Classification (Naive Bayes, Discriminant Analysis)

Classification is a fundamental topic in data science that focuses on predicting categorical outcomes based on observed data. This topic introduces students to the principles of supervised learning, in which a model is developed using labeled training data to assign observations to predefined classes. Students will learn the concepts of predictor variables, class labels, training and testing datasets, and decision boundaries. Practical examples, such as classifying patients into disease and non-disease groups, identifying spam emails, and categorizing environmental quality levels, will help students understand how classification techniques support data-driven decision-making. Emphasis will be placed on understanding how classification differs from regression and how the characteristics of a dataset influence the selection of an appropriate classification method.

The material focuses on two important classification methods: Naive Bayes and Discriminant Analysis. Naive Bayes is a probabilistic classification method based on Bayes’ theorem, which calculates the probability that an observation belongs to a particular class given its predictor variables. Students will explore prior probabilities, likelihoods, posterior probabilities, and the conditional independence assumption underlying the Naive Bayes classifier. Discriminant analysis, including Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA), will be introduced as a method for classifying observations based on differences in the distributions of predictor variables across groups. Students will learn how LDA assumes a common covariance matrix across classes, whereas QDA allows each class to have its own covariance matrix, resulting in different decision boundaries. Through examples and comparisons, students will understand the assumptions, advantages, and limitations of these approaches.

Furthermore, students will learn to implement and evaluate Naive Bayes and discriminant analysis models using R and Python. The material will cover data preparation, model fitting, class prediction, and the interpretation of predicted class probabilities. Students will also explore classification performance measures, including accuracy, sensitivity, specificity, precision, recall, and confusion matrices, to assess how well a model distinguishes between classes. Practical exercises will demonstrate how to compare classification methods, identify potential sources of misclassification, and select an appropriate model based on the research objective and characteristics of the data. By the end of this topic, students are expected to develop, evaluate, and interpret classification models and understand their applications in statistical analysis and real-world prediction problems