Classification (Logistic Regression)

Logistic regression is a fundamental statistical method in data science used to model the relationship between predictor variables and a categorical outcome. Unlike linear regression, which predicts a continuous numerical value, logistic regression estimates the probability that an observation belongs to a particular class, most commonly a binary outcome such as yes/no, diseased/non-diseased, or success/failure. This topic introduces students to the principles of binary classification, the logistic function, and the logit transformation, which converts probabilities into log-odds. Students will learn how logistic regression incorporates one or more predictor variables to estimate the probability of an outcome and how the model supports classification and data-driven decision-making in fields such as health, economics, environmental science, and social research.

The material covers the mathematical formulation of logistic regression, including the logistic function and the relationship between predictor variables and the probability of an event. Students will learn to interpret regression coefficients, odds, and odds ratios (ORs) to understand how changes in predictor variables are associated with the likelihood of an outcome. The topic also introduces maximum likelihood estimation, hypothesis testing, p-values, confidence intervals, and model fit assessment. Students will explore how predicted probabilities are converted into class labels using a classification threshold, commonly 0.5, and understand how changing this threshold affects classification decisions. Practical examples will demonstrate how logistic regression can be applied to predict disease status, student performance categories, or other binary outcomes using multiple explanatory variables.

Furthermore, students will learn to implement and evaluate logistic regression models using R and Python. The material covers data preparation, model fitting, prediction, and interpretation of statistical outputs, as well as evaluation using a confusion matrix and performance measures such as accuracy, sensitivity, specificity, precision, recall, and the area under the receiver operating characteristic curve (ROC-AUC). Students will also examine important considerations, including independence of observations, multicollinearity, influential observations, and the assumption of a linear relationship between continuous predictors and the log-odds of the outcome. By the end of this topic, students are expected to develop, interpret, and evaluate logistic regression models, understand their strengths and limitations, and apply them to real-world classification problems while recognizing that predictive performance depends on data quality, model assumptions, and appropriate validation.