Regression and Prediction (Simple Linear Regression)

Simple linear regression is a fundamental statistical method in data science used to examine and describe the relationship between two numerical variables. This topic introduces students to the concepts of an independent variable (predictor) and a dependent variable (outcome), with the aim of understanding how changes in one variable are associated with changes in another. Students will learn the simple linear regression model, expressed as Y = B0 + B1(X) + e, where Y represents the outcome, X represents the predictor, B0 is the intercept, B1 is the slope, and e represents random error. Practical examples, such as examining the relationship between study time and examination scores or between temperature and electricity consumption, will help students understand the application of regression in real-world situations.

The material covers the estimation and interpretation of regression coefficients using the ordinary least squares (OLS) method, which determines the regression line by minimizing the sum of squared residuals. Students will learn to interpret the intercept and slope, construct scatter plots, fit regression lines, and evaluate the strength and direction of relationships between variables. The topic also introduces important measures of model performance, including the coefficient of determination R^2, which describes the proportion of observed variation in the outcome explained by the fitted linear model. In addition, students will examine statistical inference for regression coefficients, including hypothesis testing, p-values, and confidence intervals, to assess whether the data provide evidence of a linear association between the predictor and outcome.

Furthermore, students will learn to evaluate the assumptions and limitations of simple linear regression, including linearity, independence of errors, constant error variance (homoscedasticity), and approximate normality of residuals for conventional statistical inference. They will explore residual plots and other diagnostic tools to identify potential problems, such as outliers, influential observations, and departures from linearity. Through practical exercises using R and Python, students will develop the ability to fit regression models, interpret statistical outputs, assess model adequacy, and generate predictions for new observations. By the end of this topic, students are expected to apply simple linear regression appropriately and communicate their findings clearly, while recognizing that an observed association does not necessarily imply causation and that predictions should be made within the context and limitations of the available data.