Introduction

In the world of linear regression, the Ordinary Least Squares (OLS) method plays a crucial role in estimating the parameters that define the relationship between variables. Understanding the properties of OLS estimators and their sampling distributions is essential for grasping the reliability and significance of regression results. In this chapter, we’ll delve into the fundamental properties of OLS estimators and explore their sampling distributions, shedding light on the statistical foundation that underpins linear regression analysis.

Properties of OLS Estimators

• Unbiasedness: One of the key qualities of an estimator is being unbiased, which means that, on average, the estimator’s expected value is equal to the true parameter value. In OLS, estimators are unbiased, ensuring that the estimates converge to the true population parameters as sample size increases.

• Efficiency: An efficient estimator has the smallest possible variance among the class of consistent estimators. OLS estimators are efficient under the assumptions of the Gauss-Markov theorem when assumptions hold, making them optimal in terms of minimal variance.

Sampling Distributions of OLS Estimators

Understanding the sampling distributions of OLS estimators is crucial for hypothesis testing and constructing confidence intervals.

1) Normality of Sampling Distribution: When the classical assumptions of linear regression are met (linearity, independence, constant variance, and normality), the sampling distribution of OLS estimators is approximately normal. This normality is vital for conducting hypothesis tests and calculating confidence intervals.

2) $\mathbf{t}$-Distribution for Hypothesis Testing: For small sample sizes, the sampling distribution of OLS estimators follows the t-distribution, a close relative of the normal distribution. This is particularly important when performing hypothesis tests on individual regression coefficients.

Example: Suppose we’re analyzing the relationship between years of experience and salary. An OLS regression yields an estimated coefficient of 2500 for the experience variable. To determine if this coefficient is statistically significant, we can use the t-distribution and calculate atstatistic using the formula:

$$t=\frac{(\hat{\beta}-\beta_0)}{\operatorname{SE}(\hat{\beta})}$$ where

• $\hat{\beta}$ is the estimated coefficient,
• $\beta_0$ is the hypothesized population parameter (usually 0), and
• $S E(\hat{\beta})$ is the standard error of the coefficient estimate.

By comparing the t-statistic to critical values from the $t$-distribution, we can make informed decisions about the significance of the coefficient.