Introduction

In the realm of statistical theory, the Law of Large Numbers (LLN) and the Central Limit Theorem (CLT) hold a prominent place, providing profound insights into the behavior of sample statistics. In this chapter, we will explore how these two fundamental theorems apply specifically to the sample mean. We will unravel the principles behind LLN and CLT and demonstrate their significance in understanding the stability and distribution of the sample mean.

Law of Large Numbers (LLN)

The Law of Large Numbers asserts a crucial principle: as the size of a sample increases, the sample mean approaches the population mean. In simpler terms, with a sufficiently large sample, the sample mean becomes a reliable estimator of the population mean. This theorem underscores the stabilizing effect of larger sample sizes on estimation.

For example, consider flipping a fair coin. As the number of coin flips increases, the proportion of heads will converge towards 0.5, which is the true probability of heads. In finance, LLN implies that as the number of observations in a financial data series grows, the average return over time converges towards the expected return.

Central Limit Theorem (CLT)

The Central Limit Theorem, a cornerstone of statistical theory, postulates that the distribution of the sample mean of a sufficiently large sample becomes approximately normal, regardless of the underlying distribution of the population. This has immense practical implications, as it enables us to make inferences about the sample mean using the properties of the normal distribution.

Imagine rolling a fair six-sided die multiple times and calculating the mean of each set of rolls. According to the CLT, as the number of sets of rolls increases, the distribution of the sample means will approximate a normal distribution, even though the individual rolls are not normally distributed.

Application to the Sample Mean

Both LLN and CLT are particularly relevant when examining the sample mean. LLN assures us that a large enough sample provides a reliable estimate of the population mean, while CLT enables us to make probabilistic statements about the distribution of the sample mean.

Mathematically,

For LLN:

$$\lim _{n \rightarrow \infty} \frac{1}{n} \sum_i X_i=\mu \text {, }$$

where $X_i$ represents individual observations, $n$ is the sample size, $\mu$ is the population mean, and $i$ varies from 1 to $n$.

For CLT:

$$\bar{X} \sim N\left(\mu, \frac{\sigma^2}{n}\right) \text {, }$$

where $\bar{X}$ is the sample mean, $\mu$ is the population mean, $\sigma^2$ is the population variance, and $n$ is the sample size.