Econometrics: OLS estimates

Let X and Y be column vectors and the sample has the size n:

The vector beta contains both the coefficient of X as well as the coefficient for the intercept.

This is equivalent to writing this equation:

$y_i= \beta_0 + \beta_1 \cdot x_i + \epsilon$

We will assume that the expected value of the error term is 0 (this can be done by construction), and that X is uncorrelated to epsilon. Assume the following is for population data.

$Cov(X,\epsilon)= E[X(Y-X' \cdot \beta)]=0$

Let's prove that statement:

$Cov(X,\epsilon)= E[X \cdot \epsilon]-E[X] \cdot \overbrace{E[\epsilon]}^0=0$

Since we know that the covariance between X and epsilon is 0, it follows that the expected value of X times the error term must also be 0.

Since we do not have access to the actual expected value of the distribution, let's use the sample data instead:

$\hat{Cov}(x,\epsilon)= n^{-1} \sum x_i (y_i-\hat{\beta_0}-\hat{\beta_1} \cdot x_i)=0$

The hat on the covariance signifies that this is a MM estimator.

Let's use the fact that the sample mean of epsilon is also equal to 0, then:

$\bar{y}= \hat{\beta_0}+ \hat{\beta_1} \bar{x}$

Using the estimation for the covariance and that the expected value of the error term equals 0:

$\hat{Cov(x, \epsilon)}=0 \rightarrow \sum x_i(y_i-\overbrace{(\bar{y}-\hat{\beta_1}\bar{x})}^{\hat{\beta_0}}-\hat{\beta_1}x_i=0$

Then we have:

$\sum x_i(y_i-\bar{y})= \hat{\beta_1}\sum x_i(x_i-\bar{x}) \rightarrow \hat{\beta_1}= \frac{\sum (x_i-\bar{x}) \cdot (y_i-\bar{y})}{\sum (x_i-\bar{x})^2}= \frac{\hat{Cov}(x,y)}{\hat{V}(x)}$

To get this result, you must use the properties of the summation operator.

So far we have found the OLS estimates for beta-0 hat, beta-1 hat. To see why this is called the ordinary least square estimates, let epsilon hat be the residual and defined as follows:

$\hat{\epsilon_i}= y_i- \overbrace{\hat{\beta_0} - \hat{\beta_1x_i}}^{\hat{y_i}} \rightarrow min[\sum \epsilon_i^2]=min[\sum (y_i- \hat{\beta_0}-\hat{\beta_1}x_i)^2]$

We want to minimize this argument by finding the optimal values of beta-0 hat and beta-1 hat. Take the partial derivatives with respect to beta-0 hat and beta-1 hat and set the functions to 0 and solve for them. We'll get the same result for beta-0 hat and beta-1 hat. Thus, we have minimized the squared residuals.

We finally have our OLS regression line, which is an estimate of the population regression:

$\hat{y}=\hat{\beta_0}+\hat{\beta_1}x$

Note that the hats on the regression parameters signify that they are estimates.

Reference: Wooldridge, Jeffrey. Introductory Econometrics, 3rd edition. Thomson South-Western (2006). p 29-38.

Economics and Statistics

Search This Blog

Econometrics: OLS estimates

Labels

Comments

Post a Comment