Econometrics: OLS estimates

Let X and Y be column vectors and the sample has the size n:

The vector beta contains both the coefficient of X as well as the coefficient for the intercept.

This is equivalent to writing this equation:

$y_i= \beta_0 + \beta_1 \cdot x_i + \epsilon$

We will assume that the expected value of the error term is 0 (this can be done by construction), and that X is uncorrelated to epsilon. Assume the following is for population data.

$Cov(X,\epsilon)= E[X(Y-X' \cdot \beta)]=0$

Let's prove that statement:

$Cov(X,\epsilon)= E[X \cdot \epsilon]-E[X] \cdot \overbrace{E[\epsilon]}^0=0$

Since we know that the covariance between X and epsilon is 0, it follows that the expected value of X times the error term must also be 0.

Since we do not have access to the actual expected value of the distribution, let's use the sample data instead:

$\hat{Cov}(x,\epsilon)= n^{-1} \sum x_i (y_i-\hat{\beta_0}-\hat{\beta_1} \cdot x_i)=0$

The hat on the covariance signifies that this is a MM estimator.

Let's use the fact that the sample mean of epsilon is also equal to 0, then:

$\bar{y}= \hat{\beta_0}+ \hat{\beta_1} \bar{x}$

Using the estimation for the covariance and that the expected value of the error term equals 0:

$\hat{Cov(x, \epsilon)}=0 \rightarrow \sum x_i(y_i-\overbrace{(\bar{y}-\hat{\beta_1}\bar{x})}^{\hat{\beta_0}}-\hat{\beta_1}x_i=0$

Then we have:

$\sum x_i(y_i-\bar{y})= \hat{\beta_1}\sum x_i(x_i-\bar{x}) \rightarrow \hat{\beta_1}= \frac{\sum (x_i-\bar{x}) \cdot (y_i-\bar{y})}{\sum (x_i-\bar{x})^2}= \frac{\hat{Cov}(x,y)}{\hat{V}(x)}$

To get this result, you must use the properties of the summation operator.

So far we have found the OLS estimates for beta-0 hat, beta-1 hat. To see why this is called the ordinary least square estimates, let epsilon hat be the residual and defined as follows:

$\hat{\epsilon_i}= y_i- \overbrace{\hat{\beta_0} - \hat{\beta_1x_i}}^{\hat{y_i}} \rightarrow min[\sum \epsilon_i^2]=min[\sum (y_i- \hat{\beta_0}-\hat{\beta_1}x_i)^2]$

We want to minimize this argument by finding the optimal values of beta-0 hat and beta-1 hat. Take the partial derivatives with respect to beta-0 hat and beta-1 hat and set the functions to 0 and solve for them. We'll get the same result for beta-0 hat and beta-1 hat. Thus, we have minimized the squared residuals.

We finally have our OLS regression line, which is an estimate of the population regression:

$\hat{y}=\hat{\beta_0}+\hat{\beta_1}x$

Note that the hats on the regression parameters signify that they are estimates.

Reference: Wooldridge, Jeffrey. Introductory Econometrics, 3rd edition. Thomson South-Western (2006). p 29-38.

Comments

Macroeconomics: balance of payments accounts

Balance of payment is the sum of three separate accounts and must always be equal to 0 (given no statistical error): Current account: the net flow of funds exchanged for goods and services, and monetary gifts that flow in and out of a country. This is often used as an indicator for net export. Financial account: also known as the capital account, is the net flow of funds for investment in real assets (direct investments) or financial assets (stocks or bonds) into a nation. Official reserves: in order to balance the two accounts above, a country must have a reserve of foreign money. It measures the net effect of all money flows from the other accounts. If the current account is positive, it must be the case that the sum of the financial account and official reserves is negative. Current account (CA) components: Balance of trade in goods: spending by consumers and firms on imported and exported goods. Exported goods have a positive effect on the current account, whereas importe...

Macroeconomics: equilibrium in the AD/AS model

Real output and price level in the AD/AS model: The short run equilibrium in the AD/AS model, is when the intersection between the short run aggregate supply curve and the aggregate demand curve determines the level of output and price level. This short equilibrium output can be compared to the full-employment (i.e long run equilibrium) output by adding the long run aggregate supply curve in the graph. If the difference between the short run equilibrium output and full employment output is negative than we have a recessionary gap. This could be caused by a decrease in households' consumption, a fall in private investments, or a decrease in government spending. If the difference is positive, we have an inflationary gap, where the short run equilibrium output is bigger than its full employment output. This can be caused by an increase in households expenditures, expansionary fiscal policy, etc... If the gap is zero, this means that t...

Microeconomics: Shift in Supply

We recently talked about the factors that cause a shift in the demand. It is now time to discuss the factors that shift the supply curve which cause the equilibrium price and demand to change. Determinants of Supply: changes in supply are caused by changes in the price of inputs, the number of firms in the market, technology, changes in the price of related goods and services, and changes in expected prices. An increase (decrease) in the price of an input results in less (more) supply as per unit production costs rise (fall). More competition increases supply, and less competition leads to less supply. If a new firm enters the market for good X, then the supply of good X increases. Improvement in technology can result in an increase in the ability of producers to supply their products. For example, the invention of the printing press increased the supply of books. An increase (decrease) in the price of a related good leads to an increase (decrease) in the supply of the other...

Economics and Statistics

Search This Blog