Econometrics: OLS estimates

Let X and Y be column vectors and the sample has the size n:

The vector beta contains both the coefficient of X as well as the coefficient for the intercept.

This is equivalent to writing this equation:

$y_i= \beta_0 + \beta_1 \cdot x_i + \epsilon$

We will assume that the expected value of the error term is 0 (this can be done by construction), and that X is uncorrelated to epsilon. Assume the following is for population data.

$Cov(X,\epsilon)= E[X(Y-X' \cdot \beta)]=0$

Let's prove that statement:

$Cov(X,\epsilon)= E[X \cdot \epsilon]-E[X] \cdot \overbrace{E[\epsilon]}^0=0$

Since we know that the covariance between X and epsilon is 0, it follows that the expected value of X times the error term must also be 0.

Since we do not have access to the actual expected value of the distribution, let's use the sample data instead:

$\hat{Cov}(x,\epsilon)= n^{-1} \sum x_i (y_i-\hat{\beta_0}-\hat{\beta_1} \cdot x_i)=0$

The hat on the covariance signifies that this is a MM estimator.

Let's use the fact that the sample mean of epsilon is also equal to 0, then:

$\bar{y}= \hat{\beta_0}+ \hat{\beta_1} \bar{x}$

Using the estimation for the covariance and that the expected value of the error term equals 0:

$\hat{Cov(x, \epsilon)}=0 \rightarrow \sum x_i(y_i-\overbrace{(\bar{y}-\hat{\beta_1}\bar{x})}^{\hat{\beta_0}}-\hat{\beta_1}x_i=0$

Then we have:

$\sum x_i(y_i-\bar{y})= \hat{\beta_1}\sum x_i(x_i-\bar{x}) \rightarrow \hat{\beta_1}= \frac{\sum (x_i-\bar{x}) \cdot (y_i-\bar{y})}{\sum (x_i-\bar{x})^2}= \frac{\hat{Cov}(x,y)}{\hat{V}(x)}$

To get this result, you must use the properties of the summation operator.

So far we have found the OLS estimates for beta-0 hat, beta-1 hat. To see why this is called the ordinary least square estimates, let epsilon hat be the residual and defined as follows:

$\hat{\epsilon_i}= y_i- \overbrace{\hat{\beta_0} - \hat{\beta_1x_i}}^{\hat{y_i}} \rightarrow min[\sum \epsilon_i^2]=min[\sum (y_i- \hat{\beta_0}-\hat{\beta_1}x_i)^2]$

We want to minimize this argument by finding the optimal values of beta-0 hat and beta-1 hat. Take the partial derivatives with respect to beta-0 hat and beta-1 hat and set the functions to 0 and solve for them. We'll get the same result for beta-0 hat and beta-1 hat. Thus, we have minimized the squared residuals.

We finally have our OLS regression line, which is an estimate of the population regression:

$\hat{y}=\hat{\beta_0}+\hat{\beta_1}x$

Note that the hats on the regression parameters signify that they are estimates.

Reference: Wooldridge, Jeffrey. Introductory Econometrics, 3rd edition. Thomson South-Western (2006). p 29-38.

Comments

Macroeconomics: multiplier and crowding out effects

Multiplier effect: whenever any of the components of AD increases, the increase in GDP will be greater than the initial increase in expenditures. The impact on GDP of a particular increase in spending depends on the proportion of the new income that is taken out of the system to the proportion that continues to circulate in the economy. The multiplier effect tells us the impact a particular change in one the components of AD will have on the total income (GDP). Let k denote the spending multiplier, which is a function of MPC and MPS. The larger the marginal propensity to consume, the larger the spending multiplier. Notice that the larger the MPC, the greater the impact a particular change in the spending variables will have on the nation's GDP. The crowding out effect: If government spending increases without an increase in taxes, the government must borrow funds from the private sector to finance its deficit, thereby increasing the interest rate. This increase in interest ...

Microeconomics: Shift in Supply

We recently talked about the factors that cause a shift in the demand. It is now time to discuss the factors that shift the supply curve which cause the equilibrium price and demand to change. Determinants of Supply: changes in supply are caused by changes in the price of inputs, the number of firms in the market, technology, changes in the price of related goods and services, and changes in expected prices. An increase (decrease) in the price of an input results in less (more) supply as per unit production costs rise (fall). More competition increases supply, and less competition leads to less supply. If a new firm enters the market for good X, then the supply of good X increases. Improvement in technology can result in an increase in the ability of producers to supply their products. For example, the invention of the printing press increased the supply of books. An increase (decrease) in the price of a related good leads to an increase (decrease) in the supply of the other...

Macroeconomics: Fiscal Policy and short run effects of fiscal and monetary policies

Fiscal Policy: government spending and taxation, aimed at expending or contracting the level of macroeconomic activity in a nation. A tax increase (decrease) will raise (lower) households' disposable income, leading to more (less) consumption. Furthermore, firms will increase (decrease) the number of investment as they get to keep a larger (smaller) share of their profits. An increase (decrease) in government spending affects the variable G that defines the aggregate demand. Government spending also leads to a change in household income, affecting the level of consumption. Tax multiplier: just like any multiplier so far, it is just the sum of a converging geometric series: Say the MPC is equal to 70%, then it must be the case that the MPS is 30%. Thus, the tax multiplier, t, is equal to about -2.32. In other words, for every dollar that goes to tax revenue, total spending decreases by about 2.32 dollars. Note that a tax decrease would be negative, and thus have a positi...

Economics and Statistics

Search This Blog