Econometrics: OLS estimates

Let X and Y be column vectors and the sample has the size n:

The vector beta contains both the coefficient of X as well as the coefficient for the intercept.

This is equivalent to writing this equation:

$y_i= \beta_0 + \beta_1 \cdot x_i + \epsilon$

We will assume that the expected value of the error term is 0 (this can be done by construction), and that X is uncorrelated to epsilon. Assume the following is for population data.

$Cov(X,\epsilon)= E[X(Y-X' \cdot \beta)]=0$

Let's prove that statement:

$Cov(X,\epsilon)= E[X \cdot \epsilon]-E[X] \cdot \overbrace{E[\epsilon]}^0=0$

Since we know that the covariance between X and epsilon is 0, it follows that the expected value of X times the error term must also be 0.

Since we do not have access to the actual expected value of the distribution, let's use the sample data instead:

$\hat{Cov}(x,\epsilon)= n^{-1} \sum x_i (y_i-\hat{\beta_0}-\hat{\beta_1} \cdot x_i)=0$

The hat on the covariance signifies that this is a MM estimator.

Let's use the fact that the sample mean of epsilon is also equal to 0, then:

$\bar{y}= \hat{\beta_0}+ \hat{\beta_1} \bar{x}$

Using the estimation for the covariance and that the expected value of the error term equals 0:

$\hat{Cov(x, \epsilon)}=0 \rightarrow \sum x_i(y_i-\overbrace{(\bar{y}-\hat{\beta_1}\bar{x})}^{\hat{\beta_0}}-\hat{\beta_1}x_i=0$

Then we have:

$\sum x_i(y_i-\bar{y})= \hat{\beta_1}\sum x_i(x_i-\bar{x}) \rightarrow \hat{\beta_1}= \frac{\sum (x_i-\bar{x}) \cdot (y_i-\bar{y})}{\sum (x_i-\bar{x})^2}= \frac{\hat{Cov}(x,y)}{\hat{V}(x)}$

To get this result, you must use the properties of the summation operator.

So far we have found the OLS estimates for beta-0 hat, beta-1 hat. To see why this is called the ordinary least square estimates, let epsilon hat be the residual and defined as follows:

$\hat{\epsilon_i}= y_i- \overbrace{\hat{\beta_0} - \hat{\beta_1x_i}}^{\hat{y_i}} \rightarrow min[\sum \epsilon_i^2]=min[\sum (y_i- \hat{\beta_0}-\hat{\beta_1}x_i)^2]$

We want to minimize this argument by finding the optimal values of beta-0 hat and beta-1 hat. Take the partial derivatives with respect to beta-0 hat and beta-1 hat and set the functions to 0 and solve for them. We'll get the same result for beta-0 hat and beta-1 hat. Thus, we have minimized the squared residuals.

We finally have our OLS regression line, which is an estimate of the population regression:

$\hat{y}=\hat{\beta_0}+\hat{\beta_1}x$

Note that the hats on the regression parameters signify that they are estimates.

Reference: Wooldridge, Jeffrey. Introductory Econometrics, 3rd edition. Thomson South-Western (2006). p 29-38.

Comments

Macroeconomics: multiplier and crowding out effects

Multiplier effect: whenever any of the components of AD increases, the increase in GDP will be greater than the initial increase in expenditures. The impact on GDP of a particular increase in spending depends on the proportion of the new income that is taken out of the system to the proportion that continues to circulate in the economy. The multiplier effect tells us the impact a particular change in one the components of AD will have on the total income (GDP). Let k denote the spending multiplier, which is a function of MPC and MPS. The larger the marginal propensity to consume, the larger the spending multiplier. Notice that the larger the MPC, the greater the impact a particular change in the spending variables will have on the nation's GDP. The crowding out effect: If government spending increases without an increase in taxes, the government must borrow funds from the private sector to finance its deficit, thereby increasing the interest rate. This increase in interest ...

Econometrics: Bivariate population model

Hello I'm finally back from my extended break. I thought that we should start studying Econometrics. Let's begin by analyzing a simple bivariate regression. Assume that this equation describes the relationship between two variables X and Y. We say that Y is the dependent variable, whereas X is the independent variable. In other words, we assume that Y (the output) depends on X (the input). Epsilon is the error term, it represents other factors that affect Y. The error term must be uncorrelated with the variable X so that we do not need to include them in our regression, and thus the coefficient of X (beta) should not change, even though Epsilon also determines Y. beta-0 is the constant term. It tells us what would be Y if X=0. beta-1 is the effect on Y if X changes by one unit. To see this, assume X=education is a continuous function, let's take the derivative of Y=wage with respect to X: Thus, if education goes up by one unit, we should expect, on average, wage to go up ...

Exercise: short and long run effects of a fiscal policy

You are in charge of the government budget for the year 2021. You are told that schools and public roads need to be updated, and you estimate an appropriate budget in order to carry out the task. Describe what will happen in the short run to the economy, and what type of inflation do we see? Describe the supply side effect from this policy, explain what happens in the long run. Assume that there is no crowding out effect. First, note that updating public infractures, assuming taxes stay the same, will require an increase in government spending, which will also have an impact on the investment and consumption level in the economy (remember the spending multiplier?). Therefore, the aggregate demand curve will shift to the right. This will cause inflation (i.e positive change in the price level) and a higher output in the short-run. Note that if we were to assume a large crowding out effect, the short run aggregate supply curve would shift to the left (increase in production c...

Economics and Statistics

Search This Blog