Econometrics: OLS estimates

Let X and Y be column vectors and the sample has the size n:

The vector beta contains both the coefficient of X as well as the coefficient for the intercept.

This is equivalent to writing this equation:

$y_i= \beta_0 + \beta_1 \cdot x_i + \epsilon$

We will assume that the expected value of the error term is 0 (this can be done by construction), and that X is uncorrelated to epsilon. Assume the following is for population data.

$Cov(X,\epsilon)= E[X(Y-X' \cdot \beta)]=0$

Let's prove that statement:

$Cov(X,\epsilon)= E[X \cdot \epsilon]-E[X] \cdot \overbrace{E[\epsilon]}^0=0$

Since we know that the covariance between X and epsilon is 0, it follows that the expected value of X times the error term must also be 0.

Since we do not have access to the actual expected value of the distribution, let's use the sample data instead:

$\hat{Cov}(x,\epsilon)= n^{-1} \sum x_i (y_i-\hat{\beta_0}-\hat{\beta_1} \cdot x_i)=0$

The hat on the covariance signifies that this is a MM estimator.

Let's use the fact that the sample mean of epsilon is also equal to 0, then:

$\bar{y}= \hat{\beta_0}+ \hat{\beta_1} \bar{x}$

Using the estimation for the covariance and that the expected value of the error term equals 0:

$\hat{Cov(x, \epsilon)}=0 \rightarrow \sum x_i(y_i-\overbrace{(\bar{y}-\hat{\beta_1}\bar{x})}^{\hat{\beta_0}}-\hat{\beta_1}x_i=0$

Then we have:

$\sum x_i(y_i-\bar{y})= \hat{\beta_1}\sum x_i(x_i-\bar{x}) \rightarrow \hat{\beta_1}= \frac{\sum (x_i-\bar{x}) \cdot (y_i-\bar{y})}{\sum (x_i-\bar{x})^2}= \frac{\hat{Cov}(x,y)}{\hat{V}(x)}$

To get this result, you must use the properties of the summation operator.

So far we have found the OLS estimates for beta-0 hat, beta-1 hat. To see why this is called the ordinary least square estimates, let epsilon hat be the residual and defined as follows:

$\hat{\epsilon_i}= y_i- \overbrace{\hat{\beta_0} - \hat{\beta_1x_i}}^{\hat{y_i}} \rightarrow min[\sum \epsilon_i^2]=min[\sum (y_i- \hat{\beta_0}-\hat{\beta_1}x_i)^2]$

We want to minimize this argument by finding the optimal values of beta-0 hat and beta-1 hat. Take the partial derivatives with respect to beta-0 hat and beta-1 hat and set the functions to 0 and solve for them. We'll get the same result for beta-0 hat and beta-1 hat. Thus, we have minimized the squared residuals.

We finally have our OLS regression line, which is an estimate of the population regression:

$\hat{y}=\hat{\beta_0}+\hat{\beta_1}x$

Note that the hats on the regression parameters signify that they are estimates.

Reference: Wooldridge, Jeffrey. Introductory Econometrics, 3rd edition. Thomson South-Western (2006). p 29-38.

Comments

Macroeconomics: multiplier and crowding out effects

Multiplier effect: whenever any of the components of AD increases, the increase in GDP will be greater than the initial increase in expenditures. The impact on GDP of a particular increase in spending depends on the proportion of the new income that is taken out of the system to the proportion that continues to circulate in the economy. The multiplier effect tells us the impact a particular change in one the components of AD will have on the total income (GDP). Let k denote the spending multiplier, which is a function of MPC and MPS. The larger the marginal propensity to consume, the larger the spending multiplier. Notice that the larger the MPC, the greater the impact a particular change in the spending variables will have on the nation's GDP. The crowding out effect: If government spending increases without an increase in taxes, the government must borrow funds from the private sector to finance its deficit, thereby increasing the interest rate. This increase in interest ...

Microeconomics: Firms and cost in the short run

In Economics, the term short run refers to a time period where at least one variable of interest does not change . In our case, the short run for a firm is when at least one input (labor, land, capital) stays fixed. Usually land and capital are considered fixed in the short run. If an input is fixed during a period time, no matter how much the total product a firm produces, its cost stays the same. This cost is commonly known as fixed cost (FC). Examples of fixed costs: rent, property taxes, loan payments. Labor is often considered to be a part of the variable cost (VC) . Variable cost can be defined as the cost a firm has control over during the short run. Unlike fixed cost, variable cost increases (decreases) as a firm's total product increases (decreases). Examples of variable costs include: utility bills, wages, raw materials A firm's total cost (TC) is the sum of its variable and fixed costs. As you can see, the fixed cost...

Microeconomics: Factor Markets

Definition: Factor markets: markets for the factors of production (example: labor and capital). Markets are formed whenever consumers and producers meet to exchange goods or services. Deriving factor demand: the demand for goods or services in the product markets creates demand for the factors of production. An increase (decrease) in demand for good X leads the suppliers to increase their production thereby increasing (decreasing) the demand for the factors of production. Marginal revenue product: The demand for the factor of production is formed by multiplying a firm's marginal revenue by its marginal product. Remember that by taking the derivative of the TR function with respect to Q we are able to find the MR. Marginal product on the other hand is found by taking the derivative of the production function with respect to a factor of production (L or K for example). Marginal revenue product (MRP): the change in total revenue when one more input is employed. It decrea...

Economics and Statistics

Search This Blog