Econometrics: OLS estimates

Let X and Y be column vectors and the sample has the size n:

The vector beta contains both the coefficient of X as well as the coefficient for the intercept.

This is equivalent to writing this equation:

$y_i= \beta_0 + \beta_1 \cdot x_i + \epsilon$

We will assume that the expected value of the error term is 0 (this can be done by construction), and that X is uncorrelated to epsilon. Assume the following is for population data.

$Cov(X,\epsilon)= E[X(Y-X' \cdot \beta)]=0$

Let's prove that statement:

$Cov(X,\epsilon)= E[X \cdot \epsilon]-E[X] \cdot \overbrace{E[\epsilon]}^0=0$

Since we know that the covariance between X and epsilon is 0, it follows that the expected value of X times the error term must also be 0.

Since we do not have access to the actual expected value of the distribution, let's use the sample data instead:

$\hat{Cov}(x,\epsilon)= n^{-1} \sum x_i (y_i-\hat{\beta_0}-\hat{\beta_1} \cdot x_i)=0$

The hat on the covariance signifies that this is a MM estimator.

Let's use the fact that the sample mean of epsilon is also equal to 0, then:

$\bar{y}= \hat{\beta_0}+ \hat{\beta_1} \bar{x}$

Using the estimation for the covariance and that the expected value of the error term equals 0:

$\hat{Cov(x, \epsilon)}=0 \rightarrow \sum x_i(y_i-\overbrace{(\bar{y}-\hat{\beta_1}\bar{x})}^{\hat{\beta_0}}-\hat{\beta_1}x_i=0$

Then we have:

$\sum x_i(y_i-\bar{y})= \hat{\beta_1}\sum x_i(x_i-\bar{x}) \rightarrow \hat{\beta_1}= \frac{\sum (x_i-\bar{x}) \cdot (y_i-\bar{y})}{\sum (x_i-\bar{x})^2}= \frac{\hat{Cov}(x,y)}{\hat{V}(x)}$

To get this result, you must use the properties of the summation operator.

So far we have found the OLS estimates for beta-0 hat, beta-1 hat. To see why this is called the ordinary least square estimates, let epsilon hat be the residual and defined as follows:

$\hat{\epsilon_i}= y_i- \overbrace{\hat{\beta_0} - \hat{\beta_1x_i}}^{\hat{y_i}} \rightarrow min[\sum \epsilon_i^2]=min[\sum (y_i- \hat{\beta_0}-\hat{\beta_1}x_i)^2]$

We want to minimize this argument by finding the optimal values of beta-0 hat and beta-1 hat. Take the partial derivatives with respect to beta-0 hat and beta-1 hat and set the functions to 0 and solve for them. We'll get the same result for beta-0 hat and beta-1 hat. Thus, we have minimized the squared residuals.

We finally have our OLS regression line, which is an estimate of the population regression:

$\hat{y}=\hat{\beta_0}+\hat{\beta_1}x$

Note that the hats on the regression parameters signify that they are estimates.

Reference: Wooldridge, Jeffrey. Introductory Econometrics, 3rd edition. Thomson South-Western (2006). p 29-38.

Comments

Macroeconomics: real vs nominal gross domestic product

Nominal GDP: A nominal GDP measures the value of a nation's output produced in a year, given the prices charged for that year. However, if prices increase while the output level stays constant in a given year, the nominal GDP will drastically increase, even though the level of production is equal to that of the previous year. Estimating real GDP: To determine the actual change in output from one year to the next, we must adjust the nominal value of a country's output in a year based on changes in the average price level during that year. Inflation is when the price level increases, whereas deflation is when the price level is decreasing. To determine whether the nominal GDP overestimates or underestimates the output of a country, one must know whether the average price level has increased or decreased. In order to achieve this task can use a price index known as the GDP deflator. The GDP deflator: Real GDP is calculated by replacing the prices for that current year with tho...

Microeconomics: Firms and cost in the short run

In Economics, the term short run refers to a time period where at least one variable of interest does not change . In our case, the short run for a firm is when at least one input (labor, land, capital) stays fixed. Usually land and capital are considered fixed in the short run. If an input is fixed during a period time, no matter how much the total product a firm produces, its cost stays the same. This cost is commonly known as fixed cost (FC). Examples of fixed costs: rent, property taxes, loan payments. Labor is often considered to be a part of the variable cost (VC) . Variable cost can be defined as the cost a firm has control over during the short run. Unlike fixed cost, variable cost increases (decreases) as a firm's total product increases (decreases). Examples of variable costs include: utility bills, wages, raw materials A firm's total cost (TC) is the sum of its variable and fixed costs. As you can see, the fixed cost...

Macroeconomics: the money market and loanable funds market

The money market: The money market combines the supply of money and the demand for money for a country. The supply curve (Sm) is inelastic and is determined by the central bank. The demand curve (Dm) is the horizontal summation of the asset and transaction demand curves and is sloping downward. The intersection between the Sm and Dm is the equilibrium interest rate for a country. A change in the nominal interest rate implies that either or both curves (i.e Sm and Dm) have shifted to the right or left. An increase in nominal GDP (P*Y) will lead to an increase in the transaction demand for money which will shift the total demand (Dm) to the right. The nominal interest rate goes up. A decrease in the level of output (Y) will shift the Dm curve to the left. The Sm curve will shift whenever the central bank decides to change the money supply due to a monetary policy decision. The Fed expands the supply of excess reserves in commercial banks, by purchasing bonds from banks, Sm shi...

Economics and Statistics

Search This Blog