Two-Variable Regression Model faces estimation problems in formulas: The method which is used to fit a regression line to a given sample data is called Ordinary Least Square (OLS) method or Least Square (LS).

We discussed Method of Least Square (LS), Ordinary Least Square (OLS)

This method consists of minimizing the sum of the squares of the difference between observed values and the corresponding values of the dependent variable obtained from the equation i.e. Yi & Ŷi. In this method, we find the values of parameters that minimize the sum of squares of residuals.

Ŷi = β_{1} + β_{2}Xi + µi ————– P.R.F

E(Yi/Xi) = β_{1} + β_{2}Xi ————-P.R.Line

Yi = β_{1} + β_{2} Xi +Ûi —————-S.R.F

Ŷi = β_{1} + β_{2} Xi ———————S.R.Line

Since we are fitting a line to the sample data so we minimize (∑Ûi²) i.e. sample residual sum of squares.

Ûi = yi — Ŷi

Ûi = yi — β_{1} – β_{2} Xi 1:1:2 —–n

∑Ûi^{2} = ∑(yi – β_{1} – β_{2}Xi)^{2 }

We have to find the values of β_{1} & β_{2} which will minimize this function.

Table of Contents

## Partial Derivatives in Two-Variable Regression Model

To minimize this function take its **partial derivatives** w.r.to β_{1} & β_{2} equate them to zero.

∂∑Ûi^{2}/∂βi = 2∑(Yi-β_{1}-β_{2}Xi) (-1) =0

∂∑Ûi^{2}/∂β_{2} = 2∑(yi-β_{1}-β_{2}Xi)(-Xi)=0

Simplifying we get ∑yi-nβ_{1}-β_{2}∑Xi=0

∑xiyi-β_{1}∑xi-β_{2}∑Xi^{2} =0

## Normal Equations in Two-Variable Regression Model

These are called **normal equations.** Solving them simultinously we get formulas for β_{1} & β_{2}

There xi = xi-x

yi = yi – y

β_{2 } = n∑xiyi – ∑xi∑yi / n∑xi^{2} – (∑xi)^{2} = ∑xiyi / ∑xi^{2}

β_{1} = ∑xi^{2}∑yi – ∑xi∑xiyi / n∑xi^{2}-(∑xi)^{2} = y-β_{2}X

Given the data on x & y we can find the values of β_{1 }& β_{2}

**Model in Deviation Form **

yi = β_{2}xi = ûi

yi = β_{2}xi

Where yi = Yi – Ŷ

xi = Xi – X

ûi = (yi – Ŷi) = (yi – β_{2}xi)

So we have to minimize ∑ûi^{2} w.r.to β_{2}

∑ûi^{2} = ∑(yi – Ŷi)^{2} = ∑(yi –β_{2}xi)^{2}

∂∑ûi^{2}/∂β_{2} = 2∑(yi – β_{2}xi) (-xi)=0

∑xiyi – β_{2}∑xi^{2} = 0

So β_{2} = ∑xiyi / ∑xi^{2}

### Assumption of Classical Linear Regression Model

In regression analysis our objective is not only to estimate values of β_{1 }& β_{2} but also draw inferences about the true of β_{1 }& β_{2} (i.e. population parameters from sample parameters).

For this purpose, we not only need the specific functional form among variables but we need to make some assumptions about variables, parameters, and disturbance term µi. These assumptions make the OLS as CLS.

The CLS assumptions are as follows:

1- The regression model is linear in parameters i.e. power of parameters is always one.

yi = β_{1} + β_{2} Xi +ui —— linear in parameters

yi = β_{1} + (β_{2})^{2} Xi +ui ——— Non linear in parameters

2- The values of independent variable X are assumed to be fixed across repeated samples i.e if we draw more than one sample from the same population the values of the X variable remain the same.

3- Given the value of X, the mean value or expected value of the residual term (ui) is zero i.e.

E(ui/xi) =0

4- Given the value of X, the variance of residual term ui is same for all observations. i.e.

var(ui/xi) = E [ui- E(ui/xi)]^{2} E[ui/xi]^{2} = δ^{2}

This is also called Homoscedasticity.

5- Given any two values of X i.e. Xi & Xj where i≠j the correlation between two ui & uj (i≠j) is zero.

cov(ui,uj/xi,xj) = E[ui-E(ui)/xi)][ui-E(ui)/xj)]

= E[(ui/xi)(ui/xj)] = 0

This assumption is also called no Autocorrelation or no serial correlation assumption. It implies that if ui,uj are plotted against each other they show no systematically

pattern.

6- Zero covariance between ui &xi i.e. the disturbance term ui and explanatory variable x are uncorrelated.

cov(ui,xi) = E[ui-E(ui)] [xi-E(xi)] = E[ui(xi-E(xi)] since E(ui) =0

= E(uixi)- E(xi) E(ui) where E(ui) =0 so

= E(uixi) =0

7- The number of observations n must be greater than the number of parameters k to be estimated i.e. n >k

8- The regression model has correctly specified i.e. the functional form of the equation is correctly selected. e.g A linear model is fitted to the marginal cost function would be an incorrect one because MC is always no linear.

9- The x values in a given sample must not all the same e.g. x1 = x2 = x3 = x4 etc is not correct.