Two-Variable Regression Model: Estimation Problem

Two-Variable Regression Model faces estimation problems in formulas: The method which is used to fit a regression line to a given sample data is called Ordinary Least Square (OLS) method or Least Square (LS).

We discussed Method of Least Square (LS), Ordinary Least Square (OLS)

This method consists of minimizing the sum of the squares of the difference between observed values and the corresponding values of the dependent variable obtained from the equation i.e. Yi & Ŷi. In this method, we find the values of parameters that minimize the sum of squares of residuals.

Ŷi = β1 + β2Xi + µi ————– P.R.F

E(Yi/Xi) = β1 + β2Xi ————-P.R.Line

Yi = β1 + β2 Xi +Ûi —————-S.R.F

Ŷi = β1 + β2 Xi ———————S.R.Line

Since we are fitting a line to the sample data so we minimize (∑Ûi²) i.e. sample residual sum of squares.

Ûi = yi — Ŷi

Ûi = yi — β1 – β2 Xi 1:1:2 —–n

∑Ûi2 = ∑(yi – β1 – β2Xi)2

We have to find the values of β1 & β2 which will minimize this function.

Partial Derivatives in Two-Variable Regression Model

To minimize this function take its partial derivatives β1 & β2 equate them to zero.

∂∑Ûi2/∂βi = 2∑(Yi-β12Xi) (-1) =0

∂∑Ûi2/∂β2 = 2∑(yi-β12Xi)(-Xi)=0

Simplifying we get     ∑yi-nβ12∑Xi=0

∑xiyi-β1∑xi-β2∑Xi2 =0

Normal Equations in Two-Variable Regression Model

These are called normal equations. Solving them simultinously we get formulas for β1 & β2

There   xi  =  xi-x

yi  =  yi – y

β2  =  n∑xiyi – ∑xi∑yi / n∑xi2 – (∑xi)2 = ∑xiyi / ∑xi2

β1 = ∑xi2∑yi – ∑xi∑xiyi / n∑xi2-(∑xi)2 = y-β2X

Given the data on x & y we can find the values of β1 & β2

Model in Deviation Form

yi = β2xi = ûi

yi = β2xi

Where              yi = Yi – Ŷ

xi = Xi – X

ûi = (yi – Ŷi) = (yi – β2xi)

So we have to minimize ∑ûi2 β2

∑ûi2 = ∑(yi – Ŷi)2 = ∑(yi –β2xi)2

∂∑ûi2/∂β2 = 2∑(yi – β2xi) (-xi)=0

∑xiyi – β2∑xi2 = 0

So β2 = ∑xiyi / ∑xi2

Assumption of Classical Linear Regression Model

In regression analysis our objective is not only to estimate values of β1 & β2 but also draw inferences about the true of β1 & β2 (i.e. population parameters from sample parameters).

For this purpose, we not only need the specific functional form among variables but we need to make some assumptions about variables, parameters, and disturbance term µi. These assumptions make the OLS as CLS.

The CLS assumptions are as follows:

1- The regression model is linear in parameters i.e. power of parameters is always one.

yi = β1 + β2 Xi +ui —— linear in parameters

yi = β1 + (β2)2 Xi +ui ——— Non linear in parameters

2- The values of independent variable X are assumed to be fixed across repeated samples i.e if we draw more than one sample from the same population the values of the X variable remain the same.

3- Given the value of X, the mean value or expected value of the residual term (ui) is zero i.e.

E(ui/xi) =0

classical linear model positive
classical linear model positive

4- Given the value of X, the variance of residual term ui is same for all observations. i.e.

var(ui/xi) = E [ui- E(ui/xi)]2 E[ui/xi]2 = δ2

This is also called Homoscedasticity.

5- Given any two values of X i.e. Xi & Xj where i≠j the correlation between two ui & uj (i≠j) is zero.

cov(ui,uj/xi,xj) = E[ui-E(ui)/xi)][ui-E(ui)/xj)]

= E[(ui/xi)(ui/xj)] = 0


This assumption is also called no Autocorrelation or no serial correlation assumption. It implies that if ui,uj are plotted against each other they show no systematically

Classical Linear Model negative
Classical Linear Model negative


6- Zero covariance between ui &xi i.e. the disturbance term ui and explanatory variable x are uncorrelated.

cov(ui,xi) = E[ui-E(ui)] [xi-E(xi)] = E[ui(xi-E(xi)] since E(ui) =0

= E(uixi)- E(xi) E(ui) where E(ui) =0 so

= E(uixi) =0

classical linear model noac
classical linear model noac

7- The number of observations n must be greater than the number of parameters k to be estimated i.e. n >k

8- The regression model has correctly specified i.e. the functional form of the equation is correctly selected. e.g A linear model is fitted to the marginal cost function would be an incorrect one because MC is always no linear.

9- The x values in a given sample must not all the same e.g. x1 = x2 = x3 = x4 etc is not correct.

Leave a Comment

%d bloggers like this: