Two-Variable Regression Model faces estimation problems in formulas: The method which is used to fit a regression line to a given sample data is called Ordinary Least Square (OLS) method or Least Square (LS).
We discussed Method of Least Square (LS), Ordinary Least Square (OLS)
This method consists of minimizing the sum of the squares of the difference between observed values and the corresponding values of the dependent variable obtained from the equation i.e. Yi & Ŷi. In this method, we find the values of parameters that minimize the sum of squares of residuals.
Ŷi = β1 + β2Xi + µi ————– P.R.F
E(Yi/Xi) = β1 + β2Xi ————-P.R.Line
Yi = β1 + β2 Xi +Ûi —————-S.R.F
Ŷi = β1 + β2 Xi ———————S.R.Line
Since we are fitting a line to the sample data so we minimize (∑Ûi²) i.e. sample residual sum of squares.
Ûi = yi — Ŷi
Ûi = yi — β1 – β2 Xi 1:1:2 —–n
∑Ûi2 = ∑(yi – β1 – β2Xi)2
We have to find the values of β1 & β2 which will minimize this function.
Partial Derivatives in Two-Variable Regression Model
To minimize this function take its partial derivatives w.r.to β1 & β2 equate them to zero.
∂∑Ûi2/∂βi = 2∑(Yi-β1-β2Xi) (-1) =0
∂∑Ûi2/∂β2 = 2∑(yi-β1-β2Xi)(-Xi)=0
Simplifying we get ∑yi-nβ1-β2∑Xi=0
Normal Equations in Two-Variable Regression Model
These are called normal equations. Solving them simultinously we get formulas for β1 & β2
There xi = xi-x
yi = yi – y
β2 = n∑xiyi – ∑xi∑yi / n∑xi2 – (∑xi)2 = ∑xiyi / ∑xi2
β1 = ∑xi2∑yi – ∑xi∑xiyi / n∑xi2-(∑xi)2 = y-β2X
Given the data on x & y we can find the values of β1 & β2
Model in Deviation Form
yi = β2xi = ûi
yi = β2xi
Where yi = Yi – Ŷ
xi = Xi – X
ûi = (yi – Ŷi) = (yi – β2xi)
So we have to minimize ∑ûi2 w.r.to β2
∑ûi2 = ∑(yi – Ŷi)2 = ∑(yi –β2xi)2
∂∑ûi2/∂β2 = 2∑(yi – β2xi) (-xi)=0
∑xiyi – β2∑xi2 = 0
So β2 = ∑xiyi / ∑xi2
Assumption of Classical Linear Regression Model
In regression analysis our objective is not only to estimate values of β1 & β2 but also draw inferences about the true of β1 & β2 (i.e. population parameters from sample parameters).
For this purpose, we not only need the specific functional form among variables but we need to make some assumptions about variables, parameters, and disturbance term µi. These assumptions make the OLS as CLS.
The CLS assumptions are as follows:
1- The regression model is linear in parameters i.e. power of parameters is always one.
yi = β1 + β2 Xi +ui —— linear in parameters
yi = β1 + (β2)2 Xi +ui ——— Non linear in parameters
2- The values of independent variable X are assumed to be fixed across repeated samples i.e if we draw more than one sample from the same population the values of the X variable remain the same.
3- Given the value of X, the mean value or expected value of the residual term (ui) is zero i.e.
4- Given the value of X, the variance of residual term ui is same for all observations. i.e.
var(ui/xi) = E [ui- E(ui/xi)]2 E[ui/xi]2 = δ2
This is also called Homoscedasticity.
5- Given any two values of X i.e. Xi & Xj where i≠j the correlation between two ui & uj (i≠j) is zero.
cov(ui,uj/xi,xj) = E[ui-E(ui)/xi)][ui-E(ui)/xj)]
= E[(ui/xi)(ui/xj)] = 0
This assumption is also called no Autocorrelation or no serial correlation assumption. It implies that if ui,uj are plotted against each other they show no systematically
6- Zero covariance between ui &xi i.e. the disturbance term ui and explanatory variable x are uncorrelated.
cov(ui,xi) = E[ui-E(ui)] [xi-E(xi)] = E[ui(xi-E(xi)] since E(ui) =0
= E(uixi)- E(xi) E(ui) where E(ui) =0 so
= E(uixi) =0
7- The number of observations n must be greater than the number of parameters k to be estimated i.e. n >k
8- The regression model has correctly specified i.e. the functional form of the equation is correctly selected. e.g A linear model is fitted to the marginal cost function would be an incorrect one because MC is always no linear.
9- The x values in a given sample must not all the same e.g. x1 = x2 = x3 = x4 etc is not correct.