html
y = Xb + e
where y is an n by 1 vector of responses,
x is an n by p matrix of predictors,
b is an p by 1 vector of parameters,
e is an n by 1 vector of random disturbances.
Let X = Q*R where Q and R come from a QR Decomposition of X. Q is orthogonal and R is triangular. Both these matrices are useful for calculating many regression diagnostics.
The least squares estimator for b is:
b = R\(Q'*y);
Q from QR Decomposition.
R from QR Decomposition.
Regression Coefficients .
Covariance of regression coefficients.
Fitted values of the response data.
Residuals .
Mean Squared Error.
Leverage.
Hat Matrix.
Delete-1 Variance.
Delete-1 Coefficients.
Standardized Residuals .
Studentized Residuals .
Change in Regression Coefficients.
Change in Fitted Values.
Scaled Change in Fitted Values.
Change in Covariance.
Cook's Distance.
[Q,R] = qr(X,0);
This is the so-called economy sized QR decomposition. Q is n by p. Q is also orthogonal. That is:
Q'*Q = I (the identity matrix)
[Q,R] = qr(X,0);
This is the so-called economy sized QR decomposition. R is p by p and triangular. This makes solving linear systems simple.
b = R\(Q'*y);
If you only wanted the coefficients and did not want to use Q and R later, then:
b = X\y;
is the simplest code.
%Inverse of R
ri = R\eye(p);
%inv(X'*X)
xtxi = ri*ri';
%Covariance of the coefficients.
covb = xtxi*mse;
MSE is mean squared error.
covb is a p by p matrix. The diagonal elements are the variances of the individual coefficients.
y = Xb + e
where y is an n by 1 vector of responses,
x is an n by p matrix of predictors,
b is an p by 1 vector of parameters,
e is an n by 1 vector of random disturbances.
Let X = Q*R where Q and R come from a QR Decomposition of X. Q is orthogonal and R is triangular. Both these matrices are useful for calculating many regression diagnostics.
The least squares estimator for b is:
b = R\(Q'*y);
Plugging this estimate for b into the model equation (leaving out e) we have:
yhat = X*b = X*(R\(Q'*y))
where yhat is an n by 1 vector of fitted (or predicted) values of y.
yhat be an n by 1 vector of fitted (or predicted) values of y.
Then the residuals, also n by 1 are:
r = y - yhat
In English, the residuals are the observed values minus the predicted values.
The formula is:
mse = r'*r./(n-p)
where r is the n by 1 vector of residuals.
n is the number of observations.
p is the number of unknown parameters.
In general, the farther a point is from the center of the input space, the more leverage it has.
The leverage vector is an n by 1 vector of the leverages of all the observations. It is the diagonal of the hat matrix.
MATLAB code
h = diag(Q*Q');
MATLAB code.
H = Q*Q';
yhat = H*y;
MATLAB code:
stanres = residuals./(sqrt(mse*(1-h)));
where mse is the mean squared error.
and h is the leverage vector.
See also: Studentized Residuals .
MATLAB code:
stanres = residuals./(sqrt(s2i*(1-h)));
where s2i is the delete-1 variance.
and h is the leverage vector.
See also: Standardized Residuals .
MATLAB code:
dffit = h.*residuals./(1-h)
where h is the leverage vector.
MATLAB code:
sdffit = sqrt(h./(1-h)).*e_i)
where e_i is the vector of studentized residuals.
and h is the leverage vector.
MATLAB code:
d = residuals.*residuals.*(h./(1-h))./p; where d is Cook's distance.
h is the leverage vector.
and p is the number of unknown parameters.