Least Squares Estimation & Weighted Least Squares Estimation
If the number of equations(linear or non linear ) is more than the no of variables to be determined least squares estimation is used to find a compromising solution.First let us discuss the linear least square problem ,then we turn our attention to the non linear and weighted least square problems.
The linear least square problem is determined as follows.
Determine X
which satisfy the set of equations
Determine X
which satisfy the set of equations
A*X=b
(1)
(1)
where A is a m X n matrix where m is the number of linear equations.
X =[x1 x2 ...xn] the solution vector of nX1 ,where m>n
so A is not a square matrix it's rank may be equal or less than 'n'.
To solve this system of equations the concept of least squares is used.
The variable J is the summation of least squares. The solution vector which minimize the ‘’J is called least squares solution.
J=(A*X-b)T*(A*X-b) (2)
By applying khun tucker conditions the condition for optimality can be derived as .
AT*A*X=AT*b (3)
If the term AT*A is having rank of ‘n’ the solution is easily determined bys solving (3).If it is less than ‘n’ Orthogonal-triangular decomposition. is used to solve the system of equations.
% Example 1. Start with
%program to solve linear least squares pproblem
clear;
clc;
A = [ 1 2 3
4 5 6
7 8 9
10 11 12 ]
% This is a rank-deficient matrix; the middle column is the average of the other two columns. The rank deficiency is revealed by the factorization: [Q,R] = qr(A)
b = [1;3;5;7]
[Q,R] = qr(A)
x = R\(R'\(A'*b))
r = b - A*x
e = R\(R'\(A'*r))
x = x + e;
Solution
A =
1 2 3
4 5 6
7 8 9
10 11 12
b =
1
3
5
7
Q =
-0.0776 -0.8331 0.5456 -0.0478
-0.3105 -0.4512 -0.6919 0.4704
-0.5433 -0.0694 -0.2531 -0.7975
-0.7762 0.3124 0.3994 0.3748
R =
-12.8841 -14.5916 -16.2992
0 -1.0413 -2.0826
0 0 -0.0000
0 0 0
Warning: Rank deficient, rank = 2, tol = 2.2550e-014.
x =
0.5000
0
0.1667
r =
1.0e-013 *
0.3575
0.1732
-0.0089
-0.1954
Warning: Rank deficient, rank = 2, tol = 2.2550e-014.
e =
1.0e-013 *
-0.2709
0
0.2095
% Example 1. Start with
%program to solve linear least squares pproblem
clear;
clc;
A = [ 1 2 3
4 5 6
7 8 9
10 11 12 ]
% This is a rank-deficient matrix; the middle column is the average of the other two columns. The rank deficiency is revealed by the factorization: [Q,R] = qr(A)
b = [1;3;5;7]
[Q,R] = qr(A)
x = R\(R'\(A'*b))
r = b - A*x
e = R\(R'\(A'*r))
x = x + e;
Solution
A =
1 2 3
4 5 6
7 8 9
10 11 12
b =
1
3
5
7
Q =
-0.0776 -0.8331 0.5456 -0.0478
-0.3105 -0.4512 -0.6919 0.4704
-0.5433 -0.0694 -0.2531 -0.7975
-0.7762 0.3124 0.3994 0.3748
R =
-12.8841 -14.5916 -16.2992
0 -1.0413 -2.0826
0 0 -0.0000
0 0 0
Warning: Rank deficient, rank = 2, tol = 2.2550e-014.
x =
0.5000
0
0.1667
r =
1.0e-013 *
0.3575
0.1732
-0.0089
-0.1954
Warning: Rank deficient, rank = 2, tol = 2.2550e-014.
e =
1.0e-013 *
-0.2709
0
0.2095
No comments:
Post a Comment