From: fas on
Hi for Tikhonov regulaization of Ax-b we can write e.g

A = rand(1000,2000);
b = rand(1000,1);

% regularize
lamba = 0.1;
A = [A;lambda^2*eye(2000)];
b = [b;zeros(2000,1)];
x = lsqr(A,b);

However, for Generalized Tikhonov regularization, can any one suggest
me how to do it matlab.
Thanks.

From: John D'Errico on
fas wrote:
>
>
> Hi for Tikhonov regulaization of Ax-b we can write e.g
>
> A = rand(1000,2000);
> b = rand(1000,1);
>
> % regularize
> lamba = 0.1;
> A = [A;lambda^2*eye(2000)];
> b = [b;zeros(2000,1)];
> x = lsqr(A,b);
>
> However, for Generalized Tikhonov regularization, can any one
> suggest
> me how to do it matlab.
> Thanks.

The simple Tikhonov is a bias towards
zero. In effect, it pulls all the
coefficients towards zero with the
same amount of "force".

A generalized Tikhonov is simply a
different style of bias. A diagonal
covariance matrix would cause each
element to be biased towards zero
by varying amounts. A covariance
matrix with off-diagonal elements
will cause a bias for some linear
combinations of your variables
towards zero.

A variation of generalized Tikhonov
that I've used to good effect is seen
in gridfit (and in other tools I've
written.) Here the bias is towards
smoothness, as defined by the
appropriate linear combinations of
the unknowns.

<http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=8998&objectType=FILE>

You can formulate a generalized
Tikhonov problem in two ways, to be
solved by a tool like quadprog, or
by any linear least squares solver.
LSQR is one, but \ will do as well.

Thus, solve the regression problem

A*x = b

but with a generalized Tikhonov term
such that we wish to minimize the
sum

(norm(A*x-b))^2 + lambda*x'*Q*x

for some fixed value of lambda.

The simple Tikhonov results from
an identity matrix for Q.

If Q has full rank, then the trick
is to build a cholesky factorization
of Q=L'*L. Then we solve the linear
least squares problem

x = [A;lambda*L]\[b;zeros(n,1)]

Or use lsqr if you prefer.

If Q is rank deficient, then you
can use an ldl or udu factorization.

HTH,
John D'Errico
From: spasmous on

fas wrote:
> Hi for Tikhonov regulaization of Ax-b we can write e.g
>
> A = rand(1000,2000);
> b = rand(1000,1);
>
> % regularize
> lamba = 0.1;
> A = [A;lambda^2*eye(2000)];
> b = [b;zeros(2000,1)];
> x = lsqr(A,b);
>
> However, for Generalized Tikhonov regularization, can any one suggest
> me how to do it matlab.
> Thanks.


Following wikipedia (http://en.wikipedia.org/wiki/
Tikhonov_regularization):

P = diag(rand(100,1));
Q = diag(rand(200,1));
x0 = rand(200,1);
lambda = 0.1;

A = rand(100,200);
b = rand(100,1);
x = lsqr(A,b); % minimizes ||Ax-b||

A2 = [sqrt(P)*A;lambda*sqrt(Q)];
b2 = [sqrt(P)*b;lamba*sqrt(Q)*x0];

x2 = lsqr(A2,b2); % minimizes ||sqrt(P)*(Ax-b)||^2 + lambda^2*||
sqrt(Q)*(x-x0)||^2

From: fas on
Hi,Thanks
I would like to know that
for the case beta||Ax-b|| +lambda^2||x||, where beta is the weighting
factor is following solution that you give me works. As I thought
Generalized Tikhonov regularization means weigting factor. Or in other
words I want weithed least square minimization with regularization.
Can you comment.
Thanks,
On Mar 9, 7:13 am, "spasmous" <spasm...(a)gmail.com> wrote:
> fas wrote:
> > Hi for Tikhonov regulaization of Ax-b we can write e.g
>
> > A = rand(1000,2000);
> > b = rand(1000,1);
>
> > % regularize
> > lamba = 0.1;
> > A = [A;lambda^2*eye(2000)];
> > b = [b;zeros(2000,1)];
> > x = lsqr(A,b);
>
> > However, for Generalized Tikhonov regularization, can any one suggest
> > me how to do it matlab.
> > Thanks.
>
> Following wikipedia (http://en.wikipedia.org/wiki/
> Tikhonov_regularization):
>
> P = diag(rand(100,1));
> Q = diag(rand(200,1));
> x0 = rand(200,1);
> lambda = 0.1;
>
> A = rand(100,200);
> b = rand(100,1);
> x = lsqr(A,b); % minimizes ||Ax-b||
>
> A2 = [sqrt(P)*A;lambda*sqrt(Q)];
> b2 = [sqrt(P)*b;lamba*sqrt(Q)*x0];
>
> x2 = lsqr(A2,b2); % minimizes ||sqrt(P)*(Ax-b)||^2 + lambda^2*||
> sqrt(Q)*(x-x0)||^2


From: spasmous on
On Mar 12, 5:48 pm, "fas" <faisalmu...(a)gmail.com> wrote:
> Hi,Thanks
> I would like to know that
> for the case beta||Ax-b|| +lambda^2||x||, where beta is the weighting
> factor is following solution that you give me works. As I thought
> Generalized Tikhonov regularization means weigting factor. Or in other
> words I want weithed least square minimization with regularization.
> Can you comment.
> Thanks,


No comment. Go and read the wikipedia article then formulate a
coherent question and post it.