Generalized Tikhonov regularization lsqr
in [Matlab]
|
Prev: How to install a toolbox ?
Next: GUIDANCE REQUIRED FOR AN ERROR USING LINKAGEIN BIOINFORMATICS TOOLBOX
From: fas on 7 Mar 2007 22:32 Hi for Tikhonov regulaization of Ax-b we can write e.g A = rand(1000,2000); b = rand(1000,1); % regularize lamba = 0.1; A = [A;lambda^2*eye(2000)]; b = [b;zeros(2000,1)]; x = lsqr(A,b); However, for Generalized Tikhonov regularization, can any one suggest me how to do it matlab. Thanks.
From: John D'Errico on 8 Mar 2007 09:13 fas wrote: > > > Hi for Tikhonov regulaization of Ax-b we can write e.g > > A = rand(1000,2000); > b = rand(1000,1); > > % regularize > lamba = 0.1; > A = [A;lambda^2*eye(2000)]; > b = [b;zeros(2000,1)]; > x = lsqr(A,b); > > However, for Generalized Tikhonov regularization, can any one > suggest > me how to do it matlab. > Thanks. The simple Tikhonov is a bias towards zero. In effect, it pulls all the coefficients towards zero with the same amount of "force". A generalized Tikhonov is simply a different style of bias. A diagonal covariance matrix would cause each element to be biased towards zero by varying amounts. A covariance matrix with off-diagonal elements will cause a bias for some linear combinations of your variables towards zero. A variation of generalized Tikhonov that I've used to good effect is seen in gridfit (and in other tools I've written.) Here the bias is towards smoothness, as defined by the appropriate linear combinations of the unknowns. <http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=8998&objectType=FILE> You can formulate a generalized Tikhonov problem in two ways, to be solved by a tool like quadprog, or by any linear least squares solver. LSQR is one, but \ will do as well. Thus, solve the regression problem A*x = b but with a generalized Tikhonov term such that we wish to minimize the sum (norm(A*x-b))^2 + lambda*x'*Q*x for some fixed value of lambda. The simple Tikhonov results from an identity matrix for Q. If Q has full rank, then the trick is to build a cholesky factorization of Q=L'*L. Then we solve the linear least squares problem x = [A;lambda*L]\[b;zeros(n,1)] Or use lsqr if you prefer. If Q is rank deficient, then you can use an ldl or udu factorization. HTH, John D'Errico
From: spasmous on 8 Mar 2007 15:13 fas wrote: > Hi for Tikhonov regulaization of Ax-b we can write e.g > > A = rand(1000,2000); > b = rand(1000,1); > > % regularize > lamba = 0.1; > A = [A;lambda^2*eye(2000)]; > b = [b;zeros(2000,1)]; > x = lsqr(A,b); > > However, for Generalized Tikhonov regularization, can any one suggest > me how to do it matlab. > Thanks. Following wikipedia (http://en.wikipedia.org/wiki/ Tikhonov_regularization): P = diag(rand(100,1)); Q = diag(rand(200,1)); x0 = rand(200,1); lambda = 0.1; A = rand(100,200); b = rand(100,1); x = lsqr(A,b); % minimizes ||Ax-b|| A2 = [sqrt(P)*A;lambda*sqrt(Q)]; b2 = [sqrt(P)*b;lamba*sqrt(Q)*x0]; x2 = lsqr(A2,b2); % minimizes ||sqrt(P)*(Ax-b)||^2 + lambda^2*|| sqrt(Q)*(x-x0)||^2
From: fas on 12 Mar 2007 20:48 Hi,Thanks I would like to know that for the case beta||Ax-b|| +lambda^2||x||, where beta is the weighting factor is following solution that you give me works. As I thought Generalized Tikhonov regularization means weigting factor. Or in other words I want weithed least square minimization with regularization. Can you comment. Thanks, On Mar 9, 7:13 am, "spasmous" <spasm...(a)gmail.com> wrote: > fas wrote: > > Hi for Tikhonov regulaization of Ax-b we can write e.g > > > A = rand(1000,2000); > > b = rand(1000,1); > > > % regularize > > lamba = 0.1; > > A = [A;lambda^2*eye(2000)]; > > b = [b;zeros(2000,1)]; > > x = lsqr(A,b); > > > However, for Generalized Tikhonov regularization, can any one suggest > > me how to do it matlab. > > Thanks. > > Following wikipedia (http://en.wikipedia.org/wiki/ > Tikhonov_regularization): > > P = diag(rand(100,1)); > Q = diag(rand(200,1)); > x0 = rand(200,1); > lambda = 0.1; > > A = rand(100,200); > b = rand(100,1); > x = lsqr(A,b); % minimizes ||Ax-b|| > > A2 = [sqrt(P)*A;lambda*sqrt(Q)]; > b2 = [sqrt(P)*b;lamba*sqrt(Q)*x0]; > > x2 = lsqr(A2,b2); % minimizes ||sqrt(P)*(Ax-b)||^2 + lambda^2*|| > sqrt(Q)*(x-x0)||^2
From: spasmous on 13 Mar 2007 15:40
On Mar 12, 5:48 pm, "fas" <faisalmu...(a)gmail.com> wrote: > Hi,Thanks > I would like to know that > for the case beta||Ax-b|| +lambda^2||x||, where beta is the weighting > factor is following solution that you give me works. As I thought > Generalized Tikhonov regularization means weigting factor. Or in other > words I want weithed least square minimization with regularization. > Can you comment. > Thanks, No comment. Go and read the wikipedia article then formulate a coherent question and post it. |