Prev: Use of kstest() function in Matlab
Next: Sort vector with out using sort function
From: Eric Diaz on 26 Apr 2010 17:09
Dear Bruno,

> 1- Are you asking how to compute the Jacobian of the non-linear projection? It is obvious that the Jacobian where users must provide is the derivative of the model with respect to the optimized parameter(s), and nothing more.
>>
In the example setup, the function called by lsqcurvefit returns my yhat and Jacobian, without the use of separation.

Now, if I were to implement the same approach as given by Matlab in their technical solution document, then there is confusion.

The confusion lies wherein
the example they give has linear coefficients as the 2nd output, and my function has the Jacobian as the 2nd output. If I implement backslash for the linear terms, then would I have 3 outputs for my function?

Using the setup example: would I get?

function [Sig, lincoeff, Jacob] = myfuncSep(p,xdata,ydata)
% This function is called by lsqcurvefit.
% p is a vector which contains the coefficients of the
% equation. xdata is the data set passed to lsqcurvefit.

B=p;

% Separately solve for the linear coefficients
systemeq =[exp(-xdata (:)/B)];
lincoeff = systemeq\ydata(:);
A=lincoeff(1);

Sig = A.*exp(-xdata./B);
Jp1 = exp(-xdata./B);
Jp2 = A.*(xdata./(B.^2).*exp(-xdata./B);
Jacob = [Jp1; Jp2]';
end


> 2- If yes, do you know how to compute the Jacobian in 1 above?
>>Isn't it just as you mention in 1), where it is the derivative of the model with respect to the optimized parameters?

> 3- If no, do you know how to compute the Jacobian of the original problem (i.e., without projection, I guess the answer is yes that this point from what I read)?
>>Yes

> 4- If yes are you able to compute 2 from 3?
>>Well, I would assume so, but perhaps my assumption is incorrect.

> 5- If no have you heard about Golub/Pereyra formula?
>>Yes, however, not from the original paper. I have been looking at a more recent publication by Osborne 2007 "separable least squares, variable projection, and the gauss-newton algorithm" dedicated to Golub for his 75th birthday.



> 6- If no try to get this paper: "G. H. Golub and V. Pereyra, The differentiation of pseudoinverses and nonlinear least squares problems whose variables separate, SIAM J. Numer. Anal., 10 (1973), pp. 413–432.'
>>I will look into it

Thanks for your interest and pointers,

Regards,

Eric Diaz
From: Bruno Luong on 26 Apr 2010 17:39
"Eric Diaz" <eric.diaz(a)gmail.com> wrote in message <hr4vdg$h41$1(a)fred.mathworks.com>...

>
> Using the setup example: would I get?
>
> function [Sig, lincoeff, Jacob] = myfuncSep(p,xdata,ydata)
> % This function is called by lsqcurvefit.
> % p is a vector which contains the coefficients of the
> % equation. xdata is the data set passed to lsqcurvefit.
>
> B=p;
>
> % Separately solve for the linear coefficients
> systemeq =[exp(-xdata (:)/B)];
> lincoeff = systemeq\ydata(:);
> A=lincoeff(1);
>
> Sig = A.*exp(-xdata./B);
> Jp1 = exp(-xdata./B);
> Jp2 = A.*(xdata./(B.^2).*exp(-xdata./B);
> Jacob = [Jp1; Jp2]';
> end

I have no idea why they return lincoeff in the second output, but obviously as they do not turn on the Jacobian option, it does not interfere. In you case, if you turn the Jacobian option "on", you should return the Jacobian as second output. In this specific example Jacobian is just a column vector, since there is only one (1) parameter.

>
>
> > 2- If yes, do you know how to compute the Jacobian in 1 above?
> >>Isn't it just as you mention in 1), where it is the derivative of the model with respect to the optimized parameters?

Yes,

>
> > 3- If no, do you know how to compute the Jacobian of the original problem (i.e., without projection, I guess the answer is yes that this point from what I read)?
> >>Yes
>
> > 4- If yes are you able to compute 2 from 3?
> >>Well, I would assume so, but perhaps my assumption is incorrect.

You can, by the Golub/Pereyra formula. In some cases, this formula is too complicated for practical use, and people often prefer making an approximation of the Jacobian by finite difference. I believe this is more or less equivalent to turning the Jacobian option 'off' and let the minimizer in charge of the estimation of the gradient.

>
> > 5- If no have you heard about Golub/Pereyra formula?
> >>Yes, however, not from the original paper. I have been looking at a more recent publication by Osborne 2007 "separable least squares, variable projection, and the gauss-newton algorithm" dedicated to Golub for his 75th birthday.

Yeap, there are a tone of papers on this topic, especially when the linear-separable problems come up, GP's formula and GN's algorithm are often used. I have used it in my Free-knot spline FEX submission.

Good luck,

Bruno
First  |  Prev  | 
Pages: 1 2 3
Prev: Use of kstest() function in Matlab
Next: Sort vector with out using sort function