Questions about the optimzation method.
in [Matlab]
|
From: Antony on 18 Jul 2010 10:58 Hi, dear all. I wonder whether there is a method that computes the large-scale non-linear least squares problem Ax=b. x is a sparse matrix written in vector form, A is a large-scale matrix (around 400x10000) and b is also a large vector ( around 400x1). There are also some constraints considering the undertermined property of Ax=b. So, how to solve such a problem when there are some additional non-linear constraints of x are added? While I see some work in solving Ax=b, I can not find a solution especially targeting large-scale sparse matrix, not to mention additional non-linear constraints. I ever tried fmincon and lsqnonlin, but the program often stops because of the out-of-memory problem. So could please give me some advice on how to solve such an Ax=b problem? Thanks a lot for your help! Mutang
From: Brian Borchers on 18 Jul 2010 11:48 400x10000 simply isn't large. It might be helpful to exploit the sparsity of the A matrix in solving your problem, but even if it were necessary to work with A as a fully dense matrix, that shouldn't be a problem. You'll need to give more detail about your additional constraints before anyone can reasonably answer your question.
From: Antony on 18 Jul 2010 21:08 Thanks Brian. I wanted to attach an equation to this page yesterday but I don't know how to attach. So before showing the equation, I wonder whether you know how to show an equation in this page? Or I simply write the equation in text format without mathematical symbols here? Thanks. Brian Borchers <borchers.brian(a)gmail.com> wrote in message <0f935e8a-8ddf-4925-aa0c-901d7de55a92(a)f6g2000yqa.googlegroups.com>... > 400x10000 simply isn't large. It might be helpful to exploit the > sparsity of the A matrix in solving your problem, but even if it were > necessary to work with A as a fully dense matrix, that shouldn't be a > problem. > > You'll need to give more detail about your additional constraints > before anyone can reasonably answer your question.
From: Brian Borchers on 18 Jul 2010 22:17 On Jul 18, 7:08 pm, "Antony " <mutang.b...(a)gmail.com> wrote: > Thanks Brian. I wanted to attach an equation to this page yesterday but I don't know how to attach. So before showing the equation, I wonder whether you know how to show an equation in this page? Or I simply write the equation in text format without mathematical symbols here? Thanks. If possible, put your equation in text format. If that's not possible, most readers of this group understand LaTeX notation.
From: Antony on 19 Jul 2010 00:37
Brian Borchers <borchers.brian(a)gmail.com> wrote in message <b4dd9eeb-e56c-4448-9e52-717916ce8465(a)y4g2000yqy.googlegroups.com>... > > If possible, put your equation in text format. If that's not > possible, most readers of this group understand LaTeX notation. OK. Thank you! I try to write the inverse problem as below: min_x {(Ax-b)^2 + \lambda_1 ||x|| + \lambda_2 ||gradient(x)||} s.t. x>0 where Ax-b is refomulated from Ax=b, ||x|| is the Tikhonov regularizer and ||gradient(x)|| is the smoothness regularizer. I have tried to solve this minimization problem with fmincon and lsqnonlin. I will list the questions below: 1) when I use fmincon, it seems that only interior-point algorithm works. For interior-point, When the MaxFunEvals is relatively small ( around 5000), i can obtain the result although it is incorrect. But when I set the MaxFunEvals to be 10000, there will be out of memory error because the total virtual memory will be used more than 1.65 Gigabytes. 2) when I use lsqnonlin and try all algorithms supported (subspace trust region method, Levenberg-Marquardt and line search), none of them works. For the trust-region-reflective algorithm, the nonlinear system of equations cannot be underdetermined; The Levenberg-Marquardt algorithm does not handle bound constraints. At the moment, I don't know how to cope with this minimization problem. Thanks a lot for your suggestions and helps! Antony |