Questions about the optimzation method.
in [Matlab]
|
From: Bruno Luong on 19 Jul 2010 07:54 "Bruno Luong" <b.luong(a)fogale.findmycountry> wrote in message <i21ale$e37$1(a)fred.mathworks.com>... > > > > Minimizing ||x||^0.8 is non linear and yes, you do need non-linear optimization tool. > > Btw ||x||^0.8 is not differentiable and it's not convex. This is nasty for the minimizer. To avoid numerical problem, the trick I sometime use is to replace ||x||^0.8 by (||x||^2+epsilon)^1.6, where x is a small number compared to the expected 2-norm of the solution. Do not select it too small so as the replacement is effective. Bruno
From: Antony on 19 Jul 2010 08:19 "Bruno Luong" <b.luong(a)fogale.findmycountry> wrote in message <i21edd$9fn$1(a)fred.mathworks.com>... > "Bruno Luong" <b.luong(a)fogale.findmycountry> wrote in message <i21ale$e37$1(a)fred.mathworks.com>... > > > > > > Minimizing ||x||^0.8 is non linear and yes, you do need non-linear optimization tool. > > > > Btw ||x||^0.8 is not differentiable and it's not convex. This is nasty for the minimizer. > > To avoid numerical problem, the trick I sometime use is to replace ||x||^0.8 by (||x||^2+epsilon)^1.6, where x is a small number compared to the expected 2-norm of the solution. Do not select it too small so as the replacement is effective. > > Bruno Thank you very much, Bruno. You mean (||x||+epsilon)^1.6? It will make ||x||^0.8 more complex, I feel. Also thanks a lot for your suggesion. I am trying to find a tool solving the non-linear optimization with constraints of ||x||^0.8 and squared gradient. I find a similar work is called Lasso (http://www-stat.stanford.edu/~tibs/lasso.html). Even there is a tool called SPAMS (Sparse Modeling Software (SPAMS): http://www.di.ens.fr/willow/SPAMS/). But Lasso can not cope with my problem. Anyone else can help me now? *-*. Thanks a lot. -Antony
From: Bruno Luong on 19 Jul 2010 15:26 "Antony " <mutang.bing(a)gmail.com> wrote in message <i21fs2$blc$1(a)fred.mathworks.com>... > "Bruno Luong" <b.luong(a)fogale.findmycountry> wrote in message <i21edd$9fn$1(a)fred.mathworks.com>... > > "Bruno Luong" <b.luong(a)fogale.findmycountry> wrote in message <i21ale$e37$1(a)fred.mathworks.com>... > > > > > > > > Minimizing ||x||^0.8 is non linear and yes, you do need non-linear optimization tool. > > > > > > Btw ||x||^0.8 is not differentiable and it's not convex. This is nasty for the minimizer. > > > > To avoid numerical problem, the trick I sometime use is to replace ||x||^0.8 by (||x||^2+epsilon)^1.6, where x is a small number compared to the expected 2-norm of the solution. Do not select it too small so as the replacement is effective. > > > > Bruno > > Thank you very much, Bruno. You mean (||x||+epsilon)^1.6? It will make ||x||^0.8 more complex, I feel. > Oops what I meant if (||x||^2+epsilon)^0.4. IMO, it is not more complex, it makes the function differentiable and locally convex at ||x||=0. Those kind of "details" in more important than the length of the code or formula. I'll stop here. Bruno
From: Antony on 19 Jul 2010 21:00
"Bruno Luong" <b.luong(a)fogale.findmycountry> wrote in message <i228sa$sd1$1(a)fred.mathworks.com>... .... > > Thank you very much, Bruno. You mean (||x||+epsilon)^1.6? It will make ||x||^0.8 more complex, I feel. > > > > Oops what I meant if (||x||^2+epsilon)^0.4. IMO, it is not more complex, it makes the function differentiable and locally convex at ||x||=0. Those kind of "details" in more important than the length of the code or formula. > > I'll stop here. > > Bruno I see, thanks a lot, Bruno. I appreciate your help! |