smallest singular value of large, tall, sparse matrix
in [Matlab]
|
Prev: NI USB-6501
Next: Multiple Tabs with dlmwrite
From: Jakob Jørgensen on 27 Jul 2010 16:17 I have a large, tall, sparse M-by-N matrix A, where for instance M=100000 and N=80000. I want to determine whether or not rank(A) = N. As far as I can see, the problem is equivalent to determining whether or not the N'th singular value of A is nonzero. Commands rank and svd fail since A is sparse, and svds tries to do a factorization, and runs out of memory or takes forever. sprank gives an upper bound of the rank, but returns N for my problem, which is not helpful. Any other suggestions on how to determine whether or not rank(A) = N? Thank you.
From: Matt J on 27 Jul 2010 16:45 "Jakob Jørgensen" <jhjspam(a)gmail.com> wrote in message <i2nes3$18v$1(a)fred.mathworks.com>... > I have a large, tall, sparse M-by-N matrix A, where for instance M=100000 and N=80000. I want to determine whether or not rank(A) = N. As far as I can see, the problem is equivalent to determining whether or not the N'th singular value of A is nonzero. Commands rank and svd fail since A is sparse, and svds tries to do a factorization, and runs out of memory or takes forever. sprank gives an upper bound of the rank, but returns N for my problem, which is not helpful. Any other suggestions on how to determine whether or not rank(A) = N? Thank you. ================== I don't know how reliable this is for what you're trying to do, but if you do A\zeros(N,1) you should get a warning if the solver perceives A to be rank deficient.
From: Bruno Luong on 27 Jul 2010 17:48 "Jakob Jørgensen" <jhjspam(a)gmail.com> wrote in message <i2nes3$18v$1(a)fred.mathworks.com>... > I have a large, tall, sparse M-by-N matrix A, where for instance M=100000 and N=80000. I want to determine whether or not rank(A) = N. As far as I can see, the problem is equivalent to determining whether or not the N'th singular value of A is nonzero. Commands rank and svd fail since A is sparse, and svds tries to do a factorization, and runs out of memory or takes forever. sprank gives an upper bound of the rank, but returns N for my problem, which is not helpful. Any other suggestions on how to determine whether or not rank(A) = N? Thank you. Have you tried SVDS(A, 1, 0) or EIGS of H := A'*A (size N x N) with 'SM' option? Bruno
From: Jakob Heide Jørgensen on 27 Jul 2010 19:34 "Matt J " <mattjacREMOVE(a)THISieee.spam> wrote in message <i2ngh4$ki2$1(a)fred.mathworks.com>... > "Jakob Jørgensen" <jhjspam(a)gmail.com> wrote in message <i2nes3$18v$1(a)fred.mathworks.com>... > > I have a large, tall, sparse M-by-N matrix A, where for instance M=100000 and N=80000. I want to determine whether or not rank(A) = N. As far as I can see, the problem is equivalent to determining whether or not the N'th singular value of A is nonzero. Commands rank and svd fail since A is sparse, and svds tries to do a factorization, and runs out of memory or takes forever. sprank gives an upper bound of the rank, but returns N for my problem, which is not helpful. Any other suggestions on how to determine whether or not rank(A) = N? Thank you. > ================== > > I don't know how reliable this is for what you're trying to do, but if you do > A\zeros(N,1) > you should get a warning if the solver perceives A to be rank deficient. Thank you for the quick reply. For small A, your suggestion seems to work. Unfortunately Matlab returns a segmentation fault, when I try to run it for my large A on our high memory server. I will try to determine why. The documentation of mldivide says that the rank is determined from a QR factorization with pivoting. Using R = qr(A,0) I get the same segmentation fault.
From: Matt J on 28 Jul 2010 02:03
"Jakob Heide Jørgensen" <jhjspam(a)gmail.com> wrote in message <i2nqdt$r7l$1(a)fred.mathworks.com>... > Thank you for the quick reply. > > For small A, your suggestion seems to work. Unfortunately Matlab returns a segmentation fault, when I try to run it for my large A on our high memory server. I will try to determine why. > > The documentation of mldivide says that the rank is determined from a QR factorization with pivoting. Using R = qr(A,0) I get the same segmentation fault. ======================= And when you try Bruno's solution? |