how to transpose large matrix?
in [Shell]
|
From: Greg Russell on 4 Jun 2010 04:12 "Janis Papanagnou" <janis_papanagnou(a)hotmail.com> wrote in message news:hua1jh$4j3$1(a)news.m-online.net... > You're already aware of the problem that you have in principle with > this type of task. One more thought; are your matrices fully filled > or sparsely populated? Excellemt point. > In the latter case you might be able to use > the all-in-memory approach anyway, because you'd need just allocate > the actual values and leave the zero-values away. Are you assuming that zero values comstitute "sparse" elements of the array? The OP made no such indication, only that "... columns [are] separated by tabs, rows [are] separated by newlines." Null values e.g. ...\t\t... are far more reasonable for the sparse entries, but of course the OP can define the data structure that is of question. > (In any language, like awk, that supports associative arrays this > would require just a few lines of code.) Please educate us in such simplicity ... please. Even such well-known routines as the R/S/SPlus http://cran.r-project.org tramspose function, or the Fortran transpose routines from http://ornl.gov (U.S. Oak Ridge National Laboratories) that we used 20+ years ago on such arrays, are more than "a few lines of code." If you can provide the "few lines" of awk necessary then you too can be famous rather than merely flippant.
From: Janis Papanagnou on 4 Jun 2010 04:29 Greg Russell wrote: > "Janis Papanagnou" <janis_papanagnou(a)hotmail.com> wrote in message > news:hua1jh$4j3$1(a)news.m-online.net... > >> You're already aware of the problem that you have in principle with >> this type of task. One more thought; are your matrices fully filled >> or sparsely populated? > > Excellemt point. > >> In the latter case you might be able to use >> the all-in-memory approach anyway, because you'd need just allocate >> the actual values and leave the zero-values away. > > Are you assuming that zero values comstitute "sparse" elements of the array? No, that's an arbitrary value, generally it's problem dependent. > The OP made no such indication, only that "... columns [are] separated by > tabs, rows [are] separated by newlines." > > Null values e.g. ...\t\t... are far more reasonable for the sparse entries, > but of course the OP can define the data structure that is of question. > >> (In any language, like awk, that supports associative arrays this >> would require just a few lines of code.) > > Please educate us in such simplicity ... please. Sure. Here's one way to implement it... awk ' BEGIN { FS = "\t" } { for (col=1; col<=NF; col++) if ($col) matrix[NR,col] = $col } END { for (col=1; col<=NF; col++) for (row=1; row<=NR; row++) printf("%s%s", ((row,col) in matrix) ? matrix[row,col] : 0, (row<NR) ? FS : RS) } ' Janis > > Even such well-known routines as the R/S/SPlus http://cran.r-project.org > tramspose function, or the Fortran transpose routines from http://ornl.gov > (U.S. Oak Ridge National Laboratories) that we used 20+ years ago on such > arrays, are more than "a few lines of code." > > If you can provide the "few lines" of awk necessary then you too can be > famous rather than merely flippant. > > >
From: Janis Papanagnou on 4 Jun 2010 04:30 > Greg Russell wrote: >> >> If you can provide the "few lines" of awk necessary then you too can be >> famous rather than merely flippant. BTW, neither one of your choices is what I am aiming at. Janis
From: Thomas 'PointedEars' Lahn on 4 Jun 2010 05:26 Janis Papanagnou wrote: > [...] (In any language, like awk, that supports associative arrays this > would require just a few lines of code.) If your matrices not sparsely > populated then follow the way of using temporary files if your memory is > limited. Temporary files and arrays are unnecessary. You can read and print the columns as rows one by one. PointedEars
From: Maxwell Lol on 4 Jun 2010 07:56
Thomas 'PointedEars' Lahn <PointedEars(a)web.de> writes: > Janis Papanagnou wrote: > > > Temporary files and arrays are unnecessary. You can read and print the > columns as rows one by one. Yes, by re-reading the input file each time you want a new row. |