Repeat Data reading in one file
in [Mathematica]
|
From: graser on 16 Dec 2009 06:19 Hello I am posting my question.. The question is I have multiple files with following data structure.. Header..1 Header..2 1 3.12 2. 2.54 .. .. .. Header...2 1. 1.24 2. 5.44 .. .. .. .. Header...2 1. 1.24 2. 5.44 .. .. .. .. I tried to take out only the number in each data set. So I used a script which I learned from this group like that.. FileList = FileNames["*.txt"]; Block[{str}, str = OpenRead[#]; Skip[str, Record, 1, NullRecords -> False]; dat[#] = ReadList[str, Table[Number, {i, 1, 2}], RecordSeparators -> "Header..2]"]; Close[str]] & /@ FileList But with this script, it just stop reading in first data set in each file and at the end of file, it said {{1, 3.12}, {2, 2.54},..... $Failed} .. Do I have to use any loop for that? Help me Please.. Thanks
From: dh on 17 Dec 2009 07:23 Hi, here is one possibility: - import data as table - drop first 3 lines - split by second header. This puts data in matrices. - extract the matrices Assu,e the data file looks like: s = "Header..1 Header..2 1 3.12 2. 2.54 Header..2 1. 1.24 2. 5.44" the the following code will extract the data and put it into matrices: st = ImportString[s, "Table"];(*import as string*) st = Drop[st, 3](*drop first 3 lines*) st = SplitBy[st, MatchQ[#, {_?NumberQ, _?NumberQ}] &](*make matrices*) st = Select[st, Length[#] > 1 &] This would produce a list containing two matrices: {{{1, 3.12}, {2., 2.54}}, {{1., 1.24}, {2., 5.44}}} Daniel graser wrote: > Hello > > I am posting my question.. > > The question is > > I have multiple files with following data structure.. > > Header..1 > > Header..2 > 1 3.12 > 2. 2.54 > . > . > . > > Header...2 > 1. 1.24 > 2. 5.44 > . > . > . > . > > Header...2 > 1. 1.24 > 2. 5.44 > . > . > . > . > > I tried to take out only the number in each data set. > So I used a script which I learned from this group like that.. > > > FileList = FileNames["*.txt"]; > > Block[{str}, str = OpenRead[#]; > Skip[str, Record, 1, NullRecords -> False]; > dat[#] = > ReadList[str, Table[Number, {i, 1, 2}], > RecordSeparators -> "Header..2]"]; > Close[str]] & /@ FileList > > > But with this script, it just stop reading in first data set in each > file and at the end of file, it said {{1, 3.12}, {2, 2.54},..... > $Failed} .. > > Do I have to use any loop for that? > > Help me Please.. > > Thanks > > >
From: Bill Rowe on 17 Dec 2009 07:27 On 12/16/09 at 6:17 AM, graser(a)gmail.com (graser) wrote: >I am posting my question.. >The question is >I have multiple files with following data structure.. >Header..1 >Header..2 1 3.12 >2. 2.54 You haven't asked a question nor made it clear what you want to do with the files you describe. Perhaps all you need is a pointer on how to read these files with Mathematica. If so, the function likely to be most useful to you is Import. Probably Import[filename,"Table"] will work. For details on using Import see the online documentation available in the Documentation Center.
From: graser on 18 Dec 2009 06:23 On Dec 17, 7:27 am, Bill Rowe <readn...(a)sbcglobal.net> wrote: > On 12/16/09 at 6:17 AM, gra...(a)gmail.com (graser) wrote: > > >I am posting my question.. > >The question is > >I have multiple files with following data structure.. > >Header..1 > >Header..2 1 3.12 > >2. 2.54 > > You haven't asked a question nor made it clear what you want to > do with the files you describe. Perhaps all you need is a > pointer on how to read these files with Mathematica. If so, the > function likely to be most useful to you is Import. Probably > > Import[filename,"Table"] > > will work. For details on using Import see the online > documentation available in the Documentation Center. Hi I am sorry that I wasn't clear to make my question. My question was 1. Each data file has same structure like above.. 2. I want to take out only Numbers or I want to skip headers or comments. 3. In each file, there are a couple of lines of header 1 and multiple data repeats with header 2. But the number of repetition is different in different file. 4. So I want to save only the number data with mathematica.. 5. Import is not working at all.. 6. Tried to with script.. 7. Need you guys..help! I hope it is more clear than before.. Thanks
From: Bill Rowe on 19 Dec 2009 06:25 On 12/18/09 at 6:23 AM, graser(a)gmail.com (graser) wrote: >On Dec 17, 7:27 am, Bill Rowe <readn...(a)sbcglobal.net> wrote: >>On 12/16/09 at 6:17 AM, gra...(a)gmail.com (graser) wrote: >>>I am posting my question.. The question is I have multiple files >>>with following data structure.. Header..1 Header..2 1 3.12 2. 2.54 >>You haven't asked a question nor made it clear what you want to do >>with the files you describe. Perhaps all you need is a pointer on >>how to read these files with Mathematica. If so, the function >>likely to be most useful to you is Import. Probably >>Import[filename,"Table"] >>will work. For details on using Import see the online documentation >>available in the Documentation Center. >My question was >1. Each data file has same structure like above.. >2. I want to take out only Numbers or I want to skip headers or comments. >3. In each file, there are a couple of lines of header 1 and multiple >data repeats with header 2. >But the number of repetition is different in different file. >4. So I want to save only the number data with mathematica.. >5. Import is not working at all.. When you say Import is "not working at all..." what do you mean? Not reading the file information? Or reading more than what you want? Assuming it is the later, then likely the simplest way to achieve your goal given the structure you've indicated would be data = Cases[Import[filename, "Table"], {_?NumericQ,__}] if you want data to be just the numeric data or headers = DeleteCases[Import[filename, "Table"], {_?NumericQ,__}] if you want to only retain the non-numeric lines. This should be adeguate for moderately sized files. If the files are really large, then this will be rather slow due to the overhead Import has and the fact you are reading in more of the file then you really need to. You can read only the stuff you want from the files by opening a stream and writing your own functions to read in the data making use of the built-in function Read. But this is definitely more work and is unlikely to be worthwhile for moderately sized files.
|
Pages: 1 Prev: ParallelTable slows down computation Next: Help needed using NDSolve and NIntegrate |