Prev: ellipse points regularly spaced, or inverse of the incomplete
Next: Energy of signal
From: Camron Call on 26 May 2010 15:21
I want to read a file that is organized with semicolons (;) as a delimeter. The file is organized into 4 columns where each row contains a variable name, value, units, in/out designation.

Example:

Variable.With.Various.Unique.Levels.name1;Value;Units;in
Variable.With.Various.Unique.Levels.name2;Value;Units;out

Value could be anything
1
10946.0386113404
19173.112909708834,0,-722.71884819229354
"2006.6,0; 2000.60,131.63; 1982.65,272.563;" <-- This one is my problem
or any string

Units will always be a string
The last column will always be a string, 'in' or 'out'

I would just use textscan with ; as delim, but Value can sometimes have ; that are between double quotes. " " Making textscan harder to use. I would like anything inside the double quotes to be read as a string to be processed later.

Ideas?

Please ask for clarification if needed.
From: TideMan on 26 May 2010 16:54
On May 27, 7:21 am, "Camron Call" <camronc...(a)gmail.cam> wrote:
> I want to read a file that is organized with semicolons (;) as a delimeter.  The file is organized into 4 columns where each row contains a variable name, value, units, in/out designation.  
>
> Example:
>
> Variable.With.Various.Unique.Levels.name1;Value;Units;in
> Variable.With.Various.Unique.Levels.name2;Value;Units;out
>
> Value could be anything
> 1
> 10946.0386113404
> 19173.112909708834,0,-722.71884819229354
> "2006.6,0; 2000.60,131.63; 1982.65,272.563;"      <-- This one is my problem
> or any string
>
> Units will always be a string
> The last column will always be a string, 'in' or 'out'
>
> I would just use textscan with ; as delim, but Value can sometimes have ; that are between double quotes. " "  Making textscan harder to use.  I would like anything inside the double quotes to be read as a string to be processed later.
>
> Ideas?
>
> Please ask for clarification if needed.  

So why not read everything in as a string?

BTW, your examples are as clear as mud.
The numerical example does not seem to match:
Variable.With.Various.Unique.Levels.name1;Value;Units;in

Can you just show us a few lines from the file without editorial
comments?
From: Camron Call on 26 May 2010 19:26
The rows are formatted with:
variable name ; value ; units ; in/out

The data is a text file with rows like the following:

Aircraft.Mass.Calibration.Name1;1;;in
Aircraft.Mass.Calibration.Name2;1;;in
Aircraft.Mass.Calibration.Name3;12345.6789;lb;in
Aircraft.Mass.Certification.Name4;123456;lb;in
Aircraft.Mass.Certification.Name5;12345;lb;in
Aircraft.Mass.Design.Category1.Name6;12345.6789,0,-1234.92;;out
Aircraft.Mass.Design.Name7;12345.678;lb;out
Aircraft.Operations.Name8;1.234;;in
Aircraft.Operations.Name9;123;kts;in
Aircraft.Operations.Name10;1.2;;in
Aircraft.Components.Category2.Name11;7556.63837287839;gal;out
Aircraft.Components.Category3.Category4.Name12;"2006.6,0;2000.60,131.63; ";;out
FlightPerformance.DesignMission.Category5.Name13;True;;in
FlightPerformance.DesignMission.Category6.Name14;"10; 30; 50; 70; 90; 110; ";;in

I wondered if there is any way to get this data into the workspace without using fgetl in a loop and parsing each line individually. The final goal is to preserve the variable hierarchy in a cell array, or structure or dataset or something and be able to access the variable name, value, units, and in/out.
From: TideMan on 26 May 2010 19:48
On May 27, 11:26 am, "Camron Call" <camronc...(a)gmail.cam> wrote:
> The rows are formatted with:
> variable name ; value ; units ; in/out
>
> The data is a text file with rows like the following:
>
> Aircraft.Mass.Calibration.Name1;1;;in
> Aircraft.Mass.Calibration.Name2;1;;in
> Aircraft.Mass.Calibration.Name3;12345.6789;lb;in
> Aircraft.Mass.Certification.Name4;123456;lb;in
> Aircraft.Mass.Certification.Name5;12345;lb;in
> Aircraft.Mass.Design.Category1.Name6;12345.6789,0,-1234.92;;out
> Aircraft.Mass.Design.Name7;12345.678;lb;out
> Aircraft.Operations.Name8;1.234;;in
> Aircraft.Operations.Name9;123;kts;in
> Aircraft.Operations.Name10;1.2;;in
> Aircraft.Components.Category2.Name11;7556.63837287839;gal;out
> Aircraft.Components.Category3.Category4.Name12;"2006.6,0;2000.60,131.63; ";;out
> FlightPerformance.DesignMission.Category5.Name13;True;;in
> FlightPerformance.DesignMission.Category6.Name14;"10; 30; 50; 70; 90; 110; ";;in
>
> I wondered if there is any way to get this data into the workspace without using fgetl in a loop and parsing each line individually.  The final goal is to preserve the variable hierarchy in a cell array, or structure or dataset or something and be able to access the variable name, value, units, and in/out.

Aah, I see the problem now.
The Value column is fairly chaotic, isn't it?
Sometimes it has a number - that's easy.
Sometimes it has text - that's easy
But sometimes it is a string enclosed in double quotes, and if that is
the case the stuff inside the double quotes is delimited by either a
semicolon or a comma. So, textscan should ignore the semicolons if
they are inside double quotes.

I'm afraid this is so idiosyncratic that I don't see any alternative
to using fgetl in a loop and parsing line by line.
Not much help I'm afraid.
From: Walter Roberson on 26 May 2010 23:06
Camron Call wrote:

> Aircraft.Components.Category2.Name11;7556.63837287839;gal;out
> Aircraft.Components.Category3.Category4.Name12;"2006.6,0;2000.60,131.63;
> ";;out FlightPerformance.DesignMission.Category5.Name13;True;;in
> FlightPerformance.DesignMission.Category6.Name14;"10; 30; 50; 70; 90;
> 110; ";;in

> I wondered if there is any way to get this data into the workspace
> without using fgetl in a loop and parsing each line individually. The
> final goal is to preserve the variable hierarchy in a cell array, or
> structure or dataset or something and be able to access the variable
> name, value, units, and in/out.

If it were me, I would pre-process the input file into another form,
using perl or awk or sed or an editor like vi.
 |  Next  |  Last
Pages: 1 2
Prev: ellipse points regularly spaced, or inverse of the incomplete
Next: Energy of signal