Prev: Autocalibration for camera
Next: apparently simple graphic won't print when SaveAs[] PDF
From: jihane on 27 Feb 2010 03:13
I am currently having trouble achieving something on mathematica and I would greatly appreciate your help.
My problem is the following:

I want to construct a loop that will read data and when data<0.1, put it data in a list and do so until data>0.1. the problem is the data fluctuates and i have kind of sinusoidals if i plot the raw data.What I want to do is ask the program to start puting the data<0.1 in a list until data>0.1 and then stop, then plot the data. Then the list would be reinitialized and the program would start from where it stopped and when it reaches values of data<0.1 start putting the data in the list again and stop again when data>0.1 and plot the data and so on so forth. I would then have plots of each kind of sinusoidals individually and not all of them together in one plot as i did previously.

Thank you very much

From: Bob Hanlon on 28 Feb 2010 05:01

data = RandomReal[{0, .125}, 25];

plotData = SplitBy[
data /. x_?(# > .1 &) -> Null,
NumericQ] /.
{Null, ___} -> Sequence[];

nColumns = 2;

Grid[Partition[Join[
ListLinePlot[#,
Frame -> True, Axes -> False,
ImageSize -> 256,
PlotRange -> {-0.005, 0.105},
PlotMarkers -> {Automatic, 8}] & /@
plotData,
Table["", {nColumns - 1}]],
nColumns]]


Bob Hanlon

---- jihane <jihane.ajaja(a)mail.mcgill.ca> wrote:

=============
I am currently having trouble achieving something on mathematica and I would greatly appreciate your help.
My problem is the following:

I want to construct a loop that will read data and when data<0.1, put it data in a list and do so until data>0.1. the problem is the data fluctuates and i have kind of sinusoidals if i plot the raw data.What I want to do is ask the program to start puting the data<0.1 in a list until data>0.1 and then stop, then plot the data. Then the list would be reinitialized and the program would start from where it stopped and when it reaches values of data<0.1 start putting the data in the list again and stop again when data>0.1 and plot the data and so on so forth. I would then have plots of each kind of sinusoidals individually and not all of them together in one plot as i did previously.

Thank you very much



From: jihane on 2 Mar 2010 04:45
Mr hanlon:

thank you very much for the help.
I tried to implement this in my code but i didn't succeed. If you would like to check my code and perhaps indicate to me how i could implement it in my code I would really appreciate:

<< PlotLegends`

(* Change Flight number in labelling *)

SetDirectory[NotebookDirectory[]];

cutOffFreq = 20;
fileTime = 85;


(**********************USER INPUT***********************************)
\

(*Specify column numbers within text file to be extracted*)
fineTimeCol = 1;

X0Col = 6;
Y0Col = 7;
Z0Col = 8;

(*Total number of columns in the text file*)
numberOfColumns = 12;


(*For RMS plots*)
(*Specify how often to calculate RMS values; i.e. every ___ seconds*)
\

RMSseconds = 1;

(*For PSD plots*)
(*Choose how often to integrate PSD plots to find RMS contribution*)
\

fileTime = 85;
numSeconds = 60;




RawDataPlots = True;

(********************************************************************)


(*The samples are collected at 32Hz for a85 minute file*)

sampleFreq = 32;

(*Read data from file*)
data = Import["UofT_Nanowire_Flight1 - Copy.dat"];

listOfZeroes = Table[0, {i, numberOfColumns}];

(*Specify which columns to read*)
td = ReplacePart[listOfZeroes, fineTimeCol -> 1];

x0d = ReplacePart[listOfZeroes, X0Col -> 1];
y0d = ReplacePart[listOfZeroes, Y0Col -> 1];
z0d = ReplacePart[listOfZeroes, Z0Col -> 1];

(*Split data into appropriate columns, drop "EndOfFile" at the end of \
the list*)


X0D = Drop[data.x0d, -1];
Y0D = Drop[data.y0d, -1];
Z0D = Drop[data.z0d, -1];


(********************************************************************************************************************)
\

(*Start analysis for each group of X, Y, Z*)
(********************************************************************************************************************)

(*Assign the X, Y, Z lists*)

XD = X0D;
YD = Y0D;
ZD = Z0D;

XDNew = {};
YDNew = {};
ZDNew = {};

LZ = Length[ZD];

For[j = 1, j <= LZ, j++;
If[ZD[[j]] < 0.05, ZDNew = Append[ZDNew, ZD[[j]]];
XDNew = Append[XDNew, XD[[j]]]; YDNew = Append[YDNew, YD[[j]]]];
];


(*xMean= mean of raw data rounded to 6 decimal places*)
xMean = N[Mean[XD], 6];
yMean = N[Mean[YD], 6];
zMean = N[Mean[ZD], 6];

xMeanNew = N[Mean[XDNew], 6];
yMeanNew = N[Mean[YDNew], 6];
zMeanNew = N[Mean[ZDNew], 6];

(*Demean the raw data*)
XDemeaned = XD - xMean;
YDemeaned = YD - yMean;
ZDemeaned = ZD - zMean;

XDemeanedNew = XDNew - xMeanNew;
YDemeanedNew = YDNew - yMeanNew;
ZDemeanedNew = ZDNew - zMeanNew;

(*Create list of course time based on number of data points and \
sample frequency*)
TD = (Range[Length[XD]] - 1)/sampleFreq;
TDNew = (Range[Length[ZDNew]] - 1)/sampleFreq;


(*Time interval*)
dt = TD[[2]] - TD[[1]];
dtNew = TDNew[[2]] - TDNew[[1]];


(******************Raw Data Plots************************************)

(*Plot raw data plots if specified*)
If[RawDataPlots,
(* 'time' is a list of numbers representing the time at which each \
data point was collected *)
time = TD/60;
timeNew = TDNew/60;


(* 'XDV' is a two dimensional list represengting the voltage \
measurements versus time *)
XDV = Transpose[{time, XD}];
YDV = Transpose[{time, YD}];
ZDV = Transpose[{time, ZD}];

XDVNew = Transpose[{timeNew, XDNew}];
YDVNew = Transpose[{timeNew, YDNew}];
ZDVNew = Transpose[{timeNew, ZDNew}];


(* The following exports plots of the acceleration data for each \
axis into the folder specified at the beginning of this file *)
xName = "X-Axis Accelerometer Measurement.gif";
yName = "Y-Axis Accelerometer Measurement.gif" ;
zName = "Z-Axis Accelerometer Measurement.gif";

xOut = StringForm["X-Axis Accelerometer Measurement - Flight 1"];
yOut = StringForm["Y-Axis Accelerometer Measurement - Flight 1"];
zOut = StringForm["Z-Axis Accelerometer Measurement - Flight 1"];

Export[xName,
Framed[ListLinePlot[XDVNew, PlotRange -> All,
PlotLabel -> Style[xOut, 20], Frame -> True,
FrameLabel -> {"Time (min)",
"Measured Acceleration (g/\!\(\*SubscriptBox[\"g\", \
\"0\"]\))"}, LabelStyle -> Directive[15], ImageSize -> Scaled[1],
Axes -> False]]];
Export[yName,
Framed[ListLinePlot[YDVNew, PlotRange -> All,
PlotLabel -> Style[yOut, 20], Frame -> True,
FrameLabel -> {"Time (min)",
"Measured Acceleration (g/\!\(\*SubscriptBox[\"g\", \
\"0\"]\))"}, LabelStyle -> Directive[15], ImageSize -> Scaled[1],
Axes -> False]]];
Export[zName,
Framed[ListLinePlot[ZDVNew, PlotRange -> All,
PlotLabel -> Style[zOut, 20], Frame -> True,
FrameLabel -> {"Time (min)",
"Measured Acceleration (g/\!\(\*SubscriptBox[\"g\", \
\"0\"]\))"}, LabelStyle -> Directive[15], ImageSize -> Scaled[1],
Axes -> False]]];
];

From: Bob Hanlon on 2 Mar 2010 08:00

xd = RandomReal[{0, 1}, 10];

yd = RandomReal[{0, 1}, 10];

zd = RandomReal[{0, .1}, 10];

use Pick to select elements of xd and yd based on corresponding element of zd

xdnew = Pick[xd, zd, _?(# < .05 &)];

ydnew = Pick[yd, zd, _?(# < .05 &)];

zdnew = Select[zd, # < .05 &];


N[Mean[xd], 6] does not round

Mean[xd] // InputForm

0.45700132031309676

N[Mean[xd], 6] // InputForm

0.45700132031309676

Round[Mean[xd], 10.^-6] // InputForm

0.457001

But then, why would you round in the middle of your calculations?


Bob Hanlon

---- jihane <jihane.ajaja(a)mail.mcgill.ca> wrote:

=============
Mr hanlon:

thank you very much for the help.
I tried to implement this in my code but i didn't succeed. If you would like to check my code and perhaps indicate to me how i could implement it in my code I would really appreciate:

<< PlotLegends`

(* Change Flight number in labelling *)

SetDirectory[NotebookDirectory[]];

cutOffFreq = 20;
fileTime = 85;


(**********************USER INPUT***********************************)
\

(*Specify column numbers within text file to be extracted*)
fineTimeCol = 1;

X0Col = 6;
Y0Col = 7;
Z0Col = 8;

(*Total number of columns in the text file*)
numberOfColumns = 12;


(*For RMS plots*)
(*Specify how often to calculate RMS values; i.e. every ___ seconds*)
\

RMSseconds = 1;

(*For PSD plots*)
(*Choose how often to integrate PSD plots to find RMS contribution*)
\

fileTime = 85;
numSeconds = 60;




RawDataPlots = True;

(********************************************************************)


(*The samples are collected at 32Hz for a85 minute file*)

sampleFreq = 32;

(*Read data from file*)
data = Import["UofT_Nanowire_Flight1 - Copy.dat"];

listOfZeroes = Table[0, {i, numberOfColumns}];

(*Specify which columns to read*)
td = ReplacePart[listOfZeroes, fineTimeCol -> 1];

x0d = ReplacePart[listOfZeroes, X0Col -> 1];
y0d = ReplacePart[listOfZeroes, Y0Col -> 1];
z0d = ReplacePart[listOfZeroes, Z0Col -> 1];

(*Split data into appropriate columns, drop "EndOfFile" at the end of \
the list*)


X0D = Drop[data.x0d, -1];
Y0D = Drop[data.y0d, -1];
Z0D = Drop[data.z0d, -1];


(********************************************************************************************************************)
\

(*Start analysis for each group of X, Y, Z*)
(********************************************************************************************************************)

(*Assign the X, Y, Z lists*)

XD = X0D;
YD = Y0D;
ZD = Z0D;

XDNew = {};
YDNew = {};
ZDNew = {};

LZ = Length[ZD];

For[j = 1, j <= LZ, j++;
If[ZD[[j]] < 0.05, ZDNew = Append[ZDNew, ZD[[j]]];
XDNew = Append[XDNew, XD[[j]]]; YDNew = Append[YDNew, YD[[j]]]];
];


(*xMean= mean of raw data rounded to 6 decimal places*)
xMean = N[Mean[XD], 6];
yMean = N[Mean[YD], 6];
zMean = N[Mean[ZD], 6];

xMeanNew = N[Mean[XDNew], 6];
yMeanNew = N[Mean[YDNew], 6];
zMeanNew = N[Mean[ZDNew], 6];

(*Demean the raw data*)
XDemeaned = XD - xMean;
YDemeaned = YD - yMean;
ZDemeaned = ZD - zMean;

XDemeanedNew = XDNew - xMeanNew;
YDemeanedNew = YDNew - yMeanNew;
ZDemeanedNew = ZDNew - zMeanNew;

(*Create list of course time based on number of data points and \
sample frequency*)
TD = (Range[Length[XD]] - 1)/sampleFreq;
TDNew = (Range[Length[ZDNew]] - 1)/sampleFreq;


(*Time interval*)
dt = TD[[2]] - TD[[1]];
dtNew = TDNew[[2]] - TDNew[[1]];


(******************Raw Data Plots************************************)

(*Plot raw data plots if specified*)
If[RawDataPlots,
(* 'time' is a list of numbers representing the time at which each \
data point was collected *)
time = TD/60;
timeNew = TDNew/60;


(* 'XDV' is a two dimensional list represengting the voltage \
measurements versus time *)
XDV = Transpose[{time, XD}];
YDV = Transpose[{time, YD}];
ZDV = Transpose[{time, ZD}];

XDVNew = Transpose[{timeNew, XDNew}];
YDVNew = Transpose[{timeNew, YDNew}];
ZDVNew = Transpose[{timeNew, ZDNew}];


(* The following exports plots of the acceleration data for each \
axis into the folder specified at the beginning of this file *)
xName = "X-Axis Accelerometer Measurement.gif";
yName = "Y-Axis Accelerometer Measurement.gif" ;
zName = "Z-Axis Accelerometer Measurement.gif";

xOut = StringForm["X-Axis Accelerometer Measurement - Flight 1"];
yOut = StringForm["Y-Axis Accelerometer Measurement - Flight 1"];
zOut = StringForm["Z-Axis Accelerometer Measurement - Flight 1"];

Export[xName,
Framed[ListLinePlot[XDVNew, PlotRange -> All,
PlotLabel -> Style[xOut, 20], Frame -> True,
FrameLabel -> {"Time (min)",
"Measured Acceleration (g/\!\(\*SubscriptBox[\"g\", \
\"0\"]\))"}, LabelStyle -> Directive[15], ImageSize -> Scaled[1],
Axes -> False]]];
Export[yName,
Framed[ListLinePlot[YDVNew, PlotRange -> All,
PlotLabel -> Style[yOut, 20], Frame -> True,
FrameLabel -> {"Time (min)",
"Measured Acceleration (g/\!\(\*SubscriptBox[\"g\", \
\"0\"]\))"}, LabelStyle -> Directive[15], ImageSize -> Scaled[1],
Axes -> False]]];
Export[zName,
Framed[ListLinePlot[ZDVNew, PlotRange -> All,
PlotLabel -> Style[zOut, 20], Frame -> True,
FrameLabel -> {"Time (min)",
"Measured Acceleration (g/\!\(\*SubscriptBox[\"g\", \
\"0\"]\))"}, LabelStyle -> Directive[15], ImageSize -> Scaled[1],
Axes -> False]]];
];



From: Luci Ellis on 2 Mar 2010 08:02
Dear jihane,

I am not Bob Hanlon, but let's see if I can also be of some help. It's
hard to be sure without seeing your data if this would work. I am
assuming that your data file is some sort of matrix of numbers, with
"End Of File" at the end of each column, as your code suggests.

On 2010-03-02 20:45:49 +1100, jihane said:
>
> << PlotLegends`
>
> (* Change Flight number in labelling *)
>
> SetDirectory[NotebookDirectory[]];
>
> cutOffFreq = 20;
> fileTime = 85;
>
>
> (**********************USER INPUT***********************************)
> \
>
> (*Specify column numbers within text file to be extracted*)
> fineTimeCol = 1;
>
> X0Col = 6;
> Y0Col = 7;
> Z0Col = 8;
>
> (*Total number of columns in the text file*)
> numberOfColumns = 12;
>
>
> (*For RMS plots*)
> (*Specify how often to calculate RMS values; i.e. every ___ seconds*)
> \
>
> RMSseconds = 1;
>
> (*For PSD plots*)
> (*Choose how often to integrate PSD plots to find RMS contribution*)
> \
>
> fileTime = 85;
> numSeconds = 60;
>
>
>
>
> RawDataPlots = True;
>
> (********************************************************************)
>
>
> (*The samples are collected at 32Hz for a85 minute file*)
>
> sampleFreq = 32;
>
> (*Read data from file*)
> data = Import["UofT_Nanowire_Flight1 - Copy.dat"];
>
> listOfZeroes = Table[0, {i, numberOfColumns}];
>
> (*Specify which columns to read*)
> td = ReplacePart[listOfZeroes, fineTimeCol -> 1];
>
> x0d = ReplacePart[listOfZeroes, X0Col -> 1];
> y0d = ReplacePart[listOfZeroes, Y0Col -> 1];
> z0d = ReplacePart[listOfZeroes, Z0Col -> 1];
>
> (*Split data into appropriate columns, drop "EndOfFile" at the end of \
> the list*)
>
>
> X0D = Drop[data.x0d, -1];
> Y0D = Drop[data.y0d, -1];
> Z0D = Drop[data.z0d, -1];


Here is the first thing I'd change. Instead of ReplacePart and
listOfZeroes and so on, why not just use:

X0D = Most[data[[All,X0Col]];
Y0D = Most[data[[All,X0Col]];
Z0D = Most[data[[All,X0Col]];

note that Most[] is equivalent to Drop[x,-1]

>
>
> (********************************************************************************************************************)
\

(*Start
>
> analysis for each group of X, Y, Z*)
> (********************************************************************************************************************)

(*Assign
>
> the X, Y, Z lists*)
>
> XD = X0D;
> YD = Y0D;
> ZD = Z0D;
>
> XDNew = {};
> YDNew = {};
> ZDNew = {};
>
> LZ = Length[ZD];
>
> For[j = 1, j <= LZ, j++;
> If[ZD[[j]] < 0.05, ZDNew = Append[ZDNew, ZD[[j]]];
> XDNew = Append[XDNew, XD[[j]]]; YDNew = Append[YDNew, YD[[j]]]];
> ];

This is where Bob Hanlon's post is relevant. He intended you to replace
the For[] loop above with something like this.

SplitBy[data /. x_?(# > .1 &) -> Null, NumericQ] /. {Null, ___} -> Sequence[]

But because you are really wanting to split a three-vector (X, Y, Z) by
the value of Z, it's a bit trickier.

You need the full syntax of Replace (/.) to specify the level of the
list you are replacing at.

plotData = SplitBy[Replace[Transpose[{X0D,Y0D,Z0D}], x_?(Last[#] > .005
&) -> Null, 1], Head] /. {Null} ->
Sequence[]

To see what is going on, let's break it up. Here are your triples

data = RandomReal[{0, .1}, {30, 3}]

{{0.0695089, 0.041404, 0.000962538}, {0.0542868, 0.0356201,
0.0172699}, {0.0126049, 0.0977735, 0.0267022}, {0.0320958, 0.0640111,
0.0670509}, {0.0248244, 0.0637462, 0.0426013}, {0.0539459, 0.0567951,
0.0899871}, {0.0837307, 0.0309503, 0.0575072}, {0.0844311, 0.035911,
0.0186672}, {0.0266531, 0.0824807, 0.078472}, {0.0546202, 0.0441246,
0.0317306}, {0.0935794, 0.0847413, 0.0922612}, {0.0984859, 0.0546285,
0.000364843}, {0.0189445, 0.00818944, 0.0883822}, {0.00642631, 0.0539467,
0.0780225}, {0.0626157, 0.0372797, 0.0154142}, {0.0317539, 0.0259473,
0.0125016}, {0.0580047, 0.0659501, 0.0684925}, {0.0333913, 0.0775575,
0.0620592}, {0.0303167, 0.0248388, 0.067798}, {0.00594988, 0.0886844,
0.0430903}, {0.0577544, 0.0335647, 0.00181778}, {0.0535661, 0.049339,
0.0987746}, {0.0832517, 0.00895941, 0.0507834}, {0.0284413, 0.0289574,
0.027981}, {0.0943071, 0.000771151, 0.0806029}, {0.0440539, 0.0653912,
0.0788237}, {0.0167852, 0.045078, 0.0358629}, {0.0972686, 0.0207125,
0.0452583}, {0.085462, 0.0508639, 0.0438656}, {0.0441169, 0.0377907,
0.021743}}

You want to get rid of the triples where the last element is greater
than the threshold.

In[42]:= Replace[data, x_?(Last[#] > .08 &) -> Null, 1]
Out[42]= {{0.0695089, 0.041404, 0.000962538}, {0.0542868, 0.0356201,
0.0172699}, {0.0126049, 0.0977735, 0.0267022}, {0.0320958, 0.0640111,
0.0670509}, {0.0248244, 0.0637462, 0.0426013}, Null, {0.0837307, 0.0309503,
0.0575072}, {0.0844311, 0.035911, 0.0186672}, {0.0266531, 0.0824807,
0.078472}, {0.0546202, 0.0441246, 0.0317306}, Null, {0.0984859, 0.0546285,
0.000364843}, Null, {0.00642631, 0.0539467, 0.0780225}, {0.0626157,
0.0372797, 0.0154142}, {0.0317539, 0.0259473, 0.0125016}, {0.0580047,
0.0659501, 0.0684925}, {0.0333913, 0.0775575, 0.0620592}, {0.0303167,
0.0248388, 0.067798}, {0.00594988, 0.0886844, 0.0430903}, {0.0577544,
0.0335647, 0.00181778}, Null, {0.0832517, 0.00895941,
0.0507834}, {0.0284413, 0.0289574, 0.027981}, Null, {0.0440539, 0.0653912,
0.0788237}, {0.0167852, 0.045078, 0.0358629}, {0.0972686, 0.0207125,
0.0452583}, {0.085462, 0.0508639, 0.0438656}, {0.0441169, 0.0377907,
0.021743}}

As you can see, what we have after this step is a list of elements,
some of which are triples (Head is List) and some of which are the
value Null (Head is Symbol). So we can split them according to each
element's Head.

In[60]:= SplitBy[Replace[data, x_?(Last[#] > .08 &) -> Null, 1], Head]

Out[60]= {{{0.0695089, 0.041404, 0.000962538}, {0.0542868, 0.0356201,
0.0172699}, {0.0126049, 0.0977735, 0.0267022}, {0.0320958, 0.0640111,
0.0670509}, {0.0248244, 0.0637462, 0.0426013}}, {Null}, {{0.0837307,
0.0309503, 0.0575072}, {0.0844311, 0.035911, 0.0186672}, {0.0266531,
0.0824807, 0.078472}, {0.0546202, 0.0441246,
0.0317306}}, {Null}, {{0.0984859, 0.0546285,
0.000364843}}, {Null}, {{0.00642631, 0.0539467, 0.0780225}, {0.0626157,
0.0372797, 0.0154142}, {0.0317539, 0.0259473, 0.0125016}, {0.0580047,
0.0659501, 0.0684925}, {0.0333913, 0.0775575, 0.0620592}, {0.0303167,
0.0248388, 0.067798}, {0.00594988, 0.0886844, 0.0430903}, {0.0577544,
0.0335647, 0.00181778}}, {Null}, {{0.0832517, 0.00895941,
0.0507834}, {0.0284413, 0.0289574, 0.027981}}, {Null}, {{0.0440539,
0.0653912, 0.0788237}, {0.0167852, 0.045078, 0.0358629}, {0.0972686,
0.0207125, 0.0452583}, {0.085462, 0.0508639, 0.0438656}, {0.0441169,
0.0377907, 0.021743}}}

And then use the final replacement rule {Null..} ->Sequence[] to "get
rid" of the {Null} lists. The double dot should handle cases where
there are successive Null elements in a sublist, which don't appear in
my test data.

You are left with lists of triples. You can plot each subset of triples
using something like this:

Grid[Partition[Graphics3D[{Blue, Thick, Line[#]}] & /@ plotData, 2]]

Or if that isn't what you want, you can split the triples up again into
columns and plot against time.

>
> (*xMean= mean of raw data rounded to 6 decimal places*)
Why do you need to round the data? And did you know you could do all
this in less code like this?

{XDemeaned, YDemeaned, ZDemeaned, XDemeanedNew, YDemeanedNew, ZDemeanedNew } =
Standardize[#, Mean,( 1 &)]& /@ {XD, YD, ZD, XDNew, YDNew, ZDNew}

> xMean = N[Mean[XD], 6];
> yMean = N[Mean[YD], 6];
> zMean = N[Mean[ZD], 6];
>
> xMeanNew = N[Mean[XDNew], 6];
> yMeanNew = N[Mean[YDNew], 6];
> zMeanNew = N[Mean[ZDNew], 6];
>
> (*Demean the raw data*)
> XDemeaned = XD - xMean;
> YDemeaned = YD - yMean;
> ZDemeaned = ZD - zMean;
>
> XDemeanedNew = XDNew - xMeanNew;
> YDemeanedNew = YDNew - yMeanNew;
> ZDemeanedNew = ZDNew - zMeanNew;
>
> (*Create list of course time based on number of data points and \
> sample frequency*)
> TD = (Range[Length[XD]] - 1)/sampleFreq;
> TDNew = (Range[Length[ZDNew]] - 1)/sampleFreq;
>
>
> (*Time interval*)
> dt = TD[[2]] - TD[[1]];
> dtNew = TDNew[[2]] - TDNew[[1]];


{extra code snipped}

I hope that helps.

Regards,
Luci



 |  Next  |  Last
Pages: 1 2
Prev: Autocalibration for camera
Next: apparently simple graphic won't print when SaveAs[] PDF