Prev: data storage
Next: a question about HMM functions in MATLAB
From: Joshua Arnott on 21 Apr 2010 13:52
Hello everyone,

I have some temperature profile data (elevation [Z] in meters, temperature [T] in celcius), sampled every 0.001m. See figure linked below; left plot is an entire profile, right plot is a smaller section of the same profile.

http://bit.ly/b0MKPv

The fluid this is measured from is quite stratified; there are distinct points at which the gradient (dT/dZ) changes. My goal is to identify these break points.

My difficulties in doing this are a result of noise (at least, what I consider noise) on the data. The smallest stratified sections I wish to identify are around 15 samples (in the example figure, this one is around 60 samples) and so I need to denoise or simplify the signal in some way. The problem I have come across is that (for example, with wavelet denoising on the first derivative) I find often when I smooth the signal enough to remove the noise in sections between the knots I want to identify, exact points at which the changes I want to detect occur are blurred as a result of the smoothing and I am one or two samples out in their location. I really want to be able to identify the exact knot; something I expected easy to do (as it is "by eye") but which I have found is not.

I have experimented with the SLM and BSFK tools available on the FEX, which seem to perform well on the larger stratification structure of the profile, but due to the number of samples (around 14000) and unknown number of knots seem unsuitable for the finer resolution results I desire.

Any direction on this would be appreciated.

Thanks,
Josh.
From: Bruno Luong on 21 Apr 2010 14:10
Joshua,
You might take a simpler method such as Douglas and Peucker algorithm. I believe there is an implementation in FEX, which I haven't tested.

Bruno
From: TideMan on 21 Apr 2010 16:25
On Apr 22, 5:52 am, "Joshua Arnott" <j.arn...(a)lancs.ac.uk.matlab>
wrote:
> Hello everyone,
>
> I have some temperature profile data (elevation [Z] in meters, temperature [T] in celcius), sampled every 0.001m. See figure linked below; left plot is an entire profile, right plot is a smaller section of the same profile.
>
> http://bit.ly/b0MKPv
>
> The fluid this is measured from is quite stratified; there are distinct points at which the gradient (dT/dZ) changes. My goal is to identify these break points.
>
> My difficulties in doing this are a result of noise (at least, what I consider noise) on the data. The smallest stratified sections I wish to identify are around 15 samples (in the example figure, this one is around 60 samples) and so I need to denoise or simplify the signal in some way. The problem I have come across is that (for example, with wavelet denoising on the first derivative) I find often when I smooth the signal enough to remove the noise in sections between the knots I want to identify, exact points at which the changes I want to detect occur are blurred as a result of the smoothing and I am one or two samples out in their location. I really want to be able to identify the exact knot; something I expected easy to do (as it is "by eye") but which I have found is not.
>
> I have experimented with the SLM and BSFK tools available on the FEX, which seem to perform well on the larger stratification structure of the profile, but due to the number of samples (around 14000) and unknown number of knots seem unsuitable for the finer resolution results I desire.
>
> Any direction on this would be appreciated.
>
> Thanks,
> Josh.

Rather than blindly denoising, have you looked at the individual
wavelet details?
For example, detail 1 may be noise, but detail 2 may contain enough of
the signal to identify the turning points.
You may find a pattern there.

BTW, wavelet decomposition of such a signal will have wild end
effects.
What I do as a pre-processing step is to draw a line through the end
points and subtract that from the signal.
yramp=linspace(y(1),y(end),length(y))';
y=y - yramp;
This brings the end points to zero and reduces end effects, sometimes
to zero.
You can add the linear ramp in again (to the approximation) later if
necessary.
 | 
Pages: 1
Prev: data storage
Next: a question about HMM functions in MATLAB