Prev: why RMSE obtained from stepwisefit and regstats differs?
Next: The number provided for each value
From: TideMan on 8 May 2010 18:40
On May 9, 8:38 am, emceeaye <dumathechee...(a)yahoo.com> wrote:
> On May 8, 4:16 am, Rune Allnor <all...(a)tele.ntnu.no> wrote:
>
>
>
> > On 8 Mai, 10:52, emceeaye <dumathechee...(a)yahoo.com> wrote:
>
> > > Hi Matlab experts,
> > > My background is in functional magnetic resonance imaging time series
> > > data analysis, and I recently realized that there's no reason l can't
> > > also apply the same statistical procedures I use for my research to
> > > stock forecasting.
>
> > Yes, there is.
>
> > MRI is based on very specific physical relationships that can
> > be expressed compactly and conveniently by means of maths.
>
> > Stocks indexes / prices are random series that rely on unpredictable,
> > unforeseen factors, including human psychological factors like fear,
> > nervousness and mass hysteria.
>
> > Within the past couple of months Europe have experienced a couple
> > of factors that were unforeseeable, like the Iceland volcano
> > eruption and the Greek economical crisis. There is no way any
> > stock market model can include or account for such factors.
>
> > Rune
>
> Hi Rune,
> Part of what you say is true and part is not.  1) While stock prices
> are influenced by "unpredictable" and "unforseen" fluctuations in
> values such as those seen in the recent volatility in stock market
> prices and the other examples you gave, the prices certainly don't
> "RELY" on them as you stated--after all, they are the exceptions
> rather than the rules.
>
> 2) Certainly evidence suggests that fMRI data is based on biological
> (or "physical" relationships) as you suggest, but there are a lot of
> unknowns about the physiological significance of the BOLD signal
> (e.g., is it blood flow, oxygen consumption, glucose consumption, a
> combination of all, or a consequence of magnetic field
> inhomogeneities, and the list goes on).  Furthermore, the signal
> changes being analyzed often do not reflect the underlying
> psychological/cognitive process that the researcher is intending to
> measure and the conclusions drawn and interpretations made from the
> results of analyzing this data are often leaps in logic and not
> necessarily accurate (happens all the time).  Also, Just as human
> "nervousness", and "self-consciousness" may influence stock prices,
> these same types of unpredictable, uncontrollable and unverifiable
> emotions/states and traits of the subjects may contaminate the sampled
> BOLD signal as well, which ultimately interferes with the
> intepretability of the data (happens all the time). so results of fMRI
> data may be expressed "conveniently" and "compactly" by math, but what
> is being deduced or extrapolated from the results is often far from
> "convenient" or "compact".
>
> Routine techniques are often applied to fMRI BOLD data to reduce
> "noise" and "outliers" because they don't fit the expected behavior of
> the signal--why is this done? to eliminate the influence of anomalies
> and unexpected behavior in the signal being sampled (basically because
> the researcher cannot explain the reason for them and so discards the
> raw data for them). These same procedures of dealing with outliers and
> noise may also be applied to stock time series data to eliminate their
> influence on what the researcher is trying to determine by analyzing
> the data.  Does this mean that stock prices time series don't exhibit
> more unpredictable volatility than fMRI BOLD data?  Maybe or maybe
> not.  That is one of the questions I would like to answer.
>
> One way to compensate for these unexpected fluctuations in the stock
> market is to increase the sampling period (e.g., from 1 year to 2, 3,
> 4 years worth of values)--by increasing the sampling period you reduce
> the impact of sudden unexpected fluctuations on the overall
> statistical results. Either way, I don't have to know much about
> econometrics to say that stock prices are certainly not "random" as
> you stated.  However, there are certainly many factors influencing the
> values and it's trying to account for as many of these factors as
> possible and determine their relative contribution to the results that
> may be a bit of a challenge.
>
> emceeaye

Perhaps "random" is the wrong word. Chaotic may be better.
There is no hope of ever forecasting a random series. There is no
correlation from one point in time to the next. But a chaotic series
arises from nonlinear interaction of the components and if we ever
become clever enough, we will be able to solve it.
So there is hope, but it's a pretty faint hope right now.

From: Liam Zaidel on 9 May 2010 20:35
On May 8, 4:43 am, "Wayne King" <wmkin...(a)gmail.com> wrote:
> emceeaye <dumathechee...(a)yahoo.com> wrote in message <3dfbec32-85bf-4bb5-a01a-b3479188e...(a)g1g2000pro.googlegroups.com>...
> > Hi Matlab experts,
> > My background is in functional magnetic resonance imaging time series
> > data analysis, and I recently realized that there's no reason l can't
> > also apply the same statistical procedures I use for my research to
> > stock forecasting.  I want to analyze historical time series data for
> > stocks that I'd like to import into Matlab? I have yet to use Matlab,
> > and so this project will be a good chance for me to get my feet wet.
> > For the time-series data, I plan to look at different sampling rates
> > (e.g., closing prices for the day; price 4 times a day, etc.) for
> > different ranges of time (e.g., past 1/2 year, past 1 year, past 2
> > years, past 5 years, etc.).
>
> > It would be great to get insights from people with experience with
> > time series data about the pre-processing steps necessary to analyze
> > this data.  Traditionally for fMRI data, there are a series of pre-
> > processing steps researchers go through before analyzing different
> > time series of signal values (e.g., remove linear trend, smooth the
> > data, etc.) in order to free the data of contaminating artifact and to
> > make it safe to analyze (which will in turn make the statistics that
> > are based on them interpretable).  Can I expect that the assumptions
> > that must be made about stock time series data before safe analysis is
> > possible are the same as for fMRI signal fluctuations time series data
> > (e.g. normal distribution, homogeneity of variance, etc.)?  If so, do
> > they require specific pre-processing steps to prepare it for analysis
> > (detrending, removing outliers, etc.), and are these procedures
> > available in Matlab?  I also wonder if there is an uncomplicated (or
> > as uncomplicated as possible) book that can do a good job of answering
> > these questions that anyone can recommend?
>
> > I would really appreciate any help with this as it will go a long way
> > for me.
>
> > Thank you in advance.
>
> Hi, I think you will find that all time series analysis problems share some similarities, but certainly each domain has some unique elements and problems as well. Specifically, the "homogeneity of variance" assumption that you referred to is violated consistently in econometric time series. As Rune points out, there are often shocks in financial time series.
>
> MATLAB provides a lot of tools in the Econometrics Toolbox for modeling conditional heteroskedasticity like that encountered regularly in financial time series. Start with Hamilton's textbook on Time Series Analysis. You will find a lot of things that are familiar from working on fMRI time series and a lot that is probably not familiar.
>
> Wayne


Hi everyone,

Thank you for your thoughts and book suggestions so far!

Emceeaye
From: Dan on 10 May 2010 14:49

When all you have is a hammer, everything looks like a nail.
I (humbly) suggest that you forget about potential fMRI/stock similarities and simply analyze the data.

emceeaye <dumathecheetah(a)yahoo.com> wrote in message <ab240ee4-f11f-486a-884e-58debbdbb6bc(a)23g2000pre.googlegroups.com>...
> On May 8, 4:16 am, Rune Allnor <all...(a)tele.ntnu.no> wrote:
> > On 8 Mai, 10:52, emceeaye <dumathechee...(a)yahoo.com> wrote:
> >
> > > Hi Matlab experts,
> > > My background is in functional magnetic resonance imaging time series
> > > data analysis, and I recently realized that there's no reason l can't
> > > also apply the same statistical procedures I use for my research to
> > > stock forecasting.
> >
> > Yes, there is.
> >
> > MRI is based on very specific physical relationships that can
> > be expressed compactly and conveniently by means of maths.
> >
> > Stocks indexes / prices are random series that rely on unpredictable,
> > unforeseen factors, including human psychological factors like fear,
> > nervousness and mass hysteria.
> >
> > Within the past couple of months Europe have experienced a couple
> > of factors that were unforeseeable, like the Iceland volcano
> > eruption and the Greek economical crisis. There is no way any
> > stock market model can include or account for such factors.
> >
> > Rune
>
> Hi Rune,
> Part of what you say is true and part is not. 1) While stock prices
> are influenced by "unpredictable" and "unforseen" fluctuations in
> values such as those seen in the recent volatility in stock market
> prices and the other examples you gave, the prices certainly don't
> "RELY" on them as you stated--after all, they are the exceptions
> rather than the rules.
>
> 2) Certainly evidence suggests that fMRI data is based on biological
> (or "physical" relationships) as you suggest, but there are a lot of
> unknowns about the physiological significance of the BOLD signal
> (e.g., is it blood flow, oxygen consumption, glucose consumption, a
> combination of all, or a consequence of magnetic field
> inhomogeneities, and the list goes on). Furthermore, the signal
> changes being analyzed often do not reflect the underlying
> psychological/cognitive process that the researcher is intending to
> measure and the conclusions drawn and interpretations made from the
> results of analyzing this data are often leaps in logic and not
> necessarily accurate (happens all the time). Also, Just as human
> "nervousness", and "self-consciousness" may influence stock prices,
> these same types of unpredictable, uncontrollable and unverifiable
> emotions/states and traits of the subjects may contaminate the sampled
> BOLD signal as well, which ultimately interferes with the
> intepretability of the data (happens all the time). so results of fMRI
> data may be expressed "conveniently" and "compactly" by math, but what
> is being deduced or extrapolated from the results is often far from
> "convenient" or "compact".
>
> Routine techniques are often applied to fMRI BOLD data to reduce
> "noise" and "outliers" because they don't fit the expected behavior of
> the signal--why is this done? to eliminate the influence of anomalies
> and unexpected behavior in the signal being sampled (basically because
> the researcher cannot explain the reason for them and so discards the
> raw data for them). These same procedures of dealing with outliers and
> noise may also be applied to stock time series data to eliminate their
> influence on what the researcher is trying to determine by analyzing
> the data. Does this mean that stock prices time series don't exhibit
> more unpredictable volatility than fMRI BOLD data? Maybe or maybe
> not. That is one of the questions I would like to answer.
>
> One way to compensate for these unexpected fluctuations in the stock
> market is to increase the sampling period (e.g., from 1 year to 2, 3,
> 4 years worth of values)--by increasing the sampling period you reduce
> the impact of sudden unexpected fluctuations on the overall
> statistical results. Either way, I don't have to know much about
> econometrics to say that stock prices are certainly not "random" as
> you stated. However, there are certainly many factors influencing the
> values and it's trying to account for as many of these factors as
> possible and determine their relative contribution to the results that
> may be a bit of a challenge.
>
> emceeaye
From: emceeaye on 11 May 2010 00:28
On May 10, 11:49 am, "Dan" <sambe...(a)gmail.com> wrote:
> When all you have is a hammer, everything looks like a nail.
> I (humbly) suggest that you forget about potential fMRI/stock similarities and simply analyze the data.
>
> emceeaye <dumathechee...(a)yahoo.com> wrote in message <ab240ee4-f11f-486a-884e-58debbdbb...(a)23g2000pre.googlegroups.com>...
> > On May 8, 4:16 am, Rune Allnor <all...(a)tele.ntnu.no> wrote:
> > > On 8 Mai, 10:52, emceeaye <dumathechee...(a)yahoo.com> wrote:
>
> > > > Hi Matlab experts,
> > > > My background is in functional magnetic resonance imaging time series
> > > > data analysis, and I recently realized that there's no reason l can't
> > > > also apply the same statistical procedures I use for my research to
> > > > stock forecasting.
>
> > > Yes, there is.
>
> > > MRI is based on very specific physical relationships that can
> > > be expressed compactly and conveniently by means of maths.
>
> > > Stocks indexes / prices are random series that rely on unpredictable,
> > > unforeseen factors, including human psychological factors like fear,
> > > nervousness and mass hysteria.
>
> > > Within the past couple of months Europe have experienced a couple
> > > of factors that were unforeseeable, like the Iceland volcano
> > > eruption and the Greek economical crisis. There is no way any
> > > stock market model can include or account for such factors.
>
> > > Rune
>
> > Hi Rune,
> > Part of what you say is true and part is not.  1) While stock prices
> > are influenced by "unpredictable" and "unforseen" fluctuations in
> > values such as those seen in the recent volatility in stock market
> > prices and the other examples you gave, the prices certainly don't
> > "RELY" on them as you stated--after all, they are the exceptions
> > rather than the rules.
>
> > 2) Certainly evidence suggests that fMRI data is based on biological
> > (or "physical" relationships) as you suggest, but there are a lot of
> > unknowns about the physiological significance of the BOLD signal
> > (e.g., is it blood flow, oxygen consumption, glucose consumption, a
> > combination of all, or a consequence of magnetic field
> > inhomogeneities, and the list goes on).  Furthermore, the signal
> > changes being analyzed often do not reflect the underlying
> > psychological/cognitive process that the researcher is intending to
> > measure and the conclusions drawn and interpretations made from the
> > results of analyzing this data are often leaps in logic and not
> > necessarily accurate (happens all the time).  Also, Just as human
> > "nervousness", and "self-consciousness" may influence stock prices,
> > these same types of unpredictable, uncontrollable and unverifiable
> > emotions/states and traits of the subjects may contaminate the sampled
> > BOLD signal as well, which ultimately interferes with the
> > intepretability of the data (happens all the time). so results of fMRI
> > data may be expressed "conveniently" and "compactly" by math, but what
> > is being deduced or extrapolated from the results is often far from
> > "convenient" or "compact".
>
> > Routine techniques are often applied to fMRI BOLD data to reduce
> > "noise" and "outliers" because they don't fit the expected behavior of
> > the signal--why is this done? to eliminate the influence of anomalies
> > and unexpected behavior in the signal being sampled (basically because
> > the researcher cannot explain the reason for them and so discards the
> > raw data for them). These same procedures of dealing with outliers and
> > noise may also be applied to stock time series data to eliminate their
> > influence on what the researcher is trying to determine by analyzing
> > the data.  Does this mean that stock prices time series don't exhibit
> > more unpredictable volatility than fMRI BOLD data?  Maybe or maybe
> > not.  That is one of the questions I would like to answer.
>
> > One way to compensate for these unexpected fluctuations in the stock
> > market is to increase the sampling period (e.g., from 1 year to 2, 3,
> > 4 years worth of values)--by increasing the sampling period you reduce
> > the impact of sudden unexpected fluctuations on the overall
> > statistical results. Either way, I don't have to know much about
> > econometrics to say that stock prices are certainly not "random" as
> > you stated.  However, there are certainly many factors influencing the
> > values and it's trying to account for as many of these factors as
> > possible and determine their relative contribution to the results that
> > may be a bit of a challenge.
>
> > emceeaye

Hi Dan,
yes, I like your analogy. I've grown accustomed to dealing with the
dilemma of analyzing data with tools that are too crude to do it
justice. But the best I can do is hit the hammer as squarely on the
head of the nail as possible...and, coming from an fMRI time series
analysis perspective, I'm trying to facilitate this by learning the
similarities and differences in important considerations, particularly
with respect to pre-processing steps before performing the analysis.
I can't analyze the data until I know what the differences are to
implement necessary changes. For instance, I hope to find out (from
reading and from people here) what some good ideal reference waveforms
are to model stock time series, at least with respect to the degree of
the polynomial that should be used--So far, people who have responded
to this thread have characterized stock time series data as "chaotic"
and "random", so at least I know that it won't be a linear model. In
fMRI, if one doesn't model the signal in order to come up with an
ideal waveform, they often choose prescribed waveforms factoring in
things like the delay part, rise part, fall part (including a post-
undershoot), and a restoration part (from the undershoot) of the
signal in coming up with the structure of a model time course to fit
to the data. While I know there is variability in approach, are there
preferred ways of modeling the time series in stock data?

Thanks in advance.
From: Joe on 11 May 2010 06:13
On May 10, 11:49 am, "Dan" <sambe...(a)gmail.com> wrote:
> When all you have is a hammer, everything looks like a nail.
> I (humbly) suggest that you forget about potential fMRI/stock similarities and simply analyze the data.
>
> emceeaye <dumathechee...(a)yahoo.com> wrote in message <ab240ee4-f11f-486a-884e-58debbdbb...(a)23g2000pre.googlegroups.com>...
> > On May 8, 4:16 am, Rune Allnor <all...(a)tele.ntnu.no> wrote:
> > > On 8 Mai, 10:52, emceeaye <dumathechee...(a)yahoo.com> wrote:
>
> > > > Hi Matlab experts,
> > > > My background is in functional magnetic resonance imaging time series
> > > > data analysis, and I recently realized that there's no reason l can't
> > > > also apply the same statistical procedures I use for my research to
> > > > stock forecasting.
>
> > > Yes, there is.
>
> > > MRI is based on very specific physical relationships that can
> > > be expressed compactly and conveniently by means of maths.
>
> > > Stocks indexes / prices are random series that rely on unpredictable,
> > > unforeseen factors, including human psychological factors like fear,
> > > nervousness and mass hysteria.
>
> > > Within the past couple of months Europe have experienced a couple
> > > of factors that were unforeseeable, like the Iceland volcano
> > > eruption and the Greek economical crisis. There is no way any
> > > stock market model can include or account for such factors.
>
> > > Rune
>
> > Hi Rune,
> > Part of what you say is true and part is not.  1) While stock prices
> > are influenced by "unpredictable" and "unforseen" fluctuations in
> > values such as those seen in the recent volatility in stock market
> > prices and the other examples you gave, the prices certainly don't
> > "RELY" on them as you stated--after all, they are the exceptions
> > rather than the rules.
>
> > 2) Certainly evidence suggests that fMRI data is based on biological
> > (or "physical" relationships) as you suggest, but there are a lot of
> > unknowns about the physiological significance of the BOLD signal
> > (e.g., is it blood flow, oxygen consumption, glucose consumption, a
> > combination of all, or a consequence of magnetic field
> > inhomogeneities, and the list goes on).  Furthermore, the signal
> > changes being analyzed often do not reflect the underlying
> > psychological/cognitive process that the researcher is intending to
> > measure and the conclusions drawn and interpretations made from the
> > results of analyzing this data are often leaps in logic and not
> > necessarily accurate (happens all the time).  Also, Just as human
> > "nervousness", and "self-consciousness" may influence stock prices,
> > these same types of unpredictable, uncontrollable and unverifiable
> > emotions/states and traits of the subjects may contaminate the sampled
> > BOLD signal as well, which ultimately interferes with the
> > intepretability of the data (happens all the time). so results of fMRI
> > data may be expressed "conveniently" and "compactly" by math, but what
> > is being deduced or extrapolated from the results is often far from
> > "convenient" or "compact".
>
> > Routine techniques are often applied to fMRI BOLD data to reduce
> > "noise" and "outliers" because they don't fit the expected behavior of
> > the signal--why is this done? to eliminate the influence of anomalies
> > and unexpected behavior in the signal being sampled (basically because
> > the researcher cannot explain the reason for them and so discards the
> > raw data for them). These same procedures of dealing with outliers and
> > noise may also be applied to stock time series data to eliminate their
> > influence on what the researcher is trying to determine by analyzing
> > the data.  Does this mean that stock prices time series don't exhibit
> > more unpredictable volatility than fMRI BOLD data?  Maybe or maybe
> > not.  That is one of the questions I would like to answer.
>
> > One way to compensate for these unexpected fluctuations in the stock
> > market is to increase the sampling period (e.g., from 1 year to 2, 3,
> > 4 years worth of values)--by increasing the sampling period you reduce
> > the impact of sudden unexpected fluctuations on the overall
> > statistical results. Either way, I don't have to know much about
> > econometrics to say that stock prices are certainly not "random" as
> > you stated.  However, there are certainly many factors influencing the
> > values and it's trying to account for as many of these factors as
> > possible and determine their relative contribution to the results that
> > may be a bit of a challenge.
>
> > emceeaye

Not sure what your point is--First, I guess you're saying his
knowledge of stock analysis is limited, and then you're suggesting to
him in the same breath to analyze the data anyway?? Seems to me that
if he has experience with fMRI time series, then he is prepared to
deal with many of the challenges posed by that of stocks analysis. It
must take one with only a hammer to know that everything looks like a
nail.
First  |  Prev  |  Next  |  Last
Pages: 1 2 3
Prev: why RMSE obtained from stepwisefit and regstats differs?
Next: The number provided for each value