Prev: 14 bit pipelineadc
Next: how to use neural network to tell true or false of a realtimeinput?
From: Simeon on 21 Jul 2010 23:23
Thanks Greg, for your suggestion

N (number of input variables) is small, complex stimuli always make observers unconfident. I will say N <=6

M (number of observations) is not decided yet, but won't be large, because every subject is paid, not free. I will say M <1000.

Each subject feeds his response to the network. We need the network update upon every new input, which means 1 observation/update.

By the way, would you please explain a little bit about "using cluster centers as the calibration set", any example is appreciated.

Greg Heath <heath(a)alumni.brown.edu> wrote in message <07b2e4c9-b87a-40d8-a74f-66ade75fdd01(a)g19g2000yqc.googlegroups.com>...
> On Jul 19, 2:44 pm, "Simeon " <simeonyu1...(a)yahoo.com> wrote:
> > Problem description:
> > A neural network is needed by the psychology lab to determine whether the subject gives right response to the features of the given object.
> >
> > all data in this problem is taken from N dimension feature space, e.g, [shape,size,color] for given objects. the intensity of each dimension is digitalized to [1,10].
> >
> > Training data is given (MxN matrix). A neural network will be trained with the given data. After training, the network should give positive response when the most frequent feature vector is given from input.
> >
> > The critical part is, after trained with given data, "the most frequent feature vector" might keep changing because the subject might gives different response other than the "the most frequent feature vector" we got from the training data.
> > How to make the network keeps updating?
> >
> > Any suggestion is appreciated.
>
> Suggestions are size and time dependent.
>
> What are M(number of input variables) and
> N(Number of observations/cases)?
>
> How often will you be updating the model?
> How many observations per update?
>
> If N is huge and updates are large and frequent,
> it may be prudent to maintain a smaller calibration
> set that summarizes the salent characteristics of
> past data. Typically, this can be obtained
> by intially clustering the training data and using
> cluster centers as the calibration set. When
> new data is encountered, both the calibration set
> and the net are updated.
>
> Hope this helps.
>
> Greg
From: Walter Roberson on 21 Jul 2010 23:49
Simeon wrote:

> By the way, would you please explain a little bit about "using cluster
> centers as the calibration set", any example is appreciated.

Well, for example, suppose you maintain k-means clustering information,
and when you get a new sample you add it randomly to one of the clusters
and then iterate a few times until the clusters settle down. As most
values would already be right, it shouldn't be very much movement or
much in the way of iterations.

Just giving an example, not suggesting that k-means is suitable for your
situation.


Your specification is incomplete. You want the most frequent feature
vector to give a positive response, and you want to effectively retrain
for every response. The implication of that combination is that as a
particular feature vector starts becoming more common and starts taking
the lead, then there must be a point at which two or more feature
vectors will be equally the most frequent ones. Your description does
not say what to do in that case: hold the old one until a new one
surpasses it, or respond positively to _all_ feature vectors that are
currently equally favored ?


It isn't clear to me that a neural network is a good approach to the
task when the continuous update need is taken into account.
From: Walter Roberson on 22 Jul 2010 02:49
Walter Roberson wrote:

> Your specification is incomplete. You want the most frequent feature
> vector to give a positive response, and you want to effectively retrain
> for every response. The implication of that combination is that as a
> particular feature vector starts becoming more common and starts taking
> the lead, then there must be a point at which two or more feature
> vectors will be equally the most frequent ones. Your description does
> not say what to do in that case: hold the old one until a new one
> surpasses it, or respond positively to _all_ feature vectors that are
> currently equally favored ?

In thinking about this point more, it seems to me that neural networks,
especially those with floating point weights, would tend to learn a
weighted average of the input signals, and as such their idea of "the
most frequent feature vector" would seldom or rarely exactly match any
one (integral) input feature vector.

I guess it might be possible to in effect implement a "round" step that
would snap to one particular integral vector, but I would not generally
expect that vector to necessarily be a match for any actual input.

If you imagine constructing an N dimensional space whose values are the
number of occurrences of that N dimensional point, then the training
process would generally be akin to going to the center of the space and
finding the steepest gradient relative to that point: as you probably
know, a single steepest gradient measurement is seldom sufficient to
find the global minima of a surface.

If you were to add 1 to each of your feature values (which if I recall
currently start from 0) then you could use the resulting vector as an
index into an N dimensional array of counts (might as well use uint16
for storage efficiency.) Detecting whether a particular input vector
matched the most frequent feature vector would then be equivalent to
detecting whether the value stored at that the indexed array location
was the current maxima of the array, and "retraining" after each sample
would logically correspond to adding 1 to the indexed location.

There are a variety of strategies that could be used to improve the
efficiency of updating the maximum value of the N dimensional array;for
example marginal values along the later dimensions can cut down the
search and increase its locality greatly. Or considering that you would
have a large space and at most 1000 points of it filled, simply using
sparse double arrays and using max() over the non-zero entries would
probably be fine for efficiency.

As I indicated earlier, your needs do not seem to me to be well suited
to a NN implementation.
 | 
Pages: 1
Prev: 14 bit pipelineadc
Next: how to use neural network to tell true or false of a realtimeinput?