Prev: latex problem in textbox
Next: DICOM SPM5 preprocessing error messages
From: Kishore on 2 Nov 2009 20:45
Hello,

I am trying to classify a 4 class problem (each class has 20 features ) using neural network.
So, in order to reduce the complexity, i used newff function to get the weights.

The problem is i am not very familiar with newff function usage ( the samples are not classified properly- same sample set is being classified welll using k nearest neighbour and bayesian techniques).

It would be great if i can get feed back on the usage of this newff sequence.

%%%%%%%%

net = newff(training_data',group',no_hiddenLayer);
% Create a new feed forward network. 20 neurons in the hidden layer.
%training data is a matrix of training samles.
%group is a matrix, where each row is for example [1 0 0 0] or [0 1 0 0] or [0 0 1 %0] or [0 0 0 1] based on the class to which the training sample belongs.
%so basically i have 4 outputs.




% training parameters -i took this from an example
net.trainParam.goal = 0.1;
net.trainParam.show = 20;
net.trainParam.epochs = 40;
net.trainParam.mc = 0.95;

[net,tr] = train(net,training_data',group');

W = net.IW{1};
V = net.LW{2};

W =W';
V = V';

the best weights are obtained.

Now i do the testing...and classify based on the 4 outputs( which ever is maximum).

Is this approach correct =or am i missing something?


Thanks
From: Kishore on 2 Nov 2009 21:24
Ok, one more thing i need to tell,

The network assumes 2 layers, but i have implemented equations ( back propagation for 1 layer - so this is the problem i guess).

But how do i do the testing using standard function.

( i have testing data as well, which function should i pass it to?)

Thanks,
From: ade77 on 2 Nov 2009 22:08
I did exactly the same project last year. Your code seems correct. I will assume the following based on the description of the problem:

1. You have 20 rows of input features.
2. You need to classify the problem into 4 possible outcomes.
unless you need to pass the weights into another program, the codes you have written:
W = net.IW{1};
> V = net.LW{2};
>
> W =W';
> V = V'
is completely unnecessay, since MATLAB will stote the weights in the network(net).

Back to the problem, once you have the network trained, all you have to do is test the network.

test = sim(net,new_input).
The trick here is that your output will produce 4 elements, the one that is closest to 1 is your classification.

For example if your classification is [red green blue orange], and you get
[1.2 2.4 5.6 3] , then red is your classification. because 1.2 is closest to 1

In most cases, you will get negative, you need to decide based on your input features, if you will use absoulte values. For example,
test = sim(net, new_inputs) gives [0.7 1.5 -0.8 3.2], if you take absolute value, then -0.8 is closest to 1, hence blue is your classification.

Finally, I have assumed that you correctly put the inputs and outputs in your training, and you normalize the inputs.

if you are still confused, feel free to let me know

One last thing, you can just let MATLAB use its default parameters, and all you need to change is the number of neurons, and try 2 or more hidden layers, since your input features are many.
From: Kishore on 3 Nov 2009 17:39
Hello,

Thanks for the input. I will try this.


Thanks!
From: Greg Heath on 8 Nov 2009 05:42
On Nov 2, 8:45 pm, "Kishore " <kishore3...(a)yahoo.co.in> wrote:
> Hello,
>
> I am trying to classify a 4 class problem (each class has
> 20 features ) using neural network.
> So, in order to reduce the complexity, i used newff function to
> get the weights.
>
> The problem is i am not very familiar with newff function usage
> ( the samples are not classified properly- same sample set is being
> classified welll using k nearest neighbour and bayesian techniques).
>
> It would be great if i can get feed back on the usage of this
> newff sequence.
>
> %%%%%%%%
>
> net = newff(training_data',group',no_hiddenLayer);

help newff
doc newff

You have accepted the PURELIN output default. For classification,
LOGSIG is superior. The output then represents the posterior
probability (conditional on the input) for class "1".

> % Create a new feed forward network. 20 neurons in the hidden layer.

H = 20 may be far too many. It's best to use the smallest
satisfactory
value which is usually found by trial and error.

For an I-H-O MLP, Ntrn training vectors, size(p) = [I Ntrn],
size(t) = [ O Ntrn] and training without regularization, a good rule
of thumb is Neq >> Nw where

Neq = Ntrn*O = Ntrn*4 = No. of training equations.
Nw = (I+1)*H+(H+1)*O = No. of unknown weights


Search on

greg-heath Neq Nw
greg-heath pretraining advice newbies

for details

> %training data is a matrix of training samles.
> %group is a matrix, where each row is for example
> [1 0 0 0] or [0 1 0 0] or [0 0 1 %0] or [0 0 0 1]
> based on the class to which the training sample belongs.

Each column should be an output target.

size(t) = [ 4 Ntrn]

I don't recommend using the transpose operator in
the call of newff.

> %so basically i have 4 outputs.
>
> % training parameters -i took this from an example
> net.trainParam.goal = 0.1;

Looks OK:

You would get lower than this if the correct class
output is 0.7 and the other outputs are 0.3

> net.trainParam.show = 20;
> net.trainParam.epochs = 40;
> net.trainParam.mc = 0.95;

delete the last 2 and settle for the defaults in TRAINLM.

help trainlm
doc trainlm

> [net,tr] = train(net,training_data',group');


> W = net.IW{1};
> V = net.LW{2};

Incorrect see the documentation

> W =W';
> V = V';
>
> the best weights are obtained.

It is not necessary to explicitly extract the weights
(and the very important thresholds in net.b).
>
> Now i do the testing...and classify based on the 4 outputs
> ( which ever is maximum).
>
> Is this approach correct or am i missing something?

For any input x with correct outputs y0

size(x) = [ I N]
size(y0) = [ O N]
y = sim(net,x);
error = y0-y;
MSE = mse(error)


Hope this helps.

Greg

 |  Next  |  Last
Pages: 1 2 3
Prev: latex problem in textbox
Next: DICOM SPM5 preprocessing error messages