From: Luigi Giaccari on
"Manos " <ezoulias(a)in.gr> wrote in message <h0gn49$729$1(a)fred.mathworks.com>...
> Dear friends,
> I am a Matlab2008a user and I have already run many and various Neural Nets.
> I use the following code to run the Neural net 100 times, its time with different test, val and training set, for this i use my function split_to_nn_sets_and_sim_for_suspicious_spec
>
> for i=1:1:100
>
> [HeadLines,NMSIMF,MSIMF,MSIMF1,MSIMF0,NMSIMF1,NMSIMF0,MSIM1,MSIMY1,MSIM0,MSIMY0,NMSIM1,NMSIMY1,NMSIM0,NMSIMY0,NMTRF,MTRF,MTRF1,MTRF0,NMTRF1,NMTRF0,MTR1,MTRY1,MTR0,MTRY0,NMTR1,NMTRY1,NMTR0,NMTRY0,NMVALF,MVALF,MVALF1,MVALF0,NMVALF1,NMVALF0,MVAL1,MVALY1,MVAL0,MVALY0,NMVAL1, NMVALY1, NMVAL0, NMVALY0,MT1,MTY1,MT0,MTY0,NMT1,NMTY1,NMT0,NMTY0,ResultsNM,ResultsM,ResultsNMold,ResultsMEid1,ResultsMEid0,ResultsNMEid1,ResultsNMEid0,YResultsMEid1,YResultsMEid0,YResultsNMEid1,YResultsNMEid0,YResultsNM,YResultsM,YResultsNMold,TotalSimResult,YResultsSimTotal,TotalTrainResult,YResultsTrainTotal,TotalValResult,YResultsValTotal,TotalTestResult,YResultsTestTotal,YResultsTrainTotalNum,YResultsSimTotalNum,YResultsTestTotalNum,YResultsValTotalNum] = split_to_nn_sets_and_sim_for_suspicious_spec(textdata,data,0.05,0.6,0.2,1);
> %%then I use newff
> %traincgp
> net1107 = newff(transpose(TotalTrainResult),transpose(YResultsTrainTotalNum),[10 10],{},'traincgp');
> VV.P = transpose(TotalValResult);
> VV.T = transpose(YResultsValTotalNum);
> TV.P = transpose(TotalTestResult);
> TV.T = transpose(YResultsTestTotalNum);
> SV.P = transpose(TotalSimResult);
> SV.T = transpose(YResultsSimTotalNum);
> net1107.trainParam.epochs = 1000;
> net1107.trainParam.goal = 1e-3;
> net1107.trainParam.time = 500;
> net1107.trainParam.max_fail = 25;
> net1107.divideFcn = '';
> %I do the trainning
> [net1107 tr1107] =
> train(net1107,transpose(TotalTrainResult),transpose(YResultsTrainTotalNum),[],[],VV,TV);
> % I do the simulation
> [Y1107,Pf,Af,E1107,perf1107] = sim(net1107,SV.P,[],[],SV.T);
>
> %I calculate the confusion matrix for each time
> [Current.c,Current.cm,Current.ind,Current.per] = confusion(SV.T,Y1107);
>
> % I use this part to have a table of sensitivities and specificities of each iteration
> Sensitivityall1107(i,:) = Current.per(1,3)
> Specificityall1107(i,:) = Current.per(2,3)
> Overallall1107(i,:) = 1 - Current.c
>
> %I calculate the mean of sensitivity and specificity of the table each time, the last %turn I have the mean sensitivity and specificity of 100 times calculations
> meansens1107 = mean(Sensitivityall1107)
> meanspec1107 = mean(Specificityall1107)
> meanover1107 = mean(Overallall1107)
>
> end
>
> and then I plot
> plot(meansens1107,1 - meanspec1107,'-.or')
>
> But this is not the ROC curve I need, I thing?
> Have you got any idea how to find the ROC function of all iterations, I know how to do for the one but not as average of more?
>
> Best regards,
> Manolis

http://www.advancedmcode.org/roc-curve.html