perfcurve
in [Matlab]
|
Prev: mat lab
Next: prob with integration
From: Ilya Narsky on 9 Dec 2009 16:42 "nivas k" <nivas_07(a)yahoo.com> wrote in message news:hfopu8$l3m$1(a)fred.mathworks.com... > "Ilya Narsky" <inarsky(a)mathworks.com> wrote in message > <hfongv$cn9$1(a)fred.mathworks.com>... >> >> "nivas k" <nivas_07(a)yahoo.com> wrote in message >> news:hfoj58$oqf$1(a)fred.mathworks.com... >> > "Ilya Narsky" <inarsky(a)mathworks.com> wrote in message >> > <hfmlns$o87$1(a)fred.mathworks.com>... >> >> >> >> "nivas k" <nivas_07(a)yahoo.com> wrote in message >> >> news:hfmhfa$qtf$1(a)fred.mathworks.com... >> >> > Hello >> >> > >> >> > I am working on two class problem. I need to plot receiver operating >> >> > characteristic(ROC) curve using [x y]=perfcurve(labels, scores, >> >> > posclass). >> >> > >> >> > classification using nearest mean classifier. for each test signal I >> >> > have >> >> > two distances (distance from both means), i can get the decision >> >> > based >> >> > on >> >> > distance. >> >> > >> >> > But, I dont understand how to write scores, in the perfcurve. >> >> > >> >> > Any kind of suggestions will be appreciated. >> >> > >> >> > Thanks >> >> > Nivas. >> >> >> >> Nivas, >> >> >> >> I am not quite sure what you are asking. But let's see if this helps. >> >> For >> >> every observation you can compute a vector of scores, one score per >> >> class. >> >> The observation is then assigned to the class with the highest score - >> >> that's the convention assumed by perfcurve. For perfcurve, you need to >> >> choose posclass (positive class) and provide a vector of scores for >> >> this >> >> class, one score per observation. >> >> >> >> In your problem, you assign observations to classes using distances. I >> >> am >> >> guessing that you assign an observation to the class that gives you >> >> the >> >> smallest distance. This is opposite to the convention assumed by >> >> perfcurve. >> >> What you need to do is convert your distances to scores. What exact >> >> transformation you use for this conversion is not important. You >> >> should >> >> end >> >> up with the same ROC curve. You can try, for example, score=-dist. >> >> >> >> -Ilya >> >> >> > >> > Let me explain the problem. >> > >> > Let say I have test signals from 2 classes x1 and x2, each has 100 >> > observations and 10 features. >> > m1 and m2 are the corresponding means. >> > I need to find class of test signal(1st signal from x1). >> > d1(1)=norm(x1(1,:)-m1); >> > d1(2)=norm(x1(1,:)-m2); >> > >> > I assign a class to x1(1,:) based on d1 values. >> >> How exactly do you assign a class based on d1 values? Is it "assign class >> 1 >> if d1(1)<d1(2)" or something else? >> >> > I will have 100 by 2 array for all signals in x1 and similarly 100 by 2 >> > array for signals in x2. >> > >> > If I need to plot ROC for this experiment what scores should I write in >> > perfcurve? >> > >> > Thank you in advance. >> > >> > Nivas. >> > > yes, if d1(1) < d1(2), i assign class 1. Classification score is a number that represents confidence of classification. The higher the classification score, the more confidence you have that this observation comes from this class. You need to combine these two numbers, d1(1) and d1(2), into one number (score) that represents confidence of classification into class 1. You can do that in a number of ways. Thinking of d1 and d2 as distances, I would naively go with score = d1(2)-d1(1); But keep in mind that a) I do not know enough about the nature of your problem, and b) there may be several reasonable solutions for the same problem. You could also try, for example score = d1(2)/d1(1); Or perhaps score = -d1(1); Hope this helps. |