perfcurve
in [Matlab]
|
Prev: mat lab
Next: prob with integration
From: nivas k on 8 Dec 2009 16:46 Hello I am working on two class problem. I need to plot receiver operating characteristic(ROC) curve using [x y]=perfcurve(labels, scores, posclass). classification using nearest mean classifier. for each test signal I have two distances (distance from both means), i can get the decision based on distance. But, I dont understand how to write scores, in the perfcurve. Any kind of suggestions will be appreciated. Thanks Nivas.
From: Ilya Narsky on 8 Dec 2009 17:59 "nivas k" <nivas_07(a)yahoo.com> wrote in message news:hfmhfa$qtf$1(a)fred.mathworks.com... > Hello > > I am working on two class problem. I need to plot receiver operating > characteristic(ROC) curve using [x y]=perfcurve(labels, scores, posclass). > > classification using nearest mean classifier. for each test signal I have > two distances (distance from both means), i can get the decision based on > distance. > > But, I dont understand how to write scores, in the perfcurve. > > Any kind of suggestions will be appreciated. > > Thanks > Nivas. Nivas, I am not quite sure what you are asking. But let's see if this helps. For every observation you can compute a vector of scores, one score per class. The observation is then assigned to the class with the highest score - that's the convention assumed by perfcurve. For perfcurve, you need to choose posclass (positive class) and provide a vector of scores for this class, one score per observation. In your problem, you assign observations to classes using distances. I am guessing that you assign an observation to the class that gives you the smallest distance. This is opposite to the convention assumed by perfcurve. What you need to do is convert your distances to scores. What exact transformation you use for this conversion is not important. You should end up with the same ROC curve. You can try, for example, score=-dist. -Ilya
From: nivas k on 9 Dec 2009 11:27 "Ilya Narsky" <inarsky(a)mathworks.com> wrote in message <hfmlns$o87$1(a)fred.mathworks.com>... > > "nivas k" <nivas_07(a)yahoo.com> wrote in message > news:hfmhfa$qtf$1(a)fred.mathworks.com... > > Hello > > > > I am working on two class problem. I need to plot receiver operating > > characteristic(ROC) curve using [x y]=perfcurve(labels, scores, posclass). > > > > classification using nearest mean classifier. for each test signal I have > > two distances (distance from both means), i can get the decision based on > > distance. > > > > But, I dont understand how to write scores, in the perfcurve. > > > > Any kind of suggestions will be appreciated. > > > > Thanks > > Nivas. > > Nivas, > > I am not quite sure what you are asking. But let's see if this helps. For > every observation you can compute a vector of scores, one score per class. > The observation is then assigned to the class with the highest score - > that's the convention assumed by perfcurve. For perfcurve, you need to > choose posclass (positive class) and provide a vector of scores for this > class, one score per observation. > > In your problem, you assign observations to classes using distances. I am > guessing that you assign an observation to the class that gives you the > smallest distance. This is opposite to the convention assumed by perfcurve. > What you need to do is convert your distances to scores. What exact > transformation you use for this conversion is not important. You should end > up with the same ROC curve. You can try, for example, score=-dist. > > -Ilya > Let me explain the problem. Let say I have test signals from 2 classes x1 and x2, each has 100 observations and 10 features. m1 and m2 are the corresponding means. I need to find class of test signal(1st signal from x1). d1(1)=norm(x1(1,:)-m1); d1(2)=norm(x1(1,:)-m2); I assign a class to x1(1,:) based on d1 values. I will have 100 by 2 array for all signals in x1 and similarly 100 by 2 array for signals in x2. If I need to plot ROC for this experiment what scores should I write in perfcurve? Thank you in advance. Nivas.
From: Ilya Narsky on 9 Dec 2009 12:41 "nivas k" <nivas_07(a)yahoo.com> wrote in message news:hfoj58$oqf$1(a)fred.mathworks.com... > "Ilya Narsky" <inarsky(a)mathworks.com> wrote in message > <hfmlns$o87$1(a)fred.mathworks.com>... >> >> "nivas k" <nivas_07(a)yahoo.com> wrote in message >> news:hfmhfa$qtf$1(a)fred.mathworks.com... >> > Hello >> > >> > I am working on two class problem. I need to plot receiver operating >> > characteristic(ROC) curve using [x y]=perfcurve(labels, scores, >> > posclass). >> > >> > classification using nearest mean classifier. for each test signal I >> > have >> > two distances (distance from both means), i can get the decision based >> > on >> > distance. >> > >> > But, I dont understand how to write scores, in the perfcurve. >> > >> > Any kind of suggestions will be appreciated. >> > >> > Thanks >> > Nivas. >> >> Nivas, >> >> I am not quite sure what you are asking. But let's see if this helps. For >> every observation you can compute a vector of scores, one score per >> class. >> The observation is then assigned to the class with the highest score - >> that's the convention assumed by perfcurve. For perfcurve, you need to >> choose posclass (positive class) and provide a vector of scores for this >> class, one score per observation. >> >> In your problem, you assign observations to classes using distances. I am >> guessing that you assign an observation to the class that gives you the >> smallest distance. This is opposite to the convention assumed by >> perfcurve. >> What you need to do is convert your distances to scores. What exact >> transformation you use for this conversion is not important. You should >> end >> up with the same ROC curve. You can try, for example, score=-dist. >> >> -Ilya >> > > Let me explain the problem. > > Let say I have test signals from 2 classes x1 and x2, each has 100 > observations and 10 features. > m1 and m2 are the corresponding means. > I need to find class of test signal(1st signal from x1). > d1(1)=norm(x1(1,:)-m1); > d1(2)=norm(x1(1,:)-m2); > > I assign a class to x1(1,:) based on d1 values. How exactly do you assign a class based on d1 values? Is it "assign class 1 if d1(1)<d1(2)" or something else? > I will have 100 by 2 array for all signals in x1 and similarly 100 by 2 > array for signals in x2. > > If I need to plot ROC for this experiment what scores should I write in > perfcurve? > > Thank you in advance. > > Nivas.
From: nivas k on 9 Dec 2009 13:23
"Ilya Narsky" <inarsky(a)mathworks.com> wrote in message <hfongv$cn9$1(a)fred.mathworks.com>... > > "nivas k" <nivas_07(a)yahoo.com> wrote in message > news:hfoj58$oqf$1(a)fred.mathworks.com... > > "Ilya Narsky" <inarsky(a)mathworks.com> wrote in message > > <hfmlns$o87$1(a)fred.mathworks.com>... > >> > >> "nivas k" <nivas_07(a)yahoo.com> wrote in message > >> news:hfmhfa$qtf$1(a)fred.mathworks.com... > >> > Hello > >> > > >> > I am working on two class problem. I need to plot receiver operating > >> > characteristic(ROC) curve using [x y]=perfcurve(labels, scores, > >> > posclass). > >> > > >> > classification using nearest mean classifier. for each test signal I > >> > have > >> > two distances (distance from both means), i can get the decision based > >> > on > >> > distance. > >> > > >> > But, I dont understand how to write scores, in the perfcurve. > >> > > >> > Any kind of suggestions will be appreciated. > >> > > >> > Thanks > >> > Nivas. > >> > >> Nivas, > >> > >> I am not quite sure what you are asking. But let's see if this helps. For > >> every observation you can compute a vector of scores, one score per > >> class. > >> The observation is then assigned to the class with the highest score - > >> that's the convention assumed by perfcurve. For perfcurve, you need to > >> choose posclass (positive class) and provide a vector of scores for this > >> class, one score per observation. > >> > >> In your problem, you assign observations to classes using distances. I am > >> guessing that you assign an observation to the class that gives you the > >> smallest distance. This is opposite to the convention assumed by > >> perfcurve. > >> What you need to do is convert your distances to scores. What exact > >> transformation you use for this conversion is not important. You should > >> end > >> up with the same ROC curve. You can try, for example, score=-dist. > >> > >> -Ilya > >> > > > > Let me explain the problem. > > > > Let say I have test signals from 2 classes x1 and x2, each has 100 > > observations and 10 features. > > m1 and m2 are the corresponding means. > > I need to find class of test signal(1st signal from x1). > > d1(1)=norm(x1(1,:)-m1); > > d1(2)=norm(x1(1,:)-m2); > > > > I assign a class to x1(1,:) based on d1 values. > > How exactly do you assign a class based on d1 values? Is it "assign class 1 > if d1(1)<d1(2)" or something else? > > > I will have 100 by 2 array for all signals in x1 and similarly 100 by 2 > > array for signals in x2. > > > > If I need to plot ROC for this experiment what scores should I write in > > perfcurve? > > > > Thank you in advance. > > > > Nivas. > yes, if d1(1) < d1(2), i assign class 1. |