Prev: JSH: Maybe they're fakes?. No.....
Next: JSH: Understanding prime gaps issue
From: Luna Moon on 14 Jul 2010 14:08
On Jul 14, 2:07 pm, Luna Moon <lunamoonm...(a)gmail.com> wrote:
> HI all,
>
> suppose I have a treatment, and would like to evaluate the
> effectiveness of this treatment.
>
> how shall I design the stat test to see if the treatment is
> significant or not?
>
> Here is how I think of it:
>
> H0: the treatment is not effective;   Ha: the treatment is effective
>
> I then collect the treatment experiment data, lets say I repeat 100
> times and collected 100 data points.
>
> When the treatment is not in place, there is a score or benchmark
> which is 50.
>
> When the treatment is on and I repeat and collect 100 times - my 100
> data points are real numbers roughly from 0 to 100.
>
> I observe that most of the times the real numbers are above 50, but
> sometimes they are below 50.
>
> The problem is that if they are above 50, they beat 50 by only a small
> margin;
>
> but if they are below 50, they underperform 50 by a very large margin.
>
> So is the treatment effective or not?
>
> That's the question I try to answer.
>
> I don't know the distribution of the 100 data points.
>
> But is there a way to devise the test to assess the statistical
> significance of the treatment?
>
> Maybe t-test? How does t-test help me in this scenario?
>
> Thank you!

Could anybody help me? Thank you!
From: Greg Heath on 21 Jul 2010 22:03
On Jul 14, 2:08 pm, Luna Moon <lunamoonm...(a)gmail.com> wrote:
> On Jul 14, 2:07 pm, Luna Moon <lunamoonm...(a)gmail.com> wrote:
>
>
>
>
>
> > HI all,
>
> > suppose I have a treatment, and would like to evaluate the
> > effectiveness of this treatment.
>
> > how shall I design the stat test to see if the treatment is
> > significant or not?
>
> > Here is how I think of it:
>
> > H0: the treatment is not effective;   Ha: the treatment is effective
>
> > I then collect the treatment experiment data, lets say I repeat 100
> > times and collected 100 data points.
>
> > When the treatment is not in place, there is a score or benchmark
> > which is 50.

100 repetitians all yield 50??

> > When the treatment is on and I repeat and collect 100 times - my 100
> > data points are real numbers roughly from 0 to 100.
>
> > I observe that most of the times the real numbers are above 50, but
> > sometimes they are below 50.
>
> > The problem is that if they are above 50, they beat 50 by only a small
> > margin;
>
> > but if they are below 50, they underperform 50 by a very large margin.
>
> > So is the treatment effective or not?

Why don't you explain, in the same detail, the distribution
corresponding to no treatment:

tabulating min,median,mean,standard deviation and max
or deciles might be a good start.

> > That's the question I try to answer.
>
> > I don't know the distribution of the 100 data points.

You know the 100 values. Try summarizing them.

> > But is there a way to devise the test to assess the statistical
> > significance of the treatment?
>
> > Maybe t-test? How does t-test help me in this scenario?
>
> > Thank you!
>
> Could anybody help me? Thank you

Start with adequate summaries for both distributions.

Hope this helps.

Greg
From: Robert Dodier on 22 Jul 2010 20:34

> On Jul 14, 2:07 pm, Luna Moon <lunamoonm...(a)gmail.com> wrote:

> > suppose I have a treatment, and would like to evaluate the
> > effectiveness of this treatment.
>
> > how shall I design the stat test to see if the treatment is
> > significant or not?
>
> > Here is how I think of it:
>
> > H0: the treatment is not effective;   Ha: the treatment is effective
>
> > I then collect the treatment experiment data, lets say I repeat 100
> > times and collected 100 data points.
>
> > When the treatment is not in place, there is a score or benchmark
> > which is 50.
>
> > When the treatment is on and I repeat and collect 100 times - my 100
> > data points are real numbers roughly from 0 to 100.
>
> > I observe that most of the times the real numbers are above 50, but
> > sometimes they are below 50.
>
> > The problem is that if they are above 50, they beat 50 by only a small
> > margin;
>
> > but if they are below 50, they underperform 50 by a very large margin.
>
> > So is the treatment effective or not?
>
> > That's the question I try to answer.
>
> > I don't know the distribution of the 100 data points.
>
> > But is there a way to devise the test to assess the statistical
> > significance of the treatment?
>
> > Maybe t-test? How does t-test help me in this scenario?

Significance tests such as the t-test are a hack
that was invented to avoid (1) computing probability
of hypotheses (which doesn't exist according to
frequentist theories of probability), and (2) to
avoid bringing cost or utility into play.

My advice to you is to dump the significance test
mumbo jumbo and just work towards the expected utility
of different treatments. Then you can recommend the
treatment that has the greatest expected utility.

The math isn't hard; the difficulty is just the
mental effort to jettison the frequentist stuff which
you have already invested a lot of time and effort
towards learning.

A good introductory book is "Making Hard Decisions"
by Robert Clemen.

Good luck.

Robert Dodier
 | 
Pages: 1
Prev: JSH: Maybe they're fakes?. No.....
Next: JSH: Understanding prime gaps issue