The Power of a Sample

The Power of a Sample

We use a sample to estimate a parameter from a population. Sometimes the sample just doesn’t have the ability to discern a change when it actually occurs.

In hypothesis testing, we establish a null and alternative hypothesis. We are setting up an experiment to determine if there is sufficient evidence that a process has changed in some way. The Type II Error, β is a measure of the probability of not concluding the alternative hypothesis is true when in reality it is true.

The power, 1-β, reflects the ability of the sample to correctly lead us to the conclusion that an actual change has occurred when in reality it actually has.

The Difference to Detect Matters

With the Type I Error we are concerned with the distribution related to the null hypothesis, which we are assuming hasn’t changed. If in reality there isn’t any change and the null hypothesis is true, the error is related to the sample drawn resulting in believing a change has occurred. In essence, the samples came mostly from a tail of the distribution, while this is rare, it is possible. It is about the probability of the sample not representing the mean of the underlying population.

For the Type II Error, we now need to consider a second distribution. In this…