Comparing Results

Making predictions is only one use of statistics. Suppose you have recently developed a new headache/pain remedy that you call Ache‐Away. Should you produce Ache‐Away in quantity and make it available to the public? That would depend upon, among other factors, whether Ache‐Away is more effective than the old remedy. How can you determine that?

One way might be to administer both remedies to two separate groups of people, collect data on the results, and then statistically analyze that data to determine if Ache‐Away is more effective than the old remedy. And what if the results of this test showed Ache‐Away to be more effective? How certain can you be that this particular test administration is indicative of all tests of these two remedies? Perhaps the group taking the old remedy (the control group) and the group taking Ache‐Away (the treatment group) were so dissimilar that the results were due not to the pain remedies but to the differences between the groups.

It is possible that the results of this test are far off the results that you would get if you tried the test several more times. You certainly do not want to foist a questionable drug upon an unsuspecting public based upon atypical test results. How certain can you be that you can put your faith in the results of your tests? You can see that the problems of comparing headache remedy results can produce headaches of their own.