Some Preliminary Thoughts on Statistics
by H.P.L. Molloy and T Newfields
|". . . the post-positivist position on epistemology, the study of knowledge, contends convincingly that we can never prove positive hypotheses: we can only falsify hypotheses."|
[ p. 2 ]Let us consider an example. Suppose you had the idea: "All of the cars in Tokyo are less than ten years old." How would you test this? We can walk though the streets and look at all the cars. Suppose we found 700 cars, all of them under ten years old. Have we proved our hypothesis? No: the next car might be over than ten years old. In fact, no matter how many cars we check we have to keep on looking, because it is possible that the next car could be over a decade old.
|"Type 1 errors . . . occur when the null hypothesis is rejected when it shouldn't be."|
[ p. 3 ]Alpha, p, or significance, values should be chosen in the planning stages of a study - before collecting any data. You should have good reasons for not using 0.05 or 0.01, but also should also have reasons for choosing either 0.05 or 0.01. Is your study one that is only descriptive or is it exploratory, done in preparation for more rigorous research? You're probably safe with 0.05, as no big decisions depend on the chances of making an error. Is your study one that will affect people's lives (such as one involving entrance examinations)? You might want to use the more conservative 0.01 value. In medical research, values of 0.001 or 0.0001 are often chosen since people may die because of a mistaken decision.
|"[A] Type 2 error . . . [is] the chance of retaining the null hypothesis when it is not true."|
[ p. 4 ]The first strategy is to use a more liberal alpha value: a value of 0.05 gives greater power than an alpha value of 0.01, if all other things are equal. As mentioned above, however, it is a bad (and unethical) idea to change your alpha value after you've begun your study.