What is the role of statistics in scientific inquiry?

Answer 1

Statistics (tests, and not just summarized data) really does onebasic thing. When I design and carry out some experimental model, clearly it is to see if the treatments that I have done (treatments can be literally any intended, measurable manipulation of the subjects) will produce the effects that I predict. After I do the experiment and gather up all the data, how do I decide if my 'treatment' was in fact effective? With statistics, I have a reasonable way to answer the question. Again, in the broadest possible terms: I can create an imaginary computer version of my experimental model [scientists don't actually have to do this every time they do a study; it's already been done] and then put the computer to work randomly generating values for imaginary subjects. Why would I do this? You can see that if I do this with, say the same number of imaginary subjects as I have in my live study, and I do it (on the computer) over and over again, thousands or tens of thousands of times, I will get a wide variety of "results" out of the computer. Most of the results will show nothing of particular interest. A small number of results will "show" very clear significant results in the direction that I would want, and a small number would clearly be significant in the 'wrong' direction (remember, we are just looking at data). But by comparing my actual results with the computer model, I am able to say some objective things about my hypothesis. I might be able to say, for example, "The computer model data that resembles or is better than my real data only happens 5% of the time! This means that if my treatment is completely meaningless and has no affect at all, [like the completely random data generated by the computer] and if I were able to do this exact experiment the very same way 100 times, I would expect to get results that "look" this good only about 5 times out of the 100, by nothing but chance. This would represent a confidence level of 95%. I am not certain beyond all possible doubt that my treatment produced the results, but I can actually quantify my degree of certainty, and this method of establishing confidence is understood by fellow researchers.