What are the limitations of statistics?

Answer 1

First of all, there's the sample size. If you don't have a big enough sample, you can't give a very reliable answer. Of course, with limited resources it might not be possible to collect a large sample. There's a tradeoff between sample size and reliability.

Also, there are aspects of statistics which are countrintuitive and tend to confuse people. For instance, Simpson's Paradox. We all know that lots of sunlight is good for crops. But if you do a survey of crop yield and compare it to weather records, you might actually find that the sunnier it is, the less the crops grow. This is different from the problem of small sample size outlined above. I'm not talking about a few freak years with lots of pests throwing the statistics out. The problem is that when it's sunny, it generally isn't raining. So the years with lots of sunshine have lower crop yields because they have less rain. It is possible to correct for this, by looking at records of both sunshine and rainfall. Basically, if you're looking for a direct relationship between two variables, you need to think of all the other variables that might have an effect and take those into account. But I think this increases the sample size needed, if you overdo it.

Another fallacy is to assume that correlation implies causality. For instance, you could do a survey and discover that poorer neighbourhoods have more crime. You might conclude that living in such a place makes someone a criminal. But this might not be the case. It might be that wealth affects both crime and place of residence, i.e. poor people tend to live in certain places (because they can't afford to live anywhere else), and poor people often turn to crime (to feed their families). Or there might be some other variable we haven't thought of yet.

Actually, statistics is an exact science. It is the application of the results that leads to problems.

Perhaps the most commonly encountered limitation of statistics is the misunderstanding that a statistical measure can be used as a measure of the accuracy of a measurement.

Statistics, in general, provide very little information on the intrinsic accuracy of a measurement. Statistics can only provide an estimate of the minimal error that might be in the measurement. The actual error can be much greater than the minimal (statistical) error.

Another way to put it is that statistics measure the variability of a measurment, not the accuracy of a measurment.