Generally, there is no problem comparing means for groups of different sizes unless the sizes of some groups are extremely different from the others. You can consider the mean of a larger group as being weightier than the means of smaller groups. What you cannot do is take a collection of means and calculate a 'grand mean' by simply calculating the mean of the means. This would be 'weighing' all the means as the same, or in other words, assuming that each of the the means was generated by the same number of scores. It helps to compare variance or standard deviation as well as means, because this tells you something about possible problems with selection or group size. Statistical software packages make this easier if you are comparing means in the context of a stats test. Whether or not you can treat the groups as having equal variances will have some impact on your test's confidence levels.
Chat with our AI personalities