drdolittle wrote:disco_barred wrote:bilbobaggins wrote:The differences are not necessarily statistically significant.

Statistical significance is a measure used to determine whether or not an observed measurement based on a sample is likely to be representative of the group. Berkeley released its median data in 2009, and it did so in 2010. The figures are not based on a random sample, but rather the entire population of enrolled Boalt students. Thus the term 'statistical significance' has no application to this situation.

Random & dependent sampling is not necessary to determine statistical significance. More sophisticated statistical analyses like one-way ANOVA (vs. a simple Student's t-test, for ex.) would do the job of comparing different and independent samples perfectly well, given the full data set. Yearly GPAs and LSATs have means and distributions, and it would be straightforward to determine whether those means are statistically different year to year. Also, just looking at the numbers released, a problem is that we don't have the exact values down to the decimal fraction, especially for the LSAT, nor do we know the yearly distributions. For example, if the 2009 low LSAT average was 164.5, rounded up to 165, and the 2010 low LSAT were 163.4, rounded down to 163, then the year to year differences would be accentuated by rounding, i.e. from actually being only 1.1 apart to apparently being 2 apart. And not knowing the full distributions of scores each year makes such comparisons impossible to really interpret anyway.

1) Of course statistical significance is a relevant concept. “Significance” is defined as a difference of a magnitude unlikely to be the result of random score variation. If your question was whether or not the medians dropped at Berkeley, then statistical significance is of no concern. Unfortunately, that question is also of no value. The valuable question is whether or not the difference in LSAT and GPA indicates that the 2010 entering class is different than the 2009 class. All of the reasons that people have suggested rely upon this inference, and this inference relies upon a statistically significant difference.

2) LSAT and GPA are ordinal (rather than interval) scales. This is why the median is reported instead of the average. You can’t average them without introducing error. The statistical procedures that people have suggested all require averages.

3) None of this matters since we have no variability data.

4) Even a statistically significant drop in medians would not mean very much unless it was repeated for 2011. Two-point-five percent of all law schools would be expected to have a statistically significant decline in their medians every year due to random variation (assuming a p < .05 significance level).