Nightcrawler wrote:JoeSeperac wrote:
Joe, would July 2018 takers benefit from the F18 super low scale?
Statistically, you will do better on a July exam if you can score high on the MBE. The February exam itself is not more difficult, but the examinees generally score lower, particularly on the MBE. These lower MBE scores negatively affect the scale that is applied to arrive at a final score. Scaling is done to increase the reliability of the exam. Your individual score on a particular exam doesn't affect your final score – if you had the 50th best MBE score and 90th best Essay score before scaling, you will still have the 50th best MBE score and 90th best Essay score after scaling. However, to account for the lower reliability of the non-MBE components of the exam, these components are scaled to the mean MBE for that administration. For example, nationally, the average MBE scaled score on July exams (from 1974-2016) was 142 while the average MBE on February exams (from 1974-2016) was 136.7. For scaling, the lower the mean MBE, the lower the scale. According to NCBE, "[s]caling the essays to the MBE is an essential step in ensuring that scores have a consistent meaning over time.
When essay scores are not scaled to the MBE, they tend to remain about the same: for example, it is common for the average raw July essay score to be similar to the average February score even if the July examinees are known to be more knowledgeable on average than the February examinees. Using raw essay scores rather than scaled essay scores tends to provide an unintended advantage to some examinees and an unintended disadvantage to others." The Bar Examiner May 2005. (see
http://seperac.com/pdf/740205_testing.pdf) (emphasis added)
You can see this effect in the New York pass rates between July and February. In New York, over the past 5 years of reported data, for the demographic of First Timers (NY ABA Law Schools), there were 18,393 examinees who took the exam in July and 15,382 examinees passed the exam for an overall pass rate of 83.6%. In contrast, there were 1,601 First Timers (NY ABA Law Schools) who took the exam in February and 1,074 examinees passed the exam for an overall pass rate of 67.1%. These graduates of New York ABA Schools should possess a similar level of proficiency (for example, pro bono scholars usually take the Feb exam). Accordingly you would expect consistent pass rates among these first time candidates between the July and February exams. However, between 2012-2016, there was a 16.5% difference in pass rates between July and February for First-Time examinees who are graduates of New York ABA Schools. Put simply, a high tide lifts all boats - the higher the MBE mean, the higher the scale.
