TheJuryMustDie wrote:Hi Joe, I understand. I just don't think it makes (complete) rational sense to wholly scale the essays to the MBE when there are limitless variables that can affect the mean of any given MBE administration. I met candidates in New York for example who only came to the exam because they feared being sanctioned for repeated withdrawers and spent less than 20 minutes randomly bubbling their scantron sheets. Whatever these candidates score would obviously form part of the mean's calculation, wouldn't they? Considering the number of such candidates could be huge, scaling the essays to the MBE in such circumstances would appear an unfair penalty on takers of such administration.
Can you please tell if the essays are scaled to the national mean of the MBE or jurisdictional MBE's mean? And for the UBE states, do they all have to scale their essays to the MBE or that's down to individual jurisdiction's discretion which scaling mechanism they adopt?
It's my understanding that every jurisdiction determines their scale based on the MBE scores of the examinees in that jurisdiction rather than using a national scale. I also believe that every UBE jurisdiction scales the MEE/MPT scores to the MBE. It's hard to answer this because the state bar examiners release very little about their methods - I can only make indirect observations after looking at score reports.
As much as you may dislike scaling, it leads to a more reliable score. For example, prior to July 2012, Michigan scaled the essays to the MBE. Then, for the July 2012 exam, the Michigan Board of Law Examiners changed the grading formula and stopped scaling the raw essay points on the Michigan portion of the bar exam. From what I have read on this, the Michigan bar examiners felt the quality of the Michigan examinees' essays was diminishing, but their MBE scores were improving, resulting in higher pass rates due to the better MBE scores coupled with the scaling of essay scores to the MBE. What has likely happening was that Michigan examinees were focusing on the MBE portion of the exam at the expense of the written portion. Since the written portion was scaled to the MBE using an equating method, the overall decline in performance on the essay portion was masked by the MBE equating. In one of the articles I read, the Michigan Board of Law Examiners were complaining that the examinee essays were pretty poor as compared to prior years. Thus, the Michigan bar examiners stopped scaling their essays to the MBE in July 2012. However, this resulted in a precipitious drop in the July 2012 Michigan pass rates (between 1995-2011 the July pass rate was about 75% and then the July 2012 pass rate was 57% - the lowest it had even been prior to that was 59% in 1995). What happened after that is exactly what was predicted by NCBE – the pass rates between July and February started becoming consistent, even though the pool of candidates in July were considered more knowledgeable (between 1995-2011 the July pass rate was about 75% while the Feb pass rate was about 67%; for the period between 2012-2014, July pass rate was about 61% while the Feb pass rate was about 62%). By not scaling, the 2012-2014 February Michigan examinees were passing the exam at a higher percentage than July examinees who should be more knowledgeable. In July 2014, Michigan flip-flopped and went back to scaling essays (likely to stop giving an unintended advantage to February examinees).
In the end, with or without scaling, a licensure exam is basically a form of economic protectionism - a bar exam passing score operates as an arbitrary limit on the number of attorneys licensed in a jurisdiction. For example, in a few states (Vermont, North Dakota, Alaska, and South Dakota), less than 100 examinees take that state's bar exam each year. With such a small group, one would think that could be possible for each and every candidate to be qualified to practice law (if minimal competency was indeed the measure). However, the 20-year pass rate average for these states was between 64%-82%. In Alaska, the highest pass rate over the past ten years was 70%; in Vermont, the highest pass rate over the past ten years was 71%; in North Dakota, the highest pass rate over the past ten years was 83%; and in South Dakota, the highest pass rate over the past ten years was 94%.