byag wrote:Hi Joe @JoeSeperac,
I took the February 2018 NY UBE exam and received a 162.6 written and a 134.4 MBE. Can you please tell me where my scores rank? Your website was extremely useful to me; due in part to your analysis (and other resources) I was able to increase my total UBE score from 247 (July '17) to 297. I am now preparing for the FL exam and need to raise my MBE score to FL's required 136 (missed it by 1.6!!) so I look forward to using your website again in the coming months. Thanks!
Is it just me or do so many people this time around have really high essay scores compared to MBE scores? I have seen many people post @joe for analysis and their essays be 15-20 sometimes 30 points higher than MBE.
Did you score high on essays or MPT? Makes me think there is something in a question or two where some people just got it correct (like civ pro sanctions rule, property, and crim law fit for trial rule) and shot them ahead of curve.
I answered you by email but I'm answering here too in case anyone else is interested.
Those samples are too small to make any type of prediction. For example, in F17, of the 100+ examinees who sent me their scores (still a really small sample), the difference between MBE and Written Scores ranged from a max of 43 to a min of -28 with an average difference of 3. My guess is that if you look at the entire population, the average difference between MBE and Written Scores will always be less than one (as in the MO data).
As much as everyone dislikes scaling, it leads to a more reliable score. For example, prior to July 2012, Michigan scaled the essays to the MBE. Then, for the July 2012 exam, the Michigan Board of Law Examiners changed the grading formula and stopped scaling the raw essay points on the Michigan portion of the bar exam. From what I have read on this, the Michigan bar examiners felt the quality of the Michigan examinees' essays was diminishing, but their MBE scores were improving, resulting in higher pass rates due to the better MBE scores coupled with the scaling of essay scores to the MBE. What has likely happening was that Michigan examinees were focusing on the MBE portion of the exam at the expense of the written portion. Since the written portion was scaled to the MBE using an equating method, the overall decline in performance on the essay portion was masked by the MBE equating. In one of the articles I read, the Michigan Board of Law Examiners were complaining that the examinee essays were pretty poor as compared to prior years. Thus, the Michigan bar examiners stopped scaling their essays to the MBE in July 2012. However, this resulted in a precipitious drop in the July 2012 Michigan pass rates (between 1995-2011 the July pass rate was about 75% and then the July 2012 pass rate was 57% - the lowest it had even been prior to that was 59% in 1995). What happened after that is exactly what was predicted by NCBE – the pass rates between July and February started becoming consistent, even though the pool of candidates in July were considered more knowledgeable (between 1995-2011 the July pass rate was about 75% while the Feb pass rate was about 67%; for the period between 2012-2014, July pass rate was about 61% while the Feb pass rate was about 62%). By not scaling, the 2012-2014 February Michigan examinees were passing the exam at a higher percentage than July examinees who should be more knowledgeable. In July 2014, Michigan flip-flopped and went back to scaling essays (likely to stop giving an unintended advantage to February examinees).
As much as the MBE drives whether you pass, the MBE average will drive the scaling of the essays. For example, a few years back NY tested a really weird corporations question. Many examinees bombed it (the passing essays were only 300-400 hundred words long whereas typically a passing essay was 800 words long). But it really didn’t affect the scoring or scale. Everyone will just get rank-ordered and then the scale adjusts the score to reflect the overall knowledge-level of that administration of examinees.