Mbe Scoring

Discussions related to the bar exam are found in this forum
dabigchina

Silver
Posts: 1424
Joined: Mon Jan 13, 2014 2:22 am

Re: Mbe Scoring

Postby dabigchina » Sat Jul 21, 2018 2:34 pm

JoeSeperac wrote:
ladybug1989 wrote:Sorry, this might be a noob question.

I'm taking the CA bar and got a 130/200 on the Kaplan Final. The instructor said for scaling purposes to add 20 points, which would mean 150. Does that mean my scaled MBE for CA would be roughly a 1500? I would like to know so I can use the calculator that's posted above. Thanks.


For true scaling purposes, as I mention above, in J17, a 115 raw MBE (66% correct) would have resulted in a 141.2 scaled MBE. So if you are about 65% correct on the J18 MBE, you should see a 140-145 MBE. I think what Kaplan is saying is that examinees that get a 130/200 on the Kaplan exam get a 1500 scaled on the CA MBE (which would be about 72% correct on the actual MBE). This is because you study more after the Kaplan test and increase your knowledge, boosting your overall % correct by exam-time.

Appreciate the responses. Can you also shed some light on CA essay grading? Specifically, what percentile would you estimate a 55, 60, and 65 are?

User avatar
CLSGumbo

New
Posts: 43
Joined: Tue Jul 19, 2016 7:04 pm

Re: Mbe Scoring

Postby CLSGumbo » Sat Jul 21, 2018 5:04 pm

JoeSeperac wrote:



Joe, could you explain why on your UBE calculator for DC/Alabama/Colorado which use the 6 point scoring system, there can be such a huge difference between passing for the same MBE and essay grades? For example, getting all 3's in the Colorado MEE/MPT will reach 266 but getting all 3's in DC/Alabama would not, assuming a 133 MBE

Ramonlivliv

New
Posts: 1
Joined: Tue Apr 24, 2018 11:49 am

Re: Mbe Scoring

Postby Ramonlivliv » Sat Jul 21, 2018 9:02 pm

JoeSeperac wrote:
ladybug1989 wrote:Sorry, this might be a noob question.

I'm taking the CA bar and got a 130/200 on the Kaplan Final. The instructor said for scaling purposes to add 20 points, which would mean 150. Does that mean my scaled MBE for CA would be roughly a 1500? I would like to know so I can use the calculator that's posted above. Thanks.


For true scaling purposes, as I mention above, in J17, a 115 raw MBE (66% correct) would have resulted in a 141.2 scaled MBE. So if you are about 65% correct on the J18 MBE, you should see a 140-145 MBE. I think what Kaplan is saying is that examinees that get a 130/200 on the Kaplan exam get a 1500 scaled on the CA MBE (which would be about 72% correct on the actual MBE). This is because you study more after the Kaplan test and increase your knowledge, boosting your overall % correct by exam-time.


This is just my take on everything and Kaplans final. I think Kaplan gives you too many scaled points for the final. They say it is because the exam is more "difficult". I thought it was pretty easy, got a 75% on it, which would give me a 170 scaled score. Not even close to what I got on the UBE. I got a 149 scaled MBE this February but at least I passed in a UBE state. I got a 144 scaled MBE in July 2017 and failed the CBX, so I am taking the CBX again. (Did 72 percent on that final) I feel you get so used to the style of Kaplan MBE questions that you get surprised when you take the actual MBE. I used Adaptibar since March and I feel way better doing real MBE questions than with Kaplan. I just use Kaplan for the other stuff. So, my take is, don't get a false sense of security if you score high on Kaplan exams, keep studying and know that the hypos might sound different than the ones you are used to. Good luck!

JoeSeperac

Bronze
Posts: 244
Joined: Thu Feb 16, 2017 3:30 pm

Re: Mbe Scoring

Postby JoeSeperac » Sun Jul 22, 2018 3:30 pm

CLSGumbo wrote:Joe, could you explain why on your UBE calculator for DC/Alabama/Colorado which use the 6 point scoring system, there can be such a huge difference between passing for the same MBE and essay grades? For example, getting all 3's in the Colorado MEE/MPT will reach 266 but getting all 3's in DC/Alabama would not, assuming a 133 MBE


The calculators are correct. The difference is due to the scaling. Basically, the higher the MBE mean or the lower the standard deviation, the higher the scale. While these jurisdictions do not release the MBE means for each exam, CO must have a higher MBE mean than DC and both are much higher than Alabama. I explain it here along with a calculator for you to test:

https://seperac.com/zcalc-raw-scaled.php

JoeSeperac

Bronze
Posts: 244
Joined: Thu Feb 16, 2017 3:30 pm

Re: Mbe Scoring

Postby JoeSeperac » Sun Jul 22, 2018 3:37 pm

ladybug1989 wrote:If a friend who took the NY bar last year got a scaled MBE score of 160 (since NY releases scores), does that mean in CA, if he took the same MBE, he would have received a scaled score of 1600? Or does CA do its own independent scaling and therefore his scaled score in CA might be different? Thanks.


Yes. For example, an examinee in California who scored a 1561 on the F18 MBE answered the same number of MBE questions correctly as an examinee in NY who scored a 156.1 on the F18 MBE (146/175 correct).

JoeSeperac

Bronze
Posts: 244
Joined: Thu Feb 16, 2017 3:30 pm

Re: Mbe Scoring

Postby JoeSeperac » Sun Jul 22, 2018 3:57 pm

dabigchina wrote:Appreciate the responses. Can you also shed some light on CA essay grading? Specifically, what percentile would you estimate a 55, 60, and 65 are?


I made a crude estimate below. I converted each essay average to what MBE score they would represent and then compared those to the J17 National %tiles and then made a crude adjustment to what I would expect to be the CA %tiles (CA examinees score better than national examinees)

Image

For example, if you scored 55 on all your essays and PT on the J17 CA bar exam, it contributed the same amount of points to your final score as a 124 MBE would have. Nationally, a 124 MBE would have meant you were better than only 15.7% of examinees nationwide in J17 and better than only about 15.4% of examinees in CA, and I am presuming these MBE %tiles are consistent with the written %tiles (usually they are very close).

Cop2lawyerNYC

New
Posts: 56
Joined: Sat Jul 21, 2018 7:51 pm

Re: Mbe Scoring

Postby Cop2lawyerNYC » Sun Jul 22, 2018 4:03 pm

JoeSeperac wrote:
ladybug1989 wrote:If a friend who took the NY bar last year got a scaled MBE score of 160 (since NY releases scores), does that mean in CA, if he took the same MBE, he would have received a scaled score of 1600? Or does CA do its own independent scaling and therefore his scaled score in CA might be different? Thanks.


Yes. For example, an examinee in California who scored a 1561 on the F18 MBE answered the same number of MBE questions correctly as an examinee in NY who scored a 156.1 on the F18 MBE (146/175 correct).


146/175 = 83% correct. I would have thought that 83% correct would put you in 166 scaled-territory?

JoeSeperac

Bronze
Posts: 244
Joined: Thu Feb 16, 2017 3:30 pm

Re: Mbe Scoring

Postby JoeSeperac » Sun Jul 22, 2018 4:55 pm

Cop2lawyerNYC wrote:
JoeSeperac wrote:
ladybug1989 wrote:If a friend who took the NY bar last year got a scaled MBE score of 160 (since NY releases scores), does that mean in CA, if he took the same MBE, he would have received a scaled score of 1600? Or does CA do its own independent scaling and therefore his scaled score in CA might be different? Thanks.


Yes. For example, an examinee in California who scored a 1561 on the F18 MBE answered the same number of MBE questions correctly as an examinee in NY who scored a 156.1 on the F18 MBE (146/175 correct).


146/175 = 83% correct. I would have thought that 83% correct would put you in 166 scaled-territory?


I should mention that all my J17 & F18 scaled-to-raw conversions are guesstimates. For various reasons, NCBE doesn't want examinees to know their raw scores, so the only way to figure them out is to look at a lot of scores and figure out all the permutations. If I can get more sub-scores from failing J17 & F18 examinees, I can better estimate these values:
https://seperac.com/subscoreform.php

As to the low scaling, it's probably because it was a February exam. A 156 MBE in F18 was 92nd percentile (only 8% of F18 examinees nationwide did better than a 156 MBE). Don't worry about scaling - just get 65% correct or better on the MBE and you will be in an excellent position to pass any state except CA.

dabigchina

Silver
Posts: 1424
Joined: Mon Jan 13, 2014 2:22 am

Re: Mbe Scoring

Postby dabigchina » Sun Jul 22, 2018 5:34 pm

JoeSeperac wrote:
dabigchina wrote:Appreciate the responses. Can you also shed some light on CA essay grading? Specifically, what percentile would you estimate a 55, 60, and 65 are?


I made a crude estimate below. I converted each essay average to what MBE score they would represent and then compared those to the J17 National %tiles and then made a crude adjustment to what I would expect to be the CA %tiles (CA examinees score better than national examinees)

Image

For example, if you scored 55 on all your essays and PT on the J17 CA bar exam, it contributed the same amount of points to your final score as a 124 MBE would have. Nationally, a 124 MBE would have meant you were better than only 15.7% of examinees nationwide in J17 and better than only about 15.4% of examinees in CA, and I am presuming these MBE %tiles are consistent with the written %tiles (usually they are very close).

Thank you Joe. You are my hero.

Nightcrawler

Bronze
Posts: 180
Joined: Wed Apr 04, 2018 12:02 pm

Re: Mbe Scoring

Postby Nightcrawler » Tue Jul 31, 2018 3:19 pm

JoeSeperac wrote:


Joe, would July 2018 takers benefit from the F18 super low scale?

Jdsquared87

New
Posts: 88
Joined: Tue Mar 13, 2018 4:45 pm

Re: Mbe Scoring

Postby Jdsquared87 » Tue Jul 31, 2018 5:39 pm

Hey joe seperec!

I used your calculator for the f18 ube which said i would pass but i failed with a 257 (125 mbe 133 mee). I scored in the 99-100 range (50%)on the kaplan mid term and final but was scoring in the 65% range on the other practice questions. I felt i did horrible on the essays 2 of which i completely made up law for and still managed a 133.


I took the j18 ube and used kaplan and emmanuels s&e. This time my kaplan midterm was a 110 and my final was a 130 (65%) my practice quizzes were in the 70 to 80 % range and my emmanuels final was 64%. This time i felt extremely confident on all 6 essays and mpt. I've used your calculator which says its too close to call (close to failing) and now I'm freaking out.

Any insight?

User avatar
NoBladesNoBows

Silver
Posts: 1156
Joined: Fri Aug 22, 2014 7:39 pm

Re: Mbe Scoring

Postby NoBladesNoBows » Wed Aug 01, 2018 11:14 am

Jdsquared87 wrote:Hey joe seperec!

I used your calculator for the f18 ube which said i would pass but i failed with a 257 (125 mbe 133 mee). I scored in the 99-100 range (50%)on the kaplan mid term and final but was scoring in the 65% range on the other practice questions. I felt i did horrible on the essays 2 of which i completely made up law for and still managed a 133.


I took the j18 ube and used kaplan and emmanuels s&e. This time my kaplan midterm was a 110 and my final was a 130 (65%) my practice quizzes were in the 70 to 80 % range and my emmanuels final was 64%. This time i felt extremely confident on all 6 essays and mpt. I've used your calculator which says its too close to call (close to failing) and now I'm freaking out.

Any insight?

If you don't mind me asking, what did the calculator predict you would get?

Jdsquared87

New
Posts: 88
Joined: Tue Mar 13, 2018 4:45 pm

Re: Mbe Scoring

Postby Jdsquared87 » Wed Aug 01, 2018 11:28 am

NoBladesNoBows wrote:
Jdsquared87 wrote:Hey joe seperec!

I used your calculator for the f18 ube which said i would pass but i failed with a 257 (125 mbe 133 mee). I scored in the 99-100 range (50%)on the kaplan mid term and final but was scoring in the 65% range on the other practice questions. I felt i did horrible on the essays 2 of which i completely made up law for and still managed a 133.


I took the j18 ube and used kaplan and emmanuels s&e. This time my kaplan midterm was a 110 and my final was a 130 (65%) my practice quizzes were in the 70 to 80 % range and my emmanuels final was 64%. This time i felt extremely confident on all 6 essays and mpt. I've used your calculator which says its too close to call (close to failing) and now I'm freaking out.


Any insight?

If you don't mind me asking, what did the calculator predict you would get?



Specifically, a 265 and saying 'too close to call.'

The first time it said i would score around the 270 range, i failed.

Cop2lawyerNYC

New
Posts: 56
Joined: Sat Jul 21, 2018 7:51 pm

Re: Mbe Scoring

Postby Cop2lawyerNYC » Wed Aug 01, 2018 11:46 am

Jdsquared87 wrote:Hey joe seperec!

I used your calculator for the f18 ube which said i would pass but i failed with a 257 (125 mbe 133 mee). I scored in the 99-100 range (50%)on the kaplan mid term and final but was scoring in the 65% range on the other practice questions. I felt i did horrible on the essays 2 of which i completely made up law for and still managed a 133.


I took the j18 ube and used kaplan and emmanuels s&e. This time my kaplan midterm was a 110 and my final was a 130 (65%) my practice quizzes were in the 70 to 80 % range and my emmanuels final was 64%. This time i felt extremely confident on all 6 essays and mpt. I've used your calculator which says its too close to call (close to failing) and now I'm freaking out.

Any insight?


Just curious how you define “extremely confident on all 6 esssays...” you think you nailed all 6 down? Or 4/6?...
If you got a 133 with crappy essays and you nailed it this time around you should be good to go. A median essay is 143; couple with a 125 on the mbe and you’re all set.
Most people missed question #3. If you got this along with the other 5 then you’d certainly be above median.

Jdsquared87

New
Posts: 88
Joined: Tue Mar 13, 2018 4:45 pm

Re: Mbe Scoring

Postby Jdsquared87 » Wed Aug 01, 2018 11:57 am

Cop2lawyerNYC wrote:
Jdsquared87 wrote:Hey joe seperec!

I used your calculator for the f18 ube which said i would pass but i failed with a 257 (125 mbe 133 mee). I scored in the 99-100 range (50%)on the kaplan mid term and final but was scoring in the 65% range on the other practice questions. I felt i did horrible on the essays 2 of which i completely made up law for and still managed a 133.


I took the j18 ube and used kaplan and emmanuels s&e. This time my kaplan midterm was a 110 and my final was a 130 (65%) my practice quizzes were in the 70 to 80 % range and my emmanuels final was 64%. This time i felt extremely confident on all 6 essays and mpt. I've used your calculator which says its too close to call (close to failing) and now I'm freaking out.

Any insight?


Just curious how you define “extremely confident on all 6 esssays...” you think you nailed all 6 down? Or 4/6?...
If you got a 133 with crappy essays and you nailed it this time around you should be good to go. A median essay is 143; couple with a 125 on the mbe and you’re all set.
Most people missed question #3. If you got this along with the other 5 then you’d certainly be above median.


I felt like I did well on all of the essays. I did, however, run out of time on the 2nd mpt so i didn't finish the last small portion but other than that, i feel alright about my written portion. Honestly though, who the hell knows...the grading is so subjective.

User avatar
NoBladesNoBows

Silver
Posts: 1156
Joined: Fri Aug 22, 2014 7:39 pm

Re: Mbe Scoring

Postby NoBladesNoBows » Wed Aug 01, 2018 11:59 am

Jdsquared87 wrote:
NoBladesNoBows wrote:
Jdsquared87 wrote:Hey joe seperec!

I used your calculator for the f18 ube which said i would pass but i failed with a 257 (125 mbe 133 mee). I scored in the 99-100 range (50%)on the kaplan mid term and final but was scoring in the 65% range on the other practice questions. I felt i did horrible on the essays 2 of which i completely made up law for and still managed a 133.


I took the j18 ube and used kaplan and emmanuels s&e. This time my kaplan midterm was a 110 and my final was a 130 (65%) my practice quizzes were in the 70 to 80 % range and my emmanuels final was 64%. This time i felt extremely confident on all 6 essays and mpt. I've used your calculator which says its too close to call (close to failing) and now I'm freaking out.


Any insight?

If you don't mind me asking, what did the calculator predict you would get?



Specifically, a 265 and saying 'too close to call.'

The first time it said i would score around the 270 range, i failed.

Thanks for the response. Is this the calculator based on demographic/testing/gpa information or the one where you estimate how you did on the individual sections?

Jdsquared87

New
Posts: 88
Joined: Tue Mar 13, 2018 4:45 pm

Re: Mbe Scoring

Postby Jdsquared87 » Wed Aug 01, 2018 12:04 pm

NoBladesNoBows wrote:
Jdsquared87 wrote:
NoBladesNoBows wrote:
Jdsquared87 wrote:Hey joe seperec!

I used your calculator for the f18 ube which said i would pass but i failed with a 257 (125 mbe 133 mee). I scored in the 99-100 range (50%)on the kaplan mid term and final but was scoring in the 65% range on the other practice questions. I felt i did horrible on the essays 2 of which i completely made up law for and still managed a 133.


I took the j18 ube and used kaplan and emmanuels s&e. This time my kaplan midterm was a 110 and my final was a 130 (65%) my practice quizzes were in the 70 to 80 % range and my emmanuels final was 64%. This time i felt extremely confident on all 6 essays and mpt. I've used your calculator which says its too close to call (close to failing) and now I'm freaking out.


Any insight?

If you don't mind me asking, what did the calculator predict you would get?



Specifically, a 265 and saying 'too close to call.'

The first time it said i would score around the 270 range, i failed.

Thanks for the response. Is this the calculator based on demographic/testing/gpa information or the one where you estimate how you did on the individual sections?



No...the one I'm referring to is the demographic based calculator.

User avatar
NoBladesNoBows

Silver
Posts: 1156
Joined: Fri Aug 22, 2014 7:39 pm

Re: Mbe Scoring

Postby NoBladesNoBows » Wed Aug 01, 2018 12:06 pm

Jdsquared87 wrote:
NoBladesNoBows wrote:
Jdsquared87 wrote:
NoBladesNoBows wrote:
Jdsquared87 wrote:Hey joe seperec!

I used your calculator for the f18 ube which said i would pass but i failed with a 257 (125 mbe 133 mee). I scored in the 99-100 range (50%)on the kaplan mid term and final but was scoring in the 65% range on the other practice questions. I felt i did horrible on the essays 2 of which i completely made up law for and still managed a 133.


I took the j18 ube and used kaplan and emmanuels s&e. This time my kaplan midterm was a 110 and my final was a 130 (65%) my practice quizzes were in the 70 to 80 % range and my emmanuels final was 64%. This time i felt extremely confident on all 6 essays and mpt. I've used your calculator which says its too close to call (close to failing) and now I'm freaking out.


Any insight?

If you don't mind me asking, what did the calculator predict you would get?



Specifically, a 265 and saying 'too close to call.'

The first time it said i would score around the 270 range, i failed.

Thanks for the response. Is this the calculator based on demographic/testing/gpa information or the one where you estimate how you did on the individual sections?



No...the one I'm referring to is the demographic based calculator.

It seems like it dings you somewhat substantially for it being your second time taking it, which is probably accurate overall but based on the fact that you were so close the first time might mean it's not as accurate a predictor as it would be for others who failed miserably. Sounds like you were well prepared and confident this time, and fwiw myself, and pretty much everyone I've talked to (even people top of the class) felt like it was much more difficult than they expected lol. Good luck!

Jdsquared87

New
Posts: 88
Joined: Tue Mar 13, 2018 4:45 pm

Re: Mbe Scoring

Postby Jdsquared87 » Wed Aug 01, 2018 12:15 pm

NoBladesNoBows wrote:
Jdsquared87 wrote:
NoBladesNoBows wrote:
Jdsquared87 wrote:
NoBladesNoBows wrote:
Jdsquared87 wrote:Hey joe seperec!

I used your calculator for the f18 ube which said i would pass but i failed with a 257 (125 mbe 133 mee). I scored in the 99-100 range (50%)on the kaplan mid term and final but was scoring in the 65% range on the other practice questions. I felt i did horrible on the essays 2 of which i completely made up law for and still managed a 133.


I took the j18 ube and used kaplan and emmanuels s&e. This time my kaplan midterm was a 110 and my final was a 130 (65%) my practice quizzes were in the 70 to 80 % range and my emmanuels final was 64%. This time i felt extremely confident on all 6 essays and mpt. I've used your calculator which says its too close to call (close to failing) and now I'm freaking out.


Any insight?

If you don't mind me asking, what did the calculator predict you would get?



Specifically, a 265 and saying 'too close to call.'

The first time it said i would score around the 270 range, i failed.

Thanks for the response. Is this the calculator based on demographic/testing/gpa information or the one where you estimate how you did on the individual sections?



No...the one I'm referring to is the demographic based calculator.

It seems like it dings you somewhat substantially for it being your second time taking it, which is probably accurate overall but based on the fact that you were so close the first time might mean it's not as accurate a predictor as it would be for others who failed miserably. Sounds like you were well prepared and confident this time, and fwiw myself, and pretty much everyone I've talked to (even people top of the class) felt like it was much more difficult than they expected lol. Good luck!




Oh without a doubt it knocks me out for it being my 2nd administration attempt. Thank you for the kind words- it does give me some hope.

I absolutely felt far more prepared this time around- and having felt underprepared last time- it was a welcome relief. Failing by 9 points was humbling- I don't think I took the preparation as serious the first time around but I can honestly say that I began studying again the day i found out i failed.

Again, I know in my mind I feel better about the entirety of my written portion of the test- but my crux and where I failed before was the mbe. I just pray I did well on the mbe or well enough on the mee to carry me to a 266.

JoeSeperac

Bronze
Posts: 244
Joined: Thu Feb 16, 2017 3:30 pm

Re: Mbe Scoring

Postby JoeSeperac » Thu Aug 09, 2018 10:36 am

Nightcrawler wrote:
JoeSeperac wrote:


Joe, would July 2018 takers benefit from the F18 super low scale?


Statistically, you will do better on a July exam if you can score high on the MBE. The February exam itself is not more difficult, but the examinees generally score lower, particularly on the MBE. These lower MBE scores negatively affect the scale that is applied to arrive at a final score. Scaling is done to increase the reliability of the exam. Your individual score on a particular exam doesn't affect your final score – if you had the 50th best MBE score and 90th best Essay score before scaling, you will still have the 50th best MBE score and 90th best Essay score after scaling. However, to account for the lower reliability of the non-MBE components of the exam, these components are scaled to the mean MBE for that administration. For example, nationally, the average MBE scaled score on July exams (from 1974-2016) was 142 while the average MBE on February exams (from 1974-2016) was 136.7. For scaling, the lower the mean MBE, the lower the scale. According to NCBE, "[s]caling the essays to the MBE is an essential step in ensuring that scores have a consistent meaning over time. When essay scores are not scaled to the MBE, they tend to remain about the same: for example, it is common for the average raw July essay score to be similar to the average February score even if the July examinees are known to be more knowledgeable on average than the February examinees. Using raw essay scores rather than scaled essay scores tends to provide an unintended advantage to some examinees and an unintended disadvantage to others." The Bar Examiner May 2005. (see http://seperac.com/pdf/740205_testing.pdf) (emphasis added)

You can see this effect in the New York pass rates between July and February. In New York, over the past 5 years of reported data, for the demographic of First Timers (NY ABA Law Schools), there were 18,393 examinees who took the exam in July and 15,382 examinees passed the exam for an overall pass rate of 83.6%. In contrast, there were 1,601 First Timers (NY ABA Law Schools) who took the exam in February and 1,074 examinees passed the exam for an overall pass rate of 67.1%. These graduates of New York ABA Schools should possess a similar level of proficiency (for example, pro bono scholars usually take the Feb exam). Accordingly you would expect consistent pass rates among these first time candidates between the July and February exams. However, between 2012-2016, there was a 16.5% difference in pass rates between July and February for First-Time examinees who are graduates of New York ABA Schools. Put simply, a high tide lifts all boats - the higher the MBE mean, the higher the scale.

JoeSeperac

Bronze
Posts: 244
Joined: Thu Feb 16, 2017 3:30 pm

Re: Mbe Scoring

Postby JoeSeperac » Thu Aug 09, 2018 11:19 am

Jdsquared87 wrote:Hey joe seperec!

I used your calculator for the f18 ube which said i would pass but i failed with a 257 (125 mbe 133 mee). I scored in the 99-100 range (50%)on the kaplan mid term and final but was scoring in the 65% range on the other practice questions. I felt i did horrible on the essays 2 of which i completely made up law for and still managed a 133.


I took the j18 ube and used kaplan and emmanuels s&e. This time my kaplan midterm was a 110 and my final was a 130 (65%) my practice quizzes were in the 70 to 80 % range and my emmanuels final was 64%. This time i felt extremely confident on all 6 essays and mpt. I've used your calculator which says its too close to call (close to failing) and now I'm freaking out.

Any insight?


As I mention on the UBE Score Estimator page, your MBE practice scores (assuming the MBE practice questions are of sufficient difficulty and representative of the topics tested) will give you the most insight as to whether or not you will pass the UBE. You may have had MBE endurance problems on your first attempt to score 65% in practice but 50% on simulated exams (that's a big drop-off). Did you ever send me your scores? Based on your scaled MBE score of 125, your estimated raw MBE score was about 93/175 correct. This means you answered about 53% of the graded MBE questions correctly which ties in with your simulated exam performance.

Your second exam practice stats look a lot better to me. If your performance was genuine (not based on recall of the questions from your 1st attempt), I would be reasonably confident in your scoring about a 135 MBE or higher.

A reminder to anyone who fails the bar exam, I can give you a ton of useful insight into your scoring if you fill out the following form:
https://seperac.com/scoreform.php

Nightcrawler

Bronze
Posts: 180
Joined: Wed Apr 04, 2018 12:02 pm

Re: Mbe Scoring

Postby Nightcrawler » Fri Aug 10, 2018 10:47 am

JoeSeperac wrote:
Nightcrawler wrote:
JoeSeperac wrote:


Joe, would July 2018 takers benefit from the F18 super low scale?


Statistically, you will do better on a July exam if you can score high on the MBE. The February exam itself is not more difficult, but the examinees generally score lower, particularly on the MBE. These lower MBE scores negatively affect the scale that is applied to arrive at a final score. Scaling is done to increase the reliability of the exam. Your individual score on a particular exam doesn't affect your final score – if you had the 50th best MBE score and 90th best Essay score before scaling, you will still have the 50th best MBE score and 90th best Essay score after scaling. However, to account for the lower reliability of the non-MBE components of the exam, these components are scaled to the mean MBE for that administration. For example, nationally, the average MBE scaled score on July exams (from 1974-2016) was 142 while the average MBE on February exams (from 1974-2016) was 136.7. For scaling, the lower the mean MBE, the lower the scale. According to NCBE, "[s]caling the essays to the MBE is an essential step in ensuring that scores have a consistent meaning over time. When essay scores are not scaled to the MBE, they tend to remain about the same: for example, it is common for the average raw July essay score to be similar to the average February score even if the July examinees are known to be more knowledgeable on average than the February examinees. Using raw essay scores rather than scaled essay scores tends to provide an unintended advantage to some examinees and an unintended disadvantage to others." The Bar Examiner May 2005. (see http://seperac.com/pdf/740205_testing.pdf) (emphasis added)

You can see this effect in the New York pass rates between July and February. In New York, over the past 5 years of reported data, for the demographic of First Timers (NY ABA Law Schools), there were 18,393 examinees who took the exam in July and 15,382 examinees passed the exam for an overall pass rate of 83.6%. In contrast, there were 1,601 First Timers (NY ABA Law Schools) who took the exam in February and 1,074 examinees passed the exam for an overall pass rate of 67.1%. These graduates of New York ABA Schools should possess a similar level of proficiency (for example, pro bono scholars usually take the Feb exam). Accordingly you would expect consistent pass rates among these first time candidates between the July and February exams. However, between 2012-2016, there was a 16.5% difference in pass rates between July and February for First-Time examinees who are graduates of New York ABA Schools. Put simply, a high tide lifts all boats - the higher the MBE mean, the higher the scale.


Thank you. Although, is there a relationship between the J18 and the F18 scale? In other words, when they equate the J18 MBE questions to the previous bar exam to adjust for difficulty, do they equate them to the performance of previous July exams or previous February exams? And in the case they compare them just to the previous exam in time (F18 in this case), would the J18 equation be particularly beneficial to takers because the F18 scale was lower than usual?

JoeSeperac

Bronze
Posts: 244
Joined: Thu Feb 16, 2017 3:30 pm

Re: Mbe Scoring

Postby JoeSeperac » Sun Aug 12, 2018 12:16 pm

Nightcrawler wrote:Thank you. Although, is there a relationship between the J18 and the F18 scale? In other words, when they equate the J18 MBE questions to the previous bar exam to adjust for difficulty, do they equate them to the performance of previous July exams or previous February exams? And in the case they compare them just to the previous exam in time (F18 in this case), would the J18 equation be particularly beneficial to takers because the F18 scale was lower than usual?


Yes, there is likely a relationship between F18 performance and the J18 scale. The MBE scale is determined by two things: (1) how the current administration does on the equator questions and (2) how difficult the non-equator questions are. I believe the scale is primarily determined based on how administrations test on the MBE equators. According to NCBE, "[t]wo sets of 'equator' items are carefully selected from two old test forms and inserted in a new test form. In terms of content coverage and statistical properties, each of these sets of equator items is like a miniature version of the previous full length test. Next, the balance of the new test form is developed to meet total test specifications." For example, the J18 MBE may draw 30 questions from a F18 MBE exam and 30 questions from the J17 MBE exam. If one administration does better than another on the equator items, that administration receives a higher scale. Obviously, July administrations will score better on the MBE than February administrations, leading to a higher scale for July candidates. I regard this as the most important part of the scaling because NCBE also regards it as very important. For example, “[a]s Dr. Mary Sandifer, the recently retired Deputy Director of Testing for the National Conference, discussed at one of the plenary sessions, a security breach of the MBE threatens the viability of the equating chain for the exam by exposing questions selected to be equators on future tests. Reworking an equating chain is a costly process, and test items with strong performance indicators are lost to the standardization process. The high cost of just one missing test booklet is measured not only in terms of the threat to the equating chain, but also in the additional outlay of staff and resources necessary to develop new test forms to replace the exposed materials. Equators that are “lost” to bar review courses (via post-test examinee debriefings or via examinee “ringers” who memorize questions) are also damaging. The effect of giving some applicants prior access to test questions undermines overall confidence in the scores and gives some examinees an unfair advantage.” The other factor is an adjustment for the difficulty of the non-equator questions. I have no idea how this is done or its impact.



Return to “Bar Exam Prep and Discussion Forum?

Who is online

Users browsing this forum: No registered users and 9 guests