Page 20 of 20

Re: The "Ask @JoeSeperac" Thread

Posted: Sat Oct 10, 2020 2:55 pm
by AlexFergusonLS
Thanks for all your helpful advice Joe.

Re: The "Ask @JoeSeperac" Thread

Posted: Sat Oct 10, 2020 3:01 pm
by Anonymous User
Joe,

I had some family health issues that preventing me from properly studying. I ended up only studying for the MBE with zero essay or MPT prep and only intensively for 2 weeks beforehand. I'm starting a biglaw job shortly and am worried I will fail.

However, my 3 practice scores on 50 problem sets on Themis beforehand were 66 68 and 67. I graduated from a T10 in the top 10% of my class. My LSAT was 171. This gives me some hope based on what you've said before.

I didn't know one of the essay topics at all, and just made stuff up, and the other two were mediocre at best.

Any thoughts on whether I stand a good shot of passing or should I start starting for February now?

Also - you have some anecdotes of people with terrible essay results still passing. Do you have any links to examples of bad essays that still passed?

Re: The "Ask @JoeSeperac" Thread

Posted: Sun Oct 11, 2020 2:29 pm
by JoeSeperac
Anonymous User wrote:
Sat Oct 10, 2020 3:01 pm
Joe,

I had some family health issues that preventing me from properly studying. I ended up only studying for the MBE with zero essay or MPT prep and only intensively for 2 weeks beforehand. I'm starting a biglaw job shortly and am worried I will fail.

However, my 3 practice scores on 50 problem sets on Themis beforehand were 66 68 and 67. I graduated from a T10 in the top 10% of my class. My LSAT was 171. This gives me some hope based on what you've said before.

I didn't know one of the essay topics at all, and just made stuff up, and the other two were mediocre at best.

Any thoughts on whether I stand a good shot of passing or should I start starting for February now?

Also - you have some anecdotes of people with terrible essay results still passing. Do you have any links to examples of bad essays that still passed?

Based on your limited practice scores, I predict an MBE score between 137-147 on the J20 exam. This is a very rough estimate. If you get an MBE score of 140 or better, I expect you to pass regardless of your essay/MPT performance. A relevant anecdote:

If you're anything like I was (and I'm guessing at least 60-70% of you are), try as you might, you're obsessing over what you think--you don't know anything for sure--you "fucked up/completely bombed/totally blew" on the test yesterday, be it on an essay, the MPT, and/or MBE question(s).

If it's any consolation (and I sincerely hope it is), here's how things went for me last July. Of course, I also recognize every state is different/many of you took the UBE so there is not necessarily any overlap on our experiences beyond the MBE...

New York - Passed despite having:

1) Rushed through the state multiple choice (50 questions) because I did not have adequate time to read many of them, and this included a lot of guesswork.

2) Been very unsure about a handful of the essay sub-prompts.

3) Essentially, though not entirely, skipping the MPT (I wrote out an overall format/outline with intro, headings, and vague conclusion before panicking for nearly 40 minutes over what to write and how, ultimately foraying any chance I had to write something of substance).

New Jersey - Passed despite having:

1) Writing nothing more than a basic "intro" to the essay, i.e. "Dear Client, you have asked me to consider ____ and ____; here are my findings." I had a complete breakdown over the question, had no idea what to write, found myself questioning everything I knew, and ultimately moved on without writing anything more than some generic legal conclusions so as not to leave the question entirely blank.

2) Completely broken down in my car during the break--this entailed rocking myself back and forth, trying not to lose my shit entirely and cry like a baby for ~45 minutes, and convincing myself I was an utter failure for feeling how I felt/the essay mishap.

3) Shotgun approaching the last essay out of desperation, lack of time, and overall anxiety from the morning portion.

New York/New Jersey, Collectively - I was convinced beyond a reasonable doubt (don't hate me) I had done terribly on the MBE and it was going to destroy my chances on either test--especially in light of the above mishaps. My score was pleasantly surprising, and played a large role (I think) in my passing both tests.

I'm not here to give anyone false hope. Just because it worked out for me doesn't mean it will work out for you. I understand that mindset. I agree with that mindset. However, I just want those reading this (and every other similar post/rant by a co-taker/anything else you're reading online, and will continue to read while you wait 5-?? weeks for your results) to know that how you felt and feel are not conclusive of whether or not you passed/failed (one, or both tests for those of you that took two).

For what it's worth, my best advice now is to keep moving forward with your life and plans. Keep looking for work if you're still searching, and go into your clerkships and jobs if you have them feeling confident that things may very well go smoothly and all of the stress and panic was for nothing.

Re: The "Ask @JoeSeperac" Thread

Posted: Mon Oct 12, 2020 12:13 am
by aa19
Hi Joe.

Can you please help me out with a prediction?

For my review, I did 4806 MBE questions and came in around 68%. What does that translate to in terms of MBE score for an exam?

Is this an accurate score? I did so many questions and I saw some of the questions more than once and I knew what legal principal to apply for a quick answer. With that said, I am concerned that seeing the questions more than once somewhat inflated my score. Is that a valid concern to have ?

Re: The "Ask @JoeSeperac" Thread

Posted: Mon Oct 12, 2020 9:35 pm
by JoeSeperac
aa19 wrote:
Mon Oct 12, 2020 12:13 am
Hi Joe.

Can you please help me out with a prediction?

For my review, I did 4806 MBE questions and came in around 68%. What does that translate to in terms of MBE score for an exam?

Is this an accurate score? I did so many questions and I saw some of the questions more than once and I knew what legal principal to apply for a quick answer. With that said, I am concerned that seeing the questions more than once somewhat inflated my score. Is that a valid concern to have ?

Yes and no. The familiarity isn't a bad thing if you can generally explain for a question why the right answer is the right answer and why the wrong answers are wrong. Can you fill out my UBE Score Estimator and break down your source(s) for the MBE questions (inlcude your email). I can then give you a better estimate:

http://www.seperac.com/zcalc-passcalc.php

Re: The "Ask @JoeSeperac" Thread

Posted: Tue Oct 13, 2020 4:06 pm
by Anonymous User
Hi Joe,

I took the July 20 Oregon Bar exam and scored a 316.

MBE: 159
MEE: 157

Could you please let me know where this falls nationally/what my raw MBE score is?

Thanks!

Re: The "Ask @JoeSeperac" Thread

Posted: Tue Oct 13, 2020 9:14 pm
by JoeSeperac
Anonymous User wrote:
Tue Oct 13, 2020 4:06 pm
Hi Joe,

I took the July 20 Oregon Bar exam and scored a 316.

MBE: 159
MEE: 157

Could you please let me know where this falls nationally/what my raw MBE score is?

Thanks!

Congratulations on passing the J20 OR bar exam. Based on your scaled MBE score of 159, your estimated raw MBE score was about 141/175 correct (based on the last time an MBE scale was released in 2013). This means you answered about 80.6% of the graded MBE questions correctly. This places you in the 84.9% percentile for the MBE. This means that 15.1% of Jul examinees nationwide did better than you on the MBE based on your scaled MBE score of 159 (based on national data for the past 7 years). Based on a total score of 316, your written score was 157, which would have placed you in the 81.6% percentile among examinees nationwide (meaning that 18.4% of examinees nationwide would have scored better than you on the MEE/MPT). Although NCBE does not release percentiles for total UBE scores, based on the MBE and written percentiles, this would place you in the 83.3% percentile among examinees nationwide based on your total score of 316 (meaning that 16.8% of examinees nationwide scored better than you on the UBE). Please keep in mind this is just an estimate and may be incorrect.

If you don't mind, how many questions did you answer in practice, from what sources (e.g. Barbri, Kaplan, Adaptibar, NCBE) and what was your overall % correct?

Re: The "Ask @JoeSeperac" Thread

Posted: Thu Oct 15, 2020 6:27 pm
by kinglew
Hi Joe

Any idea or predictions on how the remote exams (100 MCQ 3 MEE 1 MPTs) will be scaled/graded? do you think states like IL will try to equate the score to July ube averages?

Re: The "Ask @JoeSeperac" Thread

Posted: Fri Oct 16, 2020 4:52 pm
by Anonymous User
kinglew wrote:
Thu Oct 15, 2020 6:27 pm
Hi Joe

Any idea or predictions on how the remote exams (100 MCQ 3 MEE 1 MPTs) will be scaled/graded? do you think states like IL will try to equate the score to July ube averages?
And when will scores be released?

Re: The "Ask @JoeSeperac" Thread

Posted: Fri Oct 16, 2020 8:42 pm
by JoeSeperac
kinglew wrote:
Thu Oct 15, 2020 6:27 pm
Hi Joe

Any idea or predictions on how the remote exams (100 MCQ 3 MEE 1 MPTs) will be scaled/graded? do you think states like IL will try to equate the score to July ube averages?

I haven't a clue and I am guessing the bar examiners aren't looking forward to it due to its unreliability.

First, a quick primer on reliability. Reliability is the consistency or repeatability of the scores produced by a measurement procedure; the precision in the scores yielded by a measurement instrument. Reliability means obtaining consistent results, regardless of what version of the test is given. Essentially, if a group were tested again using different questions or if the examinations were scored again using different graders, the new rank order of examinees would be very similar to the first one. Perfect reliability is when you administer two completely different tests covering the same topics and an examinee’s scores on the two tests are identical (this of course is impossible). Reliability can also be interpreted as a correlation coefficient, with values between 0.0 and 1.0. The higher the coefficient (to a maximum of 1.0), the stronger the relationship. Higher values for reliability reflect greater precision and less random error, and low values for reliability reflect a higher proportion of random error and therefore less precision. According to NCBE, "using a variety of techniques, NCBE conducts ongoing evaluations of the quality of the questions on the bar exam. For each test, we evaluate the reliability of the scores; reliability is an essential component of validity."

The industry standard is that any score used for decision making should have a reliability of at least 0.90. The 6-hour MBE has a reliability of 0.9 according to NCBE (lately it has gone up to 0.92). In contrast, NCBE found that for the essays to have a reliability of 0.90, they needed to be 16 hours long with 32 different essay questions. NCBE found that for the MPT to have a reliability of 0.90, it needed to be 33 hours long with 22 different MPT items. see The Bar Examiner: Volume 77, Number 3, August 2008.

In the past, NCBE scaled the less-reliable written to the reliable MBE to improve the overall reliability of the total score. However, a half-MBE is problematic. Based on studies, longer tests yield more reliable scores than shorter tests. In a 2004 paper on the MBE, the author stated that “The coefficient alpha measure of reliability for each section (100 items) was .80. Using the Spearman-Brown formula, the reliability of the entire test (200 items) would be estimated at .89 and is consistent with other licensure examinations that cover a broad domain.” See http://journals.sagepub.com/doi/abs/10. ... 4405282483

This means the half-MBE is unreliable and doesn’t meet the industry standard for a licensure exam. Even worse, imagine how reliable only 3 MEE and 1 MPT are. This concerns me because reliability is absolutely crucial to high stakes exams such as bar exams. The less reliable the exam, the more likely someone who should have passed will fail and vice versa.

Re: The "Ask @JoeSeperac" Thread

Posted: Fri Oct 16, 2020 9:03 pm
by JoeSeperac
Anonymous User wrote:
Fri Oct 16, 2020 4:52 pm
kinglew wrote:
Thu Oct 15, 2020 6:27 pm
Hi Joe

Any idea or predictions on how the remote exams (100 MCQ 3 MEE 1 MPTs) will be scaled/graded? do you think states like IL will try to equate the score to July ube averages?
And when will scores be released?

You ask an interesting question. It has taken an average of 88 days for NY to release July results (based on data from 2016-2019). Since there are half as many written items to grade and fewer examinees, it is arguable the results could be released within 30-40 days, assuming all the graders who are available over the summer are likewise available in October. Also, there has been a trend in NY to release results earlier and earlier. For example, results were released last year in NY on Oct 23, 2019, the earliest NY ever released July results. If I had to wildly guess, I would say Nov 6.

Re: The "Ask @JoeSeperac" Thread

Posted: Sun Oct 18, 2020 6:20 pm
by Anonymous User
I took the UBE in a state that requires a 260 to pass and publishes a "pass" list but physically mails you your score. I was on the pass list, thank goodness, but I need to transfer my score to a state that requires a 270 and won't know for several days if my score is high enough. I'm going crazy. What's the likelihood of someone getting a 260 but not a 270?

Re: The "Ask @JoeSeperac" Thread

Posted: Wed Oct 21, 2020 11:33 pm
by JoeSeperac
Anonymous User wrote:
Sun Oct 18, 2020 6:20 pm
I took the UBE in a state that requires a 260 to pass and publishes a "pass" list but physically mails you your score. I was on the pass list, thank goodness, but I need to transfer my score to a state that requires a 270 and won't know for several days if my score is high enough. I'm going crazy. What's the likelihood of someone getting a 260 but not a 270?

Congrats. Based on a 2005 study in New York, first-time domestic educated candidates scored an average total score of 291 so there is a good chance. See http://www.nybarexam.org/press/ncberep.pdf

Re: The "Ask @JoeSeperac" Thread

Posted: Thu Oct 22, 2020 4:28 pm
by blair.waldorf
Joe, your predicted score for me was basically spot on. Thank you for all you do. I thought I may have failed and your predictor brought me a lot of comfort.

Re: The "Ask @JoeSeperac" Thread

Posted: Fri Oct 23, 2020 12:20 am
by Anonymous User
JoeSeperac wrote:
Fri Oct 16, 2020 8:42 pm
kinglew wrote:
Thu Oct 15, 2020 6:27 pm
Hi Joe

Any idea or predictions on how the remote exams (100 MCQ 3 MEE 1 MPTs) will be scaled/graded? do you think states like IL will try to equate the score to July ube averages?

I haven't a clue and I am guessing the bar examiners aren't looking forward to it due to its unreliability.

First, a quick primer on reliability. Reliability is the consistency or repeatability of the scores produced by a measurement procedure; the precision in the scores yielded by a measurement instrument. Reliability means obtaining consistent results, regardless of what version of the test is given. Essentially, if a group were tested again using different questions or if the examinations were scored again using different graders, the new rank order of examinees would be very similar to the first one. Perfect reliability is when you administer two completely different tests covering the same topics and an examinee’s scores on the two tests are identical (this of course is impossible). Reliability can also be interpreted as a correlation coefficient, with values between 0.0 and 1.0. The higher the coefficient (to a maximum of 1.0), the stronger the relationship. Higher values for reliability reflect greater precision and less random error, and low values for reliability reflect a higher proportion of random error and therefore less precision. According to NCBE, "using a variety of techniques, NCBE conducts ongoing evaluations of the quality of the questions on the bar exam. For each test, we evaluate the reliability of the scores; reliability is an essential component of validity."

The industry standard is that any score used for decision making should have a reliability of at least 0.90. The 6-hour MBE has a reliability of 0.9 according to NCBE (lately it has gone up to 0.92). In contrast, NCBE found that for the essays to have a reliability of 0.90, they needed to be 16 hours long with 32 different essay questions. NCBE found that for the MPT to have a reliability of 0.90, it needed to be 33 hours long with 22 different MPT items. see The Bar Examiner: Volume 77, Number 3, August 2008.

In the past, NCBE scaled the less-reliable written to the reliable MBE to improve the overall reliability of the total score. However, a half-MBE is problematic. Based on studies, longer tests yield more reliable scores than shorter tests. In a 2004 paper on the MBE, the author stated that “The coefficient alpha measure of reliability for each section (100 items) was .80. Using the Spearman-Brown formula, the reliability of the entire test (200 items) would be estimated at .89 and is consistent with other licensure examinations that cover a broad domain.” See http://journals.sagepub.com/doi/abs/10. ... 4405282483

This means the half-MBE is unreliable and doesn’t meet the industry standard for a licensure exam. Even worse, imagine how reliable only 3 MEE and 1 MPT are. This concerns me because reliability is absolutely crucial to high stakes exams such as bar exams. The less reliable the exam, the more likely someone who should have passed will fail and vice versa.
Based off your predictor, it says that I passed with 40+ points to spare. How does this coincide with me taking the 100 MCQ. 3 MEE, 1 MPT? does that apply to the remote exam as well (in regards to PASSING not necessarily the ube score)

Re: The "Ask @JoeSeperac" Thread

Posted: Fri Oct 23, 2020 11:08 am
by JoeSeperac
Based off your predictor, it says that I passed with 40+ points to spare. How does this coincide with me taking the 100 MCQ. 3 MEE, 1 MPT? does that apply to the remote exam as well (in regards to PASSING not necessarily the ube score)

With an "abnormal" exam, I can't say how effective the score prediction will be, but I would expect it to be less reliable if the "abnormal" exam is less reliable. It's impossible to know because the states release so little about their methodologies. For example, I'm aware of one examinee who sued the North Dakota Board of Law Examiners and sought discovery, requesting information about grading techniques, how raw scores are converted, whether the board performs periodic assessments of its scores, whether a report is available of psychometric procedures, how the essay scoring judges are trained, and whether a content analysis has been conducted. The North Dakota Supreme Court ruled that these records were confidential. His petition for writ of certiorari to the Supreme Court of North Dakota was denied by the U.S. Supreme Court. The examinee later passed and didn't pursue it any further, but this illustrates the difficulty one faces in trying to get the bar examiners to explain anything about their methodologies. When I try to figure these things out, I often feel like I am walking around in the dark, reaching around for the light switch.

Re: The "Ask @JoeSeperac" Thread

Posted: Mon Oct 26, 2020 2:47 am
by Anonymous User
JoeSeperac wrote:
Fri Oct 23, 2020 11:08 am
Based off your predictor, it says that I passed with 40+ points to spare. How does this coincide with me taking the 100 MCQ. 3 MEE, 1 MPT? does that apply to the remote exam as well (in regards to PASSING not necessarily the ube score)

With an "abnormal" exam, I can't say how effective the score prediction will be, but I would expect it to be less reliable if the "abnormal" exam is less reliable. It's impossible to know because the states release so little about their methodologies. For example, I'm aware of one examinee who sued the North Dakota Board of Law Examiners and sought discovery, requesting information about grading techniques, how raw scores are converted, whether the board performs periodic assessments of its scores, whether a report is available of psychometric procedures, how the essay scoring judges are trained, and whether a content analysis has been conducted. The North Dakota Supreme Court ruled that these records were confidential. His petition for writ of certiorari to the Supreme Court of North Dakota was denied by the U.S. Supreme Court. The examinee later passed and didn't pursue it any further, but this illustrates the difficulty one faces in trying to get the bar examiners to explain anything about their methodologies. When I try to figure these things out, I often feel like I am walking around in the dark, reaching around for the light switch.
Apparently remote exam states have hired a psychometrician to normalize the results to mirror previous year passage rate. Even so, should I expect to take your predictor with reliability than if this was a normal UBE exam?

Re: The "Ask @JoeSeperac" Thread

Posted: Tue Oct 27, 2020 7:00 pm
by JoeSeperac
Anonymous User wrote:
Mon Oct 26, 2020 2:47 am
Apparently remote exam states have hired a psychometrician to normalize the results to mirror previous year passage rate. Even so, should I expect to take your predictor with reliability than if this was a normal UBE exam?

Yes, until the bar examiners go back to a "normal" exam, I would regard the predictor as less reliable. Right now, some F21 exams will be online (e.g. NY, PA) while others will be in person. I expect the MBE questions used in the online exam to be different and inferior to the MBE questions used in the in-person exam, making them less reliable. This is because the online questions will likely be discarded after the exam due to the risk of being compromised (you know someone somewhere is going to record the screen somehow).

Disclaimer: this is just my opinion, not gospel.

Re: The "Ask @JoeSeperac" Thread

Posted: Mon Nov 16, 2020 12:46 pm
by Anonymous User
Hi Joe,

I took the July 20 UBE.

UBE: 324
MBE: 166.7

Can you predict my national percentile and MBE raw score?

Re: The "Ask @JoeSeperac" Thread

Posted: Wed Nov 18, 2020 5:37 pm
by JoeSeperac
Anonymous User wrote:
Mon Nov 16, 2020 12:46 pm
Hi Joe,

I took the July 20 UBE.

UBE: 324
MBE: 166.7

Can you predict my national percentile and MBE raw score?

Sorry for the delayed follow up. Based on your scaled MBE score of 166.7, your estimated raw MBE score was about 87% correct (based on the last time an MBE scale was released in 2013). This places you in the 94.5% percentile for the MBE. This means that 5.5% of Jul examinees nationwide did better than you on the MBE based on your scaled MBE score of 166.7 (based on national data for the past 7 years).

Based on a total score of 324, your written score was 157.3, which would have placed you in the 82.1% percentile among examinees nationwide (meaning that 17.9% of examinees nationwide would have scored better than you on the MEE/MPT).

Although NCBE does not release percentiles for total UBE scores, based on the MBE and written percentiles, this would place you in the 88.3% percentile among examinees nationwide based on your total score of 324 (meaning that 11.7% of examinees nationwide scored better than you on the UBE). Please keep in mind this is just an estimate and may be incorrect.

Congrats!