L’Étranger wrote:Anyone able to breakdown for me what the paragraph below actually means (preferably with examples)? --Thx.
"The Committee of Bar Examiners (Committee) utilizes a grading procedure designed to ensure the difficulty of passing the examination remains unchanged from one administration of the examination to another. The statistical technique, called scaling, converts scores on the written portion (essay questions and PTs) to the same scale of measurement as the MBE. MBE raw scores are converted to scale scores to adjust the results for possible differences in average question difficulty across different administrations of the examination. As a result of this step, a given MBE scale score indicates the same level of proficiency regardless of the administration of the examination on which it was earned. Converting the total written raw scores to the same scale of measurement as the MBE adjusts for possible differences in average question difficulty and Grader performance across different administrations of the examination."
The bar is a standardized test. Standardized
means, in my own words, that the results of each administration are easily interchangeable. So a score of 1440.0000 from 10 years ago is is theoretically the same level of performance as 1440.0000 from today. It's easier to compare the scores across each test.
However, the difficulty of each test varies. Graders vary over administrations and during the grading period and across essays. They can't make a test that's exactly the same difficulty, unless each test has the same questions and same conditions. So they need statisticians to scale the raw scores into scaled scores.
Raw scores are what the graders give you, e.g., 65 on your essay (or 125 on your MBE). You could say this score is standardized internally within the brotherhood of bar graders, particularly through their calibration process. A 65 given to one examinee's Essay 1 is theoretically the same level of performance as another examinee's Essay 1. I believe they calibrate for each essay, so a 65 on Essay 2 isn't necessarily as good or bad, which makes sense because there is no reliable way to compare two essays.
Scaled scores are what written raw scores are converted into. You can't look at your 125/200 raw MBE score and add or compare that to your 65. That's like trying to open a Word document with Paint.
It doesn't matter because whether a bar grader was high while grading your essay because like the law, we apply utilitarian assumptions: Using the miracle of statistical analysis, your total written raw score can be plugged into a post hoc calculated formula that is unique to each administration, based on difficulty, performance of examinees and/or what the graders had for breakfast. These contingencies are baked into the formula, and the output thereof is the scaled written score. Now it's compatible with your MBE score!
Now it's like opening a file with Adobe Reader—both documents containing words or images can be understood with one program (grossly simplified example, deal with it).
BTW, the formula looks suspiciously like, and is in fact, a simple linear equation (y = mx + b). Maybe they just gather all the scores and plot them and wiggle the graph around until they feel guilty about not letting enough people pass. But don't question them because they know more about statistics than either of us here.
You might notice a raw score differs by a large, lockstep amount in steps of 5 (or 2.5 if averaged--which is a gripe I'll save for another day), while a scaled score is more continuous and fluid to the several significant figures (i.e., lots of decimal numbers make it more accurate). This helps better compare the scaled scores across administrations.
Anyway, after that it is a simple matter of considering the weight of each section. Written is 65%, and MBE is 35%, so multiply each scaled score with the percentages and add them up to get a total scaled score that captures your total performance with 1 number rather than: "Here are 8 written scores and 1 MBE score."