Is Law School Grading Arbitrary? Forum

[cinephile]

It is easily proven to not be arbitrary, by the fact that you can see clear patterns in students' scores across classes.

What? Do you have something to back this up? Almost everyone I know has a mix of As and Bs and everything in between with no coherent pattern.

[dingbat]

That being said, I don't think grades are a strong indicator of how well you actually knew the material, or how "smart" you are.

It's not about knowing the material, it's about understanding the concepts and being able to figure out how they apply.

Having been on the client side (corporate/transactional), we're not paying hundreds of dollars an hour just for someone to regurgitate the law - we could look that up ourselves (and sometimes did). What we paid for was for someone to take our specific fact pattern and figure out the most likely outcomes (as well as the likelihood of those outcomes), and, if needed, to recommend reasonable and feasible changes to our fact patterns to ensure the result we want.

[dingbat]

How predictive do you guys think law school grades are of future lawyerly success? Do those exams in law school encapsulate all that is essential to being a good lawyer?

depends on what kind of law you want to practice.
If all you care about is assisting first-time home buyers in closing, then you don't need to be able to perform the kind of analysis involved in international tax structuring,m which again is a different skill-set vis-a-vis prosecuting violent crime

[lesananas]

That being said, I don't think grades are a strong indicator of how well you actually knew the material, or how "smart" you are.

It's not about knowing the material, it's about understanding the concepts and being able to figure out how they apply.

Having been on the client side (corporate/transactional), we're not paying hundreds of dollars an hour just for someone to regurgitate the law - we could look that up ourselves (and sometimes did). What we paid for was for someone to take our specific fact pattern and figure out the most likely outcomes (as well as the likelihood of those outcomes), and, if needed, to recommend reasonable and feasible changes to our fact patterns to ensure the result we want.

Having also worked at a firm, in finance/transactional, I understand that. My point was never trying to address what makes someone a good lawyer, rather that there are many thing that hold people back on law school exams that are less relevant to practice. For instance, I am a type-A perfectionist (who isn't, i know) and literally could NOT stop myself from correcting typos, re-writing sentences to sound better, etc. On an exams this sunk me, as I didn't finish in time. In practice, if I didn't correct typos, I'd be in a whole lot of trouble. Granted, if you get a B-/C then maybe you actually did have trouble applying law to facts and that will be an issue going forward. But what separated me from bottom 10% one year to honors the next had nothing to do with understanding the law OR applying it well. It had to do with time allocation during an exam. That was what I was trying to get across.

edit: as a disclaimer, I go to a T-14 school were pretty much everyone is really smart and understands the material. which is why things like typing speed can actually make a difference. I'm not sure what I'm saying applies to, for example, an unranked law school.

[dingbat]

That being said, I don't think grades are a strong indicator of how well you actually knew the material, or how "smart" you are.

It's not about knowing the material, it's about understanding the concepts and being able to figure out how they apply.

Having been on the client side (corporate/transactional), we're not paying hundreds of dollars an hour just for someone to regurgitate the law - we could look that up ourselves (and sometimes did). What we paid for was for someone to take our specific fact pattern and figure out the most likely outcomes (as well as the likelihood of those outcomes), and, if needed, to recommend reasonable and feasible changes to our fact patterns to ensure the result we want.

Having also worked at a firm, in finance/transactional, I understand that. My point was never trying to address what makes someone a good lawyer, rather that there are many thing that hold people back on law school exams that are less relevant to practice. For instance, I am a type-A perfectionist (who isn't, i know) and literally could NOT stop myself from correcting typos, re-writing sentences to sound better, etc. On an exams this sunk me, as I didn't finish in time. In practice, if I didn't correct typos, I'd be in a whole lot of trouble. Granted, if you get a B-/C then maybe you actually did have trouble applying law to facts and that will be an issue going forward. But what separated me from bottom 10% one year to honors the next had nothing to do with understanding the law OR applying it well. It had to do with time allocation during an exam. That was what I was trying to get across.

so time management is not a vital skill for someone being billed at hundreds of dollars per hour, billed in short increments?

[lesananas]

That being said, I don't think grades are a strong indicator of how well you actually knew the material, or how "smart" you are.

It's not about knowing the material, it's about understanding the concepts and being able to figure out how they apply.

Having been on the client side (corporate/transactional), we're not paying hundreds of dollars an hour just for someone to regurgitate the law - we could look that up ourselves (and sometimes did). What we paid for was for someone to take our specific fact pattern and figure out the most likely outcomes (as well as the likelihood of those outcomes), and, if needed, to recommend reasonable and feasible changes to our fact patterns to ensure the result we want.

Having also worked at a firm, in finance/transactional, I understand that. My point was never trying to address what makes someone a good lawyer, rather that there are many thing that hold people back on law school exams that are less relevant to practice. For instance, I am a type-A perfectionist (who isn't, i know) and literally could NOT stop myself from correcting typos, re-writing sentences to sound better, etc. On an exams this sunk me, as I didn't finish in time. In practice, if I didn't correct typos, I'd be in a whole lot of trouble. Granted, if you get a B-/C then maybe you actually did have trouble applying law to facts and that will be an issue going forward. But what separated me from bottom 10% one year to honors the next had nothing to do with understanding the law OR applying it well. It had to do with time allocation during an exam. That was what I was trying to get across.

so time management is not a vital skill for someone being billed at hundreds of dollars per hour, billed in short increments?

are you trying to be an asshole? Of course it is. I NEVER had time management problems in work. We're talking about a 3 hour exam, your first year of law school, when you've never been in a situation like that before, and realizing a little too late (i.e. when the instructor called time) that you probably should have typed faster. You're acting like because I didn't want to spit out misspelled word vomit on a test, I would willingly spend too much time on a project or piss away a client's money.

[dingbat]

We're talking about a 3 hour exam, your first year of law school, when you've never been in a situation like that before, and realizing a little too late (i.e. when the instructor called time) that you probably should have typed faster.

you never had a timed exam before?

I don't mean to be an asshole, but, I'm not sure where the problem lies.
Yes, it sucks that your entire grade is based on one exam, and yes there are many things that might hold people back on an exam, but, a lot of those same issues could also hold someone back in real life, and screwing up on a single matter can get you fired, sued for malpractice, or disbarred if you're a practicing attorney. Law school exams actually give you a break in one respect which is that typos don't matter half as much

[rayiner]

How predictive do you guys think law school grades are of future lawyerly success? Do those exams in law school encapsulate all that is essential to being a good lawyer?

If not, then are they arbitrary ways of measuring students' aptitude for legal work? ...maybe arbitrary isn't the best word here for this specific question, but more like are the exams "inept" for the task?

So, here, I'm no longer asking whether or not the grading and exam outcomes are arbitrary (in the traditional sense of the word), but rather whether these exams are good predictors of future lawyerly success? One would think that with such high stakes that the exams ought to be!

I suppose it has some, weak, significance. Fred Bartlit still touts on his firm bio that he had the "top academic record in the history of the college of law" at U of I. John Paul Stevens had the highest GPA in the history of Northwestern.

I don't think it's necessarily that law exams measure anything particularly important, but rather that certain skills that are helpful on legal exams (being able to break down a problem quickly, being able to game the system, being able to figure out what the professor wants to hear) are also helpful in being a lawyer.

[lesananas]

We're talking about a 3 hour exam, your first year of law school, when you've never been in a situation like that before, and realizing a little too late (i.e. when the instructor called time) that you probably should have typed faster.

you never had a timed exam before?

I don't mean to be an asshole, but, I'm not sure where the problem lies.
Yes, it sucks that your entire grade is based on one exam, and yes there are many things that might hold people back on an exam, but, a lot of those same issues could also hold someone back in real life, and screwing up on a single matter can get you fired, sued for malpractice, or disbarred if you're a practicing attorney. Law school exams actually give you a break in one respect which is that typos don't matter half as much

Fair enough. I'm not saying grades aren't potentially indicative of certain skills. I just think you're extrapolating too much from one semester of bad grades as a 1L. Often that's all people have to go on, so its understandable to use them. But I just can't agree that you can judge a person's overall skill set from such a small representation, or that bad grades = bad lawyer. You're treating me like I have some personal deficiency because I've gotten a few Bs before. Not to mention, how do you explain people who get the whole range of grades in one semester? If exam performance was truly indicative of how well they could apply facts to law, or their overall time-management skills, or their attention to detail, wouldn't grades be more consistant?

[dingbat]

Fair enough. I'm not saying grades aren't potentially indicative of certain skills. I just think you're extrapolating too much from one semester of bad grades as a 1L. Often that's all people have to go on, so its understandable to use them. But I just can't agree that you can judge a person's overall skill set from such a small representation, or that bad grades = bad lawyer. You're treating me like I have some personal deficiency because I've gotten a few Bs before. Not to mention, how do you explain people who get the whole range of grades in one semester? If exam performance was truly indicative of how well they could apply facts to law, or their overall time-management skills, or their attention to detail, wouldn't grades be more consistant?

nah, I was just counter-arguing your point because it was incomplete.
exam performance isn't entirely correlated of how good an attorney one will become, but it is somewhat indicative - it has some correlation. More importantly, it's indicative how you compare to the rest of your class, not how you compare against all other lawyers. I strongly suspect that someone in the bottom 10-25% range at Columbia is as good as someone in the top 10-25% range at CUNY, although that's just guesswork.

[ksllaw]

What about subjectivity in grading?

In undergraduate, it was commonly understood that in non-STEM courses there was some room for subjectivity in grading. Two profs. might grade the same English lit. paper differently.

...It wasn't like in math, physics, bio, engineering, etc., where you could either solve that differential equation or not. So, e.g., if two people both solved everything on their math exams correctly, then they could both get the 100. That's because there IS a universally acknowledged correct answer to those math q's.

But what about with law exams? Is there ONE correct answer you can give and everything else is incorrect? Or, does one have to argue a case for a particular position in which there is no universally agree upon answer? And if so, does subjectivity then creep into the grading process?

[ph14]

What about subjectivity in grading?

In undergraduate, it was commonly understood that in non-STEM courses there was some room for subjectivity in grading. Two profs. might grade the same English lit. paper differently.

...It wasn't like in math, physics, bio, engineering, etc., where you could either solve that differential equation or not. So, e.g., if two people both solved everything on their math exams correctly, then they could both get the 100. That's because there IS a universally acknowledged correct answer to those math q's.

But what about with law exams? Is there ONE correct answer you can give and everything else is incorrect? Or, does one have to argue a case for a particular position in which there is no universally agree upon answer? And if so, does subjectivity then creep into the grading process?

Generally, exams are written so that there are some correct answers, but mainly ambiguous issues that could come out either way. There is some subjectivity (i.e., your professor might have a different view on it than another professor, etc.), but a professor can differentiate. Although say on a policy question, there is a bit more room for subjectivity.

[ben4847]

It is easily proven to not be arbitrary, by the fact that you can see clear patterns in students' scores across classes.

What? Do you have something to back this up? Almost everyone I know has a mix of As and Bs and everything in between with no coherent pattern.

Yes. All the people who have straight A's. All the people who have straight B-'s. And all the people who have about a 3.3 after 1L and then approximately the same in 2L, and in 3L.

Sure, to a student who has 2 A-'s, 2 B+'s, and a B in every semester, it can seem like it is random. But when you compare that student's pretty consistent record of that rough proportion to other students with other pretty consistent records, it begins to look more orderly.

[rayiner]

What about subjectivity in grading?

In undergraduate, it was commonly understood that in non-STEM courses there was some room for subjectivity in grading. Two profs. might grade the same English lit. paper differently.

...It wasn't like in math, physics, bio, engineering, etc., where you could either solve that differential equation or not. So, e.g., if two people both solved everything on their math exams correctly, then they could both get the 100. That's because there IS a universally acknowledged correct answer to those math q's.

But what about with law exams? Is there ONE correct answer you can give and everything else is incorrect? Or, does one have to argue a case for a particular position in which there is no universally agree upon answer? And if so, does subjectivity then creep into the grading process?

So the law can in most cases be broken down into rules that are implicated in particular situations (issues). The rules are broken down into elements. Elements may be governed by rules which themselves have elements. E.g. a contract is formed if there is: 1) an offer; 2) an acceptance; 3) consideration.

The fact pattern will implicate numerous issues. The issues are governed by rules. Specific facts in the fact pattern will be relevant to elements of the rules. So a fact pattern might mention that Sally accepted the offer by phone after Mary had mailed a revocation of the offer that had not yet been received by Sally. These facts "plug in" to the rule that governs whether contracts are formed.

The key to understanding how law exams are graded is to realize that the professor doesn't give a shit what your position is on each issue. He doesn't care whether you think the contract was really formed or not. What he wants to see is the analysis. He wants to see you recognize the issue, state the proper rule, and plug the relevant facts from the fact pattern into the elements of the rule. E.g. the fact that Mary mailed the revocation is relevant to the "offer" element of the rule, and the fact that Sally accepted by phone is relevant to the "acceptance" element of the rule. Neither are relevant to the "consideration" element of the rule.

How the professor grades is to give you a point for recognizing the issue, one for stating the rule, and one for each time you apply the relevant fact to the relevant element of the proper rule. If a relevant fact could cause an element to go one way or the other, then you "argue both sides." Your final score is just based on how many points you rack up.

Law exams are almost always written so the issues cannot be clearly resolved. But the process is quite objective because you're not getting points based on resolving the issue, you're getting points for hitting the elements with relevant facts. There is a little bit of subjectivity in how the professor evaluates how well you apply the relevant fact to the relevant elements, but professors do not try to make fine distinctions. There might just be a couple of points for each element: you get the maximum for a really good application that clearly captures the subtleties of the element, and get zero points for not even realizing what facts are relevant to the element.

This is very different from an undergraduate exam where a professor is evaluating your essay holistically based on how well you support a position. You get a good grade simply by hitting as many relevant points as possible with credible applications of the relevant elements to the relevant facts from the fact pattern.

[rayiner]

It is easily proven to not be arbitrary, by the fact that you can see clear patterns in students' scores across classes.

What? Do you have something to back this up? Almost everyone I know has a mix of As and Bs and everything in between with no coherent pattern.

It's mathematic fact. If there was no consistent pattern in grades, the standard deviation of grades would approach zero. I.e. the top of the class and the bottom of the class would cluster very close to the curve median. But that's obviously not what happens.

E.g. at NU, top 20% is around 3.7 after 1L, while median is around 3.4. While it may be true that top 20% people and median people all have a mix of A/A-/B+/B, it's simply mathematical fact that the people in the top 20% are unlikely to have many B+/B while people in the median are.

[cinephile]

It is easily proven to not be arbitrary, by the fact that you can see clear patterns in students' scores across classes.

What? Do you have something to back this up? Almost everyone I know has a mix of As and Bs and everything in between with no coherent pattern.

It's mathematic fact. If there was no consistent pattern in grades, the standard deviation of grades would approach zero. I.e. the top of the class and the bottom of the class would cluster very close to the curve median. But that's obviously not what happens.

E.g. at NU, top 20% is around 3.7 after 1L, while median is around 3.4. While it may be true that top 20% people and median people all have a mix of A/A-/B+/B, it's simply mathematical fact that the people in the top 20% are unlikely to have many B+/B while people in the median are.

Right, at the top and bottom of the curve there's consistency. But at median? How many people have straight B+s and how many are like A/A-/B+/B and it all evens out?

[Hutz_and_Goodman]

Another approach to considering the arbitrariness of grades is to see how people who attend a given school with a given scholarship level tend to do. In other words, if you took NYU and broke the class down into five segments: those who received 0-$30k; those who received $31-60k; those who received $60-90k; those who received $90k-120k; and those who received $120k+, what chance would someone in a given segment have of being in the top 10%?

So for the proposition:
Should I attend school A with scholarship amount B, and if I do what is my chance of being top (10%/20%/30%)?

I think the best thing to do is to find people who have attended A with scholarship amount B (or close to it), and ask them how they did.

[ben4847]

It is easily proven to not be arbitrary, by the fact that you can see clear patterns in students' scores across classes.

What? Do you have something to back this up? Almost everyone I know has a mix of As and Bs and everything in between with no coherent pattern.

It's mathematic fact. If there was no consistent pattern in grades, the standard deviation of grades would approach zero. I.e. the top of the class and the bottom of the class would cluster very close to the curve median. But that's obviously not what happens.

E.g. at NU, top 20% is around 3.7 after 1L, while median is around 3.4. While it may be true that top 20% people and median people all have a mix of A/A-/B+/B, it's simply mathematical fact that the people in the top 20% are unlikely to have many B+/B while people in the median are.

Right, at the top and bottom of the curve there's consistency. But at median? How many people have straight B+s and how many are like A/A-/B+/B and it all evens out?

I think the top and bottom prove it isn't arbitrary, but even those of whom you speak tend to pretty consistently be around median in most semesters.

[BruceWayne]

I think people are looking at this in the wrong way. Those who are saying it's arbitrary think that if it isn't that would mean that the people at the median and the bottom aren't smart while those at the top are and don't want to believe that. And a lot of those who think that it isn't feel the same way but have no problem touting that it does mean that (lol).

In reality the grading definitely isn't arbitrary. The problem is that most people don't know what the grading is based off of. i.e it feels random because people are basically taking a test but in an odd sort of way (outside of obviously, a certain base level of legal knowledge) they don't know what they are being tested for.

[IAFG]

Another approach to considering the arbitrariness of grades is to see how people who attend a given school with a given scholarship level tend to do. In other words, if you took NYU and broke the class down into five segments: those who received 0-$30k; those who received $31-60k; those who received $60-90k; those who received $90k-120k; and those who received $120k+, what chance would someone in a given segment have of being in the top 10%?

So for the proposition:
Should I attend school A with scholarship amount B, and if I do what is my chance of being top (10%/20%/30%)?

I think the best thing to do is to find people who have attended A with scholarship amount B (or close to it), and ask them how they did.

I think you'd be underwhelmed by the predictive power of scholarship.

[ksllaw]

In reality the grading definitely isn't arbitrary. The problem is that most people don't know what the grading is based off of. i.e it feels random because people are basically taking a test but in an odd sort of way (outside of obviously, a certain base level of legal knowledge) they don't know what they are being tested for.

Hi batman
Hmmm, do you feel that's fair, given the stakes of law school?

To clarify, you're saying that obviously students know the general content of the exam (e.g., Ch. X - Y of Book Z), but don't know:

a.) the format (how questions will be asked or what will be required)
b.) how grading will be done

Correct?

I've had a.) come up in undergraduate courses before, where the professor wouldn't tell us the format of an exam and just said to come prepared "knowing the material."

[Hutz_and_Goodman]

Another approach to considering the arbitrariness of grades is to see how people who attend a given school with a given scholarship level tend to do. In other words, if you took NYU and broke the class down into five segments: those who received 0-$30k; those who received $31-60k; those who received $60-90k; those who received $90k-120k; and those who received $120k+, what chance would someone in a given segment have of being in the top 10%?

So for the proposition:
Should I attend school A with scholarship amount B, and if I do what is my chance of being top (10%/20%/30%)?

I think the best thing to do is to find people who have attended A with scholarship amount B (or close to it), and ask them how they did.

I think you'd be underwhelmed by the predictive power of scholarship.

At my school I have a lot of information for the top five of the segments above for 2008-present, and there's a tremendous correlation between this segment of scholarship award and grades.

[rayiner]

It is easily proven to not be arbitrary, by the fact that you can see clear patterns in students' scores across classes.

What? Do you have something to back this up? Almost everyone I know has a mix of As and Bs and everything in between with no coherent pattern.

It's mathematic fact. If there was no consistent pattern in grades, the standard deviation of grades would approach zero. I.e. the top of the class and the bottom of the class would cluster very close to the curve median. But that's obviously not what happens.

E.g. at NU, top 20% is around 3.7 after 1L, while median is around 3.4. While it may be true that top 20% people and median people all have a mix of A/A-/B+/B, it's simply mathematical fact that the people in the top 20% are unlikely to have many B+/B while people in the median are.

Right, at the top and bottom of the curve there's consistency. But at median? How many people have straight B+s and how many are like A/A-/B+/B and it all evens out?

There tends to be less consistency between grades in the middle of the curve, but look at it this way: you're much more likely to see a median (= 3.4) transcript that's (A B+ B A- B+ B+ B+) than one that is (A A A B B B B).

[rayiner]

I think people are looking at this in the wrong way. Those who are saying it's arbitrary think that if it isn't that would mean that the people at the median and the bottom aren't smart while those at the top are and don't want to believe that. And a lot of those who think that it isn't feel the same way but have no problem touting that it does mean that (lol).

In reality the grading definitely isn't arbitrary. The problem is that most people don't know what the grading is based off of. i.e it feels random because people are basically taking a test but in an odd sort of way (outside of obviously, a certain base level of legal knowledge) they don't know what they are being tested for.

Right. The consistency I think mostly comes from the fact that everyone tries the same approach for all exams, and what works tends to work for every exam and what does't work tends to not work for every exam.

[dingbat]

Another approach to considering the arbitrariness of grades is to see how people who attend a given school with a given scholarship level tend to do. In other words, if you took NYU and broke the class down into five segments: those who received 0-$30k; those who received $31-60k; those who received $60-90k; those who received $90k-120k; and those who received $120k+, what chance would someone in a given segment have of being in the top 10%?

So for the proposition:
Should I attend school A with scholarship amount B, and if I do what is my chance of being top (10%/20%/30%)?

I think the best thing to do is to find people who have attended A with scholarship amount B (or close to it), and ask them how they did.

I think you'd be underwhelmed by the predictive power of scholarship.

At my school I have a lot of information for the top five of the segments above for 2008-present, and there's a tremendous correlation between this segment of scholarship award and grades.

This makes me very happy considering I took the money...