Alex Johnson, UVA Professor discusses problems w/ LSAT Forum

[180kickflip]

Check out this report from LSAC. On page 24, it gives grade distribution curves for all the reported ethnicities for the 2011-2012 year (scaled score from 120-180 on the horizontal axis, and percentage of the group who achieved that score on the vertical axis).

The number of AA takers scoring 170+ is virtually zero, which agrees closely with what you'd expect with a mean of ~142 and a standard deviation of ~8.7.

There's a thread in the URM section that discusses this at some length. It's called "black by the numbers" or something. It seems generally accepted to everyone posting there that the total number of AA 170+ scorers is typically well under 20. I think this video (which may also be referenced in that black by the numbers thread and certainly has been discussed often in the URM section) is interesting as an example of what goes on, but absolutely not representative of the actual numbers.

[iamgeorgebush]

I'm all about LSAT being a better predictive measure than GPA, but I just wanted to add that the fact of grade inflation does not mean that GPA is worthless. There has been inflation of the dollar over time ($1 when my parents went were my age is equivalent to about $3 now), but that doesn't mean that dollars have absolutely no value and we should just rely on the gold standard.

[lawschool22]

I'm all about LSAT being a better predictive measure than GPA, but I just wanted to add that the fact of grade inflation does not mean that GPA is worthless. There has been inflation of the dollar over time ($1 when my parents went were my age is equivalent to about $3 now), but that doesn't mean that dollars have absolutely no value and we should just rely on the gold standard.

It's not inflation itself, but the fact it occurs at different rates at different schools, that makes the comparability aspect difficult. It's like saying, "oh I have 50 yen and 1 dollar, so my yen must be worth more."

ETA: Also, inflation shrinks the band of available GPAs. If you're only giving out 3.5-4.0 vs maybe 2.0-4.0, you have fewer discrete levels of differentiation that you can provide, so ranking students by GPA becomes more difficult, as GPAs begin to cluster. I'm not saying this is something the LSAT solves, since range restriction happens with it as well. But it is something caused by inflation that has worked to make GPAs less useful than they may have been in the past.

[zhenders]

Lame ancient factually ungrounded youtube video makes forum revive lame ancient factually ungrounded discussion.

This happened ITT.

[lawschool22]

Lame ancient factually ungrounded youtube video makes forum revive lame ancient factually ungrounded discussion.

This happened ITT.

There really was only one person posting factually ungrounded things. I wouldn't call the entire discussion factually ungrounded

[iamgeorgebush]

I'm all about LSAT being a better predictive measure than GPA, but I just wanted to add that the fact of grade inflation does not mean that GPA is worthless. There has been inflation of the dollar over time ($1 when my parents went were my age is equivalent to about $3 now), but that doesn't mean that dollars have absolutely no value and we should just rely on the gold standard.

It's not inflation itself, but the fact it occurs at different rates at different schools, that makes the comparability aspect difficult. It's like saying, "oh I have 50 yen and 1 dollar, so my yen must be worth more."

ETA: Also, inflation shrinks the band of available GPAs. If you're only giving out 3.5-4.0 vs maybe 2.0-4.0, you have fewer discrete levels of differentiation that you can provide, so ranking students by GPA becomes more difficult, as GPAs begin to cluster. I'm not saying this is something the LSAT solves, since range restriction happens with it as well. But it is something caused by inflation that has worked to make GPAs less useful than they may have been in the past.

On the first point, yes, the differing rates at different schools may be a problem, but isn't that just part of a larger problem, which is differing standards of evaluation at different schools? It seems to me like differing rates of grade inflation are only one part of that issue. It's like blaming the yen being worth less than the dollar on inflation, when the yen might have always been worth less than the dollar. It's not "inflation" that presents the issues; it's the difficulty of determining proper exchange rates, which is caused by myriad factors.

Definitely true on the last point; this is, IMO, the worst thing about grade inflation. When everyone's getting grades in the B- to A range, that doesn't leave much room for nuance.

[lawschool22]

I'm all about LSAT being a better predictive measure than GPA, but I just wanted to add that the fact of grade inflation does not mean that GPA is worthless. There has been inflation of the dollar over time ($1 when my parents went were my age is equivalent to about $3 now), but that doesn't mean that dollars have absolutely no value and we should just rely on the gold standard.

It's not inflation itself, but the fact it occurs at different rates at different schools, that makes the comparability aspect difficult. It's like saying, "oh I have 50 yen and 1 dollar, so my yen must be worth more."

ETA: Also, inflation shrinks the band of available GPAs. If you're only giving out 3.5-4.0 vs maybe 2.0-4.0, you have fewer discrete levels of differentiation that you can provide, so ranking students by GPA becomes more difficult, as GPAs begin to cluster. I'm not saying this is something the LSAT solves, since range restriction happens with it as well. But it is something caused by inflation that has worked to make GPAs less useful than they may have been in the past.

On the first point, yes, the differing rates at different schools may be a problem, but isn't that just part of a larger problem, which is differing standards of evaluation at different schools? It seems to me like differing rates of grade inflation are only one part of that issue. It's like blaming the yen being worth less than the dollar on inflation, when the yen might have always been worth less than the dollar. It's not "inflation" that presents the issues; the difficulty of pinpointing exchange rates.

Definitely true on the last point; this is, IMO, the worst thing about grade inflation. When everyone's getting grades in the B- to A range, that doesn't leave much room for nuance.

To your first point, it isn't really about pinpointing the exchange rate. GPA inflation has really progressed over time, and at different rates for different schools:



Here's a quote from the guy who did the study that produced that chart, Stuart Rojstaczer:

"In the 1930s, the average GPA at American colleges and universities was about 2.35, a number that corresponds with data compiled by W. Perry in 1943. By the 1950s, the average GPA was about 2.52. GPAs took off in the 1960s with grades at private schools rising faster than public schools, lulled in the 1970s, and began to rise again in the 1980s at a rate of about 0.10 to 0.15 increase in GPA per decade. The grade inflation that began in the 1980s has yet to end. . . . These trends may help explain why private school students are disproportionately represented in Ph.D. study in science and engineering and why they tend to dominate admission into the most prestigious professional schools."

[Kevinlomax]

I'm all about LSAT being a better predictive measure than GPA, but I just wanted to add that the fact of grade inflation does not mean that GPA is worthless. There has been inflation of the dollar over time ($1 when my parents went were my age is equivalent to about $3 now), but that doesn't mean that dollars have absolutely no value and we should just rely on the gold standard.

It's not inflation itself, but the fact it occurs at different rates at different schools, that makes the comparability aspect difficult. It's like saying, "oh I have 50 yen and 1 dollar, so my yen must be worth more."

ETA: Also, inflation shrinks the band of available GPAs. If you're only giving out 3.5-4.0 vs maybe 2.0-4.0, you have fewer discrete levels of differentiation that you can provide, so ranking students by GPA becomes more difficult, as GPAs begin to cluster. I'm not saying this is something the LSAT solves, since range restriction happens with it as well. But it is something caused by inflation that has worked to make GPAs less useful than they may have been in the past.

On the first point, yes, the differing rates at different schools may be a problem, but isn't that just part of a larger problem, which is differing standards of evaluation at different schools? It seems to me like differing rates of grade inflation are only one part of that issue. It's like blaming the yen being worth less than the dollar on inflation, when the yen might have always been worth less than the dollar. It's not "inflation" that presents the issues; the difficulty of pinpointing exchange rates.

Definitely true on the last point; this is, IMO, the worst thing about grade inflation. When everyone's getting grades in the B- to A range, that doesn't leave much room for nuance.

To your first point, it isn't really about pinpointing the exchange rate. GPA inflation has really progressed over time, and at different rates for different schools:



Here's a quote from the guy who did the study that produced that chart, Stuart Rojstaczer:

"In the 1930s, the average GPA at American colleges and universities was about 2.35, a number that corresponds with data compiled by W. Perry in 1943. By the 1950s, the average GPA was about 2.52. GPAs took off in the 1960s with grades at private schools rising faster than public schools, lulled in the 1970s, and began to rise again in the 1980s at a rate of about 0.10 to 0.15 increase in GPA per decade. The grade inflation that began in the 1980s has yet to end. . . . These trends may help explain why private school students are disproportionately represented in Ph.D. study in science and engineering and why they tend to dominate admission into the most prestigious professional schools."

this makes me so angry

[liammial]

hahaha

There really was only one person posting factually ungrounded things. I wouldn't call the entire discussion factually ungrounded

numerous strawmans-->no factual basis for anything you said-->passive aggressiveness

maybe take a debate class before you get to law school?

[lawschool22]

hahaha

There really was only one person posting factually ungrounded things. I wouldn't call the entire discussion factually ungrounded

numerous strawmans-->no factual basis for anything you said-->passive aggressiveness

maybe take a debate class before you get to law school?

Yep, debate class will be very helpful for my 1L exams I'm sure.

[iamgeorgebush]

I'm all about LSAT being a better predictive measure than GPA, but I just wanted to add that the fact of grade inflation does not mean that GPA is worthless. There has been inflation of the dollar over time ($1 when my parents went were my age is equivalent to about $3 now), but that doesn't mean that dollars have absolutely no value and we should just rely on the gold standard.

It's not inflation itself, but the fact it occurs at different rates at different schools, that makes the comparability aspect difficult. It's like saying, "oh I have 50 yen and 1 dollar, so my yen must be worth more."

ETA: Also, inflation shrinks the band of available GPAs. If you're only giving out 3.5-4.0 vs maybe 2.0-4.0, you have fewer discrete levels of differentiation that you can provide, so ranking students by GPA becomes more difficult, as GPAs begin to cluster. I'm not saying this is something the LSAT solves, since range restriction happens with it as well. But it is something caused by inflation that has worked to make GPAs less useful than they may have been in the past.

On the first point, yes, the differing rates at different schools may be a problem, but isn't that just part of a larger problem, which is differing standards of evaluation at different schools? It seems to me like differing rates of grade inflation are only one part of that issue. It's like blaming the yen being worth less than the dollar on inflation, when the yen might have always been worth less than the dollar. It's not "inflation" that presents the issues; the difficulty of pinpointing exchange rates.

Definitely true on the last point; this is, IMO, the worst thing about grade inflation. When everyone's getting grades in the B- to A range, that doesn't leave much room for nuance.

To your first point, it isn't really about pinpointing the exchange rate. GPA inflation has really progressed over time, and at different rates for different schools:



Here's a quote from the guy who did the study that produced that chart, Stuart Rojstaczer:

"In the 1930s, the average GPA at American colleges and universities was about 2.35, a number that corresponds with data compiled by W. Perry in 1943. By the 1950s, the average GPA was about 2.52. GPAs took off in the 1960s with grades at private schools rising faster than public schools, lulled in the 1970s, and began to rise again in the 1980s at a rate of about 0.10 to 0.15 increase in GPA per decade. The grade inflation that began in the 1980s has yet to end. . . . These trends may help explain why private school students are disproportionately represented in Ph.D. study in science and engineering and why they tend to dominate admission into the most prestigious professional schools."

The problem with this type of analysis is that it doesn't take into account changes in selectivity over time. One might be tempted to say, looking at that graph, that there is more grade inflation amongst private schools than amongst public schools. But it might just be that private schools' selectivity has increased at a higher rate than public schools' selectivity. (I have no idea whether this is the case; I just wanted to point out a possible alternate cause.)

That's why, IMO, if you wanted to gauge the ease of obtaining good grades at various different UGs, the best way to go about it would be to weigh each school's selectivity against its avg uGPA and their compare the schools with one another in order to create some "GPA exchange rate." Yes, it'd be complicated and imperfect, but it'd be a much more accurate measure than looking at trends over time. And it really cuts to the core of the matter.

[cron1834]

I don't have the studies on hand but the bias is actually often internal. For example, women who are asked their gender on a test before taking it tend to do worse than men whereas, when they are not asked this question, they perform identically to their male counterparts. The same is true for African Americans to white test takers. Asians often do better on standardized math tests if they are asked their race than their white counterparts, but perform equally well if that question is not asked. It's an interesting phenomenon. It's attributed to subconscious, societal biases. Essentially different groups sabotage themselves. I'll see if I can find that study; it was a fascinating read. We covered it extensively in a couple of my UG courses.

These differences-of-means in NO WAY approximate the differences between ethnic subgroups; it's just wildly misleading not to point out how marginal they are.

This is (tentatively) supported on the LSAT by the fact that test-takers who don't answer the demographic information tend to score higher overall than any other group.

This is also misleading when decontextualized. Without knowing the composition of the population, it's meaningless.

[A. Nony Mouse]

The problem with this type of analysis is that it doesn't take into account changes in selectivity over time. One might be tempted to say, looking at that graph, that there is more grade inflation amongst private schools than amongst public schools. But it might just be that private schools' selectivity has increased at a higher rate than public schools' selectivity. (I have no idea whether this is the case; I just wanted to point out a possible alternate cause.)

That's why, IMO, if you wanted to gauge the ease of obtaining good grades at various different UGs, the best way to go about it would be to weigh each school's selectivity against its avg uGPA and their compare the schools with one another in order to create some "GPA exchange rate." Yes, it'd be complicated and imperfect, but it'd be a much more accurate measure than looking at trends over time. And it really cuts to the core of the matter.

I've said this already so apologies for the repetition, but I don't think we really have any evidence of significant increases in selectivity that parallel evidence of grade inflation. In fact, I think it's much more likely that as college tuitions have increased, and private schools especially have become more expensive (not being accountable to the state), private schools have had to work harder to market themselves and cater to the paying customer by giving them what they and their helicopter parents want/expect for their money, which is higher grades. There's definitely an attitude of "I'm not paying $45k a year for little Buffy to get Cs!" among parents sending their kids to the more expensive private colleges. And at least the ones I'm familiar with are extremely sensitive to marketing themselves to well-off families.

[iamgeorgebush]

The problem with this type of analysis is that it doesn't take into account changes in selectivity over time. One might be tempted to say, looking at that graph, that there is more grade inflation amongst private schools than amongst public schools. But it might just be that private schools' selectivity has increased at a higher rate than public schools' selectivity. (I have no idea whether this is the case; I just wanted to point out a possible alternate cause.)

That's why, IMO, if you wanted to gauge the ease of obtaining good grades at various different UGs, the best way to go about it would be to weigh each school's selectivity against its avg uGPA and their compare the schools with one another in order to create some "GPA exchange rate." Yes, it'd be complicated and imperfect, but it'd be a much more accurate measure than looking at trends over time. And it really cuts to the core of the matter.

I've said this already so apologies for the repetition, but I don't think we really have any evidence of significant increases in selectivity that parallel evidence of grade inflation. In fact, I think it's much more likely that as college tuitions have increased, and private schools especially have become more expensive (not being accountable to the state), private schools have had to work harder to market themselves and cater to the paying customer by giving them what they and their helicopter parents want/expect for their money, which is higher grades. There's definitely an attitude of "I'm not paying $45k a year for little Buffy to get Cs!" among parents sending their kids to the more expensive private colleges. And at least the ones I'm familiar with are extremely sensitive to marketing themselves to well-off families.

Haven't acceptance rates of top private schools gone down over the years? And haven't the average standardized test scores of incoming freshman at these schools gone up? Sure, those two things are not perfectly indicative of increasing standards of admissions, but it aligns with anecdotal evidence I've heard from people in my parents' generation who have said that they don't think they'd get into the UGs they attended if they were to apply today. Yeah, anecdotal evidence is anecdotal evidence (and boomer anecdotal evidence is even worse), but it says something.

As regards the parents-expecting-higher-grades phenomenon, I attended a top private UG but am not aware of this, so I'm skeptical of it being as widespread as you claim. Maybe it happens for legacies whose parents donate a lot of money, but for the rank-and-file? Eh.

[TigerDude]

passive aggressiveness

also irony

[A. Nony Mouse]

Haven't acceptance rates of top private schools gone down over the years?

I haven't seen any evidence of this. However, if application numbers have gone up, acceptance rates could go down without the quality of applicant/average student changing in any way. Technically, there would be greater selectivity in the sense that more people are getting rejected, but that selectivity doesn't equal higher caliber of student.

And haven't the average standardized test scores of incoming freshman at these schools gone up?

Haven't seen evidence of this either, but if we're speculating about changes over time, I'd like to know how average standardized test scores have changed over time, period.

Sure, those two things are not perfectly indicative of increasing standards of admissions, but it aligns with anecdotal evidence I've heard from people in my parents' generation who have said that they don't think they'd get into the UGs they attended if they were to apply today. Yeah, anecdotal evidence is anecdotal evidence (and boomer anecdotal evidence is even worse), but it says something.

As regards the parents-expecting-higher-grades phenomenon, I attended a top private UG but am not aware of this, so I'm skeptical of it being as widespread as you claim. Maybe it happens for legacies whose parents donate a lot of money, but for the rank-and-file? Eh.

The concern that private colleges/universities have with marketing themselves to applicants (and their families) was very, very, very evident at a number of schools where I taught. I'm sure it varies somewhat according to institutional culture, but it's a real thing. And it's not something that's necessarily evident to students.

[flyingboy]

This is one of the articles that Johnson referred to: http://www.repository.law.indiana.edu/c ... ontext=ilj.

One interesting proposal he makes (pg.317-18) is that the LSAC should make the LSAT school specific and thus non-comparable from school to school, which would presumably retain the LSAT's predictive powers while also reducing reliance on the U.S. News rankings.

[iamgeorgebush]

Haven't acceptance rates of top private schools gone down over the years?

I haven't seen any evidence of this. However, if application numbers have gone up, acceptance rates could go down without the quality of applicant/average student changing in any way. Technically, there would be greater selectivity in the sense that more people are getting rejected, but that selectivity doesn't equal higher caliber of student.

Evidence: http://talk.collegeconfidential.com/uni ... nford.html

Perhaps the greater selectivity doesn't equal a higher caliber of student, but that hypothesis certainly makes more assumptions than the converse, that this increased selectivity is indeed linked to a higher caliber of student. One possible explanation: there is more need-based financial aid now than there was in earlier times, meaning that smart but poor students can attend schools that were previously unavailable to them. This would particularly impact schools like Harvard that pay the entire tuition of needy undergraduates.

And haven't the average standardized test scores of incoming freshman at these schools gone up?

Haven't seen evidence of this either, but if we're speculating about changes over time, I'd like to know how average standardized test scores have changed over time, period.

I can't find anything on this, but come on, this is pretty common knowledge. If you want to find something to disprove me, I'm all ears.

Sure, those two things are not perfectly indicative of increasing standards of admissions, but it aligns with anecdotal evidence I've heard from people in my parents' generation who have said that they don't think they'd get into the UGs they attended if they were to apply today. Yeah, anecdotal evidence is anecdotal evidence (and boomer anecdotal evidence is even worse), but it says something.

As regards the parents-expecting-higher-grades phenomenon, I attended a top private UG but am not aware of this, so I'm skeptical of it being as widespread as you claim. Maybe it happens for legacies whose parents donate a lot of money, but for the rank-and-file? Eh.

The concern that private colleges/universities have with marketing themselves to applicants (and their families) was very, very, very evident at a number of schools where I taught. I'm sure it varies somewhat according to institutional culture, but it's a real thing. And it's not something that's necessarily evident to students.[/quote]
Ok, but even if they do market themselves in this way, it doesn't mean that they actually do pad GPAs. They might just hint at it in order to convince the parents to pay up!

[A. Nony Mouse]

Perhaps the greater selectivity doesn't equal a higher caliber of student, but that hypothesis certainly makes more assumptions than the converse, that this increased selectivity is indeed linked to a higher caliber of student. One possible explanation: there is more need-based financial aid now than there was in earlier times, meaning that smart but poor students can attend schools that were previously unavailable to them. This would particularly impact schools like Harvard that pay the entire tuition of needy undergraduates.

I think part of the issue is the ever-increasing expectation that everyone has to get a BA to succeed at anything. So more and more people apply to schools, but it doesn't mean they're better students.

Ok, but even if they do market themselves in this way, it doesn't mean that they actually do pad GPAs. They might just hint at it in order to convince the parents to pay up!

I probably wasn't clear, but that's not what I meant. Everyone I know who teaches college (including many who've been doing so for decades) complains about the increasing expectations on the part of students/parents that schools are there to make students happy, a big part of which is not giving low grades. When you're looking at private schools that rely on marketing and branding and selling themselves to students, those kinds of expectations likely have an effect that they don't have on public schools (especially given the demographic differences between private/public schools). It's anecdotal, but extremely widespread, and based on experience with many many profs at many schools.

And within academia (by which I mean teachers teaching, and institutions analyzing grades administered over time) grade inflation is widely accepted as a function of giving higher grades, not as a function of smarter students.

[iamgeorgebush]

I think part of the issue is the ever-increasing expectation that everyone has to get a BA to succeed at anything. So more and more people apply to schools, but it doesn't mean they're better students.

Aren't we talking about top private UGs? The kind of person that gets a BA just because s/he is expected to get one in order to succeed at something is not the type of person that gets into Harvard. My argument is that because more and more people are shooting for college and because top schools have much more robust need-based financial aid packages than they did in the past, top schools like Harvard are able to wrangle in a higher caliber of students than they have been able to in the past. If grade inflation at these schools has outpaced grade inflation certain other less prestigious schools (and I have yet to see evidence of this), perhaps that explains it.

I probably wasn't clear, but that's not what I meant. etc. etc.

Hmm, interesting. These people you know in academia, at what sorts of colleges do they teach? I could imagine that being the case for some schools, but I just have a hard time imagining the profs at my own private UG alma mater giving students higher grades because of that pressure. Then again, my alma mater is thought to be very difficult and suffer from less "grade inflation" or w/e than most other private UGs, so maybe it's an exception to the rule.

And within academia (by which I mean teachers teaching, and institutions analyzing grades administered over time) grade inflation is widely accepted as a function of giving higher grades, not as a function of smarter students.

I don't dispute that grade inflation is mostly a function of giving higher grades, although I did read one prof write that he thinks that at his school, the higher caliber of students has contributed to it somewhat.