LSAC settles LSAT Disability Lawsuit Forum

[A. Nony Mouse]

I don't think we know enough about cognitive performance to say that double the time or whatever is appropriate to disability X.

Sure we do. There are all sorts of people studying cognition for the purpose of determining appropriate accommodations. It's not like LSAC is going to give a blind person double-time and call it a day - there's tons of research into this issue.

[LSAT Hacks (Graeme)]

I don't think we know enough about cognitive performance to say that double the time or whatever is appropriate to disability X.

Sure we do. There are all sorts of people studying cognition for the purpose of determining appropriate accommodations. It's not like LSAC is going to give a blind person double-time and call it a day - there's tons of research into this issue.

Do they know about how extra time works on a test like the SAT vs. the LSAT though?

On most tests I ever did, the amount of time allotted was more than sufficient for me to do them at an easy pace (I test well). The LSAT was stressful.

This is a common experience. Even 15 minutes extra time would raise most people's scores by 10-20 points. The agreement requires the LSAC to grant any accommodations that were granted on another test. To me, this is the central point.

[papercut]

I don't think we know enough about cognitive performance to say that double the time or whatever is appropriate to disability X.

Sure we do. There are all sorts of people studying cognition for the purpose of determining appropriate accommodations. It's not like LSAC is going to give a blind person double-time and call it a day - there's tons of research into this issue.

Given how many disabilities exist on a spectrum, and that they are often mixed with other disabilities there's no way that we can get an "appropriate accommodation" down to the minute, or whatever fraction of 35 minutes you'd like.

[ScottRiqui]

In and of itself, I'm not sure how I feel about getting rid of flagging. In the case of an accommodated/flagged score, what was the "right" way for schools to consider it for admissions purposes? Give it less weight than an unflagged score? Give it the same weight as an unflagged score? Ignore it completely?

If the "right" thing was to assume that the accommodations given were appropriate to the disability, then why were scores ever flagged in the first place?

I don't think we know enough about cognitive performance to say that double the time or whatever is appropriate to disability X.

This is why it's important to keep the testing conditions the same.

Wanting to keep the test conditions the same is an argument against accommodations of any sort, which obviously isn't the right answer (legally-speaking).

I'm just curious what the original rationale was for the "give accommodations and flag the score" plan in the first place. It just seems strange to give accommodations (ostensibly to help level the field for the disabled testers), but then flag the score anyway. How were schools supposed to treat the flagged score in making their admissions decision?

[papercut]

(ostensibly to help level the field for the disabled testers)

There's no such thing as simply leveling the playing field on curved test. When you help one, you harm another.

[papercut]

How were schools supposed to treat the flagged score in making their admissions decision?

Differently. How exactly isn't really all that important. The schools should know who got extra time.

[ScottRiqui]

(ostensibly to help level the field for the disabled testers)

There's no such thing as simply leveling the playing field on curved test. When you help one, you harm another.

The LSAT isn't "curved" in that manner. Unlike a law school exam, there's nothing mathematically preventing every LSAT taker in the room (or in the entire test center, or even in an entire administration) from getting 170+. Did you think that by prepping for the LSAT and improving your score, you were lowering someone else's score to "balance the books"?

[papercut]

(ostensibly to help level the field for the disabled testers)

There's no such thing as simply leveling the playing field on curved test. When you help one, you harm another.

The LSAT isn't "curved" in that manner. Unlike a law school exam, there's nothing mathematically preventing every LSAT taker in the room (or in the entire test center, or even in an entire administration) from getting 170+. Did you think that by prepping for the LSAT and improving your score, you were lowering someone else's score to "balance the books"?

I know exactly how the LSAT is "curved." The curve gets set by prior LSAT takers. Just because you're not* setting the curve for the ppl that are taking the LSAT with you, doesn't mean you're not setting it for someone down the road.

[A. Nony Mouse]

Do they know about how extra time works on a test like the SAT vs. the LSAT though?

On most tests I ever did, the amount of time allotted was more than sufficient for me to do them at an easy pace (I test well). The LSAT was stressful.

This is a common experience. Even 15 minutes extra time would raise most people's scores by 10-20 points. The agreement requires the LSAC to grant any accommodations that were granted on another test. To me, this is the central point.

Let's put it this way - I personally don't know how the extra time works, but I'm quite sure that in the vast testing industry there are lots of people who've studied this.

Given how many disabilities exist on a spectrum, and that they are often mixed with other disabilities there's no way that we can get an "appropriate accommodation" down to the minute, or whatever fraction of 35 minutes you'd like.

But extra time isn't the only accommodation available to (or needed by) people with disabilities.

[papercut]

Do they know about how extra time works on a test like the SAT vs. the LSAT though?

On most tests I ever did, the amount of time allotted was more than sufficient for me to do them at an easy pace (I test well). The LSAT was stressful.

This is a common experience. Even 15 minutes extra time would raise most people's scores by 10-20 points. The agreement requires the LSAC to grant any accommodations that were granted on another test. To me, this is the central point.

Let's put it this way - I personally don't know how the extra time works, but I'm quite sure that in the vast testing industry there are lots of people who've studied this.

Given how many disabilities exist on a spectrum, and that they are often mixed with other disabilities there's no way that we can get an "appropriate accommodation" down to the minute, or whatever fraction of 35 minutes you'd like.

But extra time isn't the only accommodation available to (or needed by) people with disabilities.

Fair enough. I can see the propriety for things like bigger type, more room on the page, etc. The time I'm really skeptical about.

[LSAT Hacks (Graeme)]

Do they know about how extra time works on a test like the SAT vs. the LSAT though?

On most tests I ever did, the amount of time allotted was more than sufficient for me to do them at an easy pace (I test well). The LSAT was stressful.

This is a common experience. Even 15 minutes extra time would raise most people's scores by 10-20 points. The agreement requires the LSAC to grant any accommodations that were granted on another test. To me, this is the central point.

Let's put it this way - I personally don't know how the extra time works, but I'm quite sure that in the vast testing industry there are lots of people who've studied this.

Given how many disabilities exist on a spectrum, and that they are often mixed with other disabilities there's no way that we can get an "appropriate accommodation" down to the minute, or whatever fraction of 35 minutes you'd like.

But extra time isn't the only accommodation available to (or needed by) people with disabilities.

I doubt many people have studied extra time on the LSAT, beyond the LSAC. It wasn't an issue until....today. And, as I wrote above, the agreement compels the LSAC to automatically grant most allowances from other tests. If that includes extra time, then this will apply accommodations meant for other tests to the LSAC.

If that part of the agreement only refers to things like more paper, reading software, etc. then I'm actually all for it. Time is the only one that's a concern. The agreement wasn't clear on this issue.

Since LSAC agreed to it, and they're not fools, I'm guessing they've got a way to make this work without destroying the test.

Edit: My speculations are wrong. The agreement explicitly covers time. Have a look at section 5a here:

--LinkRemoved--

[A. Nony Mouse]

I agree that time is tougher than other accommodations.

[ScottRiqui]

(ostensibly to help level the field for the disabled testers)

There's no such thing as simply leveling the playing field on curved test. When you help one, you harm another.

The LSAT isn't "curved" in that manner. Unlike a law school exam, there's nothing mathematically preventing every LSAT taker in the room (or in the entire test center, or even in an entire administration) from getting 170+. Did you think that by prepping for the LSAT and improving your score, you were lowering someone else's score to "balance the books"?

I know exactly how the LSAT is "curved." The curve gets set by prior LSAT takers. Just because you're not* setting the curve for the ppl that are taking the LSAT with you, doesn't mean you're not setting it for someone down the road.

Even if LSAC is no longer allowed to flag accommodated scores in the reports they send to schools, there's no reason at all why they couldn't leave the accommodated scores out of the equating process when determining the curve for future administrations. In fact, it would be kind of dumb for them to leave them in.

[papercut]

Even if LSAC is no longer allowed to flag accommodated scores in the reports they send to schools, there's no reason at all why they couldn't leave the accommodated scores out of the equating process when determining the curve for future administrations. In fact, it would be kind of dumb for them to leave them in.

Sure they could do that, and probably will.

I should have made a more general point.

It's not just about getting higher LSAT scores, it's about getting into law schools.

Law school admission are still a zero sum game. If you help someone get an invalid, higher LSAT score you will hurt someone else when it comes to admissions.

[ScottRiqui]

Law school admission are still a zero sum game. If you help someone get an invalid, higher LSAT score you will hurt someone else when it comes to admissions.

What makes an LSAT score "invalid", though? A disabled tester simply scoring higher than he would have without the accommodations? That seems to require starting from the premise that accommodations give a boost that's disproportionate relative to the disability.

[LSAT Hacks (Graeme)]

Even if LSAC is no longer allowed to flag accommodated scores in the reports they send to schools, there's no reason at all why they couldn't leave the accommodated scores out of the equating process when determining the curve for future administrations. In fact, it would be kind of dumb for them to leave them in.

But what happens if 5-10% of LSAT takers start getting accommodations? That will seriously mess with the curve, especially at the upper percentiles.

This isn't a crazy scenario. Many students don't bother seeking extra time on the SAT and other tests (cases of mild ADHD, etc.). It's not that crucial for many of them. But on the LSAT, people are DESPERATE for extra time.

There are many postsecondary tests in the USA. The new consent agreement obliges the college board to honour extra time requests granted by any post-secondary test. (GRE, ACT, SAT, GMAT, MCAT, and lesser tests I'm not aware of).

Have a look at paragraph 5a of the consent agreement: --LinkRemoved--

For those candidates whose documentation establishes that they previously were approved to receive testing accommodations on any standardized examination offered in the United States related to applications for post-secondary admission,4 and with respect to the testing accommodations for which they were previously approved, LSAC shall require no more documentation than proof of the approval for such testing accommodations, and certification by the candidate through a checkmark box on the candidate form that the candidate is still experiencing the functional limitations caused by the disability(ies) for which testing accommodations were approved. Acceptable proof of prior testing accommodations shall consist of a letter or similar documentation from the other test sponsor confirming that testing accommodations were approved and specifically identifying what those approved testing accommodations were. Upon receipt of such proof in accordance with LSAC’s established deadlines, without further inquiry or request for additional documentation, LSAC shall grant those previously approved testing accommodations, or the equivalent testing accommodation offered on the LSAT,5 with respect to requests for extended time up to double time6 as well as certain other testing accommodations that are listed in Exhibit 1.

The process really does seem to be automatic. And the stakes are SO high. Anyone who can quality for extra time on any of the manyof post-secondary tests in America can now apply for accommodation with one of those tests.

You're rightly thinking of students who have serious disabilities that need accommodation. It's been a nightmare for them to get help. That's what prompted this lawsuit. There seem to be about 1,000 of them per year.

I expect there are 10x their number who will seek extra time by hook or by crook once these new rules go through. That's about 10% of people who take the LSAT. There were 100,000 administrations of the LSAT last year, and that includes those who took the test 2-3 times.

This has the potential to seriously skew the test.

[A. Nony Mouse]

For those candidates whose documentation establishes that they previously were approved to receive testing accommodations on any standardized examination offered in the United States related to applications for post-secondary admission,4 and with respect to the testing accommodations for which they were previously approved, LSAC shall require no more documentation than proof of the approval for such testing accommodations, and certification by the candidate through a checkmark box on the candidate form that the candidate is still experiencing the functional limitations caused by the disability(ies) for which testing accommodations were approved. Acceptable proof of prior testing accommodations shall consist of a letter or similar documentation from the other test sponsor confirming that testing accommodations were approved and specifically identifying what those approved testing accommodations were. Upon receipt of such proof in accordance with LSAC’s established deadlines, without further inquiry or request for additional documentation, LSAC shall grant those previously approved testing accommodations, or the equivalent testing accommodation offered on the LSAT,5 with respect to requests for extended time up to double time6 as well as certain other testing accommodations that are listed in Exhibit 1.

What about the red bit? That seems to suggest it's not necessarily taking the same accommodation on, say, the SAT, and getting that exact accommodation on the LSAT.

[LSAT Hacks (Graeme)]

For those candidates whose documentation establishes that they previously were approved to receive testing accommodations on any standardized examination offered in the United States related to applications for post-secondary admission,4 and with respect to the testing accommodations for which they were previously approved, LSAC shall require no more documentation than proof of the approval for such testing accommodations, and certification by the candidate through a checkmark box on the candidate form that the candidate is still experiencing the functional limitations caused by the disability(ies) for which testing accommodations were approved. Acceptable proof of prior testing accommodations shall consist of a letter or similar documentation from the other test sponsor confirming that testing accommodations were approved and specifically identifying what those approved testing accommodations were. Upon receipt of such proof in accordance with LSAC’s established deadlines, without further inquiry or request for additional documentation, LSAC shall grant those previously approved testing accommodations, or the equivalent testing accommodation offered on the LSAT,5 with respect to requests for extended time up to double time6 as well as certain other testing accommodations that are listed in Exhibit 1.

What about the red bit? That seems to suggest it's not necessarily taking the same accommodation on, say, the SAT, and getting that exact accommodation on the LSAT.

Good catch. That's covered in footnotes five and six. Here's the footnote text:

For example, LSAC shall provide a candidate who submits appropriate documentation that she received double time for the GRE with extended time for both the multiple choice sections and the writing section of the LSAT – despite the fact that the GRE may not specify the sections on which a candidate received extended time. Likewise, a candidate may submit appropriate documentation showing that she previously received the testing accommodation of “large block answer sheet” on the ACT. If that exact testing accommodation is not offered for the LSAT, LSAC shall grant the closest equivalent testing accommodation provided for the LSAT – such as an alternate non-Scantron answer sheet.

6 Although candidates who previously received in excess of double time will be granted double time under this provision, LSAC will consider the balance of requested extended time, and any other testing accommodations not previously received on a standardized examination related to applications for post-secondary admission, under the provisions of Paragraphs 5(b)-(d).

I believe this means that the equivalent of 50% extra time on test X is 50% extra time on the LSAT. That wording is in there to deal with differences in the specifics of how the other tests grant accommodations.

[papercut]

Law school admission are still a zero sum game. If you help someone get an invalid, higher LSAT score you will hurt someone else when it comes to admissions.

What makes an LSAT score "invalid", though? A disabled tester simply scoring higher than he would have without the accommodations?

What I mean by "invalid" is that it is a score that doesn't predict 1L grades to the same degree, at the same schools.

That seems to require starting from the premise that accommodations give a boost that's disproportionate relative to the disability.

Look, I think some accommodations are just fine (bigger font). It's the extra time that I'm not happy with.

You'd have to see if accommodated LSAT scores for the disabled applicant pool have the same predictive power. If not, the boost is disproportionate.

This would be an almost impossible task, because you'd have very small samples sizes after grouping applicants by disability+accommodation degree/type.

I think the better solution is to either have everyone take the test under the same conditions, and then the disabled applicants can write a diversity statement or addendum on their difficulties or, we give accommodations and flag them.

[ScottRiqui]

Even if LSAC is no longer allowed to flag accommodated scores in the reports they send to schools, there's no reason at all why they couldn't leave the accommodated scores out of the equating process when determining the curve for future administrations. In fact, it would be kind of dumb for them to leave them in.

But what happens if 5-10% of LSAT takers start getting accommodations? That will seriously mess with the curve, especially at the upper percentiles.

With 100,000+ takers every year, I bet that LSAC could leave 10% of them out of the equating process and still have no trouble generating curves for future administrations. On the whole, the sanctity of the LSAT will remain intact.

Now, papercut is right that at the individual level, some applicants are going to be shut out because competing applicants will have higher scores than they would have without accommodations. But I was addressing what I perceived to be his point that "accommodations are bad" in general.

I agree that if people start "gaming the system" in large numbers, it's going to cause problems. But I'm not ready to throw out the baby with the bathwater and agree that accommodations in general give an unfair advantage.

[A. Nony Mouse]

What I mean by "invalid" is that it is a score that doesn't predict 1L grades to the same degree, at the same schools.

I get that concern. But the LSAT's predictive ability is far from perfect. It also seems to me that if someone has a significant enough disability to get accommodations on the LSAT, they may well get accommodations in law school, in which case the LSAT's predictive power would likely hold good.

[cotiger]

From time spent tutoring the SAT/ACT, the only kids who ever have a time crunch are those without accommodation. Accommodated testing effectively means untimed, and I'd imagine this advantage holds even more true in such a time-crunched test like the LSAT.

Yeah, I'm sure that LSAC (and the College Board, etc) have plenty of smart people studying this. But ultimately, adherence to the ADA and fear of lawsuits from kids of rich families who've snagged their diagnoses will take precedence over predictive validity.

[UnicornHunter]

Will schools eventually be forced to offer extra time with no flagging too?

What's the reasoning for requiring this on entrance exams but not in the schools themselves?

Interested in this. If schools are not required to grant accommodations to everyone who gets extra time on the LSAT, I want to defer a year. Put me on a curve in LS with people who abuse this, I'd love it. OTOH, if schools also have to grant extra time to anyone who can get an ADD diagnosis, that kind of sucks.

[Learn_Live_Hope]

..

[LSAT Hacks (Graeme)]

Even if LSAC is no longer allowed to flag accommodated scores in the reports they send to schools, there's no reason at all why they couldn't leave the accommodated scores out of the equating process when determining the curve for future administrations. In fact, it would be kind of dumb for them to leave them in.

But what happens if 5-10% of LSAT takers start getting accommodations? That will seriously mess with the curve, especially at the upper percentiles.

With 100,000+ takers every year, I bet that LSAC could leave 10% of them out of the equating process and still have no trouble generating curves for future administrations. On the whole, the sanctity of the LSAT will remain intact.

Now, papercut is right that at the individual level, some applicants are going to be shut out because competing applicants will have higher scores than they would have without accommodations. But I was addressing what I perceived to be his point that "accommodations are bad" in general.

I agree that if people start "gaming the system" in large numbers, it's going to cause problems. But I'm not ready to throw out the baby with the bathwater and agree that accommodations in general give an unfair advantage.

My worry is that if people can game the system, then they will. That's how systems work in the US. The SAT is heavily gamed – I'm part of that industry.

The LSAT has been, to a surprisingly extent, ungameable. There are probably 20,000 people every year who would pay $5,000+ for a reliable 10-15 point boost on the LSAT. They've tried courses and everything else, and they've plateau. And their future is riding on that score. $5,000 is nothing.

For that kind of money, you can buy yourself into a diagnosis, register for another post-secondary test, get extra time, and use section 5a to transfer that automatically to the LSAT by sending in a letter. This might take 1-2 months.

I've been on the receiving end of hundreds of desperate emails complaining about time issues. "I can score 170 with enough time, but when I try it timed, I get 151. Help!!!!!!!!" And I can't help. Not to the extent they want. Right now, they're stuck.

This agreement, prima facie, has opened a chink in that armor. I can see this chink clearly. I'm not going to exploit it for money, but others will. Networks will form in the major cities between test prep consultants and "lenient" psychologists. Guides will be written for which tests are easiest to get accommodations for, and how to send those results to LSAC.

Now, maybe the system has safeguards that aren't written into the consent agreement. But if the agreement works as written, what's stopping people from gaming the system en masse?

It's already happening with the SAT, except the stakes for extra time aren't as high, so fewer people are exploiting it.

I agree with you on your other points, but I don't think you've integrated systems thinking into your approach.