Smaller Curves & Their Effects on the LSAT

mst
Posts: 925
Joined: Sun Jun 06, 2010 3:01 am

Smaller Curves & Their Effects on the LSAT

Postby mst » Thu Jun 10, 2010 11:18 pm

TL;DR Version: LSAT SCORE SYSTEM BLOWS. LETS CHANGE IT?!

LSAT RC Version using 8th grade grammar: As I sat outside my test room this week, in between Sections 3 and 4, my brain ran over a certain question I know I had incorrectly bubbled, trying to reinforce the answer/number combination to memory so that if I wanted to, I could fix it in section 4. Now, I know that bubbling prior sections is against the rules, and I know that it's a lot to risk, if it means getting caught by the proctor and basically being banished from law school. But I also realized something at that moment: that one question, at my performance level, would undoubtedly make the difference between my score being a 1** and a 1**+1.

All too often, people around the country realize this when considering whether to make slight changes to their scantrons, and the vast majority of them, from my experience, are willing to go the extra step for the extra point: whether that means keeping the pencil up for a few extra seconds to finish your section, filling in missed bubbles, or correcting missed answers outside of the alloted time.

Was it always like this? To some extent, sure. But taking a look at the decreasing curve amounts over the last 20 years, the importance in the upper echelons of scoring (164+) of individual points has become vastly more important in achieving certain scores. Coupled with this has been the increased importance of LSAT scores in law school admissions, and thus the narrowed ranges acceptable at the nations top programs.

For example, as demonstrated by LSN and other sites, the difference between a 169 and 170 will in many cases make a noticeable difference in the strength of your application at certain schools. And when that difference in points between a 169 and a 170 is defined by a single question, slight-cheating can become almost too temptatious to avoid.

While this obviously points to growing issues in the realm of cheating, I believe it raises more points about the structure of the LSAT scoring system and it's place in law school admissions. Generally speaking, the LSAT does represent the only real system for objectively measuring candidates against each other. But the scores that the test is producing no longer represent the distinct difference in performance that they did 10 to 20 years ago. Whereas 20 years ago the difference between two certain scores could be a couple questions, those same two scores could today could be separated by as little as 1 question.

Schools have virtually claimed certain and distinct LSAT scores as the brackets for admitted students, and performance at certain levels is now key. Whether or not it is the intention of the schools, the practical effect this has had is the following on students taking the LSAT: "One question will get me into a much better law school. If one question means I will get in as opposed to won't get in, that must mean they think that question represents the needed level of ability to get into their school." I doubt that any school really believes that their is a tangible difference in quality between somebody who got a 173 and a 174, but all evidence points to the fact that this is a critically important difference in whether or not to admit a student to certain schools.

It seems LSAT has recognized this trend and has tried to put more focus on "score ranges" when reporting scores to schools. For example, they emphasize that a 167 fits in the 166-168 range and that a 167 is not some distinct level of performance. However, as schools compete desperately for LSAT based rankings, those ranges importance seems to all but disappear.

So what's the solution here?

Do you guys believe that the time has come for a major overhaul in the way that the LSAT is scored, or at least a change in the way that the raw scores are translated to comparable scores/percentiles?

Would you agree more or less with a system that perhaps didn't distinguish between 170-175, 176-180, etc., (although this would still leave borderline areas where students would be tempted to cheat or schools would be tempted to advance into...)?

If you could propose a new system to fix this issue, what would it be? If you think that the current system is better than any alternative, why?
Last edited by mst on Fri Jun 11, 2010 2:50 pm, edited 3 times in total.

User avatar
Bildungsroman
Posts: 5548
Joined: Sun Apr 11, 2010 2:42 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby Bildungsroman » Thu Jun 10, 2010 11:26 pm

tl;dr

ITT: cheater attempts to justify cheating; fails.

mst
Posts: 925
Joined: Sun Jun 06, 2010 3:01 am

Re: Smaller Curves & Their Affects on the LSAT

Postby mst » Thu Jun 10, 2010 11:29 pm

Big assumption there, but go ahead and make it? All I claimed to do was spend my precious 10 minutes of LSAT break time comprehending the risk of cheating...not a crime.

User avatar
voice of reason
Posts: 264
Joined: Thu Oct 29, 2009 12:18 am

Re: Smaller Curves & Their Affects on the LSAT

Postby voice of reason » Thu Jun 10, 2010 11:37 pm

The question really is what is the standard error of the estimate. This is an arcane statistical point but it's what the issue comes down to. It is not whether the difference between a 173 and a 174 is 1 question or 2. It is whether the test produces precise measurements of the underlying ability trait, where precision is indicated by a narrow score band and imprecision is indicated by a wide one.

The problems with collapsing scores into 170-175 and 176-180 are that this would elide real differences between test-takers who would end up with the same score and that the same problem you identify would exist at the margin between the two scores. Combining scores in those ranges would make things worse.

Scores are subject to error in a way very much analogous to how surveys are subject to sampling error. The solution to this problem, all else being equal, is a longer test. Unless you can write better questions, the only way to get more precise measurement is to ask more questions. Would you rather sit for the LSAT in 3 sessions of 6 hours each spread over 3 weeks, and pay 3 times as much for it? That would be the direction to go in order to get scores where a 1 point difference is meaningful and where there is no meaningful difference in ability between people with the same scores. I don't think we want to put up with that. And I don't think schools would benefit much from it because the power of the LSAT to identify more desirable students will never be spectacular.

User avatar
romothesavior
Posts: 14772
Joined: Fri Jun 26, 2009 4:29 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby romothesavior » Thu Jun 10, 2010 11:39 pm

tl;dr

tb;dr

(tb = too boring)

User avatar
Ragged
Posts: 1509
Joined: Wed Oct 21, 2009 12:39 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby Ragged » Thu Jun 10, 2010 11:39 pm

tl;dr, but at first I thought the thread was about the female form.

I'll read it eventually when I'm bored.

Renzo
Posts: 4265
Joined: Tue Dec 02, 2008 3:23 am

Re: Smaller Curves & Their Affects on the LSAT

Postby Renzo » Thu Jun 10, 2010 11:42 pm

Ragged wrote:tl;dr, but at first I thought the thread was about the female form.

I'll read it eventually when I'm bored.

Yep, except I won't read it. But smaller is always better, as long as we're still talking physique.

User avatar
Ragged
Posts: 1509
Joined: Wed Oct 21, 2009 12:39 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby Ragged » Thu Jun 10, 2010 11:47 pm

Renzo wrote:
Ragged wrote:tl;dr, but at first I thought the thread was about the female form.

I'll read it eventually when I'm bored.

Yep, except I won't read it. But smaller is always better, as long as we're still talking physique.


I will go ahead and disagree with you there in the words of an epic song: I like big butts and I can not lie....

mst
Posts: 925
Joined: Sun Jun 06, 2010 3:01 am

Re: Smaller Curves & Their Affects on the LSAT

Postby mst » Thu Jun 10, 2010 11:50 pm

Voice, you make a good point concerning a more realistic approach to start to distinguish the quality of students, in a more accurate manner. But you acknowledge it's impractical.

I previously acknowledged that the core problem I initially pointed out would still exist if the LSAT reported broader score ranges to schools rather than scores. However, the frequency of a tester approaching the break in a score range would be less frequent, and one could assume that the student would be less enticed to cheat, and that the schools wouldn't be forced to differentiate between candidates that in all likelihood demonstrate equal ability despite a difference of 1-2 questions on the test.

With this method, the test structure wouldn't have to change, the scores from candidates who had taken the test previously would still be applicable, and law schools would feel less pressure to respond to what most could agree are unimportant LSAT differences.

Tautology
Posts: 434
Joined: Thu Mar 18, 2010 12:40 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby Tautology » Fri Jun 11, 2010 12:12 am

Schools know that the difference between a 170 and a 171 is small, but that doesn't mean it's not meaningful.

The problem is that you are looking at this from the perspective of the test taker, which, understandable as it is, is not the perspective of the schools when they're choosing who to accept. From the school's point of view, there is a population of students with 170s and a population of students with 171s. Within each population there will be some who got lucky on some questions and overachieved, some who got unlucky on a question or two and underachieved, and some who are right where they're supposed to be. On average though, the population with 171 has higher achieving students. From the school's point of view, on average you're going to get better students taking the 171s than taking the 170s. That difference is small, but small distinctions are still distinctions. The best schools get way more applicants than they can accept, and they get to make small distinctions among them when deciding who they want.

User avatar
World B. Free
Posts: 103
Joined: Sat Nov 21, 2009 2:34 am

Re: Smaller Curves & Their Affects on the LSAT

Postby World B. Free » Fri Jun 11, 2010 12:13 am

Tautology wrote:Schools know that the difference between a 170 and a 171 is small, but that doesn't mean it's not meaningful.


+1

mst
Posts: 925
Joined: Sun Jun 06, 2010 3:01 am

Re: Smaller Curves & Their Affects on the LSAT

Postby mst » Fri Jun 11, 2010 12:51 am

I'm not suggesting it's not meaningful. I believe it does have meaning to some extent. But I don't think anybody believes it has such meaning to the extent that schools will rarely accept students outside of a 3 point range, or that the cutoff line on LSATs for accepted students (non URM) is so noticeable and apparent, for the sole sake that those score differences are so vitally important. If schools really believe that 172/171 difference is that meaningful, that is sad. We all know the effect of a missed bubble, or a broken pencil. Or a sneeze that stops you from finishing that last question. If schools are at the point where they truly believe they must downgrade entire applicant pools of students with scores one point lower because the performance is going to be on average worse, that is sad.

However, there's plenty of reasons to believe this is not the case. Much evidence points to the fact that schools are generally more holistic in their reviews than that. They take into account a myriad of factors for decisions. Yet you claim that they willingly choose to have such stringent LSAT points solely for the case of their ability to predict future performance?

I don't think that's the case. And from most news articles I've read, and LSAC research papers, either do many admission committees, law professors, deans, or students. All the above had one thing in common: they believe that LSAT scores play too much of a role in decisions because schools are motivated to raise their averages in order to maintain or improve their national ranking.

User avatar
Chimica
Posts: 85
Joined: Wed May 26, 2010 3:30 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby Chimica » Fri Jun 11, 2010 1:12 am

I think that while it is possible to bubble in sections you aren't actually doing, it just doesn't help you.

You need an amazing memory; it distracts you from the section at hand and most people would do better with full focused attention on the 35 minutes they have on the section they can read directly; it gets you one point...maybe. How much is lost from loss of focus?

So, I don't think that LSAC forces this and I don't think that the cheating actually helps.

(I may be biased. I've taught and most cheaters are stupid, lazy and eventually get caught or fail before they are officially caught-granted, we may have missed the really smart ones, but it was probabely easier to just learn the damn material )

User avatar
malfurion
Posts: 159
Joined: Sat Mar 06, 2010 5:40 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby malfurion » Fri Jun 11, 2010 8:50 am

--ImageRemoved--

User avatar
malfurion
Posts: 159
Joined: Sat Mar 06, 2010 5:40 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby malfurion » Fri Jun 11, 2010 9:06 am

I don't think the cheating idea is a valid premise for your argument. People who are willing to cheat are going to continue to do so, even if the scoring system changes. These are the same people that have been cheating on stuff since 3rd grade. I remember reading about studies that showed that people would continue to cheat on tests even when they were told ahead of time that their results on the test did not matter. I think it's incredibly rare that someone who has never cheated before in their lives would decide to cheat for the first time due to the LSAT scoring system.

That said, I do agree that it's ridiculous how a one-point difference in scores can have such an impact on admissions, given that the stated margin of error for the test is greater than one point (you could make a good LR flaw question out of that, actually). In my opinion, the absurd dependence on the school rankings system is more to blame for this than the actual LSAT scoring system, though.

User avatar
balzern
Posts: 347
Joined: Tue May 25, 2010 2:27 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby balzern » Fri Jun 11, 2010 9:10 am

malfurion wrote:I don't think the cheating idea is a valid premise for your argument. People who are willing to cheat are going to continue to do so, even if the scoring system changes. These are the same people that have been cheating on stuff since 3rd grade. I remember reading about studies that showed that people would continue to cheat on tests even when they were told ahead of time that their results on the test did not matter. I think it's incredibly rare that someone who has never cheated before in their lives would decide to cheat for the first time due to the LSAT scoring system.

That said, I do agree that it's ridiculous how a one-point difference in scores can have such an impact on admissions, given that the stated margin of error for the test is greater than one point (you could make a good LR flaw question out of that, actually). In my opinion, the absurd dependence on the school rankings system is more to blame for this than the actual LSAT scoring system, though.



+1000000000000000

User avatar
pinkzeppelin
Posts: 232
Joined: Mon Dec 07, 2009 11:51 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby pinkzeppelin » Fri Jun 11, 2010 9:10 am

*Effects

User avatar
suspicious android
Posts: 938
Joined: Tue Feb 09, 2010 4:54 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby suspicious android » Fri Jun 11, 2010 11:42 am

Here's my solution: let people look ahead or back in their booklet, let them bubble in answers from previous sections. Start up section one, have the proctor say "you have one hour, forty-five minutes to complete the next three sections." After that, they collect your booklets and before you start section 4 & 5, you get a new booklet with only two sections. You have 70 minutes to complete that section. This would reward people who are perfect in one section and have the time management skills to make sure they get through the whole thing, which seems like it'd be a valuable skill to test. They might need to make some material a bit harder because I think this would intend to increase the scores for people who are already in the 170's, but that shouldn't be too hard to adjust.

This is a really self-serving suggestion, since I finished my two LR sections with about 5 minutes left and spent that time furious that I couldn't look back at my LG section. But I think it really could mitigate the effect of one crazy question/LG costing someone 5-6 points, which doesn't seem to me to be a sensible way to measure people's ability. Also, for people who have already hit the ceiling in one section (that is, they can almost always get -0 in significantly less than 35 minutes), the LSAT isn't measuring that increased ability compared to someone who just barely gets all the questions right in 35 minutes. This change could address that.

schnoodle
Posts: 50
Joined: Thu Nov 26, 2009 7:57 am

Re: Smaller Curves & Their Affects on the LSAT

Postby schnoodle » Fri Jun 11, 2010 11:53 am

sweet jesus, it's EFFECTS, NOT AFFECTS. further, re-read your OP. is "tempestuous" what you were looking for there? i thought school place a lot of value on good writing. go to your local community college and take a couple english courses before thinking about retaking the lsat/applying to law school.

next time you think a word might be right and might make you sound smarter to a bunch of anonymous strangers, do us all a favor and put it away. go with a word you know.

User avatar
3|ink
Posts: 7331
Joined: Wed Dec 16, 2009 5:23 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby 3|ink » Fri Jun 11, 2010 12:27 pm

TS; AR

nireca
Posts: 66
Joined: Wed Jun 09, 2010 8:18 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby nireca » Fri Jun 11, 2010 2:23 pm

schnoodle wrote:sweet jesus, it's EFFECTS, NOT AFFECTS. further, re-read your OP. is "tempestuous" what you were looking for there? i thought school place a lot of value on good writing. go to your local community college and take a couple english courses before thinking about retaking the lsat/applying to law school.

next time you think a word might be right and might make you sound smarter to a bunch of anonymous strangers, do us all a favor and put it away. go with a word you know.


woah, this dude is clearly a grammar bad ass, that or just a pedantic douche.

mst
Posts: 925
Joined: Sun Jun 06, 2010 3:01 am

Re: Smaller Curves & Their Affects on the LSAT

Postby mst » Fri Jun 11, 2010 2:39 pm

Was going for temptatious, but it got spell corrected...sorry broha. And honestly, I still can't get the effects/affects thing straight, I always gloss over that mistake, but thank you for pointing that out. I will mosey on down to da local commoontitty college right away sir' so I can learns me some proppa's grammEr for the spider-webs! Afta's alls, we all know the schnoodle be gittin upset every time he see someones use a words wrong! Best not let him nears that 4th LSAT Logic game thread!!!

sumus romani
Posts: 565
Joined: Fri Jan 15, 2010 6:04 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby sumus romani » Fri Jun 11, 2010 2:46 pm

Just a small point, but sometimes one answer difference can be two actual point differences. Someimes in the mid 170's, there is a gap, which no one can score (so, just as an illustration, -8 is a 174 and -7 is a 176). A free sample LSAT from LSAC has this feature, as does the Oct 09 test, believe (and some other tests too).

User avatar
citrustang
Posts: 110
Joined: Sun Jul 12, 2009 5:22 pm

Re: Smaller Curves & Their Affects on the LSAT

Postby citrustang » Fri Jun 11, 2010 2:54 pm

suspicious android wrote:Here's my solution: let people look ahead or back in their booklet, let them bubble in answers from previous sections. Start up section one, have the proctor say "you have one hour, forty-five minutes to complete the next three sections." After that, they collect your booklets and before you start section 4 & 5, you get a new booklet with only two sections. You have 70 minutes to complete that section. This would reward people who are perfect in one section and have the time management skills to make sure they get through the whole thing, which seems like it'd be a valuable skill to test. They might need to make some material a bit harder because I think this would intend to increase the scores for people who are already in the 170's, but that shouldn't be too hard to adjust.

This is a really self-serving suggestion, since I finished my two LR sections with about 5 minutes left and spent that time furious that I couldn't look back at my LG section. But I think it really could mitigate the effect of one crazy question/LG costing someone 5-6 points, which doesn't seem to me to be a sensible way to measure people's ability. Also, for people who have already hit the ceiling in one section (that is, they can almost always get -0 in significantly less than 35 minutes), the LSAT isn't measuring that increased ability compared to someone who just barely gets all the questions right in 35 minutes. This change could address that.

This is a really interesting idea.

mst
Posts: 925
Joined: Sun Jun 06, 2010 3:01 am

Re: Smaller Curves & Their Affects on the LSAT

Postby mst » Fri Jun 11, 2010 3:09 pm

citrustang wrote:
suspicious android wrote:Here's my solution: let people look ahead or back in their booklet, let them bubble in answers from previous sections. Start up section one, have the proctor say "you have one hour, forty-five minutes to complete the next three sections." After that, they collect your booklets and before you start section 4 & 5, you get a new booklet with only two sections. You have 70 minutes to complete that section. This would reward people who are perfect in one section and have the time management skills to make sure they get through the whole thing, which seems like it'd be a valuable skill to test. They might need to make some material a bit harder because I think this would intend to increase the scores for people who are already in the 170's, but that shouldn't be too hard to adjust.

This is a really self-serving suggestion, since I finished my two LR sections with about 5 minutes left and spent that time furious that I couldn't look back at my LG section. But I think it really could mitigate the effect of one crazy question/LG costing someone 5-6 points, which doesn't seem to me to be a sensible way to measure people's ability. Also, for people who have already hit the ceiling in one section (that is, they can almost always get -0 in significantly less than 35 minutes), the LSAT isn't measuring that increased ability compared to someone who just barely gets all the questions right in 35 minutes. This change could address that.

This is a really interesting idea.


It is a very unique idea, and it has a lot of merit. Plenty of cons though: People don't have the same 3 sections first, so those who have certain forms will be at a distinct advantage depending on their circumstances. I think that this could be avoided in some regards if they ensured everyone had the same first 3 sections, just in different order.

At the very least, this clears up peoples ability to cheat in many circumstances, and even clears up much of their motivation. The problem with score differentials still be rather meaningless yet critically important would still exist.




Return to “LSAT Prep and Discussion Forum”

Who is online

Users browsing this forum: Google [Bot], packerboy31489, PresidentIJohnson and 9 guests