Why not just use the raw scores? (Now: Why not scale 0-100?)

User avatar
USAO-vet
Posts: 53
Joined: Thu May 26, 2011 6:52 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby USAO-vet » Mon Dec 09, 2013 10:18 pm

Nova wrote:
USAO-vet wrote:
Nova wrote:
USAO-vet wrote:The real question is, why not give you the fucking raw score immediately, pending the actual score being curved? At least that way one would have a decent perspective on how they've done. I mean, fuck, it's a god damn Scranton -- run that shit and give me some feedback.

the lsat isn't curved

its equated before the test is administered


Equated based on what, LSAC's guess? How about the questions that get tossed? I find it pretty fucking difficult to believe that they don't analyze difficultly and determine a curve based about how the actual test takers do? If true, that's quite retarded.

see SOJ's post in the link i posted. Its not retarded because they already know how difficult the test is based on prior administrations. they can still tweak the equated scores after the administration if something seems really off.


That a good point. If they have already seen how people did on the questions when they were given as experimentals LSAC can make a decent assessment on difficulty.

User avatar
ScottRiqui
Posts: 3640
Joined: Mon Nov 29, 2010 8:09 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby ScottRiqui » Mon Dec 09, 2013 10:19 pm

USAO-vet wrote:
Nova wrote:
USAO-vet wrote:The real question is, why not give you the fucking raw score immediately, pending the actual score being curved? At least that way one would have a decent perspective on how they've done. I mean, fuck, it's a god damn Scranton -- run that shit and give me some feedback.

the lsat isn't curved

its equated before the test is administered


Equated based on what, LSAC's guess? How about the questions that get tossed? I find it pretty fucking difficult to believe that they don't analyze difficultly and determine a curve based about how the actual test takers do? If true, that's quite retarded.


All of the questions appeared previously in experimental sections, so LSAC knows how various types of scorers did on each question. This allows them to create a new test already knowing how to equate it so that scaled scores will be comparable across different administrations.

I'm not sure what causes a question to be removed from scoring, but it doesn't happen very often. Possibly the questions are changed inconsequentially between when they're experimental and when they're "actual", and sometimes they inadvertently screw up something in the process?

User avatar
lawschool22
Posts: 3875
Joined: Thu Jul 25, 2013 5:47 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby lawschool22 » Mon Dec 09, 2013 10:19 pm

USAO-vet wrote:
That a good point. If they have already seen how people did on the questions when they were given as experimentals LSAC can make a decent extremely precise assessment on difficulty.


ftfy

User avatar
lawschool22
Posts: 3875
Joined: Thu Jul 25, 2013 5:47 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby lawschool22 » Mon Dec 09, 2013 10:20 pm

ScottRiqui wrote:
USAO-vet wrote:
Nova wrote:
USAO-vet wrote:The real question is, why not give you the fucking raw score immediately, pending the actual score being curved? At least that way one would have a decent perspective on how they've done. I mean, fuck, it's a god damn Scranton -- run that shit and give me some feedback.

the lsat isn't curved

its equated before the test is administered


Equated based on what, LSAC's guess? How about the questions that get tossed? I find it pretty fucking difficult to believe that they don't analyze difficultly and determine a curve based about how the actual test takers do? If true, that's quite retarded.


All of the questions appeared previously in experimental sections, so LSAC knows how various types of scorers did on each question. This allows them to create a new test already knowing how to equate it so that scaled scores will be comparable across different administrations.

I'm not sure what causes a question to be removed from scoring, but it doesn't happen very often. Possibly the questions are changed inconsequentially between when they're experimental and when they're "actual", and sometimes they inadvertently screw up something in the process?


I think also if a test taker contests a certain question and there is credence to the claim they can be removed. I'm sure this is very very rare though.

User avatar
USAO-vet
Posts: 53
Joined: Thu May 26, 2011 6:52 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby USAO-vet » Mon Dec 09, 2013 10:24 pm

lawschool22 wrote:
USAO-vet wrote:
That a good point. If they have already seen how people did on the questions when they were given as experimentals LSAC can make a decent extremely precise assessment on difficulty.


ftfy


8===D :o

User avatar
ScottRiqui
Posts: 3640
Joined: Mon Nov 29, 2010 8:09 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby ScottRiqui » Mon Dec 09, 2013 10:25 pm

lawschool22 wrote:
USAO-vet wrote:
That a good point. If they have already seen how people did on the questions when they were given as experimentals LSAC can make a decent extremely precise assessment on difficulty.


ftfy


I don't know that I'd hang my hat on "extremely precise". Do we really know that the curve isn't adjusted, perhaps significantly, after the test is administered? For all we know, they may think it's going to be a "-9 test" when they write it, but it ends up being a "-12" by the time scores are released.
Last edited by ScottRiqui on Mon Dec 09, 2013 10:27 pm, edited 1 time in total.

User avatar
lawschool22
Posts: 3875
Joined: Thu Jul 25, 2013 5:47 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby lawschool22 » Mon Dec 09, 2013 10:26 pm

USAO-vet wrote:
lawschool22 wrote:
USAO-vet wrote:
That a good point. If they have already seen how people did on the questions when they were given as experimentals LSAC can make a decent extremely precise assessment on difficulty.


ftfy


8===D :o


Come on...all in good fun

User avatar
USAO-vet
Posts: 53
Joined: Thu May 26, 2011 6:52 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby USAO-vet » Mon Dec 09, 2013 10:27 pm

lawschool22 wrote:
USAO-vet wrote:
lawschool22 wrote:
USAO-vet wrote:
That a good point. If they have already seen how people did on the questions when they were given as experimentals LSAC can make a decent extremely precise assessment on difficulty.


ftfy


8===D :o


Come on...all in good fun


Same here, bro.

User avatar
lawschool22
Posts: 3875
Joined: Thu Jul 25, 2013 5:47 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby lawschool22 » Mon Dec 09, 2013 10:29 pm

ScottRiqui wrote:
lawschool22 wrote:
USAO-vet wrote:
That a good point. If they have already seen how people did on the questions when they were given as experimentals LSAC can make a decent extremely precise assessment on difficulty.


ftfy


I don't know that I'd hang my hat on "extremely precise". Do we really know that the curve isn't adjusted, perhaps significantly, after the test is administered?


I have skimmed some of LSAC's technical publications on the methodology, and it seems that they have a pretty sophisticated understanding of the relative difficulty of a given question. But someone with a better knowledge of stats could probably chime in. We're probably splitting hairs, but I just wanted to make the point that they really do get a huge sample size for a given question and can ascertain pretty well the likelihood that a 150 scorer vs 160 scorer vs 170 scorer will get it correct.

I would be surprised if they curve it after the fact, but I guess it could happen if they saw something really screwy with the distribution of correct vs incorrect answers scaled by score.

User avatar
ScottRiqui
Posts: 3640
Joined: Mon Nov 29, 2010 8:09 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby ScottRiqui » Mon Dec 09, 2013 10:36 pm

lawschool22 wrote:
ScottRiqui wrote:
lawschool22 wrote:
USAO-vet wrote:
That a good point. If they have already seen how people did on the questions when they were given as experimentals LSAC can make a decent extremely precise assessment on difficulty.


ftfy


I don't know that I'd hang my hat on "extremely precise". Do we really know that the curve isn't adjusted, perhaps significantly, after the test is administered?


I have skimmed some of LSAC's technical publications on the methodology, and it seems that they have a pretty sophisticated understanding of the relative difficulty of a given question. But someone with a better knowledge of stats could probably chime in. We're probably splitting hairs, but I just wanted to make the point that they really do get a huge sample size for a given question and can ascertain pretty well the likelihood that a 150 scorer vs 160 scorer vs 170 scorer will get it correct.

I would be surprised if they curve it after the fact, but I guess it could happen if they saw something really screwy with the distribution of correct vs incorrect answers scaled by score.


True, and I momentarily forgot just how damn many people take the LSAT every year. With tens of thousands of people taking each question as an experimental, they probably do have a good idea of how people will do when they see it as a "real" question.

User avatar
midwest17
Posts: 1686
Joined: Sat Aug 31, 2013 5:27 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby midwest17 » Mon Dec 09, 2013 10:37 pm

ScottRiqui wrote:
lawschool22 wrote:
USAO-vet wrote:
That a good point. If they have already seen how people did on the questions when they were given as experimentals LSAC can make a decent extremely precise assessment on difficulty.


ftfy


I don't know that I'd hang my hat on "extremely precise". Do we really know that the curve isn't adjusted, perhaps significantly, after the test is administered? For all we know, they may think it's going to be a "-9 test" when they write it, but it ends up being a "-12" by the time scores are released.


Given that every section has appeared as a complete experimental (not just the questions have appeared in experimentals) it's not clear to me why you would think the distribution on test day is more informative than the info they have when they write it.

User avatar
ScottRiqui
Posts: 3640
Joined: Mon Nov 29, 2010 8:09 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby ScottRiqui » Mon Dec 09, 2013 10:40 pm

On a related note, I remember after the October test, LSAC released the number of cancellations fairly early on in the waiting period. Assuming that includes the takers who canceled by filling in the two "cancellation" bubbles on the form, that means that LSAC has scanned the forms pretty shortly after everyone's done taking the test. I'm really curious what goes on during the rest of the waiting period.

User avatar
ScottRiqui
Posts: 3640
Joined: Mon Nov 29, 2010 8:09 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby ScottRiqui » Mon Dec 09, 2013 10:43 pm

midwest17 wrote:
ScottRiqui wrote:
lawschool22 wrote:
USAO-vet wrote:
That a good point. If they have already seen how people did on the questions when they were given as experimentals LSAC can make a decent extremely precise assessment on difficulty.


ftfy


I don't know that I'd hang my hat on "extremely precise". Do we really know that the curve isn't adjusted, perhaps significantly, after the test is administered? For all we know, they may think it's going to be a "-9 test" when they write it, but it ends up being a "-12" by the time scores are released.


Given that every section has appeared as a complete experimental (not just the questions have appeared in experimentals) it's not clear to me why you would think the distribution on test day is more informative than the info they have when they write it.


Ah, I didn't realize that entire sections made the transition from "experimental" to "actual" intact.

User avatar
lawschool22
Posts: 3875
Joined: Thu Jul 25, 2013 5:47 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby lawschool22 » Mon Dec 09, 2013 10:51 pm

ScottRiqui wrote:
Ah, I didn't realize that entire sections made the transition from "experimental" to "actual" intact.


Yep that's definitely an important distinction.

User avatar
midwest17
Posts: 1686
Joined: Sat Aug 31, 2013 5:27 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby midwest17 » Mon Dec 09, 2013 10:54 pm

lawschool22 wrote:
ScottRiqui wrote:
Ah, I didn't realize that entire sections made the transition from "experimental" to "actual" intact.


Yep that's definitely an important distinction.


I can't remember where I read that this happens, but it was an official looking document. Basically, there are two kinds of experimental sections. The first is just trying out questions (or games or passages), with no expectation that they will form a coherent unit. The other is trying out (hopefully) finished sections, to make sure they work together and figure out how they should impact the curve when they're included in an actual test.

notalobbyist
Posts: 124
Joined: Sun Jul 14, 2013 1:25 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby notalobbyist » Mon Dec 09, 2013 11:36 pm

They should use the SAT 1600 number scale.

User avatar
ScottRiqui
Posts: 3640
Joined: Mon Nov 29, 2010 8:09 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby ScottRiqui » Tue Dec 10, 2013 12:01 am

notalobbyist wrote:They should use the SAT 1600 number scale.


With a little bit of jiggering, they basically are. Each SAT section goes from 200-800, in ten-point increments. Lop off the trailing zero and tack on a leading 1, and you get 120-180.

User avatar
midwest17
Posts: 1686
Joined: Sat Aug 31, 2013 5:27 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby midwest17 » Tue Dec 10, 2013 12:03 am

notalobbyist wrote:They should use the SAT 1600 number scale.


You're clearly old.

User avatar
Nova
Posts: 9116
Joined: Sun Apr 15, 2012 8:55 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby Nova » Tue Dec 10, 2013 12:05 am

ScottRiqui wrote:
notalobbyist wrote:They should use the SAT 1600 number scale.


With a little bit of jiggering, they basically are. Each SAT section goes from 200-800, in ten-point increments. Lop off the trailing zero and tack on a leading 1, and you get 120-180.

GMAT is 200-800 too

User avatar
ScottRiqui
Posts: 3640
Joined: Mon Nov 29, 2010 8:09 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby ScottRiqui » Tue Dec 10, 2013 12:12 am

Nova wrote:
ScottRiqui wrote:
notalobbyist wrote:They should use the SAT 1600 number scale.


With a little bit of jiggering, they basically are. Each SAT section goes from 200-800, in ten-point increments. Lop off the trailing zero and tack on a leading 1, and you get 120-180.

GMAT is 200-800 too


And that's in ten-point increments too, right? I wonder if there's something statistically useful about having 61 discrete possible scores?

User avatar
Nova
Posts: 9116
Joined: Sun Apr 15, 2012 8:55 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby Nova » Tue Dec 10, 2013 12:13 am

ScottRiqui wrote:
Nova wrote:
ScottRiqui wrote:
notalobbyist wrote:They should use the SAT 1600 number scale.


With a little bit of jiggering, they basically are. Each SAT section goes from 200-800, in ten-point increments. Lop off the trailing zero and tack on a leading 1, and you get 120-180.

GMAT is 200-800 too


And that's in ten-point increments too, right? I wonder if there's something statistically useful about having 61 discrete possible scores?

yeah. hmm idk


GRE was 200-800 by 10pts too but they recently changed it.

tomwatts
Posts: 1551
Joined: Wed Sep 16, 2009 12:01 am

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby tomwatts » Tue Dec 10, 2013 2:15 am

The claim is that a 120-180 scale makes small differences look small and big differences look big. So if you say you got a 155 as opposed to a 150, that sounds like a modestly higher score, but if you say you got a 35 as opposed to a 30, it sounds like a bigger difference.

I'm pretty sure that that's nonsense, but that's what the GRE people said when they switched to a 130-170 scale.

User avatar
neprep
Posts: 1066
Joined: Fri Jul 26, 2013 11:16 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby neprep » Tue Dec 10, 2013 3:36 am

There's actually a small section of the PowerScore bible that addresses this issue. From what I remember, the 0-100 or 0-60 scale is discarded for cosmetic purposes: It sounds better to say "this student received a 125" than it does to say "this student received a 5." Although I think even David Killoran is speculating.

User avatar
JuTMSY4
Posts: 265
Joined: Sat Jun 27, 2009 3:40 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby JuTMSY4 » Tue Dec 10, 2013 3:35 pm

midwest17 wrote:
notalobbyist wrote:They should use the SAT 1600 number scale.


You're clearly old.


Are people now graduating from college who took the new 2400 point SAT? When I hear a new SAT score now, I'm either super impressed (1590) when I shouldn't be, or utterly confused (2200)

User avatar
Bildungsroman
Posts: 5548
Joined: Sun Apr 11, 2010 2:42 pm

Re: Why not just use the raw scores? (Now: Why not scale 0-100?)

Postby Bildungsroman » Tue Dec 10, 2013 3:44 pm

JuTMSY4 wrote:
midwest17 wrote:
notalobbyist wrote:They should use the SAT 1600 number scale.


You're clearly old.


Are people now graduating from college who took the new 2400 point SAT?

I'm a 3L and I took the 2400-point SAT in high school.




Return to “LSAT Prep and Discussion Forum”

Who is online

Users browsing this forum: Baidu [Spider], ebmus11, freekick and 10 guests