it seems like exams are not really equated

User avatar
swtlilsoni
Posts: 399
Joined: Tue Jun 09, 2009 1:00 am

it seems like exams are not really equated

Postby swtlilsoni » Thu Dec 08, 2011 12:58 am

I just realized something.

When a test is "harder" it causes people to screw up on a good number of problems. For example on the dec test plenty of people screwed up on 4+ more problems than usual with either LG or RC or both. However when lsac "makes up" for a harder test by curving (equating) they only give us like two or three points extra. But this doesn't really make up for the extra difficulty because the difficulty caused most people to miss more than that. Thus people's scores do drop because of difficulty. So a 175 in a harder test isn't really equal to a 175 on an easier test because it's simply harder to get.
Last edited by swtlilsoni on Thu Dec 08, 2011 1:47 am, edited 1 time in total.

09042014
Posts: 18282
Joined: Wed Oct 14, 2009 10:47 pm

Re: it seems like exams are not really equated

Postby 09042014 » Thu Dec 08, 2011 1:03 am

swtlilsoni wrote:I just realized something.

When a test is "harder" it causes people to screw up on a good number of problems. For example on the dec test plenty of people screwed up on 4+ more problems than usual with either LG or RC or both. However when lsac "makes up" for a harder test by curving (equating) they only give us like two or three points extra. But this doesn't really make up for the extra difficulty because the difficulty caused most people to miss more than that.

For example say someone who consistently scores gets -0 on LG and LR and -1 on RC takes a June exam and gets -1 on RC -0 on LG and LR (with a -9 curve). Let's say they ended up with a 180. Then say they take a dec exam which ends up having an insanely hard RC and LG but equal LR. They get -4 on RC and -1 on LG (with a -13 curve). So they end up with below 180 (177 or 178).
How is this equated? A 180 in dec clearly is not equal to a 180 in June because it's harder to get (because the curve did not accurately make up for the change in difficulty).


You presumed they don't to prove they don't. What a shitty fucking post using shitty logic.

User avatar
swtlilsoni
Posts: 399
Joined: Tue Jun 09, 2009 1:00 am

Re: it seems like exams are not really equated

Postby swtlilsoni » Thu Dec 08, 2011 1:05 am

What are you talking about? I'm giving you an example in which it isn't.


Desert Fox wrote:
swtlilsoni wrote:I just realized something.

When a test is "harder" it causes people to screw up on a good number of problems. For example on the dec test plenty of people screwed up on 4+ more problems than usual with either LG or RC or both. However when lsac "makes up" for a harder test by curving (equating) they only give us like two or three points extra. But this doesn't really make up for the extra difficulty because the difficulty caused most people to miss more than that.

For example say someone who consistently scores gets -0 on LG and LR and -1 on RC takes a June exam and gets -1 on RC -0 on LG and LR (with a -9 curve). Let's say they ended up with a 180. Then say they take a dec exam which ends up having an insanely hard RC and LG but equal LR. They get -4 on RC and -1 on LG (with a -13 curve). So they end up with below 180 (177 or 178).
How is this equated? A 180 in dec clearly is not equal to a 180 in June because it's harder to get (because the curve did not accurately make up for the change in difficulty).


You presumed they don't to prove they don't. What a shitty fucking post using shitty logic.

09042014
Posts: 18282
Joined: Wed Oct 14, 2009 10:47 pm

Re: it seems like exams are not really equated

Postby 09042014 » Thu Dec 08, 2011 1:09 am

Because a test that causes 170 level test takers (which is how TLS defines "the curve") to get 4 more wrong, doesn't cause a 180 level test taker to get 4 more.

Which is why when you look at the number wrong for 180 its always from 0-2, and usually 1.

Those extra hard questions aren't really all that much harder for someone who gets them all right.

User avatar
swtlilsoni
Posts: 399
Joined: Tue Jun 09, 2009 1:00 am

Re: it seems like exams are not really equated

Postby swtlilsoni » Thu Dec 08, 2011 1:17 am

Desert Fox wrote:Because a test that causes 170 level test takers (which is how TLS defines "the curve") to get 4 more wrong, doesn't cause a 180 level test taker to get 4 more.

Which is why when you look at the number wrong for 180 its always from 0-2, and usually 1.

Those extra hard questions aren't really all that much harder for someone who gets them all right.



hm okay that makes sense. But then wouldn't the same apply to a 170 level taker? They get 4 more wrong but the "equating" gives less points back so they end up with a lower score due to difficulty?

tng11
Posts: 95
Joined: Sun Nov 14, 2010 6:37 pm

Re: it seems like exams are not really equated

Postby tng11 » Thu Dec 08, 2011 1:48 am

...
Last edited by tng11 on Thu Jan 05, 2012 10:23 am, edited 1 time in total.

baaron008
Posts: 35
Joined: Sat Dec 03, 2011 3:36 pm

Re: it seems like exams are not really equated

Postby baaron008 » Thu Dec 08, 2011 2:11 am

tng11 wrote:I don't get the point of your post.

The way I understand it, is that LSAC equates scores and sets the curves by matching the number of questions a given test taker at each score gets wrong from the experimental sections, then sets a raw score that is required to achieve that score. They don't set a linear scale from an arbitrary score (e.g. 170) which is probably what you're thinking of. For 180 test takers, because they have no problem with more difficult questions, their raw score doesn't change very often, they've mastered the material to a point where any error might just be a misbubble or a missed word. As you slide down the scale, there starts to be more variation, as a 170 test taker may be tripped up by some more nuanced/tricky problems, which the 180 test taker assumedly has no problem with. Hence, as the number of hard problems in any LSAT fluctuates, the raw scores tend to jump all over the place.


What this guy said. They are definitely equated, otherwise more people would have wildly different min and max scores on PTS because there is no way that LSAC can make every test equally difficult unless they gave the exact same test over and over.

User avatar
swtlilsoni
Posts: 399
Joined: Tue Jun 09, 2009 1:00 am

Re: it seems like exams are not really equated

Postby swtlilsoni » Thu Dec 08, 2011 2:13 am

tng11 wrote:I don't get the point of your post.

The way I understand it, is that LSAC equates scores and sets the curves by matching the number of questions a given test taker at each score gets wrong from the experimental sections, then sets a raw score that is required to achieve that score. They don't set a linear scale from an arbitrary score (e.g. 170) which is probably what you're thinking of. For 180 test takers, because they have no problem with more difficult questions, their raw score doesn't change very often, they've mastered the material to a point where any error might just be a misbubble or a missed word. As you slide down the scale, there starts to be more variation, as a 170 test taker may be tripped up by some more nuanced/tricky problems, which the 180 test taker assumedly has no problem with. Hence, as the number of hard problems in any LSAT fluctuates, the raw scores tend to jump all over the place.

I'm having trouble explaining what I mean. But what I'm trying to say is that generally when a test is difficult people miss more than what is accounted for by the curve so scores do end up dropping. Sometimes it's -9 to get 170 and if it's difficult they'll change it to -12. However the difficulty caused a good amount of people to miss MORE than just three problems. So some people will drop below 170 when they wouldn't have otherwise. Thus the difficulty made it harder to get a 170 (it caused people to miss more and the "curve" did not account for all those that were missed.

On this past exam many people are complaining that they guessed on many more problems than usual. They hope for a generous "curve"/"equating" to make up for that. However gaining 1 or 2 points leniency by the "curve" is not going to make up for all those problems they guessed on. So their score will still be lower than usual.

tng11
Posts: 95
Joined: Sun Nov 14, 2010 6:37 pm

Re: it seems like exams are not really equated

Postby tng11 » Thu Dec 08, 2011 2:33 am

...
Last edited by tng11 on Thu Jan 05, 2012 10:23 am, edited 2 times in total.

Flanker1067
Posts: 658
Joined: Tue Dec 29, 2009 12:47 pm

Re: it seems like exams are not really equated

Postby Flanker1067 » Thu Dec 08, 2011 2:35 am

swtlilsoni wrote:
tng11 wrote:I don't get the point of your post.

The way I understand it, is that LSAC equates scores and sets the curves by matching the number of questions a given test taker at each score gets wrong from the experimental sections, then sets a raw score that is required to achieve that score. They don't set a linear scale from an arbitrary score (e.g. 170) which is probably what you're thinking of. For 180 test takers, because they have no problem with more difficult questions, their raw score doesn't change very often, they've mastered the material to a point where any error might just be a misbubble or a missed word. As you slide down the scale, there starts to be more variation, as a 170 test taker may be tripped up by some more nuanced/tricky problems, which the 180 test taker assumedly has no problem with. Hence, as the number of hard problems in any LSAT fluctuates, the raw scores tend to jump all over the place.

I'm having trouble explaining what I mean. But what I'm trying to say is that generally when a test is difficult people miss more than what is accounted for by the curve so scores do end up dropping. Sometimes it's -9 to get 170 and if it's difficult they'll change it to -12. However the difficulty caused a good amount of people to miss MORE than just three problems. So some people will drop below 170 when they wouldn't have otherwise. Thus the difficulty made it harder to get a 170 (it caused people to miss more and the "curve" did not account for all those that were missed.

On this past exam many people are complaining that they guessed on many more problems than usual. They hope for a generous "curve"/"equating" to make up for that. However gaining 1 or 2 points leniency by the "curve" is not going to make up for all those problems they guessed on. So their score will still be lower than usual.



lulz. Is your point that different people will do differently on different exams? Great Job!

The "curve" DOES account for the amount more that people miss, THAT IS HOW THEY SET IT.

hotchkiss1
Posts: 26
Joined: Sun Dec 04, 2011 12:48 pm

Re: it seems like exams are not really equated

Postby hotchkiss1 » Thu Dec 08, 2011 2:45 am

of course not, it's 67% accurate at predicting first year grade, the margin of error is significant :idea:

User avatar
swtlilsoni
Posts: 399
Joined: Tue Jun 09, 2009 1:00 am

Re: it seems like exams are not really equated

Postby swtlilsoni » Thu Dec 08, 2011 2:47 am

My point is not that different people do differently, it's the the same person would do differently.



Flanker1067 wrote:
swtlilsoni wrote:
tng11 wrote:I don't get the point of your post.

The way I understand it, is that LSAC equates scores and sets the curves by matching the number of questions a given test taker at each score gets wrong from the experimental sections, then sets a raw score that is required to achieve that score. They don't set a linear scale from an arbitrary score (e.g. 170) which is probably what you're thinking of. For 180 test takers, because they have no problem with more difficult questions, their raw score doesn't change very often, they've mastered the material to a point where any error might just be a misbubble or a missed word. As you slide down the scale, there starts to be more variation, as a 170 test taker may be tripped up by some more nuanced/tricky problems, which the 180 test taker assumedly has no problem with. Hence, as the number of hard problems in any LSAT fluctuates, the raw scores tend to jump all over the place.

I'm having trouble explaining what I mean. But what I'm trying to say is that generally when a test is difficult people miss more than what is accounted for by the curve so scores do end up dropping. Sometimes it's -9 to get 170 and if it's difficult they'll change it to -12. However the difficulty caused a good amount of people to miss MORE than just three problems. So some people will drop below 170 when they wouldn't have otherwise. Thus the difficulty made it harder to get a 170 (it caused people to miss more and the "curve" did not account for all those that were missed.

On this past exam many people are complaining that they guessed on many more problems than usual. They hope for a generous "curve"/"equating" to make up for that. However gaining 1 or 2 points leniency by the "curve" is not going to make up for all those problems they guessed on. So their score will still be lower than usual.



lulz. Is your point that different people will do differently on different exams? Great Job!

The "curve" DOES account for the amount more that people miss, THAT IS HOW THEY SET IT.

ahnhub
Posts: 578
Joined: Fri Jun 10, 2011 3:14 pm

Re: it seems like exams are not really equated

Postby ahnhub » Thu Dec 08, 2011 2:49 am

The problem is you are applying anecdotal evidence. You are saying that a "hard" test, on average, will cause a test-taker to really struggle or have to guess on 4-5 questions more than they would have had on a "normal" test, and yet the curve will only allow 2-3 more wrong questions for the same score. But this is largely a perception based on the data available to you, which is people talking on TLS forums. There will be others who struggled or guessed on less than 4-5 questions. And remember if a person has to guess on five questions, they are actually 70% likely to get one right.

I do agree that psychologically it seems that a very difficult test should be more lenient. December 2010 (-15 curve for 170) was an absolute monster (2 ridiculously hard games, fairly challenging RC, medium LR with a few whoppers thrown in), while June 2011 (-11 curve, easy games, hard RC, easy LR) seemed pretty easy in comparison; and the 4-point difference doesn't seem to account for it psychologically. But it generally does, at the end of the day.

I will say this: depending on what your shortcomings are, there probably are types of people who will do better on an "easy curve" or "hard curve" test. If you are extremely accurate in reading and thinking (you never make those stupid reading mistakes or assumptions that cause you to go, "duh" when you find out the answer) you will fair well on an "easy curve" test, because you will knock all of the straightforward questions out of the park. If you are a little sloppier, and tend to make those kinds of mistakes, but have excellent comprehension skills (you can think deeply into a problem), you will probably be best served with a "hard curve" test, because you can get most of the hard questions and will be punished less for the dumb mistakes you always make.

ahnhub
Posts: 578
Joined: Fri Jun 10, 2011 3:14 pm

Re: it seems like exams are not really equated

Postby ahnhub » Thu Dec 08, 2011 2:51 am

of course not, it's 67% accurate at predicting first year grade, the margin of error is significant


Whoa, 67% is ridiculously high. I've seen: LSAT+GPA together creates an index which accounts for only 20% of your total grade.

User avatar
rinkrat19
Posts: 13915
Joined: Sat Sep 25, 2010 5:35 am

Re: it seems like exams are not really equated

Postby rinkrat19 » Thu Dec 08, 2011 2:56 am

OP also seems to be ignoring the fact that the "curve" is set by using the real scores that real people got doing those sections as experimentals. LSAC's not just guessing on how hard they think the test is and arbitrarily assigning it a number between -8 and -14. Real people did those sections on past tests, the sections were scored, and the curve was set to make the median score a 151. Making sure that -X number works the same for a 140-scorer, a 165-scorer, and a 178-scorer (hint: it doesn't, because they are in totally different places taking the test, as others have already explained) is not really the point of the curve. It's to set the median at 151, and nothing else, so that scores from different testing days/years can be easily compared.

09042014
Posts: 18282
Joined: Wed Oct 14, 2009 10:47 pm

Re: it seems like exams are not really equated

Postby 09042014 » Thu Dec 08, 2011 3:27 am

swtlilsoni wrote:
Desert Fox wrote:Because a test that causes 170 level test takers (which is how TLS defines "the curve") to get 4 more wrong, doesn't cause a 180 level test taker to get 4 more.

Which is why when you look at the number wrong for 180 its always from 0-2, and usually 1.

Those extra hard questions aren't really all that much harder for someone who gets them all right.



hm okay that makes sense. But then wouldn't the same apply to a 170 level taker? They get 4 more wrong but the "equating" gives less points back so they end up with a lower score due to difficulty?


The equating figures out roughly how many more they got wrong. You are assuming, for no reason, they get less.

User avatar
Ludo!
Posts: 4764
Joined: Wed Nov 09, 2011 1:22 pm

Re: it seems like exams are not really equated

Postby Ludo! » Thu Dec 08, 2011 3:45 am

I just realized that every time somebody starts a post with "I just realized something" the something was pulled out of their ass

User avatar
Jeffort
Posts: 1896
Joined: Wed Jun 18, 2008 4:43 pm

Re: it seems like exams are not really equated

Postby Jeffort » Thu Dec 08, 2011 4:03 am

OP: One of the many skills the LSAT is designed to measure is time management since it is a critical skill in order to perform well in law school.

If you camp out too long on a few difficult questions because you cannot determine the credited answer choice you get punished by not having time to analyze other questions in the section, thus sacrificing the opportunity to gain raw points by solving other possibly easier questions in the section.

Take note that there has NEVER been an LSAT where the SCALE required minus zero to obtain a perfect 180 scaled score.

Also people, please stop saying curve!! The LSAT is graded on an equated scale, not on a curve. Reading posts with the C word is like hearing fingernails scratching on a chalkboard. Image

Fyo'Couch
Posts: 136
Joined: Tue Dec 06, 2011 7:38 pm

Re: it seems like exams are not really equated

Postby Fyo'Couch » Thu Dec 08, 2011 10:26 am

I know what the OP is getting at, and I have often wondered about this as well. I think what you're trying to say is that although harder tests account for the difficulty within sections, its harder to gauge how a harder test affects a test-taker cumulatively, throughout the test. Presumably, the test-taker with a harder test is facing a psychological obstacle less present in easier test forms, which may or may not be accounted for in test-equating...although I am not sure how you would measure this.

Lets say a test taker is writing an expectedly challenging LR section, and gets exactly as many questions wrong as he should for his score-band. The test taker is unlikely to be aware that this was "supposed to be" a hard section, and it could very well assume that he bombed the section. Now, how does that affect his performance throughout the rest of the test? Is he now missing questions that the test has equated he should normally get because he is rattled from "bombing" the section? I honestly have no idea how you would account for something so arbitrary.

Personally, I've always been at odds with hoping for a harder curve BEFORE the test. From a psychological standpoint, I would much rather write a moderately easier test (Dec 11), maintaining confidence throughout, then writing a comparatively harder test (Dec 10) and dealing with the increased second-guessing and self doubt.

tropicanax
Posts: 56
Joined: Fri Jun 03, 2011 9:52 pm

Re: it seems like exams are not really equated

Postby tropicanax » Thu Dec 08, 2011 11:02 pm

i get what the OP is saying... like if a game is super hard and is very difficult to get the assumptions necesary to complete the questions the test taker will get flustered and possibly miss a lot of time to complete other questions or miss way more number of questions on that section than the easier curve accounts for (2 points or whatever). Like I usually get -0 wrong on LG, but on SuperPrep B i couldn't do an entire game on the trees in the park or whatever.. which obviously affected my score..

09042014
Posts: 18282
Joined: Wed Oct 14, 2009 10:47 pm

Re: it seems like exams are not really equated

Postby 09042014 » Thu Dec 08, 2011 11:43 pm

tropicanax wrote:i get what the OP is saying... like if a game is super hard and is very difficult to get the assumptions necesary to complete the questions the test taker will get flustered and possibly miss a lot of time to complete other questions or miss way more number of questions on that section than the easier curve accounts for (2 points or whatever). Like I usually get -0 wrong on LG, but on SuperPrep B i couldn't do an entire game on the trees in the park or whatever.. which obviously affected my score..


That isn't what OP meant.

User avatar
lovejopd
Posts: 548
Joined: Tue Dec 01, 2009 1:00 pm

Re: it seems like exams are not really equated

Postby lovejopd » Fri Dec 09, 2011 8:51 pm

rinkrat19 wrote:OP also seems to be ignoring the fact that the "curve" is set by using the real scores that real people got doing those sections as experimentals. LSAC's not just guessing on how hard they think the test is and arbitrarily assigning it a number between -8 and -14. Real people did those sections on past tests, the sections were scored, and the curve was set to make the median score a 151. Making sure that -X number works the same for a 140-scorer, a 165-scorer, and a 178-scorer (hint: it doesn't, because they are in totally different places taking the test, as others have already explained) is not really the point of the curve. It's to set the median at 151, and nothing else, so that scores from different testing days/years can be easily compared.

Wow really 151 median?...
Always been 151 from 1-65 prep test? or is it changed at some point?




Return to “LSAT Prep and Discussion Forum”

Who is online

Users browsing this forum: Bing [Bot], GioChanturia and 5 guests