Any LSAT questions written wrong and thrown out? Forum

[RZ5646]

Dear LSAT gurus,

Has anyone ever successfully challenged an LSAT question? If so, is there a list of such questions available somewhere? Seeing them and the related arguments could be beneficial.

[rinkrat19]

If you see a missing question number on a PT, that question was thrown out. It hasn't happened in the 4 years I've been watching, and to my knowledge they aren't published anywhere. There was one that people threw a hissy fit over in June 2012 and tried to get thrown out (a LR about copies of Shakespeare scripts), but it was logically sound; there was just a bunch of butthurt from people who got it wrong.

[Jeffort]

If you see a missing question number on a PT, that question was thrown out. It hasn't happened in the 4 years I've been watching, and to my knowledge they aren't published anywhere. There was one that people threw a hissy fit over in June 2012 and tried to get thrown out (a LR about copies of Shakespeare scripts), but it was logically sound; there was just a bunch of butthurt from people who got it wrong.

Do you mean the Shakespeare Hamlet First Quarto abridgement LR question - PT61 S4 #5?

I remember there was a lot of uhhm, hostile anger filled debate about that one due to the CR being extreme with absolute logical force 'WAS produced by an actor' while the facts in the stimulus don't come anywhere close to guaranteeing or strongly supporting that level of certainty nor does the stimulus evidence eliminate all or almost all other possibilities about who could have created the script.

When I first looked at that one I thought it was a flawed most strongly supported inference question that accidentally slipped through LSAC's quality control. Then I noticed that the question stem is a very rare inference LR Q type that has a much lower burden of proof/certainty-much more wiggle room for the CR since the stem doesn't say 'which of the follow is most STRONGLY supported?', but instead just asks 'which one of the following statements is most supported...?'

The CR on that one certainly wouldn't pass muster under the 'most strongly supported' question type burden of proof/level of logical certainty required for the CR, and I remember that lots of people on here that argued that it was a flawed question that should be thrown out didn't notice that the stem was different and it wasn't actually a MSS question.

RZ5646, I know of only one question (an LR question) that was removed from scoring but was available for people to see for a period of time when it was published by LSAC in an old batch (over 10 years ago) of one of the '10 LSAT's' books. I have an old copy of that '10 LSAT's' book with the question in it, but I don't remember which book or PT it's in off the top of my head right now to find it. I'm not sure if it was an LSAC mistake/oversight that got the question included in some old copies of that 10 book or if the question was deemed flawed and removed from scoring and the PT long after scores and the PT was first released. I'm pretty sure it's in a 1990's era PT. I'll try to find my copy with the question if I find some free time to dig through my boxes of old books this week. It's been a long time since I've looked at that question but I don't remember there being any significant issue in it that rendered it defective in terms of there being either two or zero AC's that logically satisfied the stem, but again it's been years since I've looked at it.

[foggynotion]

"Then I noticed that the question stem is a very rare inference LR Q type that has a much lower burden of proof/certainty-much more wiggle room for the CR since the stem doesn't say 'which of the follow is most STRONGLY supported?', but instead just asks 'which one of the following statements is most supported...?'

How so? Why do you feel that including the word "strongly' creates a greater burden of proof?

[rinkrat19]

"Then I noticed that the question stem is a very rare inference LR Q type that has a much lower burden of proof/certainty-much more wiggle room for the CR since the stem doesn't say 'which of the follow is most STRONGLY supported?', but instead just asks 'which one of the following statements is most supported...?'

How so? Why do you feel that including the word "strongly' creates a greater burden of proof?

To me, it doesn't, which is probably why I thought the answer was obvious.

(I didn't actually study formally for the LSAT, so I don't know 90% of the terms for stems and whatnot. I just did some PTs for a few weeks. Didn't realize until well after that I probably could have gotten several points higher than my 171 with a little effort.)

[ltowns1]

"Then I noticed that the question stem is a very rare inference LR Q type that has a much lower burden of proof/certainty-much more wiggle room for the CR since the stem doesn't say 'which of the follow is most STRONGLY supported?', but instead just asks 'which one of the following statements is most supported...?'

How so? Why do you feel that including the word "strongly' creates a greater burden of proof?

Because it has to do with how provable the answer choice is. If it said " most strongly supported" then the burden of proof is higher, which means that we have to find answer that's most provable. Which is often times less strongly worded language. Which is why you see the correct answer choices for this question type often beginning with "some" and "at least." A statement with those quantities is much more provable than words like "most" or "all"

On the other hand, when you have a question stem that says "most supported", that leaves more wiggle room for picking answers that may be broad in scope like "most" or "all".

SN: I'm trying to get better at internalizing concepts, so if I said something wrong, I'm happy to be corrected....but I think I'm correct lol.

[rinkrat19]

"Then I noticed that the question stem is a very rare inference LR Q type that has a much lower burden of proof/certainty-much more wiggle room for the CR since the stem doesn't say 'which of the follow is most STRONGLY supported?', but instead just asks 'which one of the following statements is most supported...?'

How so? Why do you feel that including the word "strongly' creates a greater burden of proof?

Because it has to do with how provable the answer choice is. If it said " most strongly supported" then the burden of proof is higher, which means that we have to find answer that's not strong because it's most provable. Which is why you see the correct answer choices often beginning with "some" and "at least." A statement with those quantities is much more provable than words like "most" or "all"

On the other hand, when you have a question stem that says "most supported", that leaves more wiggle roomy of pick answers that may be broad in scope like "most" or "all".

SN: I'm trying to get better at understanding this concepts, so if I said something wrong, I'm happy to be corrected....but I think I'm correct lol.

"most strongly" doesn't mean it has to be a perfect or even a strong answer. Just more strong than the others. Which is exactly what "most supports" means. They both only require you to pick the best answer relative to the other answers.

[appind]

http://www.top-law-schools.com/forums/v ... 6&t=216479

http://www.top-law-schools.com/forums/v ... 6&t=209540

[RZ5646]

I just skimmed this discussion, but wouldn't the "most supported" and the "most strongly supported" answer choice always be the same anyway? I don't see how someone could challenge a question based on "strongly," unless they claimed that none of the answer choices were strongly supported, but that would be stupid because obviously one of the answer choices is correct and "strongly supported."

The burden of proof is irrelevant because you're not judging the strength of a proposition's support in any absolute sense (i.e., whether it meets some criteria to be called "strong"), but in the relative sense of whether it is better or worse than the support for the other answer choices. An answer choice's support can be incredibly weak, but it will still be the "most strongly supported" if its support is less weak than any other answer choice. So any reasonable person should find "most supported" and "most strongly supported" interchangeable.

[ltowns1]

I just skimmed this discussion, but wouldn't the "most supported" and the "most strongly supported" answer choice always be the same anyway? I don't see how someone could challenge a question based on "strongly," unless they claimed that none of the answer choices were strongly supported, but that would be stupid because obviously one of the answer choices is correct and "strongly supported."

The burden of proof is irrelevant because you're not judging the strength of a proposition's support in any absolute sense (i.e., whether it meets some criteria to be called "strong"), but in the relative sense of whether it is better or worse than the support for the other answer choices. An answer choice's support can be incredibly weak, but it will still be the "most strongly supported" if its support is less weak than any other answer choice. So any reasonable person should find "most supported" and "most strongly supported" interchangeable.

I don't know it just seems like that would be so LSAT for them to have emphasize one word that changes the outlook on the entire question no???

[ltowns1]

"Then I noticed that the question stem is a very rare inference LR Q type that has a much lower burden of proof/certainty-much more wiggle room for the CR since the stem doesn't say 'which of the follow is most STRONGLY supported?', but instead just asks 'which one of the following statements is most supported...?'

How so? Why do you feel that including the word "strongly' creates a greater burden of proof?

Because it has to do with how provable the answer choice is. If it said " most strongly supported" then the burden of proof is higher, which means that we have to find answer that's not strong because it's most provable. Which is why you see the correct answer choices often beginning with "some" and "at least." A statement with those quantities is much more provable than words like "most" or "all"

On the other hand, when you have a question stem that says "most supported", that leaves more wiggle roomy of pick answers that may be broad in scope like "most" or "all".

SN: I'm trying to get better at understanding this concepts, so if I said something wrong, I'm happy to be corrected....but I think I'm correct lol.

"most strongly" doesn't mean it has to be a perfect or even a strong answer. Just more strong than the others. Which is exactly what "most supports" means. They both only require you to pick the best answer relative to the other answers.

"Most strongly" just means most provable, I don't disagree with that, but were there any answers that seemed more provable in relation to the other answer choices??

[rinkrat19]

"Then I noticed that the question stem is a very rare inference LR Q type that has a much lower burden of proof/certainty-much more wiggle room for the CR since the stem doesn't say 'which of the follow is most STRONGLY supported?', but instead just asks 'which one of the following statements is most supported...?'

How so? Why do you feel that including the word "strongly' creates a greater burden of proof?

Because it has to do with how provable the answer choice is. If it said " most strongly supported" then the burden of proof is higher, which means that we have to find answer that's not strong because it's most provable. Which is why you see the correct answer choices often beginning with "some" and "at least." A statement with those quantities is much more provable than words like "most" or "all"

On the other hand, when you have a question stem that says "most supported", that leaves more wiggle roomy of pick answers that may be broad in scope like "most" or "all".

SN: I'm trying to get better at understanding this concepts, so if I said something wrong, I'm happy to be corrected....but I think I'm correct lol.

"most strongly" doesn't mean it has to be a perfect or even a strong answer. Just more strong than the others. Which is exactly what "most supports" means. They both only require you to pick the best answer relative to the other answers.

"Most strongly" just means provable I don't disagree with that, but were there any answers that seemed more provable in relation to the other answer choices??

Like I said, I read it and the answer seemed obvious and I picked it.

Most strongly doesn't mean provable. It means more likely than the other choices.

[ltowns1]

"Then I noticed that the question stem is a very rare inference LR Q type that has a much lower burden of proof/certainty-much more wiggle room for the CR since the stem doesn't say 'which of the follow is most STRONGLY supported?', but instead just asks 'which one of the following statements is most supported...?'

How so? Why do you feel that including the word "strongly' creates a greater burden of proof?

Because it has to do with how provable the answer choice is. If it said " most strongly supported" then the burden of proof is higher, which means that we have to find answer that's not strong because it's most provable. Which is why you see the correct answer choices often beginning with "some" and "at least." A statement with those quantities is much more provable than words like "most" or "all"

On the other hand, when you have a question stem that says "most supported", that leaves more wiggle roomy of pick answers that may be broad in scope like "most" or "all".

SN: I'm trying to get better at understanding this concepts, so if I said something wrong, I'm happy to be corrected....but I think I'm correct lol.

"most strongly" doesn't mean it has to be a perfect or even a strong answer. Just more strong than the others. Which is exactly what "most supports" means. They both only require you to pick the best answer relative to the other answers.

"Most strongly" just means provable I don't disagree with that, but were there any answers that seemed more provable in relation to the other answer choices??

Like I said, I read it and the answer seemed obvious and I picked it.

Most strongly doesn't mean provable. It means more likely than the other choices.

I meant most provable, which to me is more likely than the other four.

[Kinky John]

(I didn't actually study formally for the LSAT, so I don't know 90% of the terms for stems and whatnot. I just did some PTs for a few weeks. Didn't realize until well after that I probably could have gotten several points higher than my 171 with a little effort.)

many humble such LSAT wow

[FloridaCoastalorbust]

i hate the lsat

[Clearly]

If you see a missing question number on a PT, that question was thrown out. It hasn't happened in the 4 years I've been watching, and to my knowledge they aren't published anywhere. There was one that people threw a hissy fit over in June 2012 and tried to get thrown out (a LR about copies of Shakespeare scripts), but it was logically sound; there was just a bunch of butthurt from people who got it wrong.

Actually June 2012 did have a question that was actually thrown out. It wasn't the Shakespeare one though. It was a LG could be true question in which 2 answer choices clearly could be true. It was thrown out before the test was even released so we didn't lose points for it, but I sure lost a lot of time redoing the whole game because "there's no way these are both right I def messed something up"

[rinkrat19]

If you see a missing question number on a PT, that question was thrown out. It hasn't happened in the 4 years I've been watching, and to my knowledge they aren't published anywhere. There was one that people threw a hissy fit over in June 2012 and tried to get thrown out (a LR about copies of Shakespeare scripts), but it was logically sound; there was just a bunch of butthurt from people who got it wrong.

Actually June 2012 did have a question that was actually thrown out. It wasn't the Shakespeare one though. It was a LG could be true question in which 2 answer choices clearly could be true. It was thrown out before the test was even released so we didn't lose points for it, but I sure lost a lot of time redoing the whole game because "there's no way these are both right I def messed something up"

Wow, I don't remember that at all.

[Clearly]

It was a 5 person game, matching or sequencing, game 2 I wanna say?

[foggynotion]

I'm looking at it the same way Rinkrat an RZ are--the two phrases mean same thing, which is that there is one answer choice that is supported more than the others.

This does not mean that there is strong support for any choice (RZ, wasn't sure if you agreed with this or not), only that there is more support for one choice than there is for the others.

[rinkrat19]

i hate the lsat

Now that I'm studying for the bar, the LSAT is a fond (distant) memory. This is SO MUCH WORSE.

[ltowns1]

i hate the lsat

Now that I'm studying for the bar, the LSAT is a fond (distant) memory. This is SO MUCH WORSE.

I believe you, my father said it was the hardest test he ever had to take. Good luck.

[rinkrat19]

i hate the lsat

Now that I'm studying for the bar, the LSAT is a fond (distant) memory. This is SO MUCH WORSE.

I believe you, my father said it was the hardest test he ever had to take.

The Fundamentals of Engineering exam was pretty terrible too, I have to say. But (A) it was only one day long (albeit an 8-hour day), and (B) majoring in engineering actually teaches you what you need to know to pass the FE. None of this spending thousands of dollars on a review course because it turns out that law school is useless bullshit.

[RZ5646]

I'm looking at it the same way Rinkrat an RZ are--the two phrases mean same thing, which is that there is one answer choice that is supported more than the others.

This does not mean that there is strong support for any choice (RZ, wasn't sure if you agreed with this or not), only that there is more support for one choice than there is for the others.

I think that there are two difference senses of "strong" in play. "Strong"-1 means having at least some determinate (and usually considerable) degree of strength, while "strong"-2 means having any strength at all. Obviously "strong"-2 is meant by the question, because otherwise we'd need to get a definition of "strong"-1 from somewhere; LSAC would need to tell us what minimal level of strength is needed to be considered "strong"-1.

So idk how test-takers got confused by this. "Strong"-1? Wtf does that mean? We have no criteria to decide whether something counts as "strong"-1 or not. It's all about whether the support is better than the support for any other answer choice. We're comparing answer choices to each other, not to some external strength scale we bring to the question from outside the test.

Analogous case:

Who's tallest? A. 5'0" B. 5'8" C. 5'9"

"None of them are 'tall', so the question is wrong"

No, obviously C is correct.

[Jeffort]

From a logical standpoint, technically the word 'strongly' being included or not in the stem does change the burden of proof for the CR. However, as others above said, it doesn't really make a difference from a practical standpoint given that it's a multiple choice test and LSAC writes questions so that there is only one answer choice that is logically supported (whether super strongly or just so so semi strongly or just supported somewhat decently) such that you'll never have to nor be required to evaluate two answers that are both supported and judge which one has stronger support to make it the CR.

With LR inference questions, there are two different types of logic going on in terms of the difference between must be true and most strongly supported/most supported question types. Inductive and deductive logic.

Must be true LR questions test you with deductive logic, under which conclusions/inferences are judged to be either valid or invalid. There is no in-between, no gray area in terms of the probability/degree of how well supported an inference/conclusion is. With those the CR must be 100% guaranteed to be true by the evidence in the stimulus such that it cannot be false without contradicting evidence in the stimulus.

Most strongly supported and most supported questions test you with inductive logic.
Under inductive logic, inferences/conclusions are evaluated by the degree of probability that it is true based on the evidence and are judged to be unsupported, weak or strong inferences. There is gray area regarding how well proven an inference/conclusion is; there is an in-between. An inductive inference/conclusion can be more or less strong, more or less weak, and just where you draw the line is not clear. Since inductive logic doesn't consider weakly supported inferences to be invalid as does deductive logic, there can be a lot of wiggle room for what qualifies as a supported inference. Most strongly supported questions require that the CR is a strong inference/conclusion based on the evidence, which means that the evidence provides enough support to make the CR a statement that has a high probability of being true based on the evidence with very very little wiggle room if any under which it could be false.

Without the word strongly in the stem, the burden of proof is dropped and a weakly supported inference (a statement that has a lot of wiggle room such that there is a decent arguable probability that the inference could actually be false under the given evidence) can qualify as the correct answer.

Under most strongly supported questions there is some but very little wiggle room allowed for whether the CR could be false under the given evidence since the stem specifies a 'strong' inference/conclusion, excluding weak ones from qualifying as a CR. The CRs for nearly all MSS questions are inferences that you'd have to jump through lots of hoops and consider highly improbable possibilities like aliens coming to earth and doing weird things in order to envision a way the CR could be false without contradicting and/or ignoring some of the given evidence.

With the rare most supported inference questions, there's simply a lot more wiggle room allowed for the possibility that the inference the CR states could be false and the CR can be an extreme/absolute logical force statement even with a body of evidence that doesn't logically get you really close to definitively proving the inference is almost certainly true. You don't have to consider highly unlikely possibilities such as aliens from outer space getting involved in order to think up a few ways how the CR in the Shakespeare question could be false since the evidence in the stimulus doesn't eliminate some reasonably/remotely possible alternatives for who could have created the Hamlet abridgment. It could have been produced by an actor or wannabe actor who had prepared and rehearsed to play a role in Hamlet (or perhaps by somebody with friends that liked to mess around doing old school cosplay in the 17th century for fun! lol) but never actually got to take the stage and act the role in a production of Hamlet.

The main complaint about that question was that the CR could easily be false/isn't close to being guaranteed or nearly guaranteed to be true by the evidence like almost all MSS question CRs are and because all the answers are absolute logical force and could be false without considering aliens invading earth possibilities. A lot of people got burned and/or flustered by it on test day (or when taking it as a timed PT) because they quickly noticed that the CRs are all extreme/absolute statements that could be false and that the stimulus evidence leaves open other author possibilities and/or got it down to the two main contenders and then panicked when they realized both could be false while trying to figure out a way to eliminate one to validate selecting the other AC. (A) (B) & (E) are easy to eliminate to get it down to (C) & (D) but then it's easy to get stuck because both of those don't appear to have strong enough support since it is an absolute 'was produced' logical force inference due to being trained to think 'could this be false' as a way of evaluating answer choices and the main contenders for process of elimination and to avoid the trap on harder questions when stuck with two contenders on MBT and MSS q's.

If LSAC had included the word 'strongly' in the stem for that question, it could easily be argued that it's a flawed question under the view that the CR is a weakly supported rather than strongly supported inductive inference from the evidence. That may be why they just removed the word for CYA purposes and/or they wrote the question with extreme answers and that rare stem on purpose with the intent to throw off people trained through prep to validate their MSS Qs final answer choice decision when uncertain between contenders by using the 'could this be false?' thought process to get to the CR by POE and/or to double check both contenders to be extra careful before making final AC decision, and/or to be skeptical about extreme/absolute answers on MSS Qs.

I think the main important takeaway from this is don't panic if you get stuck with either zero contenders or two or more contenders on an inference Q that sound like they could be false and/or that are extreme/absolute logical force. Instead of having a mini panic moment, just double check the stem for the word 'strongly' and/or just use common sense and go with the answer that is decently likely to be true based on the evidence/has some support without having to throw in unwarranted assumptions.

It wouldn't surprise me if LSAC occasionally writes ones of this type just to f*ck with people that are taught by prep books/classes to think 'could this be false?' as a way to eliminate answers for MBT and MSS Qs and/or are taught to be skeptical of absolute/extreme logical force answers for MSS questions.

[mornincounselor]

Post removed.