On omitted questions... Forum

[vuthy]

This is really just an "I'm bored and a nerd" issue, but it's been intriguing me for a while. Do you ever wonder how omitted questions affect the curve? Omitted questions probably had some defect, such as two acceptable right answers, or none -- either of which, I imagine, would lead test-takers (especially strong test-takers) to devote a good amount of time to the question, taking time away from other questions. That, in turn, seems like it would drive down overall scores, if only fractionally.

I just wonder how LSAC deals with this issue. My guess is that to a certain extent it ends up being sort of "sorry about your luck -- you really shouldn't spend 3 minutes on any question, no matter what." But I wonder if part of their complex model for determining the curve accounts for sunk time on omitted questions.

I was thinking about this for PT 66. I was LG -1. But what if I had to work through the omitted question? Granted I had plenty of time left on the LG, but you never know.

Anyway, there's that.

[RodneyRuxin]

Good test takers don't get bogged down on questions and instead come back to them at the end if they have time.

[Rotor]

Good test takers don't get bogged down on questions and instead come back to them at the end if they have time.

This.

And not-so-good test takers don't notice the problem.

[rinkrat19]

The omitted question just vanishes like it never existed, and scores for calculating the "curve" are tallied without it. I imagine that they assume an imperfect question is going to slow everyone down the same amount proportional to their score.

[vuthy]

I imagine that they assume an imperfect question is going to slow everyone down the same amount proportional to their score.

Right -- that's the part that intrigues me. I would imagine that stronger testers are actually slowed down more by defective questions (or at least some types of defective questions) more than weaker testers.

Not questioning the accuracy of the analytics...just curious about them.

[rinkrat19]

I imagine that they assume an imperfect question is going to slow everyone down the same amount proportional to their score.

Right -- that's the part that intrigues me. I would imagine that stronger testers are actually slowed down more by defective questions (or at least some types of defective questions) more than weaker testers.

Not questioning the accuracy of the analytics...just curious about them.

Well, following that logic, a REALLY strong test-taker might realize much more quickly that the question was imperfect and be confident that it would be omitted later on, and move on.

[vuthy]

Yeah. Just seems like it would be a real mess trying to sort all that out. They are really good at producing a test that generates fairly consistent scores over time. Can't even imagine all the analytical work that goes into writing, structuring, and curving each test.

[bp shinners]

I imagine that they assume an imperfect question is going to slow everyone down the same amount proportional to their score.

Right -- that's the part that intrigues me. I would imagine that stronger testers are actually slowed down more by defective questions (or at least some types of defective questions) more than weaker testers.

Not questioning the accuracy of the analytics...just curious about them.

Well, following that logic, a REALLY strong test-taker might realize much more quickly that the question was imperfect and be confident that it would be omitted later on, and move on.

But that same test-taker will also have studied the test, and in his studies learned that omitted questions are exceptionally rare, causing him to second-guess his assessment. So I clearly can not pick the answer in front of you.

[sasquatchsam]

I agree that omitted questions are likely to hurt strong test takers more than weak test takers. I personally felt that one of the LR questions on PT69 was poorly worded and did not have a clearly correct answer. In total I believe I spent nearly five minutes analyzing it. Fortunately I was able to eliminate all of the clearly incorrect answers. I feel like LR is my strong suite and ended up going -0 so I certainly feel anyones pain when it comes to these questions.