Why are some people really good at standardized testing?

acrossthelake
Posts: 4431
Joined: Sat May 16, 2009 5:27 pm

Re: Why are some people really good at standardized testing?

Postby acrossthelake » Tue Jun 29, 2010 4:22 pm

Hey-O wrote:
This is an interesting idea. Let me get what you're saying here. Working memory is really the ability to hold complex ideas in your head long enough to make connections between them.

I think we're agreeing about substance and disagreeing about the definition of memory. It is still all about the ability to make connections.


Exactly! Or to apply them or to do something to them or whatever. I think we agree as well, just thought I'd supply the actual official terms to make discussion more clear. But there's different kinds of memory(like long-term), thus only using one definition in this discussion is hampering the discussion. This is all I've studied during undergrad... :lol:

EDIT: There are many more distinctions and categories of memory than what I have detailed here. (For ex, long-term can be divided into episodic vs procedural vs semantic, etc.) but these distinctions are irrelevant to this discussion so I have left them out.

Hey-O
Posts: 719
Joined: Thu Sep 17, 2009 8:50 pm

Re: Why are some people really good at standardized testing?

Postby Hey-O » Tue Jun 29, 2010 4:50 pm

acrossthelake wrote:
Hey-O wrote:
This is an interesting idea. Let me get what you're saying here. Working memory is really the ability to hold complex ideas in your head long enough to make connections between them.

I think we're agreeing about substance and disagreeing about the definition of memory. It is still all about the ability to make connections.


Exactly! Or to apply them or to do something to them or whatever. I think we agree as well, just thought I'd supply the actual official terms to make discussion more clear. But there's different kinds of memory(like long-term), thus only using one definition in this discussion is hampering the discussion. This is all I've studied during undergrad... :lol:

EDIT: There are many more distinctions and categories of memory than what I have detailed here. (For ex, long-term can be divided into episodic vs procedural vs semantic, etc.) but these distinctions are irrelevant to this discussion so I have left them out.


In what capacity do you study cognition? I find this subject to be very interesting. Have you read this series:

http://opinionator.blogs.nytimes.com/20 ... is-part-5/

acrossthelake
Posts: 4431
Joined: Sat May 16, 2009 5:27 pm

Re: Why are some people really good at standardized testing?

Postby acrossthelake » Tue Jun 29, 2010 5:14 pm

Hey-O wrote:
In what capacity do you study cognition? I find this subject to be very interesting. Have you read this series:

http://opinionator.blogs.nytimes.com/20 ... is-part-5/


Just undergrad, though I've taken so many courses that I like to think I have a good enough understanding and that I'm recent enough with research to be able to talk about it with some degree of credibility on an online forum. Different from psych---I only do cognition(Memory, Learning, Perception, Attention, etc.) and I don't touch the other areas usually required by psych at most universities such as personality, clinical, etc. I was contemplating going for a PhD, but decided to opt out and do law instead....Haven't read this series, though I'm reading through it now and I'm familiar with the Dunning-Kruger Effect--read the original research papers in class this semester, as well as some follow-up articles and lit reviews. It's an interesting field. I recommend Rational Choice in an Uncertain World by Hastie and Dawes if you find this stuff interesting.
EDIT: You may or may not find this blog interesting as well: http://www.psychologytoday.com/blog/one-among-many

Hey-O
Posts: 719
Joined: Thu Sep 17, 2009 8:50 pm

Re: Why are some people really good at standardized testing?

Postby Hey-O » Tue Jun 29, 2010 6:51 pm

acrossthelake wrote:
Hey-O wrote:
In what capacity do you study cognition? I find this subject to be very interesting. Have you read this series:

http://opinionator.blogs.nytimes.com/20 ... is-part-5/


Just undergrad, though I've taken so many courses that I like to think I have a good enough understanding and that I'm recent enough with research to be able to talk about it with some degree of credibility on an online forum. Different from psych---I only do cognition(Memory, Learning, Perception, Attention, etc.) and I don't touch the other areas usually required by psych at most universities such as personality, clinical, etc. I was contemplating going for a PhD, but decided to opt out and do law instead....Haven't read this series, though I'm reading through it now and I'm familiar with the Dunning-Kruger Effect--read the original research papers in class this semester, as well as some follow-up articles and lit reviews. It's an interesting field. I recommend Rational Choice in an Uncertain World by Hastie and Dawes if you find this stuff interesting.
EDIT: You may or may not find this blog interesting as well: http://www.psychologytoday.com/blog/one-among-many


You might enjoy "The Black Swan" by Taleb. It is a very interesting look at the limits of statistical analysis and human predictive capabilities.

acrossthelake
Posts: 4431
Joined: Sat May 16, 2009 5:27 pm

Re: Why are some people really good at standardized testing?

Postby acrossthelake » Tue Jun 29, 2010 6:52 pm

Hey-O wrote:
You might enjoy "The Black Swan" by Taleb. It is a very interesting look at the limits of statistical analysis and human predictive capabilities.


I had a seminar that discussed that book last semester. This is true, though statistical predictions often do better than flat-out human predictions.

Hey-O
Posts: 719
Joined: Thu Sep 17, 2009 8:50 pm

Re: Why are some people really good at standardized testing?

Postby Hey-O » Tue Jun 29, 2010 7:02 pm

acrossthelake wrote:
Hey-O wrote:
You might enjoy "The Black Swan" by Taleb. It is a very interesting look at the limits of statistical analysis and human predictive capabilities.


I had a seminar that discussed that book last semester. This is true, though statistical predictions often do better than flat-out human predictions.


Really? I would be interested in this. Where are you getting this from? Is this general human predictions or this targeted predictions? For instance, I would take the prediction of a particular's student teacher over the prediction of how that student would do based on test scores.

User avatar
daesonesb
Posts: 499
Joined: Thu Oct 15, 2009 2:18 pm

Re: Why are some people really good at standardized testing?

Postby daesonesb » Tue Jun 29, 2010 7:02 pm

I've always done well on standardized tests, all the way back to the CAT testing in elementary school.

I think it might be because I don't experience test day anxiety and because I always read alot of books, ever since I was a kid.

Other than that, who knows?

acrossthelake
Posts: 4431
Joined: Sat May 16, 2009 5:27 pm

Re: Why are some people really good at standardized testing?

Postby acrossthelake » Tue Jun 29, 2010 7:15 pm

Hey-O wrote:
acrossthelake wrote:
Hey-O wrote:
You might enjoy "The Black Swan" by Taleb. It is a very interesting look at the limits of statistical analysis and human predictive capabilities.


I had a seminar that discussed that book last semester. This is true, though statistical predictions often do better than flat-out human predictions.


Really? I would be interested in this. Where are you getting this from? Is this general human predictions or this targeted predictions? For instance, I would take the prediction of a particular's student teacher over the prediction of how that student would do based on test scores.


If you take a look at that book by Hastie and Dawes they go over it. :-) Basically (almost?) all research that pits human against statistical model (made by humans) for a particular prediction finds that the model wins often even over the best performing human. Here let me quote from it.

...Paul Meehl published a highly influential book in which he reviewed approximately 20 studies comparing the clinical judgment of people(expert psychologists and psychiatrists in his study) with the linear statistical based only on relationships in the empirical data on the events of interest. In all studies evaluated, the statistical method provided more accurate predictions(or the two methods tied)...Sawyer reviewed 45 studies comparing clinical and statistical prediction. Again, there was not a single study in which clinical global judgment was superior to the statistical prediction...Sawyer...even included two studies in which the clinical judges had access to more information(an interview with each person being judged) but still did worse....

Goldberg asked experienced clinical diagnosticians to distinguish between neurosis and psychosis on the basis of personality test scores(a decision that has important implications for treatment and for insurance coverage in psychotherapeutic practice). He constructed a simple linear decision rule....Starting with a new sample of patient cases and using the patient's discharge diagnoses as the to-be-predicted criterion value, "Goldberg's rule"(the model) achieved an accuracy rate of approximately 70%. The human judges, in comparison, performed at rates from slightly above chance(50%) to 67% correct. Not even the best human judge was better than the mechanical adding-and-subtracting rule...


I led a 1-hour discussion in my seminar last semester actually about whether using numbers that prove to be more predictive than any other measure was a rational choice in law school admissions. :lol: My professor(who specializes in rational decision making) said he was actually surprised to hear that law school admissions followed the only process that researchers in decision making would agree with. There's a lot in the book about how holistic measures, such as interviews, are often poorly consistent and predictive because of the natural fallibility of human judgment.

They also had this to say about holistic versus numbers-based grad school(like, for specific subjects) admissions:

Such results bring us to an unsettling conclusion: A lot of outcomes about which we deeply care about are not very predictable. For example, it is not comforting to members of a graduate school admissions committee to know that only 23% of the variance in later faculty ratings of a student can be predicted by a unit weighing of the student's undergraduate GPA, his or her GRE score, and a measure of the student's undergraduate institution selectivity--but that is in comparison to 4% based on those committee members' global ratings of the applicant. We want to predict outcomes that are important to us. It is only rational to conclude that if one method does not predict well, something else may be better. What is not rational--in fact,it's irrational--is to conclude that this "something else" necessarily exists, and in the absence of positive supporting evidence, that it's intuitive global judgment.

Hey-O
Posts: 719
Joined: Thu Sep 17, 2009 8:50 pm

Re: Why are some people really good at standardized testing?

Postby Hey-O » Tue Jun 29, 2010 7:34 pm

I think I'll pick up the Dawes book the next time I'm at the library. I still lean towards human predictions over statical models but I'm definitely open to reading this.

My personal (completely untested) belief is that statistical models are good for general predictions, but bad at individual situations.

acrossthelake
Posts: 4431
Joined: Sat May 16, 2009 5:27 pm

Re: Why are some people really good at standardized testing?

Postby acrossthelake » Tue Jun 29, 2010 7:36 pm

Hey-O wrote:I think I'll pick up the Dawes book the next time I'm at the library. I still lean towards human predictions over statical models but I'm definitely open to reading this.

My personal (completely untested) belief is that statistical models are good for general predictions, but bad at individual situations.


I'm of the belief that statistical models & human judgment both suck at individual situations. It's a pick your poison where human judgment is like ridiculously bad. For all of our talk of intelligence on this thread, humans actually aren't very bright in general. :D

Hey-O
Posts: 719
Joined: Thu Sep 17, 2009 8:50 pm

Re: Why are some people really good at standardized testing?

Postby Hey-O » Tue Jun 29, 2010 7:38 pm

acrossthelake wrote:
Hey-O wrote:I think I'll pick up the Dawes book the next time I'm at the library. I still lean towards human predictions over statical models but I'm definitely open to reading this.

My personal (completely untested) belief is that statistical models are good for general predictions, but bad at individual situations.


I'm of the belief that statistical models & human judgment both suck at individual situations. It's a pick your poison where human judgment is like ridiculously bad. For all of our talk of intelligence on this thread, humans actually aren't very bright in general. :D


Yes, people might be stupid in general, but a person can be smart, and that is really the point I'm getting at. That a smart person in their field is better at predictions than a statistical model, because a person can change their thinking while a model can't change.

acrossthelake
Posts: 4431
Joined: Sat May 16, 2009 5:27 pm

Re: Why are some people really good at standardized testing?

Postby acrossthelake » Tue Jun 29, 2010 7:39 pm

Hey-O wrote:
acrossthelake wrote:
Hey-O wrote:I think I'll pick up the Dawes book the next time I'm at the library. I still lean towards human predictions over statical models but I'm definitely open to reading this.

My personal (completely untested) belief is that statistical models are good for general predictions, but bad at individual situations.


I'm of the belief that statistical models & human judgment both suck at individual situations. It's a pick your poison where human judgment is like ridiculously bad. For all of our talk of intelligence on this thread, humans actually aren't very bright in general. :D


Yes, people might be stupid in general, but a person can be smart, and that is really the point I'm getting at. That a smart person in their field is better at predictions than a statistical model, because a person can change their thinking while a model can't change.


In the studies mentioned, they used experts in the field though. Who did worse. The very best humans can occasionally tie.

Hey-O
Posts: 719
Joined: Thu Sep 17, 2009 8:50 pm

Re: Why are some people really good at standardized testing?

Postby Hey-O » Tue Jun 29, 2010 7:51 pm

Shit. Well, I'd better start welcoming our machine overlords.

acrossthelake
Posts: 4431
Joined: Sat May 16, 2009 5:27 pm

Re: Why are some people really good at standardized testing?

Postby acrossthelake » Tue Jun 29, 2010 7:57 pm

Hey-O wrote:Shit. Well, I'd better start welcoming our machine overlords.


Don't forget, humans create the statistical models and can refine and improve them. It's just that the human brain can't handle and weight variables of prediction that well without sitting down and writing things out and doing math. It's beyond the capacity of even really smart people with good working memories.

howcani111
Posts: 83
Joined: Sat May 29, 2010 4:19 pm

Re: Why are some people really good at standardized testing?

Postby howcani111 » Tue Jun 29, 2010 8:01 pm

ME.
Undergrad:
SAT: 1030
GPA: 4.3

obviously I got screwed when I was applying for undergrad. MY GPA (rounded cuz all of those damn APS) saved me. Also, I studied for the SAT for like a year taking prep classes like PR.

Now:
GPA 3.7
LSAT: 171

Go figure. I think I'm better with the critical thinking type stuff.

I always sucked on standardized testing.

Hey-O
Posts: 719
Joined: Thu Sep 17, 2009 8:50 pm

Re: Why are some people really good at standardized testing?

Postby Hey-O » Tue Jun 29, 2010 8:32 pm

acrossthelake wrote:
Hey-O wrote:Shit. Well, I'd better start welcoming our machine overlords.


Don't forget, humans create the statistical models and can refine and improve them. It's just that the human brain can't handle and weight variables of prediction that well without sitting down and writing things out and doing math. It's beyond the capacity of even really smart people with good working memories.


This makes sense. I think my real problem is with people using statistical models without thinking about them. That is where you come up with the problems like the recent stock market crash.

Edit: Typos

jason8821
Posts: 212
Joined: Tue Oct 20, 2009 12:42 am

Re: Why are some people really good at standardized testing?

Postby jason8821 » Tue Jun 29, 2010 11:07 pm

howcani111 wrote:ME.
Undergrad:
SAT: 1030
GPA: 4.3

obviously I got screwed when I was applying for undergrad. MY GPA (rounded cuz all of those damn APS) saved me. Also, I studied for the SAT for like a year taking prep classes like PR.

Now:
GPA 3.7
LSAT: 171

Go figure. I think I'm better with the critical thinking type stuff.

I always sucked on standardized testing.


That is what I am talking about. The people that are outliers are important because there is obviously something that goes beyond innate ability in some cases while not at all in others. I can't put my finger on it, and apparently no one else can. Perhaps you are someone who has an incredibly strong working memory, but very little in the way of long term memory, but GPA indicates more long term, so it's hard to say. Did you have a huge verbal/math split? I think people underestimate the importance of "being good at math" when taking the LSAT, if you have decent RC skills, and your very good at algebra, you should score at the very minimum mid 160's. The LR, and LG are often times destroyed by people who have those skills.

acrossthelake
Posts: 4431
Joined: Sat May 16, 2009 5:27 pm

Re: Why are some people really good at standardized testing?

Postby acrossthelake » Wed Jun 30, 2010 1:03 am

jason8821 wrote:
howcani111 wrote:ME.
Undergrad:
SAT: 1030
GPA: 4.3

obviously I got screwed when I was applying for undergrad. MY GPA (rounded cuz all of those damn APS) saved me. Also, I studied for the SAT for like a year taking prep classes like PR.

Now:
GPA 3.7
LSAT: 171

Go figure. I think I'm better with the critical thinking type stuff.

I always sucked on standardized testing.


That is what I am talking about. The people that are outliers are important because there is obviously something that goes beyond innate ability in some cases while not at all in others. I can't put my finger on it, and apparently no one else can. Perhaps you are someone who has an incredibly strong working memory, but very little in the way of long term memory, but GPA indicates more long term, so it's hard to say. Did you have a huge verbal/math split? I think people underestimate the importance of "being good at math" when taking the LSAT, if you have decent RC skills, and your very good at algebra, you should score at the very minimum mid 160's. The LR, and LG are often times destroyed by people who have those skills.


Reading skills on the SAT are actually a better predictor of LSAT score. The exams just test different things. The SAT looks at reading and math, but has an emphasis on vocab for reading and actual knowledge in types of math(algebra, geometry, etc.). The LSAT does reading, logic, and analytical reasoning, but doesn't actually require you know much in terms of vocab and LG is analytical reasoning, sure, but doesn't require knowledge of any type of learned math.
I personally found it a lot easier to make careless mistakes on the SAT because of the way the math problems were set up, whereas I did not make 'careless' errors on the LSAT. Questions vary a lot more in format for the SAT than the LSAT. Weak vocab, weak actual math knowledge and/or carelessness in the math section can hurt someone for the SAT in a way it wouldn't for the LSAT. I personally was less than impressed with the SAT, but impressed with the LSAT. The LSAT is just a better designed exam. (Note: I did well in both, so this is not out of bitterness or anything).

Tautology
Posts: 434
Joined: Thu Mar 18, 2010 12:40 pm

Re: Why are some people really good at standardized testing?

Postby Tautology » Wed Jun 30, 2010 1:17 am

Did you ever take the GRE acrossthelake? It is probably the worst test I've ever taken, although the format is interesting and different.

acrossthelake
Posts: 4431
Joined: Sat May 16, 2009 5:27 pm

Re: Why are some people really good at standardized testing?

Postby acrossthelake » Wed Jun 30, 2010 1:23 am

Tautology wrote:Did you ever take the GRE acrossthelake? It is probably the worst test I've ever taken, although the format is interesting and different.


No, although I skimmed through it once out of curiosity. It looks weird and seems to emphasize vocab to a degree I don't understand the purpose of. Why'd you take it? Yeah that is one test where I think you actually need to prep even if you're fairly bright. I have friends with large vocabularies that think the GRE is trippy.

Tautology
Posts: 434
Joined: Thu Mar 18, 2010 12:40 pm

Re: Why are some people really good at standardized testing?

Postby Tautology » Wed Jun 30, 2010 1:31 am

acrossthelake wrote:
Tautology wrote:Did you ever take the GRE acrossthelake? It is probably the worst test I've ever taken, although the format is interesting and different.


No, although I skimmed through it once out of curiosity. It looks weird and seems to emphasize vocab to a degree I don't understand the purpose of. Why'd you take it? Yeah that is one test where I think you actually need to prep even if you're fairly bright. I have friends with large vocabularies that think the GRE is trippy.


The vocab is retarded, there is no reason anyone would know half those words if they didn't specifically prep, although you can still usually eliminate a few answers without knowing some of the words. I didn't prep though and still managed to do fairly well, so I think smart people can get through it without prep, but I was a math major so math wasn't something I hadn't done in four years. Both the math and the vocab strike me as things that don't really say much about the majority of college graduates. It also has really weird percentile-to-score correspondences. I got an 800 in math which was like 90th percentile and a 650 or something in verbal which was like 94th percentile.

To answer your question, I took the GRE before the first time I went to grad school. Law school is my plan B. :D

acrossthelake
Posts: 4431
Joined: Sat May 16, 2009 5:27 pm

Re: Why are some people really good at standardized testing?

Postby acrossthelake » Wed Jun 30, 2010 1:41 am

Tautology wrote:
The vocab is retarded, there is no reason anyone would know half those words if they didn't specifically prep, although you can still usually eliminate a few answers without knowing some of the words. I didn't prep though and still managed to do fairly well, so I think smart people can get through it without prep, but I was a math major so math wasn't something I hadn't done in four years. Both the math and the vocab strike me as things that don't really say much about the majority of college graduates. It also has really weird percentile-to-score correspondences. I got an 800 in math which was like 90th percentile and a 650 or something in verbal which was like 94th percentile.

To answer your question, I took the GRE before the first time I went to grad school. Law school is my plan B. :D


Out of all standardized entrance exams I've seen, the GRE is the one where the connection between material tested and skills needed in grad school seem to have a huge disconnect. The obscure vocab is not needed in most(any?) fields and the math tested is irrelevant to those in the humanities and social sciences. The GRE looks to me like someone was like oh gee, grad school needs an entrance exam, let's just take the SAT and bastardize it.

Tautology
Posts: 434
Joined: Thu Mar 18, 2010 12:40 pm

Re: Why are some people really good at standardized testing?

Postby Tautology » Wed Jun 30, 2010 1:46 am

acrossthelake wrote:
Tautology wrote:
The vocab is retarded, there is no reason anyone would know half those words if they didn't specifically prep, although you can still usually eliminate a few answers without knowing some of the words. I didn't prep though and still managed to do fairly well, so I think smart people can get through it without prep, but I was a math major so math wasn't something I hadn't done in four years. Both the math and the vocab strike me as things that don't really say much about the majority of college graduates. It also has really weird percentile-to-score correspondences. I got an 800 in math which was like 90th percentile and a 650 or something in verbal which was like 94th percentile.

To answer your question, I took the GRE before the first time I went to grad school. Law school is my plan B. :D


Out of all standardized entrance exams I've seen, the GRE is the one where the connection between material tested and skills needed in grad school seem to have a huge disconnect. The obscure vocab is not needed in most(any?) fields and the math tested is irrelevant to those in the humanities and social sciences. The GRE looks to me like someone was like oh gee, grad school needs an entrance exam, let's just take the SAT and bastardize it.


Pretty much.

User avatar
PDaddy
Posts: 2073
Joined: Sat Jan 16, 2010 4:40 am

Re: Why are some people really good at standardized testing?

Postby PDaddy » Wed Jun 30, 2010 1:48 am

hax123 wrote:Genes.



This is true; some of it is just genetic. Standardized testing is about 30% intelligence, 30% early nurturing and environmental factors, 30% preparation, and 10% luck. The LSAT and other tests can be learned, however, the degrees of learning are different for each individual. The tests do not measure "intelligence". How could they? There are different types of intelligence, and everyone is endowed with a degree of several different types of intelligence. One of the primary reasons people suck at the LSAT is poor language skills. Another is poor reasoning skills. But, again, both of these things can be learned.

User avatar
Thomas Jefferson
Posts: 385
Joined: Sun Feb 21, 2010 6:32 pm

Re: Why are some people really good at standardized testing?

Postby Thomas Jefferson » Wed Jun 30, 2010 1:56 am

.
Last edited by Thomas Jefferson on Wed Jul 07, 2010 9:40 am, edited 1 time in total.




Return to “LSAT Prep and Discussion Forum”

Who is online

Users browsing this forum: Exabot [Bot], wildquest8200 and 3 guests