Do retakes matter?

(Applications Advice, Letters of Recommendation . . . )
User avatar
LSATSCORES2012
Posts: 770
Joined: Mon Jul 02, 2012 2:12 pm

Do retakes matter?

Postby LSATSCORES2012 » Mon May 20, 2013 7:55 pm

I know TLS wisdom suggests that retakes don't negatively affect one's application, and last night I figured out how to access retaker information on LSN and thought I'd test that hypothesis for the T14.

Edited: Analysis from elterrible and squeebz, stats geniuses - thanks to both of you for doing this

WL included, 2009-2010, probit+logit regression
squeebz wrote:Schools w/ No Statistical Difference Between Retakers and Non-Retakers - Penn, GULC, Northwestern, UVA, Yale, Michigan, and Duke

Schools Where Retakers Faired Worse - Columbia, Berkeley, Cornell, NYU, Harvard, Stanford, and Chicago


2006-2013, WL excluded, bad LSN data points excluded, logit regression
elterrible78 wrote:Quick and dirty:

Controlling for LSAT, GPA, URM, ED (where applicable), and how early the app was sent in, these are the results:

Boalt - Retakers are 19% less likely to be admitted.
Cornell - Retakers are 26% less likely to be admitted.
Harvard - Retakers are 31% less likely to be admitted.
NYU - Retakers are 62% less likely to be admitted (not a typo).
Stanford - Retakers are 40% less likely to be admitted.

It doesn't matter a bit at the rest of the T14.

All the usual LSN-related caveats apply to this.


Edited: Instead of listing my conclusions, I'm providing images for each of the T14 which help you to reach your own conclusions on the significances of any difference. In these graphs, red indicates that the acceptance rate for retakers is lower; yellow indicates no difference; and green indicates that retakers have a greater acceptance rate. Note that yellow in the lower part of the graph is possibly attributable to the fact that no one in that range is getting in. (data 2008-2012)

--ImageRemoved--
--ImageRemoved--
--ImageRemoved--
--ImageRemoved--
--ImageRemoved--
--ImageRemoved--
--ImageRemoved--
--ImageRemoved--
--ImageRemoved--
--ImageRemoved--
--ImageRemoved--
--ImageRemoved--
--ImageRemoved--
--ImageRemoved--
Last edited by LSATSCORES2012 on Wed May 22, 2013 11:24 am, edited 5 times in total.

User avatar
Dr. Dre
Posts: 2347
Joined: Mon Feb 18, 2013 7:10 pm

Re: Do retakes matter?

Postby Dr. Dre » Tue May 21, 2013 12:35 am

:shock:

NoWorries
Posts: 100
Joined: Fri Mar 15, 2013 9:05 pm

Re: Do retakes matter?

Postby NoWorries » Tue May 21, 2013 1:10 am

Damn son. Impressive work.

062914123
Posts: 1846
Joined: Sun Apr 20, 2008 2:11 pm

Re: Do retakes matter?

Postby 062914123 » Tue May 21, 2013 1:24 am

.
Last edited by 062914123 on Mon Jun 30, 2014 3:11 pm, edited 1 time in total.

User avatar
Vexed
Posts: 261
Joined: Sat Feb 16, 2013 2:55 pm

Re: Do retakes matter?

Postby Vexed » Tue May 21, 2013 1:56 am

For your categories of schools that consider or probably consider, is it in favor or against retakers? Is there anyway to control for the location of the higher score (IE, if you end up scoring lower on a single retake, does that lend to a poorer cycle at those schools than someone who scores higher the second time around?)

User avatar
jrsbaseball5
Posts: 290
Joined: Fri Mar 09, 2012 12:41 pm

Re: Do retakes matter?

Postby jrsbaseball5 » Tue May 21, 2013 2:00 am

Excellent work as always. Though when I saw 2 tailed t-test I cringed a little at the repressed memories :lol:

rebexness
Posts: 4163
Joined: Sun Jan 09, 2011 6:24 am

Re: Do retakes matter?

Postby rebexness » Tue May 21, 2013 3:22 am

I would disagree about Harvard.

User avatar
sublime
Posts: 15415
Joined: Sun Mar 10, 2013 12:21 pm

Re: Do retakes matter?

Postby sublime » Tue May 21, 2013 3:55 am

..

User avatar
bowser
Posts: 236
Joined: Tue Jul 24, 2012 2:54 am

Re: Do retakes matter?

Postby bowser » Tue May 21, 2013 4:05 am

I have no idea how your statistics work. But FWIW your list is very counter-TLS wisdom (which is generally, the higher up the rankings you go the more likely a school is to consider retakes, and only HYS really even care a little bit).

rebexness
Posts: 4163
Joined: Sun Jan 09, 2011 6:24 am

Re: Do retakes matter?

Postby rebexness » Tue May 21, 2013 4:14 am

It also depends on the number and range of retakes.

For example, a 158, 172 will be looked at differently than a 165,16,170

At least, I would guess so.

User avatar
Balthy
Posts: 668
Joined: Sat Apr 03, 2010 12:28 pm

Re: Do retakes matter?

Postby Balthy » Tue May 21, 2013 4:52 am

bowser wrote:I have no idea how your statistics work. But FWIW your list is very counter-TLS wisdom (which is generally, the higher up the rankings you go the more likely a school is to consider retakes, and only HYS really even care a little bit).



I agree. Also, NU explicitly states on their website that they only consider your top score. If their official policy was that they consider all your scores in a "holistic" manner, I could believe that this was bullshit and they really only care about keeping their medians up (and so only care about your top score). However, NU stating themselves that they only consider top score makes me think something is wrong with OPs placement of NU.

User avatar
LSATSCORES2012
Posts: 770
Joined: Mon Jul 02, 2012 2:12 pm

Re: Do retakes matter?

Postby LSATSCORES2012 » Tue May 21, 2013 10:41 am

So, as I said in the first post, there are a lot of potential flaws. What I've done is reported what the numbers say, but numbers can be misleading. For Northwestern, for instance, about 70% of the data points that I sampled showed that non-retakers, or retakers without a very low first score or retake score, had a better chance of being admitted than those who retook and ended up with their highest LSAT score being three points or more greater than their lowest.

Could it be random statistical fluctuation? Perhaps, but that's what the t-test was designed to exclude (though it can't ever exclude it entirely, just say there's a very small chance that the difference is due to random chance as opposed to another factor).

Then again, that other factor can be many things. I provided an example for Chicago, but other reasons could also apply to Northwestern. Say, for example, it is the case (hypothetically) that applicants who have been out of school for a few years are slightly less likely to retake the LSAT, instead performing well on their first try (maybe because they don't have the time to retake or because of the skills they have developed in the workforce or for whatever reason). That would explain Northwestern's results, too, because Northwestern is a school that is known for preferring candidates with work experience, and if the retakes are disproportionally being done by candidates without work experience then it will seem as though they are discriminating against retakes.

Of course, this is one possibility. Another is that Northwestern is lying, though that seems unlikely because it's not something that seems like it would benefit them.

One more thing I'd like to add: I don't think any of this should discourage anyone from retaking... I mostly posted it to put more information out there and question this piece of TLS wisdom. I definitely think retaking is good in most situations (the exception being if you've already scored very high).

User avatar
LSATSCORES2012
Posts: 770
Joined: Mon Jul 02, 2012 2:12 pm

Re: Do retakes matter?

Postby LSATSCORES2012 » Tue May 21, 2013 12:00 pm

Vexed wrote:For your categories of schools that consider or probably consider, is it in favor or against retakers? Is there anyway to control for the location of the higher score (IE, if you end up scoring lower on a single retake, does that lend to a poorer cycle at those schools than someone who scores higher the second time around?)

They we all in favor of non-retakers, except where I specified otherwise. (I think Berkeley, Yale, and UVA were the only ones in favor of retakers, and those ever so slightly (results were not statistically significant)).

I'll see what I can do about your second question...

User avatar
LSATSCORES2012
Posts: 770
Joined: Mon Jul 02, 2012 2:12 pm

Re: Do retakes matter?

Postby LSATSCORES2012 » Tue May 21, 2013 2:09 pm

To eliminate the complicated statistics (and because I think an ANOVA test might have actually been more appropriate), these numbers are the percentage of the data points I looked at where the "non-retakers" had a greater acceptance rate than the retakers (note that this does not discriminate between a 1% difference and a 50% difference).

32%: Yale
40%: Berkeley
45%: Stanford
49%: UVA
53%: Harvard
55%: Michigan
55%: Chicago
58%: Penn
60%: Georgetown
65%: Columbia
70%: Northwestern
72%: Duke
79%: Cornell
83%: NYU

Also, keep in mind that this data is from 2009-2013, and IIRC NYU only recently changed their policy on how they treat retakers.

User avatar
sinfiery
Posts: 3308
Joined: Thu Jul 14, 2011 2:55 am

Re: Do retakes matter?

Postby sinfiery » Tue May 21, 2013 2:42 pm

Wow, did not expect to see those results. Awesome, thanks for doing this.

User avatar
Balthy
Posts: 668
Joined: Sat Apr 03, 2010 12:28 pm

Re: Do retakes matter?

Postby Balthy » Tue May 21, 2013 2:58 pm

Didn't want to detract from this at all, just my initial thought after seeing NU. Still great info, especially if understood with the caveats you detailed, and a great contribution. Thanks!

User avatar
LSATSCORES2012
Posts: 770
Joined: Mon Jul 02, 2012 2:12 pm

Re: Do retakes matter?

Postby LSATSCORES2012 » Tue May 21, 2013 3:43 pm

superdingle2000 wrote:Didn't want to detract from this at all, just my initial thought after seeing NU. Still great info, especially if understood with the caveats you detailed, and a great contribution. Thanks!

No, I didn't take it that way... sorry if I came off as overly defensive. I definitely think you raise a great point. It IS puzzling why NU says this one thing, when you would expect them to try to publicly push in the other direction, and at first glance they actually do seem to do it the other way.

I think it's interesting, so thank you for pointing that out, largely because it helps to show just how misleading the data can be. Surely, I would say, it's not the case that Northwestern is actually looking at the average score or considering both scores when they say otherwise and they would benefit from not saying otherwise. So there must be some other factor contributing to that difference.

What REALLY surprises me is how Yale seems to prefer people who have taken the LSAT more than once.

BigZuck
Posts: 10874
Joined: Tue Sep 04, 2012 9:53 am

Re: Do retakes matter?

Postby BigZuck » Tue May 21, 2013 7:10 pm

Fascinating.

In my case, Duke, Cornell, Northwestern, and Michigan almost certainly did not care about my other scores and only looked at my highest. And Berkeley almost certainly did not care about the fact that I retook up into their 75th (nor the fact that I was a reapplicant. Seems like they could care less about my dedication. Or they just thought I wasn't pretty enough).

Nice work!

User avatar
supernma
Posts: 63
Joined: Thu Mar 28, 2013 9:33 pm

Re: Do retakes matter?

Postby supernma » Tue May 21, 2013 10:59 pm

I think the consensus seems to be that policies on retakes are fairly school-specific (NYU averages multiple scores, Columbia takes only the highest of multiple scores, etc). Even so, it's best to only take once if at all possible. Retakes can definitely be warranted depending on the circumstances though. :D

User avatar
sublime
Posts: 15415
Joined: Sun Mar 10, 2013 12:21 pm

Re: Do retakes matter?

Postby sublime » Tue May 21, 2013 11:04 pm

..

squeebz
Posts: 10
Joined: Wed Jun 27, 2012 1:47 pm

Re: Do retakes matter?

Postby squeebz » Tue May 21, 2013 11:43 pm

LSATSCORES2012 wrote:To eliminate the complicated statistics (and because I think an ANOVA test might have actually been more appropriate), these numbers are the percentage of the data points I looked at where the "non-retakers" had a greater acceptance rate than the retakers (note that this does not discriminate between a 1% difference and a 50% difference).

32%: Yale
40%: Berkeley
45%: Stanford
49%: UVA
53%: Harvard
55%: Michigan
55%: Chicago
58%: Penn
60%: Georgetown
65%: Columbia
70%: Northwestern
72%: Duke
79%: Cornell
83%: NYU

Also, keep in mind that this data is from 2009-2013, and IIRC NYU only recently changed their policy on how they treat retakers.


Did you consider trying a probit regression? You could regress gpa, highest LSAT, and an indicator variable on retakers on outcome. You could then look at the sign and significance level of the indicator variable. That way you could avoid all the messy within .05 GPA stuff and calculating acceptance rates.

Just some thoughts from a fellow stats nerd.

User avatar
LSATSCORES2012
Posts: 770
Joined: Mon Jul 02, 2012 2:12 pm

Re: Do retakes matter?

Postby LSATSCORES2012 » Tue May 21, 2013 11:49 pm

sublime wrote:
This was done for each school in the T-14 considering applicants between 2009 and 2013


It is possible also, with the decline in applicants, that more schools are forced to not care and cannot be as picky as they were a few years ago.

This is really bizarre...

When I change to just the 2011-2012 cycle, nearly EVERY T14 school seems to very heavily prefer retakers - even when controlling for LSAT.

I've checked for errors in my code quite a few times because this is so bizarre, and even hand checked one of the results, and just can't find any problems.

User avatar
LSATSCORES2012
Posts: 770
Joined: Mon Jul 02, 2012 2:12 pm

Re: Do retakes matter?

Postby LSATSCORES2012 » Tue May 21, 2013 11:50 pm

squeebz wrote:
LSATSCORES2012 wrote:To eliminate the complicated statistics (and because I think an ANOVA test might have actually been more appropriate), these numbers are the percentage of the data points I looked at where the "non-retakers" had a greater acceptance rate than the retakers (note that this does not discriminate between a 1% difference and a 50% difference).

32%: Yale
40%: Berkeley
45%: Stanford
49%: UVA
53%: Harvard
55%: Michigan
55%: Chicago
58%: Penn
60%: Georgetown
65%: Columbia
70%: Northwestern
72%: Duke
79%: Cornell
83%: NYU

Also, keep in mind that this data is from 2009-2013, and IIRC NYU only recently changed their policy on how they treat retakers.


Did you consider trying a probit regression? You could regress gpa, highest LSAT, and an indicator variable on retakers on outcome. You could then look at the sign and significance level of the indicator variable. That way you could avoid all the messy within .05 GPA stuff and calculating acceptance rates.

Just some thoughts from a fellow stats nerd.

I don't even know what that means, lol.

Could I send you the raw data and you can perform your own analysis?

User avatar
Balthy
Posts: 668
Joined: Sat Apr 03, 2010 12:28 pm

Re: Do retakes matter?

Postby Balthy » Tue May 21, 2013 11:56 pm

LSATSCORES2012 wrote:
superdingle2000 wrote:Didn't want to detract from this at all, just my initial thought after seeing NU. Still great info, especially if understood with the caveats you detailed, and a great contribution. Thanks!

No, I didn't take it that way... sorry if I came off as overly defensive. I definitely think you raise a great point. It IS puzzling why NU says this one thing, when you would expect them to try to publicly push in the other direction, and at first glance they actually do seem to do it the other way.

I think it's interesting, so thank you for pointing that out, largely because it helps to show just how misleading the data can be. Surely, I would say, it's not the case that Northwestern is actually looking at the average score or considering both scores when they say otherwise and they would benefit from not saying otherwise. So there must be some other factor contributing to that difference.

What REALLY surprises me is how Yale seems to prefer people who have taken the LSAT more than once.




Awww, I feel like we belong in a Canadian law school :D

squeebz
Posts: 10
Joined: Wed Jun 27, 2012 1:47 pm

Re: Do retakes matter?

Postby squeebz » Wed May 22, 2013 12:06 am

LSATSCORES2012 wrote:
squeebz wrote:
LSATSCORES2012 wrote:To eliminate the complicated statistics (and because I think an ANOVA test might have actually been more appropriate), these numbers are the percentage of the data points I looked at where the "non-retakers" had a greater acceptance rate than the retakers (note that this does not discriminate between a 1% difference and a 50% difference).

32%: Yale
40%: Berkeley
45%: Stanford
49%: UVA
53%: Harvard
55%: Michigan
55%: Chicago
58%: Penn
60%: Georgetown
65%: Columbia
70%: Northwestern
72%: Duke
79%: Cornell
83%: NYU

Also, keep in mind that this data is from 2009-2013, and IIRC NYU only recently changed their policy on how they treat retakers.


Did you consider trying a probit regression? You could regress gpa, highest LSAT, and an indicator variable on retakers on outcome. You could then look at the sign and significance level of the indicator variable. That way you could avoid all the messy within .05 GPA stuff and calculating acceptance rates.

Just some thoughts from a fellow stats nerd.

I don't even know what that means, lol.

Could I send you the raw data and you can perform your own analysis?


Probit regression is a type of regression things that occur in binary outcomes (admit/reject). The math to calculate the coefficients is pretty intense, but the code is fairly straightforward to run in R or Stata.

PM and I'll try to give it a run. My coding skills are a tad rusty though.

Just an FYI when then p-value is really high it doesn't mean schools prefer retakes. It just means that there is no difference on admissions decisions between retaking and not.




Return to “Law School Admissions Forum”

Who is online

Users browsing this forum: BigZuck, yhchoi1687 and 9 guests