Attention, to all people considering ED Forum

[cotiger]

Here is a statistical breakdown of the effect of EDing on your chances of acceptance to each school in the T14 for splitters, non-splitters, and reverse-splitters.

Quick overview:
-no boost from EDing NYU
-questionable if there's a boost at Columbia or Michigan (and it's a small effect, if it does exist)
-largest boosts from EDing are at UVA and Chicago.

The coefficients on right-hand side of this chart are the boosts associated with ED applicants for each school 2010-present:



Please note that the higher the p-value to the right of the coefficients is, the less sure we are that the effect from EDing actually exists.

Thanks to Elterrible87 for the number crunching. Here's his blog: http://admissionsbythenumbers.blogspot.com

[jstep]

Awesome thank you for this

[midwest17]

Bear in mind that there might be some confounding variables here: people who ED might tend to have weaker overall applications relative to their numbers than people who don't ED, which would cause the ED boost to be underestimated by the regression method.

I don't see a reason to doubt the relative statistics (that is, if your question is "will I get a bigger ED boost at NYU or Columbia", this method is probably reasonably accurate). But it might be understating the absolute effect, if you're trying to decide whether or not to ED in the first place.

[mojangles]

Extremely interesting - thanks for the informative post!

[cotiger]

Bear in mind that there might be some confounding variables here: people who ED might tend to have weaker overall applications relative to their numbers than people who don't ED, which would cause the ED boost to be underestimated by the regression method.

I don't see a reason to doubt the relative statistics (that is, if your question is "will I get a bigger ED boost at NYU or Columbia", this method is probably reasonably accurate). But it might be understating the absolute effect, if you're trying to decide whether or not to ED in the first place.

Fair point. Also, I should mention that that isn't my blog, but rather TLS's elterrible78.

[avth]

Thank you for this post!

[McGruff]

I constantly forget this and find myself thinking I'll ED to Penn to make up for my terrible GPA. I wish I understood statistics so this refutation of the commonly held belief that Penn has an ED bump for splitters wouldn't just look like it's coming out of a magic formula.

[cotiger]

I constantly forget this and find myself thinking I'll ED to Penn to make up for my terrible GPA. I wish I understood statistics so this refutation of the commonly held belief that Penn has an ED bump for splitters wouldn't just look like it's coming out of a magic formula.

To be fair, the results don't claim that there is no ED bump for splitters at Penn. They just claim that we can't reliably differentiate it from randomness.

Penn may well have an ED bump for splitters, but the only splitter boosts that are clear enough to say that they for sure exist come from UVA, Chicago, Michigan, and Georgetown.

Edit: To expand a bit.. when you run these regressions, they will spit out a result that represents the boost. They will also give you another number called the standard error that (generally speaking) represents your uncertainty that the number it spit out is close to the "true" boost. When you go above and below the result by twice the standard error, that represents the range of values that it is likely (with 95% confidence) that the true boost is in.

So let's say that splitter ED Penn spit out a boost of 13, but the data was really wonky so the SE was 7. That means that 95% of the time, the true value of the splitter ED boost was within the range of -1 to 27. In this case, we can't rule out the possibility that there is no ED boost at all. We call this being not statistically significant, despite the fact that it is equally likely that the ED boost is as great as 26 or 27 as it is to be non-existant.

Compare that to say ED UVA, which might return a boost of 8. But if the data is clearer, it might only have SE of 3. In that case, it is 95% likely that the true boost at UVA is between 2 and 14. Despite the fact that the maximum likely value of the true UVA boost is barely larger than the result we got for Penn, UVA's boost is the only one that is significant. We KNOW (well.. are 95% sure!) that UVAs boost is above zero. We can't say the same about Penn's.

Hopefully this clarifies a little bit about what's going on.

Edit 2: In this specific case, the result can't be a negative number, but for demonstration purposes it's easiest to think of it in this way.

[McGruff]

I constantly forget this and find myself thinking I'll ED to Penn to make up for my terrible GPA. I wish I understood statistics so this refutation of the commonly held belief that Penn has an ED bump for splitters wouldn't just look like it's coming out of a magic formula.

To be fair, the results don't claim that there is no ED bump for splitters at Penn. They just claim that we can't reliably differentiate it from randomness.

Penn may well have an ED bump for splitters, but the only splitter boosts that are clear enough to say that they for sure exist come from UVA, Chicago, Michigan, and Georgetown.

Oh okay, that makes sense.

[cotiger]

I constantly forget this and find myself thinking I'll ED to Penn to make up for my terrible GPA. I wish I understood statistics so this refutation of the commonly held belief that Penn has an ED bump for splitters wouldn't just look like it's coming out of a magic formula.

To be fair, the results don't claim that there is no ED bump for splitters at Penn. They just claim that we can't reliably differentiate it from randomness.

Penn may well have an ED bump for splitters, but the only splitter boosts that are clear enough to say that they for sure exist come from UVA, Chicago, Michigan, and Georgetown.

Oh okay, that makes sense.

Explained a bit more in an edit to the above.

[McGruff]

I constantly forget this and find myself thinking I'll ED to Penn to make up for my terrible GPA. I wish I understood statistics so this refutation of the commonly held belief that Penn has an ED bump for splitters wouldn't just look like it's coming out of a magic formula.

To be fair, the results don't claim that there is no ED bump for splitters at Penn. They just claim that we can't reliably differentiate it from randomness.

Penn may well have an ED bump for splitters, but the only splitter boosts that are clear enough to say that they for sure exist come from UVA, Chicago, Michigan, and Georgetown.

Oh okay, that makes sense.

Explained a bit more in an edit to the above.

I see. I figured 'Not Statistically Significant' meant "the difference is so slight as to be negligible", but it actually means "we just plain don't know how much of a difference there is or if one even exists." Looks like UVA is still the safer choice...

[cotiger]

I see. I figured 'Not Statistically Significant' meant "the difference is so slight as to be negligible", but it actually means "we just plain don't know how much of a difference there is or if one even exists." Looks like UVA is still the safer choice...

The thing is, Penn could have also returned a boost value of 0 with SE of 1 (likely true value of boost -2 to 2), which would also be NSS but would show the effect to be demonstrably negligible. We don't know which of these scenarios is the case bc the poster just left it as generically NSS.

[gelato]

Great post, interesting results.

However please keep in mind that this data is from people who VOLUNTEER their information, and not necessarily completely. He has a warning about this on the top of the blog, which is great. Please don't take the results too seriously, for the way the data is collected in the first place is flawed.

^ Happy to explain more; didn't want to get all research methods on this thread.

[cotiger]

Great post, interesting results.

However please keep in mind that this data is from people who VOLUNTEER their information, and not necessarily completely. He has a warning about this on the top of the blog, which is great. Please don't take the results too seriously, for the way the data is collected in the first place is flawed.

^ Happy to explain more; didn't want to get all research methods on this thread.

In this case, I think that the data collection issues are overstated.

Like Midwest pointed out, if people ED not only because they think their numbers are sketchy, but also because they think that they have weak softs for people of their same numbers, then the sizes of the boosts will look to be smaller than they actually are. However, unless the tendency to ED due to weak softs varies between schools, that won't affect the relative position of schools' ED boost effects.

There is of course LSN's general problem with the fact that people are more likely to report good results. I don't think this problem would be either greater or lesser with ED applicants, so that won't affect the results for ED boost.

There's also the general issue of applicants on LSN having stronger softs or being more prepared than the average applicant for their numbers (which IMO is not a problem, and actually makes the site more relevant to me). Similar to the last problem, I don't think this would vary between the broader applicant pool and the ED crowd, except to the extent that all ED applicants (not just LSN ED applicants) may have weaker softs, which was covered in my first point.

[grungy89]

That's good information, but I wish UT-Austin's ED process wasn't in its first year so I could have something to go off of....

[20141023]

.

[midwest17]

Bear in mind that there might be some confounding variables here: people who ED might tend to have weaker overall applications relative to their numbers than people who don't ED, which would cause the ED boost to be underestimated by the regression method.

I don't see a reason to doubt the relative statistics (that is, if your question is "will I get a bigger ED boost at NYU or Columbia", this method is probably reasonably accurate). But it might be understating the absolute effect, if you're trying to decide whether or not to ED in the first place.

I don't think the ED has anything to do with the strength of the application outside of the numbers; people usually just ED because either their LSAT or GPA (or both) are too low to give them a reasonable chance of getting in somewhere without some sort of a boost. (I've never heard of someone using ED because of a lack of softs.)

EDing because of a lack of softs does seem a little silly, but holding off from ED because you think you have particularly good softs doesn't seem ridiculous.

[cotiger]

Hey guys, I got the complete results table from Elterrible for the effect of ED from 2010-present. He said he'd post the splitter-only table (with the caveat that the data for that goes all the way back to 2003) later today.



To repost a brief explanation:

Basically, the coefficient is our estimation of what the effect on admissions likelihood is for either applying one month earlier (on the left) or ED (on the right). The p-values to the right of the coefficients are a representation of how sure we are that effect is different from zero. The lower the p-value, the more certain we are that the observed effect really exists.

We call a result with a very large p-value "Not statistically significant" because it is very unlikely that the effect is real. For instance, superficially, the coefficients indicate that applying to Duke earlier in the cycle or EDing to NYU actually make it less likely to be accepted. However, the large p-values tell us what we intuitively know: that that's ridiculous and really much more likely that there's just no effect.

As far as how high of a p-value we can accept, I think that elterrible was too aggressive with NSS on his blog. Typically, if we want to prove that something is real in economics (or medicine), we demand a 90% or 95% likelihood (p-value<.1 or .05). However, in this case we aren't trying to prove that it exists, merely looking for guidance, so a 75% likelihood that the effect exists is going to be useful information. Due to that, we can say that EDing Columbia or Michigan probably does have a small effect, albeit with the caveat that that effect might not actually exist.

[McGruff]

This is great! Thank you for following up on this! Maybe EDing Penn is the real deal.

Question though: these bumps are both the difference in likelihood between acceptance and waitlisting, and between waitlisting and rejection, right? If not, which, and if so, how can it be both?

[cotiger]

This is great! Thank you for following up on this! Maybe EDing Penn is the real deal.

Question though: these bumps are both the difference in likelihood between acceptance and waitlisting, and between waitlisting and rejection, right? If not, which, and if so, how can it be both?

It only concerns people who were eventually accepted vs rejected. People who remained on the waitlist with status unknown are left out.

[McGruff]

This is great! Thank you for following up on this! Maybe EDing Penn is the real deal.

Question though: these bumps are both the difference in likelihood between acceptance and waitlisting, and between waitlisting and rejection, right? If not, which, and if so, how can it be both?

It only concerns people who were eventually accepted vs rejected. People who remained on the waitlist with status unknown are left out.

Ah, makes sense. Thanks for all the stats explanation/clarification, this is great!