New Admissions Calculator for T14+1 Forum

[srt2021]

Hi all,

I combed through the raw data available from myLSN and ran logistical regressions on acceptances to the T14+1 (who knows what to do with Georgetown and Texas anyway?) to create a new admissions calculator that incorporates data from the past decade and a half.* Check it out here:

https://jscalc.io/calc/zt21YwP4hMpgFuoT

The calculator differs from others out there in two important ways:
(1) It doesn't rely on (potentially) limited data points, which should be useful to more unusual applicants or schools with smaller applicant pools;
(2) It incorporates a waitlist probability, which should reflect what applicants are dealing with more accurately than a simple admit-not admit scheme.

Interesting findings:
(1) There's seems to be a fairly limited boost at public schools in the T14+1 for being an in-state resident. (UVa is the exception, giving Virginia residents a pretty sizable advantage, but the numbers from LSN are too small to say conclusively.)
(2) The URM label is an enigma at LSN, with the category not lining up by race nearly as neatly as you might expect.
(3) Yield protection is real (but fairly limited) at some schools once you get to fairly high LSAT scores and GPAs, most notably among URMs, who seem much more likely to get snatched up by Yale, Stanford, and Harvard.
(4) It's not all a numbers game. The third table shows how much less variance in admissions decisions is actually explained by the three or four factors given than we might often assume. This probably has something to do with the waitlist option.

Feel free to let me know if you have any questions or thoughts re: improvement. I'm willing to consider putting up a simpler non-waitlist version if that'd be helpful to folks, too.

* the acceptance rates by LSAT/GPA/URM/ED haven't changed all that much, though there's evidence that they have decreased slightly over time.

[rowdy]

Have you considered incorporating data from admissions tables on LSAC.org for UVA and UT?

Why the 3.0/160 floor?

[Robb]

How fantastic! Would something similar be possible with respect to when people can expect to hear back from schools? That is one of mylsn's most popular features lately, but I fear that the two ways it's presented aren't that helpful. Something like "90% chance you hear back between Feb 1. and Feb 28" would be much easier.

[yyyuppp]

with a 190 LSAT and a 8.0 GPA looks like i got YPd at NU and UVA. their loss.

[UVA2B]

As a slightly more existential line of questions: what should this data calculator prove exactly, and what do you hope users will extrapolate from it? Do you see a percentage output of, say, 48% for a given school in admissions and 28% waitlist (completely made up these numbers), to mean more than what LSN and myLSN provide? And do you think that should be relied on as an accurate predictor of chances for a given applicant?

I guess I'm just wondering if this really captures what you think has been missed in other data-driven models. I'm not questioning your regression or data analysis, but I guess I'm questioning whether it actually proves anything new.

Also, if you're relying on raw data from LSN, how do you explain trying to attach fixed values on crowdsourced information? I doubt there is much statistical noise by way of fake profiles, but between fake profiles and incomplete profiles, the raw data isn't strictly reliable, to the extent it ever was. It's a great weathervane for a general notion of chances of a given outcome, but it's not absolutely trustworthy.

Forgive me if this comes off poorly, I just want you to defend your regression and how you believe an applicant should digest what it provides. Because unfortunately attaching numeric values to an objective process colored by tons of subjectivity isn't easily translatable in statistics.

I guess I want to know how much wiggle room you want your regression to maintain in its accuracy.

[dabigchina]

Neat tool, but I refuse to accept this T14+1 nomenclature.

[icechicken]

As a slightly more existential line of questions: what should this data calculator prove exactly, and what do you hope users will extrapolate from it? Do you see a percentage output of, say, 48% for a given school in admissions and 28% waitlist (completely made up these numbers), to mean more than what LSN and myLSN provide? And do you think that should be relied on as an accurate predictor of chances for a given applicant?

I have a related concern: using regressions in this way seems to smooth out the data in a way that contradicts received wisdom about the importance of certain break-points (25/50/75th percentiles, GPA floors, "wow" numbers like 4.0). For instance, if one has an LSAT of 180, this tool seems to imply that their Harvard chances smoothly increase (from 3% to 97%) as their uGPA increases from 3.25 to 3.85, with a coin flip's chance at 3.53. It's probably not that simple, especially since 180/3.5+ seems to be close to an auto-admit with a very sharp dropoff for GPAs below that.

[Eman1994]

Well if this calculator is accurate I'm not gonna have a very good cycle. I'm an AA Male with about a 3.1 and a 167 LSAT. I've already been waitlisted at Duke (which is what the calculator predicted). The only two T15 schools it predicts I get ouright admitted to are Georgetown and Texas but that's not too bad for me because Georgetown is one of my top choice schools anyway.

[sparkytrainer]

Neat tool, but I refuse to accept this T14+1 nomenclature.

Agreed. Given that GT's and Texas's employment numbers aren't close to the t13, it makes it clear there is a t13 tier, then a t20 tier that includes GT, Texas, UCLA, etc.

Its much easier to just use t13.

[J Eazy]

As a slightly more existential line of questions: what should this data calculator prove exactly, and what do you hope users will extrapolate from it? Do you see a percentage output of, say, 48% for a given school in admissions and 28% waitlist (completely made up these numbers), to mean more than what LSN and myLSN provide? And do you think that should be relied on as an accurate predictor of chances for a given applicant?

I have a related concern: using regressions in this way seems to smooth out the data in a way that contradicts received wisdom about the importance of certain break-points (25/50/75th percentiles, GPA floors, "wow" numbers like 4.0). For instance, if one has an LSAT of 180, this tool seems to imply that their Harvard chances smoothly increase (from 3% to 97%) as their uGPA increases from 3.25 to 3.85, with a coin flip's chance at 3.53. It's probably not that simple, especially since 180/3.5+ seems to be close to an auto-admit with a very sharp dropoff for GPAs below that.

Your search here is for GPAs 3.5-3.8. Most of these data points in this search are for GPAs 3.6-3.8 (12 admits, 0 waitlists, 0 rejects). For GPAs 3.5-3.6, it's 3 accepts, 0 waitlists, 2 rejects—close to OP's coin's flip chance at 3.5x.

[icechicken]

Your search here is for GPAs 3.5-3.8. Most of these data points in this search are for GPAs 3.6-3.8 (12 admits, 0 waitlists, 0 rejects). For GPAs 3.5-3.6, it's 3 accepts, 0 waitlists, 2 rejects—close to OP's coin's flip chance at 3.5x.

Hmm, that's a really good point. I wish we had enough data to be confident one way or the other.

[principalagent]

Well if this calculator is accurate I'm not gonna have a very good cycle. I'm an AA Male with about a 3.1 and a 167 LSAT. I've already been waitlisted at Duke (which is what the calculator predicted). The only two T15 schools it predicts I get ouright admitted to are Georgetown and Texas but that's not too bad for me because Georgetown is one of my top choice schools anyway.

To be fair, there’s a caveat on the page that African Americans still do a fair bit better than the URM toggle by itself suggests. The calculator seems to be tracking my cycle pretty well as another AA male (with elite undergrad).

[wmbuff]

Neat tool, but I refuse to accept this T14+1 nomenclature.

Agreed. Given that GT's and Texas's employment numbers aren't close to the t13, it makes it clear there is a t13 tier, then a t20 tier that includes GT, Texas, UCLA, etc.

Its much easier to just use t13.

I know folks keep saying T20 as a grouping, but what's the twentieth school? The rankings seem to have stabilized to show a top 19, with the 20th spot shifting around (sometimes even a tie at 19). In the 2010-18 rankings, the only times any of those 19 have fallen out of the top 19 are WUSTL in 2013 and USC in 2015-16, while the extra spot or two has included brief stints by Iowa (twice), Notre Dame, Emory (three times), Boston U (twice), Minnesota (six times), George Washington (three times), and the University of Washington. Are there 26 schools in the T20, or 19? Or is Minnesota the twentieth, due to its frequent appearances?

[srt2021]

Hey all,

Thanks for the feedback (and for testing it out!)

Responding to what y'all have raised:

First (and foremost), just want to point to the accuracy/variance tables as a reminder that whether folks end up in the admit, waitlist, or deny piles is a function of more than numbers. Spoiler alert here so you can get to the conclusion.

[+] Spoiler

Two ways we can gather this:

(1) Softs: I'd run "softs" from this site's calculators in addition to the data, and self-evaluated softs often had a pretty large impact on admit rates--problem with incorporating that data is that the folks who fill out the spreadsheets here have a ridiculously high probability of getting in despite their numbers and that LSN doesn't systematically collect the data.

(2) The waitlist pile: I re-ran the numbers for Yale (since it has the lowest percentage of variance explained), Michigan (since it has the lowest retrodictive accuracy apart from UVa), and NYU (since it's the closest to the numbers game we've come to love-hate and expect).

Counting waitlisted students as denied students, Yale's accuracy went from 74 to 85%, while variance explained went from 33 to 30%. For Michigan, accuracy went from 60 to 66% and variance explained from 39 to 21% (!!!). For NYU, accuracy went from 79 to 86% and variance explained stayed the same.

Counting waitlisted students as accepted students, Yale's accuracy went from 74 to 77%, while variance explained went from 33 to 35%. For Michigan, accuracy went from 60 to 85% (!!!) and variance explained from 39 to 49%. For NYU, accuracy went from 79 to 90% and variance explained from 67 to 70%.

The above seems to suggest that waitlisted students are pretty close to admits than they are to rejects in a way that makes it hard for any model to distinguish between them very effectively.

Most importantly, all the above suggests that in the T14 overall, while numbers play a huge role, they're absolutely not determinative. Even at NYU, at least 30% of the variance in decisions comes from other factors, while at Yale, 65% of the variance comes from other factors. That's where softs, LORs, essays, and all the minor details it'd be too complicated to factor in come into play.

Point being: when in doubt, it may make sense to apply anyway, especially if the softs are in one's favor.

To the individual points:
1. rowdy: Why the floor? Because the logistic regression equations start acting super weirdly below that floor. As an example, test out 130/1.50 for both URM and non-URM applicants (everyone but Penn, UVa, NW, and Berkeley gives exceptionally unlikely outcomes). 160/3.00 is arbitrary, granted, but it covers 91.5% of the applicants included (hence why the formulas do weird things) and ensures that the T6 doesn't start increasing folks chances the further they go down. In any case, if anyone wants to go below that, they can use the number input on the right.

2. Robb: Hearing back from schools. I haven't taken a close enough look at this using LSN data to know how straightforward it would be. I remember trying to use the spreadsheets here and failing miserably on that front. I can take a look again at some point, though!

3. UVA2B: Lots of good points, and not coming off poorly at all. A few thoughts, for what they're worth (bolding things for others' sakes, since they're the main points):

My original interest was to see how much of the variation in admissions decisions could be explained by these big three factors and to see how much was left to others (as above). What it proves pretty solidly is abstract: numbers matter--a lot--but numbers are definitely not the only things that matter or even most of what matters (e.g., both Yale acceptances yesterday went to folks who the calculator says were likely to be denied, based on the numbers alone). In that sense (to your point about what it proves that's new), it doesn't necessarily prove more than what we could possibly already visualize on something like myLSN--it just helps to quantify that reality a bit more. In particular, I'm surprised at how little variance the three factors explain and yet at how accurate they are in retrospect. Still, overall they're about as accurate in prediction as a weather forecast three days out.

As to what the percentage outputs offer, then, they're rough estimates of where folks stand based on numbers alone. No one should look at them and say "oh, my chances are super high/low at X, I'm definitely getting in/rejected"; I hope instead they provide a sense for concerned applicants that there's a chance of being accepted or waitlisted even when they're in the red and a sense for cocky applicants that there's still a chance of being waitlisted or rejected even when they're in the green. That said, the only advantage potentially, relative to LSN/myLSN, is making it a little bit easier for slightly more unusual applicants (e.g., URMs in general and certain splitters) to have that data smoothed out in a way that's shown to be successful historically (and that at the end of this cycle I want to check for that potential predictive quality).

You're definitely right that LSN is a weird place data-wise. The admit rates predicted by the formulas seem consistent with actual admissions rates, though, which I also didn't expect (I expected LSN to be significantly higher the way that TLS was). Re: data accuracy, I omitted empty LSAT scores and GPAs as well as LSAT scores below 140 and GPAs below 2.00 to do my best to control for incomplete data and misleading numbers. Still, given the thousands of records per school that hopefully folks haven't fudged too much, it's probably the best we'll get. The better term for the outcomes predicted may be precision rather than accuracy, if we were being particular.

4. ice chicken: Smoothing data and break-points. I was surprised at this, too, looking at the numbers. The percentiles and floors don't seem to matter nearly as much as people think they do per se. As J Eazy pointed out, I think what happens when we look up the data in certain ranges is that we're actually comparing different things than what we think we are. Example, re: 3.5-3.8 versus 3.2-3.5: the average for the first range is about 3.68 and the latter about 3.44. When we compare 3.5-3.8 and 3.2-3.5 for Harvard on myLSN, we get 88% and 33%. Combining the two, we're at 76%, but only because there's a significantly higher percentage of applicants in the first group than in the latter. On the calculator, we get 84% and 25%, with an average of 41% at 3.50, which is a little lower than the 50% myLSN gives at that range when you do it automatically. In any case, if you visualize the smoothed-out function, you notice a pretty exponential looking rise through that range, just as we might expect.

All around, lots of surprises for me going through this data.

[srt2021]

Also, additional caveat, having tried to see if gender mattered: self-identified females on LSN did marginally better (a few percentage points) than self-identified males, with those who didn't choose between them for whatever reason doing better than both groups (a few percentage points). Unclear why exactly, but for what it's worth.

[srt2021]

Last note for now: I really wish LSN noted more systematically if folks wrote "Why X" essays to run those as well--I'd imagine that could help explain why some schools outside the T6 have weird waitlist rates.

[dabigchina]

Neat tool, but I refuse to accept this T14+1 nomenclature.

Agreed. Given that GT's and Texas's employment numbers aren't close to the t13, it makes it clear there is a t13 tier, then a t20 tier that includes GT, Texas, UCLA, etc.

Its much easier to just use t13.

I know folks keep saying T20 as a grouping, but what's the twentieth school? The rankings seem to have stabilized to show a top 19, with the 20th spot shifting around (sometimes even a tie at 19). In the 2010-18 rankings, the only times any of those 19 have fallen out of the top 19 are WUSTL in 2013 and USC in 2015-16, while the extra spot or two has included brief stints by Iowa (twice), Notre Dame, Emory (three times), Boston U (twice), Minnesota (six times), George Washington (three times), and the University of Washington. Are there 26 schools in the T20, or 19? Or is Minnesota the twentieth, due to its frequent appearances?

Good point. It should really be T19. I think when people say T20, they are talking about the group that has stabilized around the top 19 spots.

[wmbuff]

Neat tool, but I refuse to accept this T14+1 nomenclature.

Agreed. Given that GT's and Texas's employment numbers aren't close to the t13, it makes it clear there is a t13 tier, then a t20 tier that includes GT, Texas, UCLA, etc.

Its much easier to just use t13.

I know folks keep saying T20 as a grouping, but what's the twentieth school? The rankings seem to have stabilized to show a top 19, with the 20th spot shifting around (sometimes even a tie at 19). In the 2010-18 rankings, the only times any of those 19 have fallen out of the top 19 are WUSTL in 2013 and USC in 2015-16, while the extra spot or two has included brief stints by Iowa (twice), Notre Dame, Emory (three times), Boston U (twice), Minnesota (six times), George Washington (three times), and the University of Washington. Are there 26 schools in the T20, or 19? Or is Minnesota the twentieth, due to its frequent appearances?

Good point. It should really be T19. I think when people say T20, they are talking about the group that has stabilized around the top 19 spots.

I mean, if T20 traditionally means the schools I'm talking about plus Minnesota, I guess saying T19 is throwing shade at them the way T13 throws shade at GULC. I'm legitimately curious what the twentieth school is supposed to be.

[dabigchina]

Neat tool, but I refuse to accept this T14+1 nomenclature.

Agreed. Given that GT's and Texas's employment numbers aren't close to the t13, it makes it clear there is a t13 tier, then a t20 tier that includes GT, Texas, UCLA, etc.

Its much easier to just use t13.

I know folks keep saying T20 as a grouping, but what's the twentieth school? The rankings seem to have stabilized to show a top 19, with the 20th spot shifting around (sometimes even a tie at 19). In the 2010-18 rankings, the only times any of those 19 have fallen out of the top 19 are WUSTL in 2013 and USC in 2015-16, while the extra spot or two has included brief stints by Iowa (twice), Notre Dame, Emory (three times), Boston U (twice), Minnesota (six times), George Washington (three times), and the University of Washington. Are there 26 schools in the T20, or 19? Or is Minnesota the twentieth, due to its frequent appearances?

Good point. It should really be T19. I think when people say T20, they are talking about the group that has stabilized around the top 19 spots.

I mean, if T20 traditionally means the schools I'm talking about plus Minnesota, I guess saying T19 is throwing shade at them the way T13 throws shade at GULC. I'm legitimately curious what the twentieth school is supposed to be.

I frankly don't think there should be a 20th school. I think people use the T20 nomenclature because 20 is a nice round number. However, the schools that rotate in the 20 spot are not peer institutions with Vanderbilt, UCLA, UT, GULC, and to a lesser extent, WUSTL and USC.

[enoca]

Yeah, when you look at the cardinal scores, there is a pretty clear grouping that doesn't really mesh with t14+1 or t20.

http://excessofdemocracy.com/blog/2017/ ... -presented

[srt2021]

Well that's a cool chart. Thanks for sharing.

On a different note, I did some analyses of when decisions are sent relative to folks' numbers and when folks applied. The good news is there's a relatively strong relationship (to the tune of about 30-50% of variance explained). The bad news is that there's no way to present that information that's not fairly misleading: showing it as a percentage likelihood of hearing back in a certain month has a very high misclassification rate (~70%) and showing it as a fixed date isn't terribly helpful (what does it mean that you're most likely to hear back on February 15?). So that'll remain as it is.

[sodomojo]

No discussion about the ED chances table?

These numbers seem to indicate that ED gives a very sizable boost in chances at just about every school listed. Whereas TLS conventional wisdom seems to shun ED because: (1) You're tossing away $$$, and (2) You're at best only getting a marginal bump in odds.

Is reason (2) really just a bunch of baloney then?

[pupper]

Wow, I love this! I particularly appreciate the focus on how much variance LSAT/GPA explains, which is something severely lacking in other admissions calculators.

I played around with some similar analysis earlier in the year and found that a rules-based algorithms performed better in cross validation than logistic regression for some schools. Perhaps there is something to the idea of "break-points" and "floors" for some schools? Either way, if the conventional wisdom doesn't match the data, that's good a sign it's time to adjust the conventional wisdom. We gotta "update our priors".

Have you tried any other algorithms, srt2021? I always wanted to give SVMs a shot, but never got around to it.

[srt2021]

I'm glad you found it interesting! And that's good info. Do you remember which schools they might have been? Honestly, I didn't get a chance to try out much else, partly because I made this almost as soon as I noticed the pretty surprising trends (in some cases) I described and partly because I then got caught up with finals and holidays. I might try out some other stuff next week once things aren't so hectic, though.

As to ED specifically...yeah, it's weird. There's almost certainly a boost--how much that should factor in is a different question. I'd lean toward the standard advice but mostly because of the former reason and the possibility of freaking yourself out so much that you limit your options unnecessarily.

[icechicken]

No discussion about the ED chances table?

These numbers seem to indicate that ED gives a very sizable boost in chances at just about every school listed. Whereas TLS conventional wisdom seems to shun ED because: (1) You're tossing away $$$, and (2) You're at best only getting a marginal bump in odds.

Is reason (2) really just a bunch of baloney then?

My personal theory is that ED only makes a substantial and reliable difference for applicants who check key boxes and are therefore valuable to "lock in" to the class at MSRP (e.g. a reverse-splitter at Penn). If I'm right, OP's regression-based analysis might be smoothing things out so that it appears that everybody is getting a small bump, rather than that a few people are getting a big one.

Another huge problem is that the ED sample on LSN is especially small and also probably biased toward wealthier and/or more sophisticated applicants.