The Myth of Yield Protection

(Applications Advice, Letters of Recommendation . . . )
Sandrew
Posts: 87
Joined: Sat Feb 05, 2011 6:01 pm

The Myth of Yield Protection

Postby Sandrew » Tue Feb 22, 2011 1:01 am

Edited 2/24 to admit to some shoddy statistics on my part. I've kept the original intact as evidence of my blunder and to preserve the context for later helpful comments from users mst, D. H20man, and others.

Preemptive tl;dr: Yield protection at mostT14 schools (evenUVA and Penn being two notable exceptions) is a myth. For lower-ranked schools where yield protection does occur, I offer a defense thereof. Accusations of yield protection are better described as ego protection on the part of the accuser.

Earlier today I posted the following to the GULC Waiting thread:
Sandrew wrote: Tough day Friday. I counted 21 or 22 applicants reporting waitlisted 2/18 on TLS and LSN, with numbers ranging from expected "auto-admits" to candidates with more marginal numbers.

175/3.76
174/3.9
174/3.38
173/3.90
172/3.92
171/3.55
171/3.4 URM
170/3.40
170/2.94
170/2.60 URM
169/3.89
169/3.88
167/3.80
167/3.75 (possibly double-counted)
167/3.7
167/3.68
166/3.8
165/3.85 URM
165/3.7
163/4.01
163/3.99
158/3.50 URM


I imagine many TLSers will glance at the top end of the above waitlist and immediately suspect “yield protection.” For those who were waitlisted with big numbers, for whom there is no obvious reason to think their application is otherwise weak, and who may reasonably choose to attend GULC over all other schools to which they’ve so-far been admitted, I certainly sympathize. But I'm skeptical of claims of yield protection--or at least of the motive ascribed to it (i.e. massaging the USNews rankings). Below, I explain why, and then take a look at the empirical evidence of yield protection.

Why the skepticism?

First, let’s remember that a law school’s yield (percentage of admitted students matriculating) is not among USNews’ rankings metrics. Acceptance rate is such a metric, and yes, it's possible to conceive of a nefarious strategy of waitlisting or rejecting "overqualified" candidates en mass in order to achieve a boost in this metric. But at what cost?

Waitlisting qualified candidates risks losing some of them to other schools, dragging down the "hard" metrics of median LSAT and GPA—two metrics that have a combined weight in the USNews rankings 9 times that of acceptance rate. I don't have any data on the subject, but I'd hazard a guess that highly qualified candidates are more likely to matriculate if accepted outright than if waitlisted and later accepted. Several factors could contribute to this, ranging from the psychological (impatience or aversion to uncertainty) to economic (the timing of scholarship offers from competing schools, many of which "explode"). If my suspicion is correct, this suggests to me that yield protection isn't a good strategy for juicing a school's ranking.

Empirical Analysis of Yield Protection

Let's table the accusations of rankings manipulation for a moment, and instead focus on the empirical questions of whether and where yield protection exists. Here, I'm defining yield protection as the practice of waitlisting overqualified candidates, irrespective of motive.

If yield protection exists at a school, we would expect to see higher rates of waitlisting among the most qualified candidates relative some subset of less qualified candidates. Let’s start with a prototypical example of an accused yield protector: American University.

Image

On the horizontal axis, I have grouped applicants by LSAT deciles. At AU, the first decile (bucket #1 above) corresponds to a high LSAT score of 167 and above. Vertical axis is the proportion of each bucket that got accepted/waitlisted/denied. See the sidebar below for a word of warning on interpreting the vertical axis. Acceptance rates are in green, waitlist rates in blue, and denial rates in red. I've graphed them additively (i.e. "stacked"), as they always sum to 1 by definition.

In geekspeak, what we see in the above graph is that acceptance rates are a nonmonotonic function. In layman’s terms, this means it’s as humped as a camel’s back. This graph points to strong evidence of yield protection at AU; a large chunk of candidates appear to have been waitlisted on the basis of a too-high LSAT.

Conclusion: American University yield protects.

Side note: Please be cautious of the absolute acceptance rates on the left-hand axis, both above and throughout. LSN data are demonstrably skewed toward higher-achieving candidates (i.e. those more likely to be accepted). Nevertheless, I don’t believe this “LSN skew effect” is fouling up my conclusions, since what matters for purposes of demonstrating yield protection is relative rates of acceptance/waitlisting across index deciles.

Do the Top 14 Law Schools (T14) engage in yield protection?

Let’s start with a school that not even the most jaded TLSer would accuse of yield-protecting highly qualified candidates: Columbia.

Image

As we would expect of a non-yield protecting school, acceptance rates at CLS are (for the most part) monotonically increasing with an applicant’s index. The inverse is true of rejection rates. Lastly, the waitlist rate (blue/purple) is monotonically increasing over the first through fifth deciles, and monotonically decreasing thereafter. Neat and clean. No yield protection here.

Let’s move on to more widely suspected yield protectors among the T14. See note [1] below. Some T14 do appear to use yield protection as I've defined it.

Image
Image
Image

As shown above, acceptance rates are across-the-board monotonic at these T14 law schools. The conventional wisdom (around TLS, anyway) is that UVA and Penn are more liberal with their waitlists (and stingy with their admittances), on the basis of numbers, than other T14 schools. This fact is borne out by the data. But while UVA and Penn waitlist a lot of qualified applicants, they’re no more likely (less likely, in fact) to waitlist their more-qualified applicants over less-qualified ones. These data are entirely consistent with each school’s putting stronger emphasis on so-called soft factors that don’t necessarily correlate with candidates’ hard stats, such as in-state residency at UVA or work experience at Penn. Come to think of it, this is exactly what these schools avow—a holistic approach to admittances. Nothing to get excited about here.

Conclusion: Yield protection at T14 law schools is a myth. See note [1] below. Some T14 do appear to use yield protection as I've defined it.

For good measure, let’s dip a bit lower into the USNews rankings and check out George Washington University (ranked 20th).

Image

It looks like a small degree of yield protection is possible at GWU (oddly though, it seems to be in the form of dings, not waitlists). But as an armchair statistician, I can’t say for sure whether it’s statistically significant.

Conclusion: Ambiguous.

So what explains yield protection where it’s prevalent?

While it’s apparent that top-ranked law schools don’t engage in yield protection, it’s equally apparent that some lower-ranked schools do. I explained at the outset why I’m skeptical of attributing dubious motives to the practice, but I suppose I owe an alternative narrative if I’m to defend American University for waitlisting approximately 40% of its top-scoring applicants.

Each admissions committee has an interest in building the best possible student body. Due to limited class size, AdComs obviously can't admit every qualified candidate and hope for the best. Furthermore, due to unpredictable matriculation rates, AdComs can’t just calibrate-upwards the minimum admissions criteria such that the expected number of matriculants equals their desired class size. To do so would risk an overabundance of matriculating students, potentially flooding classes with students and overwhelming the school’s scarce resources.

Predicting which admitted students are most likely to matriculate is crucial to building a class. Such predictions are difficult due to information asymmetry--a candidate knows whether she prefers, say, GWU to GULC, but the AdComs of each do not. The solution to this information problem is yield protection. By waitlisting overqualified candidates, the AdComs can use the waitlist period to solicit incremental information about each candidate’s preferences (via letters of continued interest and Why X essays). There’s nothing nefarious about yield protecting in this manner; you might even say it’s downright polite, both to the candidate applying as a safety school and, more importantly, to those applying as a reach.

Next time you see a candidate with impressive numbers waitlisted, I hope you'll hesitate to cry shenanigans for manipulation of metrics. I suspect such protests are more likely evidence of the applicant protecting his own ego than of a school protecting its yield.

Note 1: Thanks to the insightful comments from TLSers in this thread, I've changed my view on the question of yield protection at certain schools. However, I maintain my skepticism toward the rankings-manipulation motivation for the reasons discussed above.

I had two major methodological errors in my analysis that are discussed in the subsequent comments, particularly on page 3. Correcting for these errors (to the extent possible), the two graphics below demonstrate that UVA likely practices yield protection.


Image

Image
Last edited by Sandrew on Thu Feb 24, 2011 11:55 am, edited 1 time in total.

FiveSermon
Posts: 1507
Joined: Tue Dec 14, 2010 6:56 pm

Re: The Myth of Yield Protection

Postby FiveSermon » Tue Feb 22, 2011 1:05 am

Holy shit, does someone have a TL;DR version of this?

User avatar
lisjjen
Posts: 1242
Joined: Wed Jun 30, 2010 12:19 am

Re: The Myth of Yield Protection

Postby lisjjen » Tue Feb 22, 2011 1:08 am

But I got WL at UChi. Isn't that YP?

User avatar
fatduck
Posts: 4186
Joined: Mon Sep 13, 2010 10:16 pm

Re: The Myth of Yield Protection

Postby fatduck » Tue Feb 22, 2011 1:10 am

Sandrew wrote:Each admissions committee has an interest in building the best possible student body. Due to limited class size, AdComs obviously can't admit every qualified candidate and hope for the best. Furthermore, due to unpredictable matriculation rates, AdComs can’t just calibrate-upwards the minimum admissions criteria such that the expected number of matriculants equals their desired class size. To do so would risk an overabundance of matriculating students, potentially flooding classes with students and overwhelming the school’s scarce resources.


I have made this argument several times on TLS only to be called an idiot. I appreciate this post.

bhan87
Posts: 850
Joined: Mon Mar 12, 2007 8:08 pm

Re: The Myth of Yield Protection

Postby bhan87 » Tue Feb 22, 2011 1:16 am

I guess GULC just really detected my apathy when they waitlisted me :roll:

(Btw, I used the same personal statements and LORs for Berkeley and Columbia, both of which auto-admitted me)

Sandrew
Posts: 87
Joined: Sat Feb 05, 2011 6:01 pm

Re: The Myth of Yield Protection

Postby Sandrew » Tue Feb 22, 2011 1:24 am

fatduck wrote:I have made this argument several times on TLS only to be called an idiot. I appreciate this post.


You're welcome! Happy to see others appreciate my evening's labor.

lisjjen wrote:But I got WL at UChi. Isn't that YP?


No.

Ed note: I'm surprised more people don't muck about with LSN data. They can tell an interesting story. Perhaps more would if they were available in csv. It's a pity they're not. Consequently, it took a lot more work than I anticipated to build the graphs. For that reason, I won't be taking requests for particular schools' graphs, although I can provide more color on methodology if requested.

User avatar
jpSartre
Posts: 326
Joined: Sun Jan 03, 2010 11:05 am

Re: The Myth of Yield Protection

Postby jpSartre » Tue Feb 22, 2011 1:32 am

Everyone shut up. This kid is cool.

User avatar
The Stig
Posts: 567
Joined: Sat Oct 30, 2010 12:27 pm

Re: The Myth of Yield Protection

Postby The Stig » Tue Feb 22, 2011 1:33 am

great write up! it is good to see this kind of analysis on TLS

Sandrew
Posts: 87
Joined: Sat Feb 05, 2011 6:01 pm

Re: The Myth of Yield Protection

Postby Sandrew » Tue Feb 22, 2011 1:35 am

jpSartre wrote:Everyone shut up. This kid is cool.

Thank you. I'm 31. And please, everyone, speak up. Criticism/suggestions welcome.

Celtic Bhoy
Posts: 8
Joined: Fri Feb 18, 2011 4:38 pm

Re: The Myth of Yield Protection

Postby Celtic Bhoy » Tue Feb 22, 2011 1:38 am

This was definitely an interesting read so thanks for providing it! I did have one thought about it though that may or may not support your conclusions. I could be wrong, in which case don't worry about it.

This mostly applies to the T14 schools, but I don't really think that the data sets you used are appropriate for them to disprove yield protection. No matter the T14, people scoring in the bottom deciles on the LSAT are going to find enormous amounts of rejections and in some cases waitlists. The lowest 25th percentile for a T14 is a 165 on the LSAT (for Cornell). This is just outside the top scoring decile (which you noted was 167).

Because of this, the majority of people we'd expect to be "targets" and "over-qualified" for these schools are all crowded in one data set simply listed as the top decile. I know it's not the case, but hypothetically all of the waitlists on the UVA graph (just for example) could be from the 175-180 range and all acceptances below that. In that case, there would still be yield protection occurring, but it could be masked simply by the way the data is shown in your graphs. To get a better picture on yield protection (at least at the T14), I would think you'd have to break down the top two deciles (those where the majority of scores fall for these schools) into small groupings like 178-180, 175-177, etc.

I'm not sure if this would change the results, but I think it would clarify each schools trends a bit. I haven't fully thought it through though so if I'm way off let me know.

User avatar
AreJay711
Posts: 3406
Joined: Tue Jul 20, 2010 8:51 pm

Re: The Myth of Yield Protection

Postby AreJay711 » Tue Feb 22, 2011 1:40 am

Celtic Bhoy wrote:This was definitely an interesting read so thanks for providing it! I did have one thought about it though that may or may not support your conclusions. I could be wrong, in which case don't worry about it.

This mostly applies to the T14 schools, but I don't really think that the data sets you used are appropriate for them to disprove yield protection. No matter the T14, people scoring in the bottom deciles on the LSAT are going to find enormous amounts of rejections and in some cases waitlists. The lowest 25th percentile for a T14 is a 165 on the LSAT (for Cornell). This is just outside the top scoring decile (which you noted was 167).

Because of this, the majority of people we'd expect to be "targets" and "over-qualified" for these schools are all crowded in one data set simply listed as the top decile. I know it's not the case, but hypothetically all of the waitlists on the UVA graph (just for example) could be from the 175-180 range and all acceptances below that. In that case, there would still be yield protection occurring, but it could be masked simply by the way the data is shown in your graphs. To get a better picture on yield protection (at least at the T14), I would think you'd have to break down the top two deciles (those where the majority of scores fall for these schools) into small groupings like 178-180, 175-177, etc.

I'm not sure if this would change the results, but I think it would clarify each schools trends a bit. I haven't fully thought it through though so if I'm way off let me know.

User avatar
Bildungsroman
Posts: 5548
Joined: Sun Apr 11, 2010 2:42 pm

Re: The Myth of Yield Protection

Postby Bildungsroman » Tue Feb 22, 2011 1:42 am

Almost everyone who claims yield protection is doing it to cushion their ego.

User avatar
NZA
Posts: 1285
Joined: Fri Nov 19, 2010 10:01 pm

Re: The Myth of Yield Protection

Postby NZA » Tue Feb 22, 2011 1:43 am

But...then it's my fault I didn't get accepted to HYS?
:shock:

I thought I was too good for them.

User avatar
DeeCee
Posts: 1352
Joined: Mon Nov 15, 2010 4:09 am

Re: The Myth of Yield Protection

Postby DeeCee » Tue Feb 22, 2011 1:43 am

Bildungsroman wrote:Almost everyone who claims yield protection is doing it to cushion their ego.


I have noticed that a lot of people with great numbers will have no softs or not much else besides the numbers. This is probably why they are WL/dinged.

User avatar
Cade McNown
Posts: 550
Joined: Tue Jan 04, 2011 10:54 pm

Re: The Myth of Yield Protection

Postby Cade McNown » Tue Feb 22, 2011 1:45 am

Thank you for having too much time on your hands.

User avatar
DeeCee
Posts: 1352
Joined: Mon Nov 15, 2010 4:09 am

Re: The Myth of Yield Protection

Postby DeeCee » Tue Feb 22, 2011 1:46 am

Cade McNown wrote:Thank you for having too much time on your hands.


why judge the OP?

The Real Jack McCoy
Posts: 279
Joined: Wed Aug 18, 2010 3:55 pm

Re: The Myth of Yield Protection

Postby The Real Jack McCoy » Tue Feb 22, 2011 1:46 am

I'd like to see the Michigan data for waitlists with a Why Michigan is awesome versus waitlists without a Why Michigan is awesome. The same could be said for UVA, etc.
Last edited by The Real Jack McCoy on Tue Feb 22, 2011 1:49 am, edited 2 times in total.

ptblazer
Posts: 376
Joined: Sun Oct 31, 2010 11:27 pm

Re: The Myth of Yield Protection

Postby ptblazer » Tue Feb 22, 2011 1:47 am

I know you said you aren't doing more graph work, but I think an interesting analysis would be the tendencies of YP in areas where there are multiple law school options. I think geography is by far the biggest factor in whether a school practices YP and the degree at which it does. DC for example, adcomms probably safely assume that any over qualified applicant at American U is also applying to GULC and using American U as a safety school. Law schools that are the best (highest ranked) option in a particular geographic region probably practice a lesser degree of YP or not at all.
Last edited by ptblazer on Tue Feb 22, 2011 2:10 am, edited 1 time in total.

mst
Posts: 925
Joined: Sun Jun 06, 2010 3:01 am

Re: The Myth of Yield Protection

Postby mst » Tue Feb 22, 2011 1:50 am

How did you choose your index? Seems like your data could go 100 other ways depending on that. 60% gpa and 40% lsat is pretty darn optimistic for GPA folks...

Also, your basic suspicion that the numbers being worth 9 times more than acceptance or yield, etc. is kind of flawed. I would hardly be surprised if over 90% of the people being yield protected would choose "better" schools anyways. I think admissions committees find this point where they lose more students than they potentially gain in hard-number points and yield from there on. You also assume that the only factor for yield-protect is rankings, when it also adds a lot of other benefits: easier to dole out scholarships at a pace that suits people that will actually attend, better numbers to publicize as far as yield (regardless of rankings), easier to figure out quartiles earlier in the process, etc.

User avatar
jtemp320
Posts: 481
Joined: Fri Jul 30, 2010 6:27 pm

Re: The Myth of Yield Protection

Postby jtemp320 » Tue Feb 22, 2011 1:58 am

This is interesting and I definitely believe there are a lot of people who cry yield protection to make themselves feel better about an unexpected waitlist. On the other hand some schools absolutely yield protect - I'll offer the following counterexample which has been mentioned on TLS before - UC Hastings:

ttps://officialguide.lsac.org/Release/S ... AC4342.pdf

Take a look at the applicant profile on the second page. It appears that as applicants' GPAs and LSAT scores increase their odds of acceptance increase correspondingly...until a certain point - at which it becomes almost impossible for an applicant to get in...

Note the applicants who apply with a 3.5+ and a 170+...only 1 of 52 were accepted. Of the group with a 3.5+ but a 165-169, 176 of 243 were accepted. Were all those 170+ high GPA applicants quantitatively qualified but submitted qualitatively deficient applications causing the adcomms not to offer them admission despite their high numbers? Seems unlikely...

Much more likely is that UC Hastings practices pretty systematic yield protection.

That said, I think yield protection as a practice is far less widespread then many of us would like to think. UChicago, for example, (where I was waitlisted above both 75ths) probably does not yield protect. More likely they just have a small class, a very competitive process and a specific admissions philosophy.

Back to Hastings...from their perspective (or any school that practices YP), if you see an applicant with numbers high above your 75ths (even one who shows specific interest in your program) you probably have to commit significant scholarship money in order to have a chance at actually securing their enrollment. If you waitlist them and for some reason Hastings, and not Boalt is their dream school, they will try to get in off the waitlist and if you accept them then they won't hurt your yield (which would affect your ranking) and won't force you to commit limited money earlier in the cycle that you won't be able to use on other applicants who are far more likely to attend. This is even more tempting if an applicant doesn't show specific interest (even less likely to attend) or has very high numbers but weak softs making them someone who the adcomms don't want to use limited resources on.

The bottom line is - schools are playing the game too and the evidence shows yield protecting is sometimes part of it. Probably not to the extent speculated about on TLS - but at least at some places - it happens.

While it happens, I've come to terms with the fact that I didn't YP'd at Chicago (I just wasn't a special enough snowflake for Hyde Park's rigor)... and no, you probably didn't get yield protected at Harvard because they were sure you'd pick Yale instead.

dabbadon8
Posts: 767
Joined: Tue May 11, 2010 2:17 am

Re: The Myth of Yield Protection

Postby dabbadon8 » Tue Feb 22, 2011 2:04 am

I think I was YP'ed by UIUC. Dean Pless is on these boards and may have read me say something stupid ha. Continuing with saying something potentially stupid...

Seriously though, not just for my ego, but if there is no YP why do you think UIUC waitlisted three people on these boards who will be now attending Michigan, Chicago and Duke? I am not saying that it is not beyond the realm of possibility that we simply were not what they were looking for...it could be, I just find it strange that people who had otherwise successful cycles would get waitlisted somewhere that most people on these boards predicted would give them $$$. My gf, got in with an identical gpa, an LSAT 12 points lower, same UG, same major, very similar softs. Maybe dean pless really is stalking the boards.

tothePAIN
Posts: 93
Joined: Thu Sep 16, 2010 2:53 pm

Re: The Myth of Yield Protection

Postby tothePAIN » Tue Feb 22, 2011 2:06 am

@ dabbadon8

I've got higher numbers than you and a full ride to Illinois. Doesn't seem to be YP there.

Sandrew
Posts: 87
Joined: Sat Feb 05, 2011 6:01 pm

Re: The Myth of Yield Protection

Postby Sandrew » Tue Feb 22, 2011 2:07 am

Celtic Bhoy wrote:This was definitely an interesting read so thanks for providing it! I did have one thought about it though that may or may not support your conclusions. I could be wrong, in which case don't worry about it.

This mostly applies to the T14 schools, but I don't really think that the data sets you used are appropriate for them to disprove yield protection. No matter the T14, people scoring in the bottom deciles on the LSAT are going to find enormous amounts of rejections and in some cases waitlists. The lowest 25th percentile for a T14 is a 165 on the LSAT (for Cornell). This is just outside the top scoring decile (which you noted was 167).

Because of this, the majority of people we'd expect to be "targets" and "over-qualified" for these schools are all crowded in one data set simply listed as the top decile. I know it's not the case, but hypothetically all of the waitlists on the UVA graph (just for example) could be from the 175-180 range and all acceptances below that. In that case, there would still be yield protection occurring, but it could be masked simply by the way the data is shown in your graphs. To get a better picture on yield protection (at least at the T14), I would think you'd have to break down the top two deciles (those where the majority of scores fall for these schools) into small groupings like 178-180, 175-177, etc.

I'm not sure if this would change the results, but I think it would clarify each schools trends a bit. I haven't fully thought it through though so if I'm way off let me know.


Thanks for the comment. You're a bit off base, but I'll let you down gently as it's partly my fault for not clearly defining how I devised the deciles.

Each set of deciles is unique to a school, and is based on all applicants in the pool (inclusive of URM and pending status). Let me unpack that a bit.

For simplicity, let's just look at LSAT scores (it gets more complicated when you weigh them against GPA). At Columbia, the top decile of LSAT scores corresponds to a 176, meaning that among the 1,813 LSN users who applied to Columbia during the two cycles ending '09 and '10, approximately 10% had scores of 176 or above. The next three "deciles" correspond to scores of 174, 173, and 171, and the tenth decile corresponds to anything less than 163. (I use deciles in scare-quotes here to indicate that they don't exactly correspond to 10% increments, due to lumpiness in score frequencies.) As you can see, these deciles are quite tightly bound. Now, compare Columbia to, say GWU. At GWU, the top four LSAT deciles correspond to scores of 171, 169, 167 and 167. (Note: 167 is repeated because the frequency of this score is so great among LSN users who applied to GWU.)

As I alluded to, it gets more complicated when you introduce GPA (leaving you with an Index combining LSAT and GPA). But it suffices to say that the deciles are a way of breaking up the entire applicant pool for a given school into (approximately) equal-size cohorts.

I can affirm that it is not the case that all of the waitlists at UVA are among those with LSATs greater than 175. These account for just 92 out of 868 waitlists.

Hope this helps!

User avatar
Cade McNown
Posts: 550
Joined: Tue Jan 04, 2011 10:54 pm

Re: The Myth of Yield Protection

Postby Cade McNown » Tue Feb 22, 2011 2:08 am

DeeCee wrote:
Cade McNown wrote:Thank you for having too much time on your hands.


why judge the OP?


I was being earnest.

Sandrew
Posts: 87
Joined: Sat Feb 05, 2011 6:01 pm

Re: The Myth of Yield Protection

Postby Sandrew » Tue Feb 22, 2011 2:09 am

ptblazer wrote:I know you said you aren't doing more graph work, but I think an interesting analysis would be the tendencies of YP in cities where there are multiple law school options. I think geography is by far the biggest factor in whether a school practices YP and the degree at which it does. DC for example, adcomms probably safely assume that any over qualified applicant at American U is also applying to GULC and using American U as a safety school. Law schools that are the best (highest ranked) option in a particular geographic region probably practice a lesser degree of YP or not at all.


Excellent point. You may be onto something. Perhaps I'll try Fordham?




Return to “Law School Admissions Forum”

Who is online

Users browsing this forum: Baidu [Spider], Bing [Bot], Lexaholik, mccracal and 3 guests