The Myth of Yield Protection
Posted: Tue Feb 22, 2011 1:01 am
Edited 2/24 to admit to some shoddy statistics on my part. I've kept the original intact as evidence of my blunder and to preserve the context for later helpful comments from users mst, D. H20man, and others.
Preemptive tl;dr: Yield protection at mostT14 schools (evenUVA and Penn being two notable exceptions) is a myth. For lower-ranked schools where yield protection does occur, I offer a defense thereof. Accusations of yield protection are better described as ego protection on the part of the accuser.
Earlier today I posted the following to the GULC Waiting thread:
Why the skepticism?
First, let’s remember that a law school’s yield (percentage of admitted students matriculating) is not among USNews’ rankings metrics. Acceptance rate is such a metric, and yes, it's possible to conceive of a nefarious strategy of waitlisting or rejecting "overqualified" candidates en mass in order to achieve a boost in this metric. But at what cost?
Waitlisting qualified candidates risks losing some of them to other schools, dragging down the "hard" metrics of median LSAT and GPA—two metrics that have a combined weight in the USNews rankings 9 times that of acceptance rate. I don't have any data on the subject, but I'd hazard a guess that highly qualified candidates are more likely to matriculate if accepted outright than if waitlisted and later accepted. Several factors could contribute to this, ranging from the psychological (impatience or aversion to uncertainty) to economic (the timing of scholarship offers from competing schools, many of which "explode"). If my suspicion is correct, this suggests to me that yield protection isn't a good strategy for juicing a school's ranking.
Empirical Analysis of Yield Protection
Let's table the accusations of rankings manipulation for a moment, and instead focus on the empirical questions of whether and where yield protection exists. Here, I'm defining yield protection as the practice of waitlisting overqualified candidates, irrespective of motive.
If yield protection exists at a school, we would expect to see higher rates of waitlisting among the most qualified candidates relative some subset of less qualified candidates. Let’s start with a prototypical example of an accused yield protector: American University.
On the horizontal axis, I have grouped applicants by LSAT deciles. At AU, the first decile (bucket #1 above) corresponds to a high LSAT score of 167 and above. Vertical axis is the proportion of each bucket that got accepted/waitlisted/denied. See the sidebar below for a word of warning on interpreting the vertical axis. Acceptance rates are in green, waitlist rates in blue, and denial rates in red. I've graphed them additively (i.e. "stacked"), as they always sum to 1 by definition.
In geekspeak, what we see in the above graph is that acceptance rates are a nonmonotonic function. In layman’s terms, this means it’s as humped as a camel’s back. This graph points to strong evidence of yield protection at AU; a large chunk of candidates appear to have been waitlisted on the basis of a too-high LSAT.
Conclusion: American University yield protects.
Side note: Please be cautious of the absolute acceptance rates on the left-hand axis, both above and throughout. LSN data are demonstrably skewed toward higher-achieving candidates (i.e. those more likely to be accepted). Nevertheless, I don’t believe this “LSN skew effect” is fouling up my conclusions, since what matters for purposes of demonstrating yield protection is relative rates of acceptance/waitlisting across index deciles.
Do the Top 14 Law Schools (T14) engage in yield protection?
Let’s start with a school that not even the most jaded TLSer would accuse of yield-protecting highly qualified candidates: Columbia.
As we would expect of a non-yield protecting school, acceptance rates at CLS are (for the most part) monotonically increasing with an applicant’s index. The inverse is true of rejection rates. Lastly, the waitlist rate (blue/purple) is monotonically increasing over the first through fifth deciles, and monotonically decreasing thereafter. Neat and clean. No yield protection here.
Let’s move on to more widely suspected yield protectors among the T14. See note [1] below. Some T14 do appear to use yield protection as I've defined it.
As shown above, acceptance rates are across-the-board monotonic at these T14 law schools. The conventional wisdom (around TLS, anyway) is that UVA and Penn are more liberal with their waitlists (and stingy with their admittances), on the basis of numbers, than other T14 schools. This fact is borne out by the data. But while UVA and Penn waitlist a lot of qualified applicants, they’re no more likely (less likely, in fact) to waitlist their more-qualified applicants over less-qualified ones. These data are entirely consistent with each school’s putting stronger emphasis on so-called soft factors that don’t necessarily correlate with candidates’ hard stats, such as in-state residency at UVA or work experience at Penn. Come to think of it, this is exactly what these schools avow—a holistic approach to admittances. Nothing to get excited about here.
Conclusion: Yield protection at T14 law schools is a myth. See note [1] below. Some T14 do appear to use yield protection as I've defined it.
For good measure, let’s dip a bit lower into the USNews rankings and check out George Washington University (ranked 20th).
It looks like a small degree of yield protection is possible at GWU (oddly though, it seems to be in the form of dings, not waitlists). But as an armchair statistician, I can’t say for sure whether it’s statistically significant.
Conclusion: Ambiguous.
So what explains yield protection where it’s prevalent?
While it’s apparent that top-ranked law schools don’t engage in yield protection, it’s equally apparent that some lower-ranked schools do. I explained at the outset why I’m skeptical of attributing dubious motives to the practice, but I suppose I owe an alternative narrative if I’m to defend American University for waitlisting approximately 40% of its top-scoring applicants.
Each admissions committee has an interest in building the best possible student body. Due to limited class size, AdComs obviously can't admit every qualified candidate and hope for the best. Furthermore, due to unpredictable matriculation rates, AdComs can’t just calibrate-upwards the minimum admissions criteria such that the expected number of matriculants equals their desired class size. To do so would risk an overabundance of matriculating students, potentially flooding classes with students and overwhelming the school’s scarce resources.
Predicting which admitted students are most likely to matriculate is crucial to building a class. Such predictions are difficult due to information asymmetry--a candidate knows whether she prefers, say, GWU to GULC, but the AdComs of each do not. The solution to this information problem is yield protection. By waitlisting overqualified candidates, the AdComs can use the waitlist period to solicit incremental information about each candidate’s preferences (via letters of continued interest and Why X essays). There’s nothing nefarious about yield protecting in this manner; you might even say it’s downright polite, both to the candidate applying as a safety school and, more importantly, to those applying as a reach.
Next time you see a candidate with impressive numbers waitlisted, I hope you'll hesitate to cry shenanigans for manipulation of metrics. I suspect such protests are more likely evidence of the applicant protecting his own ego than of a school protecting its yield.
Note 1: Thanks to the insightful comments from TLSers in this thread, I've changed my view on the question of yield protection at certain schools. However, I maintain my skepticism toward the rankings-manipulation motivation for the reasons discussed above.
I had two major methodological errors in my analysis that are discussed in the subsequent comments, particularly on page 3. Correcting for these errors (to the extent possible), the two graphics below demonstrate that UVA likely practices yield protection.
Preemptive tl;dr: Yield protection at mostT14 schools (
Earlier today I posted the following to the GULC Waiting thread:
I imagine many TLSers will glance at the top end of the above waitlist and immediately suspect “yield protection.” For those who were waitlisted with big numbers, for whom there is no obvious reason to think their application is otherwise weak, and who may reasonably choose to attend GULC over all other schools to which they’ve so-far been admitted, I certainly sympathize. But I'm skeptical of claims of yield protection--or at least of the motive ascribed to it (i.e. massaging the USNews rankings). Below, I explain why, and then take a look at the empirical evidence of yield protection.Sandrew wrote: Tough day Friday. I counted 21 or 22 applicants reporting waitlisted 2/18 on TLS and LSN, with numbers ranging from expected "auto-admits" to candidates with more marginal numbers.
175/3.76
174/3.9
174/3.38
173/3.90
172/3.92
171/3.55
171/3.4 URM
170/3.40
170/2.94
170/2.60 URM
169/3.89
169/3.88
167/3.80
167/3.75 (possibly double-counted)
167/3.7
167/3.68
166/3.8
165/3.85 URM
165/3.7
163/4.01
163/3.99
158/3.50 URM
Why the skepticism?
First, let’s remember that a law school’s yield (percentage of admitted students matriculating) is not among USNews’ rankings metrics. Acceptance rate is such a metric, and yes, it's possible to conceive of a nefarious strategy of waitlisting or rejecting "overqualified" candidates en mass in order to achieve a boost in this metric. But at what cost?
Waitlisting qualified candidates risks losing some of them to other schools, dragging down the "hard" metrics of median LSAT and GPA—two metrics that have a combined weight in the USNews rankings 9 times that of acceptance rate. I don't have any data on the subject, but I'd hazard a guess that highly qualified candidates are more likely to matriculate if accepted outright than if waitlisted and later accepted. Several factors could contribute to this, ranging from the psychological (impatience or aversion to uncertainty) to economic (the timing of scholarship offers from competing schools, many of which "explode"). If my suspicion is correct, this suggests to me that yield protection isn't a good strategy for juicing a school's ranking.
Empirical Analysis of Yield Protection
Let's table the accusations of rankings manipulation for a moment, and instead focus on the empirical questions of whether and where yield protection exists. Here, I'm defining yield protection as the practice of waitlisting overqualified candidates, irrespective of motive.
If yield protection exists at a school, we would expect to see higher rates of waitlisting among the most qualified candidates relative some subset of less qualified candidates. Let’s start with a prototypical example of an accused yield protector: American University.
On the horizontal axis, I have grouped applicants by LSAT deciles. At AU, the first decile (bucket #1 above) corresponds to a high LSAT score of 167 and above. Vertical axis is the proportion of each bucket that got accepted/waitlisted/denied. See the sidebar below for a word of warning on interpreting the vertical axis. Acceptance rates are in green, waitlist rates in blue, and denial rates in red. I've graphed them additively (i.e. "stacked"), as they always sum to 1 by definition.
In geekspeak, what we see in the above graph is that acceptance rates are a nonmonotonic function. In layman’s terms, this means it’s as humped as a camel’s back. This graph points to strong evidence of yield protection at AU; a large chunk of candidates appear to have been waitlisted on the basis of a too-high LSAT.
Conclusion: American University yield protects.
Side note: Please be cautious of the absolute acceptance rates on the left-hand axis, both above and throughout. LSN data are demonstrably skewed toward higher-achieving candidates (i.e. those more likely to be accepted). Nevertheless, I don’t believe this “LSN skew effect” is fouling up my conclusions, since what matters for purposes of demonstrating yield protection is relative rates of acceptance/waitlisting across index deciles.
Do the Top 14 Law Schools (T14) engage in yield protection?
Let’s start with a school that not even the most jaded TLSer would accuse of yield-protecting highly qualified candidates: Columbia.
As we would expect of a non-yield protecting school, acceptance rates at CLS are (for the most part) monotonically increasing with an applicant’s index. The inverse is true of rejection rates. Lastly, the waitlist rate (blue/purple) is monotonically increasing over the first through fifth deciles, and monotonically decreasing thereafter. Neat and clean. No yield protection here.
Let’s move on to more widely suspected yield protectors among the T14. See note [1] below. Some T14 do appear to use yield protection as I've defined it.
As shown above, acceptance rates are across-the-board monotonic at these T14 law schools. The conventional wisdom (around TLS, anyway) is that UVA and Penn are more liberal with their waitlists (and stingy with their admittances), on the basis of numbers, than other T14 schools. This fact is borne out by the data. But while UVA and Penn waitlist a lot of qualified applicants, they’re no more likely (less likely, in fact) to waitlist their more-qualified applicants over less-qualified ones. These data are entirely consistent with each school’s putting stronger emphasis on so-called soft factors that don’t necessarily correlate with candidates’ hard stats, such as in-state residency at UVA or work experience at Penn. Come to think of it, this is exactly what these schools avow—a holistic approach to admittances. Nothing to get excited about here.
Conclusion: Yield protection at T14 law schools is a myth. See note [1] below. Some T14 do appear to use yield protection as I've defined it.
For good measure, let’s dip a bit lower into the USNews rankings and check out George Washington University (ranked 20th).
It looks like a small degree of yield protection is possible at GWU (oddly though, it seems to be in the form of dings, not waitlists). But as an armchair statistician, I can’t say for sure whether it’s statistically significant.
Conclusion: Ambiguous.
So what explains yield protection where it’s prevalent?
While it’s apparent that top-ranked law schools don’t engage in yield protection, it’s equally apparent that some lower-ranked schools do. I explained at the outset why I’m skeptical of attributing dubious motives to the practice, but I suppose I owe an alternative narrative if I’m to defend American University for waitlisting approximately 40% of its top-scoring applicants.
Each admissions committee has an interest in building the best possible student body. Due to limited class size, AdComs obviously can't admit every qualified candidate and hope for the best. Furthermore, due to unpredictable matriculation rates, AdComs can’t just calibrate-upwards the minimum admissions criteria such that the expected number of matriculants equals their desired class size. To do so would risk an overabundance of matriculating students, potentially flooding classes with students and overwhelming the school’s scarce resources.
Predicting which admitted students are most likely to matriculate is crucial to building a class. Such predictions are difficult due to information asymmetry--a candidate knows whether she prefers, say, GWU to GULC, but the AdComs of each do not. The solution to this information problem is yield protection. By waitlisting overqualified candidates, the AdComs can use the waitlist period to solicit incremental information about each candidate’s preferences (via letters of continued interest and Why X essays). There’s nothing nefarious about yield protecting in this manner; you might even say it’s downright polite, both to the candidate applying as a safety school and, more importantly, to those applying as a reach.
Next time you see a candidate with impressive numbers waitlisted, I hope you'll hesitate to cry shenanigans for manipulation of metrics. I suspect such protests are more likely evidence of the applicant protecting his own ego than of a school protecting its yield.
Note 1: Thanks to the insightful comments from TLSers in this thread, I've changed my view on the question of yield protection at certain schools. However, I maintain my skepticism toward the rankings-manipulation motivation for the reasons discussed above.
I had two major methodological errors in my analysis that are discussed in the subsequent comments, particularly on page 3. Correcting for these errors (to the extent possible), the two graphics below demonstrate that UVA likely practices yield protection.
