A new (?) idea for a ranking Forum

[saxamaflob]

I accepted an offer from a law school last year and deferred to work on a startup business. In the meantime, I found numbers that prompted me to design my own ranking: given that prospective students select law schools based in part on what kind of job they're already eyeing before they start their 1L year, I think we should rank a school according to the percentage not of its total student body that receives a particular kind of desirable job (top firm, SCOTUS clerkship, etc) but of the segment of its student body entering a particular kind of work who receive a "most-desirable" job in that line of work. Should we, then, rank schools according to percentage of alum clerks affiliated with Article III judges and of alum firm lawyers practicing at top firms? (That question isn't just rhetorical.) Shouldn't these numbers reflect the mix of all factors that contribute to student success in law school? (Such figures as "citations per faculty member" would, I think, be better suited for rankings of not where to study law but where to teach it.)

To get the conversation started, I found these data for 2009 from ABA re total employment, US News re Article III clerkships, and bcg attorney Search re top firms:

Data in each series are (1) clerkships top/all; (2) spots at firms top/all; (3) arithmetic mean, geometric mean
Schools sorted in descending order of both measures of the mean, which happen to produce identical rankings for these schools
Adjacent schools separated by <1% ranked identically but sorted as above

1. Stanford
22.00%/23.00%=95.65%; 54.10%/61.00%=88.69%; ar: 92.17%, ge: 92.10%

2. Yale
31.40%/35.10%=89.46%; 35.30%/41.40%=85.27%; ar: 87.36%, ge: 87.34%

3-4. Columbia
11.00%/11.00%=100.00%; 54.40%/81.70%=66.59%; ar: 83.29%, ge: 81.60%

3-4. Chicago
13.00%/13.00%=100.00%; 53.10%/81.30%=65.31%; ar: 82.66%, ge: 80.82%

5-6. Virginia, Charlottesville
11.80%/13.60%=86.76%; 52.80%/76.60%=68.93%; ar: 77.85%, ge: 77.33%

5-6. Harvard
15.50%/19.00%=81.58%; 47.60%/65.90%=72.23%; ar: 76.90%, ge: 76.76%

7. Duke
13.00%/16.00%=81.25%; 49.80%/75.40%=66.05%; ar: 73.65%, ge: 73.26%

8. Pennsylvania
12.80%/16.50%=77.58%; 50.80%/76.80%=66.15%; ar: 71.86%, ge: 71.63%

One weakness of this ranking method remains the selection bias of students: for instance, if the intention of more YLS grads is to take any clerkship over any position in any firm, then more of them should end up accepting non–Article-III positions, depressing the numbers for YLS, which is maybe what we see. (Still, Stanford appears strictly preferable to Yale in the economic sense of having greater value in both measures; we cannot say the same about strict preference among Yale, Columbia, and Chicago because Columbia and Chicago have identical placement in Article III clerkships and both lie above Yale in that regard, perhaps in large part because of the selection bias at Columbia and Chicago in favor of jobs at firms over clerkships.)

In fact, the only strict preferences these rankings establish are the following:
(1) Stanford > all
(2) Yale > all except Stanford
(3) Columbia ≥ Chicago and Columbia > Duke and Pennsylvania
(nothing about Chicago, which has the lowest performance at firms of these eight schools)
(4) Virginia > Duke and Pennsylvania
(5) Harvard > Duke and Pennsylvania
(6) Duke almost ≥ Pennsylvania, although strictly, these two do crisscross

These strict pairwise rankings aside, individual preference for clerkship or position at firm would affect which schools would be preferable to which other schools.

I think this ranking does a good job reflecting how well law schools prepare their students for greatest success in whichever of these two paths they choose, but I'd be interested to hear others' thoughts.

Besides clerkships and positions in firms, do any other paths have numbers like these that this ranking should incorporate?

[Bronte]

Frankly, I think you're gonna have to do a better job explaining what you did here. I at least am having a hard time figuring it out. Rankings that don't recognize Harvard as a top three school are usually suspect though.

[duckmoney]

tl;dr:

OP ranked schools based on the percent of prestigious jobs that new grads received. In other words, percent of Article III clerkships as a fraction of all clerkships and percent of biglaw jobs as a fraction of all firm jobs.

OP's findings show the top schools are still the top schools, just in a slightly different order.

These rankings are probably too simplistic to account for a school's entire worth, but definitely a useful addition to the literature. I would like to see these numbers for all of the T1 schools; that might be more telling than just comparing slight variations in the best schools in the nation.

[MrKappus]

Frankly, I think you're gonna have to do a better job explaining what you did here. I at least am having a hard time figuring it out. Rankings that don't recognize Harvard as a top three school are usually suspect though.

You just need to reread the post. OP's methodology is extremely limited and uses only 4 values to calculate rank, so I don't think it's nearly comprehensive enough to form a new basis for ranking schools. "What [OP] did here," however, is pretty obvious.

[NorCalBruin]

I like this.

It is limited in that it only uses a few variables, but I kind of like that. A big criticism of USNWR rankings is that it uses too many variables, giving each a seemingly arbitrary weight. This is kind of cool. Not perfect, but neat.

I mean, at the very least, I would like to see an expanded T1 list. Good work.

[thelaststraw05]

I like this.

It is limited in that it only uses a few variables, but I kind of like that. A big criticism of USNWR rankings is that it uses too many variables, giving each a seemingly arbitrary weight. This is kind of cool. Not perfect, but neat.

I mean, at the very least, I would like to see an expanded T1 list. Good work.

I agree. To some extent a lot of the variables used by USNWR are redundant.

LSAT and GPA are quantitative measures of strength of student body and hiring prestige is a measure of the respect the legal community has for the school. I'm not sure what other factors really need to be taken into account. Many on this board would argue that the most important thing ITE is the hiring prestige (which this ranking demonstrates).

Unlike USNWR, gaming THIS ranking system would be difficult and could only HELP your student body.

[nonprofit-prophet]

I like this.

It is limited in that it only uses a few variables, but I kind of like that. A big criticism of USNWR rankings is that it uses too many variables, giving each a seemingly arbitrary weight. This is kind of cool. Not perfect, but neat.

I mean, at the very least, I would like to see an expanded T1 list. Good work.

I agree. To some extent a lot of the variables used by USNWR are redundant.

LSAT and GPA are quantitative measures of strength of student body and hiring prestige is a measure of the respect the legal community has for the school. I'm not sure what other factors really need to be taken into account. Many on this board would argue that the most important thing ITE is the hiring prestige (which this ranking demonstrates).

Unlike USNWR, gaming THIS ranking system would be difficult and could only HELP your student body.

I agree, it's nice to see a stripped down ranking.

OP, could you run the numbers for the rest of the top 20? I'm curious to see where Texas, GULC, and UCLA land

[rundoxierun]

more recent Harvard numbers:
http://www.law.harvard.edu/current/care ... dents.html

[Sentry]

Any rankings where Chicago is in the top 4 is good by me.

[saxamaflob]

I like this.

It is limited in that it only uses a few variables, but I kind of like that. A big criticism of USNWR rankings is that it uses too many variables, giving each a seemingly arbitrary weight. This is kind of cool. Not perfect, but neat.

I mean, at the very least, I would like to see an expanded T1 list. Good work.

I agree. To some extent a lot of the variables used by USNWR are redundant.

LSAT and GPA are quantitative measures of strength of student body and hiring prestige is a measure of the respect the legal community has for the school. I'm not sure what other factors really need to be taken into account. Many on this board would argue that the most important thing ITE is the hiring prestige (which this ranking demonstrates).

Unlike USNWR, gaming THIS ranking system would be difficult and could only HELP your student body.

I agree, it's nice to see a stripped down ranking.

OP, could you run the numbers for the rest of the top 20? I'm curious to see where Texas, GULC, and UCLA land

I ran the numbers for schools I thought you might mean by "top 20"; here are the rankings normalized against Stanford and with just the geometric mean:

(100) Stanford University Stanford, CA—92.10%
(95) Yale University New Haven, CT—87.34%
(89) Columbia University New York, NY—81.60%
(88) University of Chicago Chicago, IL—80.82%
(84) University of Virginia Charlottesville, VA—77.33%
(83) Northwestern University Chicago, IL—76.87%
(83) Harvard University Cambridge, MA—76.76%
(80) Georgetown University Washington, DC—74.08%
(80) University of Southern California—73.91%
(80) New York University New York, NY—73.49%
(80) University of California Berkeley, CA—73.39%
(80) Duke University Durham, NC—73.26%
(78) University of Pennsylvania Philadelphia, PA—71.63%
(74) University of Michigan Ann Arbor, MI—68.18%
(73) Vanderbilt University Nashville, TN—67.16%
(69) Cornell University Ithaca, NY—63.86%
(69) University of Texas Austin, TX—63.52%
(67) University of California Los Angeles, CA—61.68%
(64) Emory University Atlanta, GA—58.64%
(57) University of Illinois—52.62%
(45) George Washington University—41.39%
(39) Boston University Boston, MA—35.54%
(37) University of Minnesota Twin Cities, MN—34.32%

more recent Harvard numbers:
--LinkRemoved-- ... dents.html

The more-recent Harvard numbers don't seem to change anything: by eyeball, clerks top/all is still hovering just below 90%, and the new numbers say nothing about firms top/all.

As for being limited, this ranking is designed only to measure the actual outcome of a student's legal education (obviously). Plenty of intangible factors—quality of life, anyone?—go into making the whole "law school experience", but how do you rank on qualitatives?

I like this.

It is limited in that it only uses a few variables, but I kind of like that. A big criticism of USNWR rankings is that it uses too many variables, giving each a seemingly arbitrary weight. This is kind of cool. Not perfect, but neat.

I mean, at the very least, I would like to see an expanded T1 list. Good work.

I agree. To some extent a lot of the variables used by USNWR are redundant.

LSAT and GPA are quantitative measures of strength of student body and hiring prestige is a measure of the respect the legal community has for the school. I'm not sure what other factors really need to be taken into account. Many on this board would argue that the most important thing ITE is the hiring prestige (which this ranking demonstrates).

Unlike USNWR, gaming THIS ranking system would be difficult and could only HELP your student body.

As for LSAT and GPA, those are just predictors for how well students do in their 1L year. Afterward, what do they matter? And then again, for instance, if Stanford can take students with lower LSATs than Harvard and produce arguably more employment success (by these rankings), shouldn't that fact weigh rather in Stanford's favor than against it?

[Bronte]

Imagine this scenario: 50 students at both school X and school Y want the best clerkships they can get. At school X, 5 students get Article III clerkships and the rest do not get clerkships at all. At school Y, 10 students get Article III clerkships and another 10 get non-AIII clerkships.

Who wins in your ranking system? Who should win?

Also: what about public interest placement?

[UnbelievableForum23]

this is really cool. Look at USC...better finish the Rothman

[The Stig]

This is a really interesting way of ranking the schools. I think it might be interesting to note the class size in relation to the rankings (I put them in bold next to the school)

I ran the numbers for schools I thought you might mean by "top 20"; here are the rankings normalized against Stanford and with just the geometric mean:

170(100) Stanford University Stanford, CA—92.10%
190(95) Yale University New Haven, CT—87.34%
384(89) Columbia University New York, NY—81.60%
~190(88) University of Chicago Chicago, IL—80.82%
370(84) University of Virginia Charlottesville, VA—77.33%
270(83) Northwestern University Chicago, IL—76.87%
561(83) Harvard University Cambridge, MA—76.76%
463(80) Georgetown University Washington, DC—74.08%
220(80) University of Southern California—73.91%
476(80) New York University New York, NY—73.49%
270(80) University of California Berkeley, CA—73.39%
228(80) Duke University Durham, NC—73.26%
250(78) University of Pennsylvania Philadelphia, PA—71.63%
361(74) University of Michigan Ann Arbor, MI—68.18%
(73) Vanderbilt University Nashville, TN—67.16%
(69) Cornell University Ithaca, NY—63.86%
(69) University of Texas Austin, TX—63.52%
(67) University of California Los Angeles, CA—61.68%
(64) Emory University Atlanta, GA—58.64%
(57) University of Illinois—52.62%
(45) George Washington University—41.39%
(39) Boston University Boston, MA—35.54%
(37) University of Minnesota Twin Cities, MN—34.32%

just from looking at that, it looks like there might be somewhat of an advantage to being a smaller school (e.g. YS placing > H; Chi > NYU, though CLS is an exception in the traditional CCN bracket)

ETA: it is interesting to note that Georgetown's (large) jumped up, while H/NYU dropped (also large)

[saxamaflob]

Imagine this scenario: 50 students at both school X and school Y want the best clerkships they can get. At school X, 5 students get Article III clerkships and the rest do not get clerkships at all. At school Y, 10 students get Article III clerkships and another 10 get non-AIII clerkships.

Who wins in your ranking system? Who should win?

First, here's one assumption that my ranking makes: that a school who prepares its students well should, on average, prepare them well for both clerkships and positions at firms. In other words, I'm assuming (perhaps without justification) that a candidate who's competitive for an Article III clerkship would likely be competitive for a job at a top firm.

Given that assumption, I'm without enough information to answer your question. Here's why: what do the 45 "failures" from school X do after not getting any clerkship? Do they go to a firm? In that case, if my autocorrelation assumption holds, then those "clerkship failures" would presumably not fare very well in the market for positions at top firms, which the rankings would reveal. I chose the geometric mean rather than the arithmetic mean for precisely this reason: school X would be "hurt" more for its poor performance in firms than it would be "helped" for its ostensibly good performance in clerkships. The geometric mean places greater emphasis on the lower number. To rise to the top of my ranking, a school has to do well in both areas—it can't easily make up for a failure in clerkships by success at firms or vice versa.

Of course, the "clerkship failures" could go into public service, which I don't include in the rankings. This problem brings me to your next question:

Also: what about public interest placement?

I wish there were a good way to rank public interest placement, but I think we have no numbers other than the percentage of the total class going into public interest, which doesn't say anything.

I think all the schools I surveyed send more than 2/3 of their students to clerkships and firms, so measuring these two variables gives a relatively good picture of the school's overall performance, especially if the autocorrelation assumption carries over from clerkships and firm positions to jobs in public interest and business.

[pride09]

Interesting stuff. Good work.

[abl]

Sax--

The problem with this methodology is that clerkships are generally both more desirable and more competitive than firm jobs (even for the tippy top firm jobs). Thus, most Y/S students who come into law school wanting appellate level clerkships and aren't competitive when it comes time to applying don't clerk with state judges--they generally instead go and work at firms. This is going to be most applicable at Y and S, where virtually any student who wants it can get a top firm job. However, it'll also apply on the margins at all other schools--more students doing clerkships attests to not only more students wanting to do clerkships at that school, but more students competitive for clerkships.

Additionally, especially at Y/S, most of the students who do law firm jobs that aren't "top" jobs are doing so by choice--many of those other firms are boutiques that are more desirable than "top" firm jobs, but are too small to show up on rankings of "top" firms, or regional firms that students choose to go to for personal/family reasons. Probably once you drop much below HYS, this stops being the case, but at YS and probably H, a student whose lone goal is a V100 firm will have little difficulty getting there.

This is not to say that what you've done isn't interesting or valuable. I just wanted to point out one (rather significant) flaw in your system. Is there some way that you could (heavily) take into account overall percentage of clerks? Yale's 31% federal appellate clerks is more impressive than Stanford's 96% "elite" clerkship ratio. I suspect the end result would be very similar, and if it resulted in YS being tied or flipping, so be it--nobody should decide between the two schools on anything other than fit. It might give you a somewhat better indication of other schools' relative weight than this, however.

[showNprove]

The problem with this ranking is illuminated by its treatment of Yale: despite sending 50% more students to Article III clerkships than Stanford, Stanford gets more points. Yale sends 200% more than Columbia does, yet Columbia gets more points in the ranking formula.

If one schools sent 5 students to Article III clerkships and 5 to the Vault 100, and the other 250 students were unemployed, it would be ranked higher than Yale.

[saxamaflob]

The problem with this ranking is illuminated by its treatment of Yale: despite sending 50% more students to Article III clerkships than Stanford, Stanford gets more points. Yale sends 200% more than Columbia does, yet Columbia gets more points in the ranking formula.

This comment begs the question outright by assuming that the ranking mistreats Yale. The point of the ranking is precisely to compensate for the phenomenon that at different schools, the gross body-count heading into clerkships and firms varies. What this ranking does is to point out that, for instance, both a lower percentage of Yalies going to clerkships get Article III positions and a lower percentage of Yalies going to firms land in the top 250 of those. Now, as for Columbia, it ranks below Yale because the use of the (geometric) mean penalizes that school's lopsided performance.

If one schools sent 5 students to Article III clerkships and 5 to the Vault 100, and the other 250 students were unemployed, it would be ranked higher than Yale.

The counterfactual is ineffective as a critique because it's, well, counterfactual. Better would be to cite a specific reason for which one school or another has gotten gipped. Or to be clear about the specific flaw(s) in the methodology.

I don't pretend this ranking is perfect (as if any ranking were perfect!), but I'm not sure the comment infra has added anything to the discussion.

[bjsesq]

Thank you for the effort. I still have to say this: as a Northwestern student, I had a good laugh at being ranked above Harvard.

[fatduck]

How did you distinguish top firms from non-top firms? NLJ250? Vault? Some other criteria?

[PomasThynchon]

For the life of me I will never understand the factoring of yield rate into the rankings. It just seems like a self-perpetuating thing. People go to Yale because it's Yale, more people give up less prestigious schools because they apply to a dozen or more places and give them up for higher-ranked schools. It just doesn't seem like a solid enough metric, in that it doesn't measure anything beyond student-perceived prestige or desirability. I care about employment/clerkship placement, LRAP and little else

[Magnificent]

but at YS and probably H, a student whose lone goal is a V100 firm will have little difficulty getting there.

is there any data to back this up?

I hear this all the time on TLS yet there is never any tangible proof.

Are people here just repeating what they've heard other people say and are now taking it as the god given truth?

[PomasThynchon]

but at YS and probably H, a student whose lone goal is a V100 firm will have little difficulty getting there.

is there any data to back this up?

I hear this all the time on TLS yet there is never any tangible proof.

Are people here just repeating what they've heard other people say and are now taking it as the god given truth?

I will also never understand the TLSers who take Harvard over Yale and Stanford. For god's sake man, go to a school without letter grades!

[Bildungsroman]

Are people here just repeating what they've heard other people say and are now taking it as the god given truth?

Welcome to TLS.

[fatduck]

For the life of me I will never understand the factoring of yield rate into the rankings. It just seems like a self-perpetuating thing. People go to Yale because it's Yale, more people give up less prestigious schools because they apply to a dozen or more places and give them up for higher-ranked schools. It just doesn't seem like a solid enough metric, in that it doesn't measure anything beyond student-perceived prestige or desirability. I care about employment/clerkship placement, LRAP and little else

Well, USNews doesn't actually use yield rate per se (percentage of offers accepted). It does use selectivity (percentage of applicants admitted), but it isn't weighted very strongly.