A new (?) idea for a ranking

(Rankings, Profiles, Tuition, Student Life, . . . )
saxamaflob
Posts: 26
Joined: Tue Dec 22, 2009 10:54 pm

A new (?) idea for a ranking

Postby saxamaflob » Mon Jan 10, 2011 12:23 pm

I accepted an offer from a law school last year and deferred to work on a startup business. In the meantime, I found numbers that prompted me to design my own ranking: given that prospective students select law schools based in part on what kind of job they're already eyeing before they start their 1L year, I think we should rank a school according to the percentage not of its total student body that receives a particular kind of desirable job (top firm, SCOTUS clerkship, etc) but of the segment of its student body entering a particular kind of work who receive a "most-desirable" job in that line of work. Should we, then, rank schools according to percentage of alum clerks affiliated with Article III judges and of alum firm lawyers practicing at top firms? (That question isn't just rhetorical.) Shouldn't these numbers reflect the mix of all factors that contribute to student success in law school? (Such figures as "citations per faculty member" would, I think, be better suited for rankings of not where to study law but where to teach it.)

To get the conversation started, I found these data for 2009 from ABA re total employment, US News re Article III clerkships, and BCG Attorney Search re top firms:

Data in each series are (1) clerkships top/all; (2) spots at firms top/all; (3) arithmetic mean, geometric mean
Schools sorted in descending order of both measures of the mean, which happen to produce identical rankings for these schools
Adjacent schools separated by <1% ranked identically but sorted as above

1. Stanford
22.00%/23.00%=95.65%; 54.10%/61.00%=88.69%; ar: 92.17%, ge: 92.10%

2. Yale
31.40%/35.10%=89.46%; 35.30%/41.40%=85.27%; ar: 87.36%, ge: 87.34%

3-4. Columbia
11.00%/11.00%=100.00%; 54.40%/81.70%=66.59%; ar: 83.29%, ge: 81.60%

3-4. Chicago
13.00%/13.00%=100.00%; 53.10%/81.30%=65.31%; ar: 82.66%, ge: 80.82%

5-6. Virginia, Charlottesville
11.80%/13.60%=86.76%; 52.80%/76.60%=68.93%; ar: 77.85%, ge: 77.33%

5-6. Harvard
15.50%/19.00%=81.58%; 47.60%/65.90%=72.23%; ar: 76.90%, ge: 76.76%

7. Duke
13.00%/16.00%=81.25%; 49.80%/75.40%=66.05%; ar: 73.65%, ge: 73.26%

8. Pennsylvania
12.80%/16.50%=77.58%; 50.80%/76.80%=66.15%; ar: 71.86%, ge: 71.63%

One weakness of this ranking method remains the selection bias of students: for instance, if the intention of more YLS grads is to take any clerkship over any position in any firm, then more of them should end up accepting non–Article-III positions, depressing the numbers for YLS, which is maybe what we see. (Still, Stanford appears strictly preferable to Yale in the economic sense of having greater value in both measures; we cannot say the same about strict preference among Yale, Columbia, and Chicago because Columbia and Chicago have identical placement in Article III clerkships and both lie above Yale in that regard, perhaps in large part because of the selection bias at Columbia and Chicago in favor of jobs at firms over clerkships.)

In fact, the only strict preferences these rankings establish are the following:
(1) Stanford > all
(2) Yale > all except Stanford
(3) Columbia ≥ Chicago and Columbia > Duke and Pennsylvania
(nothing about Chicago, which has the lowest performance at firms of these eight schools)
(4) Virginia > Duke and Pennsylvania
(5) Harvard > Duke and Pennsylvania
(6) Duke almost ≥ Pennsylvania, although strictly, these two do crisscross

These strict pairwise rankings aside, individual preference for clerkship or position at firm would affect which schools would be preferable to which other schools.

I think this ranking does a good job reflecting how well law schools prepare their students for greatest success in whichever of these two paths they choose, but I'd be interested to hear others' thoughts.

Besides clerkships and positions in firms, do any other paths have numbers like these that this ranking should incorporate?

User avatar
Bronte
Posts: 2128
Joined: Sun Jan 04, 2009 10:44 pm

Re: A new (?) idea for a ranking

Postby Bronte » Mon Jan 10, 2011 12:35 pm

Frankly, I think you're gonna have to do a better job explaining what you did here. I at least am having a hard time figuring it out. Rankings that don't recognize Harvard as a top three school are usually suspect though.

duckmoney
Posts: 886
Joined: Sun Jun 13, 2010 2:21 pm

Re: A new (?) idea for a ranking

Postby duckmoney » Mon Jan 10, 2011 12:42 pm

tl;dr:

OP ranked schools based on the percent of prestigious jobs that new grads received. In other words, percent of Article III clerkships as a fraction of all clerkships and percent of biglaw jobs as a fraction of all firm jobs.

OP's findings show the top schools are still the top schools, just in a slightly different order.

These rankings are probably too simplistic to account for a school's entire worth, but definitely a useful addition to the literature. I would like to see these numbers for all of the T1 schools; that might be more telling than just comparing slight variations in the best schools in the nation.

User avatar
MrKappus
Posts: 1685
Joined: Tue Mar 31, 2009 2:46 am

Re: A new (?) idea for a ranking

Postby MrKappus » Mon Jan 10, 2011 12:43 pm

Bronte wrote:Frankly, I think you're gonna have to do a better job explaining what you did here. I at least am having a hard time figuring it out. Rankings that don't recognize Harvard as a top three school are usually suspect though.


You just need to reread the post. OP's methodology is extremely limited and uses only 4 values to calculate rank, so I don't think it's nearly comprehensive enough to form a new basis for ranking schools. "What [OP] did here," however, is pretty obvious.

User avatar
NorCalBruin
Posts: 591
Joined: Sun Sep 19, 2010 7:58 pm

Re: A new (?) idea for a ranking

Postby NorCalBruin » Mon Jan 10, 2011 1:01 pm

I like this.

It is limited in that it only uses a few variables, but I kind of like that. A big criticism of USNWR rankings is that it uses too many variables, giving each a seemingly arbitrary weight. This is kind of cool. Not perfect, but neat.

I mean, at the very least, I would like to see an expanded T1 list. Good work.

User avatar
thelaststraw05
Posts: 1028
Joined: Fri Nov 05, 2010 7:47 am

Re: A new (?) idea for a ranking

Postby thelaststraw05 » Mon Jan 10, 2011 1:08 pm

NorCalBruin wrote:I like this.

It is limited in that it only uses a few variables, but I kind of like that. A big criticism of USNWR rankings is that it uses too many variables, giving each a seemingly arbitrary weight. This is kind of cool. Not perfect, but neat.

I mean, at the very least, I would like to see an expanded T1 list. Good work.


I agree. To some extent a lot of the variables used by USNWR are redundant.

LSAT and GPA are quantitative measures of strength of student body and hiring prestige is a measure of the respect the legal community has for the school. I'm not sure what other factors really need to be taken into account. Many on this board would argue that the most important thing ITE is the hiring prestige (which this ranking demonstrates).

Unlike USNWR, gaming THIS ranking system would be difficult and could only HELP your student body.

nonprofit-prophet
Posts: 844
Joined: Sun Oct 04, 2009 11:10 am

Re: A new (?) idea for a ranking

Postby nonprofit-prophet » Mon Jan 10, 2011 1:15 pm

thelaststraw05 wrote:
NorCalBruin wrote:I like this.

It is limited in that it only uses a few variables, but I kind of like that. A big criticism of USNWR rankings is that it uses too many variables, giving each a seemingly arbitrary weight. This is kind of cool. Not perfect, but neat.

I mean, at the very least, I would like to see an expanded T1 list. Good work.


I agree. To some extent a lot of the variables used by USNWR are redundant.

LSAT and GPA are quantitative measures of strength of student body and hiring prestige is a measure of the respect the legal community has for the school. I'm not sure what other factors really need to be taken into account. Many on this board would argue that the most important thing ITE is the hiring prestige (which this ranking demonstrates).

Unlike USNWR, gaming THIS ranking system would be difficult and could only HELP your student body.


I agree, it's nice to see a stripped down ranking.

OP, could you run the numbers for the rest of the top 20? I'm curious to see where Texas, GULC, and UCLA land

rundoxierun
Posts: 1893
Joined: Wed Dec 03, 2008 1:46 am

Re: A new (?) idea for a ranking

Postby rundoxierun » Mon Jan 10, 2011 1:16 pm


User avatar
Sentry
Posts: 1235
Joined: Tue Mar 02, 2010 6:38 pm

Re: A new (?) idea for a ranking

Postby Sentry » Mon Jan 10, 2011 1:20 pm

Any rankings where Chicago is in the top 4 is good by me.

saxamaflob
Posts: 26
Joined: Tue Dec 22, 2009 10:54 pm

Re: A new (?) idea for a ranking

Postby saxamaflob » Tue Jan 11, 2011 3:22 pm

nonprofit-prophet wrote:
thelaststraw05 wrote:
NorCalBruin wrote:I like this.

It is limited in that it only uses a few variables, but I kind of like that. A big criticism of USNWR rankings is that it uses too many variables, giving each a seemingly arbitrary weight. This is kind of cool. Not perfect, but neat.

I mean, at the very least, I would like to see an expanded T1 list. Good work.


I agree. To some extent a lot of the variables used by USNWR are redundant.

LSAT and GPA are quantitative measures of strength of student body and hiring prestige is a measure of the respect the legal community has for the school. I'm not sure what other factors really need to be taken into account. Many on this board would argue that the most important thing ITE is the hiring prestige (which this ranking demonstrates).

Unlike USNWR, gaming THIS ranking system would be difficult and could only HELP your student body.


I agree, it's nice to see a stripped down ranking.

OP, could you run the numbers for the rest of the top 20? I'm curious to see where Texas, GULC, and UCLA land


I ran the numbers for schools I thought you might mean by "top 20"; here are the rankings normalized against Stanford and with just the geometric mean:

(100) Stanford University Stanford, CA—92.10%
(95) Yale University New Haven, CT—87.34%
(89) Columbia University New York, NY—81.60%
(88) University of Chicago Chicago, IL—80.82%
(84) University of Virginia Charlottesville, VA—77.33%
(83) Northwestern University Chicago, IL—76.87%
(83) Harvard University Cambridge, MA—76.76%
(80) Georgetown University Washington, DC—74.08%
(80) University of Southern California—73.91%
(80) New York University New York, NY—73.49%
(80) University of California Berkeley, CA—73.39%
(80) Duke University Durham, NC—73.26%
(78) University of Pennsylvania Philadelphia, PA—71.63%
(74) University of Michigan Ann Arbor, MI—68.18%
(73) Vanderbilt University Nashville, TN—67.16%
(69) Cornell University Ithaca, NY—63.86%
(69) University of Texas Austin, TX—63.52%
(67) University of California Los Angeles, CA—61.68%
(64) Emory University Atlanta, GA—58.64%
(57) University of Illinois—52.62%
(45) George Washington University—41.39%
(39) Boston University Boston, MA—35.54%
(37) University of Minnesota Twin Cities, MN—34.32%

tkgrrett wrote:more recent Harvard numbers:
--LinkRemoved-- ... dents.html


The more-recent Harvard numbers don't seem to change anything: by eyeball, clerks top/all is still hovering just below 90%, and the new numbers say nothing about firms top/all.

As for being limited, this ranking is designed only to measure the actual outcome of a student's legal education (obviously). Plenty of intangible factors—quality of life, anyone?—go into making the whole "law school experience", but how do you rank on qualitatives?

thelaststraw05 wrote:
NorCalBruin wrote:I like this.

It is limited in that it only uses a few variables, but I kind of like that. A big criticism of USNWR rankings is that it uses too many variables, giving each a seemingly arbitrary weight. This is kind of cool. Not perfect, but neat.

I mean, at the very least, I would like to see an expanded T1 list. Good work.


I agree. To some extent a lot of the variables used by USNWR are redundant.

LSAT and GPA are quantitative measures of strength of student body and hiring prestige is a measure of the respect the legal community has for the school. I'm not sure what other factors really need to be taken into account. Many on this board would argue that the most important thing ITE is the hiring prestige (which this ranking demonstrates).

Unlike USNWR, gaming THIS ranking system would be difficult and could only HELP your student body.


As for LSAT and GPA, those are just predictors for how well students do in their 1L year. Afterward, what do they matter? And then again, for instance, if Stanford can take students with lower LSATs than Harvard and produce arguably more employment success (by these rankings), shouldn't that fact weigh rather in Stanford's favor than against it?

User avatar
Bronte
Posts: 2128
Joined: Sun Jan 04, 2009 10:44 pm

Re: A new (?) idea for a ranking

Postby Bronte » Tue Jan 11, 2011 3:36 pm

Imagine this scenario: 50 students at both school X and school Y want the best clerkships they can get. At school X, 5 students get Article III clerkships and the rest do not get clerkships at all. At school Y, 10 students get Article III clerkships and another 10 get non-AIII clerkships.

Who wins in your ranking system? Who should win?

Also: what about public interest placement?

UnbelievableForum23
Posts: 31
Joined: Sat Oct 30, 2010 4:57 pm

Re: A new (?) idea for a ranking

Postby UnbelievableForum23 » Tue Jan 11, 2011 3:47 pm

this is really cool. Look at USC...better finish the Rothman

User avatar
The Stig
Posts: 567
Joined: Sat Oct 30, 2010 12:27 pm

Re: A new (?) idea for a ranking

Postby The Stig » Tue Jan 11, 2011 4:02 pm

This is a really interesting way of ranking the schools. I think it might be interesting to note the class size in relation to the rankings (I put them in bold next to the school)

saxamaflob wrote:
I ran the numbers for schools I thought you might mean by "top 20"; here are the rankings normalized against Stanford and with just the geometric mean:

170(100) Stanford University Stanford, CA—92.10%
190(95) Yale University New Haven, CT—87.34%
384(89) Columbia University New York, NY—81.60%
~190(88) University of Chicago Chicago, IL—80.82%
370(84) University of Virginia Charlottesville, VA—77.33%
270(83) Northwestern University Chicago, IL—76.87%
561(83) Harvard University Cambridge, MA—76.76%
463(80) Georgetown University Washington, DC—74.08%
220(80) University of Southern California—73.91%
476(80) New York University New York, NY—73.49%
270(80) University of California Berkeley, CA—73.39%
228(80) Duke University Durham, NC—73.26%
250(78) University of Pennsylvania Philadelphia, PA—71.63%
361(74) University of Michigan Ann Arbor, MI—68.18%
(73) Vanderbilt University Nashville, TN—67.16%
(69) Cornell University Ithaca, NY—63.86%
(69) University of Texas Austin, TX—63.52%
(67) University of California Los Angeles, CA—61.68%
(64) Emory University Atlanta, GA—58.64%
(57) University of Illinois—52.62%
(45) George Washington University—41.39%
(39) Boston University Boston, MA—35.54%
(37) University of Minnesota Twin Cities, MN—34.32%


just from looking at that, it looks like there might be somewhat of an advantage to being a smaller school (e.g. YS placing > H; Chi > NYU, though CLS is an exception in the traditional CCN bracket)

ETA: it is interesting to note that Georgetown's (large) jumped up, while H/NYU dropped (also large)

saxamaflob
Posts: 26
Joined: Tue Dec 22, 2009 10:54 pm

Re: A new (?) idea for a ranking

Postby saxamaflob » Tue Jan 11, 2011 6:20 pm

Bronte wrote:Imagine this scenario: 50 students at both school X and school Y want the best clerkships they can get. At school X, 5 students get Article III clerkships and the rest do not get clerkships at all. At school Y, 10 students get Article III clerkships and another 10 get non-AIII clerkships.

Who wins in your ranking system? Who should win?


First, here's one assumption that my ranking makes: that a school who prepares its students well should, on average, prepare them well for both clerkships and positions at firms. In other words, I'm assuming (perhaps without justification) that a candidate who's competitive for an Article III clerkship would likely be competitive for a job at a top firm.

Given that assumption, I'm without enough information to answer your question. Here's why: what do the 45 "failures" from school X do after not getting any clerkship? Do they go to a firm? In that case, if my autocorrelation assumption holds, then those "clerkship failures" would presumably not fare very well in the market for positions at top firms, which the rankings would reveal. I chose the geometric mean rather than the arithmetic mean for precisely this reason: school X would be "hurt" more for its poor performance in firms than it would be "helped" for its ostensibly good performance in clerkships. The geometric mean places greater emphasis on the lower number. To rise to the top of my ranking, a school has to do well in both areas—it can't easily make up for a failure in clerkships by success at firms or vice versa.

Of course, the "clerkship failures" could go into public service, which I don't include in the rankings. This problem brings me to your next question:

Bronte wrote:Also: what about public interest placement?


I wish there were a good way to rank public interest placement, but I think we have no numbers other than the percentage of the total class going into public interest, which doesn't say anything.

I think all the schools I surveyed send more than 2/3 of their students to clerkships and firms, so measuring these two variables gives a relatively good picture of the school's overall performance, especially if the autocorrelation assumption carries over from clerkships and firm positions to jobs in public interest and business.

pride09
Posts: 87
Joined: Wed Jan 05, 2011 9:50 pm

Re: A new (?) idea for a ranking

Postby pride09 » Tue Jan 11, 2011 6:26 pm

Interesting stuff. Good work.

abl
Posts: 734
Joined: Mon Jan 19, 2009 8:07 pm

Re: A new (?) idea for a ranking

Postby abl » Mon Feb 28, 2011 9:39 pm

Sax--

The problem with this methodology is that clerkships are generally both more desirable and more competitive than firm jobs (even for the tippy top firm jobs). Thus, most Y/S students who come into law school wanting appellate level clerkships and aren't competitive when it comes time to applying don't clerk with state judges--they generally instead go and work at firms. This is going to be most applicable at Y and S, where virtually any student who wants it can get a top firm job. However, it'll also apply on the margins at all other schools--more students doing clerkships attests to not only more students wanting to do clerkships at that school, but more students competitive for clerkships.

Additionally, especially at Y/S, most of the students who do law firm jobs that aren't "top" jobs are doing so by choice--many of those other firms are boutiques that are more desirable than "top" firm jobs, but are too small to show up on rankings of "top" firms, or regional firms that students choose to go to for personal/family reasons. Probably once you drop much below HYS, this stops being the case, but at YS and probably H, a student whose lone goal is a V100 firm will have little difficulty getting there.

This is not to say that what you've done isn't interesting or valuable. I just wanted to point out one (rather significant) flaw in your system. Is there some way that you could (heavily) take into account overall percentage of clerks? Yale's 31% federal appellate clerks is more impressive than Stanford's 96% "elite" clerkship ratio. I suspect the end result would be very similar, and if it resulted in YS being tied or flipping, so be it--nobody should decide between the two schools on anything other than fit. It might give you a somewhat better indication of other schools' relative weight than this, however.

showNprove
Posts: 968
Joined: Sun Jun 08, 2008 2:52 pm

Re: A new (?) idea for a ranking

Postby showNprove » Mon Feb 28, 2011 9:52 pm

The problem with this ranking is illuminated by its treatment of Yale: despite sending 50% more students to Article III clerkships than Stanford, Stanford gets more points. Yale sends 200% more than Columbia does, yet Columbia gets more points in the ranking formula.

If one schools sent 5 students to Article III clerkships and 5 to the Vault 100, and the other 250 students were unemployed, it would be ranked higher than Yale.

saxamaflob
Posts: 26
Joined: Tue Dec 22, 2009 10:54 pm

Re: A new (?) idea for a ranking

Postby saxamaflob » Mon Feb 28, 2011 10:14 pm

showNprove wrote:The problem with this ranking is illuminated by its treatment of Yale: despite sending 50% more students to Article III clerkships than Stanford, Stanford gets more points. Yale sends 200% more than Columbia does, yet Columbia gets more points in the ranking formula.


This comment begs the question outright by assuming that the ranking mistreats Yale. The point of the ranking is precisely to compensate for the phenomenon that at different schools, the gross body-count heading into clerkships and firms varies. What this ranking does is to point out that, for instance, both a lower percentage of Yalies going to clerkships get Article III positions and a lower percentage of Yalies going to firms land in the top 250 of those. Now, as for Columbia, it ranks below Yale because the use of the (geometric) mean penalizes that school's lopsided performance.

showNprove wrote:If one schools sent 5 students to Article III clerkships and 5 to the Vault 100, and the other 250 students were unemployed, it would be ranked higher than Yale.


The counterfactual is ineffective as a critique because it's, well, counterfactual. Better would be to cite a specific reason for which one school or another has gotten gipped. Or to be clear about the specific flaw(s) in the methodology.

I don't pretend this ranking is perfect (as if any ranking were perfect!), but I'm not sure the comment infra has added anything to the discussion.

User avatar
bjsesq
TLS Poet Laureate
Posts: 13383
Joined: Fri Nov 19, 2010 3:02 am

Re: A new (?) idea for a ranking

Postby bjsesq » Mon Feb 28, 2011 10:19 pm

Thank you for the effort. I still have to say this: as a Northwestern student, I had a good laugh at being ranked above Harvard.

User avatar
fatduck
Posts: 4186
Joined: Mon Sep 13, 2010 10:16 pm

Re: A new (?) idea for a ranking

Postby fatduck » Mon Feb 28, 2011 10:24 pm

How did you distinguish top firms from non-top firms? NLJ250? Vault? Some other criteria?

User avatar
PomasThynchon
Posts: 326
Joined: Thu Jan 06, 2011 1:34 am

Re: A new (?) idea for a ranking

Postby PomasThynchon » Mon Feb 28, 2011 10:29 pm

For the life of me I will never understand the factoring of yield rate into the rankings. It just seems like a self-perpetuating thing. People go to Yale because it's Yale, more people give up less prestigious schools because they apply to a dozen or more places and give them up for higher-ranked schools. It just doesn't seem like a solid enough metric, in that it doesn't measure anything beyond student-perceived prestige or desirability. I care about employment/clerkship placement, LRAP and little else

Magnificent
Posts: 144
Joined: Sun Feb 27, 2011 1:27 pm

Re: A new (?) idea for a ranking

Postby Magnificent » Mon Feb 28, 2011 10:34 pm

abl wrote: but at YS and probably H, a student whose lone goal is a V100 firm will have little difficulty getting there.



is there any data to back this up?

I hear this all the time on TLS yet there is never any tangible proof.

Are people here just repeating what they've heard other people say and are now taking it as the god given truth?

User avatar
PomasThynchon
Posts: 326
Joined: Thu Jan 06, 2011 1:34 am

Re: A new (?) idea for a ranking

Postby PomasThynchon » Mon Feb 28, 2011 10:36 pm

Magnificent wrote:
abl wrote: but at YS and probably H, a student whose lone goal is a V100 firm will have little difficulty getting there.



is there any data to back this up?

I hear this all the time on TLS yet there is never any tangible proof.

Are people here just repeating what they've heard other people say and are now taking it as the god given truth?


I will also never understand the TLSers who take Harvard over Yale and Stanford. For god's sake man, go to a school without letter grades!

User avatar
Bildungsroman
Posts: 5548
Joined: Sun Apr 11, 2010 2:42 pm

Re: A new (?) idea for a ranking

Postby Bildungsroman » Mon Feb 28, 2011 10:36 pm

Magnificent wrote:Are people here just repeating what they've heard other people say and are now taking it as the god given truth?


Welcome to TLS.

User avatar
fatduck
Posts: 4186
Joined: Mon Sep 13, 2010 10:16 pm

Re: A new (?) idea for a ranking

Postby fatduck » Mon Feb 28, 2011 10:37 pm

PomasThynchon wrote:For the life of me I will never understand the factoring of yield rate into the rankings. It just seems like a self-perpetuating thing. People go to Yale because it's Yale, more people give up less prestigious schools because they apply to a dozen or more places and give them up for higher-ranked schools. It just doesn't seem like a solid enough metric, in that it doesn't measure anything beyond student-perceived prestige or desirability. I care about employment/clerkship placement, LRAP and little else

Well, USNews doesn't actually use yield rate per se (percentage of offers accepted). It does use selectivity (percentage of applicants admitted), but it isn't weighted very strongly.




Return to “Choosing a Law School”

Who is online

Users browsing this forum: jjcorvino, somebodyelse, Yahoo [Bot] and 4 guests