US News Law School Rankings -- 5 Failures Forum

(Applications Advice, Letters of Recommendation . . . )
User avatar
jenesaislaw

Silver
Posts: 1005
Joined: Mon May 19, 2008 6:35 pm

US News Law School Rankings -- 5 Failures

Post by jenesaislaw » Mon Mar 09, 2015 2:16 pm

I just published a piece on Law.com entitled, Five Failures of the U.S. News Rankings.

Here is the body:

Tomorrow, the law school world will overreact to slightly-shuffled U.S. News rankings. Proud alumni and worried students will voice concerns. Provosts will threaten jobs. Prospective students will confuse the annual shuffle with genuine reputational change.

Law school administrators will react predictably. They’ll articulate methodological flaws and lament negative externalities, but will nevertheless commit to the rankings game through their statements and actions. Assuring stakeholders bearing pitchforks has become part of the job description.

If the rankings measured something useful, the entire charade would be much easier to stomach. The unfortunate irony is that these rankings adversely affect the decision-making process for law school administrators and prospective law students alike. The stakes are high. Our profession and society need law schools that don’t figure inefficient metrics into annual budgets. Dollars spent chasing U.S. News rankings diverts funds away from students’ education. It also stands in the way of reducing tuition.

In this post, I examine five U.S. News rankings failures. I consider the methodology and underlying rankings theory from the perspective of a student who features job prospects prominently in his application and enrollment decisions. Considering the near universal support for prioritizing job outcomes in the process, these failures demonstrate just some of the reasons annual consternation hardly seems worth it.

Image
Word cloud of LSAT test-takers surveyed about the reasons they plan to obtain a J.D. degree. Source: Above the Law.


First, the rankings pay insufficient attention to what matters most to prospective students: job outcomes. In a survey of 600 students studying for the October 2012 LSAT, Breaking Media’s research arm found that the two most popular words associated with the students’ purpose of getting a J.D. were “career” and “work” (image above). These are not exactly shocking results. Despite the importance of job outcomes, they account for only 18% of the rank and credit schools for jobs few attend law school to pursue.

Second, the rankings use a national scope, which places schools on the same scale. Only a handful of schools have a truly national reach in job placement. The rest have a regional, in-state, or even just local reach. The relative positioning of California Western and West Virginia in the rankings is virtually meaningless. Graduates from these schools do not compete with one another.

Image
Percentages in above graphic are of the entire class.

It turns out that 158 schools place at least half of their employed class of 2013 graduates in one state. The top state destination for each school accounts for 67% of employed graduates. A much smaller 8.2% of employed graduates go to a school’s second most popular destination, with just 4.5% of employed graduates working in the third most popular destination. Only 20.4% of employed graduates (16.7% of the entire class) end up in a state other than the top three. Comparing schools across the country just doesn’t make sense.

Third, U.S. News rankings follow an ordinal system that fails to show the degree of difference between schools. Are Columbia and NYU virtually tied? Or does the two-rank difference represent a wide gulf in quality? Is the so-called difference between Columbia and NYU the same as the difference between Cornell and Vanderbilt? Students weighing school prices need to know not just which school is better but how much better it is.

Fourth, performance changes over time but year-to-year comparisons are virtually impossible using the U.S. News rankings. U.S. News will tell you that Stanford knocked Harvard out of the #2 spot in 2012-13, but the swap in rankings does not provide any clues to your average reader as to why. Stanford may have improved while Harvard declined. Or, Stanford may have improved while Harvard’s quality stayed the same. Or perhaps both schools saw a decline in quality but Harvard’s decline was more severe. In fact, if every single school saw a marked decline in quality the U.S. News rankings would not indicate that this happened. Instead, students can know only relative performance.

Finally, U.S. News inexplicably places every ABA accredited law school on the list of "The Best." The best at what? U.S. News doesn’t say. But it implies that every school on the list is good. The truth is that once costs and employment outcomes are considered in comparison to personal career goals, many schools are bad choices. The U.S. News rankings provide no help in drawing the line.

Rankings are not inherently bad. In fact, they are conceptually quite useful. They order comparable things to help people sort through more information than they know how to or can weigh. However, ranking credibility may be lost when methodologies are unsound, through irrational weighting or meaningless metrics, or when the scope is too broad. The legal profession is worse off for elevating the importance of a publication that falls victim to these flaws each and every year.

Mal Reynolds

Diamond
Posts: 12612
Joined: Mon Oct 03, 2011 12:16 am

Re: US News Law School Rankings -- 5 Failures

Post by Mal Reynolds » Mon Mar 09, 2015 2:29 pm

Great post.

runinthefront

Gold
Posts: 2151
Joined: Wed Jan 15, 2014 2:18 am

Re: US News Law School Rankings -- 5 Failures

Post by runinthefront » Mon Mar 09, 2015 2:31 pm

I think your article was great, but I do wish there was a bit more detail about the actual "methodology" that US News uses. I was vaguely familiar with the methodology, but after looking at http://www.usnews.com/education/best-gr ... s-rankings , I now feel even more disgusted by how nonsensical their approach to ranking schools is, and how much weight 0Ls place on them.
Last edited by runinthefront on Sat Jan 27, 2018 12:33 am, edited 1 time in total.

User avatar
romothesavior

Diamond
Posts: 14692
Joined: Fri Jun 26, 2009 4:29 pm

Re: US News Law School Rankings -- 5 Failures

Post by romothesavior » Mon Mar 09, 2015 2:48 pm

Stickied and copied to the Law School Admissions Forum.

Great article.

User avatar
Dafaq

Bronze
Posts: 354
Joined: Sat Feb 15, 2014 6:19 pm

Re: US News Law School Rankings -- 5 Failures

Post by Dafaq » Mon Mar 09, 2015 3:02 pm

jenesaislaw wrote:However, ranking credibility may be lost when methodologies are unsound, through irrational weighting or meaningless metrics, or when the scope is too broad. The legal profession is worse off for elevating the importance of a publication that falls victim to these flaws each and every year.
Albeit, UT is a perennial #15 on USNWR and also #15 on ATL.

Want to continue reading?

Register now to search topics and post comments!

Absolutely FREE!


User avatar
jenesaislaw

Silver
Posts: 1005
Joined: Mon May 19, 2008 6:35 pm

Re: US News Law School Rankings -- 5 Failures

Post by jenesaislaw » Mon Mar 09, 2015 3:17 pm

Dafaq wrote:
jenesaislaw wrote:However, ranking credibility may be lost when methodologies are unsound, through irrational weighting or meaningless metrics, or when the scope is too broad. The legal profession is worse off for elevating the importance of a publication that falls victim to these flaws each and every year.
Albeit, UT is a perennial #15 on USNWR and also #15 on ATL.
and...

User avatar
lacrossebrother

Platinum
Posts: 7150
Joined: Wed Jul 30, 2014 11:15 pm

Re: US News Law School Rankings -- 5 Failures

Post by lacrossebrother » Thu Apr 09, 2015 5:59 pm

I found this article to be very misleading, alarmist, populist, click-baity, etc. There are definitely problems with this set of rankings, and obviously noone should be relying entirely on them --but they're the gold standard and as all parties involved (students, deans, faculty, firms, judges) continue to rely upon them, the significance compounds. Of course people will like taking shots at them, because agreement rarely gets attention --but I think this article is a particularly bad account.

I wrote a criticism of this in the other thread that this was posted in back in March in the Choosing a law school forum (where this probably exclusively belongs)...but since this has been stickied, I'd like to also cross post my response (http://top-law-schools.com/forums/viewt ... 6#p8461468) which unfortunately never got a rebuttal. Perhaps some of my concerns can be addressed now that this is stickied.
lacrossebrother wrote:Dumb article.
1. the goal is to come up with some proxy to determine what the best law schools are. anyone can list last year's employment rankings. you'll find however that those numbers fluctuate. the fact that everyone says you want to be a lawyer --that's the point of the lsat --doesn't mean that the fucking purpose of legal education is to maximize employment outcomes only.

what's fucking even more insane is that you started an entire website called "law school transparency" whereby you challenge the notion that a single percentage of employment is an appropriate means to gauge employment :lol: :lol: . so you fucking say "only 18% of your ranking is based on this really important number..." even though you think it's a shitty number to begin. :roll: :roll: dope.

moreover, "only 18%"?? you give us absolutely no idea if that's actually high or low. do a sensititivy analysis with this number you hate so much and then offer your still super layman's opinion that "this is failure number 1."

2. i don't even get this point. it's impossible to try to determine if university of arizona is a better school than uconn because people who go to school in arizona won't want to work in uconn? No shit. Perhaps this is why they don't do a fucking 100% employment ranking?? Instead, they ask judges and employers to rate their perception of the products of these schools. They try to rate the faculty. And ya, they take a look at the fucking incoming product because it's pretty easy to see that if one school, on average, has a bunch of dopes attending and the other doesn't, the fact that they aren't in competitive markets doesn't preclude reaching a conclusion. They try to gauge the respective quality of the school's infrastructure and commitment to research by looking at the size of the library (volumes is an imprecise measure, but whatever). The fact that they attempt to do this for everyone, for their magazine, doesn't make the whole thing a failure.

3. :lol: There's a raw score you fucking moron
4. :lol: it's a failure that the magazine doesn't do a more in-depth analysis of each 2-3 swap? Mr. "only 18%" is upset that the magazine doesn't explain with data graphics somewhere what the most heavily involved coefficient contributing to each swap.
5. ok ya i agree this is dumb. what failures.

User avatar
Lawgenius

New
Posts: 11
Joined: Thu May 14, 2015 2:04 am

Re: US News Law School Rankings -- 5 Failures

Post by Lawgenius » Sun May 17, 2015 3:14 am

lacrossebrother wrote:moreover, "only 18%"?? you give us absolutely no idea if that's actually high or low. do a sensititivy analysis with this number you hate so much and then offer your still super layman's opinion that "this is failure number 1."
[/quote]

So basically they did a survey accounting only for 18 percent of the rank and expected law students to come over to these schools due to authority and popularity. Obviously the idea is gold, and misleading, and it is funny. 18 percent is a bad number, do we need to argue with this? 18/100 isn’t even 1/2 of the 1/4 of 100. So tell me why it is a good number and i will still not believe.

aretoodeetoo

New
Posts: 73
Joined: Fri Jan 09, 2015 6:05 am

Re: US News Law School Rankings -- 5 Failures

Post by aretoodeetoo » Sun May 17, 2015 3:42 am

lacrossebrother wrote:I found this article to be very misleading, alarmist, populist, click-baity, etc. There are definitely problems with this set of rankings, and obviously noone should be relying entirely on them --but they're the gold standard and as all parties involved (students, deans, faculty, firms, judges) continue to rely upon them, the significance compounds. Of course people will like taking shots at them, because agreement rarely gets attention --but I think this article is a particularly bad account.

I wrote a criticism of this in the other thread that this was posted in back in March in the Choosing a law school forum (where this probably exclusively belongs)...but since this has been stickied, I'd like to also cross post my response (http://top-law-schools.com/forums/viewt ... 6#p8461468) which unfortunately never got a rebuttal. Perhaps some of my concerns can be addressed now that this is stickied.
lacrossebrother wrote:Dumb article.
1. the goal is to come up with some proxy to determine what the best law schools are. anyone can list last year's employment rankings. you'll find however that those numbers fluctuate. the fact that everyone says you want to be a lawyer --that's the point of the lsat --doesn't mean that the fucking purpose of legal education is to maximize employment outcomes only.

what's fucking even more insane is that you started an entire website called "law school transparency" whereby you challenge the notion that a single percentage of employment is an appropriate means to gauge employment :lol: :lol: . so you fucking say "only 18% of your ranking is based on this really important number..." even though you think it's a shitty number to begin. :roll: :roll: dope.

moreover, "only 18%"?? you give us absolutely no idea if that's actually high or low. do a sensititivy analysis with this number you hate so much and then offer your still super layman's opinion that "this is failure number 1."

2. i don't even get this point. it's impossible to try to determine if university of arizona is a better school than uconn because people who go to school in arizona won't want to work in uconn? No shit. Perhaps this is why they don't do a fucking 100% employment ranking?? Instead, they ask judges and employers to rate their perception of the products of these schools. They try to rate the faculty. And ya, they take a look at the fucking incoming product because it's pretty easy to see that if one school, on average, has a bunch of dopes attending and the other doesn't, the fact that they aren't in competitive markets doesn't preclude reaching a conclusion. They try to gauge the respective quality of the school's infrastructure and commitment to research by looking at the size of the library (volumes is an imprecise measure, but whatever). The fact that they attempt to do this for everyone, for their magazine, doesn't make the whole thing a failure.

3. :lol: There's a raw score you fucking moron
4. :lol: it's a failure that the magazine doesn't do a more in-depth analysis of each 2-3 swap? Mr. "only 18%" is upset that the magazine doesn't explain with data graphics somewhere what the most heavily involved coefficient contributing to each swap.
5. ok ya i agree this is dumb. what failures.
you mean you wrote a fucking criticism of this fucking thing in the other fucking thread way back in fucking March. Fuckkkkk. to be fair i fucking agree with you. bro

Want to continue reading?

Register for access!

Did I mention it was FREE ?


artistar

New
Posts: 40
Joined: Mon Jun 01, 2015 1:02 am

Re: US News Law School Rankings -- 5 Failures

Post by artistar » Tue Jun 09, 2015 11:18 pm

The point that schools are spending money to improve rankings when they could be putting it towards lowering the cost of tuition is a good one. I've read many of these articles, but few note that.

Troianii

Silver
Posts: 542
Joined: Sun Nov 24, 2013 5:13 am

Re: US News Law School Rankings -- 5 Failures

Post by Troianii » Thu Sep 17, 2015 8:40 pm

runinthefront wrote:I think your article was great, but I do wish there was a bit more detail about the actual "methodology" that US News uses. I was vaguely familiar with the methodology, but after looking at http://www.usnews.com/education/best-gr ... s-rankings , I now feel even more disgusted by how nonsensical their approach to ranking schools is, and how much weight 0Ls place on them.
Yeah, it's really interesting to see how much weight is put on factors that, in many cases, won't even matter to students. LST has a system where the applicant picks what is most important to them, and then it rates law schools based on those factors. Personally I prefer LST's system, but the different systems (USN, ATL, LST, etc.) for the most part don't end up showing much difference. T14s remain T14s.

http://www.lstscorereports.com/state/MA/

Troianii

Silver
Posts: 542
Joined: Sun Nov 24, 2013 5:13 am

Re: US News Law School Rankings -- 5 Failures

Post by Troianii » Thu Sep 17, 2015 8:41 pm

artistar wrote:The point that schools are spending money to improve rankings when they could be putting it towards lowering the cost of tuition is a good one. I've read many of these articles, but few note that.
But it feels so much cooler to get a conditional 30k scholarship at a 50k/yr school than to get no scholarship at a 20k/yr school.

Dumb, but isn't that the truth?

User avatar
HiLine

Bronze
Posts: 210
Joined: Thu Dec 31, 2009 11:57 am

Re: US News Law School Rankings -- 5 Failures

Post by HiLine » Fri Jun 03, 2016 4:36 am

lacrossebrother wrote:I found this article to be very misleading, alarmist, populist, click-baity, etc. There are definitely problems with this set of rankings, and obviously noone should be relying entirely on them --but they're the gold standard and as all parties involved (students, deans, faculty, firms, judges) continue to rely upon them, the significance compounds. Of course people will like taking shots at them, because agreement rarely gets attention --but I think this article is a particularly bad account.

I wrote a criticism of this in the other thread that this was posted in back in March in the Choosing a law school forum (where this probably exclusively belongs)...but since this has been stickied, I'd like to also cross post my response (http://top-law-schools.com/forums/viewt ... 6#p8461468) which unfortunately never got a rebuttal. Perhaps some of my concerns can be addressed now that this is stickied.
Yet another year later, still no rebuttal. I feel bad for you.

Register now!

Resources to assist law school applicants, students & graduates.

It's still FREE!


User avatar
lacrossebrother

Platinum
Posts: 7150
Joined: Wed Jul 30, 2014 11:15 pm

Re: US News Law School Rankings -- 5 Failures

Post by lacrossebrother » Sun Jun 19, 2016 5:12 pm

HiLine wrote:
lacrossebrother wrote:I found this article to be very misleading, alarmist, populist, click-baity, etc. There are definitely problems with this set of rankings, and obviously noone should be relying entirely on them --but they're the gold standard and as all parties involved (students, deans, faculty, firms, judges) continue to rely upon them, the significance compounds. Of course people will like taking shots at them, because agreement rarely gets attention --but I think this article is a particularly bad account.

I wrote a criticism of this in the other thread that this was posted in back in March in the Choosing a law school forum (where this probably exclusively belongs)...but since this has been stickied, I'd like to also cross post my response (http://top-law-schools.com/forums/viewt ... 6#p8461468) which unfortunately never got a rebuttal. Perhaps some of my concerns can be addressed now that this is stickied.
Yet another year later, still no rebuttal. I feel bad for you.
Haha. Don't feel bad. LST has and will always remain a shittily run organization despite its noble intentions. They thrive on the law school is a "scam" concept, but can't see that the best way to fix "failures" is to work with the organizations, not against them.

Indeed, that a year has passed and US News remains the most important school ranker should make you pity LST. Not me. to be. Long live the king.

Nebby

Diamond
Posts: 31195
Joined: Sat Feb 01, 2014 12:23 pm

Re: US News Law School Rankings -- 5 Failures

Post by Nebby » Sun Jun 19, 2016 7:42 pm

Shut up Lax

Goldchain

New
Posts: 52
Joined: Sun Apr 10, 2016 4:29 pm

Re: US News Law School Rankings -- 5 Failures

Post by Goldchain » Sun Jun 19, 2016 8:37 pm

I wonder why tls quit updating the rankings. Anyone know?

thelifeofapanca

New
Posts: 1
Joined: Sat Mar 25, 2017 7:58 am

Re: US News Law School Rankings -- 5 Failures

Post by thelifeofapanca » Sun Mar 26, 2017 3:39 pm

Goldchain wrote:I wonder why tls quit updating the rankings. Anyone know?

I wonder that as well!

Get unlimited access to all forums and topics

Register now!

I'm pretty sure I told you it's FREE...


seanmitchell

New
Posts: 2
Joined: Sat Apr 29, 2017 12:41 pm

Re: US News Law School Rankings -- 5 Failures

Post by seanmitchell » Sat Apr 29, 2017 12:45 pm

Job outcomes for lawyers are dismal. Anyone planning to go to law school these days is frankly, not that bright to begin with. If you want real, solid job opportunities, get a STEM degree. And grow up.

Nebby

Diamond
Posts: 31195
Joined: Sat Feb 01, 2014 12:23 pm

Re: US News Law School Rankings -- 5 Failures

Post by Nebby » Sat Apr 29, 2017 1:18 pm

seanmitchell wrote:Job outcomes for lawyers are dismal. Anyone planning to go to law school these days is frankly, not that bright to begin with. If you want real, solid job opportunities, get a STEM degree. And grow up.
Job outcomes are actually pretty good if you go to a good school.

User avatar
cavalier1138

Moderator
Posts: 8007
Joined: Fri Mar 25, 2016 8:01 pm

Re: US News Law School Rankings -- 5 Failures

Post by cavalier1138 » Sat Apr 29, 2017 1:19 pm

seanmitchell wrote:Job outcomes for lawyers are dismal. Anyone planning to go to law school these days is frankly, not that bright to begin with. If you want real, solid job opportunities, get a STEM degree. And grow up.
Yeah! Anyone not getting a STEM degree and programming stuff into other stuff is an idiot! Especially judges, teachers, lawyers, civil servants, and pretty much everyone else who isn't in a STEM field. Idiots, every last one of them.

User avatar
Deserving Porcupine

New
Posts: 66
Joined: Sun Apr 23, 2017 10:46 am

Re: US News Law School Rankings -- 5 Failures

Post by Deserving Porcupine » Sat Apr 29, 2017 1:30 pm

cavalier1138 wrote:
seanmitchell wrote:Job outcomes for lawyers are dismal. Anyone planning to go to law school these days is frankly, not that bright to begin with. If you want real, solid job opportunities, get a STEM degree. And grow up.
Yeah! Anyone not getting a STEM degree and programming stuff into other stuff is an idiot! Especially judges, teachers, lawyers, civil servants, and pretty much everyone else who isn't in a STEM field. Idiots, every last one of them.
Yeah, also definitely no oversupply of STEM graduates with grad degrees. You basically get a tenure track academic job just for showing up and smiling nicely. Academia is not fiercely competitive, plus there are all those PhDs whose dream was to be working as a lab tech all along... Dreams really do come true. For. Every. STEM. Major.

Communicate now with those who not only know what a legal education is, but can offer you worthy advice and commentary as you complete the three most educational, yet challenging years of your law related post graduate life.

Register now, it's still FREE!


Jay2021

New
Posts: 22
Joined: Wed Sep 27, 2017 1:24 am

Re: US News Law School Rankings -- 5 Failures

Post by Jay2021 » Wed Sep 27, 2017 2:05 am

If US News is flawed, what's a better list? How am I to decide how good a law school is, which is better than what and so on?

Jay2021

New
Posts: 22
Joined: Wed Sep 27, 2017 1:24 am

Re: US News Law School Rankings -- 5 Failures

Post by Jay2021 » Wed Sep 27, 2017 2:09 am

Deserving Porcupine wrote:Dreams really do come true. For. Every. STEM. Major.
This is simply false. I have a degree in "STEM" and have been underemployed since graduating in the Great Recession. I am now switching to law as plan B.

User avatar
UVAIce

Bronze
Posts: 451
Joined: Tue Jul 10, 2012 3:10 pm

Re: US News Law School Rankings -- 5 Failures

Post by UVAIce » Wed Sep 27, 2017 8:27 am

Jay2021 wrote:
Deserving Porcupine wrote:Dreams really do come true. For. Every. STEM. Major.
This is simply false. I have a degree in "STEM" and have been underemployed since graduating in the Great Recession. I am now switching to law as plan B.
Come on man. Unless this is some meta trolling of engineers, you cannot have missed the sarcasm there.

Jay2021

New
Posts: 22
Joined: Wed Sep 27, 2017 1:24 am

Re: US News Law School Rankings -- 5 Failures

Post by Jay2021 » Wed Sep 27, 2017 9:28 am

UVAIce wrote:
Jay2021 wrote:
Deserving Porcupine wrote:Dreams really do come true. For. Every. STEM. Major.
This is simply false. I have a degree in "STEM" and have been underemployed since graduating in the Great Recession. I am now switching to law as plan B.
Come on man. Unless this is some meta trolling of engineers, you cannot have missed the sarcasm there.
Sorry man, it was really late and it went right over my head. It's a sore point for me because you see everywhere that STEM is a golden ticket and it's not.

Seriously? What are you waiting for?

Now there's a charge.
Just kidding ... it's still FREE!


Post Reply

Return to “Law School Admissions Forum”