US News Law School Rankings -- 5 Failures

(Applications Advice, Letters of Recommendation . . . )
User avatar
jenesaislaw
Posts: 996
Joined: Mon May 19, 2008 6:35 pm

US News Law School Rankings -- 5 Failures

Postby jenesaislaw » Mon Mar 09, 2015 2:16 pm

I just published a piece on Law.com entitled, Five Failures of the U.S. News Rankings.

Here is the body:

Tomorrow, the law school world will overreact to slightly-shuffled U.S. News rankings. Proud alumni and worried students will voice concerns. Provosts will threaten jobs. Prospective students will confuse the annual shuffle with genuine reputational change.

Law school administrators will react predictably. They’ll articulate methodological flaws and lament negative externalities, but will nevertheless commit to the rankings game through their statements and actions. Assuring stakeholders bearing pitchforks has become part of the job description.

If the rankings measured something useful, the entire charade would be much easier to stomach. The unfortunate irony is that these rankings adversely affect the decision-making process for law school administrators and prospective law students alike. The stakes are high. Our profession and society need law schools that don’t figure inefficient metrics into annual budgets. Dollars spent chasing U.S. News rankings diverts funds away from students’ education. It also stands in the way of reducing tuition.

In this post, I examine five U.S. News rankings failures. I consider the methodology and underlying rankings theory from the perspective of a student who features job prospects prominently in his application and enrollment decisions. Considering the near universal support for prioritizing job outcomes in the process, these failures demonstrate just some of the reasons annual consternation hardly seems worth it.

Image
Word cloud of LSAT test-takers surveyed about the reasons they plan to obtain a J.D. degree. Source: Above the Law.


First, the rankings pay insufficient attention to what matters most to prospective students: job outcomes. In a survey of 600 students studying for the October 2012 LSAT, Breaking Media’s research arm found that the two most popular words associated with the students’ purpose of getting a J.D. were “career” and “work” (image above). These are not exactly shocking results. Despite the importance of job outcomes, they account for only 18% of the rank and credit schools for jobs few attend law school to pursue.

Second, the rankings use a national scope, which places schools on the same scale. Only a handful of schools have a truly national reach in job placement. The rest have a regional, in-state, or even just local reach. The relative positioning of California Western and West Virginia in the rankings is virtually meaningless. Graduates from these schools do not compete with one another.

Image
Percentages in above graphic are of the entire class.

It turns out that 158 schools place at least half of their employed class of 2013 graduates in one state. The top state destination for each school accounts for 67% of employed graduates. A much smaller 8.2% of employed graduates go to a school’s second most popular destination, with just 4.5% of employed graduates working in the third most popular destination. Only 20.4% of employed graduates (16.7% of the entire class) end up in a state other than the top three. Comparing schools across the country just doesn’t make sense.

Third, U.S. News rankings follow an ordinal system that fails to show the degree of difference between schools. Are Columbia and NYU virtually tied? Or does the two-rank difference represent a wide gulf in quality? Is the so-called difference between Columbia and NYU the same as the difference between Cornell and Vanderbilt? Students weighing school prices need to know not just which school is better but how much better it is.

Fourth, performance changes over time but year-to-year comparisons are virtually impossible using the U.S. News rankings. U.S. News will tell you that Stanford knocked Harvard out of the #2 spot in 2012-13, but the swap in rankings does not provide any clues to your average reader as to why. Stanford may have improved while Harvard declined. Or, Stanford may have improved while Harvard’s quality stayed the same. Or perhaps both schools saw a decline in quality but Harvard’s decline was more severe. In fact, if every single school saw a marked decline in quality the U.S. News rankings would not indicate that this happened. Instead, students can know only relative performance.

Finally, U.S. News inexplicably places every ABA accredited law school on the list of "The Best." The best at what? U.S. News doesn’t say. But it implies that every school on the list is good. The truth is that once costs and employment outcomes are considered in comparison to personal career goals, many schools are bad choices. The U.S. News rankings provide no help in drawing the line.

Rankings are not inherently bad. In fact, they are conceptually quite useful. They order comparable things to help people sort through more information than they know how to or can weigh. However, ranking credibility may be lost when methodologies are unsound, through irrational weighting or meaningless metrics, or when the scope is too broad. The legal profession is worse off for elevating the importance of a publication that falls victim to these flaws each and every year.

User avatar
zombie mcavoy
Posts: 428
Joined: Mon Mar 02, 2015 10:11 pm

Re: US News Law School Rankings -- 5 Failures

Postby zombie mcavoy » Mon Mar 09, 2015 2:24 pm

motion to sticky

User avatar
ILoveYou
Posts: 490
Joined: Thu Jan 29, 2015 3:42 pm

Re: US News Law School Rankings -- 5 Failures

Postby ILoveYou » Mon Mar 09, 2015 2:26 pm

zombie mcavoy wrote:motion to sticky


+1

Mal Reynolds
Posts: 12630
Joined: Mon Oct 03, 2011 12:16 am

Re: US News Law School Rankings -- 5 Failures

Postby Mal Reynolds » Mon Mar 09, 2015 2:29 pm

Great post.

User avatar
runinthefront
Posts: 1090
Joined: Wed Jan 15, 2014 2:18 am

Re: US News Law School Rankings -- 5 Failures

Postby runinthefront » Mon Mar 09, 2015 2:31 pm

I think your article was great, but I do wish there was a bit more detail about the actual "methodology" that US News uses. I was vaguely familiar with the methodology, but after looking at http://www.usnews.com/education/best-gr ... s-rankings , I now feel even more disgusted by how nonsensical their approach to ranking schools is, and how much weight 0Ls place on them.

User avatar
romothesavior
Posts: 14772
Joined: Fri Jun 26, 2009 4:29 pm

Re: US News Law School Rankings -- 5 Failures

Postby romothesavior » Mon Mar 09, 2015 2:48 pm

Stickied and copied to the Law School Admissions Forum.

Great article.

User avatar
Dafaq
Posts: 354
Joined: Sat Feb 15, 2014 6:19 pm

Re: US News Law School Rankings -- 5 Failures

Postby Dafaq » Mon Mar 09, 2015 3:02 pm

jenesaislaw wrote:However, ranking credibility may be lost when methodologies are unsound, through irrational weighting or meaningless metrics, or when the scope is too broad. The legal profession is worse off for elevating the importance of a publication that falls victim to these flaws each and every year.

Albeit, UT is a perennial #15 on USNWR and also #15 on ATL.

User avatar
jenesaislaw
Posts: 996
Joined: Mon May 19, 2008 6:35 pm

Re: US News Law School Rankings -- 5 Failures

Postby jenesaislaw » Mon Mar 09, 2015 3:17 pm

Dafaq wrote:
jenesaislaw wrote:However, ranking credibility may be lost when methodologies are unsound, through irrational weighting or meaningless metrics, or when the scope is too broad. The legal profession is worse off for elevating the importance of a publication that falls victim to these flaws each and every year.

Albeit, UT is a perennial #15 on USNWR and also #15 on ATL.


and...

User avatar
lacrossebrother
Top 17 consensus poster
Posts: 6850
Joined: Wed Jul 30, 2014 11:15 pm

Re: US News Law School Rankings -- 5 Failures

Postby lacrossebrother » Thu Apr 09, 2015 5:59 pm

I found this article to be very misleading, alarmist, populist, click-baity, etc. There are definitely problems with this set of rankings, and obviously noone should be relying entirely on them --but they're the gold standard and as all parties involved (students, deans, faculty, firms, judges) continue to rely upon them, the significance compounds. Of course people will like taking shots at them, because agreement rarely gets attention --but I think this article is a particularly bad account.

I wrote a criticism of this in the other thread that this was posted in back in March in the Choosing a law school forum (where this probably exclusively belongs)...but since this has been stickied, I'd like to also cross post my response (viewtopic.php?f=1&t=244746#p8461468) which unfortunately never got a rebuttal. Perhaps some of my concerns can be addressed now that this is stickied.

lacrossebrother wrote:Dumb article.
1. the goal is to come up with some proxy to determine what the best law schools are. anyone can list last year's employment rankings. you'll find however that those numbers fluctuate. the fact that everyone says you want to be a lawyer --that's the point of the lsat --doesn't mean that the fucking purpose of legal education is to maximize employment outcomes only.

what's fucking even more insane is that you started an entire website called "law school transparency" whereby you challenge the notion that a single percentage of employment is an appropriate means to gauge employment :lol: :lol: . so you fucking say "only 18% of your ranking is based on this really important number..." even though you think it's a shitty number to begin. :roll: :roll: dope.

moreover, "only 18%"?? you give us absolutely no idea if that's actually high or low. do a sensititivy analysis with this number you hate so much and then offer your still super layman's opinion that "this is failure number 1."

2. i don't even get this point. it's impossible to try to determine if university of arizona is a better school than uconn because people who go to school in arizona won't want to work in uconn? No shit. Perhaps this is why they don't do a fucking 100% employment ranking?? Instead, they ask judges and employers to rate their perception of the products of these schools. They try to rate the faculty. And ya, they take a look at the fucking incoming product because it's pretty easy to see that if one school, on average, has a bunch of dopes attending and the other doesn't, the fact that they aren't in competitive markets doesn't preclude reaching a conclusion. They try to gauge the respective quality of the school's infrastructure and commitment to research by looking at the size of the library (volumes is an imprecise measure, but whatever). The fact that they attempt to do this for everyone, for their magazine, doesn't make the whole thing a failure.

3. :lol: There's a raw score you fucking moron
4. :lol: it's a failure that the magazine doesn't do a more in-depth analysis of each 2-3 swap? Mr. "only 18%" is upset that the magazine doesn't explain with data graphics somewhere what the most heavily involved coefficient contributing to each swap.
5. ok ya i agree this is dumb. what failures.

User avatar
Nebby
Posts: 21873
Joined: Sat Feb 01, 2014 12:23 pm

Re: US News Law School Rankings -- 5 Failures

Postby Nebby » Sat Apr 18, 2015 3:04 pm

Image

User avatar
Lawgenius
Posts: 12
Joined: Thu May 14, 2015 2:04 am

Re: US News Law School Rankings -- 5 Failures

Postby Lawgenius » Sun May 17, 2015 3:14 am

lacrossebrother wrote:moreover, "only 18%"?? you give us absolutely no idea if that's actually high or low. do a sensititivy analysis with this number you hate so much and then offer your still super layman's opinion that "this is failure number 1."
[/quote]

So basically they did a survey accounting only for 18 percent of the rank and expected law students to come over to these schools due to authority and popularity. Obviously the idea is gold, and misleading, and it is funny. 18 percent is a bad number, do we need to argue with this? 18/100 isn’t even 1/2 of the 1/4 of 100. So tell me why it is a good number and i will still not believe.

aretoodeetoo
Posts: 73
Joined: Fri Jan 09, 2015 6:05 am

Re: US News Law School Rankings -- 5 Failures

Postby aretoodeetoo » Sun May 17, 2015 3:42 am

lacrossebrother wrote:I found this article to be very misleading, alarmist, populist, click-baity, etc. There are definitely problems with this set of rankings, and obviously noone should be relying entirely on them --but they're the gold standard and as all parties involved (students, deans, faculty, firms, judges) continue to rely upon them, the significance compounds. Of course people will like taking shots at them, because agreement rarely gets attention --but I think this article is a particularly bad account.

I wrote a criticism of this in the other thread that this was posted in back in March in the Choosing a law school forum (where this probably exclusively belongs)...but since this has been stickied, I'd like to also cross post my response (viewtopic.php?f=1&t=244746#p8461468) which unfortunately never got a rebuttal. Perhaps some of my concerns can be addressed now that this is stickied.

lacrossebrother wrote:Dumb article.
1. the goal is to come up with some proxy to determine what the best law schools are. anyone can list last year's employment rankings. you'll find however that those numbers fluctuate. the fact that everyone says you want to be a lawyer --that's the point of the lsat --doesn't mean that the fucking purpose of legal education is to maximize employment outcomes only.

what's fucking even more insane is that you started an entire website called "law school transparency" whereby you challenge the notion that a single percentage of employment is an appropriate means to gauge employment :lol: :lol: . so you fucking say "only 18% of your ranking is based on this really important number..." even though you think it's a shitty number to begin. :roll: :roll: dope.

moreover, "only 18%"?? you give us absolutely no idea if that's actually high or low. do a sensititivy analysis with this number you hate so much and then offer your still super layman's opinion that "this is failure number 1."

2. i don't even get this point. it's impossible to try to determine if university of arizona is a better school than uconn because people who go to school in arizona won't want to work in uconn? No shit. Perhaps this is why they don't do a fucking 100% employment ranking?? Instead, they ask judges and employers to rate their perception of the products of these schools. They try to rate the faculty. And ya, they take a look at the fucking incoming product because it's pretty easy to see that if one school, on average, has a bunch of dopes attending and the other doesn't, the fact that they aren't in competitive markets doesn't preclude reaching a conclusion. They try to gauge the respective quality of the school's infrastructure and commitment to research by looking at the size of the library (volumes is an imprecise measure, but whatever). The fact that they attempt to do this for everyone, for their magazine, doesn't make the whole thing a failure.

3. :lol: There's a raw score you fucking moron
4. :lol: it's a failure that the magazine doesn't do a more in-depth analysis of each 2-3 swap? Mr. "only 18%" is upset that the magazine doesn't explain with data graphics somewhere what the most heavily involved coefficient contributing to each swap.
5. ok ya i agree this is dumb. what failures.


you mean you wrote a fucking criticism of this fucking thing in the other fucking thread way back in fucking March. Fuckkkkk. to be fair i fucking agree with you. bro

artistar
Posts: 40
Joined: Mon Jun 01, 2015 1:02 am

Re: US News Law School Rankings -- 5 Failures

Postby artistar » Tue Jun 09, 2015 11:18 pm

The point that schools are spending money to improve rankings when they could be putting it towards lowering the cost of tuition is a good one. I've read many of these articles, but few note that.

Troianii
Posts: 542
Joined: Sun Nov 24, 2013 5:13 am

Re: US News Law School Rankings -- 5 Failures

Postby Troianii » Thu Sep 17, 2015 8:40 pm

runinthefront wrote:I think your article was great, but I do wish there was a bit more detail about the actual "methodology" that US News uses. I was vaguely familiar with the methodology, but after looking at http://www.usnews.com/education/best-gr ... s-rankings , I now feel even more disgusted by how nonsensical their approach to ranking schools is, and how much weight 0Ls place on them.


Yeah, it's really interesting to see how much weight is put on factors that, in many cases, won't even matter to students. LST has a system where the applicant picks what is most important to them, and then it rates law schools based on those factors. Personally I prefer LST's system, but the different systems (USN, ATL, LST, etc.) for the most part don't end up showing much difference. T14s remain T14s.

http://www.lstscorereports.com/state/MA/

Troianii
Posts: 542
Joined: Sun Nov 24, 2013 5:13 am

Re: US News Law School Rankings -- 5 Failures

Postby Troianii » Thu Sep 17, 2015 8:41 pm

artistar wrote:The point that schools are spending money to improve rankings when they could be putting it towards lowering the cost of tuition is a good one. I've read many of these articles, but few note that.


But it feels so much cooler to get a conditional 30k scholarship at a 50k/yr school than to get no scholarship at a 20k/yr school.

Dumb, but isn't that the truth?

User avatar
HiLine
Posts: 210
Joined: Thu Dec 31, 2009 11:57 am

Re: US News Law School Rankings -- 5 Failures

Postby HiLine » Fri Jun 03, 2016 4:36 am

lacrossebrother wrote:I found this article to be very misleading, alarmist, populist, click-baity, etc. There are definitely problems with this set of rankings, and obviously noone should be relying entirely on them --but they're the gold standard and as all parties involved (students, deans, faculty, firms, judges) continue to rely upon them, the significance compounds. Of course people will like taking shots at them, because agreement rarely gets attention --but I think this article is a particularly bad account.

I wrote a criticism of this in the other thread that this was posted in back in March in the Choosing a law school forum (where this probably exclusively belongs)...but since this has been stickied, I'd like to also cross post my response (http://top-law-schools.com/forums/viewt ... 6#p8461468) which unfortunately never got a rebuttal. Perhaps some of my concerns can be addressed now that this is stickied.



Yet another year later, still no rebuttal. I feel bad for you.

Angelic414
Posts: 6
Joined: Sat Jun 18, 2016 3:20 am

Re: US News Law School Rankings -- 5 Failures

Postby Angelic414 » Sat Jun 18, 2016 3:38 am

Hello Everyone,

I'm facing a bit of a crisis. So I took my LSAT for the first time in October 2015. I studied so hard and was in the 150s for my practice exams. Unfortunately, I only got a 149 on the exam. (My gpa is terrible its 3.3 - but I did work for Legal Aid for 2 years and also volunteered in Sudan)

I decided to apply see which schools I would get into and CUNY law was one of the schools that accepted me. At first I was excited about this because CUNY is cheap, and it focuses on public interest (Which is what I'd like to study). However, I am starting to feel anxious because I know that I could've done much better on the exam plus I have read so many negative comments about CUNY on this website and countless forums.

I already submitted my deposit to CUNY but I am not sure if I should retake the exam (is it too late?), go for the first year and transfer out, or write a letter of continued interest to St. John's University (waitlisted me).

Please help! Any advice is helpful!

User avatar
stego
Posts: 2834
Joined: Thu Apr 30, 2015 3:23 am

Re: US News Law School Rankings -- 5 Failures

Postby stego » Sat Jun 18, 2016 8:24 am

Angelic414 wrote:Hello Everyone,

I'm facing a bit of a crisis. So I took my LSAT for the first time in October 2015. I studied so hard and was in the 150s for my practice exams. Unfortunately, I only got a 149 on the exam. (My gpa is terrible its 3.3 - but I did work for Legal Aid for 2 years and also volunteered in Sudan)

I decided to apply see which schools I would get into and CUNY law was one of the schools that accepted me. At first I was excited about this because CUNY is cheap, and it focuses on public interest (Which is what I'd like to study). However, I am starting to feel anxious because I know that I could've done much better on the exam plus I have read so many negative comments about CUNY on this website and countless forums.

I already submitted my deposit to CUNY but I am not sure if I should retake the exam (is it too late?), go for the first year and transfer out, or write a letter of continued interest to St. John's University (waitlisted me).

Please help! Any advice is helpful!

Are you lost? I don't think you meant to post this here.

User avatar
lacrossebrother
Top 17 consensus poster
Posts: 6850
Joined: Wed Jul 30, 2014 11:15 pm

Re: US News Law School Rankings -- 5 Failures

Postby lacrossebrother » Sun Jun 19, 2016 5:12 pm

HiLine wrote:
lacrossebrother wrote:I found this article to be very misleading, alarmist, populist, click-baity, etc. There are definitely problems with this set of rankings, and obviously noone should be relying entirely on them --but they're the gold standard and as all parties involved (students, deans, faculty, firms, judges) continue to rely upon them, the significance compounds. Of course people will like taking shots at them, because agreement rarely gets attention --but I think this article is a particularly bad account.

I wrote a criticism of this in the other thread that this was posted in back in March in the Choosing a law school forum (where this probably exclusively belongs)...but since this has been stickied, I'd like to also cross post my response (viewtopic.php?f=1&t=244746#p8461468) which unfortunately never got a rebuttal. Perhaps some of my concerns can be addressed now that this is stickied.



Yet another year later, still no rebuttal. I feel bad for you.

Haha. Don't feel bad. LST has and will always remain a shittily run organization despite its noble intentions. They thrive on the law school is a "scam" concept, but can't see that the best way to fix "failures" is to work with the organizations, not against them.

Indeed, that a year has passed and US News remains the most important school ranker should make you pity LST. Not me. to be. Long live the king.

User avatar
Nebby
Posts: 21873
Joined: Sat Feb 01, 2014 12:23 pm

Re: US News Law School Rankings -- 5 Failures

Postby Nebby » Sun Jun 19, 2016 7:42 pm

Shut up Lax

Goldchain
Posts: 53
Joined: Sun Apr 10, 2016 4:29 pm

Re: US News Law School Rankings -- 5 Failures

Postby Goldchain » Sun Jun 19, 2016 8:37 pm

I wonder why tls quit updating the rankings. Anyone know?




Return to “Law School Admissions Forum”

Who is online

Users browsing this forum: Bing [Bot], Yahoo [Bot] and 2 guests