Something wrong with the US News rankings?...

(Applications Advice, Letters of Recommendation . . . )
User avatar
Havaianas
Posts: 287
Joined: Thu Oct 22, 2009 12:48 pm

Something wrong with the US News rankings?...

Postby Havaianas » Fri Feb 26, 2010 5:19 pm

Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017

From the intro:
"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"

edit:
For non-USNews rankings I've also been looking at:
lawdragon rankings - --LinkRemoved-- ... ol_update/
super lawyers rankings - --LinkRemoved-- ... ates/2009/
Princeton review rankings - http://www.princetonreview.com/law-school-rankings.aspx
Cooley rankings - --LinkRemoved--

edit: Thanks ggocat for posting these
Leiter Rankings: http://www.leiterrankings.com/new/index.shtml
TaxProfBlog used the Princeton Review raw data across multiple categories to rank the top 50: http://taxprof.typepad.com/taxprof_blog ... iew-1.html.

Student Quality as Measured by LSAT Scores: Migration Patterns in the U.S. News Rankings Era. --LinkRemoved--? ... _id=720122
Measuring Outcomes: Post-Graduation Measures of Success in the U.S. News & World Report Law School Rankings. --LinkRemoved--? ... _id=954604
Last edited by Havaianas on Sun Feb 28, 2010 3:14 pm, edited 4 times in total.

User avatar
zozin
Posts: 3733
Joined: Thu Apr 09, 2009 10:13 pm

Re: Something wrong with the US News rankings?...

Postby zozin » Fri Feb 26, 2010 6:32 pm

Citation fail, the first page is 5% text and the rest citations.

User avatar
Richie Tenenbaum
Posts: 2162
Joined: Wed Dec 31, 2008 6:17 am

Re: Something wrong with the US News rankings?...

Postby Richie Tenenbaum » Fri Feb 26, 2010 6:39 pm

Unfortunately, there is no fixed way of converting LSAT scores into
percentile equivalents. Because students sitting for a particular LSAT
administration may do a little better or a little worse than those taking the
test on a different date, percentile equivalents will not be identical across
test administrations.


Maybe this wasn't the case in 2007, but percentile equivalents are based on the the last three years of test-takers.

User avatar
twert
Posts: 414
Joined: Sun Jul 19, 2009 6:13 pm

Re: Something wrong with the US News rankings?...

Postby twert » Fri Feb 26, 2010 6:39 pm

good read. worth the time

User avatar
Havaianas
Posts: 287
Joined: Thu Oct 22, 2009 12:48 pm

Re: Something wrong with the US News rankings?...

Postby Havaianas » Fri Feb 26, 2010 6:54 pm

Richie Tenenbaum wrote:
Unfortunately, there is no fixed way of converting LSAT scores into
percentile equivalents. Because students sitting for a particular LSAT
administration may do a little better or a little worse than those taking the
test on a different date, percentile equivalents will not be identical across
test administrations.


Maybe this wasn't the case in 2007, but percentile equivalents are based on the the last three years of test-takers.


Yeah same with the fact that now the part time scores are included. Its a little different now but I dont think the LSAT administration would make that much of a difference. I actually found this article through : http://www.nxtbook.com/nxtbooks/cypress ... st0210/#/8 in which a side not addresses the new part time inclusion and GWs consequential drop.

User avatar
D. H2Oman
Posts: 7469
Joined: Tue Feb 24, 2009 5:47 am

Re: Something wrong with the US News rankings?...

Postby D. H2Oman » Fri Feb 26, 2010 7:59 pm

Havaianas wrote:Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"



Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.

User avatar
Trifles
Posts: 159
Joined: Thu Oct 22, 2009 10:55 pm

Re: Something wrong with the US News rankings?...

Postby Trifles » Fri Feb 26, 2010 10:34 pm

D. H2Oman wrote:Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.


Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.

User avatar
OperaSoprano
Posts: 4410
Joined: Tue Nov 25, 2008 1:54 am

Re: Something wrong with the US News rankings?...

Postby OperaSoprano » Fri Feb 26, 2010 10:39 pm

Havaianas wrote:[strike]Something[/strike] Everything wrong with the US News rankings?...

itsfine
Posts: 229
Joined: Mon Aug 10, 2009 1:07 am

Re: Something wrong with the US News rankings?...

Postby itsfine » Fri Feb 26, 2010 10:42 pm

OperaSoprano wrote:
Havaianas wrote:[strike]Something[/strike] Everything wrong with the US News [strike]Rankings?[/strike] Riggings?

User avatar
twert
Posts: 414
Joined: Sun Jul 19, 2009 6:13 pm

Re: Something wrong with the US News rankings?...

Postby twert » Fri Feb 26, 2010 10:43 pm

Trifles wrote:
D. H2Oman wrote:Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.


Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.

+1

User avatar
D. H2Oman
Posts: 7469
Joined: Tue Feb 24, 2009 5:47 am

Re: Something wrong with the US News rankings?...

Postby D. H2Oman » Fri Feb 26, 2010 10:44 pm

Trifles wrote:
D. H2Oman wrote:Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.


Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.



True, but it's still a stupid hypothetical. Yale is never going to do that.

If they are going to revamp the US News rankings they should probably fix some of the real issues it has now as opposed to some hypotheical problems that aren't even close to realistic.

itsfine
Posts: 229
Joined: Mon Aug 10, 2009 1:07 am

Re: Something wrong with the US News rankings?...

Postby itsfine » Fri Feb 26, 2010 10:45 pm

twert wrote:
Trifles wrote:
D. H2Oman wrote:Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.


Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.

+1



yeah but those rankings would be 'inefficient.'

User avatar
twert
Posts: 414
Joined: Sun Jul 19, 2009 6:13 pm

Re: Something wrong with the US News rankings?...

Postby twert » Fri Feb 26, 2010 11:48 pm

D. H2Oman wrote:
Trifles wrote:
D. H2Oman wrote:Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.


Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.



True, but it's still a stupid hypothetical. Yale is never going to do that.

If they are going to revamp the US News rankings they should probably fix some of the real issues it has now as opposed to some hypotheical problems that aren't even close to realistic.

the hypothetical is really just to demonstrate that there are weird things going on with the methodology. it prefaces a long discussion of all sorts of wonky things. read the article you'll see what i mean.

Derrex
Posts: 48
Joined: Tue Jul 22, 2008 1:04 am

Re: Something wrong with the US News rankings?...

Postby Derrex » Fri Feb 26, 2010 11:50 pm

Loyola? Never even heard of it.

User avatar
D. H2Oman
Posts: 7469
Joined: Tue Feb 24, 2009 5:47 am

Re: Something wrong with the US News rankings?...

Postby D. H2Oman » Sat Feb 27, 2010 12:06 am

twert wrote:
D. H2Oman wrote:
Trifles wrote:
D. H2Oman wrote:Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.


Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.



True, but it's still a stupid hypothetical. Yale is never going to do that.

If they are going to revamp the US News rankings they should probably fix some of the real issues it has now as opposed to some hypotheical problems that aren't even close to realistic.

the hypothetical is really just to demonstrate that there are weird things going on with the methodology. it prefaces a long discussion of all sorts of wonky things. read the article you'll see what i mean.


I don't disagree, but wouldn't the only way to fix that specific problem be to weigh the LSAT even more.

User avatar
twert
Posts: 414
Joined: Sun Jul 19, 2009 6:13 pm

Re: Something wrong with the US News rankings?...

Postby twert » Sat Feb 27, 2010 12:21 am

D. H2Oman wrote:
I don't disagree, but wouldn't the only way to fix that specific problem be to weigh the LSAT even more.


yeah, but i think his solution is to reduce the importance of the reputation survey, which would in turn make the lsat a larger piece of the puzzle. i don't know if this specifically would fix the issue with harvard and yale, but its what a good portion of his criticism of the rankings is.

User avatar
twert
Posts: 414
Joined: Sun Jul 19, 2009 6:13 pm

Re: Something wrong with the US News rankings?...

Postby twert » Sat Feb 27, 2010 12:22 am

Derrex wrote:Loyola? Never even heard of it.

seriously there's three of em?

User avatar
traehekat
Posts: 3195
Joined: Thu Apr 30, 2009 4:00 pm

Re: Something wrong with the US News rankings?...

Postby traehekat » Sat Feb 27, 2010 12:34 am

twert wrote:
Derrex wrote:Loyola? Never even heard of it.

seriously there's three of em?


Bitter Lawyer fail.

User avatar
Mattalones
Posts: 537
Joined: Sun Dec 03, 2006 8:18 pm

Re: Something wrong with the US News rankings?...

Postby Mattalones » Sat Feb 27, 2010 12:36 am

The problem with the rankings has nothing to do with the rankings themselves. It has to do with the buy-in that they have.

User avatar
quickquestionthanks
Posts: 629
Joined: Mon Jan 25, 2010 7:30 pm

Re: Something wrong with the US News rankings?...

Postby quickquestionthanks » Sat Feb 27, 2010 4:06 am

Yeah, if Yale had a median 153 or Harvard had a median 180, then the acceptance rate, employment/salary, peer assessment and ALL of those factors would be different than they actually are. They would move with that variable. If anything, he proves his own point by showing that the slight variations in LSAT scores are not the exact cause for the ranking differentiation.

This is why lawyers shouldn't get involved in multiple regression analysis.

User avatar
JustDude
Posts: 354
Joined: Wed Jan 02, 2008 10:07 pm

Re: Something wrong with the US News rankings?...

Postby JustDude » Sat Feb 27, 2010 4:17 am

Havaianas wrote:Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"



Brannigan??? Is that You>!

User avatar
D. H2Oman
Posts: 7469
Joined: Tue Feb 24, 2009 5:47 am

Re: Something wrong with the US News rankings?...

Postby D. H2Oman » Sat Feb 27, 2010 4:30 am

JustDude wrote:
Havaianas wrote:Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"



Brannigan??? Is that You>!


Hey JustDude :oops: :oops: :oops:

User avatar
JustDude
Posts: 354
Joined: Wed Jan 02, 2008 10:07 pm

Re: Something wrong with the US News rankings?...

Postby JustDude » Sat Feb 27, 2010 4:33 am

D. H2Oman wrote:
JustDude wrote:

Brannigan??? Is that You>!


Hey JustDude :oops: :oops: :oops:




Quite unexpected. Hijack attempt?? I mean you know I am drunk!!

User avatar
Havaianas
Posts: 287
Joined: Thu Oct 22, 2009 12:48 pm

Re: Something wrong with the US News rankings?...

Postby Havaianas » Sat Feb 27, 2010 12:02 pm

JustDude wrote:
Havaianas wrote:Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"



Brannigan??? Is that You>!


uhh no - I'm not Brannigan. And no, I don't really care whether Yale or Harvard is #1. I'm just trying to make some tough decisions.For non-USNews rankings I've also been looking at:
the lawdragon rankings - --LinkRemoved-- and
super lawyers rankings - http://www.superlawyers.com/toplists/la ... ates/2009/
and the Princeton review rankings - http://www.princetonreview.com/law-school-rankings.aspx

wired
Posts: 472
Joined: Tue Jun 23, 2009 2:29 pm

Re: Something wrong with the US News rankings?...

Postby wired » Sat Feb 27, 2010 12:31 pm

D. H2Oman wrote:
Havaianas wrote:Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"



Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.



I see where you coming from, but methodologically, US News ought to be trying to capture different effects. Instead, what it has essentially done, is have two different variables capturing only one effect. If lawyer's/judge's opinions are a function of LSAT medians, they shouldn't be calculating both. Just my opinion.




Return to “Law School Admissions Forum”

Who is online

Users browsing this forum: Bing [Bot] and 7 guests