Page 1 of 3

Something wrong with the US News rankings?...

Posted: Fri Feb 26, 2010 5:19 pm
by Havaianas
Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017

From the intro:
"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"

edit:
For non-USNews rankings I've also been looking at:
lawdragon rankings - --LinkRemoved-- ... ol_update/
super lawyers rankings - --LinkRemoved-- ... ates/2009/
Princeton review rankings - http://www.princetonreview.com/law-school-rankings.aspx
Cooley rankings - --LinkRemoved--

edit: Thanks ggocat for posting these
Leiter Rankings: http://www.leiterrankings.com/new/index.shtml
TaxProfBlog used the Princeton Review raw data across multiple categories to rank the top 50: http://taxprof.typepad.com/taxprof_blog ... iew-1.html.

Student Quality as Measured by LSAT Scores: Migration Patterns in the U.S. News Rankings Era. --LinkRemoved--? ... _id=720122
Measuring Outcomes: Post-Graduation Measures of Success in the U.S. News & World Report Law School Rankings. --LinkRemoved--? ... _id=954604

Re: Something wrong with the US News rankings?...

Posted: Fri Feb 26, 2010 6:32 pm
by zozin
Citation fail, the first page is 5% text and the rest citations.

Re: Something wrong with the US News rankings?...

Posted: Fri Feb 26, 2010 6:39 pm
by Richie Tenenbaum
Unfortunately, there is no fixed way of converting LSAT scores into
percentile equivalents. Because students sitting for a particular LSAT
administration may do a little better or a little worse than those taking the
test on a different date, percentile equivalents will not be identical across
test administrations.
Maybe this wasn't the case in 2007, but percentile equivalents are based on the the last three years of test-takers.

Re: Something wrong with the US News rankings?...

Posted: Fri Feb 26, 2010 6:39 pm
by twert
good read. worth the time

Re: Something wrong with the US News rankings?...

Posted: Fri Feb 26, 2010 6:54 pm
by Havaianas
Richie Tenenbaum wrote:
Unfortunately, there is no fixed way of converting LSAT scores into
percentile equivalents. Because students sitting for a particular LSAT
administration may do a little better or a little worse than those taking the
test on a different date, percentile equivalents will not be identical across
test administrations.
Maybe this wasn't the case in 2007, but percentile equivalents are based on the the last three years of test-takers.
Yeah same with the fact that now the part time scores are included. Its a little different now but I dont think the LSAT administration would make that much of a difference. I actually found this article through : http://www.nxtbook.com/nxtbooks/cypress ... st0210/#/8 in which a side not addresses the new part time inclusion and GWs consequential drop.

Re: Something wrong with the US News rankings?...

Posted: Fri Feb 26, 2010 7:59 pm
by D. H2Oman
Havaianas wrote:Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"

Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.

Re: Something wrong with the US News rankings?...

Posted: Fri Feb 26, 2010 10:34 pm
by Trifles
D. H2Oman wrote: Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.
Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.

Re: Something wrong with the US News rankings?...

Posted: Fri Feb 26, 2010 10:39 pm
by OperaSoprano
Havaianas wrote:[strike]Something[/strike] Everything wrong with the US News rankings?...

Re: Something wrong with the US News rankings?...

Posted: Fri Feb 26, 2010 10:42 pm
by itsfine
OperaSoprano wrote:
Havaianas wrote:[strike]Something[/strike] Everything wrong with the US News [strike]Rankings?[/strike] Riggings?

Re: Something wrong with the US News rankings?...

Posted: Fri Feb 26, 2010 10:43 pm
by twert
Trifles wrote:
D. H2Oman wrote: Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.
Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.
+1

Re: Something wrong with the US News rankings?...

Posted: Fri Feb 26, 2010 10:44 pm
by D. H2Oman
Trifles wrote:
D. H2Oman wrote: Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.
Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.

True, but it's still a stupid hypothetical. Yale is never going to do that.

If they are going to revamp the US News rankings they should probably fix some of the real issues it has now as opposed to some hypotheical problems that aren't even close to realistic.

Re: Something wrong with the US News rankings?...

Posted: Fri Feb 26, 2010 10:45 pm
by itsfine
twert wrote:
Trifles wrote:
D. H2Oman wrote: Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.
Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.
+1

yeah but those rankings would be 'inefficient.'

Re: Something wrong with the US News rankings?...

Posted: Fri Feb 26, 2010 11:48 pm
by twert
D. H2Oman wrote:
Trifles wrote:
D. H2Oman wrote: Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.
Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.

True, but it's still a stupid hypothetical. Yale is never going to do that.

If they are going to revamp the US News rankings they should probably fix some of the real issues it has now as opposed to some hypotheical problems that aren't even close to realistic.
the hypothetical is really just to demonstrate that there are weird things going on with the methodology. it prefaces a long discussion of all sorts of wonky things. read the article you'll see what i mean.

Re: Something wrong with the US News rankings?...

Posted: Fri Feb 26, 2010 11:50 pm
by Derrex
Loyola? Never even heard of it.

Re: Something wrong with the US News rankings?...

Posted: Sat Feb 27, 2010 12:06 am
by D. H2Oman
twert wrote:
D. H2Oman wrote:
Trifles wrote:
D. H2Oman wrote: Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.
Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.

True, but it's still a stupid hypothetical. Yale is never going to do that.

If they are going to revamp the US News rankings they should probably fix some of the real issues it has now as opposed to some hypotheical problems that aren't even close to realistic.
the hypothetical is really just to demonstrate that there are weird things going on with the methodology. it prefaces a long discussion of all sorts of wonky things. read the article you'll see what i mean.
I don't disagree, but wouldn't the only way to fix that specific problem be to weigh the LSAT even more.

Re: Something wrong with the US News rankings?...

Posted: Sat Feb 27, 2010 12:21 am
by twert
D. H2Oman wrote:
I don't disagree, but wouldn't the only way to fix that specific problem be to weigh the LSAT even more.
yeah, but i think his solution is to reduce the importance of the reputation survey, which would in turn make the lsat a larger piece of the puzzle. i don't know if this specifically would fix the issue with harvard and yale, but its what a good portion of his criticism of the rankings is.

Re: Something wrong with the US News rankings?...

Posted: Sat Feb 27, 2010 12:22 am
by twert
Derrex wrote:Loyola? Never even heard of it.
seriously there's three of em?

Re: Something wrong with the US News rankings?...

Posted: Sat Feb 27, 2010 12:34 am
by traehekat
twert wrote:
Derrex wrote:Loyola? Never even heard of it.
seriously there's three of em?
Bitter Lawyer fail.

Re: Something wrong with the US News rankings?...

Posted: Sat Feb 27, 2010 12:36 am
by Mattalones
The problem with the rankings has nothing to do with the rankings themselves. It has to do with the buy-in that they have.

Re: Something wrong with the US News rankings?...

Posted: Sat Feb 27, 2010 4:06 am
by quickquestionthanks
Yeah, if Yale had a median 153 or Harvard had a median 180, then the acceptance rate, employment/salary, peer assessment and ALL of those factors would be different than they actually are. They would move with that variable. If anything, he proves his own point by showing that the slight variations in LSAT scores are not the exact cause for the ranking differentiation.

This is why lawyers shouldn't get involved in multiple regression analysis.

Re: Something wrong with the US News rankings?...

Posted: Sat Feb 27, 2010 4:17 am
by JustDude
Havaianas wrote:Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"

Brannigan??? Is that You>!

Re: Something wrong with the US News rankings?...

Posted: Sat Feb 27, 2010 4:30 am
by D. H2Oman
JustDude wrote:
Havaianas wrote:Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"

Brannigan??? Is that You>!
Hey JustDude :oops: :oops: :oops:

Re: Something wrong with the US News rankings?...

Posted: Sat Feb 27, 2010 4:33 am
by JustDude
D. H2Oman wrote:
JustDude wrote:

Brannigan??? Is that You>!
Hey JustDude :oops: :oops: :oops:


Quite unexpected. Hijack attempt?? I mean you know I am drunk!!

Re: Something wrong with the US News rankings?...

Posted: Sat Feb 27, 2010 12:02 pm
by Havaianas
JustDude wrote:
Havaianas wrote:Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"

Brannigan??? Is that You>!
uhh no - I'm not Brannigan. And no, I don't really care whether Yale or Harvard is #1. I'm just trying to make some tough decisions.For non-USNews rankings I've also been looking at:
the lawdragon rankings - --LinkRemoved-- and
super lawyers rankings - http://www.superlawyers.com/toplists/la ... ates/2009/
and the Princeton review rankings - http://www.princetonreview.com/law-school-rankings.aspx

Re: Something wrong with the US News rankings?...

Posted: Sat Feb 27, 2010 12:31 pm
by wired
D. H2Oman wrote:
Havaianas wrote:Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"

Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.

I see where you coming from, but methodologically, US News ought to be trying to capture different effects. Instead, what it has essentially done, is have two different variables capturing only one effect. If lawyer's/judge's opinions are a function of LSAT medians, they shouldn't be calculating both. Just my opinion.