Something wrong with the US News rankings?... Forum

[Havaianas]

Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017

From the intro:
"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"

edit:
For non-USNews rankings I've also been looking at:
lawdragon rankings - --LinkRemoved-- ... ol_update/
super lawyers rankings - --LinkRemoved-- ... ates/2009/
Princeton review rankings - http://www.princetonreview.com/law-school-rankings.aspx
Cooley rankings - --LinkRemoved--

edit: Thanks ggocat for posting these
Leiter Rankings: http://www.leiterrankings.com/new/index.shtml
TaxProfBlog used the Princeton Review raw data across multiple categories to rank the top 50: http://taxprof.typepad.com/taxprof_blog ... iew-1.html.

Student Quality as Measured by LSAT Scores: Migration Patterns in the U.S. News Rankings Era. --LinkRemoved--? ... _id=720122
Measuring Outcomes: Post-Graduation Measures of Success in the U.S. News & World Report Law School Rankings. --LinkRemoved--? ... _id=954604

[zozin]

Citation fail, the first page is 5% text and the rest citations.

[Richie Tenenbaum]

Unfortunately, there is no fixed way of converting LSAT scores into
percentile equivalents. Because students sitting for a particular LSAT
administration may do a little better or a little worse than those taking the
test on a different date, percentile equivalents will not be identical across
test administrations.

Maybe this wasn't the case in 2007, but percentile equivalents are based on the the last three years of test-takers.

[twert]

good read. worth the time

[Havaianas]

Unfortunately, there is no fixed way of converting LSAT scores into
percentile equivalents. Because students sitting for a particular LSAT
administration may do a little better or a little worse than those taking the
test on a different date, percentile equivalents will not be identical across
test administrations.

Maybe this wasn't the case in 2007, but percentile equivalents are based on the the last three years of test-takers.

Yeah same with the fact that now the part time scores are included. Its a little different now but I dont think the LSAT administration would make that much of a difference. I actually found this article through : http://www.nxtbook.com/nxtbooks/cypress ... st0210/#/8 in which a side not addresses the new part time inclusion and GWs consequential drop.

[D. H2Oman]

Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"

Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.

[Trifles]

Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.

Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.

[OperaSoprano]

[strike]Something[/strike] Everything wrong with the US News rankings?...

[itsfine]

[strike]Something[/strike] Everything wrong with the US News [strike]Rankings?[/strike] Riggings?

[twert]

Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.

Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.

+1

[D. H2Oman]

Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.

Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.

True, but it's still a stupid hypothetical. Yale is never going to do that.

If they are going to revamp the US News rankings they should probably fix some of the real issues it has now as opposed to some hypotheical problems that aren't even close to realistic.

[itsfine]

Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.

Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.

+1

yeah but those rankings would be 'inefficient.'

[twert]

Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.

Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.

True, but it's still a stupid hypothetical. Yale is never going to do that.

If they are going to revamp the US News rankings they should probably fix some of the real issues it has now as opposed to some hypotheical problems that aren't even close to realistic.

the hypothetical is really just to demonstrate that there are weird things going on with the methodology. it prefaces a long discussion of all sorts of wonky things. read the article you'll see what i mean.

[Derrex]

Loyola? Never even heard of it.

[D. H2Oman]

Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.

Not in one year, before that information even came out or those students hit the workforce. They would still be ranked #1 that first year.

True, but it's still a stupid hypothetical. Yale is never going to do that.

If they are going to revamp the US News rankings they should probably fix some of the real issues it has now as opposed to some hypotheical problems that aren't even close to realistic.

the hypothetical is really just to demonstrate that there are weird things going on with the methodology. it prefaces a long discussion of all sorts of wonky things. read the article you'll see what i mean.

I don't disagree, but wouldn't the only way to fix that specific problem be to weigh the LSAT even more.

[twert]

I don't disagree, but wouldn't the only way to fix that specific problem be to weigh the LSAT even more.

yeah, but i think his solution is to reduce the importance of the reputation survey, which would in turn make the lsat a larger piece of the puzzle. i don't know if this specifically would fix the issue with harvard and yale, but its what a good portion of his criticism of the rankings is.

[twert]

Loyola? Never even heard of it.

seriously there's three of em?

[traehekat]

Loyola? Never even heard of it.

seriously there's three of em?

Bitter Lawyer fail.

[Mattalones]

The problem with the rankings has nothing to do with the rankings themselves. It has to do with the buy-in that they have.

[quickquestionthanks]

Yeah, if Yale had a median 153 or Harvard had a median 180, then the acceptance rate, employment/salary, peer assessment and ALL of those factors would be different than they actually are. They would move with that variable. If anything, he proves his own point by showing that the slight variations in LSAT scores are not the exact cause for the ranking differentiation.

This is why lawyers shouldn't get involved in multiple regression analysis.

[JustDude]

Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"

Brannigan??? Is that You>!

[D. H2Oman]

Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"

Brannigan??? Is that You>!

Hey JustDude

[JustDude]

Brannigan??? Is that You>!

Hey JustDude

Quite unexpected. Hijack attempt?? I mean you know I am drunk!!

[Havaianas]

Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"

Brannigan??? Is that You>!

uhh no - I'm not Brannigan. And no, I don't really care whether Yale or Harvard is #1. I'm just trying to make some tough decisions.For non-USNews rankings I've also been looking at:
the lawdragon rankings - --LinkRemoved-- and
super lawyers rankings - http://www.superlawyers.com/toplists/la ... ates/2009/
and the Princeton review rankings - http://www.princetonreview.com/law-school-rankings.aspx

[wired]

Check out this Loyola Law professor Theodore Seto's study...
http://papers.ssrn.com/sol3/papers.cfm? ... _id=937017
From the intro:

"Indeed, as I began playing with a spreadsheet I had written to
replicate the 2007 U.S. News computations, I discovered that even if
Harvard had reported a perfect median LSAT of 180, it still would have
been ranked third. And even if Yale had reported a median LSAT of just
153 (placing it in the “fourth tier” of law schools ranked by LSAT),9 it still
would have been ranked first. Indeed, Yale would have been ranked higher
than Harvard even if both had been true – if Harvard had reported a perfect
median LSAT and Yale a 153. I was stunned. Was Yale really that much
better than Harvard in all other material respects? If not, what might the
parts of U.S. News’ methodology that led to these counterintuitive results
tell us about the validity of U.S. News’ ranking of other schools?"

Stupid hypothetical, if Yale's LSAT median plummeted down to retard land, their peer and lawyer/judge assessment scores would plummet as well.

I see where you coming from, but methodologically, US News ought to be trying to capture different effects. Instead, what it has essentially done, is have two different variables capturing only one effect. If lawyer's/judge's opinions are a function of LSAT medians, they shouldn't be calculating both. Just my opinion.