US News Ranking Discussion Amongst deans Forum

[SandyC877]

Video by UVALawSchool
http://www.youtube.com/watch?v=_7_xHsce57c

Description of the video:

"University of Virginia law professor Alex Johnson, former chair of the Law School Admissions Council and former dean of Minnesota Law School, discusses the black/white LSAT score gap and why law schools are not admitting African-American students at a rate proportional to the test-taking population. Johnson offered his remarks during a talk sponsored by the UVA Black Law Students Association on Feb. 15."

Nevermind the part about URMs and LSAT, but fastforward to 23:20 into the video (about half way). You will hear this former chair of LSAC discuss the implications of USNews and World Ranking. I think the consensus among the deans (top and bottom) regarding US News Ranking is that it is highly subjective and evil. Furthermore, I think he makes a good point that most deans do not understand math and thus try to fudge the numbers the wrong way.

But here's what concerns me:

40% of US News Law School Ranking is based on "reputation" of the school. (Survey based on judges, academics, and lawyers)
http://www.usnews.com/articles/educatio ... l?PageNr=1

First, we have no idea how the questions are phrased or if the sample is representative. The response return rate on these surveys is extremely low, at about 30%. (Keep in mind, we dont know the total number, so it could be out of 1000 responses, or 10) This alone seriously raises doubt on validity of the "reputation" system. Furthermore, there are 3 possibly serious flaws.
1) We don't know who gets surveyed and who doesn't. For all we know, they could be asking the same judges year in and year out, producing the same reinforcing outcome for the select few schools (like it is now).
2) The responses could be heavily concentrated to a single area. For example, out of 100 responses, 90 could be from D.C, 8 from New York, and 2 from Los Angeles.
3) Subjective criteria takes up 40% in its entirety. Margin of error coupled with subjective opinion about a school far outweighs the tangible and real computable criteria. This to me is seriously questionable.

Peer assessment has no bearing on my legal education. Why should I care if some old guy on a bench thinks a certain way about a school? I am a firm believer that judges and lawyers, once they graduate law school, know jack shit about how much their school's quality of education has improved/dwindled.

Here are factors I do care about but matter only by a margin.

25% on actual caliber of the student body (GPA/LSAT) + (selectivity... wtf? How does rejecting more people than others have even the slightest relevance to the quality of education or even job placement?)
20% on Job placement + bar passage
1.5% on how much money they spend on the students.
3% on Student faculty ratio
<1% on Library resources
http://www.usnews.com/articles/educatio ... l?PageNr=2

Under the ABA guideline, all ABA-approved law schools must teach a series of required curriculum. While all law schools are remarkably similar in what they teach, I do agree that Cooley Law and Yale Law education is anything but similar. However, it becomes a problem when Yale can admit the entire class of 2013 with 140/2.0, make them sit on cardboard boxes, close down the law library - and still be ranked at the top. How is this a ranking? It's a game of "whoever opened their school first wins. Forever."

I think USNEWS Ranking is seriously questionable, but it is the best rankings out there. I am not saying that we should eliminate this practice of ranking, since it's a practical way to give rightful advantages to those who studied harder for the tests/gpa, but I think USNWR needs to rework where they place the emphasis. Also, it sure as hell beats a legal world of "who does my daddy know in that firm" approach.

Discuss.

[awesomepossum]

Why reputation matters.


You care about getting a job right? The reputation of your school to judges and lawyers matters in terms of your ability to get a job from those same judges and lawyers.

[acdisagod]

You should care what the old guys think because they are the ones who are going to be hiring you.

[wired]

Here are factors I do care about but matter only by a margin.

25% on actual caliber of the student body (GPA/LSAT) + (selectivity... wtf?)
20% on Job placement + bar passage
1.5% on how much money they spend on the students.
3% on Student faculty ratio
<1% on Library resources
http://www.usnews.com/articles/educatio ... l?PageNr=2

Sounds like you could make your own rankings system then. As for a lot of other people, the judge and peer rankings are all that matter. Judges and lawyers are the ones hiring students for the best positions out of law school.

Rankings bite and cause schools to do things that many people don't like. It's been that way for a long time.

[SandyC877]

Why reputation matters.


You care about getting a job right? The reputation of your school to judges and lawyers matters in terms of your ability to get a job from those same judges and lawyers.

You should care what the old guys think because they are the ones who are going to be hiring you.

We're ranking a school, an educational institution. and why the separate 20% for the job placement statistics if we're speaking strictly in terms of ability to get a job?
Schools are essentially being ranked by 1) What an old guy thinks about a school + 2) If that old guy actually hires from that school

[beesknees]

.

[Grizz]

These might be more up your alley.

http://www.cooley.edu/rankings/

[kittenmittons]

How mad are you about your LSAT score on a scale of 120-180?

[SandyC877]

Yeah, using such a ridiculously subjective concept as reputation as a major factor in the rankings is problematic. Seems like the category for employment stats work better to quantify the implications for employment prospects.

Furthermore, USNWR does not reveal how reputation is computed and we have no idea what their sample size is. So you can easily get certain biases towards certain schools and we would never be the wiser.

Also, "reputation" can also be a result of the USNWR as well as a variable of the USNWR. We won't go into how flawed that sort of system becomes. Not saying that Yale doesn't deserve it, but in this system, no matter what Yale does, it can never NOT be awesome.

The bold part is my biggest concern. Theoretically speaking, there is nothing Yale can involuntarily do to drop from their #1 ranking. There isn't a "Lose" option for Yale and there isn't a "Win" option for rest of the schools. Unless, of course, Yale decides to quit law school altogether

[beesknees]

.

[flyingpanda]

Yeah, using such a ridiculously subjective concept as reputation as a major factor in the rankings is problematic. Seems like the category for employment stats work better to quantify the implications for employment prospects.

Furthermore, USNWR does not reveal how reputation is computed and we have no idea what their sample size is. So you can easily get certain biases towards certain schools and we would never be the wiser.

Also, "reputation" can also be a result of the USNWR as well as a variable of the USNWR. We won't go into how flawed that sort of system becomes. Not saying that Yale doesn't deserve it, but in this system, no matter what Yale does, it can never NOT be awesome.

The bold part is my biggest concern. Theoretically speaking, there is nothing Yale can involuntarily do to drop from their #1 ranking. There isn't a "Lose" option for Yale and there isn't a "Win" option for rest of the schools. Unless, of course, Yale decides to quit law school altogether

Are you kidding me? Do you not realize that the factors in the ranking are connected? If Yale dropped its medians down to Cooley's medians and suddenly had terrible job placement, do you think that its reputation would stay the same? No....

[Rand M.]

Does anyone know how the 1-5 scale works on these surveys? For instance, if Yale has a 4.8 does that mean that 80% gave a 5 and 20% gave a 4? Or is a single respondent allowed to offer scores differentiated by a tenth of a point? Just curious. Some of the survey data always seems a bit suspect. Do Lawyers and Judges really hold UVa and CC as the same 4.6? Stuff like this always makes me wonder how they shake this number out.

[GeePee]

Does anyone know how the 1-5 scale works on these surveys? For instance, if Yale has a 4.8 does that mean that 80% gave a 5 and 20% gave a 4? Or is a single respondent allowed to offer scores differentiated by a tenth of a point? Just curious. Some of the survey data always seems a bit suspect. Do Lawyers and Judges really hold UVa and CC as the same 4.6? Stuff like this always makes me wonder how they shake this number out.

The scores are given on a very vanilla 1-5 integer scale. However, the real problem with these reputation scores isn't that they exist, it's that they are responded to sparsely. The lawyer/judge rankings have something like a 23% rate of response, which is pretty pathetic. Maybe if we had a better way to make sure that the responses were of a more representative sample, they'd be more conclusive.

[wired]

Does anyone know how the 1-5 scale works on these surveys? For instance, if Yale has a 4.8 does that mean that 80% gave a 5 and 20% gave a 4? Or is a single respondent allowed to offer scores differentiated by a tenth of a point? Just curious. Some of the survey data always seems a bit suspect. Do Lawyers and Judges really hold UVa and CC as the same 4.6? Stuff like this always makes me wonder how they shake this number out.

More likely is that 95% of respondents gave it a 5, then 5% representing bitter Harvard grads gave it a 1.

[awesomepossum]

Why reputation matters.


You care about getting a job right? The reputation of your school to judges and lawyers matters in terms of your ability to get a job from those same judges and lawyers.

You should care what the old guys think because they are the ones who are going to be hiring you.

We're ranking a school, an educational institution. and why the separate 20% for the job placement statistics if we're speaking strictly in terms of ability to get a job?
Schools are essentially being ranked by 1) What an old guy thinks about a school + 2) If that old guy actually hires from that school

it's not quite the same. Percentage hired is just any job...any garbage job. reputation represents whether people making the decisions think the school is good. This goes more towards the quality of the jobs people are likely to get.

[MConchis]

Did anyone listen to the last 40 seconds about eliminating section 503, and therefore eliminating the LSAT requirement?

[Grizz]

These might be more up your alley.

http://www.cooley.edu/rankings/

That was a little too easy, sir.

Someone had to say it.

[gwuorbust]

Did anyone listen to the last 40 seconds about eliminating section 503, and therefore eliminating the LSAT requirement?

but what could they replace it with ? It is like Ron Paul saying we should abolish the IRS but not offering a replacement. In both cases the system would essentially collapse.

[klussy]

Did anyone listen to the last 40 seconds about eliminating section 503, and therefore eliminating the LSAT requirement?

but what could they replace it with ? It is like Ron Paul saying we should abolish the IRS but not offering a replacement. In both cases the system would essentially collapse.

This may be wrong...but I once heard that Caltech doesn't require/look at the SAT or ACT, but it ends up that only the very top scorers get in. Meaning that the top scores match up with the rest of the application (achievements, GPA, courseload, etc.)

[j.wellington]

These and other rankings exist because Americans (a) are inordinately obsessed with superficial tokens of status and (b) like things having things put in numerical order. They also offer a diversion from that fact that your success will in the end come down to personal performance, networking skills and sheer luck.

A minimal amount of firsthand web research can determine which schools will suit your ambitions and an honest reflection on your own work ethic can determine your odds of success. Rankings are nothing more than fodder for the shortsighted, which is unfortunately a very large crowd. I can see why they drive admissions folks nuts.

[TTH]

Arbitrary rankings determining which schools are the best is wrong.

Law schools should have a playoff.

/thread

[eandy]

How is this a ranking? It's a game of "whoever opened their school first wins. Forever."

If this were true, William & Mary would be ranked a lot higher, no?

[timertimer61]

How is this a ranking? It's a game of "whoever opened their school first wins. Forever."

If this were true, William & Mary would be ranked a lot higher, no?

i think its more of being a necessary factor, and not a sufficient factor.

[awesomepossum]

that video was really interesting though. I'm sorry that the Q&A was cut off.

[Renzo]

Here's my proposal. We'll determine the rankings in a manner analogous to US presidential elections.

Every year schools will reach out to lawyers for donations (we won't count bequests). Then we rank schools based on donations per capita for average class size. Biggest spender wins. Just like real life it will favor the old money institutions, but it will also allow successful alums to vote up their alma maters, and allow unhappy or broke grads to withhold donations, or even vote for other schools.

Instead of driving up tuition, the rankings should drive it down by providing an alternative revenue stream and rewarding efficiency.