LAW SCHOOL RANKINGS & RATINGS by CanadianWolf Forum

[CanadianWolf]

That would be the equivalent of assuming that the internet & personal computers are simply a passing fancy.
Your suggestions are certainly reasonable as footnotes if not made crystal clear by the stated purpose of the rankings.

[beesknees]

.

[Helmholtz]

Ok, give Canadian a break. I understand that taking a ranking system that puts Yale 5th is a little ridiculous. But remove ridiculous outliers like Yale and then you might have something a little more meaningful. And remember, these rankings are judging schools pretty much by their firm placement.

I would argue rankings that take only one or two components into account are more useful than USNWR because then you truly are distilling the info and comparing apples with apples. Leiter rankings, for example, let you know explicitly what criteria the schools are being compared against. So if you are most interested in firm work, you could see a ranking of schools that place highly into top firms WITH THE KNOWLEDGE that the very top schools are at a disadvantage in such a system due to the wide variety of employment opportunities they offer their students, sending grads to clerkships, academia, etc.

As a side note, my flamedar may be really off, but I think people call flame way too easily.

If you are interested in firm work, you should probably pay more attention to a ranking system that shows how easily it is to get biglaw from a particular school and not necessarily how much head to biglaw from a particular school, e.g. do we really think it's easier to get biglaw from Cornell than Yale just because Cornell has outplaced Yale in the NLJ250? If CanadianWolf wants these rankings to have an ounce of relevance, it seems like the least he could do is include clerkship and academia numbers.

[CanadianWolf]

No break needed. I do not buy that any ranking placing Yale fifth is not reasonable. I am just using more verifiable & objective data than does USNews. Assuming Yale should be ranked in any position--whether high or low--is for peer assessment opinion which , unfortunately, is heavily represented in my combined rankings & excessively included in the USNews system.
All anyone has to do is to provide the clerkship & academia data to me in an objective & verifiable form. When that is done then decisions regarding weighting of the data may have to be made; for example, does a state trial judicial clerkship get the same value as a federal judicial clerkship or as a state supreme court clerkship? Does teaching constitutional law at a community college get the same value as teaching torts at Stanford? My rankings are not portrayed to be perfect, just better than USNews for my stated purpose.
I do agree, nevertheless, that any ranking of US law schools based solely on prestige would be questionable if Yale & Harvard were not included in the top three.

[beesknees]

If you are interested in firm work, you should probably pay more attention to a ranking system that shows how easily it is to get biglaw from a particular school and not necessarily how much head to biglaw from a particular school, e.g. do we really think it's easier to get biglaw from Cornell than Yale just because Cornell has outplaced Yale in the NLJ250? If CanadianWolf wants these rankings to have an ounce of relevance, it seems like the least he could do is include clerkship and academia numbers.

I certainly wouldn't argue, even in light of firm placement rankings that it would be easier from Cornell than Yale. Like I said, you have to maintain a level head and realize that top schools like Yale and Harvard won't be ranked as high because their grads have more employment options.

I do think a ranking system that ONLY includes employment outcomes, top firm placement, clerkship, and academia numbers would be especially useful since they are outcome oriented. I think reputational scores have too much of a tendency toward regional bias AND they are self-perpetuating and easily manipulated. I would argue that if USNWR changed HOW they collected reputational scores, outcomes could easily be changed. For example, if they simply ask people to grade schools on a 1 to 5 scale, which is how I believe they do it now, its very difficult, even if a school has been steadily improving its placement performance to change because its just asking someone about their perception of a school. But what if they included recent numbers for schools, like firm placement, academia and clerkships? That might lead to some changes in reputational scores because subjective rankings like that are easy to sway by presenting more or less information. Or if they only included information on the quality of the faculty at each school, that too might skew the results a different way.

And the problem with USNWR is that we simply do not know how it is collected and the sample it is collected from.

[CanadianWolf]

Exactly correct. Just as my first year civil procedure law professor wrote on my final exam, "you understand".
Simply stated: The problem with the USNews rating & ranking methodology is that the largest component of the ratings --the peer assessment scores--has absolutely no standards--let alone being objective or verifiable.

[Helmholtz]

I certainly wouldn't argue, even in light of firm placement rankings that it would be easier from Cornell than Yale. Like I said, you have to maintain a level head and realize that top schools like Yale and Harvard won't be ranked as high because their grads have more employment options.

I do think a ranking system that ONLY includes employment outcomes, top firm placement, clerkship, and academia numbers would be especially useful since they are outcome oriented. I think reputational scores have too much of a tendency toward regional bias AND they are self-perpetuating and easily manipulated. I would argue that if USNWR changed HOW they collected reputational scores, outcomes could easily be changed. For example, if they simply ask people to grade schools on a 1 to 5 scale, that would be one thing. But what if they included recent numbers for schools, like firm placement, academia and clerkships? That might lead to some changes in reputational scores because subjective rankings like that are easy to sway by presenting more or less information.

And the problem with USNWR is that we simply do not know how it is collected and the sample it is collected from.

Agree with a lot of that, but what do you think the purpose of these "CanadianWolf rankings" are? Maybe I'm just missing the point.

[vanwinkle]

Simply stated: The problem with the USNews rating & ranking methodology is that the largest component of the ratings --the peer assessment scores--has absolutely no standards--let alone being objective or verifiable.

But your rankings are still subjective, even if you attempt to base it on objective criteria, because you're making conscious selections regarding what data to include and what data to exclude. Trying to create a ranking system using methods that don't properly balance out self-selection away from BigLaw and weights BigLaw placement too heavily is a subjective decision that leads to flawed results.

Just because your rankings are "objective" or "verifiable" doesn't make them useful or accurate. I can verify the accuracy of your method all day, but as long as your method continues to be wrong, your rankings will continue to be disrespected by anyone informed enough to realize what's wrong with them.

[beesknees]

I certainly wouldn't argue, even in light of firm placement rankings that it would be easier from Cornell than Yale. Like I said, you have to maintain a level head and realize that top schools like Yale and Harvard won't be ranked as high because their grads have more employment options.

I do think a ranking system that ONLY includes employment outcomes, top firm placement, clerkship, and academia numbers would be especially useful since they are outcome oriented. I think reputational scores have too much of a tendency toward regional bias AND they are self-perpetuating and easily manipulated. I would argue that if USNWR changed HOW they collected reputational scores, outcomes could easily be changed. For example, if they simply ask people to grade schools on a 1 to 5 scale, that would be one thing. But what if they included recent numbers for schools, like firm placement, academia and clerkships? That might lead to some changes in reputational scores because subjective rankings like that are easy to sway by presenting more or less information.

And the problem with USNWR is that we simply do not know how it is collected and the sample it is collected from.

Agree with a lot of that, but what do you think the purpose of these "CanadianWolf rankings" are? Maybe I'm just missing the point.

Oh, well, I had no point other than to not rail against his work ONLY because it puts Yale 5th. So you didn't miss anything on that front

[Helmholtz]

I certainly wouldn't argue, even in light of firm placement rankings that it would be easier from Cornell than Yale. Like I said, you have to maintain a level head and realize that top schools like Yale and Harvard won't be ranked as high because their grads have more employment options.

I do think a ranking system that ONLY includes employment outcomes, top firm placement, clerkship, and academia numbers would be especially useful since they are outcome oriented. I think reputational scores have too much of a tendency toward regional bias AND they are self-perpetuating and easily manipulated. I would argue that if USNWR changed HOW they collected reputational scores, outcomes could easily be changed. For example, if they simply ask people to grade schools on a 1 to 5 scale, that would be one thing. But what if they included recent numbers for schools, like firm placement, academia and clerkships? That might lead to some changes in reputational scores because subjective rankings like that are easy to sway by presenting more or less information.

And the problem with USNWR is that we simply do not know how it is collected and the sample it is collected from.

Agree with a lot of that, but what do you think the purpose of these "CanadianWolf rankings" are? Maybe I'm just missing the point.

Oh, well, I had no point other than to not rail against his work ONLY because it puts Yale 5th. So you didn't miss anything on that front

Yale at 5th is an absurdity in any ranking that cumulates data in a supposed method to show which schools are better. If all CanadianWolf wanted to show us was which school send a lot of students to the NLJ250 firms, I don't understand why the entire OP is not just one link to that chart.

[TCScrutinizer]

Any ranking that doesn't include total number of library volumes is worthless.

No, no... any ranking that doesn't include total square footage of library and classroom space is worthless.

[Fancy Pants]

I am just using more verifiable & objective data than does USNews.

How exactly are your data more verifiable and objective than US News when your rankings includes the US News data? Your rankings suffer from the same verifiability problems as US News does.

[beesknees]

Yale at 5th is an absurdity in any ranking that cumulates data in a supposed method to show which schools are better. If all CanadianWolf wanted to show us was which school send a lot of students to the NLJ250 firms, I don't understand why the entire OP is not just one link to that chart.

Last point is credited.

I guess the whole idea of trying to make one ULTIMATE ranking system is a flawed idea in the first place. It makes for comparing extremes easy, but very unhelpful in trying to discern the relative values of schools in close tiers... which is the decision facing many TLSers.

[Blindmelon]

Yea... all this biglaw placement discussion, etc. usually ignores not only clerkships, but prestigious fed. gov work. Someone who got in the DOJ Honors Program prob could have gotten a big firm job if they.

Self selection makes all of this "objective" ranking irrelevant.

[CanadianWolf]

You all are correct, because you're critisizing based on a reason or reasons.(Remember this if you take a negotiations seminar).
But I also said for a stated purpose & data available to me. These ratings & rankings are not perfect, and not correct--or even useful--for all purposes. Of course, I decided what data to include--I included it all. everything from USNews & everything from the National Law Journal survey results. And I decided how much weight to assign each of the two components; I assigned 50% to the National Law Journal survey results & 50% to the USNews overall scores. This accorded USNews a heavier weighting in reality because both were on a point system but the USNews assigned 100 points to its top ranked law school-Yale- and the NLJ top law school-Northwestern- only received 50 points which was the maximum.
The overall weighting effect, therefore, is one third National Law Journal rankings & two thirds USNews ratings.

[Fancy Pants]

I decided what data to include--I included it all. everything from USNews

Which means your rankings suffer from the same verifiability problems as US News, and since you think that US News has verifiability problems, you think your own rankings have verifiability problems.

So uh, thanks for the awesome rankings.

[General Tso]

If that's what you want to believe, then it is fine by me.

Would you say that you know more or less about law school rankings than one of the most cited law professors at the University of Chicago?

edit: also, I am calling flame

I am calling appeal to authority

(..but I cited the same source in an argument against SBL the other day)

[General Tso]

Exactly correct. Just as my first year civil procedure law professor wrote on my final exam, "you understand".

lmao..

[Mr. Matlock]

You all are correct, because you're critisizing based on a reason or reasons.(Remember this if you take a negotiations seminar).
But I also said for a stated purpose & data available to me. These ratings & rankings are not perfect, and not correct--or even useful--for all purposes. Of course, I decided what data to include--I included it all. everything from USNews & everything from the National Law Journal survey results. And I decided how much weight to assign each of the two components; I assigned 50% to the National Law Journal survey results & 50% to the USNews overall scores. This accorded USNews a heavier weighting in reality because both were on a point system but the USNews assigned 100 points to its top ranked law school-Yale- and the NLJ top law school-Northwestern- only received 50 points which was the maximum.
The overall weighting effect, therefore, is one third National Law Journal rankings & two thirds USNews ratings.

When you get your next semester grades, I hope you remember this little adventure you set out upon. In the meantime, try to remember:

[CanadianWolf]

FancyPants: Yes & no. I recognize that a significant portion of the USNews rating & ranking system--the peer assessment scores--is unverifiable & not objective for their results but not for mine because I accepted the USNews results in total so as to avoid individual preference or bias. I chose not to attempt to correct that portion for several reasons (especially because it probably includes significant credit for judicial clerkship & academia placements), but adding in the NLJ results did give the USNews' heaviest factor--the opinion peer assessment scores--less influence on the results.
This is similiar to the dilemma faced by appellate courts that do not have authority to conduct a de novo review of the matter before them. Simply stated = you have to work with what you are given.
So thanks for the awesome comment.

[Fancy Pants]

FancyPants: Yes & no. I recognize that a significant portion of the USNews rating & ranking system--the peer assessment scores--is unverifiable & not objective for their results but not for mine because I accepted the USNews results in total so as to avoid individual preference or bias. I chose not to attempt to correct that portion for several reasons (especially because it probably includes significant credit for judicial clerkship & academia placements), but adding in the NLJ results did give the USNews' heaviest factor--the opinion peer assessment scores--less influence on the results.
This is similiar to the dilemma faced by appellate courts that do not have authority to conduct a de novo review of the matter before them. Simply stated = you have to work with what you are given.
So thanks for the awesome comment.

No, you don't have to work with what you have. If the data are unverifiable as you claim, then your rankings which use them are just as unverifiable. You can't magically make that problem go away.

Also, correcting bad data is not "individual preference or bias" unless you are now saying that it is only your opinion that the data are bad. In that case, your position now is that it is your opinion that your rankings are bad. Kudos.

[CanadianWolf]

Another totally awesome comment that deserves a totally awesome response.
Please note that I have repeatedly written that these rankings are not perfect--probably not even awesome--just better for my stated purpose.

[insidethetwenty]

That would be the equivalent of assuming that the internet & personal computers are simply a passing fancy.
Your suggestions are certainly reasonable as footnotes if not made crystal clear by the stated purpose of the rankings.

Wait, what? I didn't say that law school rankings are a fad or that they won't be used in the future. What I said was akin to "computer rankings should be taken with a grain of salt, because a killer graphics card isn't important to you if you have no desire to watch HD movies/play HD games on your computer".

So, yeah...

[D. H2Oman]

[strike]CanadianWolf[/strike]

[Dignan]

Another totally awesome comment that deserves a totally awesome response.
Please note that I have repeatedly written that these rankings are not perfect--probably not even awesome--just better for my stated purpose.

The problem is not that your rankings are imperfect. The problem is that your rankings are useless, and people are pointing that out.

If I were to add 2009 academia placement to the USNWR rankings (see --LinkRemoved--), and then adjust it for class size, the resulting "rankings" would have Berkeley ahead of NYU and Chicago, Georgetown ahead of Virginia, and Texas ahead of Northwestern. But what would it prove? What's the point in taking one variable in isolation and then adding it to a comprehensive ranking system?