Why you should completely ignore Vault rankings Forum

[quakeroats]

How does Vault come up with its list of the Top 100 firms in the country? The first step is to compile a list of the most renowned law firms in the land by reviewing the feedback we receive from previous surveys, consulting our previous lists, poring over legal newspapers, talking to lawyers in the field and checking out other published rankings. This year, our list was made up of more than 300 law firms. We asked these 300+ law firms to distribute a password-protected online survey to their associates. Nearly 16,000 attorneys returned anonymous surveys to Vault. Associates from all over the country and the world responded. We heard from lawyers in New York, Los Angeles, San Francisco, Palo Alto, Chicago, Boston, Philadelphia, Houston, Dallas, Washington, D.C., Miami, Cleveland, Seattle, Orlando, Phoenix and Atlanta, among many other domestic locations, not to mention Hong Kong, London, Paris and beyond. The online survey asked attorneys to score each of the 300+ law firms on a scale of 1 to 10 based on how prestigious it is to work for the firm. Associates were asked to ignore any firm with which they were unfamiliar and were not allowed to rank their own firm.

We collected all the surveys and averaged the score for each firm. The firms were then ranked in order, starting with the highest average prestige score as No. 1 on down to determine the Vault Top 100. Remember that in the Top 100, Vault is not assessing firms by profit, size, lifestyle, number of deals or quality of service; we are ranking the most prestigious law firms based on the perceptions of currently practicing lawyers [sic] at peer firms.

http://www.vault.com/wps/portal/usa/ran ... rankings=1

[Transferthrowaway]

cool story, bro.

[Lieut Kaffee]

Yes, we know the methodology, and yes, Vault should be taken with a huge huge grain of salt. Still, I think in a lot of cases if you ask a practitioner/judge/prof/fellow student/human being "Which firm is better? Firm X or Firm Y" you will find that the responses comport with the Vault rankings a fair amount of the time.

[rayiner]

So basically how USNWR is done?

[dixiecupdrinking]

In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.

[dixiecupdrinking]

And anyway, if you can't impress other law students and 25-35 year-old attorneys with your V25 firm, what else is there?

[Grizz]

cool story, bro.

[rayiner]

In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.

I think one of the things people forget is that there is a distinction between what is relevant to associates and what is relevant to partners/in-house counsel. I fail to see why what current associates (i.e.: people who will be in-house and evaluating you for jobs eventually) think of a firm is less relevant to an associate than firm profits, the presence of particular super-star partners, quality of service, or other things measured by Chambers.

[dixiecupdrinking]

In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.

I think one of the things people forget is that there is a distinction between what is relevant to associates and what is relevant to partners/in-house counsel. I fail to see why what current associates (i.e.: people who will be in-house and evaluating you for jobs eventually) think of a firm is less relevant to an associate than firm profits, the presence of particular super-star partners, quality of service, or other things measured by Chambers.

Yup. Exactly.

Having said that, one might justifiably be frustrated with the threads on here that go something like, "I have two offers, one is in the city I want to live in forever, is full of great people, and does exactly the kind of work I want to do, but it's only a V50 and the other is a V25. What should I do?!"

[quakeroats]

So basically how USNWR is done?

It's even worse. Off the top of my head: there's only one datapoint. Responses are self-selected. Data collection isn't controlled. Only associates are surveyed. Firms control who can respond. We don't know what, if any, controls are in place to stop firms from gaming the rankings (we do know that law students take the V100 seriously which gives firms an incentive). Prestige isn't defined. The scale isn't tied to anything. Associates are merely asked to ignore firms they don't know. The ranking is simply an accounting of highest score to lowest with no effort to account for minor differences in score (e.g., scores 7.550, 7.551 ... 7.559 could be 10 different ranks).

[mrloblaw]

In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.

I think one of the things people forget is that there is a distinction between what is relevant to associates and what is relevant to partners/in-house counsel. I fail to see why what current associates (i.e.: people who will be in-house and evaluating you for jobs eventually) think of a firm is less relevant to an associate than firm profits, the presence of particular super-star partners, quality of service, or other things measured by Chambers.

Yup. Exactly.

Having said that, one might justifiably be frustrated with the threads on here that go something like, "I have two offers, one is in the city I want to live in forever, is full of great people, and does exactly the kind of work I want to do, but it's only a V50 and the other is a V25. What should I do?!"

Not frustrated so much as perplexed as to how those people get offers in the first place.

[mrloblaw]

So basically how USNWR is done?

It's even worse. Off the top of my head: there's only one datapoint. Responses are self-selected. Data collection isn't controlled. Only associates are surveyed. Firms control who can respond. We don't know what, if any, controls are in place to stop firms from gaming the rankings (we do know that law students take the V100 seriously which gives firms an incentive). Prestige isn't defined. The scale isn't tied to anything. Associates are merely asked to ignore firms they don't know. The ranking is simply an accounting of highest score to lowest with no effort to account for minor differences in score (e.g., scores 7.550, 7.551 ... 7.559 could be 10 different ranks).

I'm guessing that Herb Wachtell has more important concerns than gaming Vault.

[quakeroats]

So basically how USNWR is done?

It's even worse. Off the top of my head: there's only one datapoint. Responses are self-selected. Data collection isn't controlled. Only associates are surveyed. Firms control who can respond. We don't know what, if any, controls are in place to stop firms from gaming the rankings (we do know that law students take the V100 seriously which gives firms an incentive). Prestige isn't defined. The scale isn't tied to anything. Associates are merely asked to ignore firms they don't know. The ranking is simply an accounting of highest score to lowest with no effort to account for minor differences in score (e.g., scores 7.550, 7.551 ... 7.559 could be 10 different ranks).

I'm guessing that Herb Wachtell has more important concerns than gaming Vault.

I suspect he can do more than one thing at a time. Even if you're right, the same can't be said for everyone else.

[Lieut Kaffee]

So basically how USNWR is done?

It's even worse. Off the top of my head: there's only one datapoint. Responses are self-selected. Data collection isn't controlled. Only associates are surveyed. Firms control who can respond. We don't know what, if any, controls are in place to stop firms from gaming the rankings (we do know that law students take the V100 seriously which gives firms an incentive). Prestige isn't defined. The scale isn't tied to anything. Associates are merely asked to ignore firms they don't know. The ranking is simply an accounting of highest score to lowest with no effort to account for minor differences in score (e.g., scores 7.550, 7.551 ... 7.559 could be 10 different ranks).

I'm guessing that Herb Wachtell has more important concerns than gaming Vault.

You say this tongue in cheek, but the second-most important thing firms do is recruit talent. Kinda like how you'd think top-flight educators and deans would have more important things to do than game USN.

[GeePee]

In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.

I think one of the things people forget is that there is a distinction between what is relevant to associates and what is relevant to partners/in-house counsel. I fail to see why what current associates (i.e.: people who will be in-house and evaluating you for jobs eventually) think of a firm is less relevant to an associate than firm profits, the presence of particular super-star partners, quality of service, or other things measured by Chambers.

I sort of agree with this, but I think that Vault skewed heavily toward transactional work and litigators should keep this in mind when choosing firms.

[quakeroats]

In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.

I think one of the things people forget is that there is a distinction between what is relevant to associates and what is relevant to partners/in-house counsel. I fail to see why what current associates (i.e.: people who will be in-house and evaluating you for jobs eventually) think of a firm is less relevant to an associate than firm profits, the presence of particular super-star partners, quality of service, or other things measured by Chambers.

That they may be in house one day isn't particularly compelling. They aren't in house at the moment and may never be. Even if they are at some point, they're several steps removed from the real decision making. If we accept your point, why not poll college students or 1Ls along the same line?

[Blindmelon]

In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.

I think one of the things people forget is that there is a distinction between what is relevant to associates and what is relevant to partners/in-house counsel. I fail to see why what current associates (i.e.: people who will be in-house and evaluating you for jobs eventually) think of a firm is less relevant to an associate than firm profits, the presence of particular super-star partners, quality of service, or other things measured by Chambers.

I sort of agree with this, but I think that Vault skewed heavily toward transactional work and litigators should keep this in mind when choosing firms.

Also heavily scewed towards NYC.

[quakeroats]

In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.

I think one of the things people forget is that there is a distinction between what is relevant to associates and what is relevant to partners/in-house counsel. I fail to see why what current associates (i.e.: people who will be in-house and evaluating you for jobs eventually) think of a firm is less relevant to an associate than firm profits, the presence of particular super-star partners, quality of service, or other things measured by Chambers.

I sort of agree with this, but I think that Vault skewed heavily toward transactional work and litigators should keep this in mind when choosing firms.

That's our read on the data after the fact, but why do you suppose that is? Are there other glaring issues that might cause problems? Do you suppose there's a chance there are non-obvious issues one might not see that could negatively affect a decision?

[Old Gregg]

Eh, Boies, Quinn, and W&C have been rising surely but steadily through the rankings. Wouldn't say it's too skewed toward M&A firms nowadays.

[BioEBear2010]

For litigation, the problem is that Vault "undervalues" some non-national, boutique-ish firms which are wonderful to work for. Munger Tolles and Irell & Manella, to take two examples, are both small, highly respected west coast firms which many students pick over V10 firms. And Vault doesn't even rank tiny rockstar firms like Howard Rice, Keker & Van Nest, Susman Godfrey, Bartlit Beck, etc.

That being said, I think Vault rankings can and should be a part of most peoples' firm choice calculus.

[Old Gregg]

For litigation, the problem is that Vault "undervalues" some non-national, boutique-ish firms which are wonderful to work for. Munger Tolles and Irell & Manella, to take two examples, are both small, highly respected west coast firms which many students pick over V10 firms. And Vault doesn't even rank tiny rockstar firms like Howard Rice, Keker & Van Nest, Susman Godfrey, Bartlit Beck, etc.

Kind of weird "Howard Rice" trolling.

That aside, Vault is ranking national prestige. To that extent, the exclusion and underrating of the above firms is logical.

[Anonymous User]

For litigation, the problem is that Vault "undervalues" some non-national, boutique-ish firms which are wonderful to work for. Munger Tolles and Irell & Manella, to take two examples, are both small, highly respected west coast firms which many students pick over V10 firms. And Vault doesn't even rank tiny rockstar firms like Howard Rice, Keker & Van Nest, Susman Godfrey, Bartlit Beck, etc.

Kind of weird "Howard Rice" trolling.

That aside, Vault is ranking national prestige. To that extent, the exclusion and underrating of the above firms is logical.

Is Howard Rice not on the same level as those others?

[BioEBear2010]

For litigation, the problem is that Vault "undervalues" some non-national, boutique-ish firms which are wonderful to work for. Munger Tolles and Irell & Manella, to take two examples, are both small, highly respected west coast firms which many students pick over V10 firms. And Vault doesn't even rank tiny rockstar firms like Howard Rice, Keker & Van Nest, Susman Godfrey, Bartlit Beck, etc.

Kind of weird "Howard Rice" trolling.

That aside, Vault is ranking national prestige. To that extent, the exclusion and underrating of the above firms is logical.

Please forgive my Bay Area bias.

[Old Gregg]

For litigation, the problem is that Vault "undervalues" some non-national, boutique-ish firms which are wonderful to work for. Munger Tolles and Irell & Manella, to take two examples, are both small, highly respected west coast firms which many students pick over V10 firms. And Vault doesn't even rank tiny rockstar firms like Howard Rice, Keker & Van Nest, Susman Godfrey, Bartlit Beck, etc.

Kind of weird "Howard Rice" trolling.

That aside, Vault is ranking national prestige. To that extent, the exclusion and underrating of the above firms is logical.

Is Howard Rice not on the same level as those others?

Not even close, buddy.

[Anonymous User]

For litigation, the problem is that Vault "undervalues" some non-national, boutique-ish firms which are wonderful to work for. Munger Tolles and Irell & Manella, to take two examples, are both small, highly respected west coast firms which many students pick over V10 firms. And Vault doesn't even rank tiny rockstar firms like Howard Rice, Keker & Van Nest, Susman Godfrey, Bartlit Beck, etc.

Kind of weird "Howard Rice" trolling.

That aside, Vault is ranking national prestige. To that extent, the exclusion and underrating of the above firms is logical.

Is Howard Rice not on the same level as those others?

Not even close, buddy.

Good to know . Thx.

Edit: you sure?

"We know that focusing on excellence works. Our clients are among the largest and most sophisticated consumers of legal services. They include The Charles Schwab Corporation, Citigroup Global Markets Inc., Digital River, Inc., Genentech, Inc., Hewlett-Packard Co., Morgan Stanley, Pacific Gas & Electric Company, PalmSource, Inc. (now named ACCESS Systems Americas, Inc.), Prudential Securities, Inc., Sony Online Entertainment, Technology Crossover Ventures, Thomas Weisel Partners LLC, United States Olympic Committee and Wells Fargo & Company. These companies selected our firm because of our dedication to their needs, their reliance on our excellent judgment, and our consistently outstanding results. Over the years, we have handled their large and complex litigation, corporate counseling and transactional work, tax matters and precedent-setting bankruptcies intelligently, effectively and successfully.

We are known as the "lawyers' lawyers." Many of the largest firms in California regularly call upon us to handle delicate issues of conflicts of interest, disqualification, malpractice or dissolution. For the same reasons that our largest clients choose us, these law firms, which are among the most incisive purchasers of legal services, select our firm when their own interests and reputations are at stake."
--LinkRemoved--