Why you should completely ignore Vault rankings

(On Campus Interviews, Summer Associate positions, Firm Reviews, Tips, ...)
Forum rules
Anonymous Posting

Anonymous posting is only appropriate when you are revealing sensitive employment related information about a firm, job, etc. You may anonymously respond on topic to these threads. Unacceptable uses include: harassing another user, joking around, testing the feature, or other things that are more appropriate in the lounge.

Failure to follow these rules will get you outed, warned, or banned.
User avatar
quakeroats
Posts: 1399
Joined: Mon Oct 26, 2009 8:34 am

Why you should completely ignore Vault rankings

Postby quakeroats » Tue Sep 13, 2011 11:35 pm

How does Vault come up with its list of the Top 100 firms in the country? The first step is to compile a list of the most renowned law firms in the land by reviewing the feedback we receive from previous surveys, consulting our previous lists, poring over legal newspapers, talking to lawyers in the field and checking out other published rankings. This year, our list was made up of more than 300 law firms. We asked these 300+ law firms to distribute a password-protected online survey to their associates. Nearly 16,000 attorneys returned anonymous surveys to Vault. Associates from all over the country and the world responded. We heard from lawyers in New York, Los Angeles, San Francisco, Palo Alto, Chicago, Boston, Philadelphia, Houston, Dallas, Washington, D.C., Miami, Cleveland, Seattle, Orlando, Phoenix and Atlanta, among many other domestic locations, not to mention Hong Kong, London, Paris and beyond. The online survey asked attorneys to score each of the 300+ law firms on a scale of 1 to 10 based on how prestigious it is to work for the firm. Associates were asked to ignore any firm with which they were unfamiliar and were not allowed to rank their own firm.

We collected all the surveys and averaged the score for each firm. The firms were then ranked in order, starting with the highest average prestige score as No. 1 on down to determine the Vault Top 100. Remember that in the Top 100, Vault is not assessing firms by profit, size, lifestyle, number of deals or quality of service; we are ranking the most prestigious law firms based on the perceptions of currently practicing lawyers [sic] at peer firms.

http://www.vault.com/wps/portal/usa/ran ... rankings=1

Transferthrowaway
Posts: 608
Joined: Sat Jan 22, 2011 9:40 am

Re: Why you should completely ignore Vault rankings

Postby Transferthrowaway » Tue Sep 13, 2011 11:36 pm

cool story, bro.

User avatar
Lieut Kaffee
Posts: 774
Joined: Wed Nov 11, 2009 2:01 am

Re: Why you should completely ignore Vault rankings

Postby Lieut Kaffee » Tue Sep 13, 2011 11:38 pm

Yes, we know the methodology, and yes, Vault should be taken with a huge huge grain of salt. Still, I think in a lot of cases if you ask a practitioner/judge/prof/fellow student/human being "Which firm is better? Firm X or Firm Y" you will find that the responses comport with the Vault rankings a fair amount of the time.

User avatar
rayiner
Posts: 6184
Joined: Thu Dec 11, 2008 11:43 am

Re: Why you should completely ignore Vault rankings

Postby rayiner » Tue Sep 13, 2011 11:39 pm

So basically how USNWR is done?

dixiecupdrinking
Posts: 3142
Joined: Sun Oct 26, 2008 2:39 pm

Re: Why you should completely ignore Vault rankings

Postby dixiecupdrinking » Tue Sep 13, 2011 11:40 pm

In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.

dixiecupdrinking
Posts: 3142
Joined: Sun Oct 26, 2008 2:39 pm

Re: Why you should completely ignore Vault rankings

Postby dixiecupdrinking » Tue Sep 13, 2011 11:42 pm

And anyway, if you can't impress other law students and 25-35 year-old attorneys with your V25 firm, what else is there?

User avatar
Grizz
Posts: 10583
Joined: Mon Jan 04, 2010 6:31 pm

Re: Why you should completely ignore Vault rankings

Postby Grizz » Tue Sep 13, 2011 11:44 pm

Transferthrowaway wrote:cool story, bro.

User avatar
rayiner
Posts: 6184
Joined: Thu Dec 11, 2008 11:43 am

Re: Why you should completely ignore Vault rankings

Postby rayiner » Tue Sep 13, 2011 11:45 pm

dixiecupdrinking wrote:In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.


I think one of the things people forget is that there is a distinction between what is relevant to associates and what is relevant to partners/in-house counsel. I fail to see why what current associates (i.e.: people who will be in-house and evaluating you for jobs eventually) think of a firm is less relevant to an associate than firm profits, the presence of particular super-star partners, quality of service, or other things measured by Chambers.

dixiecupdrinking
Posts: 3142
Joined: Sun Oct 26, 2008 2:39 pm

Re: Why you should completely ignore Vault rankings

Postby dixiecupdrinking » Tue Sep 13, 2011 11:49 pm

rayiner wrote:
dixiecupdrinking wrote:In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.


I think one of the things people forget is that there is a distinction between what is relevant to associates and what is relevant to partners/in-house counsel. I fail to see why what current associates (i.e.: people who will be in-house and evaluating you for jobs eventually) think of a firm is less relevant to an associate than firm profits, the presence of particular super-star partners, quality of service, or other things measured by Chambers.

Yup. Exactly.

Having said that, one might justifiably be frustrated with the threads on here that go something like, "I have two offers, one is in the city I want to live in forever, is full of great people, and does exactly the kind of work I want to do, but it's only a V50 and the other is a V25. What should I do?!"

User avatar
quakeroats
Posts: 1399
Joined: Mon Oct 26, 2009 8:34 am

Re: Why you should completely ignore Vault rankings

Postby quakeroats » Tue Sep 13, 2011 11:54 pm

rayiner wrote:So basically how USNWR is done?


It's even worse. Off the top of my head: there's only one datapoint. Responses are self-selected. Data collection isn't controlled. Only associates are surveyed. Firms control who can respond. We don't know what, if any, controls are in place to stop firms from gaming the rankings (we do know that law students take the V100 seriously which gives firms an incentive). Prestige isn't defined. The scale isn't tied to anything. Associates are merely asked to ignore firms they don't know. The ranking is simply an accounting of highest score to lowest with no effort to account for minor differences in score (e.g., scores 7.550, 7.551 ... 7.559 could be 10 different ranks).

mrloblaw
Posts: 534
Joined: Fri Jul 22, 2011 3:00 pm

Re: Why you should completely ignore Vault rankings

Postby mrloblaw » Tue Sep 13, 2011 11:55 pm

dixiecupdrinking wrote:
rayiner wrote:
dixiecupdrinking wrote:In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.


I think one of the things people forget is that there is a distinction between what is relevant to associates and what is relevant to partners/in-house counsel. I fail to see why what current associates (i.e.: people who will be in-house and evaluating you for jobs eventually) think of a firm is less relevant to an associate than firm profits, the presence of particular super-star partners, quality of service, or other things measured by Chambers.

Yup. Exactly.

Having said that, one might justifiably be frustrated with the threads on here that go something like, "I have two offers, one is in the city I want to live in forever, is full of great people, and does exactly the kind of work I want to do, but it's only a V50 and the other is a V25. What should I do?!"


Not frustrated so much as perplexed as to how those people get offers in the first place.

mrloblaw
Posts: 534
Joined: Fri Jul 22, 2011 3:00 pm

Re: Why you should completely ignore Vault rankings

Postby mrloblaw » Tue Sep 13, 2011 11:57 pm

quakeroats wrote:
rayiner wrote:So basically how USNWR is done?


It's even worse. Off the top of my head: there's only one datapoint. Responses are self-selected. Data collection isn't controlled. Only associates are surveyed. Firms control who can respond. We don't know what, if any, controls are in place to stop firms from gaming the rankings (we do know that law students take the V100 seriously which gives firms an incentive). Prestige isn't defined. The scale isn't tied to anything. Associates are merely asked to ignore firms they don't know. The ranking is simply an accounting of highest score to lowest with no effort to account for minor differences in score (e.g., scores 7.550, 7.551 ... 7.559 could be 10 different ranks).


I'm guessing that Herb Wachtell has more important concerns than gaming Vault.

User avatar
quakeroats
Posts: 1399
Joined: Mon Oct 26, 2009 8:34 am

Re: Why you should completely ignore Vault rankings

Postby quakeroats » Tue Sep 13, 2011 11:58 pm

mrloblaw wrote:
quakeroats wrote:
rayiner wrote:So basically how USNWR is done?


It's even worse. Off the top of my head: there's only one datapoint. Responses are self-selected. Data collection isn't controlled. Only associates are surveyed. Firms control who can respond. We don't know what, if any, controls are in place to stop firms from gaming the rankings (we do know that law students take the V100 seriously which gives firms an incentive). Prestige isn't defined. The scale isn't tied to anything. Associates are merely asked to ignore firms they don't know. The ranking is simply an accounting of highest score to lowest with no effort to account for minor differences in score (e.g., scores 7.550, 7.551 ... 7.559 could be 10 different ranks).


I'm guessing that Herb Wachtell has more important concerns than gaming Vault.


I suspect he can do more than one thing at a time. Even if you're right, the same can't be said for everyone else.
Last edited by quakeroats on Tue Sep 13, 2011 11:59 pm, edited 1 time in total.

User avatar
Lieut Kaffee
Posts: 774
Joined: Wed Nov 11, 2009 2:01 am

Re: Why you should completely ignore Vault rankings

Postby Lieut Kaffee » Tue Sep 13, 2011 11:59 pm

mrloblaw wrote:
quakeroats wrote:
rayiner wrote:So basically how USNWR is done?


It's even worse. Off the top of my head: there's only one datapoint. Responses are self-selected. Data collection isn't controlled. Only associates are surveyed. Firms control who can respond. We don't know what, if any, controls are in place to stop firms from gaming the rankings (we do know that law students take the V100 seriously which gives firms an incentive). Prestige isn't defined. The scale isn't tied to anything. Associates are merely asked to ignore firms they don't know. The ranking is simply an accounting of highest score to lowest with no effort to account for minor differences in score (e.g., scores 7.550, 7.551 ... 7.559 could be 10 different ranks).


I'm guessing that Herb Wachtell has more important concerns than gaming Vault.

You say this tongue in cheek, but the second-most important thing firms do is recruit talent. Kinda like how you'd think top-flight educators and deans would have more important things to do than game USN.

User avatar
GeePee
Posts: 1273
Joined: Fri Jul 03, 2009 7:35 pm

Re: Why you should completely ignore Vault rankings

Postby GeePee » Wed Sep 14, 2011 10:42 am

rayiner wrote:
dixiecupdrinking wrote:In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.


I think one of the things people forget is that there is a distinction between what is relevant to associates and what is relevant to partners/in-house counsel. I fail to see why what current associates (i.e.: people who will be in-house and evaluating you for jobs eventually) think of a firm is less relevant to an associate than firm profits, the presence of particular super-star partners, quality of service, or other things measured by Chambers.

I sort of agree with this, but I think that Vault skewed heavily toward transactional work and litigators should keep this in mind when choosing firms.

User avatar
quakeroats
Posts: 1399
Joined: Mon Oct 26, 2009 8:34 am

Re: Why you should completely ignore Vault rankings

Postby quakeroats » Wed Sep 14, 2011 11:30 am

rayiner wrote:
dixiecupdrinking wrote:In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.


I think one of the things people forget is that there is a distinction between what is relevant to associates and what is relevant to partners/in-house counsel. I fail to see why what current associates (i.e.: people who will be in-house and evaluating you for jobs eventually) think of a firm is less relevant to an associate than firm profits, the presence of particular super-star partners, quality of service, or other things measured by Chambers.


That they may be in house one day isn't particularly compelling. They aren't in house at the moment and may never be. Even if they are at some point, they're several steps removed from the real decision making. If we accept your point, why not poll college students or 1Ls along the same line?

User avatar
Blindmelon
Posts: 1708
Joined: Thu Mar 26, 2009 11:13 am

Re: Why you should completely ignore Vault rankings

Postby Blindmelon » Wed Sep 14, 2011 11:34 am

GeePee wrote:
rayiner wrote:
dixiecupdrinking wrote:In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.


I think one of the things people forget is that there is a distinction between what is relevant to associates and what is relevant to partners/in-house counsel. I fail to see why what current associates (i.e.: people who will be in-house and evaluating you for jobs eventually) think of a firm is less relevant to an associate than firm profits, the presence of particular super-star partners, quality of service, or other things measured by Chambers.

I sort of agree with this, but I think that Vault skewed heavily toward transactional work and litigators should keep this in mind when choosing firms.


Also heavily scewed towards NYC.

User avatar
quakeroats
Posts: 1399
Joined: Mon Oct 26, 2009 8:34 am

Re: Why you should completely ignore Vault rankings

Postby quakeroats » Wed Sep 14, 2011 11:39 am

GeePee wrote:
rayiner wrote:
dixiecupdrinking wrote:In other words, Vault 100 = TLS for grown-ups. Total echo chamber.

Isn't there at least some merit to the rankings, though, in that for better or worse the profession can be prestige-obsessed and what other people think of your law firm may matter for future opportunities? Not to say the V100 is a perfect or even good measure of a firm's quality, but it could be good at measuring the limited thing it is actually measuring – what other young attorneys in our generation will tend to think when they see a particular firm name on a resume.


I think one of the things people forget is that there is a distinction between what is relevant to associates and what is relevant to partners/in-house counsel. I fail to see why what current associates (i.e.: people who will be in-house and evaluating you for jobs eventually) think of a firm is less relevant to an associate than firm profits, the presence of particular super-star partners, quality of service, or other things measured by Chambers.

I sort of agree with this, but I think that Vault skewed heavily toward transactional work and litigators should keep this in mind when choosing firms.


That's our read on the data after the fact, but why do you suppose that is? Are there other glaring issues that might cause problems? Do you suppose there's a chance there are non-obvious issues one might not see that could negatively affect a decision?

User avatar
Old Gregg
Posts: 5413
Joined: Thu Sep 01, 2011 1:26 pm

Re: Why you should completely ignore Vault rankings

Postby Old Gregg » Wed Sep 14, 2011 11:43 am

Eh, Boies, Quinn, and W&C have been rising surely but steadily through the rankings. Wouldn't say it's too skewed toward M&A firms nowadays.

User avatar
BioEBear2010
Posts: 745
Joined: Wed Nov 19, 2008 7:05 pm

Re: Why you should completely ignore Vault rankings

Postby BioEBear2010 » Wed Sep 14, 2011 11:52 am

For litigation, the problem is that Vault "undervalues" some non-national, boutique-ish firms which are wonderful to work for. Munger Tolles and Irell & Manella, to take two examples, are both small, highly respected west coast firms which many students pick over V10 firms. And Vault doesn't even rank tiny rockstar firms like Howard Rice, Keker & Van Nest, Susman Godfrey, Bartlit Beck, etc.

That being said, I think Vault rankings can and should be a part of most peoples' firm choice calculus.
Last edited by BioEBear2010 on Wed Sep 14, 2011 11:55 am, edited 1 time in total.

User avatar
Old Gregg
Posts: 5413
Joined: Thu Sep 01, 2011 1:26 pm

Re: Why you should completely ignore Vault rankings

Postby Old Gregg » Wed Sep 14, 2011 11:55 am

BioEBear2010 wrote:For litigation, the problem is that Vault "undervalues" some non-national, boutique-ish firms which are wonderful to work for. Munger Tolles and Irell & Manella, to take two examples, are both small, highly respected west coast firms which many students pick over V10 firms. And Vault doesn't even rank tiny rockstar firms like Howard Rice, Keker & Van Nest, Susman Godfrey, Bartlit Beck, etc.


Kind of weird "Howard Rice" trolling.

That aside, Vault is ranking national prestige. To that extent, the exclusion and underrating of the above firms is logical.

Anonymous User
Posts: 273406
Joined: Tue Aug 11, 2009 9:32 am

Re: Why you should completely ignore Vault rankings

Postby Anonymous User » Wed Sep 14, 2011 11:55 am

Fresh Prince wrote:
BioEBear2010 wrote:For litigation, the problem is that Vault "undervalues" some non-national, boutique-ish firms which are wonderful to work for. Munger Tolles and Irell & Manella, to take two examples, are both small, highly respected west coast firms which many students pick over V10 firms. And Vault doesn't even rank tiny rockstar firms like Howard Rice, Keker & Van Nest, Susman Godfrey, Bartlit Beck, etc.


Kind of weird "Howard Rice" trolling.

That aside, Vault is ranking national prestige. To that extent, the exclusion and underrating of the above firms is logical.


Is Howard Rice not on the same level as those others?

User avatar
BioEBear2010
Posts: 745
Joined: Wed Nov 19, 2008 7:05 pm

Re: Why you should completely ignore Vault rankings

Postby BioEBear2010 » Wed Sep 14, 2011 11:56 am

Fresh Prince wrote:
BioEBear2010 wrote:For litigation, the problem is that Vault "undervalues" some non-national, boutique-ish firms which are wonderful to work for. Munger Tolles and Irell & Manella, to take two examples, are both small, highly respected west coast firms which many students pick over V10 firms. And Vault doesn't even rank tiny rockstar firms like Howard Rice, Keker & Van Nest, Susman Godfrey, Bartlit Beck, etc.


Kind of weird "Howard Rice" trolling.

That aside, Vault is ranking national prestige. To that extent, the exclusion and underrating of the above firms is logical.

Please forgive my Bay Area bias.

User avatar
Old Gregg
Posts: 5413
Joined: Thu Sep 01, 2011 1:26 pm

Re: Why you should completely ignore Vault rankings

Postby Old Gregg » Wed Sep 14, 2011 11:57 am

Anonymous User wrote:
Fresh Prince wrote:
BioEBear2010 wrote:For litigation, the problem is that Vault "undervalues" some non-national, boutique-ish firms which are wonderful to work for. Munger Tolles and Irell & Manella, to take two examples, are both small, highly respected west coast firms which many students pick over V10 firms. And Vault doesn't even rank tiny rockstar firms like Howard Rice, Keker & Van Nest, Susman Godfrey, Bartlit Beck, etc.


Kind of weird "Howard Rice" trolling.

That aside, Vault is ranking national prestige. To that extent, the exclusion and underrating of the above firms is logical.


Is Howard Rice not on the same level as those others?


Not even close, buddy.

Anonymous User
Posts: 273406
Joined: Tue Aug 11, 2009 9:32 am

Re: Why you should completely ignore Vault rankings

Postby Anonymous User » Wed Sep 14, 2011 11:58 am

Fresh Prince wrote:
Anonymous User wrote:
Fresh Prince wrote:
BioEBear2010 wrote:For litigation, the problem is that Vault "undervalues" some non-national, boutique-ish firms which are wonderful to work for. Munger Tolles and Irell & Manella, to take two examples, are both small, highly respected west coast firms which many students pick over V10 firms. And Vault doesn't even rank tiny rockstar firms like Howard Rice, Keker & Van Nest, Susman Godfrey, Bartlit Beck, etc.


Kind of weird "Howard Rice" trolling.

That aside, Vault is ranking national prestige. To that extent, the exclusion and underrating of the above firms is logical.


Is Howard Rice not on the same level as those others?


Not even close, buddy.


Good to know :). Thx.

Edit: you sure?

"We know that focusing on excellence works. Our clients are among the largest and most sophisticated consumers of legal services. They include The Charles Schwab Corporation, Citigroup Global Markets Inc., Digital River, Inc., Genentech, Inc., Hewlett-Packard Co., Morgan Stanley, Pacific Gas & Electric Company, PalmSource, Inc. (now named ACCESS Systems Americas, Inc.), Prudential Securities, Inc., Sony Online Entertainment, Technology Crossover Ventures, Thomas Weisel Partners LLC, United States Olympic Committee and Wells Fargo & Company. These companies selected our firm because of our dedication to their needs, their reliance on our excellent judgment, and our consistently outstanding results. Over the years, we have handled their large and complex litigation, corporate counseling and transactional work, tax matters and precedent-setting bankruptcies intelligently, effectively and successfully.

We are known as the "lawyers' lawyers." Many of the largest firms in California regularly call upon us to handle delicate issues of conflicts of interest, disqualification, malpractice or dissolution. For the same reasons that our largest clients choose us, these law firms, which are among the most incisive purchasers of legal services, select our firm when their own interests and reputations are at stake."
--LinkRemoved--




Return to “Legal Employment”

Who is online

The online users are hidden on this forum.