The Accuracy of the T6's Self-Reported Big Law Employment #s Forum

[Bronck]

If we're using the deferral argument for 2009, can't we at least apply the same argument to a portion of the 2010 discrepancy?

I still find it odd that there would be a correlation between large class size and cooking of data.

[timbs4339]

I have data from CLS c/o 2011 EIP. It shows that 13 students accepted SA offers from Magic Circle/international firms.

Wachtell, Boies, Munger, Irell are all NLJ 250 according to this chart

http://pdfserver.amlaw.com/nlj/NLJ250_Poster2011.pdf

If I can find a list of firms not NLJ 250 but still attended EIP I can compare them to my list to see who accepted offers.

As a CLS student I would not put this past the administration (fudging numbers OR trying to hide the discrepancy from the Post.) The fact they haven't explained is very troubling.

[birdlaw117]

Really, this just stems from trying to get two metrics that measure different things to be equal, and being really concerned when they aren't. I don't see why everyone is so worked up about it here.

[AntipodeanPhil]

Really, this just stems from trying to get two metrics that measure different things to be equal, and being really concerned when they aren't. I don't see why everyone is so worked up about it here.

Because: (a) metric 1 is supposed to measure a proper subset of what metric 2 measures, and yet metric 1 is larger; and (b) CLS refuses to explain the discrepancy.

Since we all accept that law school rankings are ridiculous, employment data is the best way to compare schools. This suggests we can't trust the data.

[birdlaw117]

Really, this just stems from trying to get two metrics that measure different things to be equal, and being really concerned when they aren't. I don't see why everyone is so worked up about it here.

Because: (a) metric 1 is supposed to measure a proper subset of what metric 2 measures, and yet metric 1 is larger; and (b) CLS refuses to explain the discrepancy.

Since we all accept that law school rankings are ridiculous, employment data is the best way to compare schools. This suggests we can't trust the data.

I also like how you expect a school to be able to explain a discrepancy between numbers they report and numbers that a wholly separate organization reports. CLS really can't comment on one half of the figures, so how can they explain the discrepancy?

ETA: there is nothing out there saying that metric 1 really is measuring a proper subset of metric 2. You/Campos/whoever are simply asserting this to be the case.

[keg411]

2010 also had deferrals, though I don't know the proportion of '10 deferrals to '09 deferrals. I'd guess that's where the discrepancy comes in, but I'm not really sure. If I was considering one of these schools, I'd probably shoot an e-mail to someone there just to see if the schools can explain the difference rather than just idle-y speculate (though I'll give Campos props for bringing this up since it's interesting).

[skers]

Really, this just stems from trying to get two metrics that measure different things to be equal, and being really concerned when they aren't. I don't see why everyone is so worked up about it here.

Because: (a) metric 1 is supposed to measure a proper subset of what metric 2 measures, and yet metric 1 is larger; and (b) CLS refuses to explain the discrepancy.

Since we all accept that law school rankings are ridiculous, employment data is the best way to compare schools. This suggests we can't trust the data.

I also like how you expect a school to be able to explain a discrepancy between numbers they report and numbers that a wholly separate organization reports. CLS really can't comment on one half of the figures, so how can they explain the discrepancy?

ETA: there is nothing out there saying that metric 1 really is measuring a proper subset of metric 2. You/Campos/whoever are simply asserting this to be the case.

It is unless you assume that there a ton of NYU and Columbia grads working at magic circle firms or a fair number of firms in the nlj250 failed to report, but also didn't have a single Yale or Chicago grad.

[JusticeHarlan]

Really, this just stems from trying to get two metrics that measure different things to be equal, and being really concerned when they aren't. I don't see why everyone is so worked up about it here.

Because: (a) metric 1 is supposed to measure a proper subset of what metric 2 measures, and yet metric 1 is larger; and (b) CLS refuses to explain the discrepancy.

Since we all accept that law school rankings are ridiculous, employment data is the best way to compare schools. This suggests we can't trust the data.

I also like how you expect a school to be able to explain a discrepancy between numbers they report and numbers that a wholly separate organization reports. CLS really can't comment on one half of the figures, so how can they explain the discrepancy?

ETA: there is nothing out there saying that metric 1 really is measuring a proper subset of metric 2. You/Campos/whoever are simply asserting this to be the case.

Your point about not holding them accountable for someone else's numbers is a decent one, but I don't understand your last point. Are you saying that {the number of first years at firms of 250+ lawyers} isn't a subset of {the number of first years at firms of 160+}? There is simply no way that {the number of first years at firms of 250+ lawyers} > {the number of first years at firms of 160+} if you're counting first years the same way. It's not a simple assertion. It's math.

[timbs4339]

Really, this just stems from trying to get two metrics that measure different things to be equal, and being really concerned when they aren't. I don't see why everyone is so worked up about it here.

Because: (a) metric 1 is supposed to measure a proper subset of what metric 2 measures, and yet metric 1 is larger; and (b) CLS refuses to explain the discrepancy.

Since we all accept that law school rankings are ridiculous, employment data is the best way to compare schools. This suggests we can't trust the data.

I also like how you expect a school to be able to explain a discrepancy between numbers they report and numbers that a wholly separate organization reports. CLS really can't comment on one half of the figures, so how can they explain the discrepancy?

ETA: there is nothing out there saying that metric 1 really is measuring a proper subset of metric 2. You/Campos/whoever are simply asserting this to be the case.

But in this case they and their peer schools collect the same information for NALP reporting purposes. It doesn't seem unreasonable to ask where the discrepancy between CLS/NYU and Chicago/Yale comes from and the fact they refuse to comment is very telling.

[hung jury]

http://en.wikipedia.org/wiki/List_of_10 ... _law_firms

Notice the many, many 250+ UK-based firms. All of these would count under employment of 250+, and all of them are not included in the NLJ250. I'm surprised no one else has pointed this out?

You'd have to explain why roughly 20% of 2010 CLS/NYU grads went to foreign-based firms but nowhere near as high percentages as other T14s did. Your proposition is essentially saying that one in five CLS/NYU grads going into biglaw is going to a Magic Circle firm or the like, but almost no one else graduating from the T14 going into biglaw is.

Yes, this. What is strange about the discrepancy is it seems to center on the three top schools that place a lot of grads into NY, but it doesn't carry over to the other t14 who also place a lot of grads in NY (Penn, Cornell).

But I'm mostly posting because I find it odd that answerman3 joined the site to offer this implausible explanation.

[birdlaw117]

Really, this just stems from trying to get two metrics that measure different things to be equal, and being really concerned when they aren't. I don't see why everyone is so worked up about it here.

Because: (a) metric 1 is supposed to measure a proper subset of what metric 2 measures, and yet metric 1 is larger; and (b) CLS refuses to explain the discrepancy.

Since we all accept that law school rankings are ridiculous, employment data is the best way to compare schools. This suggests we can't trust the data.

I also like how you expect a school to be able to explain a discrepancy between numbers they report and numbers that a wholly separate organization reports. CLS really can't comment on one half of the figures, so how can they explain the discrepancy?

ETA: there is nothing out there saying that metric 1 really is measuring a proper subset of metric 2. You/Campos/whoever are simply asserting this to be the case.

Your point about not holding them accountable for someone else's numbers is a decent one, but I don't understand your last point. Are you saying that {the number of first years at firms of 250+ lawyers} isn't a subset of {the number of first years at firms of 160+}? There is simply no way that {the number of first years at firms of 250+ lawyers} > {the number of first years at firms of 160+} if you're counting first years the same way. It's not a simple assertion. It's math.

It's not simple math. NLJ250 is not based on every firm in the world, as pointed out by another poster. So in reality, it is definitionally not a subset. It's math.

ETA: It still doesn't explain the weirdness of it, but IMO it shows that there are more variables here than you guys think there are. You also assume that the figures are a snapshot of the same point in time, but I'm unaware of any reason to believe this.

[Tiago Splitter]

It's not simple math. NLJ250 is not based on every firm in the world, as pointed out by another poster. So in reality, it is definitionally not a subset. It's math.

This argument seems reasonable but doesn't answer why other schools don't have this discrepancy. NLJ250 underreporting, international hires and different treatment of deferred associates should show up in the numbers of every school.

[birdlaw117]

It's not simple math. NLJ250 is not based on every firm in the world, as pointed out by another poster. So in reality, it is definitionally not a subset. It's math.

This argument seems reasonable but doesn't answer why other schools don't have this discrepancy. NLJ250 underreporting, international hires and different treatment of deferred associates should show up in the numbers of every school.

Agreed. Yale apparently had a negative discrepancy that Campos reported at 0. So that at least shows that it works in two directions, meaning there are multiple variables. Multiple variables could create opposite discrepancies of similar values, making it appear to be close to 0. That's about all I've got, but it isn't a very rewarding explanation.

[timbs4339]

It's not simple math. NLJ250 is not based on every firm in the world, as pointed out by another poster. So in reality, it is definitionally not a subset. It's math.

This argument seems reasonable but doesn't answer why other schools don't have this discrepancy. NLJ250 underreporting, international hires and different treatment of deferred associates should show up in the numbers of every school.

There also aren't very many international firms hiring CLS students in the first place. I counted five in my EIP data.

[Bronck]

As someone else mentioned above, the discrepancy is with HCN, and not YSC, schools that place heavily into NYC and have large class sizes. Penn doesn't have the discrepancy but they also have a far smaller class size too.

Is there anything to the conjunction of those two factors?

[JusticeHarlan]

Your point about not holding them accountable for someone else's numbers is a decent one, but I don't understand your last point. Are you saying that {the number of first years at firms of 250+ lawyers} isn't a subset of {the number of first years at firms of 160+}? There is simply no way that {the number of first years at firms of 250+ lawyers} > {the number of first years at firms of 160+} if you're counting first years the same way. It's not a simple assertion. It's math.

It's not simple math. NLJ250 is not based on every firm in the world, as pointed out by another poster. So in reality, it is definitionally not a subset. It's math.

ETA: It still doesn't explain the weirdness of it, but IMO it shows that there are more variables here than you guys think there are. You also assume that the figures are a snapshot of the same point in time, but I'm unaware of any reason to believe this.

Yes, another poster brought up the international firm argument. But it makes little sense, as other posters have said, because (1) 20% going to those firms is a staggering amount, and (2) why does this affect NYU and Columbia more than Penn or Harvard (which experienced some amount of overstatement, but less per capita than either NYU or CLS)? It's not a satisfactory answer.

Now, the reporting time is an interesting hypothesis. It's worth looking into. 9 months from graduation for class of 2010 is roughly, what, mid February 2011, assuming mid May graduations? The NLJ250 numbers were published February 28 2011, so presumably the survey went out some time before that. If start dates were deferred to the point such that first years would start after the survey went out but before 9 months, I guess there could be a lag of those not yet started for NLJ purposes but who did by 9 months. Worth looking into, I guess, but the window seems somewhat tight. Also doesn't quite explain why some schools experience this more than others.

[birdlaw117]

As someone else mentioned above, the discrepancy is with HCN, and not YSC, schools that place heavily into NYC and have large class sizes. Penn doesn't have the discrepancy but they also have a far smaller class size too.

Is there anything to the conjunction of those two factors?

One could guess that self-reported data is less likely to be as accurate with a larger class size. Perhaps HCN put count firm jobs that don't specify a firm size into the 500+ size. Purely speculative, but it does seem like there is a NYC correlation, a class size correlation, and perhaps a inverse relationship with placement power.

[birdlaw117]

Your point about not holding them accountable for someone else's numbers is a decent one, but I don't understand your last point. Are you saying that {the number of first years at firms of 250+ lawyers} isn't a subset of {the number of first years at firms of 160+}? There is simply no way that {the number of first years at firms of 250+ lawyers} > {the number of first years at firms of 160+} if you're counting first years the same way. It's not a simple assertion. It's math.

It's not simple math. NLJ250 is not based on every firm in the world, as pointed out by another poster. So in reality, it is definitionally not a subset. It's math.

ETA: It still doesn't explain the weirdness of it, but IMO it shows that there are more variables here than you guys think there are. You also assume that the figures are a snapshot of the same point in time, but I'm unaware of any reason to believe this.

Yes, another poster brought up the international firm argument. But it makes little sense, as other posters have said, because (1) 20% going to those firms is a staggering amount, and (2) why does this affect NYU and Columbia more than Penn or Harvard (which experienced some amount of overstatement, but less per capita than either NYU or CLS)? It's not a satisfactory answer.

Now, the reporting time is an interesting hypothesis. It's worth looking into. 9 months from graduation for class of 2010 is roughly, what, mid February 2011, assuming mid May graduations? The NLJ250 numbers were published February 28 2011, so presumably the survey went out some time before that. If start dates were deferred to the point such that first years would start after the survey went out but before 9 months, I guess there could be a lag of those not yet started for NLJ purposes but who did by 9 months. Worth looking into, I guess, but the window seems somewhat tight. Also doesn't quite explain why some schools experience this more than others.

The fact that there are many variables that we have identified that can have some impact should make you question the methodology behind Campos' claims, since he seems to be ignoring the possibility of any of these existing.

We're still trying to get two metrics to equal one another when they measure different things. It's crazy to think you're going to reconcile them.

[Tiago Splitter]

I went back and looked at 2008 for NYU:

NLJ 250 says: 317
NYU says: 317

[birdlaw117]

I went back and looked at 2008 for NYU:

NLJ 250 says: 317
NYU says: 317

Which leads me to believe that there is still something unaccounted for unless we think there were no NYU grads that went to NLJ250 firms with fewer than 250 attorneys. I definitely don't believe that to be true.

[Tiago Splitter]

I went back and looked at 2008 for NYU:

NLJ 250 says: 317
NYU says: 317

Which leads me to believe that there is still something unaccounted for unless we think there were no NYU grads that went to NLJ250 firms with fewer than 250 attorneys. I definitely don't believe that to be true.

I don't think that's true either but a discrepancy of 10-20 attorneys at a school of NYU's size makes sense based on the other factors already discussed. As someone who will likely be attending one of these schools I'm hoping the deferment/timing issue is what's really throwing these numbers off.

[birdlaw117]

I went back and looked at 2008 for NYU:

NLJ 250 says: 317
NYU says: 317

Which leads me to believe that there is still something unaccounted for unless we think there were no NYU grads that went to NLJ250 firms with fewer than 250 attorneys. I definitely don't believe that to be true.

I don't think that's true either but a discrepancy of 10-20 attorneys at a school of NYU's size makes sense based on the other factors already discussed. As someone who will likely be attending one of these schools I'm hoping the deferment/timing issue is what's really throwing these numbers off.

It is true. It would create a "negative discrepancy." The NYU numbers would be lower than the NLJ250 numbers. Example here would be Wachtell, which is a NLJ250 firm but only has like 230 attorneys.

[booboo]

birdlaw, since you seem invested in this conversation, you could try counting the amount of people that went into a firm from the file that lists positions at firms.

I've done it once but cannot remember the numbers.

[birdlaw117]

birdlaw, since you seem invested in this conversation, you could try counting the amount of people that went into a firm from the file that lists positions at firms.

I've done it once but cannot remember the numbers.

I could. But that wouldn't really prove or show anything, since it would be moving in the opposite direction of Campos' discrepancy. I just look at it as proof that this is not as simple as everyone is trying to make it. There are multiple moving parts and without knowing exactly how all of the data is collected, trying to reconcile it futile.

[booboo]

birdlaw, since you seem invested in this conversation, you could try counting the amount of people that went into a firm from the file that lists positions at firms.

I've done it once but cannot remember the numbers.

I could. But that wouldn't really prove or show anything, since it would be moving in the opposite direction of Campos' discrepancy. I just look at it as proof that this is not as simple as everyone is trying to make it. There are multiple moving parts and without knowing exactly how all of the data is collected, trying to reconcile it futile.

I agree. And those numbers are SA positions anyway. My only contribution is that I don't think NYU is attempting to deceive anyone.