A 72.2% response rate is really quite good for a study tracking graduates from their last-known address a full 20 years after graduation. E.g. it's been only 5 years since my graduation, and I've managed to move enough that my undergraduate institution has lost my trail as far as mail begging me for alumni donations. While I imagine there is some skew in the data, I think it would be unreasonable to assume that everyone who didn't respond was dissatisfied.NR3C1 wrote:This report, like any recent employment report of graduates, has the flaw that it only includes those who respond to the survey. In all likelihood, responders are much better off and happy than non-responders. This is not a random cohort; it is biased. This is valuable data, but one has to be careful about drawings conclusions because of this and other limitations.
See page 14:Of the 360 living graduates of the University of Virginia School of Law Class of 1990, completed surveys were returned by 260; 155 of these respondents (60%) were men and 105 (40%) were women. The response rate was 72.2%.
You need to be careful drawing conclusions from any data. That's a given. However, you have to also quantify how much the unknown data can affect the known data. E.g., even if everyone who didn't report is dissatisfied with being a lawyer, 58% of the whole class is still satisfied. Pessimism is different than rationality, and it's important to note which pessimistic assumptions cannot be simultaneously true.
I think one of the most valuable things the data shows is the existence of a substantial number of legal jobs 20 years out. The presumption on TLS seems to often be that after you're finished with big law, you go back to waiting tables or whatever you did before law school. But the study shows 232 reported full-time salaries in 5 categories, all with solid averages.
E.g. It shows 64 people still working in large law firms 20 years out. We don't know how many ever worked at a large firm, but we can bound it between 148 (the number of people who started at large firms) and 360. So anywhere from 18-43% of people who ever worked at a large firm found some long-term position.
E.g. The data shows 41 people working in small firms at apparently substantial salaries after 20 years. This is a category of jobs people generally assume doesn't exist. While it's true that this isn't a relevant category for fresh graduates, because these firms take maybe one associate every year or two, it's a very relevant category for trained lawyers lateraling out of big firms.
E.g. Even if you assume all of those 28% who didn't respond had bad outcomes, it's important not to "double count" them. You can't say, on one hand, that 28% never got a decent legal job when discussing jobs after graduation, and then say, on the other hand, that 28% ended up working at Wal-Mart after leaving big law.
E.g. The weighted average salary of those who were working full-time is well over $300k. Even if we assume that everyone who didn't report is making zero, that puts the average well above $200k. Now, that doesn't tell you much about the exact distribution, I think it's reasonable to compare that number to other longitudinal studies reporting averages. The average salary of Harvard UG graduates is about $115k, in comparison.