Everybody already murdered this deeply flawed study point-by-point. The TLDR version was that by not accounting for skyrocketing tuition and choosing to end the study in 2008, the study deliberately cherry-picks the best-case scenario while ignoring many things that actually account for the extraordinary cost of attending a TTT.
To my mind, this study has a very serious flaw - that is, it derives its results around the assumption of synthetic work-life earnings. For the non-labor economists reading this, that means that the future earnings of recent JDs are predicated on the lifetime earnings of old JDs. In the classic mini-treatise "Flaws and Fallacies in Statistical Thinking," Stephen K. Campbell describes the concept of "arithmancy" as "a derogatory term suggesting that somehow the manipulation of past numbers has imbued the resulting estimates of future numbers with a kind of magic" under his section on the "Uncritical Projection of Trends."
Bear Stearns and Lehman made the uncritical projection assumption that housing prices would always rise w/r/t to subprime lending, to their doom. In point of fact, bubble inflators have ignored the sage advice that "past performance is no guarantee of future returns" to their doom since that thing with the tulips in the Netherlands in the 1600's. Yet, it is the central assumption of this study. Huh.
Campbell then offers a host of questions that I will discuss at the bottom of the post.
The study also chooses 2008 as the most recent year to include new attorneys, despite extraordinarily negative changes to the legal hiring market since then that show no sign of abating anytime in the next ten years. Huh.
And it fails to take into consideration student loans when calculating lifetime earnings premiums. Huh.
Or taxes. Huh.
And it improperly compares lifetime wages of JD holders to bachelors holders, instead of other groups who spent a similar amount of time and money in school, like doctors, dentists, MBAs, and MS holders in Computer Science. Huh.
OK, back to the central flaw. This foundation on the synthetic work-life earnings assumption ignores the possibility that the enormous structural changes in law, as acknowledged by everyone from the ABA to the NALP to the BLS, represent a Black Swan (an unpredicted, catastrophic event) for the legal profession - even as the paper otherwise incorporates data from the NALP and BLS. To analogize, Polaroid and Kodak had record earnings right before the advent of digital cameras. Now they no longer exist. Digital cameras were a black swan. I'm sure the highest salaries for phaeton builders were in the years right before the Model T was introduced. Concorde mechanics probably earned their highest salaries just before the planes were pulled out of service because of safety concerns. Scriveners probably made a great salary right until typewriters went mainstream. One can go on and on.
The advent of automated, predictive technologies for discovery and drafting, the continual relaxation of the boundaries of UPL and subsequent competition for legal work between attorneys and less-expensive compliance workers, HR personnel, paralegals, offshore professionals, real estate agents, ADR folk, and IT workers, among others; the exploding growth of the LPO sector, and so forth may prove to be a Black Swan for lawyers. The eventual repeal of IBR/PAYE and/or PSLF would be a Black Swan for law students. The repeal of GradPLUS loans (as Senators Simpson & Bowles desire, as well as the higher ed foundation duopoly of Gates and Lumina) would be a Black Swan for law schools. The study acknowledges that these are unprecedented times of change in the legal profession, but then almost incredibly waves aside all of these dark scenarios with one stock turn of phrase, we can't predict the future, even as they go back to their thesis that career trajectories for new lawyers (and new JD holders not working as attorneys) will exactly track the path of JD recipients who are now 65. It's facially absurd.
Taken hand-in-hand with the purposeful exclusion of JD-holders into the marketplace post-2008, and we have some seriously flawed methodology.
Campbell's questions regarding possible instances of uncritical projection of trends:
1. What kind of reputation does the source enjoy as a supplier of statistical estimates or as an authority on the relevant subject?
I'll leave this judgment to others, but the lack of taxes or tuition in the lifetime earnings premiums smacks of bias. Particularly since one who debt-finances a Seton Hall Law School education (where the author is employed) and has the average undergrad debt amount will pass the bar with about $260,000 in student loans ($70k * 3 + bar expenses, principalizing interest on all graduate student loans, and $28k undergrad debt). $260,000 on a 20 year payback, whether on IBR/PAYE or not, will result in an additional $220,000 in interest payments. That works out to $480,000, which is considerably higher than the 25th percentile earnings premium for JDs, where a good number of Seton Hall grads can expect to end up, at least going by their ABA employment disclosures. Gives quite an incentive to proclaim that tuition levels are just too tricky to incorporate, even though it seems that variable is considerably simpler to handle than the blended SIPPS and BLS data.
2. Does the supplier of the data have an "axe to grind"?
Well, Seton Hall has had to reduce its class size dramatically over the last few years, possibly as much as 35%, resulting in millions of dollars in lost revenue. They may also conduct untenured faculty layoffs next year, as has been widely reported. And the study's cutoff date of 2008 just coincidentally lines up with the beginning of Seton Hall's (and every other law school's) troubles. According to the most recent ABA disclosures, only 204 of 310 Class of 2012 Seton Hall grads landed full-time, long-term, bar-passage required jobs. At any salary. More importantly, only 31 of those 310 graduates landed jobs at law firms with more than 100 attorneys, which are the ONLY employers who can justify spending $70,000/year to attend Seton. In fact, the student loan calculator I used at finaid.org to find the additional interest payments on a Seton Hall payback under question 1 specifically states that one needs to make $240,000 per year in order to afford such a burden, an amount vastly higher than even the market rate for white shoe firms in NYC. Seton Hall sports similarly depressing numbers for the past several years, though of course the paper in question chooses not to include them in its analysis.
3. What supportive evidence is offered?
For the notion that synthetic work-life earnings is an appropriate method of analysis? None that I could find - it seems to be taken for granted that this is the method that should be used.
4. Does the underlying assumption, theory, or methodology seem OK?
Not really. Beyond the issues I have outlined above, the study glosses over well-examined theories regarding 21st century hiring practices and long-term unemployment (i.e. HR software filters auto-reject resumes that do not match the job's requirements, including overqualified candidates; after six months of unemployment, the unemployed - including fresh grads - are dead in the water).
5. Do the estimates appear plausible?
No. To my mind, it smacks of confirmation bias, experimenter’s bias w/r/t the exclusion of student loans and taxes in the premium calculations, hostile media effect, possibly the inverse gambler’s fallacy w/r/t “past performance is determinative of future results,” illusion of validity from the same assumption (and stemming from shortcomings in NALP and ABA salary disclosures), irrational escalation, normalcy bias w/r/t the structural changes in the legal profession (and wider economy) over the last five years, observer-expectancy effect w/r/t excluding student loans from the equation, and the Semmelweis reflex w/r/t the notion that higher levels of education no longer automatically equate to higher levels of income. But then again, I am not a labor economist. I am just one of America's tens of thousands of un/underemployed attorneys.
Oh, and within two hours of posting this on Volokh Conspiracy (a libertarian law professor-run blog), my web browser was banned from the Social Science Research Network (SSRN) where this paper was released (error 401 you are not authorized to access this webpage), so my commentary has evidently drawn blood.
