LST Score Reports (Asking TLSers to test beta product) Forum

[jenesaislaw]

Hi everybody,

Law School Transparency is soon releasing a competitor to U.S. News rankings. The beta version is available now at LSTScoreReports.com. (Note that if you click the tabs at the top, they take you back to LST.com and sometimes do not work -- they will work when the Score Reports leave beta.) I am purposefully not going to comment much more beyond this. Any feedback would be much appreciated.

The gist is that the "Score Reports" aim to focus people on the schools that best meet their career objectives. They start broad and ease you into the mountain of data we provide. Specifically, we hope people stop focusing on the U.S. News rankings because they have little connection to employment prospects beyond the top few schools. (For more on why the rankings have almost no value, see this.)

Also, check out the guides we have made available. Most of the ones available now are about the Score Reports and their different parts. Recommendations appreciated here too.

Kyle
Executive Director, Law School Transparency

[gaud]

I dig it.

[crumpetsandtea]

Woah, this is really cool! Thanks for the heads up about it (:

[Starletangel07]

Looks promising! I have spent some time on the site's employment date compilation, and it's easier than opening one school at a time from the ABA's site. Convenient.

[lovejopd]

Thanks!

However, according to LST, Virginia has the top employment score over Columbia, Stanford, and Harvard....

[dingbat]

According to this, in NY, UVA outperforms Columbia while Touro outperforms Fordham.
Not only that, GW outperforms Georgetown, Northwestern, Michigan, Cornell.
There is no possible way this makes sense.

How are you calculating this?

[Mroberts3]

I have a lot of respect for what LST is trying to do, and I agree that the US News is not a good system. However, I want to echo what the above poster said: some of this just doesn't follow common sense. Certainly some schools over/under perform their rank and certainly some T3 schools do very well in their home state/market, but I find it hard to believe that a school like Mercer does almost as well as both USC and UCLA.

Perhaps the problem is that the model just goes by the numbers: it treats someone at a 4 person firm making 60k as the same as someone in biglaw. There are plenty of reasons to not do biglaw, but the fact is that it is a "better" job in that it pays more, is harder to get, and the people that end up doing it had/will have better options all around. Maybe the bottom 20-30% of the class will struggle at both USC/UCLA and Mercer, but I think the options available to the rest are not really the same.

[Swimp]

According to this, in NY, UVA outperforms Columbia while Touro outperforms Fordham.
Not only that, GW outperforms Georgetown, Northwestern, Michigan, Cornell.
There is no possible way this makes sense.

How are you calculating this?

I noticed that. I assumed the rank took into account cost and maybe other factors too. If that's the case, of course it would be nice to have the formula spelled out, otherwise I don't think the tool will be of much use.

[JCFindley]

I like it; I like it a LOT!

While it may have results that go against the CW, much of the CW is based upon USN's faulty ranking system being ingrained in our psyche. Perhaps Touro does outperform Fordham in NY and I can guarantee Mercer outperforms USC and UCLA in Macon GA or even Dothan AL.

It would be interesting to see the metrics involved but conceptually I love it.

[dingbat]

I like it; I like it a LOT!

While it may have results that go against the CW, much of the CW is based upon USN's faulty ranking system being ingrained in our psyche. Perhaps Touro does outperform Fordham in NY and I can guarantee Mercer outperforms USC and UCLA in Macon GA or even Dothan AL.

It would be interesting to see the metrics involved but conceptually I love it.

Touro doesn't outperform Fordham in "desirable" jobs*, but then, that's the major flaw with the logic

The Employment Score represents the percentage of graduates
who have successfully started a career in the practice of law, though
it does not judge the quality of that start

Having read the methodology, the problem is that there are too many "questionable" statistics for which LSN uses an all or nothing approach, such that graduates who can't find any job and instead join up with other classmates to form a small firm are counted, whereas graduates taking a part-time or short-term position at an existing firm with the possibility of it becoming full-time or permanent are not counted.
Furthermore, there are many jobs that do not require a JD but are desirable, including consulting, working on a political campaign, or even working for a foreign government

The methodology employed by LSN is at least as flawed as USN, but in a different manner, and should be taken with a very large dose of skepticism grain of salt.

*about 1/3 are in firms of 2-10 attorneys - aka "couldn't get a job and partnered up my classmate; after subtracting solo and 2-10 attorney jobs, only 44% are employed at all (10% of those don't require a JD), and less than 16% are employed by a law firm of 11 or more attorneys

[Mroberts3]

I like it; I like it a LOT!

While it may have results that go against the CW, much of the CW is based upon USN's faulty ranking system being ingrained in our psyche. Perhaps Touro does outperform Fordham in NY and I can guarantee Mercer outperforms USC and UCLA in Macon GA or even Dothan AL.

It would be interesting to see the metrics involved but conceptually I love it.

Touro doesn't outperform Fordham in "desirable" jobs*, but then, that's the major flaw with the logic

The Employment Score represents the percentage of graduates
who have successfully started a career in the practice of law, though
it does not judge the quality of that start

Having read the methodology, the problem is that there are too many "questionable" statistics for which LSN uses an all or nothing approach, such that graduates who can't find any job and instead join up with other classmates to form a small firm are counted, whereas graduates taking a part-time or short-term position at an existing firm with the possibility of it becoming full-time or permanent are not counted.
Furthermore, there are many jobs that do not require a JD but are desirable, including consulting, working on a political campaign, or even working for a foreign government

The methodology employed by LSN is at least as flawed as USN, but in a different manner, and should be taken with a very large dose of skepticism grain of salt.

*about 1/3 are in firms of 2-10 attorneys - aka "couldn't get a job and partnered up my classmate; after subtracting solo and 2-10 attorney jobs, only 44% are employed at all (10% of those don't require a JD), and less than 16% are employed by a law firm of 11 or more attorneys

This is the point I was trying to make. I'm not trying to knock small firms, but its just too hard to know what kind of employment these jobs really provide. Biglaw jobs might suck, but at least we know they pay the bills. Government jobs pay less, but we know they pay enough.

Again, I like the data/employment focus of LST, but the conventional wisdom has some sway: just ask 90% of employers whether they want a Fordam grad or a Touro grad given the same GPA. Some might not care, but most will.

Also, I'm making these comments because hopefully there is a way to tweak the modeling (it is a beta after all), and not because I am a curmudgeon that likes to shoot down other people's ideas. I am totally willing to believe that there are T3 schools out there that are a good deal and that some "top" schools don't meet their perceived ranking. That said, I don't think the current model is accurate.

[JCFindley]

I like it; I like it a LOT!

While it may have results that go against the CW, much of the CW is based upon USN's faulty ranking system being ingrained in our psyche. Perhaps Touro does outperform Fordham in NY and I can guarantee Mercer outperforms USC and UCLA in Macon GA or even Dothan AL.

It would be interesting to see the metrics involved but conceptually I love it.

Touro doesn't outperform Fordham in "desirable" jobs*, but then, that's the major flaw with the logic

The Employment Score represents the percentage of graduates
who have successfully started a career in the practice of law, though
it does not judge the quality of that start

Having read the methodology, the problem is that there are too many "questionable" statistics for which LSN uses an all or nothing approach, such that graduates who can't find any job and instead join up with other classmates to form a small firm are counted, whereas graduates taking a part-time or short-term position at an existing firm with the possibility of it becoming full-time or permanent are not counted.
Furthermore, there are many jobs that do not require a JD but are desirable, including consulting, working on a political campaign, or even working for a foreign government

The methodology employed by LSN is at least as flawed as USN, but in a different manner, and should be taken with a very large dose of skepticism grain of salt.

*about 1/3 are in firms of 2-10 attorneys - aka "couldn't get a job and partnered up my classmate; after subtracting solo and 2-10 attorney jobs, only 44% are employed at all (10% of those don't require a JD), and less than 16% are employed by a law firm of 11 or more attorneys

This is the point I was trying to make. I'm not trying to knock small firms, but its just too hard to know what kind of employment these jobs really provide. Biglaw jobs might suck, but at least we know they pay the bills. Government jobs pay less, but we know they pay enough.

Again, I like the data/employment focus of LST, but the conventional wisdom has some sway: just ask 90% of employers whether they want a Fordam grad or a Touro grad given the same GPA. Some might not care, but most will.

Also, I'm making these comments because hopefully there is a way to tweak the modeling (it is a beta after all), and not because I am a curmudgeon that likes to shoot down other people's ideas. I am totally willing to believe that there are T3 schools out there that are a good deal and that some "top" schools don't meet their perceived ranking. That said, I don't think the current model is accurate.

I agree with both of yall except that "desirable" jobs is somewhat of a subjective term. If we are strictly talking income out of the gate, then without a doubt biglaw is at the top of the list. If we are just talking prestige then clerking for the right court may win. There are so many variables though in what people are really looking for. How about career earnings potential? There may well be plaintiff firms that will pay more in the long term if you like such things. Some folks like me want criminal law which is certainly not going to pay much right off the bat but may pay quite when I am representing Liborio Bellomo (AKA Big Barney.)

I am not arguing with yall just stating that how the rankings might define desirable jobs can not likely be done in a generalist manor.

[dingbat]

I am not arguing with yall just stating that how the rankings might define desirable jobs can not likely be done in a generalist manor.

I do nothing in the general's manor

that being beside the point, it is true that there are T3 schools worth going to, and T1s that aren't. However, it is far more nuanced than can be extrapolated by analyzing every law school in the same manner. If someone's goal is biglaw or a federal clerkship, this is far more useful. On the other hand, LSN's data is more useful if someone wants to live and work in less populous states/areas (like where a market is only served by a single law school). But, no single methodology will show, for example, that CUNY is a good option for someone looking to work for the city of NY, or that Yale is the best option for someone wanting to be a law professor.
If someone wants to weigh cost of living to salary, this might be far more useful

The thing is, it's possible to parse the data any which way you want (see here) and it's ingenuous to say "The gist is that the "Score Reports" aim to focus people on the schools that best meet their career objectives" when all it does is provide a single method of interpreting a school's employment data and adds in a geographic component.

I don't want to belittle what is actually very valuable work (compiling data and allowing schools to be ranked by percentages going to, say, federal clerkship, or public interest is great, keep it up), but the employment/underemployment score is not a better methodology than the USN rankings, and in some cases is downright misleading (Notre Dame outranking UCLA for California? Louisiana outperforming UT in Texas?)

[onionz]

I don't want to belittle what is actually very valuable work (compiling data and allowing schools to be ranked by percentages going to, say, federal clerkship, or public interest is great, keep it up), but the employment/underemployment score is not a better methodology than the USN rankings, and in some cases is downright misleading (Notre Dame outranking UCLA for California? Louisiana outperforming UT in Texas?)

Indeed, the report should certainly take into account what % of graduates end up practicing in that state. Maybe make it a minimum threshold, or weight the Employment score differently if x% don't practice in that state. At a minimum.

[dingbat]

I don't want to belittle what is actually very valuable work (compiling data and allowing schools to be ranked by percentages going to, say, federal clerkship, or public interest is great, keep it up), but the employment/underemployment score is not a better methodology than the USN rankings, and in some cases is downright misleading (Notre Dame outranking UCLA for California? Louisiana outperforming UT in Texas?)

Indeed, the report should certainly take into account what % of graduates end up practicing in that state. Maybe make it a minimum threshold, or weight the Employment score differently if x% don't practice in that state. At a minimum.

I didn't even mention that UVA outranked both Berkeley and Stanford for California.

[Tiago Splitter]

I don't want to belittle what is actually very valuable work (compiling data and allowing schools to be ranked by percentages going to, say, federal clerkship, or public interest is great, keep it up), but the employment/underemployment score is not a better methodology than the USN rankings, and in some cases is downright misleading (Notre Dame outranking UCLA for California? Louisiana outperforming UT in Texas?)

Indeed, the report should certainly take into account what % of graduates end up practicing in that state. Maybe make it a minimum threshold, or weight the Employment score differently if x% don't practice in that state. At a minimum.

The report seems to take all the schools that place anyone into a state and then just sorts those schools by employment score. Since UVA had the highest employment score it always shows up first in any state where it placed a non-negligible number of graduates.

[jenesaislaw]

Thanks!

However, according to LST, Virginia has the top employment score over Columbia, Stanford, and Harvard....

According to this, in NY, UVA outperforms Columbia while Touro outperforms Fordham.
Not only that, GW outperforms Georgetown, Northwestern, Michigan, Cornell.
There is no possible way this makes sense.

How are you calculating this?

http://www.lstscorereports.com/?r=guide ... _and_rates

Keep in mind that the Employment Score is a starting point. People need to dig into the numbers because no single number will ever tell the whole story.

some of this just doesn't follow common sense. Certainly some schools over/under perform their rank and certainly some T3 schools do very well in their home state/market, but I find it hard to believe that a school like Mercer does almost as well as both USC and UCLA.

Where does your common sense come from? That's an important thing to think about. Read this guide: http://www.lstscorereports.com/?r=guides&show=usnews

Perhaps the problem is that the model just goes by the numbers: it treats someone at a 4 person firm making 60k as the same as someone in biglaw. There are plenty of reasons to not do biglaw, but the fact is that it is a "better" job in that it pays more, is harder to get, and the people that end up doing it had/will have better options all around. Maybe the bottom 20-30% of the class will struggle at both USC/UCLA and Mercer, but I think the options available to the rest are not really the same.

The Employment Score does do that, as we discuss extensively here in the weaknesses guide: http://www.lstscorereports.com/?r=guide ... weaknesses The ES is just a starting point, however. There are other data there to see how schools stack up on different metrics.

And again re: "I think the options available to the rest are not really the same" -- what do you have to make this claim other than what you expect from U.S. News? And, does the data available support it or not? The reality is that the bottom half of USC/UCLA are really struggling right now, and they have lots of debt. The top part (30%ish) are doing okay relative to other schools, but not necessarily justifying the cost of attendance.

I noticed that. I assumed the rank took into account cost and maybe other factors too. If that's the case, of course it would be nice to have the formula spelled out, otherwise I don't think the tool will be of much use.

http://www.lstscorereports.com/?r=guide ... _and_rates

[SuperCerealBrah]

Thanks!

However, according to LST, Virginia has the top employment score over Columbia, Stanford, and Harvard....

According to this, in NY, UVA outperforms Columbia while Touro outperforms Fordham.
Not only that, GW outperforms Georgetown, Northwestern, Michigan, Cornell.
There is no possible way this makes sense.

How are you calculating this?

http://www.lstscorereports.com/?r=guide ... _and_rates

Keep in mind that the Employment Score is a starting point. People need to dig into the numbers because no single number will ever tell the whole story.

some of this just doesn't follow common sense. Certainly some schools over/under perform their rank and certainly some T3 schools do very well in their home state/market, but I find it hard to believe that a school like Mercer does almost as well as both USC and UCLA.

Where does your common sense come from? That's an important thing to think about. Read this guide: http://www.lstscorereports.com/?r=guides&show=usnews

Perhaps the problem is that the model just goes by the numbers: it treats someone at a 4 person firm making 60k as the same as someone in biglaw. There are plenty of reasons to not do biglaw, but the fact is that it is a "better" job in that it pays more, is harder to get, and the people that end up doing it had/will have better options all around. Maybe the bottom 20-30% of the class will struggle at both USC/UCLA and Mercer, but I think the options available to the rest are not really the same.

The Employment Score does do that, as we discuss extensively here in the weaknesses guide: http://www.lstscorereports.com/?r=guide ... weaknesses The ES is just a starting point, however. There are other data there to see how schools stack up on different metrics.

And again re: "I think the options available to the rest are not really the same" -- what do you have to make this claim other than what you expect from U.S. News? And, does the data available support it or not? The reality is that the bottom half of USC/UCLA are really struggling right now, and they have lots of debt. The top part (30%ish) are doing okay relative to other schools, but not necessarily justifying the cost of attendance.

I noticed that. I assumed the rank took into account cost and maybe other factors too. If that's the case, of course it would be nice to have the formula spelled out, otherwise I don't think the tool will be of much use.

http://www.lstscorereports.com/?r=guide ... _and_rates

Yea guys, you need to check the "gut feeling" and "common sense" stuff at the door. It is all based off your own assumptions on how things are rather than real data. Also, just because a school is placing more students overall into long term, full time employment technically does NOT mean that they are placing more into the more desirable jobs justifying COA. This is exactly why he is stating you need to use these numbers as a starting point.

[jenesaislaw]

Again, I like the data/employment focus of LST, but the conventional wisdom has some sway: just ask 90% of employers whether they want a Fordam grad or a Touro grad given the same GPA. Some might not care, but most will.

And yet those employers are not hiring Fordham grads either. The data are outcome-based, not based on perception.

Also, I'm making these comments because hopefully there is a way to tweak the modeling (it is a beta after all), and not because I am a curmudgeon that likes to shoot down other people's ideas. I am totally willing to believe that there are T3 schools out there that are a good deal and that some "top" schools don't meet their perceived ranking. That said, I don't think the current model is accurate.

I think the problem is that people are viewing the ES as a definitive ranking rather than one metric among many.

Let's look at the Fordham and Touro comparison. To start, this comparison makes a lot of sense because both place almost all of their graduates in New York.

Touro:
Emp. Score: +57.9%
Under-Emp. Score: -27.6%
ND Cost: 249k
101+ Firms: 2.3%
26-100 Firms: 7.7%
2-25 Firms: 29.9%
Fed Clerks: 0%

Fordham:
Emp. Score: +57.5%
Under-Emp. Score: -30.1%
ND Cost: 276k
101+ Firms: 24.8%
26-100 Firms: 5.4%
2-25 Firms: 9.3%
Fed Clerks: 2.3%

The first thing we see is that the ESs are about the same. We then unbundle the data and see that Fordham's placement in firms is mostly at large firms, and Touro is mostly at small firms. I'm going to go ahead and guess you're interested in biglaw, so you can now rule out Touro rather quickly. Fewer than 1/25 grads get those jobs.

Isn't this exactly where the conventional wisdom would get you? Fordham has a better shot at the type of employment you want than Touro.

The issue, it seems, is that people don't think Touro should be performing "as well" as Fordham on the one metric which explicitly does not judge anything other than a start to a legal career. Perhaps you should be asking instead whether it means something that Fordham is performing "as poorly" as Touro. The answer to me is pretty clearly that this is a huge problem. Fordham's non-discounted cost if $276k. Neither school has higher than a 58% Employment Score. Not even 60% of the graduates from these schools started a legal career.

[SuperCerealBrah]

I am not arguing with yall just stating that how the rankings might define desirable jobs can not likely be done in a generalist manor.

I do nothing in the general's manor

that being beside the point, it is true that there are T3 schools worth going to, and T1s that aren't. However, it is far more nuanced than can be extrapolated by analyzing every law school in the same manner. If someone's goal is biglaw or a federal clerkship, this is far more useful. On the other hand, LSN's data is more useful if someone wants to live and work in less populous states/areas (like where a market is only served by a single law school). But, no single methodology will show, for example, that CUNY is a good option for someone looking to work for the city of NY, or that Yale is the best option for someone wanting to be a law professor.
If someone wants to weigh cost of living to salary, this might be far more useful

The thing is, it's possible to parse the data any which way you want (see here) and it's ingenuous to say "The gist is that the "Score Reports" aim to focus people on the schools that best meet their career objectives" when all it does is provide a single method of interpreting a school's employment data and adds in a geographic component.

I don't want to belittle what is actually very valuable work (compiling data and allowing schools to be ranked by percentages going to, say, federal clerkship, or public interest is great, keep it up), but the employment/underemployment score is not a better methodology than the USN rankings, and in some cases is downright misleading (Notre Dame outranking UCLA for California? Louisiana outperforming UT in Texas?)

I think you are misunderstanding the stats. When you click on texas, it is not saying that LSU is placing 79% IN that state of Texas compared to UT. That is just its overall employment score. That isn't saying LSU is outperforming UT in Texas at all. I agree the layout is a bit misleading here though.

[onionz]

Thanks!

However, according to LST, Virginia has the top employment score over Columbia, Stanford, and Harvard....

According to this, in NY, UVA outperforms Columbia while Touro outperforms Fordham.
Not only that, GW outperforms Georgetown, Northwestern, Michigan, Cornell.
There is no possible way this makes sense.

How are you calculating this?

http://www.lstscorereports.com/?r=guide ... _and_rates

Keep in mind that the Employment Score is a starting point. People need to dig into the numbers because no single number will ever tell the whole story.

some of this just doesn't follow common sense. Certainly some schools over/under perform their rank and certainly some T3 schools do very well in their home state/market, but I find it hard to believe that a school like Mercer does almost as well as both USC and UCLA.

Where does your common sense come from? That's an important thing to think about. Read this guide: http://www.lstscorereports.com/?r=guides&show=usnews

Perhaps the problem is that the model just goes by the numbers: it treats someone at a 4 person firm making 60k as the same as someone in biglaw. There are plenty of reasons to not do biglaw, but the fact is that it is a "better" job in that it pays more, is harder to get, and the people that end up doing it had/will have better options all around. Maybe the bottom 20-30% of the class will struggle at both USC/UCLA and Mercer, but I think the options available to the rest are not really the same.

The Employment Score does do that, as we discuss extensively here in the weaknesses guide: http://www.lstscorereports.com/?r=guide ... weaknesses The ES is just a starting point, however. There are other data there to see how schools stack up on different metrics.

And again re: "I think the options available to the rest are not really the same" -- what do you have to make this claim other than what you expect from U.S. News? And, does the data available support it or not? The reality is that the bottom half of USC/UCLA are really struggling right now, and they have lots of debt. The top part (30%ish) are doing okay relative to other schools, but not necessarily justifying the cost of attendance.

I noticed that. I assumed the rank took into account cost and maybe other factors too. If that's the case, of course it would be nice to have the formula spelled out, otherwise I don't think the tool will be of much use.

http://www.lstscorereports.com/?r=guide ... _and_rates

I notice no response to the "UVA" problem, or generally of schools having higher employment scores in States that they really don't have an actual presence in.

Also, I think it's kind of a cop-out to say "Here's a ranking, but it's only a starting point." That's exactly what people say of the US News reports and has generally been the defense of law schools that are getting sued for posting statistics in accurate but misleading ways. If you're going to quantitatively mark schools and rank them, they should bear some meaning to their general standing or to what you're trying to say.

Also, your response to the Mercer comment is a non-response. Almost 50% of Mercer graduates go to firms of sizes 2-10 people large... common sense comes from what we know to be good employment outcomes vs. bad employment outcomes.

[jenesaislaw]

I think you are misunderstanding the stats. When you click on texas, it is not saying that LSU is placing 79% IN that state of Texas compared to UT. That is just its overall employment score. That isn't saying LSU is outperforming UT in Texas at all. I agree the layout is a bit misleading here though.

Yup, you're exactly right. I'm still trying to figure out how best to get this point across effectively. Open to suggestions.

[onionz]

I am not arguing with yall just stating that how the rankings might define desirable jobs can not likely be done in a generalist manor.

I do nothing in the general's manor

that being beside the point, it is true that there are T3 schools worth going to, and T1s that aren't. However, it is far more nuanced than can be extrapolated by analyzing every law school in the same manner. If someone's goal is biglaw or a federal clerkship, this is far more useful. On the other hand, LSN's data is more useful if someone wants to live and work in less populous states/areas (like where a market is only served by a single law school). But, no single methodology will show, for example, that CUNY is a good option for someone looking to work for the city of NY, or that Yale is the best option for someone wanting to be a law professor.
If someone wants to weigh cost of living to salary, this might be far more useful

The thing is, it's possible to parse the data any which way you want (see here) and it's ingenuous to say "The gist is that the "Score Reports" aim to focus people on the schools that best meet their career objectives" when all it does is provide a single method of interpreting a school's employment data and adds in a geographic component.

I don't want to belittle what is actually very valuable work (compiling data and allowing schools to be ranked by percentages going to, say, federal clerkship, or public interest is great, keep it up), but the employment/underemployment score is not a better methodology than the USN rankings, and in some cases is downright misleading (Notre Dame outranking UCLA for California? Louisiana outperforming UT in Texas?)

I think you are misunderstanding the stats. When you click on texas, it is not saying that LSU is placing 79% IN that state of Texas compared to UT. That is just its overall employment score. That isn't saying LSU is outperforming UT in Texas at all. I agree the layout is a bit misleading here though.

I don't think he misunderstands the stat. When you go the main site it says "Click on the State you wish to practice in." Clicking on Texas and then seeing a green 79% Employment Score next to LSU in Texas, should mean it's outperforming UT. I think he knew it didn't mean 79% of people from that school graduates there, but that it IS misleading.

[Tiago Splitter]

How would you explain the results of a state like California to an uninitiated applicant?

[jenesaislaw]

I notice no response to the "UVA" problem, or generally of schools having higher employment scores in States that they really don't have an actual presence in.

The UVA problem I responded to dealt with the impact the long-term, full-time school-funded jobs have on the Employment Score. Specifically:

School-funded jobs present an interesting issue for any measurement of employment outcomes because they can span a range of jobs from the desirable to the illusory. On one end are year-long, full-time appointments in jobs that involve substantive legal work, provide valuable experience, and genuinely advance a recent graduate's career. On the other end are part-time positions that last only a short time and are timed to coincide with the nine month employment survey. With increasing attention drawn to school-funded jobs, and several schools employing more than 10% of their graduates, any measurement of employment outcomes would be remiss if it did not take these positions into consideration.

Our Employment Score makes no adjustment for short-term and part-time jobs funded by the school because none is needed. These jobs—often created with an eye towards inflating employment statistics—are already accounted for when we discount for short-term and part-time jobs. For full-time, long-term jobs funded by the school, we could not exclude jobs in this category even if we wanted. First, we cannot justify the assumption that all (or a critical mass) of long-term, full-time jobs funded by the school require bar passage—non-legal jobs have already been excluded and we do not want to risk excluding graduates twice.

Second, some of these jobs might actually be jobs with an indefinite term instead of a definite, one-year term. (It might be tempting to exclude definite-term jobs because of the likelihood that these jobs were structured to inflate employment statistics.) Jobs in clinics, as librarians, as writing instructors, or as professors each could have an indefinite term. Overall, the uncertainty here demonstrates how critical it is that schools disclose significantly more data on school-funded jobs.

Also, I think it's kind of a cop-out to say "Here's a ranking, but it's only a starting point." That's exactly what people say of the US News reports and has generally been the defense of law schools that are getting sued for posting statistics in accurate but misleading ways. If you're going to quantitatively mark schools and rank them, they should bear some meaning to their general standing or to what you're trying to say.

Also, your response to the Mercer comment is a non-response. Almost 50% of Mercer graduates go to firms of sizes 2-10 people large... common sense comes from what we know to be good employment outcomes vs. bad employment outcomes.

a) I did not say "here is a ranking." I purposefully do not use the term ranking. The default sort is by the Employment Score, but there are a number of tables and multiple columns on each table.
b) The rankings purport to sort by best. We do no such thing. The Employment Score sort sorts by the percentage of people who started a legal career within 9 months of graduating law school. There are other points to compare.
c) Re: Mercer, good thing you can compare that now, eh? Those people started a legal career just like others. It's not a biglaw start. It's not a federal clerkship start. It's a start at a small firm, potentially with an unpredictable income. You're able to surmise this pretty readily from here.