Law School Predictor: The Thread" Forum

[YCrevolution]

..

[Bronte]

Would you all be willing to forego having the % +/-, % YOU, % MID. fields if it meant having a feature that would say something like "X% of applicants matriculated with a score at/below yours" (this was discussed a few posts ago by PrayingforHYS, Bronte, and thesealocust)?

My $0.02:

I think % +/- is actually a very important field, considering that medians drive the rankings and are thus very important to adcomms. However, %YOU, %MID, FROM 25, FROM MID, and FROM 75 could probably go, considering that the latter three are in "index units," which are fairly arbitrary, especially to the average user. You're probably using %YOU and %MID to arrive at the % +/- calculations, but you could just do % +/- directly, something like:

Code: Select all

= ( YOUR INDEX - MED INDEX ) / PERFECT INDEX 

Edit: clarity

[buddyblack]

[However, %YOU, %MID, FROM 25, FROM MID, and FROM 75 could probably go, considering that the latter three are in "index units," which are fairly arbitrary, especially to the average user.

I agree with you on your point about clarity/user interface, but I would rather keep the FROM 25th/median/75th index scores and lose the percentages because then you can quickly see where your index falls in that range rather than just it's relation to the median.

And YC, a % of applicants matriculated might be interesting, but I always feel like a "% applicants matriculated at a score below yours" is sort of useless because a 2.2/144 and a 3.5/167 are lumped together.

[Bronte]

I agree with you on your point about clarity/user interface, but I would rather keep the FROM 25th/median/75th index scores and lose the percentages because then you can quickly see where your index falls in that range rather than just it's relation to the median.

Again, just IMO, percent over/under median is the most important factor in a predictor, like LSP, that uses matriculation percentiles as its database. Remember, 25th and 75th don't matter nearly as much as median to the schools themselves. Further, one can still easily see which quartile his index score falls in by comparing it to the 25/50/75 index scores.

YC, maybe it's worth a poll?

And YC, a % of applicants matriculated might be interesting, but I always feel like a "% applicants matriculated at a score below yours" is sort of useless because a 2.2/144 and a 3.5/167 are lumped together.

This actually kind of runs contrary to what you're saying above, because a percentage matriculated is equivalent to an exact percentile. In other words, it shows you exactly (albeit via estimation/interpolation) where your index falls in the range. For example, 70% would be in the third quartile, closer to the 75th percentile.

Not trying to monopolize the discussion, just wanted to get those ideas out.

[gunners]

Yeah, I was going to write a scrapper to get all the data and convert it to a usable format for YCrevolution this weekend. Speaking of, YCrevolution what format do you prefer, I can do whatever, but I guessed that the one you would want was a csv file with username, lsat, gpa, law school, result. Correct? I can also give it to you as a database or whatever format you need.

I actually did some XML parsing into Excel as part of a research project this summer. I used java. Let me know if I can be of assistance. You too, YCrevolution.

[pissantvache]

YC, another way to cut down on the calculation time would be to create different applications for each tier (say, top 50, 51-100, 101-150, etc). Either that, or, alternatively, a landing page that would allow one to choose one's schools could be nice. As it is, I think that people usually have a sense of what schools they should look at, and it's really not valuable for me to be looking at a sea of auto-admits far below where I would ever consider applying and a sea of auto-rejects far above that point.

My $0.02:

I think % +/- is actually a very important field, considering that medians drive the rankings and are thus very important to adcomms. However, %YOU, %MID, FROM 25, FROM MID, and FROM 75 could probably go, considering that the latter three are in "index units," which are fairly arbitrary, especially to the average user. You're probably using %YOU and %MID to arrive at the % +/- calculations, but you could just do % +/- directly, something like:

Code:
= ( YOUR INDEX - MED INDEX ) / PERFECT INDEX

I wholeheartedly agree with Bronte on this point.

[CyLaw]

I actually did some XML parsing into Excel as part of a research project this summer. I used java. Let me know if I can be of assistance. You too, YCrevolution.

Thanks for the offer, but I actually finished it during lunch today, cause I was bored.

[YCrevolution]

..

[YCrevolution]

..

[YCrevolution]

..

[CyLaw]

If anybody has some easy-to-implement suggestions for LSP (in addition to those noted above), now would be a good time to tell me, so I can try and incorporate them into a new version before law school starts.

Can you make it auto-email the various admission committees so they are aware that they should admit me according to LSP

[YCrevolution]

..

[CyLaw]

I'll have it hack into their databases and auto-admit you everywhere you apply. If you could tell me how much scholarship money you'd like, I'll put that in, too.

For some reason, I don't believe you.

I think that what you have now plus the previous suggestions are plenty. Last thing you want is too much feature creep with LSP, with it doing more then really needed for the general populace. It will just make it a pain when you want to maintain or evolve the software.

[gahthelaw]

I don't know if this has been mentioned and dismissed (a quick glance around the thread isnt turning up too much), but is there any way to edit for multiple LSAT takers? I'm not sure what the best way to do this would be except for dredging up a list of who takes the highest score, who definitely averages, and then some number crunching just based on stats (i absolutely defer to people who are better than i at this for a good way).

[YCrevolution]

..

[gahthelaw]

I don't know if this has been mentioned and dismissed (a quick glance around the thread isnt turning up too much), but is there any way to edit for multiple LSAT takers? I'm not sure what the best way to do this would be except for dredging up a list of who takes the highest score, who definitely averages, and then some number crunching just based on stats (i absolutely defer to people who are better than i at this for a good way).

The implementation wouldn't be that hard, but then somebody would still have to dredge up (and I'd have to enter) what schools average/take the highest score. I'll put it as a longer-term goal.

As a general rule, plug your highest LSAT into LSP, even for schools that claim to average. I've been told by other TLS users that it's more accurate if you do that.

Thanks for the info. I've been using my higher one (mostly to make myself feel better).

I actually have a spreadsheet somewhere around listing who (in the T20) averages, takes higher, or is ambiguous along with the actual wording from the schools and I think the numbers of a few confirmed multiple lsat takers from LSN for each school (yes, I geeked out a little about this). It's on my work computer but if you eventually ever want the information let me know and I'll update it a bit, format it in maybe a more helpful way, and send it over.

[YCrevolution]

..

[SilverE2]

Ladies and gentlemen, I give you the accuracy statistics for the Law School Predictor.

These results are based off of more than 33,500 LSN-listed law school admission decisions from the 2008-09 cycle for all ABA schools listed on both LSN and LSP. The version of LSP tested was Version 2.4 (current as of July 30, 2009). A big thanks goes out to CyLaw for compiling the data from LSN; I couldn't have done it without CyLaw.

The admit rate was calculated by taking the number of admitted applicants divided by total applicants for each respective prediciton category (total applicants includes admitted, waitlisted, and rejected applicants, but does not include those who status was listed as "pending"). The prediction categories used in testing accuracy are the same as LSP's prediction engine with the exception of early prediction (which was not tested); predictions are adjusted to account for splitter-ness, weak GPAs, and URM status.

Target admit rates, based on how LSP renders predictions:
When LSP said Admit: >= 87%
When LSP said Strong Consider: ~ 69%
When LSP said Consider: ~ 50%
When LSP said Weak Consider: ~ 31%
When LSP said Deny: <= 13%

Based on the decisions processed so far:
When LSP said Admit, 8402 instances: 87.0%, +0.0%
When LSP said Strong Consider, 4842 instances: 78.0%, +9.0%
When LSP said Consider, 10066 instances: 50.6%, +0.6%
When LSP said Weak Consider, 3646 instances: 22.8%, -8.2%
When LSP said Deny, 6754 instances: 10.0%, -3.0%

Difference between actual admit rate and predicted admit rate: (prediction categories weighted evenly)
Average rate: -0.3%
Median rate: -3.0%

Summary:
Prediction categories that were right on the money: Admit, Consider
Prediction categories that were reasonably close: Deny
Prediction categories that were somewhat off: Strong Consider and Weak Consider

A closer look...
A possible explanation for the Strong/Weak Consider results is that the chance of being admitted (based on admission index scores and LSP adjustments) should look like a parabolic curve (since it should be a normal distribution), and, based on these accuracy results, a curve with a high and narrow peak in the center (the roughly median applicant).

At any rate, I'll do some more number-crunching before schools starts, especially in regard to URM applicants and splitters, and compile a more official report that'll go up on LSP.

Was this done for part time as well?

*edit*

And Early Decision? Did you test it with that as well? Thanks!

[YCrevolution]

..

[SilverE2]

Ladies and gentlemen, I give you the accuracy statistics for the Law School Predictor.

These results are based off of more than 33,500 LSN-listed law school admission decisions from the 2008-09 cycle for all ABA schools listed on both LSN and LSP. The version of LSP tested was Version 2.4 (current as of July 30, 2009). A big thanks goes out to CyLaw for compiling the data from LSN; I couldn't have done it without CyLaw.

The admit rate was calculated by taking the number of admitted applicants divided by total applicants for each respective prediciton category (total applicants includes admitted, waitlisted, and rejected applicants, but does not include those who status was listed as "pending"). The prediction categories used in testing accuracy are the same as LSP's prediction engine with the exception of early prediction (which was not tested); predictions are adjusted to account for splitter-ness, weak GPAs, and URM status.

Target admit rates, based on how LSP renders predictions:
When LSP said Admit: >= 87%
When LSP said Strong Consider: ~ 69%
When LSP said Consider: ~ 50%
When LSP said Weak Consider: ~ 31%
When LSP said Deny: <= 13%

Based on the decisions processed so far:
When LSP said Admit, 8402 instances: 87.0%, +0.0%
When LSP said Strong Consider, 4842 instances: 78.0%, +9.0%
When LSP said Consider, 10066 instances: 50.6%, +0.6%
When LSP said Weak Consider, 3646 instances: 22.8%, -8.2%
When LSP said Deny, 6754 instances: 10.0%, -3.0%

Difference between actual admit rate and predicted admit rate: (prediction categories weighted evenly)
Average rate: -0.3%
Median rate: -3.0%

Summary:
Prediction categories that were right on the money: Admit, Consider
Prediction categories that were reasonably close: Deny
Prediction categories that were somewhat off: Strong Consider and Weak Consider

A closer look...
A possible explanation for the Strong/Weak Consider results is that the chance of being admitted (based on admission index scores and LSP adjustments) should look like a parabolic curve (since it should be a normal distribution), and, based on these accuracy results, a curve with a high and narrow peak in the center (the roughly median applicant).

At any rate, I'll do some more number-crunching before schools starts, especially in regard to URM applicants and splitters, and compile a more official report that'll go up on LSP.

Was this done for part time as well?

The LSN data included everyone (FT and PT), although I only tested the LSN data against the LSP FT predictor (I only had access to the things you'd see on a LSN graph; so no way to differentiate between ED, PT, scholarships, etc.).

Gotcha. Thanks!

[OperaSoprano]

I might be misreading the data, but it appears that the people who've self selected to report their numbers have done much better than they "should have." Are we all that much savvier than the average law school applicant?

In particular, I'm thinking of applicants who want to attend schools in their strong consider range. In my experience, if someone is able to demonstrate that he or she really wants in at a strong consider school, that person will very likely be admitted. This is what happened in my own cycle, and I've seen it happen over and over again.

Is there any way to account for "persistence" factors like visiting, interviewing, Why X letters, and LOCIs? IMO these can often trump softs, and at the end of the cycle, when WL decisions need to be made quickly, they can occasionally even trump numbers. Deadatheist's recent admission to Hastings (160/3.63, non URM) is a perfect example. She worked for this, probably even harder than I worked for Fordham, though LSP called her a weak consider. I believe her persistence must have been a contributing factor in her WL admission.

I'm not sure how you could quantify this, but do you think it's credited?

I mean, I'm sure you remember the dramatic mess my cycle was, and I was decently above the median.

[YCrevolution]

..

[jackassjim]

I know nothing about forecasting, but I can't help but feel that your results are just too good to be true. I'm not sure if it's standard procedure to test the predictive validity of a model on the very data that was used to compute. It just all seems "over-fitted" to my taste. You would probably need to first compute a model using data for a specific cycle, and then test to see if it is helpful in predicting admissions results in the second cycle. (Please tell me if I'm way off here).In essence, this would mean that you can't test the predictive power of your model using the same data you used to build it.

What you can do, however, is check how much of the variance your different models can explain. That would require you to report a statistic like a pseudo-R2. Maybe the average pseudo-R2 for all the models you calculated for each school?

In any case, I was wondering: What kind of model is this anyway? A simple logit or probit? Are you interacting variables, or is it simply additive?

[CyLaw]

I don't believe that the data model for LSP was built with the LSN data, as YC did not have that data when he created the model. The LSN data was only recently used to test the LSP Model. I believe.

[jackassjim]

I don't believe that the data model for LSP was built with the LSN data, as YC did not have that data when he created the model. The LSN data was only recently used to test the LSP Model. I believe.

Good catch. I really dropped the ball on that one.