Law School Predictor: The Thread"

[SamSeaborn2016]

paratactical created this playing card for LSP:

Awesome. Although, I have to say the only acceptance I have thus far is from a "weak consider."

[YCrevolution]

..

[YCrevolution]

..

[Cavalier]

Hopefully the winning bidder isn't a 144/2.8 who's insistent that he get into Harvard.

[tomhobbes]

Hey, I just noticed that LSP says that a 4.0/180 is only 91% at/below at Yale. How does that happen?

[AngryAvocado]

Hey, I just noticed that LSP says that a 4.0/180 is only 91% at/below at Yale. How does that happen?

Well since LSAC GPA goes all the way up to 4.33, I'm guessing it just means ~9% of admitted students have >4.0 and an LSAT above 178 or so. Considering it's Yale, seems pretty possible to me.

[YCrevolution]

..

[YCrevolution]

..

[treple]

Version 3.0 Likely Features
- User selection of a specific URM race
- State residency feature for state law schools
- Significant work experience feature

Nutshell: Current tool good. URM predictability mediocre at best. Future plans: Horrible.

Analysis:

Alright, thus far watching this program progress has been completely logical. It started as a simple calculator based on indexes as well as some predictive indices to 'fill in the gaps'. Then got significantly more advanced through a compilation of LSN data, a move that made the program useful given that indices create very broad bands for accept, consider, reject. Further, it differentiated the program from other predictive tools by correlating LSN data with indices; something that had not been done before. Then an URM factor was added which used data from LSN.

This is where things start to get a little hairy because we're not correlating data off of 35,000 data points, but a fraction (13%) of that data. Assuming a roughly equal distribution of URM for each school, that correlates to between 20-40 data points per school being used to make a statistical correlation. A corollary assumption could be made that an uneven distribution of URM applications occurs for schools such that some schools may have 10 data points with others having 100. This provides an equally poor result without some indication of what schools are unevenly represented and therefore likely to be predicted better or worse (here's a feature that could be added). A claim on these results is made that that LSP may actually be a 'better' predictor for URM's than for average applicants. This claim is just absurd and incredibly misleading. First, of course they will accurately predict past results; they are fit to past results. Second, and again, obviously a statistical tool is going to be better at predicting a result based on fewer data points than more. I could fit a 100th degree polynomial to 10 points and it'll give me a correlation of 1, but fit to 1,000,000 points and that correlation won't be quite as good.

Which takes us to these future plans. As of now, there is no data available outside of LSN regarding applicants cycle. There are no indices or other data forms that provide insight into admissions URM/Work experience etc. I don't have the same assembled data that you do, so this is somewhat speculative but I would surprised if I was that far off. You are going to now begin developing predictive statistical tools based on what is likely too small of a sample size. The URM data was already an incredibly small sample, breaking that data down into subcategories and you're going to be predicting people's admissions chances on 10 data points. Of course you could add data from previous cycles, but you run into a trade-off there where the further back you go, the less reliable the results. You've stated that as of now you don't have the data for this, but to list this as a future plan with an estimated launch date in 2010 is disingenuous without any means of solving this shortcoming.

With regard to work experience. I would LOVE to know how this is even possible to incorporate outside of some arbitrary additional numeric 'boost'. How you would even develop a reasonable method for determining that 'boost' is beyond me. There is just no statistical way to incorporate work experience into a model that is developed by being fit to data points. How do you discern between 5 years of employment at McDonalds and 2 years at a Hedge Fund? Oh, and again, a shortage of data is again apparent here.

You state that an ability to interprete the results is key to using your tool. Perhaps you could consider adding a "reliability of data" category. This would provide some indication of how much data the result is based on. I would strongly contend that no one will be able to interprete the results otherwise from these modifications because there simply isn't enough data for a valuable correlation.

Another future endeavor you should probably embark on is to take your tool as developed based on 2008-2009 data and verify the reliability based 2009-2010 results. This would be the best way of solving your problem that you're testing it's predictive ability on data used to generate the predictive tool. You would just need to make a script that input 2009-2010 data into your tool and output a result. You could then compare how your tool did versus actual results and make adjustments to the algorithms. Instead of broadening the areas you're predicting, make your current tool better.

[YCrevolution]

..

[treple]

Apologies for the hostile tone that post seemed to take. I really do like what you've done with the spreadsheet, but think that there is still a bit of work to be done to make it a valuable tool in its current form.

[jaskat]

Are you saying it's mediocre in predicting URM cycles in favor of the URM or against the URM?

[kittenmittons]

+1 to treple's post. Don't overreach.

[vanwinkle]

Apologies for the hostile tone that post seemed to take. I really do like what you've done with the spreadsheet, but think that there is still a bit of work to be done to make it a valuable tool in its current form.

I'll agree with this. I didn't want to agree with you earlier personally because of how hostile it did sound initially, but you do overall make a good point. Refining and verifying the accuracy of the predictor is a lot more important than trying to add new features to what's already there.

[YCrevolution]

..

[Tangerine Gleam]

YCrev, I agree that you shouldn't go too far. GPA and LSAT are (obviously) quantifiable, but the proposed additional options just seem a bit too lofty. You've got a great thing going; something based off of lots of hard data which will keep pouring in. As it stands, LSP is an incredible tool which seems quite accurate. If you add these other complicating bells and whistles, it might increase the likelihood of inaccuracy and undermine your program's credibility. Risk/reward seems iffy...we're talking a ton of extra work (I assume) for some features which may not even improve the site much (if at all).

Residency bump makes some sense if it can be verified by data, but work expeience seems way too murky (not to mention much harder to collect LSN data on).

[pissantvache]

I agree as well.

However, I would be interested to see actual charts of the schools and the trendlines that you're putting together, somewhat akin to what's available on Chiashu. LSN is horribly limited in that regard, and, while it's good at aggregating data, it's not infrequently difficult to get a more nuanced sense of the topology of schools' cutoffs.

Features that come immediately to mind are: ability to filter datapoints based on application results, ability to compare lines-of-best-fit across schools (and create lines between the 5 results you have (deny, weak consider, etc.)), ability to look at all URM results only, etc. I don't envision you replacing LSN, but creating a better way to view the data in context, graphically, perhaps in partnership with LSN, seems like a huge boon that can only help the applicant community.

[YCrevolution]

..

[pissantvache]

Sure thing. I was just throwing out some ideas. Data is all well and good, but being able to portray in an effective way is super important, and that's a key component that LSN is notably lacking.

[laidoffjournalist]

I still think that merit payouts should have hard data and numbers available, and it would be something everyone would be interested in.

[YCrevolution]

..

[laidoffjournalist]

We all want a piece of that money pie! It annoys me that some people on LSN don't apparently indicate they are URMs... when I see someone with really low stats getting acceptance or getting a crapload of money, but a relatively blank profile, I have to assume they must be a highly sought-after URM. That or they have the most amazing story everrr. I don't know who runs LSN but I wish certain aspects were a little more reliable. Self-reported stats will never be perfect though.

LPS is still my favorite calculator. The colors are so pretty.

[the lantern]

I haven't been keeping up on LSP updates, but since I am bored out of my mind I decided to plug my numbers in again for fun. If I remember right, this bad boy was INCREDIBLY pessimistic about my chances. Today, it seems to be showing me a lot more love. Whatever changes you made are definitely more splitter friendly, so I hope that its accuracy is still close, cause it just made my day a whole lot brighter.

[LSHPFL10]

This probably sounds stupid, but I cannot figure out how to agree to the terms and conditions.

Anyone?

[YCrevolution]

..