Ask PG: Could YC admissions be replaced with a very small shell script?

https://news.ycombinator.com/item?id=627103

by dfranke • 17 years ago

17 12 17 years ago

news.ycombinator.com

If you train CRM114 (or some other classifier) on all but the most recent batch of YC applications, how does it fare at distinguishing interview vs. no-interview applications in the most recent batch? How well would it have to do in order to be alarming? :-)

There's no particular reason that I'm asking this other than that I just finished retraining CRM to take advantage of the version upgrade and was pretty impressed with the training results.

pg• 17 years ago

We did try this once on startup school applications (after the fact) and it was a pretty bad predictor.

If we ran code like this on YC applications, I suspect the most useful way to use it would be to find groups we hadn't been planning to invite to interviews that deserved a second look.

dfranke4• 17 years ago

Huh. For startup school applications that's actually a surprising result. I was under the impression that the admissions process on that was basically just to weed out the hackers from the business people, and with an application that brief I'm surprised that can't be accomplished with a keyword search.

Alex3917• 17 years ago

You mean I changed my name to Lisp MacErlang for nothing??

But seriously, the Startup School application is basically just a list of keywords, at least the way I understood it. Are there really people writing essays in there?

fizx• 17 years ago

I imagine that YC apps reflect the startup trends at the moment. An online video company that quite possibly would have been funded in 2006 would be dead in the water today. With six months between applications, and not that much historical data, this could be hard to compensate for.

SwellJoe• 17 years ago

YC tries to choose teams, not ideas, so in an ideal world they would fund the same team in 2006 as in 2009. So, it would seem like the idea parts of the application would already be more about the way the applicant talks about their idea rather than what the idea actually is. Or, perhaps one could simply drop the "idea" fields from the data and only train on the "about the team" fields.

Nonetheless, I'd have to see evidence to believe that applications could successfully be classified automatically with any reliability, just because it's such a complex problem. The reason spam and log data and experimental data is classifiable using tools like this is that it is, by it's very nature, relatively predictable. If every spam were written fresh by a different person, and selling a completely different product, it would be impossible to filter it. Likewise, if every time Apache got a new request or error it made up a little prose on the spot about the topic, it'd be pretty difficult for an automated tool to make any of sort of sense out of it.

ivankirigin2• 17 years ago

You could open up the dataset by assigning every word an id, and giving an anonymous dataset of applications in code.

dfranke4• 17 years ago

You could probably extract a good deal of information from this by examining word frequency and sentence structure. Or at least, the attempt would be too much for me to resist.

Ask PG: Could YC admissions be replaced with a very small shell script?

Related Stories