Paul: I think the problem is that you're measuring something slightly different from what everyone else wants to measure, and thus the claim in your first paragraph winds up being too strong.
One way attending an elite college helps is that it makes it easier to learn--both in that the resources are more obviously available, and in that the institution is pushing you to do work. You can graduate from Northwester Louisiana University going out drinking six nights a week and never really doing the work; you can't pull that off at Harvard. But if you're driven enough, 'easier' doesn't matter all that much; if you're determined enough, you can learn to the limits of your own abilities at Harvard or at NLU. Or, hell, you could pull a Ramanujan and teach yourself entirely out of books.
The problem is that most people aren't that driven; most people will get a lot more out of the college where it's easier to learn (I know I've needed the push more than once to keep going aggressively). But those people aren't the ones you're looking for at Y Combinator. If you need that kind of push, you're not going to succeed at a startup, because a startup requires drive and determination more than almost anything else. So your evidence says something like, "in the set of people to whom the institution matters the least, the institution doesn't matter very much." Which is probably true, but doesn't say much about the people who are going to settle down into a nice comfortable job when they graduate (doctor, lawyer, middle management), and aren't driven enough to go for broke the way you did.
Also, institution matters much more when you're doing something where letters of recommendation make a huge difference (say, applying to grad schools, like I am). When my profs write a grad school and say I'm in the top 25% of the math majors here, or whatever they wind up saying, that tells programs something; they've accepted students from my college before, and they can compare me to those other students to get an idea of how good I am. Sometimes this is explicit; I think one of my profs has written a rec letter along the lines of, "this student is about as good as student X whom you accepted three years ago." This gives them very precise data about whether they want me. But if I were going to a weaker institution, they wouldn't be able to get that sort of gauge of my ability; this makes me a riskier pick, and so I'm less likely to be accepted. But once again, this shouldn't factor much into your particular dataset.