People parrot this point a lot but it doesn't make any sense. The average SAT/ACT at the Ivy's are near perfect, the number of mediocre rich kids paying their way in is a small fraction of the student body.
And yes test scores aren't the best metric for "talent" but it is one of the better signals you get in a college application.