The short, stupid answer is: transistor count and size.
Explaining it in detail requires more background in electronics, but that's ultimately what it boils down to.
High-end analog front ends can reach the three-digit GHz (non-silicon processes admittedly, but still).