This is/was also my take. I’m skeptical that a probe-based network can be granular enough to reliably pinpoint a city, especially when some paths are much better connected than others (fewer hops, uncongested fiber, no throttling).
However, ipinfo still appears to rely on active probing to triangulate geolocation data, which suggests they believe these routing asymmetries can be modeled or averaged out in practice.