A preview of Gmail’s new look 15 years ago
another dick bar :(
HN user
esryl
5 karma
Posts0
Comments2
No posts found.
Google Bot Attempts to Crawl Shortest URLs First 15 years ago
The site adheres to a strict url structure. /state/city/id/schoolname - entering from the homepage, the only way to crawl the site 1 level at a time would be crawling the shortest urls first. this structure is also emphasised in the breadcrumbs on every page, the shortest urls are also the ones with the most internal links.
why would you crawl the site in any other way?