How to Crawl the Web Politely with Scrapyhttps://medium.com/@ScrapingHub/how-to-crawl-the-web-politely-with-scrapy-15fbe489573d#.897np3gy7 by unsettledtck • 10 years ago 1 0 10 years agoMEmedium.com