HNGlance

Top Best New Show

Beer and Concurrent HTTP Pipelines

https://damienradtke.com/post/beer-and-http-pipelines/

by kiyanwang • 10 years ago

2 1 10 years ago

DA

damienradtke.com

Related Stories

Loading related stories...

Comments

1

hrbrmstr• 10 years ago

Rather than emoticon-ing away your responsibility in a potential site DoS, perhaps read up on responsible crawling - https://webarchive.jira.com/wiki/display/Heritrix/Responsibl... + http://blog.mischel.com/2011/12/20/writing-a-web-crawler-pol... . You had no need for either unlimited or even 50 parallel crawl tasks. And, if your intent is anything but personal use, you shld prbly (re)read https://www.brewtoad.com/legal since you just got their attention in a pretty big way.

Source preview

damienradtke.com

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms