HNGlance

Top Best New Show

Ask HN: How could you make/find a spider for JSTOR articles?

https://news.ycombinator.com/item?id=5065883

by corporalagumbo • 14 years ago

1 1 14 years ago

NE

news.ycombinator.com

Related Stories

Loading related stories...

Comments

1

venomsnake• 14 years ago

Ask the federal prosecutors in Boston - they have a lot of information gathered already.

On a not so sad note - writing a high performance fully async spider is a trivial task so it is just a matter of guessing the urls

http://www.jstor.org/stable/i40055831 - this seems to be the issue format just regex for them

there the articles url are in

http://www.jstor.org/stable/41237135 this format. So parse any page for those links and keep already crawled and html downloaded.

Source preview

news.ycombinator.com

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms