Ask YC: How to web scrape 100s of sites/forms?

hi...

a potential research app requires the scraping of a few hundred websites/forms and diving into the child links to obtain the linked/parent structure ie. company->dept->title->name.

in this case, this would involve going 4 levels deep, and getting the required information.

so, does anyone know of a method/app/company that can be used to accomplish this. orm am i going to have to figure out how to get a number of cheap guys to write a bunch of python scripts!!

thanks