Show HN: Scrape.it – Change-Resilient Web Scraper

by notastartup • 12 years ago

16 16 12 years ago

dailen2• 12 years ago

Holy crap that's expensive!

notastartup4• 12 years ago

There's no metering (https://scrape.it/tour2) so you can create as many web scrapes as you want. Scrape.it constantly monitors each job to make sure the data extraction doesn't get interrupted when website changes. Should the website change, your jobs get updated automatically to continue working.

dailen2• 12 years ago

It's an impressive product, there's no doubt about it. I just happened to be looking for a WYSIWYG like tool for scraping a page for changes. This seems more like a commercial grade product than a home user hobby haha nice site btw!

warkdarrior2• 12 years ago

This is ripe for resellers to move in and make some dough.

notastartup4• 12 years ago

Can you describe what you mean by resellers?

jbob2000• 12 years ago

I can imagine some scenarios where you would use a web scraper, but I'm curious; What are people actually using a web scraper for? Does anyone have one in production?

agersant• 12 years ago

I used to run one that I made for displaying a nice view of all the art posted on conceptart.org (a very large message board which I found cumbersome to navigate).

Mandatum• 12 years ago

I use one to scrape session times from local theatre websites. They don't have the capacity to build an API so I have an agreement with them that they keep the formats the same. I've set up a script which alerts them if they've screwed it up.

martin-adams• 12 years ago

My product MyShopData.com uses web scraping for retailers to extract their own data and integrate with marketplaces.

ryeon• 12 years ago

For simple web scraping, I find that kimonolabs.com does a perfectly fine job.

notastartup4• 12 years ago

I love what they are doing with kimono and import.io

There's no free lunch lets put it that way. It's free, it's simple but limiting for doing anything heavier. It covers only a small portion of the websites. You can't crawl all the links in a website and hard to scrape data from dynamic webpages etc. Also I found that some websites wouldn't even load making it impossible to define the fields to scrape.

davidbarker• 12 years ago

I tend to stick with KimonoLabs, but I did try http://parsehub.com for a while. It's a lot more complicated, but it allows you to scrape dynamic sites.

pablohoffman• 12 years ago

We released a similar open source tool for visual scraping, earlier this year, called Portia: https://github.com/scrapinghub/portia

It's been getting quite a bit of traction and we're currently working on the integration with Scrapinghub platform (disclaimer: I work there) for those who prefer a hosted version.

Show HN: Scrape.it – Change-Resilient Web Scraper

Related Stories