Search Goes Real-Time With Scoopler (YC S08)

http://www.techcrunch.com/2009/05/08/search-goes-real-time-with-scoopler-twitter-dominates-results/

by rokhayakebe • 17 years ago

67 35 17 years ago

pclark2• 17 years ago

Cool. But until there are a lot of real time services their value is very limited.

dilanj• 17 years ago

Going by the current trend, there will only be more real time services in the future. It pays to be early in the game :)

axod2• 17 years ago

Yeah. Soon I can imagine a time when we could talk to each other online in real time. Like twitter but without a delay. But maybe that's a couple of years off.

ikayak• 17 years ago

Good execution on the trending and the real power here is the deep search on live content. Anybody who does search right has value.

ajasver8• 17 years ago

Thanks! We're working on adding more data sources to provide broader results.

okeumeni2• 17 years ago

I have one question, are they consuming other companies search API, let’s say Twitter or are they crawling and aggregate their own data? Either way it’s a great job!

ajasver8• 17 years ago

We actually crawl and index all the data ourselves. That's how we identify the most popular content.

okeumeni2• 17 years ago

Nice Work!

hachiya• 17 years ago

Is this truly crawling the sites, or querying each site's search service?

e.g. for Twitter, are you crawling all the Twitter users' URLs (seems difficult to do - how do you find new accounts' URLs?), or are you fetching results from search.twitter.com ?

vaksel• 17 years ago

isn't Google pretty close to real time anyways? I know I tend to find things on Google that I just posted 5-6 minutes ago.

ajasver8• 17 years ago

Yes Google is pretty fast for newly published static content. However, it's difficult to use for events happening right now. From the feedback we've got, people like real-time results because its engaging and effortless, a little like watching TV.

BenS• 17 years ago

This is really cool - congrats. I'd really like to see google-trend-like features but on much shorter time scales. Even a weekly-zeitgeist tied to news stories instead of having to wait for the end of year.

ajasver8• 17 years ago

For the most recent trends you can check out the hot topics. We also want to do news updates regularly - follow @scoopler and you'll get updates soon!

diego• 17 years ago

In case you haven't seen it, we do that for Twitter: http://twist.flaptor.com/

anuraggoel• 17 years ago

What is Scoopler built with?

ajasver8• 17 years ago

Scoopler is written in Python. We use Cloudant (YC s08) for storing our data. I'm planning to write a blog post about this on soon on our Posterous.

kyro• 17 years ago

Cool. I'll take a note of that using Snipd, upload it to Scribd, then post it to Reddit, then tell my friends to upvote me using Heysan or maybe Chatterous. It depends on whether I'll have enough time, since I'm always playing on DraftMix.

Anyway, good luck with Scoopler. :P

datums• 17 years ago

Nice start. I was expecting more from the twitter feed. It's an rss feed from a query. Maybe a better way is to group similar real-time results ? v2

edawerd• 17 years ago

awesome job dude! It would be awesome to have some sort of real-time alert feature, too!

ajasver8• 17 years ago

It's in the pipeline! Thanks.

hamai• 17 years ago

"One element that’s really nice, is the Peek feature, which allows you to take a look at pages being linked to in an overlay on top of the results, so you don’t have to visit the actual page."

Real-Time news for the writer: you actually visit the page. It's an iframe...

In the old days, there was something called target attribute. Neat when the target is not iframe friendly.

snprbob86• 17 years ago

This feature is really curious to me... why would I want to peek when I have a back button on my mouse?

Search Goes Real-Time With Scoopler (YC S08)

Related Stories