Google Play doesn't have an API. So I built one.

by jonursenbach • 14 years ago

52 20 14 years ago

Do you run into any query limits with aggressive scraping? A colleague wrote an android market scraper a year or two back and she was repeatedly stymied by limits on the number of queries she could make to the store before getting cut off.

jonursenbach4• 14 years ago

We don't do any aggressive scraping at the moment, so I can't really answer to that. The way our backend stuff works is that when we want to add an app to the site (http://gdgt.com/best/apps), we query for the search results, and then grab the app we're looking for.

charliesome• 14 years ago

https://github.com/pastfuture/MarketBot/blob/master/App/Andr...

    \PastFuture\MarketBot\App\Android\AmazonAppstoreApp

Welcome to Modern PHP.

TazeTSchnitzel• 14 years ago

If you want to add iTunes, here's an article on how to get the iTunes Store HTML from PHP:

http://hactheplanet.com/blog/5

jonursenbach4• 14 years ago

Yeah we're currently using their JSON API, it just hasn't been migrated over to this library yet.

kanzure4• 14 years ago

I see, it's based off of:

http://github.com/chadrem/market_bot

Edit: I would like to point out that Google Play actually does have an API. It uses protocol buffers, if you check the Google Play apk.

http://github.com/kanzure/android-market-api-py

jonursenbach4• 14 years ago

It's not official though, and the results you get back from it are incredibly varied.

kanzure4• 14 years ago

> and the results you get back from it are incredibly varied.

Can you show me? I haven't noticed.

est2• 14 years ago

Oh god parsing HTML pages.

Why don't Google Play provide JSON API like iTunes do?

http://itunes.apple.com/lookup?id=343200656

and alternatively

http://itunes.apple.com/lookup?bundleId=com.clickgamer.Angry...

The second link can be used in client app to self-checking for new version updates.

Evbn• 14 years ago

There are tons of Android Market website clones. How do they get their data?

est2• 14 years ago

apps are likely to be published on G Play first, then to 3rd party markets.

If an app is exclusively published on 3rd pary market, then it's likely distributed directly through apk , there's no way to identify them using a centralized way.

jeffehobbs• 14 years ago

Bon fortuna!

gailees• 14 years ago

Like a BOSS.

How about the Apple Store?

SwaroopH• 14 years ago

Look above [1], they already have a JSON output for apps.

[1] http://news.ycombinator.com/item?id=4634634

krob2• 14 years ago

I did something like this, but using xpath queries, only problem is when they decide to change these, you will have "fun" times retesting all the dom locations to fix your scraper when the time does come when it breaks.

kanzure4• 14 years ago

> I did something like this, but using xpath queries, only problem is when they decide to change these

That's a strong reason for going with a library that is used by multiple users (more people to distribute the workload over). But more realistically, you can just use the existing API endpoints because there are client apps in the wild that rely on those APIs working the same way-- at least until Google sunsets Google Play.

krob2• 14 years ago

there was an API which i found a while back, but it didn't work at all, all the unit tests it had failed, so my guess was that google changed their api once again and the project was worthless. The solution i came up with was the only reliable one i could build. It was only like 6 lines of code.

aphidbyteinc• 14 years ago

Nice.

Google Play doesn't have an API. So I built one.

Related Stories