Why you should not pirate Google’s geo APIs

http://petewarden.com/2013/09/09/why-you-should-stop-pirating-googles-geo-apis/

by spenrose • 13 years ago

309 113 13 years ago

Here's another reason that I wish was more prevalently discussed: it's not okay to violate Terms of Service anyway. Don't accept the terms? Don't use the service. We'd all have a field day if "big brother" violated the terms of an open-source license, but the freedom we enjoy in the community comes with a responsibility to honor our agreements.

Killswitch3• 13 years ago

I just wish people would stop being so self-entitled period.

TallGuyShort4• 13 years ago

Agreed. The tone of the article is correct, in my opinion, because this really does hurt the community. I think we'd have an easier time fighting things like SOPA, DRM, etc. if actual piracy wasn't such a big problem.

baddox• 13 years ago

I feel like I deserve to live in a world with fewer self-entitled people.

ianstallings2• 13 years ago

As my developer friend likes to say - this is why we can't have nice things.

revelation• 13 years ago

Violating the terms of a license is very different to violating the Terms of service you "sign" with a button click.

TallGuyShort4• 13 years ago

Actually I'd argue that you're more obligated to abide by something you explicitly checked "I have read and accepted..." than something that was listed somewhere in a file you downloaded with some other stuff. You probably just happen to like the terms of the GPL or ASL more than you like the terms of Google's API. I do too - but at what point are we just losing all integrity because we're too lazy to read a long document we have no intention of actually agreeing with?

ajtaylor2• 13 years ago

I really don't see the difference. Either way you're agreeing to abide by a set of rules by the owner of the service. If you don't like their rules, use one of the alternatives.

wisty• 13 years ago

OK, so on the one hand there's white (or grey) hat cases, and on the other there's spammers and scammers. Both break the ToS.

I think a big difference is whether it's for commercial purposes, or has some evil motive. Eating KFC in McDonalds is probably breaking some rule, but no-one in their right mind would really care. Selling your own fried chicken in McDonalds is much more likely to lead to trouble. A political stunt is somewhere in between, but should generally be given a bit of lenience.

So I'd say it should be OK to use a script to automate some personal task. Creating a business which relies on breaching a ToS is both wrong and stupid. And while political hacks might be a bad idea, they might also be the right thing to do.

Here's the thing about the law - it's not quite black and white. A judge or jury will always consider the circumstances, when they decide what to do with you. The same is true about companies blocking you. Sometimes, you need to use common sense, and hope that other people do to.

lmgftp• 13 years ago

This really could use more exposure. People who believe that because they only had to click "Accept" they may claim ignorance or stupidity/absurdity of the terms the accepted ought to be corrected. It's simply silly to think you may violate the terms of any agreement made, especially when that agreement is necessary to your use of the service (hence "terms" of service).

While it is clear that hacker culture embraces putting things from numerous sources together, in this case the Google Maps elevation API with crime data, it's one thing to make a cool presentation and another to make a cool presentation derived from someone else's work against their explicit wishes, expressed in the terms.

The use of software implies (by necessity or otherwise) that you wish to make/create/analyze/do-X with the software, and by using it you've accepted the fact that you will do-X, but within the terms.

It's a funny world we live in ;)

sillysaurus2• 13 years ago

If they wish people not use it in that way, then they should make it impossible to use in that way, or ban those people from using it.

Those are really the only two options, no matter how much we wish people would be honorable.

mikeash5• 13 years ago

Personally, I think it's silly to think that you may be bound to the terms of an agreement just because you clicked a button that says "Agree", without any of the fundamental attributes of a contract being present, and without even interacting with a representative of the other party in any way.

ianstallings2• 13 years ago

I've said the same in the past and usually it's to the business managers or executives. Too often they are willing to conveniently overlook licensing or service restrictions in order to get what they want or need.

A specific example - I had to convince executives that we needed to pay MPEG LA because we were using FFMPEG to encode and decode using 264 on our servers. I only convinced them after a long debate and contacting our legal department to explain the risk. It had very little to do with my personal views on licensing or patents and everything to do with professionalism. A lot of the other software developers thought this was wasting money and time, and paying off people that were somehow evil because they charge large sums for using their patented software. My stance has always been if yo don't like the terms use something else and avoid the ethical dilemma. It's just plain easier and avoids some risk.

chatman• 13 years ago

Everyone should try OpenStreetMap based APIs, e.g. MapQuest Open.

potatolicious2• 13 years ago

I tried one part of the API recently - I'm building an app that's partially intended to give local cycling directions.

Now I'm between a rock and a hard place. One the one hand some parts of the MapQuest Open API are extremely bad, on the other hand Google has an all-or-nothing license for their geo data, and their native iOS maps component sucks.

MapQuest's cycling directions in Manhattan is worse than having no directions at all. They have a specific cycling flag you can set, but results will blatantly ignore bike lanes and send you down extremely wide avenues with no bike lanes and heavy traffic. IMO using MapQuest's cycling directions, at least in NYC, is an extreme hazard, and I can't in good conscience integrate it into my app.

graue• 13 years ago

How do the directions compare to YourNavigation?

http://www.yournavigation.org/

Despite the clunky interface, I was pleasantly surprised by its biking directions: not as good as Google, but not terrible either. (I was in NYC when I tried using it for awhile.) It's fully open-source.

Pxtl• 13 years ago

Yeah, I was doing some tinkering with OSM and found a lot of that kind of additional data (bike trails and such) to be missing. Obviously I should just roll up my sleeves and contribute to OSM, but it still means that I wouldn't build an app or do large research that depends on those kinds of details with OSM as the back-end, knowing how much stuff is missing.

And yes, more municipalities need to start releasing open data.

cmelbye• 13 years ago

For what it's worth, they're currently very poor quality to Google's geocoder.

ajtaylor2• 13 years ago

MapQuest Open is very hit or miss. In Australia, the directions & geocoding are pretty good. However, in India it's atrocious - but still good enough for what we need.

jacalata• 13 years ago

His site http://www.datasciencetoolkit.org/ certainly looks interesting! One of the most impressive kinds of arguments - "I think you should stop doing x and here are the resources y I've created/curated to make it easier for you to do so."

jack-r-abbit• 13 years ago

I was thinking the same thing. Lots of people get the first part. But not so much with the second part.

dewitt• 13 years ago

FWIW, Google doesn't particularly like it when people use APIs outside the terms, either. Having to take down an app is never fun or enjoyable for anyone involved. And the cases in which the terms aren't programmatically enforced right away isn't necessarily a choice; often it is difficult to differentiate at scale between legitimate usage and disallowed usage.

And while I don't know how every other company writes their developer Terms of Services, I do know that in Google's case we measure each word carefully and try hard to strike the right balance between offering as much as feasibly possible, while trying to create something long-term sustainable (so even if the usage grows far beyond our initial goals, we're still in a position to keep them available). We're hardly perfect at it yet, but we're certainly getting a lot better over time.

Also worth noting, often times these terms are designed to honor upstream data contracts, so if for no other reason, it's worth trying to respect them because it's the right thing to do for the other relatively small parties, too.

Mithaldu2• 13 years ago

I would have some sympathy with this if you made it easy for single people to pay you to be able to use your data in a manner outside of the ToS of the free APIs.

However, for the Elevation API there are two choices:

  * free and heavily restricted
  * business api, which looks extremely sketchy †
    and is blocked to most people

The gaming world has learned this with Steam and friends already. People will pirate the hell out of digital offerings that are not available in a reasonable manner. Steam recognized this and made games available in the most convenient manner humanly possible and are making money hand-over-fist. Sure, there are still pirates in gaming, but those are the people who are either paranoid about Steam or those who truly couldn't afford it anyhow.

You're still in this pre-realization phase and have only one single entity to blame if people "misuse" your APIs: Yourself. Make the APIs available for normal people in a reasonable manner and you will find adherence to your ToS becoming the norm, not the exception.

† By sketchy i mean the facts that there is zero pricing information, it's a buzzword-slaughterhouse, and the only way to get at it is to fill out a form with a lot of information, an indication of what one might want to do and which offers no further information or even actual flesh-and-blood human contact.

joshma• 13 years ago

I don't understand this feeling of self-entitlement - who are we to judge whether or not an API has been offered "in a reasonable manner?" I'm guessing it's very resource-consuming to collect, process, and maintain the data necessary for geolocation APIs (especially given the upstream providers).

It'd be one thing for Google to be deceptive about it, offer a free service, and try to upsell you every point along the way. It's quite another thing for Google to dedicate an entire section in the API intro page [1] about Usage Limits.

Saying that "Google has nobody to blame but itself" is interesting when the blame is usually from developers to Google, not the other way around.

[1] https://developers.google.com/maps/documentation/elevation/

bluman• 13 years ago

In the specific example of the elevation data set, the data for the United States looks like it's public domain (USGS DEMs). The rest of the world may be more difficult to get high resolution but there is STRM which is reasonable ok for a lot of uses.

dll38242• 13 years ago

Really, don't bother trying to contact them, it's not worth it. If anything their business API is less sketchy than their actual sales tactics.

We had two different conversations with them and the first time the rep quoted us a rate that worked out to $72 per 1000 map impressions. The second time we spoke to a different rep who, when asked the differences between the free plan and the business plan, told us that if we were currently on the free plan outside of the ToS there would be "consequences".

Edit: I should add that afterwards we immediately switched to MapBox. Straightforward pricing, and it feels great to know that we're supporting the open source community.

DannyBee2• 13 years ago

"I would have some sympathy with this if you made it easy for single people to pay you to be able to use your data in a manner outside of the ToS of the free APIs. "

I'm not sure why you assume, among other things, that upstream contracts always make this possible.

mathattack• 13 years ago

I thought that was the original point that was missed too. Don't do it because the provider is telling you not to.

At least in this cited case, it was a throwaway project that won't be around, but folks who copy the code may be in for a shock.

lhnz• 13 years ago

You would have thought San Francisco's crime rate would have been higher with all of these unauthorised API usages... (Yep, the data was pirated by a guy in San Francisco. Ironic.)

jimmyrocks• 13 years ago

If you're looking for elevation data, the USGS provides this for free for the US.

Example: http://gisdata.usgs.gov/xmlwebservices2/elevation_service.as...

picea• 13 years ago

Also worth noting, Adam Schneider's GPS Visualiser website is excellent for quick elevation lookups globally.

Handy for me as Geoscience Australia (another good resource) unfortunately charge for their DEM data.

http://www.gpsvisualizer.com/elevation http://www.ga.gov.au/

iamleppert• 13 years ago

It's rate limited for a reason and every other so called "better alternative" will have some sort of limitations in place, if they want to stay in business. Nothing is for free. By saying people will just magically make a free, unlimited API service as long as people stop using Google's is dumb.

blackmad• 13 years ago

I've been meaning to put together a more technical blog post about the various geo pieces that we've released at foursquare. But for now, here's one that's an overview of the things we've been working on: http://engineering.foursquare.com/2013/06/11/quattroshapes-a...

The biggest piece is http://twofishes.net/, our coarse (city/neighborhood level) geocoder based on geonames data. It also serves as our coarse reverse geocoder when we combine it with polygon data, which we've released at http://quattroshapes.com/, as a combination of open government data and synthesized data.

http://zetashapes.com/ is a side project of ours, where we're seeing if people want to draw their own crowdsourced neighborhood boundaries in US by coloring in a map of basically city blocks.

I'm always eager to chat about open geodata as well as complimentary commercial solutions.

ubasu• 13 years ago

From the article: "I’m not RMS, I love open-source for very pragmatic reasons. One of the biggest is that I hate hitting black boxes when I’m debugging!"

RMS started the free software movement because he wasn't able to adapt a black box printer.

mkopinsky• 13 years ago

Even if RMS started from a pragmatic standpoint, his position has clearly moved far beyond pragmatism.

reginaldjcooper5• 13 years ago

Has it? I think it's very pragmatic to want control over your hardware and the ability to repair problems or add functionality as desired. If you consider it from what is best for society as well, it's very restrictive to have hardware where the software cannot be modified.

ChuckMcM• 13 years ago

I wonder how much of this is an outgrowth of 'hackathon culture' which is to say "Hey we can do this thing ..." and get to some demo without having the foundation bits worked out. (or more importantly the future foundation bits).

There are a lot of people who used to make 'kit cars' which was they took a VW bug chassis, removed the bodywork, and installed some fiberglass body on it. Really "cool" looking sports cars. But you couldn't really say "We're going to be the next Porsche selling our new "Wangdingo" sports car which, as it turns out, is a fiberglass body on top of a VW bug chassis. The difference between kids making a prototype which looks cool (professional even, some of these cars were really really nice) but as a 'new car' it was always a cheat since the other bits weren't engineered, they were 'acquired' and outside the plans of the original engineers for the chassis. Compared to the 'motorhome' business where the company buys a Chassis from Ford or Chevy as a chassis to build a new vehicle on.

Makes for an interesting conundrum. You can fail fast and iterate prototyping like this, and you can arrive at a saleable product for which there is no route to production because the underlying tech you rely on cannot be used 'legally' the way you want to use it, and the amount of money the product might conceivably generate is less than the cost of re-creating the crucial bit of technology from scratch.

cinquemb4• 13 years ago

It even gets more complicated as `dewitt mentioned: "And the cases in which the terms aren't programmatically enforced right away isn't necessarily a choice; often it is difficult to differentiate at scale between legitimate usage and disallowed usage."

This sets up a situation where people/companies can effectively 'get away' with it. So what is an aspiring entrepreneur to do when they are able to take advantage of a situation like this in a way that the other company isn't in a position block it (without throwing the baby out with the bath water) nor are they even trying to do what you are doing when you combine such usage with other "legal" means or data one collects themselves that is providing some value to people they tested their service/'hack' with?

tegansnyder• 13 years ago

If you are looking for a good mapping alternate checkout Leaflet.js. You can load from a variety of tile servers. I think a while back someone had a tutorial an setting up your own tile server on AWS. Actually http://tiledrawer.com/ provides a base AMI. Another good commercial solution for geocoding is http://smartystreets.com/ if you need to do batch processing.

nraynaud• 13 years ago

just to clear up a misconception, the SRTM is just unusable in it's normal form, you need a lot of time to clean it up and cross it with other sources. This guy does a lot of curation: http://www.viewfinderpanoramas.org/dem3.html

(and I also suck google's elevation data for a test because that's the simplest way to do it: http://nraynaud.github.io/webgcode/test_threejs.html )

Sukotto• 13 years ago

[Edit to add] Yes, you're right. When I compare their map side-by-side the same area in maps.google.com they are clearly different. I should have looked more carefully before asking my question.

So I guess my question should be "If they had the exact same page except using a Google map, would they be in compliance?"

It's not clear to me in what way the "Crime doesn't climb" page violates the quoted terms of use. Would someone please explain it to me?

  The Elevation API may only be used in conjunction with
  displaying results on a Google map; using elevation data
  without displaying a map for which elevation data was
  requested is prohibited.

They used the API to display results on a Google map.

Their site used the elevation data to display a map for which elevation data was requested.

I'm not being facetious. It seems to me that their use matches the requirements. So what exactly is the problem?

bdon2• 13 years ago

The map in question looks like it's actually from Yahoo! Maps: http://maps.yahoo.com/

X-Istence• 13 years ago

I don't think so, it says Nokia in the bottom right, along with CartoDB: http://cartodb.com

nucleardog• 13 years ago

Here's my problem:

1. The TOS aren't heavily enforced; thus 2. No one I work for cares about complying with the TOS; thus 3. No one is particularly supportive of my spending additional time to work within the TOS.

Using and contributing work time or resources to an open source project is going to be a hard sell until there is some pressure on the business end to change. Having our data sources cut off, or having to reimplement on different providers would help.

javajosh• 13 years ago

Okay, so I have a question: how can open source geocoding services sustain themselves? Doesn't it cost money to gather all of that data? Where does that money come from if the service is free? I would expect an OSS geocoding service to be a) seeded with data that is out of copyright (basic geography and placenames, b) spruced up with any well-regarded modern free datasources (probably government funded), c) spruced up even more from volunteers donating GPS data.

Would a geocoding service be a good candidate for a state-run utility?

timClicks• 13 years ago

Does anyone know of a good guide to DEM (Digital Elevation Model) formats? New Zealand has some great open data[0], and I expect that it would be a fun exercise to create your own elevation API from it but I don't know how to get started

[0] http://koordinates.com/#/layer/1418-nz-80m-digital-elevation...

jetd• 13 years ago

I have one comment and one question.

Q: Are OSM/Nominatim good for most countries now? When I tried it was very good to good enough for developed countries like US or UK but for others clearly not.

C: I really dont get Google mindset. They sell geoservices. OK. How much money that would bring 1mil, 10mil or 100mil in over 20bil company. What is the logic? Creating barriers? I dont know.

Why you should not pirate Google’s geo APIs

Related Stories