The Huge, Unseen Operation Behind the Accuracy of Google Maps

http://www.wired.com/2014/12/google-maps-ground-truth

by dreamweapon • 12 years ago

340 187 12 years ago

calinet63• 12 years ago

I dunno, there's been a Starbucks open a couple blocks from me for six months now. Wasn't on Google Maps until I went and added it myself!

cryptoz• 12 years ago

Well, they only re-drive most places every few years (Toronto seems to have 1-2 year gaps), so their own image sources won't necessarily have that data yet. Although, you'd think that local business listings would also be a data source and would probably be up-to-date. Do they scrape retailers' sites to learn about store locations? Guess not but would have assumed so.

Just wait for the self-driving car tests to ramp up. I think one of Google's early deployments will be a fleet of self-driving cars equipped to take streetview photos - this should enable something like weekly redrives of cities.

hnriot• 12 years ago

Keeping up with starbucks opening/closing stores would be impossible without the distributed power of their users, they pop everywhere, move regularly and aren't really critical for mapping. Luckily in SF you can't throw a stone without hitting a coffee shop, and of course, people here only use starbucks to meet people for craigslist transactions, no self respecting hipster would ever go in one.

gniv• 12 years ago

To play devil's advocate: The starbucks.com store locator is presumably up to date, so one can find openings and closings relatively easily.

Additionally, Google is now providing wi-fi for Starbucks, so they really should have all the locations.

The hipster cafes, though, seem harder to find automatically.

beering3• 12 years ago

It's incredible how Google can get so many people to work for them for no pay. Would you similarly contribute to closed-source proprietary software if given the opportunity?

zaphoyd• 12 years ago

I'd put that sort of contribution in the detailed bug report category, something I'd absolutely contribute to closed source proprietary software if I thought there was a good chance it would get fixed and that I wouldn't have to deal with it being broken anymore.

magicalist3• 12 years ago

I've filed many bugs on IE, Safari, and Chrome (for the last two not just on the Webkit/Chromium side, but also on the proprietary Safari/Chrome side).

I see your point, but the reality is that my users are there and these are also products I use every day. It's in my interest to have them work correctly.

calinet63• 12 years ago

It makes it more useful to me when others contribute, and more useful to others when I contribute. So, yes. Quite uncomplicated.

cdr• 12 years ago

Google Maps still does poorly in small towns, as far as I've seen. Not that you can really expect any better. The data in my hometown has clearly never been ground-checked (or even sanity checked by a remote employee), it's all clearly culled from webcrawelers and corrections by locals. There are businesses marked that haven't existed in a decade or more, and businesses that have existed for a decade aren't on the map. Yet another good reason to have a decent website as a small business I guess, when it's potentially the primary source of data for mapping/"local" services.

calinet63• 12 years ago

This is in a major city with a young tech-aware population, but point taken.

koenigdavidmj2• 12 years ago

Their screen capture of the Seattle Center area is particularly amusing given that their competitor Apple Maps still hasn't figured out:

1. That Broad St no longer exists for a significant portion of its length.

2. That Mercer St has been two-way for months now.

This is the major east-west road in that area, and they can't even get that right.

Yes, I've reported it regularly for months now.

serve_yay• 12 years ago

I can't believe people use it, it's embarrassingly bad. It's impossible to trust.

skygazer2• 12 years ago

In the car, I generally alternate between Google Maps, Apple Maps, TomTom and Navigon. After years of doing this, I've rarely noticed a difference in route accuracy -- in my experience they're equally likely to give a slightly faulty route, and there are better than 50/50 odds that a flaw will be shared across apps.

TomTom seems to be the winner with realtime traffic data accuracy, besting Google, and Google beats Apple, although not by much.

Google has lead in POI info, but that lead has diminished to the point of irrelevancy in the last year. Apple and Navigon have the best visuals. Google has the most naturally conversational voice guidance. And while those are nice, I still tend to prefer TomTom's accurate traffic data.

evmar• 12 years ago

In the pre-Ground Truth days, Google Maps in SF used to think you could take a right from northeastboound Market street onto the 101 onramp, which is both obviously the best route for a wide variety of routes and also not actually allowed. The combination meant that many route plans didn't work.

In looking into getting it fixed I heard that the contract for getting the map data prevented Google from making any modifications to the map data. Apple Maps might be in a similar situation -- aware of the problem but with their hands tied. Map data has a lot of legal restrictions for reasons I don't really get.

pyre• 12 years ago

The map companies probably want to make the customers dependent on continuing to get map data from them, rather than just using their map data as a 'base data set' to building their own, competing map data...

beering3• 12 years ago

I think Apple gets the short end of the stick in these deals. The company that provides that mapping data usually has ridiculous restrictions on what you can/can't do to the data (even possibly disallowing your own patches), and Apple takes all the heat from users.

mankyd2• 12 years ago

I mean, don't you think that Apple deserves to take at least a little of the blame for this? If they can't negotiate a better deal with their data sources, find a better data source, or find a way to feed corrections back to their data source, that's on them.

notatoad• 12 years ago

Apple has their name on the product, it's their responsibility to ensure it is up to par. It doesn't really matter who the fault lies with, if the data coming in isn't good enough then apple should be looking for different data sources.

jonknee2• 12 years ago

Why wouldn't Apple take all the heat from users? Steve Jobs removed a very good map from everyone's iPhone and replaced it with a buggy Apple branded map. I don't care if it's the fault of some vendor, at the root of it Apple is to blame for shipping it.

astrange• 12 years ago

I tried it out but couldn't reproduce either issue. Ignoring appearance of the road geometry, which two addresses exactly will give you a bad route?

koenigdavidmj2• 12 years ago

It doesn't ever seem to route on Broad St, though I think you're underestimating the severity of the 'appearance of the road geometry' for anyone who uses the Maps app to show a map rather than just following its directions.

If you want a bad case, ask for directions to the town of Langley, which is placed on the wrong island altogether.

TazeTSchnitzel2• 12 years ago

It's wonderful what Google are doing with Google Maps, but I can't help but wish these maps were open to use by their competitors.

Imagine a world where all the effort Google, Apple, Microsoft and Nokia put into their maps all went into OpenStreetMap instead.

cylinder2• 12 years ago

Can't understand the reasoning behind expecting a company to pour hundreds of millions of dollars and resources into something for free / no return at all.

ZeroGravitas• 12 years ago

Because once you get the incentives correctly aligned it's a massive boost in global wealth?

gvb• 12 years ago

Remember the Yellow Pages[tm]? It was a free product that made a huge amount of money by charging businesses to be included/advertised. Google Maps is the new Yellow Pages. "Let your fingers do the walking."

Many GPSes have "live google" functionality. I have one that allowed me to search for the nearest gas station or restaurant while on the road. There is huge value to Google from that.

rakoo• 12 years ago

no return at all

That's where you got it all wrong; the same common sense argument could be used against using (and contributing to) open source software in companies, and yet the most successful open source softwares are supported by companies. The thing is, contrary to the common sense argument, there is value in not reinventing what others are already doing. That's also a great way to find technical talents who have first-hand knowledge of the tools you use.

OSM is very similar here: the data is openly reusable by anybody and any change benefits everyone.

There can be a good reason for not contributing and using OSM, and that would be because Maps is such a central product to Google that they don't want anything shared with anybody else (plus they already have a lot of data and are considered as the best in class, rightfully or not)

rictic2• 12 years ago

I read the parent's post not as reasoning or arguing for a specific policy, but simply observing one of the flaws of capitalism.

Capitalism is doing much better than the other existing systems at solving the massive coordination problems of a large modern society. However, we can also tell that it is imperfect and involves some clear examples of wasted effort. Noticing this waste is valuable, even if no one has a good solution at the moment to improve on the existing system.

What's the alternative? Should we just assume that our current economic system has an optimal set of incentives that we could not improve upon?

pkh80• 12 years ago

and that will be paid for, how?

Its been said that Maps is likely Google's most expensive product to maintain. Just think about the scope of driving every road in the world as often as possible.

It will only be replaced by OSM if OSM gets to be better than Maps on its own.

Doctor_Fegg3• 12 years ago

OSM's already better than Google Maps in many ways - walking and cycling data, detail in several European countries, and so on. And the beauty of it is that it's paid for by the volunteer effort of its contributors.

7952• 12 years ago

What Google has is massive amounts of raw data. The street information may be time consuming but is still trivial from a mapping point of view, and is often done better elsewhere. But StreetView has amazing potential and is inaccessible for analysis outside of Google. APIs are great but we need raw data!

Rondom• 12 years ago

You might want to have a look at Mapillary (which is a crowd sourced StreetView-alternative). Depending on the location the quality can be quite okay and some places have images where StreetView does not have any. All the images are under a CC license. https://www.mapillary.com/

nfoz• 12 years ago

Economic competition implies massive waste of effort. Not sharing work and ideas is one of them; others include the psychological/sociological effects of competition and pressure, and also all the resources poured into advertising (which itself does not particularly give us better products/services).

This isn't unique to Google Maps; the same goes for competing pharmaceutical companies, educational institutions, and everything else. Imagine if they were all cooperating rather than competing.

However there are also some benefits to competition. Notably, the impetus to take risks of private capital in order to build new things / try new ventures. Also, the presence of choice in the market (in this case, you can choose which maps you consider more reliable).

I'd love to see more exploration into economic systems involving cooperation and choice, instead of competition. Can we have independent organizations motivated to produce different-but-similar goods, while simultaneously sharing their efforts?

TazeTSchnitzel2• 12 years ago

I'm not sure competition is always the best model. For certain things it works well, but for things like maps, I think everyone loses out if mapping data is locked in different commercial vendor silos rather than combined into a non-profit, shared single source.

mempko3• 12 years ago

This is a very detailed article. Most people do not appreciate how much work is put into building these digital maps, so it is great to see these kinds of articles.

While not as detailed, for contrast I want to post how Nokia's HERE maps builds their maps.

http://360.here.com/2014/11/17/made-usa-people-fargo-make-ma...

That is from the company blog. It shows a couple of our tools, but not nearly all. I work for HERE maps and my opinion is my own.

I just thought that people might be interested in how other companies like HERE build maps. Despite all the automation (at both google and HERE), it is still a human intensive process and there are people on the other end making it as correct as can be. Also, I can't go beyond what is written on the public blog as I am not a company spokesperson.

fs111• 12 years ago

Similar articles about TomTom from 2 years ago: http://www.pocket-lint.com/news/115883-how-tomtom-maps-are-m...

mempko3• 12 years ago

Awesome. That should be interesting to people who use Apple's maps.

bostonpete• 12 years ago

I can't imagine the amount of ambiguity that comes up in every day conversation when the name of your project/division is "here". Do you have internal language that's used to avoid such ambiguity?

mankyd2• 12 years ago

I used to work for a company called "Going". We had no alternative internal code name. (I swear, if I heard one more pun about where I was "going"...)

That being said, you'd be surprised how little you need to refer to your own company by name internally, especially in an ambiguous manner. Usually its just "Hey, can you put this up on the site?" or "It would be cool if the product did X."

mempko3• 12 years ago

I typically say "here maps". What I like about it is the URL is easy to remember. here.com

ihaveajob• 12 years ago

I used to work on a project called MOST inside a large hardware organization. Most conversations about MOST were most entertaining.

xwintermutex• 12 years ago

The LiDAR part is neat. Anyone knows if Google uses this too for mapping?

Someone12343• 12 years ago

Google Map's data keeps getting better and better, they now have lane warnings (e.g. "right two lanes," etc) and traffic warnings with the offering to re-route (if they can find a quicker route). Although Google Map constantly offering to re-route me onto a slower route is a little odd (e.g. "want to re-route? It will be a 5 minute longer trip!").

That all being said, while Google Maps as a data source is amazing, Google Maps as an app and or web-site has a lot to be desired. Up until a few versions ago they had a way to add "My Places" but no way to actually access those places from the app (what?!). They have no search history (still, today), they do have suggestions but they are just somewhat useful, they have no compass (got removed), plus they went all "minimalist" and hide all the UI elements (and removed things like offline maps, for nearly a year).

I've actually been forced to stop using Google Maps as GPS as the app keeps closing during navigation and when re-opened it has "forgotten" what I was navigating to or previously searched for. So now I am forced, mid-journey to exit the freeway just to re-open Google Maps, re-search for my destination, and then re-start navigation just hoping it won't re-close-randomly before I get there...

It doesn't crash. It just closes. It is just gone. Like "poof." Then of course the phone goes into standby as nothing is keeping the screen on. At least if it had search history the situation MIGHT be recoverable, but nope...

ParadoxOryx• 12 years ago

You can access your Maps search history here - https://history.google.com/history/lookup?hl=en&st=maps

Someone12343• 12 years ago

That's not a lot of help when I'm driving down the freeway at 65 MpH. It needs to be within the app.

tim333• 12 years ago

They have no search history

An odd thing is that if I've searched for something on Google Maps on the laptop and then open their phone app I only have to type one letter and then the name pops up of what I'd searched so there's evidently some sort of history, just not very well implemented. Another funny glitch I had with driving directions is it quite often says 'turn left' about 4 seconds after you have passed the turning which is not quite as it should be. I sometimes switch to Apple Maps of all things though I don't think their traffic info is as good.

psaintla4• 12 years ago

I agree completely, there are so many great features I think they could have added instead of changing the interface.

1.) Send directions to a mobile device. There are many times where I search for directions or map out an alternate route on my desktop. The only way I can access those directions is to email it to myself then open the email on the mobile device. Why not just store the data and let the user retrieve it on their mobile device?

2.) Better accuracy of property in rural/suburban areas. It's odd to say this but Bing Maps is actually much more accurate in rural areas.

3.) More fine grained control over street view placement.

4.) Offline maps! Not everywhere has good cell coverage.

afterburner2• 12 years ago

Offline maps never really went away, but they did get harder to manage. Type "OK maps" in the search bar to cache the visible area (it saves several zoom levels). Apparently with the newest version there's another method: http://www.cnet.com/how-to/how-to-use-google-maps-offline-on...

Agreed on the apparent deprecation of social/offline features, but the navigation does seem to be getting better.

Someone12343• 12 years ago

Send directions to a mobile device.

Yeah. How is this not a thing?! I imagine a lot of people search for things on their desktop and then want to "push" it to their phone for navigation, but yet I have to re-enter the zip into the phone to do so.

twoodfin• 12 years ago

Although Google Map constantly offering to re-route me onto a slower route is a little odd (e.g. "want to re-route? It will be a 5 minute longer trip!").

I think the point of this is to be reassuring: "Hmm... I wonder if it would be faster if I took a right here to get around this backup? Ah, Google says no."

That's increasingly important as the live traffic data gets better and it's worthwhile to use navigation to pick one of several routes you already know.

Google is still not perfect in this regard, but I'm certain my batting average at being able to "beat" its recommended route to a frequent destination has been trending down for some time.

jedc• 12 years ago

Yep, that's how I've always used/interpreted this same feature. I know the various different routes I could potentially take, and knowing Google has already calculated they're slower makes me feel better about the one I'm actually on. :)

visarga3• 12 years ago

keeps closing during navigation and when re-opened it has "forgotten" what I was navigating to or previously searched for

Maybe the good guys at Google need to learn about caching.

bhandziuk• 12 years ago

I find the new website to be downright slow, too. If I go to https://www.google.com/maps/ I have to wait too long to load. I can generally get going a lot faster using the older interface (https://www.google.com/maps/mm?authuser=0&hl=en).

But, sometimes I just go to Bing because I know it will load fast and I can scroll around super fast, too. Scrolling around in Google maps is always slow. It's like they are loading way too much stuff. All the 3D buildings...useless.

r0m4n0• 12 years ago

Agreed, they've added so much functionality to the desktop site that you can't simply search for an address without huge overhead. google earth is cool but the typical user doesn't need to load topographical details and 3d buildings when landing on the home page for maps...

MatthewMcDonald• 12 years ago

the app keeps closing during navigation

I've had a similar issue occur several times in the past month. The app doesn't close, it just stops providing navigation instructions. The worst part is that it fails silently, and it seems to be related to poor cell service. It was a little scary when I realized what had happened, as I was driving on icy roads in an unfamiliar area at night. A notification that navigation has stopped would have saved me time and headache.

I'm also not a fan of how it changes your route without telling you. I intentionally selected a routed that took major highways, but after stopping for gas the app changed my route to a shorter (by distance) trip that was on small, poorly maintained roads through rural areas (with very poor cell coverage). When combined with the navigation silently failing in poor cell coverage, this has made for a very poor user experience.

magicalist3• 12 years ago

wait, am I crazy? As soon as I touch the search bar in the mobile app the search history is right there. Someone below mentions "send to mobile" from the desktop, but searches from desktop google maps also show up in that search history.

You do need to turn it on originally, but I believe that option is also fairly prominent...

sliverstorm• 12 years ago

"want to re-route? It will be a 5 minute longer trip!"

I might jump at the chance. My stress levels are correlated much more strongly to traffic than trip time.

vlasev• 12 years ago

What irks me the most is that it almost never displays the names for the horizontal streets until you zoom in really really close. It drives me crazy sometimes. Turning the map 90 degree doesn't seem to fix this either which is really weird.

kalleboo• 12 years ago

They have no search history (still, today),

At least on iOS, there is. It even syncs with your history on the desktop maps.google.com so if you looked something up on your computer, it's right there.

efalcao• 12 years ago

There was a great talk on this at Google I/O 2013. Recording here: https://www.youtube.com/watch?v=FsbLEtS0uls

plg2• 12 years ago

difficult to imagine why apple would have even attempted to get into this space ... they must realize they can only be a second rate player. maybe that's good enough for them? doesn't fit with their story/ethos though

stephenr4• 12 years ago

Why do you assume an advertising company is better equipped/able than a technology company to run a maps/routing service?

commandar2• 12 years ago

You're asking the wrong question.

Why would an AI and automation company be better equipped/able than a commodity hardware company to run a maps/routing service?

spinchange4• 12 years ago

Setting aside how one views/classifies the companies, you don't have to assume anything: Google's maps have qualitatively better data than Apple's because they've literally been driving & photographing much of the world's roadways. Apple shows no indication of having this kind of commitment and even if they did, Google has a significant, years-ahead lead.

plg2• 12 years ago

I just meant google has such a massive head start

th0ma57• 12 years ago

Anyone have resources for mitigating the web map zoom level inaccuracy bug? I guess I know this only as the OSM bug, but it is the reason web mapping like OSM and Google Maps isn't used much by true GIS analysis. You can see this most easily by having your GPS on with your phone and using Google Maps on the web. When you zoom out your location accuracy decreases the more zoomed out you are. This doesn't seem to happen on the Maps app on my phone.

rmc10• 12 years ago

What do you mean? OSM is open data, you can display it anyway you want.

aw3c2• 12 years ago

What are you talking about?

th0ma57• 12 years ago

http://wiki.openstreetmap.org/wiki/Osmarender_bug

spinchange4• 12 years ago

It's little wonder that Google Maps is such an amazing application given all the time, intelligence, resources and just the sheer commitment Google's shown to it.

I know business considerations are what they are and often independent from others, but nothing shook my belief in Apple's commitment to quality as a defining principle more than the decision to replace Google Maps with Apple's own half-baked beta offering. Does anyone think they'll ever give maps & cartography the kind of attention to detail that Google's given it?

visarga3• 12 years ago

Google Maps is such an amazing application

Not so amazing. Just the data is amazing. The app sucks.

brandonwamboldt• 12 years ago

The app may not be great, but it's pretty good, especially for the vast majority of users. From Google's perspective, I doubt it makes sense to add features that many of their users don't need and won't use.

forrestthewoods3• 12 years ago

I'm still waiting for smartphones to be more fully crowd sourced into maps. I badly want real-time traffic data to come from smartphones.

beering3• 12 years ago

This already happens on Google Maps. Or do you mean in an open-data way?

forrestthewoods3• 12 years ago

Is the data gathered from crowdsourced smartphones? If it is then the precision is shit. Waze has been continuously unimpressive in dense urban environments. It can sometimes take 15 minutes to go two blocks through two lights. And neither Waze nor Google Maps correctly show that.

Using the GPS drains battery and there are huge privacy concerns. It's not an easy thing to do right. But I think it can be done.

alxndr• 12 years ago

Have you looked at Waze?

forrestthewoods3• 12 years ago

Yes. It's several steps in the right direction. The god damn achievements and items and ads are incredibly annoying. But it's a good step. It also isn't used by enough people. How many drivers have smartphones? How many of those are contributing traffic data? The answer is, respectively, a lot and almost none.

jjp• 12 years ago

Interesting read on TomTom's approach - http://www.pocketgpsworld.com/How-TomTom-Makes-Maps-Part-4-H...

amoshag• 12 years ago

This article shows that Waze's method of crowd sourcing maps is the right way to go. Waze is able to have realtime detailed information about every street, building, directions, etc. including all the details that were discussed in the article + much more (e.g. future road changes), and most importantly it does that in a fraction of the cost. Now it seems that the 1.3B that Google paid for it is probably a bargain

erjiang2• 12 years ago

If you're fascinated by maps and digital cartography and haven't done so yet, do check out OpenStreetMap! It's competitive to Bing, HERE, etc. but everything is open, and the data is yours to play with.

There are projects based on OSM for routing, geo search, map editing, etc. that are also free. Definitely a lot of room for both programmers and non-programmers to contribute.

http://wiki.openstreetmap.org/wiki/Main_Page

MCRed• 12 years ago

I've had a great experience with Apple Maps, which are based in part on Open Street Map (giving data back too) and TomTom. I've found it's accurate enough to just trust it, and it's excellent at detecting and rerouting when I can't do what it wanted me to (eg: traffic in the way, or I just miss the turn.)

rmc10• 12 years ago

I didn't think Apple Maps gave any data back to OSM, do you have a link? I thought it was just one dump file from a few years ago that they used.

thomersch_• 12 years ago

Apple has no official contributions to OSM. We are not even sure in which cases they use the data.

ris2• 12 years ago

"which are based in part on Open Street Map (giving data back too)"

I know for a fact this isn't true. What was your source for this?

discardorama• 12 years ago

Outside the US, Apple Maps suck ass (I'm sorry, but there's no other way to put it).

bluenose69• 12 years ago

In my region (Atlantic Canada), OSM is much more detailed, up-to-date, and accurate than Apple maps.

auganov• 12 years ago

I can second that. https://www.mapbox.com is a particularly notable technology on top of OSM. Initially I wanted to use Google maps with some customs styles but found their styling options pretty limited. Even worse it's too unreliable in China and there's no way around it. I used mapbox to easily create a nice map that incorporated our branding and then http://leafletjs.com/ to add interactivity on top of it. Very happy with it.

OvidNaso• 12 years ago

The zoom is much much faster than Google maps. Very nice.

cjslep• 12 years ago

Definitely a lot of room for both programmers and non-programmers to contribute.

As an engineer-developer hybrid, I found OSM mightily helpful when pulling midwest town data and running tornado strike simulations. I encourage contributors to look beyond the roads -- other infrastructure such as power lines or towers are rare to find in the data but can be extremely useful.

akavel• 12 years ago

Also, OSM allowed the amazing initiative of http://hotosm.org/ to be built -- a website where contributors from around the world help humanitarian organizations by adding data into OSM for various areas where people are in grave need (e.g. finding houses of people in drought areas in Africa) based on satellite & aerial imagery. I tried this, and really couldn't walk away much too far into the night, can get addicting... also I think it's something similar to analyzing photos from spy satellites during WWII, only for 100% good case! For more details, see:

http://wiki.openstreetmap.org/wiki/Humanitarian_OSM_Team

Quoting from the wiki page:

"From the beginning of OpenStreetMap it was anticipated that open, free map data would be a tremendous benefit for humanitarian aid and economic development. The idea was proved during the Haiti earthquake in 2010. HOT was incorporated in the immediate aftermath, August 2010, as a U.S. nonprofit and became a registered 501(c)3 charitable organization in 2013. Anyone is welcome to contribute to the HOT mission via our Tasking Manager - all you need is an OpenStreetMap Username. We ask only that you try to uphold the same code of conduct as our members, see our HOT Membership Code."

Surprisingly for me, quite many important buildings can be discovered without local knowledge (i.e. without "being there physically"), purely based on aerial imagery and a specially crafted guide. See e.g. the following link, with some hints and clues on how to recognize a school for a current (urgent!) pre-typhoon task http://tasks.hotosm.org/project/804:

http://wiki.openstreetmap.org/wiki/Humanitarian_OSM_Team/Typ...

timClicks• 12 years ago

If anyone wants to contribute, the online tool is really accessible and drive-by contributions (e.g. sub 10 minute) are totally fine: http://tasks.hotosm.org/

rmc10• 12 years ago

HOT arose spontanously from the Haitian earthquake. OSMers wanted to do something about the earthquake, so started mapping. Then people realised that something very useful had been created, so it was formalised into HOT.

JustSomeNobody• 12 years ago

I'm curious if you would know how to do this. I'm in the states and I would like to improve my local maps. I'm a complete newb to mapping and OSM. What I'd like to know is, what's the best way to link addresses and coordinates? Are there public databases? There are new office building and subdivisions that are not yet in OSM and I'd like to help do that. I just don't have time to enter all of them manually, so if there's a way to automate that...

bhousel2• 12 years ago

The best way for a "complete newb" to improve OpenStreetMap is to just find the place that you are interested in and click on "Edit" and start drawing the missing features.

There are public datasets and imports that happen sometimes, but these are done by people who have experience in it, and usually in cooperation with the local government GIS that controls the dataset.

erjiang2• 12 years ago

Going outside and recording what you see is still the gold standard for collecting data. Don't feel like you have to fix everything in your city - if you can write down a little bit every time you go out and then come back home and enter it in using the map editor, that's very valuable.

bgolder• 12 years ago

If you're specifically interested in adding data about addresses, you may want to look into http://openaddresses.io/

rmc10• 12 years ago

What you're talking about is "imports". OSMers are wary of them. It's possible for someone to mess up existing data, to add bad data, damage the local community etc. Most of the map in the USA was imported in late 2007 ( http://wiki.openstreetmap.org/wiki/TIGER ). The map looks done, but the data quality is pretty poor.

People are happy if you manually enter them.

mejackreed• 12 years ago

Hi, you should also look into checking to see if there is a MapTime http://maptime.io/ near you. MapTimes have sprung up all over the world and are a great way to learn about mapping and OSM.

lobster_johnson3• 12 years ago

Do you know if OSM is any good at geocoding for US addresses? We're currently using Google, but we're often bumping into the 2,500/day ceiling, and the commercial license is $10k/year.

We have looked at some alternatives such as geocod.io, but quality seems a bit lacking and they don't offer some features we need (but which we would surely be able to develop ourselves if we could use OSM as a primary database).

aldr• 12 years ago

Have you tried the mapquest open data api? http://developer.mapquest.com/web/products/open/geocoding-se...

maxerickson2• 12 years ago

It is not. Coverage is poor. At high zooms the main tiles at openstreetmap.org will show addresses, so you can go to a random place and see that there aren't any. Some areas are pretty complete (look at NYC), but there are many major cities with basically 0 addresses.

peteretep• 12 years ago

How accurate do you need? I have had great success in putting a proxy in place before your call out to Google to cache, remove addresses you don't think will geocode, and geocoding addresses you're happy with via free-sources first...

rmc10• 12 years ago

OSM is just a data base. People have made geocoding that uses the OSM database. You can install it yourself (e.g. http://wiki.openstreetmap.org/wiki/Nominatim ) or use a hosted version ( http://wiki.openstreetmap.org/wiki/Search_engines )

The advantage of OSM, is that if it's not returning the right data, you can fix it, and import it into your own hosted version.

thecodemonkey• 12 years ago

Hey,

I'm one of the Geocod.io founders. I'm very interested in hearing about the features you're looking for and discuss data quality. Feel free to reach out here or at hello@geocod.io

ChuckMcM• 12 years ago

They started Ground Truth when I was there and it really opened my eyes to just how "broken" a lot of map data sources were at the time. Something I considered to be a 'solved' problem (take aerial photograph, pull out streets, poof map!) was no where close to solved.

Some folks have said that Ingress, the Android and now iOS, real world geo game. Was in part an exercise in data collecting. Get people to photograph land marks in the real world to make "portals" while collecting GPS data associated with public landmarks for maps. And then use that data to localize things in satellite imagery, etc.

Makes for an interesting data set.

matt_kantor• 12 years ago

The majority of buildings in the U.S. are now on Google Maps.

I doubt this. Out of curiosity I looked at my family's farm and Google Maps is missing most of our outbuildings. More generally I'm sure that they come up short on backyard sheds, pool buildings, structures in dense forests, etc.

jameshart• 12 years ago

Where do you think the majority of buildings are? I mean, there's a long tail, but more than 80% of the US population lives in urban environments; the head of that density graph is massive.

Retra• 12 years ago

As a point of fact, most buildings do not exist where there are currently such things as trees, pools, and sheds... Most people, it would seem, tend to live in houses and complexes, work in factories and offices, or shop at stores and shopping centers.

cjensen• 12 years ago

Despite all of Google' effort, they still rely too much on USGS published maps. For example, just 6 miles from the GooglePlex is a road in Newark with a gate [1], but Google doesn't know you can't drive on it.

They're doing a good thing, but unless you can see that Google has driven the route and has published street view, you can't entirely trust them.

[1] 37.515696 -122.050873

rickdale• 12 years ago

Google maps is amazing. I was on my way to Detroit Airport, I live about 1.5hours away. Anyways, I wanted to double check directions so I used google maps, and it came up with directions that were going to take me 2 hours. I laughed thinking, 'oh technology, 2 steps forward, 1 step back'. So I drove down the way I already knew and was just double checking. With twenty minutes left in the drive, theres a deadstop traffic jam. Highway is closed. Turns out entire highways flooded in Detroit around 6-7am, and when I googled directions around 830am, google knew not to go that way, but I didn't. Made it to the airport 3 hours later, with a flight 6 hours after that. I had never been so impressed with google maps before this though. How did it know not to go that way so fast? I literally just drove right into it no warning or detour signs.

bhousel2• 12 years ago

They have gotten really good at this since acquiring Waze, which crowdsources traffic data.

juliendorra• 12 years ago

An interesting anecdote: the "promenade claude lévi-strauss" is a new street in Paris (around 1 year old, pedestrian only). It doesn't exist in Google Maps yet, which make Uber fail when you try to go there as Uber rely on google maps. (many people might want to go there, as it is now where all the paris urban planning administration is. I had to for a permit.)

Actually last I tried Uber/Google Maps sent you at the directly opposite corner of Paris, to the Quai Branly Museum, probably because there is something related to the anthropologist there (?).

Both Apple Maps and more interestingly the crowd sourced OpenStreetMap know about the "promenade". I'm curious to see how long it will take Google to discover this new street.

magicalist3• 12 years ago

The Uber driver app uses Apple Maps...

rmc10• 12 years ago

more interestingly the crowd sourced OpenStreetMap know about the "promenade".

Not too suprising really. OSM can be updated in a minute to add new features.

robk• 12 years ago

why not add it yourself in Map Maker?

civilian3• 12 years ago

I worked as a contractor in Kirkland on this project for about a year.

Just for context, the first image with the red and green dots--- each dot represents a 360-degree "street view" image. While consumers only get half-a-dozen a block, the operators got access to all of them. (Green are HD, red are normal def.) It was really useful for verifying that businesses had closed because the streetview dots also had the exact time that they were taken.

There was also deeplinking in Atlas to a specific dot and direction vector. So when I was submitting a change to a business, I could link to the exact sign that I saw to prove that "Mama's Teriyaki is on _that_ corner", which helped whoever QA'd my change.

It was a great first job out of college given that my degree wasn't very relevant to anything. (A lot of my coworkers in the same spot--- young, a degree that wasn't too useful, trying to find anything to do.) Out of the 400 contractors in that first Kirkland "class", I see tons of them around the Seattle tech industry now. It's been a surprisingly good network to have.

At the end of the day it was still data-entry / data-validation, and mind-numbingly boring.

ghshephard• 12 years ago

This is actually very fascinating. About how many businesses a day would an average person versus a prolific or slow person manage to update? Did you mostly work off of business signs to determine information about businesses? What if no recent pictures had been taken?

civilian3• 12 years ago

We'd also use the internet and, reluctantly, call them. We would identify ourselves as google maps, calling for info about their business.

I don't quite remember the numbers, but it was something like 40 a day? The slower and new peeps would be around 25, and the fast people could do like 80 a day. But there weren't any productivity bonuses besides not being let go, so me and my buddies would do about 10% more than the average and then goof off on the internet or do homework for our python night class.

blissfulight6• 12 years ago

Can you use the Atlas tool to correct business and other E/POIs (establishment/points of interest), or is limited to roads and other base map data? How do they evaluate spam POIs (say, locksmiths, who are prolific on Maps) in comparison to other Maps RAP (Report a problems)? Do all the teams (Places Listing Editors, Map Maker Google Reviewers, Maps Editors, etc.) use the same Atlas tool, or do they all have different tools? I use MM (Map Maker) quite a bit, and I'm curious how Google's tools work in comparison. Thanks.

civilian3• 12 years ago

Yup, you can change those things in Atlas. The "GroundTruth:Local" team did our best to accurately have 6 points of data: name, address, phone, url, business category, and pinpoint on the map. Atlas also had history for POIs, so you could see history from other operators. We wouldn't just use streetview, we'd also search for websites and, if we had to, call the business itself.

The Local team used Atlas, as well as the GroundTruth road-placement team, and the QA teams. There were different map layers and controls you could bring up.

Business category was funny because it had the most subjectivity to it. There was also a team in Hyderabad (that wouldn't make phone calls), and I once got into a dispute about whether "Olive Garden" is an American restaurant or an italian restaurant. :D

Yup, so part of the Atlas history for a POI is seeing problems that people have submitted onto a listing. When I was doing this a few years ago we had a humongous backlog of user feedback, but I'm under the impression now that they're mostly up to date.

Spammers were a huge problem. For businesses that don't want to advertise their brick-and-motor (a locksmith comes to you, not you to a locksmith) the policy was that they'd get one POI at the center of the city or zipcode they operated in. You'd think that locksmiths are bad, but florists are worse. Florists! They sell flowers to people! Horrible spammers. They also like to make fraudulent listings--- so they'll make a listing for "Competitor's Flowers" but have the phone number go to their business "Evil Flowers LLC".

username223• 12 years ago

I worked as a contractor in Kirkland on this project for about a year.

You survived the place hope goes to die? Congrats! It sounds like an unholy amalgam of data entry and a call center.

paulrademacher• 12 years ago

Civilian, you probably signed an agreement with Google to not disclose information about their internal operations, as you're doing here and below.

jedberg• 12 years ago

And yet, when I want to go from SFO to Cupertino, it absolutely refuses to offer taking 380 to 280, even when it is 10 to 15 minutes faster according to Google itself.

Apparently they still haven't fixed the whole "we have to go directly away from the destination for a little while" problem.

danellis• 12 years ago

So they have speed limit information? It would be handy if you could have that information on the navigation screen, maybe even with a warning when you exceed it.

shutupalready• 12 years ago

St. Louis streetwalker serial killer, Maury Travis, sent the local newspaper a computer generated map giving the intersection where he left a body.

Investigators determined that web-based mapping software Expedia was used--based on symbols used to mark highways and such--ruling out Mapquest, Yahoo, and others.

A single person had clicked on that intersection in Expedia in the 5 days before the map was received by the newspaper. They traced the person presumably using the IP address.

This was back in 2001.

It's odd that we still have unsolved street crime given that everyone today carries a tracking device sending back real-time geolocation data.

sytelus• 12 years ago

Another similar article appeared two years ago:

How Google Builds Its Maps—and What It Means for the Future of Everything

http://www.theatlantic.com/technology/archive/2012/09/how-go...

I think most interesting point is not covered in either of them: How much of this effort devoted towards human curation? The numbers are believed to be ~1100 full time employees and 6000 contract workers[1]. These are huge numbers compared to most competitors in the market. Assuming each correction to a map can take 15 mins on average, you can easily make 100,000 corrections a day. Again, assuming there are top 2000 cities in the world where most of the queries originates, this is about 100 corrections per city per day. This would guarantee Google maps best of the best freshness, precision and recall on most metrics. With about 10X-20X larger curation force plus algo engineering, likes of Apple or HERE have no chance. In a way this also shows Google's leadership wisdom. Maps are the most important thing on mobile and even on web. Most companies don't get this and provide minuscule budgets citing no potential revenues (for instance Ballmer cancelled Street View like effort at Microsoft). By the time they wake - if they wake up - it would be to only find that they have been outrun by such a huge gap that even a decade won't be enough to catch up.

[1] http://www.businessinsider.com/to-do-what-google-does-in-map...

lingoberry• 12 years ago

After relying on Google Maps a lot for many years living in the US, I have (maybe unsurprisingly) noticed that it doesn't work nearly as well in Europe. As in, I'd zoom in on central Stockholm and search for something common, like the name of a restaurant, and Maps would promptly swoosh me to Nicaragua where it found a place with that name. It's doing stuff like this so often that I rarely use it anymore.

The Huge, Unseen Operation Behind the Accuracy of Google Maps

Related Stories