Decision to store geocodes

https://news.ycombinator.com/item?id=36721

by glangley • 19 years ago

1 3 19 years ago

news.ycombinator.com

I am working on a website and am planning on implementing a google map on each page and would need to make between 10-50 geocode requests on each page. Is it smart to have a whole new table just for storing these geocode address or is it easier to just hit google every page refresh?

Thanks in advance.

epi0Bauqu• 19 years ago

Definitely store them, or at least cache them.

willarson• 19 years ago

Epi0Bauqu is definitely correct that you do not want to access google 10-50 times with each page refresh, that would be very slow.

However, with the number of pieces of data you are requesting you'll have to be intelligent with your storage and querying mechanism. Example: you will very likely want to fork your queries to google to make the entire batch of requests simultaneously instead of sequentially. When you start implementing a caching/storage mechanism you'll want to keep this in mind too: accessing memcached 50 times sequentially will be slower than querying PostgreSQL once (in general, although if the database is very large and has to be paged in and out of memory, etc, this may not hold). So you'll probably want to figure out a way to store these requests in your database, and have a single query that can recall all the relevant geocodes (I don't know the details of how the geocodes will relate to each other, so I can't really be specific), and then you will want to cache that entire request. This will mean that you will (when dealing with data you have already retrieved from google... and prefetching data that you expect to see wouldn't be a bad idea) have at worst one SQL query, and at best a quick access in memcached.

If you can't think of a way to group requests like this, I strongly suspect you will need to rethink your application. Performing 10-50 external requests per page refresh will put a serious damper on your site's performance, and making 50 database queries per page refresh or even 50 memcached queries per page refresh is probably untenable (although you could start caching the entire created pages, depending on what exactly your application does). To get sufficiently quick speed to do 50 queries, you'd probably have to use local memory (which is significantly quicker than memcached), but using local memory will open the door into a thread safety hell.

glangley• 19 years ago

sorry let me in explain. i will have a database of locations and it already stores the address. i havent looked to deeply into the google maps api, so i am not sure if I will make just 1 request to google for the entire map or if it requires 50 requests to get every marker on the map.

so, would it make sense to cache the results? anyone have good links for tips on memcache?

thanks already for the great feedback

Decision to store geocodes

Related Stories