Building Lob's API

http://blog.lob.com/post/61597473247/building-lobs-api

by dzhao • 13 years ago

46 19 13 years ago

blog.lob.com

eterm• 13 years ago

"Do not return total counts or loop through entire databases to get the large count"

How do you achieve this on a practical level? I'm pretty new to this kind of thing and I've been coming up against this in my code, getting the totals have been a headache.

ollyculverhouse2• 13 years ago

I wish they would have elaborated on to why not to return the total counts. Does anyone have any ideas why this is a bad idea? I thought it would be useful for the consumer so that they can account for pagination?

datboitom• 13 years ago

I just pinged one of the founders and told him to reply, so hopefully he'll come and answer your question!

bavidar2• 13 years ago

Founder here. Returning total count is a very expensive action and most users don't use the result. Therefore, by returning paginated results you can allow the users that need that data, to loop through all the paginated pages and get the total count. It may add an extra step but for 98% of users the API will run faster.

7Figures2Commas5• 13 years ago

Assume that I am building an interface that displays data I have retrieved from your API.

Are you saying that your API would require my interface to mimic your pagination, as opposed to being able to retrieve data in such a way that I could paginate as I saw fit?

GrinningFool• 13 years ago

It may make sense to expose a separate entry to retrieve the count. That way the 98% of users who don't need it can continue to ignore it, but you won't force the extra overhead of requiring the other 2% to page through all results to get the count.

While count is expensive, it seems it would be less so than full result set retrieval broken up over several requests?

You'll get some additional usage beyond the 2% who need it now (after all, if it's there people will use it while in its absence it would not be considered at all), but on the whole you should be able to find the threshold at which you'd come out ahead by offering it.

thejosh• 13 years ago

So rather than saying 139 Results found, I would have to do over a dozen HTTP requests to get the full result count? How is that expensive?

7Figures2Commas5• 13 years ago

Pretty print by default is one of those small things but makes your API that much easier to use. It makes it easy for customers who are debugging to easily read data and pleasant to use.

A pretty print option should be available, but the extra whitespace adds to the content size, which is obviously a bad thing when you have any real volume. As the primary consumer of a production API is not a person, pretty print should not be the default.

Do not return total counts or loop through entire databases to get the large count.

The data that is available to consumers of an API should be based on actual use cases, and there is often a reasonable need for a consumer to have a total count. If there are performance implications associated with generating this, which won't always be the case, it's better to address those than leave the consumers of the API without the data they need.

ollyculverhouse2• 13 years ago

Regarding your total count comment, thats what I thought and I completely agree. More often than not the extra performance hit is worth it for a better 'consumer experience'.

gfodor3• 13 years ago

gzip compression kind of minimizes this issue.

7Figures2Commas5• 13 years ago

There's still no reason to add additional whitespace to a response that isn't going to be consumed by a human.

Also, it's worth pointing out that not all of your API consumers will necessarily support compression. The best practice is to use an Accept-Encoding header to allow the consumer to explicitly request a compressed response. This ensures that you can still serve uncompressed responses where necessary.

crymer11• 13 years ago

Where's the HATEOAS?

rkv• 13 years ago

Bob Loblaw Lob Blog

Building Lob's API

Related Stories