One long, intimate month with the Facebook API

http://anilchawla.org/2011/02/23/one-intimate-month-facebook-api/

by anilchawla • 15 years ago

78 27 15 years ago

anilchawla6• 15 years ago

I should say that I've been lurking on HN for quite some time and that this is my first post :-) One motivation for writing this post was that I was surprised to see how often the Facebook API will drop/omit data that exists and that you should be able to access. I specifically call out the Breakup Notifier app (which recently got a ton of buzz) because I don't see how apps like that can report accurate information when the underlying data is unreliable. Any thoughts or similar experiences with the Facebook API?

plusbryan• 15 years ago

Thanks for sharing. Your insights & warnings have been most helpful.

baffledshrimp2• 15 years ago

We run a program that monitors a significant number of Facebook Pages. It drives me crazy because every once in a while a certain attribute, like the author, is just omitted. It'll usually show up again later on.

growt• 15 years ago

We just work with the API within the usual scope (fb connect, etc) but it breaks regularly and things sometimes change without notice. It's really a joke for a company of that size. It has become kind of a running gag in the office (like: "Skip the tests, just release it. Today we do it facebook-style")

theli0nheart• 15 years ago

So this is how Breakup Notifier does it. It checks to see that the data is there. If at any point in time, the data I'm looking for is unavailable, I mark that field as "stale", don't vouch for the validity of its contents, and remove the ability for the user of the app to view that data.

When it comes back online, I mark it as not stale, and if it differs from the previous (non-stale) version, I send out an email.

anilchawla6• 15 years ago

Gotcha. I thought you might report updates only. The problem is that people can also remove their status instead of setting it back to "Single". Focusing on value updates only is probably fine and I'm sure nobody is going to freak out if the app misses a breakup :-P In my case, I was trying to create a more comprehensive notification that covered events such as new relationships (i.e. the user does not have a relationship status and one day finally sets it) and I couldn't afford the false positives.

michael_dorfman• 15 years ago

One motivation for writing this post was that I was surprised to see how often the Facebook API will drop/omit data that exists and that you should be able to access.

I loved the article, but I'm a bit surprised that you're surprised. What else would you expect from a massive NoSQL installation? It seems to me a given that they are going to trade some data integrity (in the form of missing attributes) for speed of access. I'd assume that was a fundamental design goal of the architecture.

joe_the_user• 15 years ago

Well, the graph API reports things as JSON. Your app has to page through the resulting json and see what information is relevant and meaningful.

I'm barely starting the Facebook API but the solution I'll use is to put the code for interpretating a given query into a series of configuration files using a DSL to interpret the Json. It's mostly a series of entries like:

    foo = [bar or baz or "default"]

This way any change in graph structure can be dealt with quickly (this would be a problem for XML/XSLT except they are too slow AND they don't do incremental interpretation).

saphead• 15 years ago

Regarding the Breakup Notifier reliability: "I had my boyfriend test it out by changing his status several times waiting about 10 minutes in between, and I never received an email notification. The site supposedly crawls Facebook every ten minutes for changes so it shouldn’t take long to be notified."

http://datingdissection.com/2011/02/21/breakup-notifier-make...

codebaobab2• 15 years ago

Anil, why didn't you just ignore fields and attributes that were missing? So if education:concentration returns "CompSci" on the first call, but then is missing on the second call, don't treat that as a change. Don't consider the attribute changed until the it is present and different.

soulclap2• 15 years ago

So what if it is missing on the first call? How many calls until you get the complete picture? And which of those results is 'the truth' and reflects the current status?

anilchawla6• 15 years ago

Right, I also thought I could work around the issues by ignoring fluctuations in the data. The problem is knowing what is the correct baseline data. In your example, the education:concentration might be missing when you first start checking and only start appearing a long time later. Do you ignore the data when it first appears or report that the person just added another major at college (which may be true!)? This is a bigger problem with fields that people might completely add/remove instead of simply updating (e.g. relationship status). I tried baking in a time period (e.g. 24 hours) in which I ignored all changes and waited for the baseline data to become obvious, but even that didn't work because some data took even longer to re-appear. At some point I had to admit to myself that - with all these work-arounds - my simple feature had turned into a hack.

codebaobab2• 15 years ago

I understand that having to work around fluctuating data might produce less than desirable results. But your users might find the feature useful anyway.

baffledshrimp2• 15 years ago

Also, some things like relationship status might go from 'dating' to blank after a breakup if the person doesn't want to scream that they're single right away

soulclap2• 15 years ago

Good read. And a good reminder that you shouldn't (or can't) rely on documentation or a single request here and there when you decide to work with an API.

And that the amount of support and 'release cycles' (frequency of updates and fixes) is something that you should always consider when integrating another piece of software into your own system.

apu• 15 years ago

Nice post.

Remember ChromeSpeed? Fancy seeing you here!

http://www.cs.columbia.edu/~neeraj/projects/chromespeed/

anilchawla6• 15 years ago

Ha - ChromeSpeed! Glad to see you have the project page hosted and the video on YouTube. Maybe we should port to iPhone? Definitely an Angry Birds killer... OK, sorry to go off topic, we should catch up offline (or online elsewhere)!

michaelchisari• 15 years ago

Great stuff, I'll keep this in mind as I build out the Appleseed Graph API. Especially good that you opened up with what you like about the Graph API, often-times articles like this skip right to the criticisms.

jimiray• 15 years ago

Having built a Facebook application that uses FB video I can understand where he's coming from. The Graph API is a great concept but like so many things Facebook, only halfway executed. It always amazed me that there are so many differences between a video in FQL and a video in the Graph API. In the end I figured out that "begin ... rescue ... end" was my friend when dealing with the Facebook apis. And about any call I make to the OpenGraph API is now wrapped in them.

JacobAldridge• 15 years ago

I won't go into the specifics of the article, because there are people far more qualified than I - I did enjoy it.

There's a lesson here in marketing: can you predict which name will get more buzz, 'Breakup Notifier' or 'Friend Mail'?

There's another lesson in marketing - you sell the sizzle, not the steak, but if the steak is awful you won't sell very many for very long regardless of how sizz-tastic your marketing is.

I can't compare both these products - and I hope they're both solid.

henrikschroder• 15 years ago

I recently spent some time integrating with the Facebook Credits API, and even though it's smaller, it's a similarly frustrating experience.

The documentation is sort of there, sometimes contradictory, and I always have to resort to reading their code examples in PHP to really figure out what data is expected, how to format it, and what I get back. None of their APIs are really well thought-out, they're all slapped together to sort of have all the relevant data. I understand that it's better to ship than not, but a little thought. Please?

The greatest irony though is that if you want to make a post asking for help in the official developer forum, you're met by this page:

http://forum.developers.facebook.net/register.php?agree=Agre...

acabal• 15 years ago

I tried to integrate my site, Scribophile, with FB a few years ago when the API was still relatively new. The idea was to give the user the option of adding a wall post to their profile when they posted writing to the site.

The API had the exact same problems then... unanswered questions, confusing documentation, complex code examples. Something that seemed so simple--post a link to someone's wall, with their permission--turned into a multi-day codefest. But I finally got it working!

Until one day a few months later, when the API changed without warning. And that's what I think the biggest problem with the FB API is... they (at least back in the day) change it constantly, without much warning, and don't update documentation in a sane way. It's impossible to build once and forget.

I finally got a complaint from a user one day that the FB integration wasn't working. When I went in to check the code, I found that FB had almost completely changed their JS libraries and made significant API changes. Instead of going through that hell again, I just removed the feature.

Zakuzaa• 15 years ago

How do you deal with spoofed emails?

terhechte• 15 years ago

I'm the developer of Clarity (clarity-app.com), a Facebook Client for the Mac and I've had a similar experience. Here's another couple of things that are weird.

- you can't like pictures or even see likes for pictures

- you can't post on your wall if the attached image is located in your fcebook album: so if you want to post an image you have to upload it somewhere else.

- you can't rename friend lists

- lots of things are inaccessible if your friends have strict security settings (like their birthday, or their name)

- as outlined in the article: the notification data structure is a train wreck

- you can't approve friend requests

- some parts of the API are not documented at all, like the 'attachment' field on the posts table: it contains a dictionary, but in order to understand the keys and values you have to read the raw data from lots of posts, and even then you never know if one future entry breaks your app.

- and much more

One problem I had was that users expected much of this to work because it works on the Facebook iPhone app too. But that one is official and does not need to use the limited app API.

anilchawla6• 15 years ago

Great list of additional restrictions/shortcomings. I forgot to mention that you can't post a photo that is hosted on the Facebook CDN - that is really annoying.

kilian• 15 years ago

I can understand all the complains about the facebook api, and they're all very valid, but my use of the facebook javascript API's have been relatively easy, to be honest. The documentation is far from ideal, and you have to figure stuff out by reading the actual response codes instead of trusting the documentation, but in general it all seems to work.

I am just wall posting and uploading photos though, and not doing any real data retrieval. Maybe facebook is more focussed on API's that put stuff in instead of taking things out? ;)

One long, intimate month with the Facebook API

Related Stories