Facebook's software architecture

http://muratbuffalo.blogspot.com/2014/10/facebooks-software-architecture.html?spref=tw

by colbyaley • 12 years ago

295 22 12 years ago

muratbuffalo.blogspot.com

Yadi2• 12 years ago

Great post. Thanks this is an awesome blog post, I love it when someone summarizes the good stuff :D! I wish someone would do the same for the AWS cloud papers.

Wow TIL what Tao is and what it can do.

This part is very sexy to me: Facebook's new architecture splits the media into two categories: 1) hot/recently-added media, which is still stored in Haystack, and 2) warm media (still not cold), which is now stored in F4 storage and not in Haystack.

kukla• 12 years ago

Some AWS related links from that blog: http://muratbuffalo.blogspot.com/2010/11/dynamo-amazons-high...

http://muratbuffalo.blogspot.com/2012/10/building-fault-tole...

http://muratbuffalo.blogspot.com/2013/04/aws-summit-nyc-day-...

http://muratbuffalo.blogspot.com/2013/04/aws-summit-nyc-rest...

http://muratbuffalo.blogspot.com/2013/04/aws-summit-nyc-day-...

Yadi2• 12 years ago

Ow my! This is Christmas for me right here! Thanks again, this is awesome!

JSno• 12 years ago

the title of his blog"Facebook's software architecture" is misleading. actually these papers are talking about data storage and database. not "software architecture." Though, his blog is informative and consistent. Love it!

EpicEng• 12 years ago

Digital data and information storage is not a component of a software architecture? Of course it is, and anyone working on a similar project would need to include these layers in their architectural specifications.

oebs• 12 years ago

One could argue that software architecture is the inner structure of an application, i.e. the libraries, modules, classes, functions and, most importantly, their interfaces. The GOF design patterns are purely a matter of software architecture, for example.

System architecture on the other hand describes the different applications/services that compromise a system, including the ones that you write yourself, plus the ones that you just "use", e.g. database, storage, load balancer, etc.

I would argue that the article mostly talks about system architecture with focus on storage and I guess the parent you replied to would too.

23david2• 12 years ago

I found this part interesting: "F4 currently stores over 65PB of logical data and saves over 53PB of storage."

It looks like most of the savings here is due to optimization of the replication level of storage, according to the usage patterns of the data.

Also found it interesting that it doesn't talk about CDN usage.

mark_integerdsv• 12 years ago

What kind of footprint would a system of this nature require I wonder? Geographically speaking.

vidarh• 12 years ago

You can use 1-3 petabyte per rack as a rough yardstick (before you start counting RAID overhead etc.)

It can be more/less than this depending on your rack configuration and how big drives you use, and how dense you pack them (e.g. "standard" off the shelf servers typically peak at 24 3.5" drives per 4U, while top-loaded specialised storage enclosures reach about 50 x 3.5" drives per 4U; full racks vary between 42U and around 47U etc.)

But you could if you wanted to get this much storage into about a 5m x 5m server room....

Of course you wouldn't, since you'd want multi-datacentre redundancy etc. Often there's also no point using the biggest drives available because you need higher throughput. Or no point packing them as tight as possible because you need more processing power per drive.

nbm2• 12 years ago

What would you have liked to know about CDN usage?

(I work on the team that builds and runs the Facebook CDN infrastructure.)

23david2• 12 years ago

If the replication factor for blobs is calculated from the 'hotness' of the data (by looking at the age for example), I'm wondering how CDNs come into play.

Increasing the replication factor would allow for faster reads, but there usually wouldn't be a need for that if you offload reads of hot data to a CDN service (could be an internal CDN...).

dbarlett• 12 years ago

"An Analysis of Facebook Photo Caching" [1] discusses CDN usage and complements the original Haystack paper [2] [3] well.

[1] http://www.cs.cornell.edu/~qhuang/papers/sosp_fbanalysis.pdf

[2] https://www.usenix.org/legacy/event/osdi10/tech/full_papers/...

[3] http://muratbuffalo.blogspot.com/2010/12/finding-needle-in-h...

akurilin2• 12 years ago

On this topic, where do you guys learn more about architecture of large conglomerates of web services and the latest trends?

I read High Scalability and the occasional company blog. Is there anything else out there that might be even better? Blogs, books, forums, doesn't matter.

louyang• 12 years ago

Get a job at one of them, cant quite beat that.

akurilin2• 12 years ago

I did already. There are multiple issues with that approach: you get only flavor of how to do things, unless you keep jumping teams over and over again for years. It's extremely inefficient way of learning this.

You're also putting the burden on the folks in charge to know what's out there and pass it down to you, instead of actually doing your own research.

Also, sometimes those people don't know what they're doing and they're struggling just as much as you would be in their position, so now you're learning from people who are making it up as they go, which you could have done equally as well by yourself.

dantiberian2• 12 years ago

I've learned heaps from reading the papers that these companies publish. Google, Facebook, and Amazon all publish well written, readable, papers. If you're looking for somewhere to start, then the Dynamo paper is a classic: http://www.allthingsdistributed.com/files/amazon-dynamo-sosp...

gansai• 12 years ago

just curious: "learned heaps" --> did it just improve your knowledgebase or it actually helped you practically to solve a problem which you were facing.? I am just curious, as I am a starter to reading these kind of papers. thanks.

rawnlq• 12 years ago

This is a talk about the evolution of Dropbox's architecture from 2012: http://www.youtube.com/watch?v=PE4gwstWhmc. It is incredibly detailed (down to the exact sql schema they use for file metadata etc).

RivieraKid• 12 years ago

+1 and I'm particularly interested in Amazon's architecture, I couldn't find much info about it (besides High Scalability). Can anyone from Amazon answer questions?

shaydoc• 12 years ago

Excellent article. I like the architecture explanation. I also like the simplicity of the description, makes it really easy to understand the descision making process.

I think it is a very logical solution. Naturally when I think of facebook, its the freshness of the data that is important for me, as a user I like to know whats going on now, as opposed to older timeframes. I think it seems quite logical that it is much more optimal to manage their data this way.

Facebook's software architecture

Related Stories