How To Scale Apps The Facebook Way (Video)How To Scale Apps The Facebook Way (Video)

As critical and operational applications drive today's business, one of the biggest challenges most organizations face is how to scale those applications (especially as they become distributed) and their data. Facebook is a great example of one of the most distributed, massive applications around. At Web 2.0, Facebook's VP of Engineering, Mike Schroepfer, talked about how they tackle this challenge.You can watch the crucial clip from Schroepfer's talk in the video below.

Schroepfer pointed out that most traditional architectures scale in linear fashion -- more application usage and more data, add more servers, split up the databases. Facebook was no different. It started at one site (Harvard) and even when it spread to other schools, those schools weren't connected. But this changed in 2006 and instantly Facebook had the challenge of a huge network with massive amounts of data and overwhelming communication and collaboration.

Just to put it in perspective, users spend collectively 8 billion minutes every day on Facebook. Those users upload 2 billion photos per month, and it stores more than 20 billion photos. Facebook measured a single day with 1.2 million photos per second. And that's just photos. Add the 15,000 Facebook Connect implementations and the 5 billion API calls per day and the scaling challenges this brings is daunting, to say the least.

At some point it can't be about buying more or bigger machines. To provide massive queries against a distributed database in real time required building Multifeed, which is a distributed system with in memory cache of recent activity (it handles 50 million operations per second). But Facebook has to fil this data out with the photos and comments and likes and other information that enriches a Facebook page. To do this, it uses Memcache, an open source tool that it customized to fetch all of the data in real time. In fact, Facebook scaled Memcache to run five times faster than its original performance.

Fritz Nelson is an Executive Editor at information and the Executive Producer of TechWebTV. Fritz writes about startups and established companies alike, but likes to exploit multiple forms of media into his writing.

Follow Fritz Nelson and information on Twitter, Facebook, YouTube and LinkedIn:

Twitter @fnelson @information@IWpremium

Facebook Fritz Nelson Facebook Page information Facebook Page

YouTube TechWebTV

LinkedIn Fritz Nelson on LinkedIn information