Engineering Blog

Blog posts tagged 'Caching'

This browser tweak saved 60% of requests to Facebook

Posted about 2 months ago
blog post · Web · Caching · Backend · Performance

The work resulted in more efficient browser caches across the web and dramatically improved page load times. Read more...

Arun SharmaEngineering

Dragon: A distributed graph query engine

Posted about a year ago
blog post · Data Infrastructure · Backend · Caching · Graph

Dragon monitors real-time updates to the social graph and creates several different types of indices that improve the efficiency of fetching, filtering, and reordering the data. Read more...

Shaohua LiSoftware Engineer at Facebook

Improving software RAID with a write-ahead log

Posted about a year ago

Software RAID has some drawbacks, which can be problematic at Facebook's scale. Using a write-ahead log can address some of these issues and improve reliability of the array. Read more...

Under the hood: Broadcasting live video to millions

Posted about a year ago
blog post · Data · Mobile · Networking and Traffic · iOS · Caching · Performance

Solving for traffic spikes through load balancing and enabling RTMP playback to bring latency down to a few seconds are some of the ways we enabled seamless live video sharing on Facebook. Read more...

Martin KaFai LauSoftware Engineer at Facebook

Linux IPv6 improvement: Routing cache on demand

Posted about 2 years ago

One of our pain points in deploying IPv6 is the size of the routing tree. We've solved this scalability issue by creating routing cache on demand. Read more...

Ryan AlbrechtEngineering

Web performance: Cache efficiency exercise

Posted about 2 years ago
blog post · Web · Production Engineering · Caching · Front End

My team was discussing the parts of facebook.com that are currently uncached, and the question came up: What is the efficiency of the cache since, at Facebook, we release new code twice a day? We were worried that our release process might be negatively impacting our cache performance. Read more...

Erin GreenEngineering

Inside Wikipedia's transition to HHVM

Posted about 2 years ago
blog post · Infra · Web · Backend · Open Source · Performance · Optimization · User Experience · Caching · PHP · HipHopmore

Inside Wikipedia's transition to HHVM. Read more...

Adam ErnstiOS Developer at Facebook

Making News Feed nearly 50% faster on iOS

Posted about 2 years ago
blog post · Mobile · Data · iOS · News Feed · User Experience · Optimization · Performance · Framework · Caching · Storage · Testingmore

We realized that while Core Data had served us well in the beginning, we needed to go without some of its features to accommodate our scale. We set about replacing it with our own solution, resulting in News Feed performing nearly 50% faster on iOS. Read more...

@Scale 2014: Recap of Data Track

Posted about 2 years ago
blog post · Data · Infra · Mobile · Caching · Testing · @Scale

Facebook started the @Scale conference series as a forum for engineers to share how they solve problems in the real world. It's since evolved and grown to become a community that collaborates in the open in order to innovate further, faster, and more effectively. We saw this firsthand during the web track sessions of @Scale, where companies at all stages of growth — Wikimedia, Twitter, Khan Academy, Google, and Netflix — are pushing the boundaries of what we thought was possible in web development. For many of these teams, they're also contributing those technologies back to the open source community, ensuring that others can learn and benefit from these lessons. Read more...

Introducing mcrouter: A memcached protocol router for scaling memcached deployments

Posted about 3 years ago

Most web-based services begin as a collection of front-end application servers paired with databases used to manage data storage. As they grow, the databases are augmented with caches to store frequently-read pieces of data and improve site performance. Often, the ability to quickly access data moves from being an optimization to a requirement for a site. This evolution of cache from neat optimization to necessity is a common path that has been followed by many large web scale companies, including Facebook, Twitter[1], Instagram, Reddit, and many others. Read more...

Alex SourovEngineering

Improving Facebook on Android

Posted about 3 years ago

In an effort to connect the next five billion, Facebook began to shift to a mobile-first company about two years ago. We trained hundreds of employees on mobile development, restructured internal teams to build for all platforms, and moved to a fast-paced release cycle. Read more...

Open-sourcing Haxl, a library for Haskell

Posted about 3 years ago
blog post · Infra · Data · Web · Backend · Open Source · Caching · Languages · Security · Data Science · Analyticsmore

Today we're open-sourcing Haxl, a Haskell library that simplifies access to remote data, such as databases or web-based services. Read more...

Wyatt LloydEngineering at Facebook

An analysis of Facebook photo caching

Posted about 3 years ago
blog post · Infra · Photos · Caching

Every day people upload more than 350 million photos to Facebook as of Dec 2012 and view many more in their News Feeds and on their friends’ Timelines. Facebook stores these photos on Haystack machines that are optimized to store photos. But there is also a deep and distributed photo-serving stack with many layers of caches that delivers photos to people so they can view them. Read more...

Wyatt LloydEngineering at Facebook

An analysis of Facebook photo caching

Posted about 3 years ago
blog post · Infra · Photos · Caching

When Thrift was originally conceived, most services were relatively straightforward in design. A web server would make a Thrift request to some backend service, and the service would respond. But as Facebook grew, so did the complexity of the services. Making a Thrift request was no longer so simple. Not only did we have tiers of services (services calling other services), but we also started seeing unique feature demands for each service, such as the various compression or trace/debug needs. Over time it became obvious that Thrift was in need of an upgrade for some of our specific use cases. In particular, we sought to improve performance for asynchronous workloads, and we wanted a better way to support per-request features. Read more...

Subodh IyengarSoftware Engineer at Facebook

Introducing Conceal: Efficient storage encryption for Android

Posted about 3 years ago
blog post · Web · Infra · Data · Security · Open Source · Android · Java · Development Tools · Caching · Storage · Performancemore

Caching and storage are tricky problems for mobile developers because they directly impact performance and data usage on a mobile device. Caching helps developers speed up their apps and reduce network costs for the device owner by storing information directly on the phone for later access. However, internal storage capacity on Android phones is often limited, especially with lower to mid range phone models. A common solution for Android is to store some data on an expandable SD card to mitigate the storage cost. What many people don't realize is that Android's privacy model treats the SD card storage as a publicly accessible directory. This allows data to be read by any app (with the right permissions). Thus, external storage is normally not a good place to store private information. Read more...

Domas MituzasInfrastructure Engineer at Facebook

Flashcache at Facebook: From 2010 to 2013 and beyond

Posted about 3 years ago
blog post · Infra · Data · Storage · Caching · Performance · Optimization

We recently released a new version of Flashcache, kicking off the flashcache-3.x series. We’ve spent the last few months working on this new version, and our work has resulted in some performance improvements, including increasing the average hit rate from 60% to 80% and cutting the disk operation rate nearly in half. Read more...

Mark MarchukovEngineering

TAO: The power of the graph

Posted about 4 years ago
blog post · Data · Infra · Caching · Production Engineering

Facebook puts an extremely demanding workload on its data backend. Every time any one of over a billion active users visits Facebook through a desktop browser or on a mobile device, they are presented with hundreds of pieces of information from the social graph. Users see News Feed stories; comments, likes, and shares for those stories; photos and check-ins from their friends -- the list goes on. The high degree of output customization, combined with a high update rate of a typical user’s News Feed, makes it impossible to generate the views presented to users ahead of time. Thus, the data set must be retrieved and rendered on the fly in a few hundred milliseconds. Read more...

Laurent DemaillySoftware Engineer at Facebook

Wormhole pub/sub system: Moving data through space and time

Posted about 4 years ago
blog post · Data · Infra · Production Engineering · Caching

Over the last couple of years, we have built and deployed a reliable publish-subscribe system called Wormhole. Wormhole has become a critical part of Facebook's software infrastructure. At a high level, Wormhole propagates changes issued in one system to all systems that need to reflect those changes – within and across data centers. Read more...

Scaling memcache at Facebook

Posted about 4 years ago
blog post · Data · Infra · Caching · Production Engineering · Storage

Facebook started using memcached in August 2005 when Mark Zuckerberg downloaded it from the Internet and installed it on our Apache web servers. At that time, Facebook was starting to make increasingly sizable database queries on every page load, and page load times were significantly increasing. Providing a fast, snappy user experience has always been a high priority for Facebook, and memcached came to the rescue. Read more...

Alex GartrellEngineering

McDipper: A key-value cache for Flash storage

Posted about 4 years ago
blog post · Infra · Data · Web · Storage · Caching · Performance · Server Infrastructure · Data Centers · Photos

Memcache has been used at Facebook for everything from a look-aside cache for MySQL to a semi-reliable data-store for ads impression data. Of course RAM is relatively expensive, and for working sets that had very large footprints but moderate to low request rates, we believed we could make Memcached much more efficient. Compared with memory, flash provides up to 20 times the capacity per server and still supports tens of thousands of operations per second, so it was the obvious answer to this problem. Read more...

Sean LynchEngineering

Monitoring cache with Claspin

Posted about 5 years ago

When I started at Facebook, I joined the newly formed cache performance team in production engineering. Our goal was to get a handle on the health of our various cache systems and to facilitate quick troubleshooting, starting with answering the question, "Is this problem being caused by the cache?". Read more...

Minghui YangSoftware Engineer at Facebook

Making HPHPi Faster

Posted about 5 years ago
blog post · Infra · Web · PHP · HipHop · Open Source · Performance · Development Tools · Languages · C++ · Optimization · Caching · Testingmore

Read more...

Robert JohnsonDirector, Software Engineering at Facebook

More Details on Today's Outage

Posted about 7 years ago
blog post · Infra · User Experience · Performance · Caching

Early today Facebook was down or unreachable for many of you for approximately 2.5 hours. This is the worst outage we’ve had in over four years, and we wanted to first of all apologize for it. We also wanted to provide much more technical detail on what happened and share one big lesson learned. Read more...

Carlos BuenoFixer at Facebook

Internet Cartography

Posted about 7 years ago

A telegram from San Francisco to Hong Kong in 1901 must have taken many hops through British Empire cables to Europe, through the Middle East, and so on. London to New York was fast and direct. The vestiges of the Spanish and Portuguese Empires show up in the many links between South America, the Caribbean archipelago, and the Iberian peninsula. A cool thing is that you can measure these relative latencies yourself, using the present-day internet. If you run a website with a decent amount of worldwide traffic, you can use that traffic to map out how the internet responds with regards to you, and see how that matches with the gross structure of the 'net. I wrote about a cheap and cheerful way to generate this data last year, and the code has since been open-sourced as part of Yahoo's Boomerang measurement framework. Read more...

Akhil WableEngineering

Intro to Facebook Search

Posted about 7 years ago
blog post · Infra · Data · Web · Caching · User Experience

Connecting and sharing with others is Facebook’s primary value. That value necessitates having the ability to easily and efficiently find the people and information we care about. The search team at Facebook is focused on building a search product to enable our more than 400 million users to quickly find what they're looking for. In July 2007 we explained the complexities of serving one of the largest user bases in the world and the reasons for building our own in-house search service. Serving more than 150 million queries a day, and supporting a user base that has grown by more than 10x since then reinforces that decision. Read more...

Designing the Facebook username land rush

Posted about 8 years ago
blog post · Web · Data · Caching · User Experience · Performance · Optimization · Testing

We recently hit a milestone of 50MM usernames a few weeks ago — in just over a month since we launched usernames on June 12. Ever since we launched usernames, we’ve had a lot of people express interest in understanding how we designed the system and prepared for this big event. In a recent post, my colleague Tom Cook wrote about the site reliability and infrastructure work that we did to ensure a smooth launch. As an extension to that post, I’ll discuss some specific application and system design issues here. Launching usernames to allow over 200 million (at the time — we’re now over 250 million) people to get a username at the same time presented some really interesting performance and site reliability challenges. The two main parts of the system that needed to scale were (1) the availability checker and (2) the username assigner. Since we were pre-generating suggestions for users, we needed to check availability of all the suggested names, which placed extra load on the availability checker. Read more...

Keep Updated

Stay up-to-date via RSS with the latest open source project releases from Facebook, news from our Engineering teams, and upcoming events.

Subscribe
Facebook © 2017