Engineering Blog

Blog posts tagged 'Data'

Under the Hood: Building out the infrastructure for Graph Search

Posted about 5 years ago
blog post · Data · Infra · Graph Search

The Facebook graph is the collection of entities and their relationships on Facebook. The entities are the nodes and the relationships are the edges. One way to think of this is if the graph were represented by language, the nodes would be the nouns and the edges would be the verbs. Every user, page, place, photo, post, etc. are nodes in this graph. Edges between nodes represent friendships, check-ins, tags, relationships, ownership, attributes, etc. Read more...

Alex GartrellEngineering

McDipper: A key-value cache for Flash storage

Posted about 5 years ago
blog post · Infra · Data · Web · Storage · Caching · Performance · Server Infrastructure · Data Centers · Photos

Memcache has been used at Facebook for everything from a look-aside cache for MySQL to a semi-reliable data-store for ads impression data. Of course RAM is relatively expensive, and for working sets that had very large footprints but moderate to low request rates, we believed we could make Memcached much more efficient. Compared with memory, flash provides up to 20 times the capacity per server and still supports tens of thousands of operations per second, so it was the obvious answer to this problem. Read more...

Louis BrandySoftware engineer at Facebook

Fighting spam with pure functions

Posted about 5 years ago
blog post · Web · Infra · Data · Security · Language Tools

Like any popular Internet site, Facebook is a target for abuse. Our Site Integrity engineers rely on FXL, a domain-specific language forged in the fires of spam fighting at Facebook, to quash this abuse before it can affect our users. Feature eXtraction Language (FXL) evolved in response to our need for a fast, flexible, safe way to write rules for identifying spam. Read more...

Lars Eilstrup RasmussenDirector of Engineering at Facebook

Under the Hood: Building Graph Search Beta

Posted about 5 years ago
blog post · Infra · Data · Graph Search

This morning we announced the beta of Graph Search, a new product that lets you explore the content your friends and others have shared with you on Facebook in entirely new ways. With Graph Search, you simply enter phrases such as "My friends who live in San Francisco," "Photos of my family taken in Copenhagen," or "Dentists my friends like," and Facebook quickly displays a page of the content you've requested. Read more...

Eric BarrettData Plumber at Facebook

Under the Hood: Automated backups

Posted about 5 years ago
blog post · Web · Data · Infra · Production Engineering

Facebook has one of the largest MySQL installations in the world, with thousands of database servers in multiple regions, so it's no surprise that we have unique challenges when we take backups. Our job is to keep every piece of information you add safe, while ensuring that anything that's been deleted is purged in a timely manner. Read more...

Under the Hood: Scheduling MapReduce jobs more efficiently with Corona

Posted about 5 years ago
blog post · Data · Infra · Open Source · Storage

Nearly every team at Facebook depends on our custom-built data infrastructure for warehousing and analytics, with roughly 1,000 people across the company – technical and non-technical – using these technologies every day. Over half a petabyte of new data arrives in the warehouse every 24 hours, and ad-hoc queries, data pipelines, and custom MapReduce jobs process this raw data around the clock to generate more meaningful features and aggregations. Read more...

Under the Hood: Building the App Center recommendation engine

Posted about 5 years ago
blog post · Infra · Data · Platform

As more apps on Facebook Platform have launched over the years, the types of apps available have become more diverse, making it crucial that people see the most relevant and highest quality apps in channels like news feed and App Center. Read more...

Sean LynchEngineering

Monitoring cache with Claspin

Posted about 5 years ago

When I started at Facebook, I joined the newly formed cache performance team in production engineering. Our goal was to get a handle on the health of our various cache systems and to facilitate quick troubleshooting, starting with answering the question, "Is this problem being caused by the cache?". Read more...

Philip SuSoftware engineer at Facebook

Announcing Facebook Engineering in London

Posted about 5 years ago
blog post · Mobile · Web · Data · Compute · London · Recruiting

Today we have some exciting news: We're now hiring for a new engineering team in London. Read more...

Paul TarjanWeb hacker at Facebook

Under the Hood: Timeline apps behind Facebook engineering

Posted about 5 years ago
blog post · Web · Infra · Data · Culture · Open Source · Timeline

A big part of working at Facebook is dogfooding our own products. Before new features roll out to the public, our team uses them frequently and thoroughly to figure out what we can do better. So when we started building Open Graph, we were all eager to give Open Graph apps a test run by creating our own and making the internal tools we were already using more social. Here's a look at some of our favorite in-house Facebook Open Graph apps for work and play, and what we've learned along the way that can help you, too. Read more...

Andrew RyanEngineering

Under the Hood: Hadoop Distributed Filesystem reliability with Namenode and Avatarnode

Posted about 6 years ago
blog post · Infra · Data · Storage · Open Source

The Hadoop Distributed Filesystem (HDFS) forms the basis of many large-scale storage systems at Facebook and throughout the world. Our Hadoop clusters include the largest single HDFS cluster that we know of, with more than 100 PB physical disk space in a single HDFS filesystem. Optimizing HDFS is crucial to ensuring that our systems stay efficient and reliable for users and applications on Facebook. Read more...

Karan ManglaSoftware engineer at Facebook

Under the Hood: Building the Location API

Posted about 6 years ago
blog post · Data · Infra · Platform

Earlier this week, we added the ability to tag location in stories published from apps on the open graph. Like with any update you make from your composer, location can now be added to any photo, link, or update written from an app back to Facebook, allowing people to add more context to their stories. Read more...

MySQL and Database Engineering: Mark Callaghan

Posted about 6 years ago

Mark Callaghan joined Facebook in 2009. Now, working from home in Oregon on the database infrastructure team, he does whatever it takes to keep the database running and users like Trekkie George Takei, a favorite person to get updates from amongst Mark's team, happy. Read on for Mark's insights into fast-storage challenges, Facebook's MySQL roadmap, and more. Read more...

Harry LiEngineering

Announcing the 2012-2013 Facebook Fellows

Posted about 6 years ago
blog post · Data · Web · Mobile · Culture · Compute · Research · Recruiting

We began the Facebook Fellowship Program in 2010 to foster ties to the academic community and support the research of promising computer science Ph.D. students. Since then, our Fellows have shown remarkable progress in their study of computer security, compiler technology, distributed systems, machine learning, and more. Because of our past winners' great successes, we decided to more than double the number of Fellowships for the 2012-2013 program. Read more...

Mick JohnsonDirector, Product Management at Facebook

Under the Hood: Mobile Timeline

Posted about 6 years ago
blog post · Data · Infra · Mobile · Timeline

Completely rewriting one of the fundamental pages of mobile Facebook was a huge engineering task – and required a lot of design work to get a model that would scale to all the interfaces we needed to support. All the stories needed to be re-implemented for mobile, with new designs to suit smaller viewports, different styling to match the capabilities of different devices, browsers and platforms, and additional interactions unique to touch devices to help navigate all of the new content. Fortunately the back-end work handled both mobile and desktop just fine, and by f8 in September we had completely designed and built working versions of all the stories, tabs and interactions. We’d been working round the clock in Palo Alto and Seattle for months, so to finally watch mobile Timeline scroll smoothly up the screen during the keynote was an amazing feeling. Read more...

Ryan MackEngineering

Building Timeline: Scaling up to hold your life story

Posted about 6 years ago
blog post · Data · Infra · Web · Timeline · User Experience

Timeline isn’t just a bold new look for Facebook­—it’s also the product of a remarkably ambitious engineering effort. While our earlier profile pages surfaced a few days or weeks of activity, from the onset we knew that with Timeline we had to think in terms of years and even decades. At a high level we needed to scan, aggregate, and rank posts, shares, photos and check-ins to surface the most significant events over years of Facebook activity. Read more...

David AlvesEngineering

Announcing Facebook’s 2012 Hacker Cup

Posted about 6 years ago
blog post · Web · Data · Culture · Compute · Hacking · Recruiting

Hacking is core to how we build at Facebook. Whether we’re building a prototype for a major product like Timeline at a Hackathon, creating a smarter search algorithm, or tearing down walls at our new headquarters, we’re always hacking to find better ways to solve problems. Read more...

Haiping ZhaoSenior Server Engineer at Facebook

Meet a Facebook Engineer: Haiping Zhao

Posted about 6 years ago
blog post · Culture · Data · HipHop · Testing · Languages

At Facebook, our engineers collaborate to create an open environment where ideas win and are executed quickly. Each week, our engineers will give you a look into what it's like to ideate and build at Facebook in our new "Meet a Facebook Engineer" Q&A series. Check back weekly to hear from different engineers about what problems they're passionate about solving right now, what they're up to at Facebook and what advice they have for you. Read more...

Arturo BejarEngineering

Compassion Research Day

Posted about 6 years ago
blog post · Data · Culture · Compute · Research · Academics

Read more...

Qiang WuInfrastructure Software Engineer at Facebook

Keeping the Site Reliable While Moving Fast

Posted about 6 years ago
blog post · Infra · Web · Data · Culture · Optimization · Performance · Backend · Platform · Chat · PHPmore

Read more...

Carlos BuenoFixer at Facebook

Doppler: Internet Radar

Posted about 6 years ago
blog post · Infra · Data · Web · Mobile · Performance · Optimization

The basic strategy for all performance and optimization work is the delicious measurement sandwich: measure, change something, then measure again. Detailed network measurements are especially hard to do because we only control one side of the transaction, our own servers. So we design network experiments that are lightweight, continuous, and gather as many samples as possible, even at the expense of detail and accuracy. A billion data points can cover a lot of methodological sins. Read more...

Yael MaguireEngineering
Donn LeeEngineering at Facebook

Facebook and World IPv6 Day

Posted about 7 years ago
blog post · Infra · Data · Testing

Read more...

Sanjeev KumarDirector Engineering at Facebook

How Project Triforce Prepared our Software Stack for Prineville

Posted about 7 years ago
blog post · Infra · Data · Compute · Hardware · Open Compute · Prineville · Data Centers · MySQL

For the first few years of Facebook’s existence, we served our users from data centers in a single region in Northern California. As the site grew, we added a second region of data centers in Virginia in 2007, and this year, we launched our third region in Prineville, Oregon. Read more...

Keep Updated

Stay up-to-date via RSS with the latest open source project releases from Facebook, news from our Engineering teams, and upcoming events.

Subscribe
Facebook © 2017