Engineering Blog

Blog posts tagged 'Data'

Jay TangEngineering

Building the Presto community

Posted about 2 years ago
blog post · Data · Infra · Analytics · Performance · Open Source

When we launched Presto, we saw dramatic query performance improvement across multiple internal Hadoop clusters. Read more...

Ed WolfProduct manager at Facebook

Instrumenting meetings at Facebook

Posted about 2 years ago
blog post · Data · Culture

Data from calendars, motion sensors, and videoconferencing allows us to analyze how meeting space is utilized so employees get the resources they need. Read more...

Meghan MarquezCommunications/PR at Facebook

Inside @Scale 2015

Posted about 2 years ago
blog post · Data · Mobile · @Scale · Open Source · Development Tools

A thousand engineers from hundreds of companies joining together to share lessons learned and best practices for building systems and applications at scale. Read more...

Lee ByronMobile hacker at Facebook

GraphQL: A data query language

Posted about 2 years ago
blog post · Mobile · Data · @Scale · Open Source · Languages · News Feed · IDE · Design Tools · Development Tools

GraphQL is a data query language and runtime designed and used at Facebook to request and deliver data to mobile and web apps since 2012. Read more...

Timothy YungEngineering Manager at Facebook

Relay: Declarative data for React applications

Posted about 2 years ago

We've been working on a solution to simplify the process of retrieving server data. Read more...

Grantland ChewEngineering

The Parse SDK: What's inside?

Posted about 2 years ago
blog post · Mobile · Data · Storage · Open Source

In this post, we'll unpack a few of the most challenging aspects of building the Parse SDKs — structuring an asynchronous API, decoupling architecture, and achieving API consistency. Read more...

Erin GreenEngineering

Using ISC Kea DHCP in our data centers

Posted about 2 years ago

Inside Facebook's transition to ISC Kea. Read more...

Ganapathy KrishnamoorthyEngineering at Facebook

Inside Data@Scale 2015

Posted about 2 years ago
blog post · Data · @Scale · Storage

This week, hundreds of engineers gathered in Seattle for the Data@Scale event for discussion and collaboration toward better solutions for scaling data storage and processing. Read more...

Recommending items to more than a billion people

Posted about 3 years ago
blog post · Infra · Data · Backend · Artificial Intelligence

Recommending items to more than a billion people using matrix factorization collaborative filtering. Read more...

Kristie ChowEngineering

Spam Fighting @Scale Recap

Posted about 3 years ago
blog post · Data · Web · Mobile · @Scale · Security · Platform · Data Science · Graph · User Experience

Hundreds of spam-fighting professionals gathered to hear engineers at Facebook, Pinterest, Dropbox, Yelp, and LinkedIn discuss techniques for fighting spam. Read more...

Under the hood: Facebook’s cold storage system

Posted about 3 years ago

Finding a place for images to live so they can be instantly available is a recurring scale challenge for Facebook. Read more...

How RocksDB is used in osquery

Posted about 3 years ago
blog post · Infra · Data · Backend · Security · Framework · Analytics · Storage · Open Source

Using RocksDB as osquery's embedded database allows osquery to store and access data in a fast, persistent way, enabling our team to solve some technical problems we'll detail in this blog. Read more...

Charity MajorsProduction Engineering Manager at Facebook

Parse now running MongoDB on RocksDB

Posted about 3 years ago
blog post · Data · Infra · Backend

Parse is running MongoDB on RocksDB and we're loving it. Read more...

Introducing ComponentKit: Functional and declarative UI on iOS

Posted about 3 years ago
blog post · Mobile · Data · iOS · Open Source · Performance · News Feed · Development Tools · Design Tools

We're open-sourcing ComponentKit, a native iOS library for functional and declarative UI. It is inspired by React and is used in News Feed in the Facebook iOS app. Read more...

Dain SundstromEngineering

Even faster: Data at the speed of Presto ORC

Posted about 3 years ago
blog post · Data · Backend · Open Source · Analytics · Performance · Testing

The Presto ORC reader is available in open source, and it's being used at Facebook, showing good results. Read more...

Hu LiEngineering

Introducing "Yosemite": the first open source modular chassis for high-powered microservers

Posted about 3 years ago
blog post · Infra · Data · Performance · Open Compute · Open Source · Compute · Data Centers · Hardware

To provide our infrastructure with capacity that scales out with the demand, we designed a modular chassis that contains high-powered system-on-a-chip (SoC) processor cards, code-named “Yosemite.” Today, we're proposing the Yosemite design as a contribution to the Open Compute Project. Read more...

Tomer BarEngineering at Facebook

Faster Photos in Facebook for iOS

Posted about 3 years ago
blog post · Mobile · Data · iOS · Photos · Performance · User Experience · Storage · News Feed

Faster Photos in Facebook for iOS. Read more...

Favorite Hacks of 2014

Posted about 3 years ago
blog post · Culture · Infra · Data · Language Tools · Hack · Android · London

We've celebrated a lot of big moments and launches over the last year - Facebook's 10th anniversary, Internet.org, and the new Hack language, to name a few. But while our teams were cranking away on major releases, we also took some time out for good old-fashioned hacking. We had 17 hackathons this year in Menlo Park, New York, Seattle, London, Vancouver, Tel Aviv, as well as a castle and a museum. Facebookers started over 500 projects focused on infrastructure, product, culture, and even coffee! Here are some of our favorite hacks from 2014 - may they inspire your own hacks in the new year!. Read more...

Ryan MackEngineering

Data @Scale - Boston Recap

Posted about 3 years ago
blog post · Data · Infra

Facebook started the @Scale conference series as a forum for engineers to share how they solve problems in the real world. It's since evolved and grown to become a community that collaborates in the open in order to innovate further, faster, and more effectively. We saw this firsthand during the web track sessions of @Scale, where companies at all stages of growth — Wikimedia, Twitter, Khan Academy, Google, and Netflix — are pushing the boundaries of what we thought was possible in web development. For many of these teams, they're also contributing those technologies back to the open source community, ensuring that others can learn and benefit from these lessons. Read more...

Audience Insights query engine: In-memory integer store for social analytics

Posted about 3 years ago
blog post · Web · Data · Infra · Production Engineering · Analytics · Data Science

A query engine with a hybrid integer store that organizes data in memory and on flash disks so that a query can process terabytes of data in real time. Read more...

Adam ErnstiOS Developer at Facebook

Making News Feed nearly 50% faster on iOS

Posted about 3 years ago
blog post · Mobile · Data · iOS · News Feed · User Experience · Optimization · Performance · Framework · Caching · Storage · Testingmore

We realized that while Core Data had served us well in the beginning, we needed to go without some of its features to accommodate our scale. We set about replacing it with our own solution, resulting in News Feed performing nearly 50% faster on iOS. Read more...

@Scale 2014: Recap of Data Track

Posted about 3 years ago
blog post · Data · Infra · Mobile · Caching · Testing · @Scale

Facebook started the @Scale conference series as a forum for engineers to share how they solve problems in the real world. It's since evolved and grown to become a community that collaborates in the open in order to innovate further, faster, and more effectively. We saw this firsthand during the web track sessions of @Scale, where companies at all stages of growth — Wikimedia, Twitter, Khan Academy, Google, and Netflix — are pushing the boundaries of what we thought was possible in web development. For many of these teams, they're also contributing those technologies back to the open source community, ensuring that others can learn and benefit from these lessons. Read more...

Phil DibowitzProduction Engineer at Facebook

Facebook, configuration management, community, and open source

Posted about 3 years ago
blog post · Infra · Data · Open Source · Production Engineering

Last year we began speaking at conferences around the world about our approach to managing hundreds of thousands of servers. We had outgrown our existing system and needed something new. We wanted a system that would let any engineer make any change they needed to any systems they owned via simple data-driven APIs while also scaling to Facebook's huge infrastructure, and while also minimizing the size of the team that would have to own the system. We designed a new paradigm and built a framework to bring it to life. At the core of that framework is Chef — but the way we ended up using Chef is pretty unique. We wanted to share how and why we made those choices and the benefits they brought us. Read more...

Lessons from Deploying MySQL GTID at Scale

Posted about 3 years ago
blog post · Data · MySQL · Production Engineering · Open Source

Global Transaction ID (GTID) is one of the most compelling new features of MySQL 5.6. It provides major benefits in failover, point-in-time backup recovery, and hierarchical replication, and it's a prerequisite for crash-safe multi-threaded replication. Over the course of the last few months, we enabled GTID on every production MySQL instance at Facebook. In the process, we learned a great deal about deployment and operational use of the feature. We plan to open source many of our server-side fixes via WebScaleSQL, as we believe others in the scale community can learn from this and benefit from the work we've done. Read more...

James PearceEngineering at Facebook

Introducing TODO: Working together to make open source easier

Posted about 3 years ago
blog post · Web · Data · Mobile · Infra · @Scale · Open Source

Today at @Scale 2014 we joined a number of other companies in launching a new open source collaboration called TODO. The group — whose name is a backronym for “talk openly, develop openly” — was formed to address the challenges that companies like ours have encountered in consuming open source software and running open source programs. Read more...

Introducing mcrouter: A memcached protocol router for scaling memcached deployments

Posted about 3 years ago

Most web-based services begin as a collection of front-end application servers paired with databases used to manage data storage. As they grow, the databases are augmented with caches to store frequently-read pieces of data and improve site performance. Often, the ability to quickly access data moves from being an optimization to a requirement for a site. This evolution of cache from neat optimization to necessity is a common path that has been followed by many large web scale companies, including Facebook, Twitter[1], Instagram, Reddit, and many others. Read more...

@Scale 2014 Agenda is Live!

Posted about 3 years ago
blog post · Web · Data · Infra · Mobile · Culture · Open Source · Open Compute · @Scale

@Scale conferences are regular industry events for engineers who build or maintain systems that are designed for scale. Read more...

Keep Updated

Stay up-to-date via RSS with the latest open source project releases from Facebook, news from our Engineering teams, and upcoming events.

Subscribe
Facebook © 2017