Engineering Blog

Blog posts tagged 'Data'

Using Apache Spark for large-scale language model training

Posted about 4 months ago
blog post · Data · Data Infrastructure · Analytics

The pipeline is modular, readable, and more maintainable, with reductions in both resource usage and data landing time. Read more...

Justin TellerEngineering

Beringei: A high-performance time series storage engine

Posted about 4 months ago

Beringei powers most of the performance and health monitoring at Facebook while enabling engineers and analysts to make decisions quickly with accurate, real-time data. Read more...

Tobias TieckeEngineering at Facebook

Open population datasets and open challenges

Posted about 6 months ago

Sharing data about how people are aggregated around the world can help solve challenges such as connectivity, infrastructure planning, humanitarian aid, and disaster response. Read more...

A comparison of state-of-the-art graph processing systems

Posted about 7 months ago
blog post · Data · Graph · Data Infrastructure · Performance · Backend

The study measured the relative performance and ability of two systems to handle large graphs, focusing on performance and usability. Read more...

Lauren RuganiTechnology Communications at Facebook

Highlights from @Scale 2016

Posted about 9 months ago
blog post · Data · Mobile · @Scale · Tooling · Video · Open Source · Artificial Intelligence

Engineers representing hundreds of companies gathered to discuss the challenges and opportunities of building apps and systems at scale. Read more...

Lauren RuganiTechnology Communications at Facebook

Facebook announces new tech at @Scale 2016

Posted about 9 months ago
blog post · Data · Mobile · @Scale · Video · Tooling · Performance

New data storage technologies, 360 video improvements, and performance tools were revealed throughout the day. Read more...

Apache Spark @Scale: A 60 TB+ production use case

Posted about 9 months ago
blog post · Data · Infra · Data Infrastructure · Analytics · Backend · Open Source

Through a series of performance and reliability improvements, we were able to scale Spark to handle a TB-scale entity ranking system in production. Read more...

Yoshinori MatsunobuDatabase Engineer at Facebook

MyRocks: A space- and write-optimized MySQL database

Posted about 9 months ago
blog post · Data · Infra · Storage · MySQL · Backend · Data Infrastructure

Deploying MyRocks to a database tier in one of our data center regions enabled a 50 percent reduction in storage requirements. Read more...

Smaller and faster data compression with Zstandard

Posted about 9 months ago
blog post · Data · Performance · Storage

With a performance-first design optimized for modern CPUs, Facebook's new compression algorithm translates directly to faster data transfer and smaller storage requirements. Read more...

Erin GreenEngineering

@Scale 2016 lineup announced!

Posted about 10 months ago

Registration for the 2016 @Scale conference is now open. Read more...

Lighting the way to deep machine learning

Posted about 11 months ago

Open source Torchnet helps researchers and developers build rapid and reusable prototypes of learning systems in Torch. Read more...

Surendra VermaEngineering at Facebook

Data @Scale, June 2016 — Recap

Posted about 11 months ago
blog post · Data · @Scale

Engineers working on large-scale storage systems and analytics discuss challenges and collaborate on new solutions. Read more...

Introducing DeepText: Facebook's text understanding engine

Posted about 12 months ago
blog post · Infra · Data · Artificial Intelligence · Research · News Feed

DeepText can understand with near-human accuracy the textual content of several thousand posts per second, spanning more than 20 languages. Read more...

Phil DibowitzProduction Engineer at Facebook

Facebook Chef cookbooks

Posted about a year ago
blog post · Infra · Data · Open Source · Production Engineering

This suite of cookbooks — along with a sample 'init' cookbook — will allow anyone who wants to use our model of Chef in their own environment to get started easily and quickly. Read more...

Ryan MackEngineering
Gautam RoySoftware engineer at Facebook

How we built Facebook Lite for every Android phone and network

Posted about a year ago

FB Lite is the fastest-growing version of Facebook, 100 million users in under nine months. Read more...

Connecting the world with better maps

Posted about a year ago

By applying computer vision techniques to satellite imagery, we can identify how populations are distributed in remote locations and determine the best way to provide connectivity in those areas. Read more...

NetNORAD: Troubleshooting networks via end-to-end probing

Posted about a year ago

NetNORAD troubleshoots issues independently of device polling to help keep Facebook's massive networking infrastructure up and running. Read more...

Chris MarraProduct Manager at Facebook

Favorite hacks of 2015

Posted about a year ago

The passion people have for ideas generated at hackathons results in everything from new products to open source tools. Read more...

Shaohua LiSoftware engineer at Facebook

Improving software RAID with a write-ahead log

Posted about a year ago

Software RAID has some drawbacks, which can be problematic at Facebook's scale. Using a write-ahead log can address some of these issues and improve reliability of the array. Read more...

Under the hood: Broadcasting live video to millions

Posted about a year ago
blog post · Data · Mobile · Networking and Traffic · iOS · Caching · Performance

Solving for traffic spikes through load balancing and enabling RTMP playback to bring latency down to a few seconds are some of the ways we enabled seamless live video sharing on Facebook. Read more...

Jay TangEngineering

Building the Presto community

Posted about 2 years ago
blog post · Data · Infra · Analytics · Performance · Open Source

When we launched Presto, we saw dramatic query performance improvement across multiple internal Hadoop clusters. Read more...

Ed WolfProduct Manager at Facebook

Instrumenting meetings at Facebook

Posted about 2 years ago
blog post · Data · Culture

Data from calendars, motion sensors, and videoconferencing allows us to analyze how meeting space is utilized so employees get the resources they need. Read more...

Meghan MarquezCommunications/PR at Facebook

Inside @Scale 2015

Posted about 2 years ago
blog post · Data · Mobile · @Scale · Open Source · Development Tools

A thousand engineers from hundreds of companies joining together to share lessons learned and best practices for building systems and applications at scale. Read more...

Lee ByronMobile hacker at Facebook

GraphQL: A data query language

Posted about 2 years ago
blog post · Mobile · Data · @Scale · Open Source · Languages · News Feed · IDE · Design Tools · Development Tools

GraphQL is a data query language and runtime designed and used at Facebook to request and deliver data to mobile and web apps since 2012. Read more...

Timothy YungEngineering

Relay: Declarative data for React applications

Posted about 2 years ago

We've been working on a solution to simplify the process of retrieving server data. Read more...

Grantland ChewEngineering

The Parse SDK: What's inside?

Posted about 2 years ago
blog post · Mobile · Data · Storage · Open Source

In this post, we'll unpack a few of the most challenging aspects of building the Parse SDKs — structuring an asynchronous API, decoupling architecture, and achieving API consistency. Read more...

Keep Updated

Stay up-to-date via RSS with the latest open source project releases from Facebook, news from our Engineering teams, and upcoming events.

Facebook © 2017