Engineering Blog

Blog posts tagged 'Open Source'

Introducing mcrouter: A memcached protocol router for scaling memcached deployments

Posted about 3 years ago

Most web-based services begin as a collection of front-end application servers paired with databases used to manage data storage. As they grow, the databases are augmented with caches to store frequently-read pieces of data and improve site performance. Often, the ability to quickly access data moves from being an optimization to a requirement for a site. This evolution of cache from neat optimization to necessity is a common path that has been followed by many large web scale companies, including Facebook, Twitter[1], Instagram, Reddit, and many others. Read more...

@Scale 2014 Agenda is Live!

Posted about 3 years ago
blog post · Web · Data · Infra · Mobile · Culture · Open Source · Open Compute · @Scale

@Scale conferences are regular industry events for engineers who build or maintain systems that are designed for scale. Read more...

James PearceEngineering at Facebook

9.9 million lines of code and still moving fast - Facebook open source in 2014

Posted about 3 years ago
blog post · Mobile · Culture · Data · Infra · @Scale · Open Source

The first six months of 2014 have been very busy for our open source program. In the spirit of the World Cup, we thought it was time for a half-time review of some of the highlights so far. Read more...

Introducing “Wedge” and “FBOSS,” the next steps toward a disaggregated network

Posted about 4 years ago

We’re big believers in the value of disaggregation – of breaking down traditional data center technologies into their core components so we can build new systems that are more flexible, more scalable, and more efficient. This approach has guided Facebook from the beginning, as we’ve grown and expanded our infrastructure to connect more than 1.28 billion people around the world. Read more...

Nick PetroEngineering

F8 Developer Conference - Hacker Way Recap

Posted about 4 years ago
blog post · Data · Infra · Mobile · Web · Performance · Optimization · Open Source · Hack · Messages · Design Tools · Android · iOSmore

Over 1,700 developers traveled to the Concourse Exhibition Center in San Francisco for Facebook’s F8 Developer Conference last week. Read more...

Open-sourcing Haxl, a library for Haskell

Posted about 4 years ago
blog post · Infra · Data · Web · Backend · Open Source · Caching · Languages · Security · Data Science · Analyticsmore

Today we're open-sourcing Haxl, a Haskell library that simplifies access to remote data, such as databases or web-based services. Read more...

HydraBase – The evolution of HBase@Facebook

Posted about 4 years ago
blog post · Data · Infra · Messages · Analytics · Storage · Platform · Open Source

When we revamped Messages in 2010 to integrate SMS, chat, email and Facebook Messages into one inbox, we built the product on open-source Apache HBase, a distributed key value data store running on top of HDFS, and extended it to meet our requirements. At the time, HBase was chosen as the underlying durable data store because it provided the high write throughput and low latency random read performance necessary for our Messages platform. In addition, it provided other important features, including horizontal scalability, strong consistency, and high availability via automatic failover. Since then, we’ve expanded the HBase footprint across Facebook, using it not only for point-read, online transaction processing workloads like Messages, but also for online analytics processing workloads where large data scans are prevalent. Today, in addition to Messages, HBase is used in production by other Facebook services, including our internal monitoring system, the recently launched Nearby Friends feature, search indexing, streaming data analysis, and data scraping for our internal data warehouses. Read more...

Kimon TsinterisEngineering Manager at Facebook

Introducing Pop, the animation engine behind Paper

Posted about 4 years ago
blog post · Mobile · iOS · Open Source · Framework · Design Tools

A couple of months ago we launched Paper, a fluid and beautiful way to explore and share stories. Today we’re open-sourcing Pop, the animation engine behind the application’s smooth animations and transitions. Using dynamic instead of traditional static animations, Pop drives the scrolling, bouncing, and unfolding effects that bring Paper to life. Read more...

Joel PobarEngineering

Hack Developer Day Recap

Posted about 4 years ago

A few weeks ago, Facebook introduced and open-sourced Hack, a gradually-typed programming language for HHVM that interoperates seamlessly with PHP. Yesterday we hosted our first Hack Developer Day to bring together members of the developer community for a deeper dive into Hack and HHVM. Read more...

Large-scale graph partitioning with Apache Giraph

Posted about 4 years ago
blog post · Infra · Data · Open Source · Graph · Graph Search · Performance · Optimization

Facebook’s architecture relies on various services that answer queries about people and their friends. Because of the size of the dataset, number of queries per second, and latency requirements, many of these systems cannot run on a single machine. Instead, people and their metadata are sharded across several machines. In such a distributed environment, answering queries might require communication among all these servers. Read more...

Andrei AlexandrescuResearch Scientist at Facebook

Under the Hood: warp, a fast C and C++ preprocessor

Posted about 4 years ago
blog post · Infra · Open Source · Languages

Today we are open-sourcing warp, a fast preprocessor for the C and C++ languages, written by Walter Bright in a joint project with Facebook. Read more...

Hack: a new programming language for HHVM

Posted about 4 years ago
blog post · Infra · Culture · Backend · Open Source · Hack · Development Tools · HipHop · Languages · Platform · Menlo Parkmore

Today we're releasing Hack, a programming language we developed for HHVM that interoperates seamlessly with PHP. Hack reconciles the fast development cycle of PHP with the discipline provided by static typing, while adding many features commonly found in other modern programming languages. Read more...

Lyrica McTiernanEngineering

Open sourcing PUE/WUE dashboards

Posted about 4 years ago
blog post · Infra · Open Source

Last April, we launched public dashboards that visualize real-time energy and water efficiency in our Oregon and North Carolina data centers. We’re proud of our data center efficiency, and we wanted to demystify data centers and share more about what our operations really look like. Read more...

Scott GoodsonEngineering Manager for Paper at Facebook

Building Paper

Posted about 4 years ago

A few weeks ago we launched Paper, a new app to explore content from your friends and the world around you—with built-in access to all the core features of Facebook. Through the process of implementing the fresh design and creating about 20 new categories of content, the team has developed new frameworks and architectural approaches to address the toughest challenges we encountered. In a series of blog posts, tech talks, and open source releases, we hope to give a comprehensive overview of the key parts of Paper's implementation and share some of our most valuable lessons learned while building this interaction-rich app on iOS. Read more...

Bryan KlimtEngineering

Building and open-sourcing Bolts, a mobile developer tools library

Posted about 4 years ago
blog post · Mobile · Infra · Open Source · Platform

When Thrift was originally conceived, most services were relatively straightforward in design. A web server would make a Thrift request to some backend service, and the service would respond. But as Facebook grew, so did the complexity of the services. Making a Thrift request was no longer so simple. Not only did we have tiers of services (services calling other services), but we also started seeing unique feature demands for each service, such as the various compression or trace/debug needs. Over time it became obvious that Thrift was in need of an upgrade for some of our specific use cases. In particular, we sought to improve performance for asynchronous workloads, and we wanted a better way to support per-request features. Read more...

Emil HesslowSoftware engineer at Facebook

HHVM - Implementing MySQLi

Posted about 4 years ago

The MySQLi extension is very popular amongst PHP developers. We knew we had to support it in our HHVM runtime, and thought it would be interesting to share the process we went through to implement it. Read more...

Andrei AlexandrescuResearch Scientist at Facebook

Under the Hood: Building and open-sourcing flint

Posted about 4 years ago
blog post · Infra · Open Source

Flint, Facebook’s lint program, issues the lint errors and warnings appear automatically in our code review system (phabricator) alongside each proposed code change, notifying the programmer where something may be going wrong. Flint has become important to the work that we do at Facebook, and I'm very excited to open-source flint so that everyone can check out what we’ve built and try it for themselves. Read more...

Dave WatsonWebmaster at Facebook

Under the Hood: Building and open-sourcing fbthrift

Posted about 4 years ago
blog post · Infra · Open Source

When Thrift was originally conceived, most services were relatively straightforward in design. A web server would make a Thrift request to some backend service, and the service would respond. But as Facebook grew, so did the complexity of the services. Making a Thrift request was no longer so simple. Not only did we have tiers of services (services calling other services), but we also started seeing unique feature demands for each service, such as the various compression or trace/debug needs. Over time it became obvious that Thrift was in need of an upgrade for some of our specific use cases. In particular, we sought to improve performance for asynchronous workloads, and we wanted a better way to support per-request features. Read more...

A Decade of Building Facebook

Posted about 4 years ago
blog post · Culture · Web · Data · Mobile · Menlo Park · Open Source · Open Compute · Data Centers · Luleå

Today we're celebrating Facebook's 10th anniversary. Check out a timeline of the engineering milestones that have built the infrastructure supporting 1.23 billion users, 201.6 billion friend connections, 400 billion shared photos, and 7.8 trillion messages sent since the start of 2012. Read more...

Subodh IyengarSoftware engineer at Facebook

Introducing Conceal: Efficient storage encryption for Android

Posted about 4 years ago
blog post · Web · Infra · Data · Security · Open Source · Android · Java · Development Tools · Caching · Storage · Performancemore

Caching and storage are tricky problems for mobile developers because they directly impact performance and data usage on a mobile device. Caching helps developers speed up their apps and reduce network costs for the device owner by storing information directly on the phone for later access. However, internal storage capacity on Android phones is often limited, especially with lower to mid range phone models. A common solution for Android is to store some data on an expandable SD card to mitigate the storage cost. What many people don't realize is that Android's privacy model treats the SD card storage as a publicly accessible directory. This allows data to be read by any app (with the right permissions). Thus, external storage is normally not a good place to store private information. Read more...

Scaling Mercurial at Facebook

Posted about 4 years ago
blog post · Infra · Open Source · Performance · Optimization

With thousands of commits a week across hundreds of thousands of files, Facebook's main source repository is enormous--many times larger than even the Linux kernel, which checked in at 17 million lines of code and 44,000 files in 2013. Given our size and complexity—and Facebook's practice of shipping code twice a day--improving our source control is one way we help our engineers move fast. Read more...

James PearceEngineering at Facebook

2013: A Year of Open Source at Facebook

Posted about 4 years ago
blog post · Data · Mobile · Web · Infra · Open Source · Languages

Open source has always been a huge part of the Facebook engineering philosophy. 2013 has been a great year for our open source program, with a significant number of new projects that we're really proud of, a renewed commitment to run and maintain them actively, and a desire to work with the vibrant communities that have built up around them. Read more...

Will BaileySoftware engineer at Facebook

Under the Hood: Building and open-sourcing the Rebound animation library for Android

Posted about 4 years ago
blog post · Mobile · Open Source · Android

About a month ago, Facebook hosted Mobile@Scale, the second in our series of small technical conferences with speakers from Facebook, LinkedIn, Pinterest, Dropbox, and Twitter. During the conference, we announced a new open-source spring dynamics animation library for Android called Rebound. We've seen a lot of interest in this project on GitHub, so I'd like to take this opportunity share some of the motivations and concepts behind this library, as well as some tips on how you can integrate it into your own applications to create physics-based animations. Read more...

Dhruba BorthakurEngineering

Under the Hood: Building and open-sourcing RocksDB

Posted about 4 years ago
blog post · Data · Infra · Backend · Production Engineering · Open Source · Storage

Every time one of the 1.2 billion people who use Facebook visits the site, they see a completely unique, dynamically generated home page. There are several different applications powering this experience--and others across the site--that require global, real-time data fetching. Read more...

Jay BorensteinEngineering

Facebook Open Academy: Bringing open source to CS curricula

Posted about 4 years ago
blog post · Culture · Open Source · Hacking · Academics

Open source is a huge part of Facebook engineering. Whether we're building new data infrastructure tools like Corona and Presto to manage our warehouses on Hadoop, releasing a new mobile build tool like Buck, or improving PHP runtime with HHVM, open source projects are integral to our operations. Read more...

Michael BolinSoftware engineering at Facebook

Buck: How we build Android apps at Facebook

Posted about 5 years ago
blog post · Mobile · Android · Tooling · Open Source · Platform

During my first Facebook hackathon in July 2012, I decided to create a new build system for Android that would address our needs. My primary objective was to create a build tool that favored the creation of many small modules rather than a handful of large modules. I hypothesized that once we had clear, fine-grained dependencies, we would be able to dig into the issue of slow build times. Read more...

Tim ArmstrongEngineering

LinkBench: A database benchmark for the social graph

Posted about 5 years ago
blog post · Data · Infra · Graph · MySQL · Performance · Optimization · Open Source · Testing · Storage

MySQL offers a good mix of flexibility, performance, and administrative ease, but the database engineering team continues to explore alternatives to MySQL for storing social graph data. There are several generic open-source benchmarks that could provide a starting point for comparing database systems. However, the gold standard for database benchmarking is to test the performance of a system on the real production workload, since synthetic benchmarks often don't exercise systems in the same way. When making decisions about a significant component of Facebook's infrastructure, we need to understand how a database system will really perform in Facebook's production workload. Read more...

Keep Updated

Stay up-to-date via RSS with the latest open source project releases from Facebook, news from our Engineering teams, and upcoming events.

Subscribe
Facebook © 2017