February 9, 2016Security · Open Source

In pursuit of secure open source software

Mike ArpaiaTed Reed

In late 2014, we released osquery to the open source community. It's now an increasingly important element of maintaining insight into the security of Facebook infrastructure. As such, it's held to incredibly strict security standards to ensure we're not introducing new vulnerabilities into our network. We also committed to a high standard of code quality when we open-sourced it because we want to build a community of trust with a secure software development ecosystem. In this same vein, we believe it's important for people who use osquery to know what we do to keep it secure.

Constant vigilance

Perhaps the most important aspect of writing secure software is constantly evaluating the security implications of every contribution made. If you're the author, you continually reason about potential ways in which a new feature could be abused, and you ask a lot of questions like these:

  • Are there configurable options that influence the behavior of this feature?
  • Can those options be influenced to make the feature do something undesirable?
  • Can a non-privileged user influence the operations of the software in an effort to carry out privileged actions?
  • If you're manually allocating new data structures on the heap, have you zeroed those data structures before and after using them?
  • Are you using “copy and move” operations that require explicit size parameters, e.g., snprintf vs. sprintf?
  • What assumptions are you making about library API constraints?
  • The actual list of questions that we, as defensive programmers, ask ourselves is much longer, but the point is that secure development requires us to proactively think about how an attacker might exploit our software. To better understand an attacker's point of view and develop a more offensive mindset, I suggest reading Mark Dowd's The Art of Software Security Assessment. Additional resources for implementing common design patterns securely in C and C++ include John Viega's Secure Programming Cookbook for C and C++: Recipes for Cryptography, Authentication, Input Validation & More and Robert Seacord's Secure Coding in C and C++.

    Full-pipeline awareness

    Even when developers think about the security implications of the software they create, it's important to bake security into the complete software development, build, and distribution pipeline. At this point in time, osquery uses a set of Jenkins build infrastructure to create pull requests, run unit and integration tests, run analyzers, build packages, sign packages, etc. If we were to assume the codebase itself is secure, the ability to arbitrarily modify the behavior of packaging hosts could still enable malicious behavior. That's why we also invest in the security of our build and packaging infrastructure.

    The osquery team puts a significant amount of effort into monitoring and protecting the attack surface of our Jenkins infrastructure. We presented on these efforts at the EkoParty security conference several months ago, and you can watch the video of that talk below. The Jenkins build infrastructure actually consists of two sets of build hosts — one is dedicated to building pull requests and unit tests, and the other is dedicated to building release packages.

    Integrating with bug bounty

    Facebook has a robust and mature bug bounty program for engaging with WhiteHat security researchers. When we launched osquery, we also added it to the Facebook bug bounty program and published a callout on our main GitHub project README file to encourage independent researchers to ethically submit vulnerability reports about the project.

    This callout included a link to additional resources like “Bug hunting osquery,” which outlines some tips and tricks to find vulnerabilities more easily and clarifies what we consider a high-priority attack surface. Most osquery team members come from an offensive security background, so we openly embrace our bug bounty program as an effective way to interact with bright, motivated security researchers from all around the world.

    We are also happy to reward creative and well-documented approaches to fuzzing and performing static or dynamic analysis against the codebase. Recently, we rewarded Luke Li for his effort using AFL to find a loosely documented SQLite Virtual Table API that led to a controlled crash.

    Third-party assessment

    In addition to our commitment to secure software development, we recognize the inevitable truth that new vulnerabilities can be introduced into any codebase over time. In an effort to mitigate this risk and add fresh eyes on the osquery codebase, we contracted NCC Group to participate in a security assessment of osquery. Although no critical or high-risk vulnerabilities were identified, we quickly issued fixes for the found vulnerabilities in a new version of osquery. The NCC Group report was delivered to us on October 23, 2015, and we released osquery version 1.6.0 on November 4, 2015. Today, we're sharing the vulnerability report in its entirety with the security community. You can download it here.

    Putting it all together

    We're very pleased with the community and developer ecosystem that has grown around osquery — and we're always excited for new members to join. Check out the osquery project at https://osquery.io/ and the code at https://github.com/facebook/osquery. We also have a public wiki at https://osquery.readthedocs.org/en/stable/. Join our Slack at https://osquery-slack.herokuapp.com/.

    Keep Updated

    Stay up-to-date via RSS with the latest open source project releases from Facebook, news from our Engineering teams, and upcoming events.

    Facebook © 2017