rocksdb

Created November 14, 2013Updated 5 hours ago1945 Forks8794 Stars6617 Commits501 Open IssuesInfra · Backend · Data Infrastructure · Storage

RocksDB is an embeddable persistent key-value store for fast storage. RocksDB can also be the foundation for a client-server database but our current focus is on embedded workloads.

RocksDB builds on LevelDB to be scalable to run on servers with many CPU cores, to efficiently use fast storage, to support IO-bound, in-memory and write-once workloads, and to be flexible to allow for innovation.

We benchmarked LevelDB and found that it was unsuitable for our server workloads. The benchmark results look awesome at first sight, but we quickly realized that those results were for a database whose size was smaller than the size of RAM on the test machine - where the entire database could fit in the OS page cache. When we performed the same benchmarks on a database that was at least 5 times larger than main memory, the performance results were dismal.

By contrast, we've published the RocksDB benchmark results for server side workloads on Flash. We also measured the performance of LevelDB on these server-workload benchmarks and found that RocksDB solidly outperforms LevelDB for these IO bound workloads. We found that LevelDB's single-threaded compaction process was insufficient to drive server workloads. We saw frequent write-stalls with LevelDB that caused 99-percentile latency to be tremendously large. We found that mmap-ing a file into the OS cache introduced performance bottlenecks for reads. We could not make LevelDB consume all the IOs offered by the underlying Flash storage.

RocksDB can be used by applications that need low latency database accesses. A user-facing application that stores the viewing history and state of users of a website can potentially store this content on RocksDB. A spam detection application that needs fast access to big data sets can use RocksDB. A graph-search query that needs to scan a data set in realtime can use RocksDB. RocksDB can be used to cache data from Hadoop, thereby allowing applications to query Hadoop data in realtime. A message-queue that supports a high number of inserts and deletes can use RocksDB.

Created November 14, 2013Updated 5 hours ago1945 Forks8794 Stars6617 Commits501 Open IssuesInfra · Backend · Data Infrastructure · Storage

@rocksdb

RocksDB

RT @markcallaghan: For more from @axboe on recent improvements to Linux for storage see https://t.co/qBZov1kqmR. The 4.6 kernel has been go…...

RocksDB

RT @seemohan: @facebook's @SiyingD speaks on #RocksDB @RocksDB #KeyValueStore #KVS at #HPTS2017 #Asilomar https://t.co/6gdx4gr535...

RocksDB

Next RocksDB meetup at Facebook HQ on December 4th! https://t.co/VlS6p7eVim...

On Facebook

RocksDB

Video of the 3rd Annual RocksDB Meetup is now in youtube: https://www.youtube.com/playlist?list=PLb0IAmt7-GS2DTBGK1JS31JuDoQDuXbZp...

RocksDB

Our 3rd annual RocksDB meetup at Facebook HQ is only a week away. Agenda and sign-up here: https://www.meetup.com/RocksDB/events/234778003/...

RocksDB

RocksDB meetup is tonight! Hope to see you there! https://www.facebook.com/groups/rocksdb.dev/permalink/1018389214926259/...

Keep Updated

Stay up-to-date via RSS with the latest open source project releases from Facebook, news from our Engineering teams, and upcoming events.

Subscribe
Facebook © 2017