SplinterDB: Closing the NVMe Bandwidth Gap
January 29, 2020 (NSH 3305)

NVMe and NVRAM present new challenges and opportunities to the world of storage systems. For example, RocksDB, a state-of-the-art key-value store, may use 30% or less of a single device’s bandwidth for insertions, even when using many cores. Because key-value store performance is essential to overall application performance, this can be a critical bottleneck.

In this talk, I discuss how to understand and model performance on NVMe and how to design theoretically optimal data structures which are also fast in practice.

I present SplinterDB, a key-value store which can saturate an NVMe device with low IO amplification under a wide range of parameters. SplinterDB outperforms RocksDB by 6-9x on write-heavy workloads, and by 30-80% on read-heavy workloads.

Bio:

Alex Conway is a PhD student in Computer Science at Rutgers University, advised by Martin Farach-Colton. His research is focused on external memory theory and storage systems. His recent publications include work on high performance key-value stores, file system aging, external memory hash tables and the analysis of storage optimization heuristics.

He is a member of the BetrFS team. BetrFS, a prototype file system, is designed around the principles of write-optimization and is built using variants of Bε-trees.