SCS Faculty Candidate

  • Gates Hillman Centers
  • ASA Conference Room
  • Post-doctoral Associate
  • PDOS Group, Computer Science and Artificial Intelligence Laboratory (CSAIL)
  • Massachusetts Institute of Technology

New Abstractions for High-Performance Datacenter Applications

Developing high-performance datacenter applications is complex and time-consuming today, as developers must understand and correctly implement subtle interactions between different backend systems. I describe an approach that redesigns core datacenter systems around new abstractions to substantially reduce their complexity while also improving their performance. This saves expensive developer time, uses datacenter servers more efficiently, and can enable new, previously impossible systems and applications.

I illustrate the impact of such redesigns with the example of Noria, a new system that recasts web application backends—i.e., databases and caches—as a streaming dataflow computation with a novel abstraction of partial state. Noria's partially-stateful dataflow brings classic databases' familiar query flexibility to scalable dataflow systems, simplifying applications and improving the backend's efficiency. For example, Noria increases the request load handled by a single server by 5-70x compared to state-of-the-art backends. Additional new abstractions from my research increase the efficiency of other datacenter systems (e.g., cluster schedulers), or enable qualitatively new kinds of systems, for example to protect private user data against exposure through application bugs.

Malte Schwarzkopf is a postdoc at MIT CSAIL, where he is a member of the Parallel and Distributed Operating Systems (PDOS) group. He received both his B.A. and Ph.D. from the University of Cambridge, and Malte and his collaborators have received an NSDI Best Paper Award and a EuroSys Best Student Paper Award. In his research, Malte designs and builds systems that aim to be both efficient and easy to use, and some of these systems have impacted industry practice.

Faculty Host: Mahadev Satyanarayanan

Computer Science Department

For More Information, Please Contact: