The Need for Speed Behind Everyday Computing

Researchers in Carnegie Mellon University's Kenneth C. Griffin School of Computer Science have developed new techniques to make computer caching systems faster and more efficient by improving how they decide which information to keep readily available and which to remove.

The CMU team set out to better understand how existing cache-management techniques affect computer performance and identify ways to reduce the work involved in managing caches without sacrificing their effectiveness.

A cache is a small, fast storage space that keeps frequently requested information close so computers don't have to retrieve it from a much slower source every time it's needed. Caches operate throughout the digital infrastructure people use every day, from databases and operating systems to websites and video services. As people browse the internet, every page load accesses many caches.

The challenge with caches is deciding what information deserves their limited space. A cache eviction algorithm makes those decisions, keeping frequently accessed information available while removing items less likely to be needed. But managing those decisions can also slow a computer down, especially when several requests need to be handled simultaneously.

One of the most common approaches, called least recently used (LRU), assumes that information accessed recently is likely to be needed again soon. Each time a computer requests an item in the cache, the algorithm moves it to the top of an ordered list. When space is needed, the least recently accessed item is removed.

"The problem is that maintaining this list takes work. Every time an item moves, the computer must coordinate that change with other operations," said Rashmi Vinayak, an associate professor in the Computer Science Department (CSD). "On modern servers with dozens or even hundreds of processing cores working simultaneously, those operations can create delays as different parts of the system wait for access to the same information."

To address this issue, Vinayek worked with the study's lead author Juncheng Yang, a CSD Ph.D. alumnus now at Harvard University, and CSD Ph.D. student Ziyue Qiu to investigate several ways to reduce the amount of work involved in keeping that list up to date. Some approaches wait rather than moving information every time it is accessed. Others give recently accessed information another chance to stay in the cache before removing it. These small changes can make the cache faster to manage while keeping the information people are most likely to need close at hand.

The team compared five existing cache-management approaches and found that some could make caches faster to manage without making them less effective. They also developed two new cache-management techniques that build on these ideas, helping computers handle requests more efficiently without sacrificing performance. But there's no one-size-fits-all approach.

"There's no one optimal algorithm," Yang said. "Different algorithms work well for different situations. Our findings give system designers a clearer understanding of which approaches are best for different workloads, rather than relying on trial and error."

Improving cache efficiency can help digital services respond faster, handle more requests and make better use of computing resources. Without caching, retrieving information for each request could take seconds or longer. With it, a webpage loads in a fraction of that time.

The research received a Best Paper Honorable Mention at the 2026 Very Large Data Bases conference, one of the leading conferences in data storage and database systems research.

By better understanding how computers manage the information they use most, the researchers hope to help build digital systems that can keep pace with growing demands, making the technology behind everyday online experiences faster and more efficient. To learn more, visit the project website.