Overview

Despite continued innovations in disk design, input/output performance has not kept pace with concurrent increases in processor speeds. Much research has focused on developing algorithms to avoid input/output or hide input/output latency in an attempt to redress this widening gap. Many studies have shown that with advance knowledge of access patterns, file systems can improve input/output performance by selecting policies appropriate for the resource demands. Unfortunately, access patterns may be complex or data dependent, and therefore unknown a priori.

Our thesis is that the file system can automatically detect qualitative file access patterns both locally (per parallel program thread) and globally (per parallel program) and use this information to dynamically choose appropriate file system policies. We propose two complementary methods for automatic classification, based on neural networks and hidden Markov models, respectively. Global classifications are created from a combination of local classifications and additional access pattern information. We map qualititative classifications and quantitative statistics to file system policies shown to improve performance for those access patterns.

We have implemented this classification framework as extensions to the Portable Parallel File System (PPFS) testbed. Experimental results on sequential and parallel scientific applications demonstrate the utility of this approach.

More Information

• Tara Madhyastha, "Automatic Classification of Input/Output Access Patterns," Ph.D. Thesis, Aug 1997.
• Tara Madhyastha and Daniel A. Reed, "Input/Output Access Pattern Classification Using Hidden Markov Models," Fifth Workshop on Input/Output in Parallel and Distributed Systems, Nov 1997.
• Tara Madhyastha and Daniel A. Reed, "Exploiting Global Input/Output Access Pattern Classification," Proceedings of SC'97, Nov 1997.
• Tara M. Madhyastha and Daniel A. Reed, "Intelligent, Adaptive File System Policies," Frontiers of Massively Parallel Computation, Oct 1996.
• Tara M. Madhyastha, Christopher L. Elford and Daniel A. Reed, "Optimizing Input/Output Using Adaptive File System Policies," Fifth NASA Goddard Conference on Mass Storage Systems and Technologies, Sept 1996.