Proposal for a Common Parallel File System Programming Interface 1.0

Peter Corbett{1}, Jean-Pierre Prost{1}, Chris Demetriou, Garth Gibson, Erik Riedel, Jim Zelenka, Yuqun Chen{2}, Ed Felten{2}, Kai Li{2}, John Hartman{3}, Larry Peterson{3}, Brian Bershad{4}, Alec Wolman{4}, Ruth Aydt{5}

October 1996
School of Computer Science
Carnegie Mellon University
Pittsburgh, PA 15213
sio-os-api@cs.princeton.edu
http://www.cs.cmu.edu/Web/Groups/PDL/SIO

{1}IBM T. J. Watson Research Center
{2}Department of Computer Science, Princeton University
{3}Department of Computer Science, The University of Arizona
{4}Computer Science & Engineering, University of Washington
{5}Department of Computer Science, University of Illinois at Urbana-Champaign

Abstract

This document proposes an interface to parallel file systems intended for use with a variety of parallel computers. This proposal is based on the separation of programmer convenience functions from high-performance enabling functions. We propose that the former be supported above this interface, possibly in client libraries. The latter, functions that enable high performance, are defined by this proposed API under the assumption that these functions are more likely to need system and vendor-specific support.

Specifically, this proposal includes functions which support reading and writing with scatter-gather addressing for memory and file ranges, and asynchronous operations. It also includes mechanisms that permit client control over client caching, and file access and layout hints. Finally, it includes a mechanism by which this API can be extended and extensions for fast file copy and batching collective I/O operations.

Also appears as Technical Report CACR-130, Scalable I/O Initiative, Caltech Center for Advanced Computing Research, Pasadena, CA, November 1996.

Click here for the full paper in pdf or postscript