Streams Library
The Streams Library
Authors: Scott McKay, Bill Chiles, Marc Ferguson, and Eliot Miranda.
Note: This document was written by Harlequin, Inc., and not by the Gwydion Project.
Editor: Andrew Shires.
Date: 06 Feb 1997.
0.1 About this document
0.1.1 Acknowledgments
We’d like to thank the other people who have been instrumental in the production of this proposal: Jonathan Bachrach, Dave Berry, John Dunning, Chris Fry, Paul Haahr, William Lott, Rob Maclachlan, Tim McNerney, Tony Mann, Keith Playford, Robert Stockton, and Tucker Withington.
0.1.2 Discussing error conditions
This document uses two special terms in discussions of error conditions.
When we say that something is an error, we mean that the result is undefined. In particular, we do not mean that a Streams implementation must signal an error condition; that is the implementor’s choice. So, for instance, the following text, from page 79, means only that the result of using unread-element in the case described is undefined:
It is an error to apply unread-element to an element that is not the element most recently read from the stream.
Only when we specifically mention signaling do we mean that a Streams implementation must signal an error condition. Note that we may not, in such a case, say exactly which error condition must be signaled; if we do not say so, the choice is again up to the implementor. In this text from the description of stream-position-setter on page 77, for instance, we state that an implementation must signal an error, but we do not say what error must be signaled:
When position is a <stream-position>, if it is invalid for some reason, this function signals an error.
By contrast, the following text from the description of read-element on page 67 says exactly which error must be signaled:
If the end of the stream is encountered and no value was supplied for on-end-of-stream, read-element signals an <end-of-stream-error> condition.
0.2 Goals of the library
The Dylan Streams library aims to provide:
- A generic, easy-to-use interface for streaming over sequences and files. The same high-level interface for consuming or producing is available irrespective of the type of stream, or the types of the elements being streamed over.
- Efficiency, especially for the common case of file I/O.
- Access to an underlying buffer management protocol.
- An extensible framework. Other areas of functionality that require a stream interface should be easy to integrate with the library.
The proposal presents the design of a Streams library that meets these goals using Dylan’s built-in sequences and a buffered disk file interface.
The proposal does not address a number of related issues, including:
- A standard object-printing package such as Smalltalk’s printOn: or Lisp’s print-object, or a formatted printing facility such as Lisp’s format. Additional libraries are expected to provide these facilities.
- General object dumping and loading.
- A comprehensive range of I/O facilities for using memory-mapped files, network connections, and so on. Such facilities should be easy to add to the Streams library because of its extensible framework.
- An interface for naming files.
- An interface to operating system functionality, such as file renaming or deleting operations.
0.3 Concepts
A stream provides sequential access to an aggregate of data, such as a Dylan sequence or a disk file. Streams grant this access according to a metaphor of reading and writing: elements can be read from streams or written to them.
Streams are represented as Dylan objects, and all are general instances of the class <stream>, which the Streams library defines.
We say that a stream is established over the data aggregate. Hence, a stream providing access to the string "hello world" is said to be a stream over the string "hello world".
Streams permitting reading operations are called input streams. Input streams allow elements from the underlying data aggregate to be consumed. Conversely, streams permitting writing operations are called output streams. Output streams allow elements to be written to the underlying data aggregate. Streams permitting both kinds of operations are called input-output streams.
The library provides a set of functions for reading elements from an input stream. These functions hide the details of indexing, buffering, and so on. For instance, the function read-element reads a single data element from an input stream.
The following expression binds stream to an input stream over the string "hello world":
let stream = make(<string-stream>, contents: "hello world");
The first invocation of read-element on stream returns the character ’h’, the next invocation ’e’, and so on. Once a stream has been used to consume all the elements of the data, the stream is said to be at its end. This condition can be tested with the function stream-at-end?. The following code fragment applies function to all elements of the sequence:
let stream = make(<sequence-stream>, contents: seq);
while (~stream-at-end?(stream))
function(read-element(stream));
end;
When all elements of a stream have been read, further calls to read-element result in the <end-of-stream-error> condition being signalled. An alternative end-of-stream behavior is to have a distinguished end-of-stream value returned. You can supply such an end-of-stream value as a keyword argument to the various read functions; the value can be any object. Supplying an end-of-stream value to a read function is more efficient than asking whether a stream is at its end on every iteration of a loop.
The library also provides a set of functions for writing data elements to an output stream. Like the functions that operate upon input streams, these functions hide the details of indexing, growing an underlying sequence, buffering for a file, and so on. For instance, the function write-element writes a single data element to an output stream.
The following forms bind stream to an output stream over an empty string and create the string "I see!", using the function stream-contents to access all of the stream’s elements.
let stream = make(<byte-string-stream>, direction: #"output");
write-element(stream, ’I’);
write-element(stream, ’ ’);
write(stream, "see");
write-element(stream, ’!’);
stream-contents(stream);
Calling write on a sequence has the same effect as calling write-element on all the elements of the sequence. However, it is not required that write be implemented directly in terms of write-element; it might be implemented more efficiently, especially for buffered streams.
Some streams are positionable; that is, they permit random access to their elements. Postionable streams allow you to set the position at which the stream will be accessed by the next operation. The following example uses positioning to return the character ‘w’ from a stream over the string "hello world":
let stream = make(<string-stream>, contents: "hello world");
stream-position(stream) := 6;
read-element(stream);
The following example returns a string, but the contents of the first ten characters are undefined:
let stream = make(<string-stream>, direction: #"output");
adjust-stream-position(stream, 10);
write(stream, "whoa!");
stream-contents(stream);
You can request a sequence containing all of the elements of a positionable stream by calling stream-contents on it. The sequence returned never shares structure with any underlying sequence that might be used in future by the stream. For instance, the string returned by calling stream-contents on an output <string-stream> will not be the same string as that being used to represent the string stream.
When making an input <string-stream>, you can cause the stream to produce elements from any subsequence of the supplied string. For example:
read-to-end(make(<string-stream>,
contents: "hello there, world",
start: 6,
end: 11));
This example evaluates to "there". The interval (start, end) includes the index start but excludes the index end. This is consistent with standard Dylan functions over sequences, such as copy-sequence. The read-to-end function is one of a number of convenient utility functions for operating on streams and returns all the elements up to the end of the stream from the stream’s current position.
0.3.1 Streams, growing sequences, and object identity
When writing to output streams over sequences, Dylan may from time to time need to grow the underlying sequence that it is using to represent the stream data.
Consider the example of an output stream instantiated over an empty string. As soon as a write operation is performed on the stream, it is necessary to replace the string object used in the representation of the string stream. As well as incurring the cost of creating a new string, the replacement operation can affect the integrity of other references to the string within the program.
To guarantee that alias references to a sequence used in an output <sequence-stream> will have access to any elements written to the sequence via the stream, supply a <stretchy-vector> to make. A stream over a stretchy vector will use the same stretchy vector throughout the stream’s existence.
For example:
let sv = make(<stretchy-vector>);
let stream = make(<sequence-stream>,
contents: sv,
direction: #"output");
write(stream,#(1, 2, 3, 4, 5, 6, 7, 8, 9));
write(stream,"ABCDEF");
values(sv, stream-contents(stream));
The example returns two values. Each value is the same (\==) stretchy vector:
(1, 2, 3, 4, 5, 6, 7, 8, 9, ’A’, ’B’, ’C’, ’D’, ’E’, ’F’)
If a stretchy vector is not supplied, the result is different:
let v = make(<vector>, size: 5);
let stream = make(<sequence-stream>,
contents: v,
direction: #"output");
write(stream,#(1, 2, 3, 4, 5, 6, 7, 8, 9));
write(stream,"ABCDEF");
values(v, stream-contents(stream));
This example returns as its first value the original vector, whose contents are undefined, but the second value is a new vector:
(1, 2, 3, 4, 5, 6, 7, 8, 9, ’A’, ’B’, ’C’, ’D’, ’E’, ’F’)
This difference arises because the output stream in the second example does not use a stretchy vector to hold the stream data. A vector of at least 15 elements is necessary to accommodate the elements written to the stream, but the vector supplied, v, can hold only 5. Since the stream cannot change v’s size, it must allocate a new vector each time it grows.
0.4 Stream classes
The exported streams class heterarchy is as follows:
Open abstract class
A subclass of <stream> supporting the Stream Extension Protocol and the Buffer Access Protocol. It is not instantiable.
Streams of this class support the buffer-size: init-keyword, which can be used to suggest the size of the stream’s buffer. However, the instantiated stream might not use this value: it is taken purely as a suggested value. For example, a stream that uses a specific device’s hardware buffer might use a fixed buffer size regardless of the value passed with the buffer-size: init-keyword.
<file-stream>
Open abstract class
The class of streams over disk files. It is a subclass of <positionable-stream> and <buffered-stream>.
When you instantiate this class, an implementation-dependent, indirect instance of it is created. The file being streamed over is opened immediately upon creating the stream.
The class supports several init-keywords: locator:, direction:, if-exists:, and if-does-not-exist:.
<sequence-stream>
Open instantiable class
The class of streams over sequences. It is a subclass of <positionable-stream>.
The <sequence-stream> class can be used for streaming over all sequences, but there are also subclasses <string-stream>, <byte-string-stream>, and <unicode-string-stream>, which are specialized for streaming over strings.
The class supports several init-keywords: contents:, direction:, start:, and end:.
<string-stream>
Open instantiable class
The class of streams over strings. It is a subclass of <sequence-stream>.
The class supports the same init-keywords as <sequence-stream>.
<byte-string-stream>
Open instantiable class
The class of streams over byte strings. It is a subclass of <string-stream>.
The class supports the same init-keywords as <sequence-stream>.
<unicode-string-stream>
Open instantiable class
The class of streams over Unicode strings. It is a subclass of <string-stream>.
The class supports the same init-keywords as <sequence-stream>.
0.4.1 Creating streams
The following functions are used to create streams.
0.4.1.1 File streams
File streams are intended only for accessing the contents of files. They are not intended to provide a general file handling facility of renaming, deleting, moving, parsing directory names and so on.
make file-stream-class
G.f method
make file-stream-class #key locator direction if-exists if-does-not-exist
buffer-size element-type encoding
=> file-stream-instance
Creates and opens a stream over a file.
Returns a new instance of a concrete subclass of <file-stream> that streams over the contents of the file referenced by locator. To determine the concrete subclass to be instantiated, this method calls the generic function type-for-file-stream (see page 62.)
The file-stream-class argument is the class <file-stream>.
The locator init-keyword should be a string naming a file. If the Locators library is in use, locator should be an instance of <locator> or a string that can be coerced to one.
The direction init-keyword specifies the direction of the stream.
The if-exists and if-does-not-exist init-keywords specify actions to take if the file named by locator does or does not already exist when the stream is created. These init-keywords are discussed in more detail in Section 0.4.1.2 on page 62.
The buffer-size init-keyword is explained in <buffered-stream>, page 60.
The element-type init-keyword specifies the type of the elements in the file named by locator. This allows file elements to be represented abstractly; for instance, contiguous elements could be treated as a single database record. This init-keyword defaults to something useful, potentially based on the properties of the file; <byte-character> and <unicode-character> are likely choices. See Section 0.4.1.2 on page 62.
type-for-file-stream
Open generic function
type-for-file-stream locator element-type #rest all-keys => file-stream-type
Returns the kind of file-stream class to instantiate for a given file. The method for make on <file-stream> calls this function to determine the class of which it should create an instance.
0.4.1.2 Options when creating file streams
When creating file streams, you can can supply the following init-keywords to make in addition to the usual direction:
The if-exists init-keyword allows you to specify an action to take if the file named by locator already exists. The options are:
#"new-version"
If the underlying file system supports file versioning, a new version of the file is created. This is the default when the stream’s direction is #"output".
If the file system does not support file versioning, the implementation should substitute one of the other if-exists behaviors; the #"replace" behavior is a good choice.
#"truncate" If the file exists, it is truncated, setting the size of the file to 0. If the file does not exist, create a new file.
The if-does-not-exist init-keyword allows you to specify an action to take if the file named by locator does not exist. The options are:
Because creating a file stream always involves an attempt to open the underlying file, the aforementioned error conditions will occur during file stream instance initialization.
If an implementation checks file permissions when creating and opening file streams, and it detects an attempt to read a file for which the user has no read permission, or to write a file for which the user has no write permission, then an <invalid-file-permissions-error> condition is signalled at the time the file stream is created.
The element-type init-keyword controls how the elements of the underlying file are accessed.
Implementation Note: Ideally, element-type could be any valid Dylan type such as limited(<integer>, min: 0, max: 255) or <unicode-character>. This approach may make it possible to implement a potentially inefficient but general set of file streams. Unfortunately the current language definition does not include adequate support for this approach, so we specify instead an interim minimum set of three element types. The element types are for the time being exported from the streams module of the Streams library.
The three possible element types are:
<byte-character>
The file is accessed as a sequence of 8-bit characters.
<unicode-character>
The file is accessed as a sequence of 16-bit Unicode characters.
Portability Note: Portable code can count on the existence of these three element types, but implementations are free to provide more.
0.4.1.3 Sequence streams
There are make methods on <sequence-stream>, <string-stream>, <byte-string-stream> and <unicode-string-stream>. The make methods on <sequence-stream> and <string-stream> might not create direct instances of those classes, but instead an instance of a subclass determined by type-for-sequence-stream.
make sequence-stream-class
G.f. method
make sequence-stream-class #key contents direction start end
=> sequence-stream-instance
Creates and opens a stream over a sequence.
This method returns a general instance of <sequence-stream>. To determine the concrete subclass to be instantiated, this method calls the generic function type-for-sequence-stream (see page 65).
The sequence-stream-class argument is the class <sequence-stream>.
The contents init-keyword is a general instance of <sequence> which is used as the input for input streams, and as the initial storage for an output stream. If contents is a stretchy vector, then it is the only storage used by the stream.
The direction init-keyword specifies the direction of the stream. It must be one of #"input", #"output", or #"input-output"; the default is #"input".
The start and end init-keywords are only valid when direction is #"input". They specify the portion of the sequence to create the stream over: start is inclusive and end is exclusive. The default is to stream over the entire sequence: start is by default 0, and end is contents.size.
type-for-sequence-stream
Open generic function
type-for-sequence-stream sequence => sequence-stream-type
Returns the sequence-stream class to instantiate over a given sequence object. The method for make on <sequence-stream> calls this function to determine the concrete subclass of <sequence-stream> that it should instantiate.
There are type-for-sequence-stream methods for each of the string object classes. These methods return a stream class object that the particular Streams implementation considers appropriate.
make string-stream-class
G.f. method
make string-stream-class #key contents direction start end
=> string-stream-instance
Creates and opens a stream over a string.
This method returns an instance of <string-stream>. If supplied, contents must be an instance of <string>. The string-stream-class argument is the class <string-stream>. The direction, start, and end init-keywords are as for make on <sequence-stream>.
make byte-string-stream-class
G.f. method
make byte-string-stream-class #key contents direction start end
=> byte-string-stream-instance
Creates and opens a stream over a byte string.
This method returns a new instance of <byte-string-stream>. If supplied, contents must be an instance of <byte-string>.
The byte-string-stream-class argument is the class <byte-string-stream>. The direction, start, and end init-keywords are as for make on <sequence-stream>.
make unicode-string-stream-class
G.f. method
make unicode-string-stream-class #key contents direction start end
=> unicode-string-stream-instance
Creates and opens a stream over a Unicode string.
This method returns a new instance of <unicode-string-stream>. If supplied, contents must be an instance of <unicode-string>.
The unicode-string-stream-class argument is the class <unicode-string-stream>. The direction, start, and end init-keywords are as for make on <sequence-stream>.
0.4.2 Closing streams
When creating new stream classes it may be necessary to add a method to the close function, even though it is not part of the Stream Extension Protocol.
close
Open generic function
close stream #key #all-keys => ()
Closes stream, an instance of <stream>.
0.5 Reading and writing from streams
It is an error to call any of these functions on a buffered stream while its buffer is held.
0.5.1 Reading from streams
The following are the basic functions for reading from streams. To implement a new input stream that is not a <buffered-stream>, you must provide methods for read-element, stream-input-available?, peek, read, read-into!, and discard-input. If you implement a new stream that is a <positionable-stream>, you might need to supply a new method for unread-element.
read-element
Open generic function
read-element input-stream #key on-end-of-stream => element-or-eof
Returns the next element in the stream. If the stream is not at its end, the stream is advanced so that the next call to read-element returns the next element along in input-stream.
The on-end-of-stream keyword allows you to specify a value to be returned if the stream is at its end. If the stream is at its end and no value was supplied for on-end-of-stream, read-element signals an <end-of-stream-error> condition.
If no input is available and the stream is not at its end, read-element blocks until input becomes available.
See also unread-element, page 79.
peek
Open generic function
peek input-stream #key on-end-of-stream => element-or-eof
Behaves as read-element does, but the stream position is not advanced.
read
Open generic function
read input-stream n #key on-end-of-stream => sequence-or-eof
Returns a sequence of the next n elements from input-stream.
The type of the sequence returned depends on the type of the stream’s underlying aggregate. For instances of <sequence-stream>, the type of the result is given by type-for-copy of the underlying aggregate. For instances of <file-stream>, the result is a vector that can contain elements of the type returned by calling stream-element-type on the stream.
The stream position is advanced so that subsequent reads start after the n elements.
If the end of the stream is reached before all n elements have been read, the behavior is as follows.
If the on-end-of-stream argument is supplied, it is returned as the value of read.
If the on-end-of-stream argument was not supplied, and at least one element was read from the stream, then an <incomplete-read-error> condition is signalled. When signalling this condition, read supplies two values: a sequence of the elements that were read successfully, and n.
If the on-end-of-stream argument was not supplied, and no elements were read from the stream, an <end-of-stream-error> condition is signalled.
If the stream is not at its end, read blocks until input becomes available.
Implementation Note: Buffered streams are intended to provide a very efficient implementation of read, particularly when the result is an instance of <byte-string>, <unicode-string>, or <byte-vector>.
read-into!
Open generic function
read-into! input-stream n sequence #key start on-end-of-stream => count-or-eof
Reads the next n elements from input-stream, and inserts them into a mutable sequence starting at the position start. Returns the number of elements actually inserted into sequence unless the end of the stream is reached, in which case the on-end-of-stream behavior is as for read.
If the sum of start and n is greater than the size of sequence, read-into! reads only enough elements to fill sequence up to the end. If sequence is a stretchy vector, no attempt is made to grow it.
If the stream is not at its end, read-into! blocks until input becomes available.
Implementation Note: Buffered streams are intended to provide a very efficient implementation of read, particularly when the result is an instance of <byte-string>, <unicode-string>, or <byte-vector>.
discard-input
Open generic function
discard-input input-stream => ()
Discards any pending input from input-stream, both buffered input and, if possible, any input that might be at the stream’s source.
This operation is principally useful for "interactive" streams, such as TTY streams, to discard unwanted input after an error condition arises. There is a default method on <stream> so that applications can call this function on any kind of stream. The default method does nothing.
stream-input-available?
Open generic function
stream-input-available? input-stream => available?
Returns #t if input-stream would not block on read-element, otherwise it returns #f.
This function differs from stream-at-end?. When stream-input-
available? returns #t, read-element will not block, but it may detect that it is at the end of the stream’s source, and consequently inspect the on-end-of-stream argument to determine how to handle the end of stream.
0.5.2 Convenience functions for reading from streams
The following is a small set of convenient reading functions that search for particular elements in a stream. These functions behave as though they were implemented in terms of the more primitive functions described in Section 0.5.1.
read-to
Function
read-to input-stream element #key on-end-of-stream test => sequence-or-eof found?
Returns a new sequence containing the elements of input-stream from the stream’s current position to the first occurrence of element. The result does not contain element.
The second return value is #t if the read terminated with element, or #f if the read terminated by reaching the end of the stream’s source. The "boundary" element is consumed, that is, the stream is left positioned after element.
The read-to function determines whether the element occurred by calling the test function test, which defaults to \==. The test function must accept two arguments. The order of the arguments is the element retrieved from the stream first and element second.
The type of the sequence returned is the same that returned by read. The end-of-stream behavior is the same as that of read-line.
read-through
Function
read-through input-stream element #key on-end-of-stream test
=> sequence-or-eof found?
This function is the same as read-to, except that element is included in the resulting sequence.
If the element is not found, the result does not contain it. The stream is left positioned after element.
read-to-end
Function
read-to-end input-stream => sequence
Returns a sequence of all the elements up to, and including, the last element of input-stream, starting from the stream’s current position.
The type of the result sequence is as described for read. There is no special end-of-stream behavior; if the stream is already at its end, an empty collection is returned.
skip-through
Function
skip-through input-stream element #key test => found?
Positions input-stream after the first occurrence of element, starting from the stream’s current position. Returns #t if the element was found, or #f if the end of the stream was encountered. When skip-through does not find the element, it leaves input-stream positioned at the end.
The skip-through function determines whether the element occurred by calling the test function test, which defaults to \==. The test function must accept two arguments. The order of the arguments is the element retrieved from the stream first and element second.
0.5.3 Writing to streams
The following are the basic functions for writing to streams.
To implement a new output stream that is not a <buffered-stream>, you must provide methods for write-element, write, force-output, and discard-output.
write-element
Open generic function
write-element output-stream element => ()
Writes element to output-stream at the stream’s current position. It is an error if the type of element is inappropriate for the strream’s underlying aggregate.
If the stream is positionable, and it is not positioned at its end,
write-element overwrites the element at the current position and then advance the stream position.
write
Open generic function
write output-stream sequence #key start end => ()
Writes the elements of sequence to output-stream, starting at the stream’s current position.
The elements in sequence are accessed in the order defined by the forward iteration protocol on <sequence>. This is effectively the same as the following:
do (method (elt) write-element(stream, elt) end, sequence);
sequence;
If supplied, start and end delimit the portion of sequence to write to the stream. The value of start is inclusive and that of end is exclusive. They default to 0 and sequence.size, respectively.
If the stream is positionable, and it is not positioned at its end, write overwrites elements in the stream and then advance the stream’s position to be beyond the last element written.
Implementation Note: Buffered streams are intended to provide a very efficient implementation of write, particularly when sequence is an instance of <byte-string>, <unicode-string>, <byte-vector>, or <buffer>, and the stream’s element type is the same as the element type of sequence.
force-output
Open generic function
force-output output-stream => ()
Forces any pending output from output-stream’s buffers to its destination.
When creating new stream classes it may be necessary to add a method to the force-output function, even though it is not part of the Stream Extension Protocol.
synchronize-output
Open generic function
synchronize-output output-stream => ()
Forces any pending output from output-stream’s buffers to its destination. Before returning to its caller, synchronize-output also attempts to ensure that the output reaches the stream’s destination before, thereby synchronizing the output destination with the application state.
When creating new stream classes it may be necessary to add a method to the synchronize-output function, even though it is not part of the Stream Extension Protocol.
discard-output
Open generic function
discard-output output-stream => ()
Attempts to abort any pending output for output-stream.
A default method on <stream> is defined, so that applications can call this function on any sort of stream. The default method does nothing.
0.5.4 Reading and writing by lines
The following functions facilitate line-based input and output operations.
The newline sequence for string streams is a sequence comprising the single newline character \n. For character file streams, the newline sequence is whatever sequence of characters the underlying platform uses to represent a newline. For example, on MSDOS platforms, the sequence might comprise two characters: a carriage return followed by a linefeed.
Implementation Note: The functions described in this section are potentially an interim solution to one aspect of the more general problem of encoding and data translation. At some point, these functions may be moved into, or subsumed by, another higher level library that deals with the encoding problems in a better way. Note that no other functions in the Streams library do anything to manage the encoding of newlines; calling write-element on the character \n does not cause the \n character to be written as the native newline sequence, except by coincidence.
read-line
Open generic function
read-line input-stream #key on-end-of-stream => string-or-eof newline?
Returns a new string containing all the input in input-stream up to the next newline sequence.
The resulting string does not contain the newline sequence. The second value returned is #t if the read terminated with a newline or #f if the read terminated because it came to the end of the stream.
The type of the result string is chosen so that the string can contain characters of input-stream’s element type. For example, if the element type is <byte-character>, the string will be a <byte-string>.
If input-stream is at its end immediately upon calling read-line (that is, the end of stream appears to be at the end of an empty line), then the end-of-stream behavior and the interpretation of on-end-of-stream is as for read-element.
read-line-into!
Open generic function
read-line-into! input-stream string #key start on-end-of-stream grow? => string-or-eof newline?
Fills string with all the input from input-stream up to the next newline sequence. The string must be a general instance of <string> that can hold elements of the stream’s element type.
The input is written into string starting at the position start, which is an integer defaulting to 0.
The second return value is #t if the read terminated with a newline, or #f if the read completed by getting to the end of the input stream.
If grow? is #t, and string is not large enough to hold all of the input, this function takes one of the two following actions.
If string is stretchy, read-line-into! grows it enough to hold the input.
If string is not stretchy, read-line-into! creates a new string which it writes to and returns instread. The resulting string holds all the original elements of string, except where read-line-into! overwrites them with input from input-stream.
In a manner consistent with the intended semantics of grow?, when grow? is true and start is greater than or equal to string.size, read-line-into! grows string to accomodate the start index and the new input.
If grow? is #f (the default) and string is not large enough to hold the input, the function signals an error.
The end-of-stream behavior and the interpretation of on-end-of-stream is the same as that of read-line.
write-line
Open generic function
write-line output-stream string #key start end => ()
Writes string followed by a newline sequence to output-stream.
The default method behaves as though it calls write on string and then calls new-line, with output-stream locked across both calls.
If supplied, start and end delimit the portion of string to write to the stream. They default to 0 and string.size respectively.
new-line
Open generic function
new-line output-stream => ()
Writes a newline sequence to output-stream.
A method for new-line is defined on <string-stream> that writes the character \n to the string stream.
0.5.5 Querying streams
The following functions can be used to determine various properties of a stream.
To implement a new stream you must provide methods for stream-open?, stream-at-end?, and stream-element-type.
stream-open?
Open generic function
stream-open? stream => open?
Returns #t if stream is open and #f if it is not.
stream-element-type
Open generic function
stream-element-type stream => element-type
Returns the element type of stream as a Dylan <type>.
stream-at-end?
Open generic function
stream-at-end? stream => boolean
Returns #t if the stream is at its end and #f if it is not. For input streams, it returns #t if a call to read-element with no supplied keyword arguments would signal an <end-of-stream-error>.
This function differs from stream-input-available?. See the description of that function on page 69.
For output-only streams, this function always returns #f.
0.5.6 Positionable stream protocol
The following comprises the protocol for positionable streams.
To implement a new positionable stream you must provide methods for stream-position, stream-position-setter, adjust-stream-position, stream-size, stream-contents, and unread-element.
A stream position can be thought of as a natural number that indicates how many elements into the stream the stream’s current location is. However, it is not always the case that a single integer contains enough information to reposition a stream. Consider the case of an "uncompressing" file stream that requires additional state beyond simply the file position to be able to get the next input character from the compressed file.
The Streams library addresses this problem by introducing the class <stream-position>, which is subclassed by various kinds of stream implementations that need to maintain additional state. A stream can be repositioned as efficiently as possible when stream-position-setter is given a value previously returned by stream-position on that stream.
It is also legal to set the position of a stream to an integer position. However, for some types of streams, to do so might be slow, perhaps requiring the entire contents of the stream up to that point to be read.
<stream-position>
Abstract class
A direct subclass of <object>. It is used to represent positions within streams of a certain kind: those for which a natural number is not sufficient to fully describe the position in the stream. For example, a stream that supports compression will have some state associated with each position in the stream that a single integer is not sufficient to represent.
The <stream-position> class is disjoint from the class <integer>.
stream-position
Open generic function
stream-position positionable-stream => position
Returns the current position of positionable-stream for reading or writing.
The value returned can be either an instance of <stream-position> or an integer. When the value is an integer, it is an offset from position zero, and is in terms of the stream’s element type. For instance, in a Unicode stream, a position of 4 means that 4 Unicode characters have been read.
stream-position-setter
Open generic function
stream-position-setter position positionable-stream => new-position
Changes the stream’s position for reading or writing to position.
The position can be either an integer or a <stream-position>.
When it is an integer, if it is less than zero or greater than positionable-stream.stream-size this function signals an error.
When position is a <stream-position>, if it is invalid for some reason, this function signals an error. Streams are permitted to restrict the position to being a member of the set of values previously returned by calls to stream-position on the same stream.
The position may also be #"start", meaning that the stream should be positioned at its start, or #"end", meaning that the stream should be positioned at its end.
adjust-stream-position
Open generic function
adjust-stream-position positionable-stream delta #key from => new-position
Moves the position of positionable-stream to be offset delta elements from the position indicated by from. The new position is returned.
The value of from can be one of the symbols #"current", #"start", and #"end". The default value is #"current". When from is #"start", the stream is positioned relative to the beginning of the stream. When from is #"end", the stream is positioned relative to its end.
Using adjust-stream-position to set the position of a stream to be beyond its current last element causes the underlying aggregate to be grown to a new size. When extending the underlying aggregate for a stream, it is undefined what the unwritten elements of the aggregate will contain.
as
G.f. method
as integer-class stream-position => integer
Coerces a <stream-position> to an integer. The integer-class argument is the class <integer>.
stream-size
Open generic function
stream-size positionable-stream => size
Returns the number of elements in positionable-stream.
For input streams, this number is the number of elements that were available when the stream was created. It is unaffected by any read operations that might have been performed on the stream.
For output and input-output streams, this number is the number of elements that were available when the stream was created (just as with input streams), added to the number of elements written past the end of the stream (regardless of any repositioning operations).
It is assumed that there is no more than one stream open on the same source or destination at a time; that there are no shared references to files by other processes; that there are no alias references to underlying sequences, or any other such situations. In such situations, the behavior of stream-size is undefined.
stream-contents
Open generic function
stream-contents positionable-stream #key clear-contents? => sequence
Returns a sequence that contains all of positionable-stream’s elements from its start to its end, regardless of its current position. The type of the returned sequence is as for read. See page 68.
Note: For streams whose direction is #"input", stream-contents signals an error. Use read-to-end instead on such streams. See page 70.
If clear-contents? is #t (the default), this function sets the size of the stream to zero, and the position to the stream’s start. Thus the next call to stream-contents will return only the elements written after the previous call to stream-contents.
unread-element
Open generic function
unread-element positionable-stream element => element
"Unreads" the last element from positionable-stream. That is, it returns element to the stream so that the next call to read-element will return element. The stream must be a <positionable-stream>.
It is an error (i) to apply unread-element to an element that is not the element most recently read from the stream; (ii) to call unread-element twice in succession; (iii) to unread an element if the stream is at its initial positionl; and (iv) to unread an element after explicitly setting the stream’s position.
0.5.7 Using file streams
The following are operations that pertain to file streams.
close
G.f. method
close file-stream #key abort wait?
Closes a file stream. This method frees whatever it can of any underlying system resources held on behalf of the stream.
If abort is false (the default), any pending data is forced out and synchronized with the file’s destination. If abort is true, then any errors caused by closing the file are ignored. Furthermore, if abort is true, the file should be restored to its initial state if possible (for example, on a versioned file system, the previous version of the file should be restored as the latest version).
with-open-file
Macro
with-open-file (stream-var = locator, #rest keys) body end
Provides a safe mechanism for working with file streams. The macro creates a file stream and binds it to stream-var, evaluates a body of code within the context of this binding, and then closes the stream. The macro calls close upon exiting body.
The locator argument should evaluate to a valid argument to
as(<locator>, locator)
The values of the last expression in body are returned.
The keys are passed to the make method on <file-stream>.
For example, the following expression yields the contents of file foo.text as a <byte-vector>:
with-open-file (fs = ("foo.text", element-type: <byte>))
read-to-end(fs)
end;
It is roughly equivalent to:
begin
let hidden-fs = #f; // In case the user bashes fs variable
block ()
hidden-fs := make(<file-stream>,
locator: "foo.text", element-type: <byte>);
let fs = hidden-fs;
read-to-end(fs);
cleanup
if (hidden-fs) close(hidden-fs) end;
end block;
end;
0.6 Locking streams
Stream locks have multilocking semantics. That is, a single thread can lock a stream more than once, but the thread must unlock the stream for each time it locked the stream. Furthermore, threads waiting for a stream lock are expected to do so by blocking, not by "spinning".
This allows a high-level printing routine to lock a stream across several calls to output functions, ensuring all the output is contiguous at the stream’s destination. For example, the write-line function locks its stream argument and then calls the write and new-line functions. The write function locks its stream argument by calling get-output-buffer, but because of the multilocking semantics, the call to write within write-line does not block waiting for a lock. The same thing happens with new-line. Before returning, write-line unlocks the stream so that other routines may call output functions on the stream or get the stream’s buffer for direct manipulation.
The Locking Protocol isolates access to a stream so that only one thread may use the stream at any time. In a single-threaded Dylan implementation, these functions do nothing.
The Buffer Access Protocol functions that get a buffer first lock the stream, and those functions that release the buffer unlock the stream. Thus, getting a buffer both isolates access to the stream for a single thread and ensures that the single thread does not try to get the stream’s buffer multiple times while already holding the buffer.
The Buffer Access Protocol isolates access to a buffer within a single thread to prevent reentrancy problems and programming mistakes. Essentially, the lightweight buffer locking ensures that applications do not call output functions that directly manipulate a stream’s buffer from within routines that are already directly manipulating the stream’s buffer. This situation must be forbidden because the inner call to get the buffer cannot reliably return the state of the stream’s buffer while the application already holds the buffer.
stream-locked?
Function
stream-locked? stream => boolean
Returns #t if stream is locked and #f if it is not.
lock-stream
Function
lock-stream stream => ()
If stream is already locked by another thread, this function waits until it gets the lock.
In a single-threaded Dylan implementation, this function does nothing.
unlock-stream
Function
unlock-stream stream => ()
In a multi-threaded Dylan implementation, stream locks can be taken more than once by a thread, so they must be unlocked for each time they are locked. This function signals an error if it is invoked in order to release a stream’s lock and the current thread does not actually hold that lock.
In a single-threaded Dylan implementation, this function does nothing.
with-stream-locked
Macro
with-stream-locked (stream) body end
Evalutes body in a context in which stream is locked.
Upon exiting the body, the code resulting from the macro unlocks the stream once, as a block cleanup.
0.7 Using buffered streams
A goal of the streams library is to provide efficient support for general use of buffered I/O. At the same time, programmers using the library should not need to be concerned with buffering in most cases. For most uses of buffered streams, the buffering is transparent, but programs requiring more control can access buffering functionality when appropriate. This section describes the available buffering functionality.
0.7.1 Overview
A buffered stream maintains some sort of buffer. All buffered streams use the sealed class <buffer> for their buffers. You can suggest a buffer size when creating buffered streams, but normally you do not need to do so. Streams implementations should choose buffer sizes that are appropriate for the stream’s source or destination.
Instances of the class <buffer> also contain some state information. This state information includes an index where reading or writing should begin, and an index that is the end of input to be read, or the end of space available for writing.
Buffered streams also maintain a held state, indicating whether the application has taken the buffer for a stream and has not released it yet. When a thread already holds the buffer for a stream, it is an error to get the buffer again (or any other buffer for the same stream); this property holds for the one thread of a single-threaded Dylan implementation as well as for multi-threaded implementations.
0.7.2 Buffer access protocol
This section describes the functions that users of buffered streams invoke to manipulate stream buffers directly. These functions use corresponding functions from the Stream Extension Protocol to do their work. For example, get-input-buffer calls do-get-input-buffer. All Stream Extension Protocol functions are named with a do- prefix on the corresponding Buffer Access Protocol function. Users should never call the Stream Extension Protocol directly.
Threaded Dylan implementations should place system-dependent mutual exclusion calls in the Buffer Access Protocol functions. All streams implementations are encouraged to check the buffer-held state in the Buffer Access Protocol functions, and these functions should signal an error if the buffer is already held. The separation of the Buffer Access Protocol and the Stream Extension Protocol allows users to more portably extend the buffered stream protocol to new stream types. Programmers avoid the following design and maintenance hassles:
- Deciding whether to write for a threaded or non-threaded Dylan implementation.
- Rewriting mutual exclusion code for each Dylan implementation.
- Having to duplicate mutual exclusion code wherever the application uses stream buffers directly.
Programs that manipulate buffers are entirely responsible for tracking the buffer’s state and informing the stream of any changes to the buffer’s state. The technique for using a buffer is to get the input or output buffer and its bounds, consume or produce data, update the buffer’s state, and then release the buffer. The new bounds are indicated by calling buffer-next-setter and buffer-end-setter. Using buffers directly interoperates with using higher-level functions such as read-element, read, write-element, write, and so on. This is because higher-level stream operations are implemented in terms of the Buffer Access Protocol for instances of <buffered-stream>.
0.7.2.1 Useful types when using buffers
<byte>
Type
A type representing limited integers in the range 0 to 255 inclusive.
<byte-character>
Type
A type representing 8-bit characters that instances of <byte-string> can contain.
<unicode-character>
Type
A type representing Unicode characters that instances of <unicode-string> can contain.
<byte-vector>
Type
A subtype of <vector> whose element-type is <byte>.
<buffer>
Sealed instantiable class
A subclass of <vector> whose element-type is <byte>.
Instances of <buffer> contain a data vector and two indices: the inclusive start and the exclusive end of valid data in the buffer. The accessors for these indexes are called buffer-next and buffer-end. This class supports three init-keywords: size:, next: (defaults to zero), and end: (defaults to zero).
Note that size: is not taken as a suggestion of the size the user would like, as with the value passed with buffer-size: to make on <buffered-stream>; if you supply a value with the size: init-keyword, that size is allocated, or, if that is not possible, an error is signalled, as with making any vector.
<buffer-index>
Type
All buffer-index return values and parameters are of this type. See also Harlequin and CMU’s proposal for <integer>.
0.7.2.2 Using buffers for input
get-input-buffer
Function
get-input-buffer buffered-stream, #key wait?, bytes => buffer-or-false
Calls do-get-input-buffer to return a buffer for the stream. You should never call do-get-input-buffer directly.
The stream extension function signals an error if buffered-stream is an output-only stream, or if it cannot return a buffer for any reason.
The wait? keyword is a <boolean> and defaults to #t. It indicates whether get-input-buffer should ensure that there is valid data in the input buffer, and block for input if necessary.
When wait? is #f, this function returns the stream’s current buffer in whatever state it is in.
When wait? is #t, do-get-input-buffer waits until there is some valid data in the buffer. If do-get-input-buffer needs to wait for valid data, and it encounters the end of the stream before any valid data arrives, then do-get-input-buffer returns #f.
When bytes is supplied, wait? has no effect. The bytes keyword is an
<integer> or #f, and is the minimum number of bytes that must be present in the buffer when it is returned. If do-get-input-buffer cannot return a buffer with the specified minimum number of bytes, it signals an <incomplete-read-error>.
Values supplied for bytes should be small relative to a buffer’s size. If you need to read large numbers of bytes at once, specify a large buffer size when making the stream and do not specify any values for the bytes keyword parameter. When bytes is not #f, get-input-buffer may block for input if necessary.
If an application thread calls get-input-buffer, and another thread already holds any buffer the stream owns, then this function might block. Multi-threaded implementations should eventually return. In multi-threaded implementations this function calls lock-stream before calling do-get-input-buffer, and it does not release the lock.
If an application’s thread calls this function, and that same thread already holds the buffer, this situation is a re-entrancy programmer error. The stream cannot return the buffer and guarantee that its state is up to date with input consumption that occurred between the first call to this function and the current call. Implementations should detect and signal this error.
release-input-buffer
Function
release-input-buffer buffered-stream => ()
Call this function to announce that you have finished using the stream’s current buffer. There may still be valid input in the buffer even though you have finished with it.
This function calls do-release-input-buffer so that the stream can perform any record-keeping or management tasks it has. You should never call do-release-input-buffer directly.
After consuming input from the buffer, you must update the buffer’s state by assigning to buffer-next before releasing the buffer. When a user of the Buffer Access Protocol sets buffer-end, the results are undefined.
If the calling application does not hold the buffer, this function signals an error.
In multi-threaded implementations, release-input-buffer releases the lock obtained in get-input-buffer.
with-input-buffer
Macro
with-input-buffer (buffer-var = exp, #key wait?, bytes) body end;
Calls get-input-buffer on exp, wait?, and bytes, binding buffer-var to the result, then executes body within the scope of the buffer binding.
After the body has executed, with-input-buffer calls release-input-buffer and returns any values returned by body.
next-input-buffer
Function
next-input-buffer buffered-stream #key wait?, bytes => buffer-or-false
Call this function to get more input when you are already holding buffered-stream’s buffer. If the application does not hold the stream’s buffer, this function signals an error.
This function calls do-next-input-buffer to get more input. You should never call do-next-input-buffer directly.
You must update the buffer’s state by assigning to buffer-next before getting the next buffer; otherwise, it might seem that the current buffer still has valid input in it. When buffer-next = buffer-end there is no valid input remaining in the current buffer.
If a user of the Buffer Access Protocol sets buffer-end while holding the stream’s buffer, the results are undefined.
The stream extension function may return a new <buffer> object or the current one with buffer-next and buffer-end updated to reflect the location of the new input data. You should not make any assumptions about which option a stream’s implementation uses.
The do-next-input-buffer function signals an error when the current buffer is not held for input.
The wait? and bytes arguments are the same as for get-input-buffer.
input-available-at-source?
Function
input-available-at-source? buffered-stream => available?
Returns #t when buffered-stream’s source has any available input or when the stream is at the end of its source. If this function returns #t, the next call to next-input-buffer will not block.
This function calls do-input-available-at-source? to get more input. You should never call do-input-available-at-source? directly.
Call this function while you are holding a stream’s buffer. If the application does not hold the stream’s buffer, this function signals an error.
The stream extension function signals an error when the current buffer is not held for input.
0.7.2.3 Using buffers for output
get-output-buffer
Function
get-output-buffer buffered-stream, #key bytes => buffer
Calls do-get-output-buffer to return a buffer for buffered-stream. You should never call do-get-output-buffer directly.
The stream extension function signals an error if the stream is an input only stream, or if it cannot return a buffer for any reason. The resulting buffer is never completely full of pending output.
The bytes keyword is an <integer>, and is the minimum number of bytes that must be available for writing in the buffer when it is returned. It defaults to 1. If do-get-output-buffer cannot return a buffer with the specified minimum number of bytes, it signals an error.
Values supplied for bytes should be small relative to a buffer’s size. If you need to write large numbers of bytes at once, specify a large buffer size when making the stream and do not specify any values for the bytes keyword parameter.
If an application’s thread calls get-output-buffer and another thread already holds any buffer for the stream, the function might block. Multi-threaded implementations should eventually return. In multi-threaded implementations this function calls lock-stream before calling do-get-input-buffer, and it does not release the lock.
If an application’s thread calls this function, and that same thread already holds the buffer, this situation is a re-entrancy programmer error. The stream cannot return the buffer and guarantee that its state is up to date with output written between the first call to this function and the current call. Implementations should detect and signal this error.
release-output-buffer
Function
release-output-buffer buffered-stream => ()
Call this function to announce that you have finished using the stream’s current buffer. When you have written output to the buffer, you must update the buffer’s state by assigning to buffer-next before releasing the buffer. When a user of the Buffer Access Protocol sets buffer-end, the results are undefined.
If the application does not hold the buffer, this function signals an error.
This function calls do-release-output-buffer so that the stream can perform any record-keeping or management tasks it has. You should never call do-release-output-buffer directly.
In multi-threaded implementations, release-output-buffer releases the lock obtained in get-output-buffer.
with-output-buffer
Macro
with-output-buffer (buffer-var = exp, #key bytes) body end;
Calls get-output-buffer on exp and bytes, binding buffer-var to the result, then executes body within the scope of the buffer binding.
After the body has executed, with-output-buffer calls release-output-buffer and returns any values returned by body.
next-output-buffer
Macro
next-output-buffer buffered-stream #key bytes => buffer
Call this function to get the next buffer for writing output when you are already holding a stream’s buffer. If the application does not hold the stream’s buffer, this function signals an error.
This function calls do-next-output-buffer to get the next output buffer. You should never call do-next-output-buffer directly.
You must update the buffer’s state by assigning to buffer-next before getting the next buffer; otherwise, it might seem that the current buffer is empty (or does not contain the output you just stored in it). When buffer-next = buffer-end there is no pending output in the current buffer.
When a user of the Buffer Access Protocol sets buffer-end, the results are undefined.
The stream extension function may return a new <buffer> object or the current one with buffer-next updated to indicate the next available element to be written for output. You should not make any assumptions about which option a stream’s implementation uses.
The do-next-output-buffer function signals an error when the current buffer is not held for output.
The do-next-output-buffer function may force output to a stream’s destination, but it is not required to do so. It might empty the current buffer into secondary buffers or get a new buffer from a queue, while not forcing any output.
The bytes argument is the same as for get-output-buffer, but do-next-output-buffer signals any error associated with this parameter.
0.7.3 Copying to and from buffers
All generic sequence operations work on buffers. The Streams library provides some additional functions to compensate for lacking functionality in Dylan’s sequence operations. There are also a few functions for updating a buffer’s state.
buffer-next
Function
buffer-next buffer => buffer-index
For buffers held for input, this function returns the location of the next valid byte of input to be read. When the location equals buffer-end, there is no more valid input data in the buffer.
For buffers held for output, this function returns the location of the next free byte that should be written in the buffer, and when the location equals buffer-end, the buffer is full of pending output.
buffer-next-setter
Function
buffer-next-setter new-value buffer => new-value
Sets buffer’s next index.
buffer-end
Function
buffer-end buffer => buffer-index
For buffers held for input, this function returns the index of the exclusive end of data to be read. For buffers held for output, this function returns the size of the buffer.
See also buffer-end-setter, below.
buffer-end-setter
Function
buffer-end-setter new-value buffer => new-value
Note: Users of the Buffer Access Protocol functions should not set this accessor because the results are undefined; however, stream implementors may need to set this accessor inside Stream Extension Protocol methods, which execute while users are holding a stream buffer.
buffer-subsequence
Open generic function
buffer-subsequence buffer result-class start end => sequence
Returns an instance of the class result-class, filled with the elements from the buffer from the index start to the index end.
The start and end arguments are both instances of <buffer-index>.
Implementation Note: This function should be implemented as efficiently as possible. There should be sealed methods for when result-class is an instance of <byte-string>, <byte-vector>, and <buffer>.
copy-into-buffer!
Open generic function
copy-into-buffer! buffer buffer-start sequence #key start end => ()
Copies the portion of sequence given by the indices start and end into buffer.
The buffer is filled starting at buffer-start.
The start index defaults to 0, while end defaults to sequence.size.
If buffer is too small to hold the data, an error is signalled.
Implementation Note: This function should be implemented as efficiently as possible. There should be sealed methods for when sequence is an instance of <byte-string>, <byte-vector>, or <buffer>.
copy-from-buffer!
Open generic function
copy-from-buffer! buffer buffer-start sequence #key start end => ()
Copies a portion of buffer starting at buffer-start into sequence. The number of elements copied is determined by start and end, which are indices into sequence. The sequence is filled starting at start, which defaults to 0, and ending at end, which defaults to sequence.size. If sequence is too small to hold the data, an error is signalled.
Implementation Note: This function should be implemented as efficiently as possible. There should be sealed methods for when sequence is an instance of <byte-string>, <byte-vector>, or <buffer>.
0.7.4 Stream extension protocol
These are the generic functions to which implementors of streams add methods when extending the stream protocol to new subclasses of <buffered-stream>. Although the close function is not described here, you might need to add a method to it when implementing a new stream. See close, page 67.
0.7.4.1 Creating new input or input-output streams
do-get-input-buffer
Open generic function
do-get-input-buffer buffered-stream #key wait? bytes => buffer-or-f
The methods of this generic function implement get-input-buffer for new streams. These methods can assume that the stream is locked, so no other thread can access it. Methods must signal an error if the stream is an output-only stream, or if they cannot return a buffer for any reason. If this function’s methods cannot return a buffer with the specified minimum number of bytes, they signal an <incomplete-read-error>.
do-release-input-buffer
Open generic function
do-release-input-buffer buffered-stream => ()
The methods of this generic function implement release-input-buffer for new streams. These methods can assume that the stream is locked, so no other thread can access it.
do-next-input-buffer
Open generic function
do-next-input-buffer buffered-stream #key wait?, bytes => buffer-or-false
The methods of this generic function implement next-input-buffer for new streams. These methods can assume that the stream is locked, so no other thread can access it. Methods must signal an error if the current buffer is not held for input, or they for any reason cannot return a buffer, or a buffer with the minimum number of bytes specified by bytes (an <incomplete-read-error>).
do-input-available-at-source?
Open generic function
do-input-available-at-source? buffered-stream => available?
The methods of this generic function implement input-available-at-source? for new streams. These methods can assume that the stream is locked, so no other thread can access it. Methods must signal an error when the current buffer is not held for input.
0.7.4.2 Creating new output or input-output streams
do-get-output-buffer
Open generic function
do-get-output-buffer buffered-stream #key bytes => buffer
The methods of this generic function implement get-output-buffer for new streams. These methods can assume that the stream is locked, so no other thread can access it. Methods must signal an error if the stream is an input only stream, or if they cannot return a buffer for any reason.
The resulting buffer is never completely full of pending output. If this function’s methods cannot return a buffer with the minimum number of bytes specified by bytes, they must signal an error.
do-release-output-buffer
Open generic function
do-release-output-buffer buffered-stream => ()
The methods of this generic function implement release-output-buffer for new streams. These methods can assume that the stream is locked, so no other thread can access it.
do-next-output-buffer
Open generic function
do-next-output-buffer buffered-stream #key bytes => buffer
The methods of this generic function implement next-output-buffer for new streams. These methods must signal an error when they cannot return a buffer for any reason, or the current buffer is not held for output. If this function’s methods cannot return a buffer with the minimum number of bytes specified by bytes, they must signal an error.
Methods may force output to buffered-stream’s destination, but they are not required to do so. They might empty the current buffer into secondary buffers or get a new buffer from a queue, while not forcing any output.
0.8 Conditions
The class definitions for the error conditions are as follows. There is no recovery protocol defined for any of these errors. Every condition described in this section that takes an init-keyword has a slot accessor for the value supplied, and the name of the accessor function is the name of the condition class (without the angle brackets) to which a hyphen and the name of the init-keyword is appended.
<end-of-stream-error>
Error
Signalled when one of the read functions reaches the end of an input stream. It is a subclass of <error>. It takes one init-keyword, stream:.
<incomplete-read-error>
Error
Signalled when input functions are reading a required number of elements but they read the end of the stream before completing the required read. It is a subclass of <end-of-stream-error>. It takes two additional init-keywords, sequence: and count:. The sequence is whatever input was read before reaching the end of the stream. The count is the number of elements that were requested to be read.
<file-error>
Error
The base class for all errors related to file I/O. It is a subclass of <error>. It takes one init keyword, locator:.
<file-exists-error>
Error
Signalled when an output file stream creation function tries to create a file that already exists. It is a subclass of <file-error>.
<file-does-not-exist-error>
Error
Signalled when an input file stream creation function tries to read a file that does not exist. It is a subclass of <file-error>.
<invalid-file-permissions-error>
Error
Signalled when one of the file stream creation functions tries to access a file in a manner for which the user does not have permission. It is a subclass of <file-error>.
0.9 Wrapper streams
Sometimes stream data requires conversion before an application can use it: you might have a stream over a file of EBCDIC characters which you would prefer to handle as their ASCII equivalents, or you might need to encrypt or decrypt file data.
Wrapper streams provide a mechanism for working with streams which require such conversion. Wrapper streams hold on to an underlying stream, delegating to it most of the operations that implement streaming. The wrapper stream carries out appropriate processing in its own implementations of the streaming protocol.
The Dylan Streams Library includes a base class called <wrapper-stream> upon which other wrapping streams can be implemented.
define class <wrapper-stream> (<stream>)
slot inner-stream :: <stream>,
required-init-keyword: inner-stream:;
end class;
A subclass of <wrapper-stream> can "pass on" functions such as read-element and write-element by simply delegating these operations to the inner stream:
define method read-element (ws :: <io-wrapper-stream>);
read-element(ws.inner-stream)
end method;
define method write-element (ws :: <io-wrapper-stream>, element);
write-element(ws.inner-stream,element)
end method;
Assuming that <io-wrapper-stream> delegates all other operations to its inner stream, the following would suffice to implement a 16-bit Unicode character stream wrapping an 8-bit character stream.
define class <unicode-stream> (<io-wrapper-stream>) end class;
define method read-element (s :: <unicode-stream>)
=> ch :: <unicode-character>;
with-stream-locked (s)
let first-char = read-element(s.inner-stream);
let second-char = read-element(s.inner-stream)
end;
convert-byte-pair-to-unicode(first-char, second-char)
end method;
define method write-element (s :: <unicode-stream>,
c :: <character>);
let (first-char, second-char) = convert-unicode-to-byte-pair(c);
with-stream-locked (s)
write-element(s.inner-stream, first-char);
write-element(s.inner-stream, second-char)
end;
c
end method;
define method stream-position (s :: <unicode-stream>)
=> p :: <integer>;
truncate/(stream-position(s.inner-stream), 2)
end method;
define method stream-position-setter (p :: <integer>,
s :: <unicode-stream>);
stream-position(s.inner-stream) := p * 2
end method;
0.9.1 Wrapper streams and delegation
One problem with wrapper streams is the need for a wrapper stream to intercept methods invoked by its inner stream. For example, consider two hypothetical streams, <interactive-stream> and <dialog-stream>, the latter a subclass of <wrapper-stream>. Both of these classes have a method called prompt. The <interactive-stream> class specializes read thus:
define method read ( s :: <interactive-stream>,
n :: <integer>,
#key on-end-of-stream);
prompt(s);
next-method()
end method;
If a <dialog-stream> is used to wrap an <interactive-stream> then an invocation of read on the <dialog-stream> will call prompt on the inner
<interactive-stream>, not on the <dialog-stream>, as desired. The problem is that the <dialog-stream> wants to delegate some tasks to its inner stream, but handle some other tasks itself.
Some languages, notably Self, support such delegation in the language. Dylan, in keeping with other generic-function-based languages, does not, but experience with streams packages in a number of other generic-function-based languages has shown that it is very useful to provide delegation for wrapper streams. So we provide an explicit implementation of delegation in the streams system.
Delegation is implemented by the use of the outer-stream slot, which is defined in the base class <stream>:
define abstract class <stream> (<object>)
slot outer-stream :: <stream>, init-keyword: outer-stream:;
end class;
outer-stream is used instead of the stream itself whenever a stream invokes one of its other protocol methods.
Note: A stream must not use inner-stream if performing recursion, since this would cause an infinite regress.
A correct implementation of the read method in the example above would be as follows:
define method read ( stream :: <interactive-stream>,
n :: <integer>
#key on-end-of-stream)
prompt(s.outer-stream);
next-method()
end method;
The initialize method on <stream> is defined to set the outer-stream slot to be the stream itself. The initialize method on <wrapper-stream> is specialized to set the outer-stream slot to be the "parent" stream:
define method initialize (stream :: <wrapper-stream>,
#key on, #rest all-keys);
an-inner-stream.outer-stream := stream;
next-method()
end method;
Implementation Note: One disadvantage of this scheme is that it may prevent useful optimizations, such as inlining. If this is an important performance issue in circumstances where wrapper streams are not necessary, then it is relatively simple to provide an implementation of the streams module that omits the delegation mechanism.
0.10 Wrapper stream protocol
<wrapper-stream>
Open instantiable class
The class that implements the basic wrapper-stream functionality.
It takes a required init-keyword inner-stream:, which is used to specify the wrapped stream.
The <wrapper-stream> class implements default methods for all of the stream protocol functions described in this document. Each default method on <wrapper-stream> simply "trampolines" to its inner stream.
inner-stream
Open generic function
inner-stream wrapper-stream => wrapped-stream
Returns the stream wrapped by wrapper-stream.
inner-stream-setter
Open generic function
inner-stream-setter stream wrapper-stream => stream
Wraps stream with wrapper-stream. It does so by setting the inner-stream slot of wrapper-stream to stream, and the outer-stream slot of stream to wrapper-stream.
Note: Applications should not set inner-stream and outer-stream slots directly. The inner-stream-setter function is for use only when implementing stream classes.
outer-stream
Open generic function
outer-stream stream => wrapping-stream
Returns the stream that is wrapping stream.
outer-stream-setter
Open generic function
outer-stream-setter wrapper-stream stream => wrapper-stream
Sets the outer-stream slot of stream to wrapper-stream.
Note: Applications should not set inner-stream and outer-stream slots directly. The inner-stream-setter function is for use only when implementing stream classes.
Copyright 1994, 1995, 1996, 1997 Carnegie Mellon University. All rights reserved.
Send comments and bug reports to gwydion-bugs@cs.cmu.edu