An Introduction to Serpent

Roger B. Dannenberg

Serpent is a programming language inspired by Python. Like Python, it has a simple, minimal syntax, dynamic typing, and support for object-oriented programming. Serpent also draws inspiration from XLisp, Squeak, SmallTalk, Ruby, and Basic. Why another language? Serpent is designed for use in real-time systems, especially interactive multimedia systems. Serpent is unique in providing the following combination of features: I recommend Python to anyone who does not need these features. Serpent has some other differences from Python that reflect both the designer's personal tastes and limited resources:

A quick table of contents:

[types] [syntax] [expressions] [statements] [declarations]
[built-in functions] [special variables] [standard libraries]
[Installation Notes]
[midi and time] [graphical user interfaces] [threads] [networking] [windows shell file operations]
[Aura extensions] [extending Serpent]

Types

Serpent has the following primitive types: and the following structured types: and the following types that users ordinarily don't think of as types: There are several other types used internally that are not visible to programmers. There is no Boolean type. Instead, the special value nil represents false, and the symbol t represents true. The global variables true and false are bound to t and nil and should be used for readability.

Syntax

Serpent syntax, like that of Python, uses indentation to indicate structure. Statements are grouped, not by brackets, but by indenting them to the same level, each statement on a separate line. If a statement "group" has just one statement, it does not need to go on a new line, but this is not yet implemented.

Expressions

Serpent expressions use the following operators, listed in order of precedence:
., [], **

(unary) +, (unary) -, ~, not

*, /, %, &

+, -, |, ^, <<, >>

<, <=, =, !=, >, >=, is, is not, in, not in

and

or

Other expressions are formed by calling methods and functions. Parameters are passed by value, and functions may have optional, keyword, and arbitrary numbers of parameters. Keyword parameters are passed by writing the keyword followed by "=", as in:
foo(size = 23)
Only parameters declared as keyword or dictionary parameters can be passed using the keyword notation. All other parameters are positional.

Any expression can be used as a statement.

Statements

Statements should be familiar to programmers, so they will be described here by example and at most a few comments. Note that because of the indentation-based grouping, semicolons are not used (except in the print statement for formatting).

Load and Require

The "load" and "require" statements load a file, executing each statement and compiling each function and class sequentially in the order read from the file. The file may contain its own "load" and "require" statements, which may be immediate commands at the top level or run-time load commands excuted within a function or method. The standard file extension is appended to the filename if no file is found without the extension.
load "myfile"
require "myfile"
The difference between load and require is that load always compiles and executes the contents of the file, whereas require first checks to see if the file has been loaded (or "required") previously. If so, the file is not loaded again. Use load when the file contains a script of actions that may be performed multiple times. Use require for files that contain class and function definitions that should be loaded at most one time.

A search path is used to locate source files. Under Windows, the search path is stored in the registry as the SERPENTPATH value in HKEY_LOCAL_MACHINE/SOFTWARE/CMU/Serpent. Under Linux and Mac OS X, the environment variable SERPENTPATH holds the search path.

The search path is a list of paths separated by colons (:) in Linux or OS X or by semicolons (;) in Windows.

Please read these installation notes for more details on setting up a search path.

The search for a file begins by testing the supplied filename for a ".srp" suffix. If none is found, ".srp" is appended to the filename. Then the compiler tries to open the file in the load directory. The load directory is current directory at the time Serpent was started, or, if an initial file was specified on the command line, the directory of that file. If the file is not found in the load directory, the file is searched for sequentially in each directory in the search path until the file is found.

A list of loaded files is kept in an array bound to the global variable files_loaded. The only the filenames with extensions are stored here. Therefore, when a require command is executed, Serpent can check to see if the file has been loaded without appending each path from the search path.

A list (array) of path names of the currently loading files is kept on the global variable paths_currently_loading. When a nested load begins, the new path is appended to the array, and when the load completes, the path is popped from the end of the array. An empty array indicates that no load is in progress.

Assignment

identifier = expression
an_array[index] = expression
a_dictionary[key] = expression
an_object.field_name = expression

Conditional

if condition1:
    stmt1
    stmt2
elif condition2:
    stmt3
    stmt4
else:
    stmt5
    stmt6

Iteration

There are three forms of iteration constructs, starting with the familiar "while":
while condition:
    stmt1
    stmt2
In the first "for loop" form, the "variable" must be declared as a local variable, and the "by" part is optional:
for variable = expression1 to expression2 by expression3:
    stmt1
    stmt2
In the second "for loop" form, "expression1" must evaluate to an array, and "variable" is bound to each value in the array:
for variable in expression1:
    stmt1
    stmt2
At present, there are no equivalents to C's "break" and "continue" statements, but these will be added.

Print

print expr1, expr2, expr3; expr4
The print statement writes characters to the standard output, which is the file whose handle is the value of global variable stdout. Each expression is printed. A comma outputs one space while a semicolon outputs no space. If there is no trailing comma or semicolon, a newline is output after the last expression in the print statement. A print statement has the value nil.

Return

return expression
The expression is optional; nil is returned if no expression is provided.

Exceptions

There is no exception mechanism, but one may be added in the future.

Declarations

Serpent functions and classes are created by declaration. Within classes, member variables and methods are declared. Within functions, local variables are declared. Global variables do not need to be declared. Symbols and global variables are equivalent: every symbol has a slot to hold the value of a global, and every global is implemented by creating a symbol.

Parameter Lists

Simple, positional parameters are declared in the parameter list by simply naming them with comma separators:
def foo(p1, p2, p3):
Parameters can also be specified as optional (the parameter can be omitted, a default value can be provided), keyword (the formal parameter is named by the caller, the parameter may have a default value), rest (there can only be one "rest" parameter; it is initialized to an array containing the value of all left-over actual positional parameters), dictionary (there can only be one "dictionary" parameter; it is initialized to a dictionary containing the values of all left-over keyword parameters), and required (standard positional parameters).
def bar(p1, optional p2 = 5, keyword p3 = 6, rest p4, dictionary p5):
This function could be called by, for example:
bar(1, p3 = 3, d1 = 4), or

bar(1, 2, 3, 4, 5)

For optional and keyword parameters, a default value may be provided. The syntax is "= exp" where expr is either a constant (a number, a string, or a symbol) or a global variable identifier. If the value is a global variable identifier, the value of that variable at compile time is used. If the value changes at run-time, this will have no effect on the default parameter value. The expr may not be an expression involving functions or operators.

Functions

def foo(p1, p2, p3):
    var x
    stmt1
    stmt2
    return expression
Functions return the value of the last statement if there is no return statement. Remember that statements may be expressions, allowing a functional style:
def add1(x):
    x + 1

Local Variables

As shown above, local variables are declared using "var". Locals may be initialized, and multiple locals can be declared with a single "var" declaration. The declaration may occur anywhere in the function, but it must occur before the first use of the variable.
var x = 1, y = 2

Classes

class myclass(superclass):
    var instance_var
    def method1(p1):
        instance_var = p1
Classes specify instance variables (without initialization) and methods, which look just like function declarations except they are nested within the "class" construct. A class may inherit from one superclass. Within a method, the keyword "this" refers to the object. To create an object of the class, use the class name as a constructor:
x = myclass(5)
When an object is created, the init method, if any, is called, and parameters provided at object creation are passed to init. The init method return value is ignored, so it is not necessary to explicitly return this. Within the init method of a subclass, there should be a call to super using function syntax and with parameters appropriate to the superclass's init method. For example, if the superclass is myclass, and the init method of myclass takes one argument, then there should be a call that looks like super(5). The return value of this call should be ignored.

Member variables may be accessed directly using "." as in x.instance_var.

Built-in Functions and Methods

Most built-in functions and methods are known to the compiler, which checks and enforces the parameter counts and translates calls to efficient virtual machine instructions. You can define methods with matching names, but the number of parameters must match. The implementation may assume the "object" for some built-in methods is an array, file, or string. This is an implementation restriction that should be corrected.
array(n)
create array of length n, each element initialized to nil
dict(n)
create an empty dictionary expected to grow to size n (n is a hint)
abs(x)
absolute value of integer or float
int(x)
conversion to 64-bit integer from a float (by truncation) or string
real(x)
conversion to 64-bit double from an integer or string
within(x, y, epsilon)
true if x is within epsilon of y (three reals)
pow(x, y)
x to the power y, x and y are floats
x ** y
x to the power y, x and y are floats
x | y
bitwise or of x and y (two integers)
x ^ y
bitwise exclusive or of x and y (two integers)
x & y
bitwise and of x and y (two integers)
x << n
x shifted n bits left (two integers)
x >> n
x shifted n bits right (two integers)
~x
the bits of integer x inverted
len(s)
length of s, an array or string
idiv(i, j)
integer division: i divided by j
min(s)
minimum value in s, an array of numbers
max(s)
maximum value in s, an array of numbers
cos(x)
cosine of x, a float
sin(x)
sine of x, a float
tan(x)
tangent of x, a float
exp(x)
natural exponent of x, a float
sqrt(x)
square root of x, a float
rem(n, m)
remainder of n divided by m, two integers
random()
random float from [0 to 1)
s.append(x)
extend sequence s (an array or string) by element x; note: use + operator to append two sequences; if s is a string, it is not modified but a new string is constructed
s.unappend()
remove and return last element of s, an array
s.last()
return last element of s, an array or string
s.set_len(n)
set the length of array s to n; truncate or append nil as necessary; truncate string s or pad with spaces to make new string of length n
s.count(x)
count x's in s
s.index(x)
index of first x in array s (see find function for string searching)
s.insert(i, x)
insert x as new ith element of an array or string; if i is negative, insert at len + i; use append to insert at end of s. If s is an array, one element is inserted. If s is a string, x must be a string and the entire string is inserted.
s.uninsert(i [, j])
remove s[i] (through s[j-1]), where s is an array or string.
s.reverse()
reverse order of sequence s, an array or string.
s.sort([f])
sort elements of array s in increasing order, or use f (a symbol) to compare using a global function; f(x, y) returns true iff x should be after y in the sorted sequence. Elements of s may be arrays, in which case the first element of each array is the sort key.
s.resort([f])
if all but the last element of array s are sorted in decreasing order, this will sort s. Use this function to implement a priority queue. Insert by appending an element and calling resort(). Remove the lease element by calling unappend(). Elements of s may be arrays, in which case the first element of each array is the sort key.
a.clear()
remove all items from a, a dictionary or array
a.copy()
shallow copy of a
a.has_key(k)
t if k is key in a
a.keys()
keys of dictionary a
a.values()
values of dictionary a
a.get(k [, f])
gets item in a with key k (f is returned if no item found, nil is returned if no f is given)
a.remove(x)
remove item with key x from dictionary a, or remove first element equal to x from array a
f.close()
close a file
f.flush()
flush a file
f.fileno()
file number
f.read([size])
read up to size (default 4096) bytes from file f. Returns empty string if end-of-file is reached.
f.readline([size])
read line of characters including newline from f, limit number of bytes to size (default 255). Return nil if end of file is reached.
f.readlines([sizehint])
read lines and return in list (if sizehint is given, then read about that many bytes rather than reading to eof). Long lines (> 255 chars) may be split.
f.readvalue()
read and parse a constant (integer, real, string, or symbol). Note that dictionaries and arrays are not parsed. Whitespace is skipped. Anything that starts with something other than a digit, a period, a single quote, or a double quote is parsed as a symbol (terminated by whitespace).
f.seek(offset, whence)
position in file (whence = 0 means absolute, 1 means relative, 2 means relative to the end)
f.tell()
return current position in file
f.token()
skip over white space in file f; read string of non-whitespace characters terminated by a whitespace. The maximum token length is 255 characters.
f.unread(c)
push a single character (a string of length 1) back to an input file
f.write(str)
write string to file
f.writelines(list)
write strings to file
f.closed()
boolean status
f.mode()
string that file was opened with
f.name()
string that file was opened with
chr(i)
convert int, an ascii character code, to a one-character ascii string; if i is 0, the string is empty
hex(i)
convert int to hex string
oct(i)
convert int to octal string
hash(object)
return a hash value
id(object)
address of the object as an integer
intern(string)
convert string to atom
isinstance(object, class)
is object a direct or indirect instance of class?
issubclass(class1, class2)
is class1 a direct or indirect subclass of class2?
funcall(function, arg1, arg2, ...)
call function (an atom) with arguments
apply(function, argarray)
call function (an atom) with arguments taken from an array (there is no provision for sending keyword or dictionary parameters in this way)
open(filename, mode)
file open, mode is a string (see fopen() in C stdio library)
ord(c)
inverse of chr(): returns the integer that is the ascii code for the first character in c, a string; if c is empty, zero is returned
cwd()
return current working directory name
rename(oldpath, newpath)
rename a file. Returns nil on success, or errno on error. errno is system dependent.
mkdir(path)
make the specified directory
isdir(path)
return true iff path names a directory
repr(object)
machine readable string representation of object
round(x, n)
round x to n digits to the right of the decimal point; returns Real if n > 0, otherwise Integer; n can be negative.
sizeof(object)
size in bytes of actual memory used by object
send(object, method, arg1, arg2, ...)
invoke method (an atom) on object with the given arguments
sendapply(object, method, argarray)
invoke method (an atom) on object with the arguments given in an array (no provision is made for keyword and dictionary parameters)
str(object)
string representation of object (not necessarily machine readable)
type(object)
return type of object (an atom)
unlink(filename)
unlink (delete) a file. Returns nil on success, or errno on error. errno is system dependent.
flatten(array)
convert array of strings to one string
strcat(a, b)
concatenate two strings
subseq(s, start [, end])
a subsequence of string or array, returns a new string or array consisting of elements from start through end-1. The default value for end is len(s). If start and/or end is negative, it is interpreted as len(s)-start or len(s)-end, respectively. E.g. subseq([0,1,2,3],-2,-1] is [2].
isupper(s)
is character in A:Z?
islower(s)
is character in a:z?
isalpha(s)
is character in A:Z,a:z?
isdigit(s)
is character in 0:9?
isalnum(s)
isdigit(s) || isalpha(s)?
toupper(s)
convert letters to upper case in s
tolower(s)
convert letters to lower case in s
find(string, pattern [, start [, end])
search string for pattern contained within start:end, no match returns -1
exit()
stop execution
frame_previous(frame)
return previous stack frame
frame_variables(frame)
return dictionary of variables and their values
frame_pc(frame)
program counter of frame
frame_method(frame)
method name of frame
frame_class(frame)
class of frame
runtime_exception_nesting()
nesting level of exceptions
frame_get()
get current frame.
error(s)
generate a run-time error with message s. This normally invokes the debugger.
trace(n)
Set trace mode.  Bit 0 controls machine tracing: instructions are disassembled and printed as they are executed, and the contents of the stack are displayed. Bit 1 controls compiler tracing: tokens are printed as they are parsed, and other debugging output related to the compiler and code generation are printed.
dbg_gc_watch(x)
Turns on debugging information in garbage collector for address x. Only one address may be watched. This only has an effect when Serpent is compiled with the flag _GCDEBUG.
When a built-in operation encounters the wrong types and the first argument is an object, then the call is converted into a method lookup. (This is not implemented for all primitives yet.)

It is illegal to define a global function with the name of a built-in function.

Serpent does not have first-class functions. Instead, functions are represented by atoms (call the corresponding global function) or by objects (call a method of the object).

Special Variables

command_line_arguments
An array of strings from the command line.
dbg_trace_output_disable
In the debug version (compiled with _DEBUG defined), Serpent copies every output byte to the trace() function, a platform dependent debugging output function. If you set this variable to non-nil, this trace output is disabled. This can make the program run faster in debug mode.
files_loaded
An array of files that have been loaded so far. The require statement uses this list to decide whether to load a file.
paths_currently_loading
An array of strings naming the paths of the set of nested loads in progress. If you want to load a file relative to a source file, you should save paths_currently_loading.last() in a statement that is executed while the file is being loaded.
runtime_exception
If defined as a global function, this function will be called by Serpent to handle execution exceptions within Serpent code. See debug.srp, which uses this feature to implement a simple debugger.

Standard Libraries

Serpent comes with a number of files in the lib directory, and normally this directory should be on the Serpent search path. Serpent libraries are evolving with use. Feel free to contribute new libraries and methods of general utility. The list below is intended as a rough guide. Please read documentation in the source files themselves for more detail.
debug
Serpent has a primitive debugger. When you load the debug library, errors are passed to some library code that can print a stack trace and examine some variable values. Also, you can exit back to the command line prompt. Type RETURN to the debugger for a summary of commands.
regression
The Regression class can be used to perform (linear) regression.
statistics
The Statistics class can be used to perform simple statistics such as mean and standard deviation. It can also buffer a set of statistical samples and save them to a file.
stredit
The String_edit class is intended to edit files, e.g. perform global query-and-replace operations. The original purpose was editing templates to automatically generate HTML as part of the serpent software release process, but any file or string editing might make use of this class.
strparse
The String_parse class is intended to parse data files of all kinds one line at a time. You pass in a string you want to parse, and then sequentially advance through the string skipping spaces, reading words, integers, floats, special characters, or whatever. Highly recommended for all your text input needs.
utils
This is a grab-bag of handy functions, including: irandom to get random integers, change_file_suffix to replace a file name suffix, file_from_path to extract the file name from a full path, and pad to pad a string to a given length.

Midi and Time Functions

Serpent has an interface to PortMidi and PortTime, a cross-platform library for MIDI I/O and millisecond-accuracy time. Details are here. See also the thread functions (below) that implement periodic timer callbacks, and Windows Shell File Operations (below) that include a local_time() function.

Graphical User Interface

Serpent has a simple cross-platform interface to wxWidgets. Because wxWidgets takes over the main application thread, there is a version of serpent (called wxserpent) that gives control to wxWidgets. Some further details here here.

Thread Interface and Functions

Serpent has a very primitive interface to allow multiple threads. It is strongly recommended that you do not depend heavily on this facility. It was created to support a course and is not intended for "real" use. The facility is limited to the creation of one additional thread that loads a file and then periodically calls a function. The two threads run in a single process but have no shared variables. The only possibility of communication is through message queues. Two queues are set up and initialized to hold up to 100 strings of up to 100 characters each. Only strings may be sent and received. To build Serpent with these thread functions, link Serpent with the objects obtained from threadcreate.cpp and threadhack.cpp.
thread_create(period, filename, mempool)
thread_send(thread_id, string)
enqueue string for receipt by the other thread. Return the number of strings sent (0 if the queue is full, 1 if the send is successful.) thread_id should be the thread id of the caller, not the destination. (Use the global variable thread_id.
thread_receive(thread_id)
check the queue and if there is a message from the other thread, return the message as a string. If there is no message, "" (the empty string) is returned. Note that it is possible to send an empty string, but this will be indistinguishable from no message (an empty queue). thread_id should be the thread id of the caller, not the thread that sent the message. (Use the global variable thread_id).

Network Interface and Functions

If Serpent is compiled with NETWORK defined, then some basic communications functions are built-in. They are defined in this section.
server_create(portno)
Create a socket, bind it to portno, and listen for client connections. A socket descriptor (number) is returned. -1 is returned to indicate an error.
server_accept(socket)
Accept a client request on socket, which was created by server_create. If the return value is nil, then no client request is pending (this is a non-blocking call). If the return value is -1, an error occurred. Otherwise, the return value is socket that can be used to read the client request. Under Windows, calling this function initiates a blocking accept call in another thread. In order to call server_connect or socket_receive, you must continue (re)calling server_accept until it returns something other than nil. To terminate the blocked accept, try closing the server socket and then re-calling server_accept to read the error return.
server_connect(name, portno)
Establish a connection with a server using its name and port number. The result is a socket, nil if no result is available yet (this is a non-blocking call), or -1 if there is an error. If nil is returned, you must re-call server_connect until a non-nil result is obtained.
socket_receive(socket, n)
Read up to n bytes of data from socket. Returns a string if successful, nil if no input is available (this is a non-blocking call), and otherwise returns an integer error code. The socket is normally obtained from server_accept or server_connect. If nil is returned, the read is still in progress, and you must re-call socket_receive until a non-nil result is obtained.
socket_send(socket, string)
Send a string to the given socket, which is normally obtained from server_accept or server_connect. Returns the number of bytes sent or -1 on error.
socket_close(socket)
Close a socket.

Windows Shell File Operations

The Win32 version of Serpent includes an interface to "Shell File Operations" that perform tasks such as copying directories. These functions are:
sfo_copy_directory(from_path, to_path)
Copy a directory named by from_path to to_path (both arguments are strings).
sfo_delete(path)
Delete a file or directory named by path (a string).
create_directory(path)
Create a directory named by path (a string).
listdir(path)
Obtain a directory listing of path (a string). The result is an array of strings.
local_time()
Return the local time as an array of integers, organized as follows: [seconds, minutes, hours, day-of-month, month, year, day-of-week, day-of-year, dst], where dst is 1 for daylight savings time and 0 otherwise.

Aura Extensions

[Note: this section is obsolete due to a conversion from attribtue/value messages to remote method invocation in Aura.] Serpent has additional statements to support Aura message passing:
connect object-id from object-id
connect object-id to object-id
set attribute to value after delay
object-id <- set attribute = value at timestamp
set attribute to value
As illustrated by these examples, the "<-", "at", and "after" parts are optional. Attributes are named by symbols. A Serpent dictionary maps a symbol to the corresponding Aura object, and values are converted to match the type of the attribute.

The classes "Aura_object" and "Aura_active" are built-in and may be subclassed. When a subclass is defined, a new Aura prototype is created, the class name (as a symbol) is added to the Serpent/Aura dictionary with a reference to the prototype, and the Aura prototype is initialized to forward messages to the new class.

Subclasses of "Aura_object" and "Aura_active" may have methods defined by the following form:

on attribute:
    statements
This declares a method to be called when the attribute is set (the method is named "_on_" appended to the attribute name. An instance variable is also declared if it does not already exist.

A dictionary is used to specify the interface between Aura messages and the Serpent object. The dictionary is stored as the instance variable aura_interface. The dictionary is a mapping from attributes to arrays with the following elements:

To build the aura_interface dictionary, every class should provide a method named build_aura_interface that constructs the interface dictionary. This method is called when the class is defined in order to create the prototype object.

Extending Serpent

Serpent can be extended with functions and data types. The interface between Serpent and external code is generated semi-automatically using the Serpent program interface.srp. Not all C types are supported, and the mapping between Serpent and C types has some restrictions and special cases, so sometimes the developer must create some "glue" functions to translate between Serpent and C. The supported types are as follows:
Type name in interface description Converted to/from this type Type within Serpent
long int64 Integer
short int64 Integer
int int64 Integer
char char String
string char * String
double double Real
float double Real
bool FVal Symbol
any FVal --
Object_ptr Object_ptr Object
FILE * FILE * File
All interfaces are explicitly indicated by adding comments to C code as follows:

/*SER type function_name [c_name] (type1, type2, type3, ...) PENT*/

Generates an interface to function_name, which refers to the Serpent name for the function. If the C name is different, it is specified between square brackets. (Note that bracket characters actually appear in the comment; they are not meta-syntax characters.) The types are parameter types as shown in the first column of the  table of types shown above. In addition, "external" types may be specified as extern typename.

Finally, there are cases where the function should have access to the virtual machine, which is a C++ object of type Machine. Serpent programs cannot access the virtual machine as an object, so it is impossible to explicitly pass the machine as a parameter. However, if the type is specified as Machine, a pointer to the machine of the caller will be passed automatically. Since the parameter is implicit, the generated function in Serpent will have one less parameter than the corresponding C function.

/*SER class Class_name PENT*/
Serpent can be extended with new types using this form of comment.
/*SER variable = value PENT*/
Global variables in Serpent can be initialized to a value using this form of comment. The value must be a string, integer, symbol, or real constant. No expressions are allowed. When in doubt, value is converted to symbol.

Interfaces to Classes and Structures

The files extclass.h and extclass.cpp give an example of an interface to a class. Note the use of the class Descriptor to describe the external class to the Serpent run-time system. Descriptor is subclassed to build a descriptor for the new type. This is not an automatic operation; you must build your own descriptor subclass.

Interfaces to Functions

The files extfuncdemo.h and extfuncdemo.cpp provide an example of an interface to ordinary functions written in C or C++.

Building an Interface

Interfaces based on one or more .h file are generated by loading "interface.srp" and calling the interf function as shown in the following example:
interf("smid", ["midi.h", "midiserpent.h"])
The first parameter specifies the output file name (without the .cpp extension). This name is also used for some internal names that must be generated. The second parameter is a list of files to process. Each of these files is included in the generated output file using an #include directive, so if you want a file included,  list it even if it contains no /*SER ... PENT*/ comments.

In order to find header files not in the current directory, you can provide a search path as follows:

interface_search_path = ["..\\midi\\", ...]