Schedule of Classes

We outline the course, its goals, and talk about various administrative issues. We also introduce the language ML which is used throughout the course

We introduce declarations which evaluate to environments. An environment collects a set of bindings of variables to values which can be used in subsequent declarations or expressions. We also discuss the rules of scope which explain how references to identifiers are resolved. This is somewhat tricky for recursive function declarations.

We review the methods of mathematical and complete induction and show how they can be applied to prove the correctness of ML functions. Key is an understanding of the operational semantics of ML. Induction can be a difficult proof technique to apply, since we often need to generalize the theorem we want to prove, before the proof by induction goes through. Sometimes, this requires considerable ingenuity. We also introduce clausal function definitions based on pattern matching.

One of the most important features of ML is that it allows the definition of new types with so-called datatype declarations. This means that programs can be written to manipulate the data in a natural representation rather than in complex encodings. This goes hand-in-hand with clausal function definitions using pattern matching on given data types. We introduce lists and polymorphic datatypes and functions

We discuss the method of structural induction on recursively defined types. This technique parallels standard induction on predicates, but has a unique character of its own, and arises often in programming. We also discuss tail recursion, a form of recursion that is somewhat like the use of loops in imperative programming. This form of recursion is often especially efficient and easy to analyze. Accumulator arguments play an important role in tail recursion. As examples we consider recursively defined lists and trees

We discuss higher order functions, specifically, passing functions as arguments, returning functions as values, and mapping functions over recursive data structures. Key to understanding functions as first class values is understanding the lexical scoping rules. We discuss staged computation based on function currying

A functor is a parameterized module that acts as a kind of function which takes zero or more structures as arguments and returns a new structure as result. Functors greatly facilitate hierarchical organization in large programs. In particular, as discussed in the next few lectures, they can enable a clean separation between the details of particular definition and higher-level structure, allowing the implementation of "generic" algorithms that are easier to debug and maintain, and that maximize code reuse

Continuations act as "functional accumulators." The basic idea of the technique is to implement a function f by defining a tail-recursive function f' that takes an additional argument, called the continuation. This continuation is a function; it encapsulates the computation that should be done on the result of f. In the base case, instead of returning a result, we call the continuation. In the recursive case we augment the given continuation with whatever computation should be done on the result. Continuations can be used to advantage for programming solutions to a variety of problems. In today's lecture we'll look at a simple example where continuations are used to efficiently manage a certain pattern of control. We'll see a related and more significant example in an upcoming lecture when we look at regular expressions

Regular expressions - and their underlying finite-state automata--are useful in many different applications, and are central to text processing languages and tools such as awk, Perl, emacs and grep. Regular expression pattern matching has a simple and elegant implementation in SML using continuation passing

Exceptions play an important role in the system of static and dynamic checks that make SML a safe language. Exceptions are the first type of effect that we will encounter; they may cause an evaluation to be interrupted or aborted. We have already seen simple uses of exceptions in the course, primarily to signal that invariants are violated or exceptional boundary cases are encountered. We now look a little more closely at what exceptions are and how they can be used. In addition to signaling error conditions, exceptions can sometimes also be used in backtracking search procedures or other patterns of control where a computation needs to be partially undone

The same problem can be solved using several, very different, techniques. We examine a classic puzzle, the n-queens problem, and compare solutions that use exceptions, continuations and functions that return options

The programming techniques used so far in the course have, for the most part, been "purely functional". Some problems, however, are more naturally addressed by keeping track of the "state" of an internal machine. Typically this requires the use of mutable storage. ML supports mutable cells, or references, that store values of a fixed type. The value in a mutable cell can be initialized, read, and changed (mutated), and these operations result in effects that change the store. Programming with references is often carried out with the help of imperative techniques. Imperative functions are used primarily for the way they change storage, rather than for their return values

Previously, within the purely functional part of ML, we saw that all values were persistent. At worst, a binding might shadow a previous binding. As a result our queues and dictionaries were persistent data structures. Adding an element to a queue did not change the old queue; instead it created a new queue, possibly sharing values with the old queue, but not modifying the old queue in any way. Now that we are able to create cells and modify their contents we can create ephemeral data structures. These are data structures that change over time. The main advantage of such data structures is their ability to maintain state as a shared resource among many routines. Another advantage in some cases is the ability to write code that is more time-efficient than purely functional code. The disadvantages are error and complexity: our routines may accidentally and irreversibly change the contents of a data structure; variables may be aliases for each other. As a result it is much more difficult to prove the correctness of code involving ephemeral data structures. As always, it is a good idea to keep mutation to a minimum and to be careful about enforcing invariants. We present two examples. First, we consider a standard implementation of hash tables. We use arrays to implement generic hash tables as a functor parameterized by an abstract hashable equality type. Second, we revisit the queue data structure, now defining an ephemeral queue. The queue signature clearly indicates that internal state is maintained. Our implementation uses a pair of reference cells containing mutable lists, and highlights some of the subtleties involved when reasoning about references We end the lecture with a few words about ML's value restriction. The value restriction is enforced by the ML compiler in order to avoid runtime type errors. All expressions must have well-defined lexically-determined static types

Functions in ML are evaluated eagerly, meaning that the arguments are reduced before the function is applied. An alternative is for function applications and constructors to be evaluated in a lazy manner, meaning expressions are evaluated only when their values are needed in a further computation. Lazy evaluation can be implemented by "suspending" computations in function values. This style of evaluation is essential when working with potentially infinite data structures, such as streams, which arise naturally in many applications. Streams are lazy lists whose values are determined by suspended computations that generate the next element of the stream only when forced to do so

We continue with streams, and complete our implementation by introducing a memoizing delay function. Memoization ensures that a suspended expression is evaluated at most once. When a suspension is forced for the first time, its value is stored in a reference cell and simply returned when the suspension is forced again. The implementation that we present makes a subtle and elegant use of a "self-modifying" code technique with circular references

In this and the next lecture we discuss the computability of functions in ML. By the Church-Turing thesis this is the same notion of computability as we have in recursion theory, with Turing machines, etc. There are two main ideas to show that certain functions are not computable: diagonalization (which is a direct argument), and problem reduction (which shows that a problem is undecidable by giving a reduction from another undecidable problem)

Although the Halting problem says that there is no hope to build a program that will give a yes/no answer to many problems of interest, it is possible to write programs that will return a yes answer but may run forever when the answer is no. This is called a semi-decision procedure. For some other problems, it is always possible to correctly return a no, but a positive answer may never be returned. There is also a class of problems for which either a yes or a no answer may not be returned in finite time.

Many applications require some form of tokenization or lexical analysis to be carried out as a preprocessing step. Examples include compiling programming languages, processing natural languages, or manipulating HTML pages to extract structure. As an example, we study a lexical analyzer for a simple language of arithmetic expressions

Context-free grammars arise naturally in a variety of applications. The "Abstract Syntax Charts" in programming language manuals are one instance. The underlying machine for a context-free language is a pushdown automaton, which maintains a read-write stack that allows the machine to "count"

In this lecture we continue our discussion of context-free grammars, and demonstrate their role in parsing. Shift-reduce parsing uses a stack to delay application of rewrite rules, enabling operator precedence to be enforced. Recursive descent parsing is another style that uses recursion in a way that mirrors the grammar productions. Although parser generator tools exist for restricted classes of grammars, a direct implementation can allow greater flexibility and better error handling. We present an example of a shift-reduce parser for a grammar of arithmetic expressions

We now put together lexical analysis and parsing with evaluation. The result is an interpreter that evaluates arithmetic expressions directly, rather than by constructing an explicit translation of the code into an intermediate language, and then into machine language, as a compiler does. Our first example uses the basic grammar of arithmetic expressions, interpreting them in terms of operations over the rational numbers. In this and the next lecture we extend this simple language to include conditional statements, variable bindings, function definitions, and recursive functions

All techniques discussed so far assumed a sequential model of computation, where all the code was executed on a single machine. Whenever computation takes places on several processors at the same time, we have concurrency. One form of concurrency is parallel computing, where the computation is split among the available processors. Another form is distributed computing where independent machines collaborate by exchanging information. In this lecture, we introduce this general taxonomy and discuss some of the issues that distinguish concurrent from sequential systems, especially communication. We explore the main techniques to address these problems.

Networked applications are an especially important case of distributed systems. We explore a few of the main concepts by showing how to write a simple web server.

Combinators are functions of functions, that is, higher-order functions used to combine functions. One example is ML's composition operator o. The basic idea is to think at the level of functions, rather than at the level of values returned by those functions. Combinators are defined using the pointwise principle. Currying makes this easy in ML. We first discuss combinators of functions of type int -> int. Then we discuss rewriting our regular expression matcher using combinators. We using staging. The regular expression pattern matching is in one stage, the character functions are in another