Chapter 1

Introduction

Now, the question, What is a judgement? is no small question, because the notion of judgement is just about the first of all the notions of logic, the one that has to be explained before all the others, before even the notions of proposition and truth, for instance.

— Per Martin-Löf

On the Meanings of the Logical Constants and the Justifications of the Logical Laws [ML85a]

In everyday computing we deal with a variety of different languages. Some of them such as C, C++, Ada, ML, or Prolog are intended as general purpose languages. Others like Emacs Lisp, Tcl, TEX, HTML, csh, Perl, SQL, Visual Basic, VHDL, or Java were designed for specific domains or applications. We use these examples to illustrate that many more computer science researchers and system developers are engaged in language design and implementation than one might at first suspect. We all know examples where ignorance or disregard of sound language design principles has led to languages in which programs are much harder to write, debug, compose, or maintain than they should be. In order to understand the principles which guide the design of programming languages, we should be familiar with their theory. Only if we understand the properties of complete languages as opposed to the properties of individual programs, do we understand how the pieces of a language fit together to form a coherent (or not so coherent) whole.

As I hope these notes demonstrate, the theory of programming languages does not require a deep and complicated mathematical apparatus, but can be carried out in a concrete, intuitive, and computational way. With a only a few exceptions, the material in these notes has been fully implemented in a meta-language, a so-called logical framework. This implementation encompasses the languages we study, the algorithms pertaining to these languages (e.g., compilation), and the proofs of their properties (e.g., compiler correctness). This allows hands-on experimentation with the given languages and algorithms and the exploration of variants and extensions. We now briefly sketch our approach and the organization of these notes.

1.1 The Theory of Programming Languages

The theory of programming languages covers diverse aspects of languages and their implementations. Logically first are issues of concrete syntax and parsing. These have been relatively well understood for some time and are covered in numerous books. We therefore ignore them in these notes in order to concentrate on deeper aspects of languages.

The next question concerns the type structure of a language. The importance of the type structure for the design and use of a language can hardly be overemphasized. Types help to sort out meaningless programs and type checking catches many errors before a program is ever executed. Types serve as formal, machine-checked documentation for an implementation. Types also specify interfaces to modules and are therefore important to obtain and maintain consistency in large software systems.

Next we have to ask about the meanings of programs in a language. The most immediate answer is given by the operational semantics which specifies the behavior of programs, usually at a relatively high level of abstraction.

Thus the fundamental parts of a language specification are the syntax, the type system, and the operational semantics. These lead to many meta-theoretic questions regarding a particular language. Is it effectively decidable if an input expression is well-typed? Do the type system and the operational semantics fit together? Are types needed during the execution of a program? In these notes we investigate such questions in the context of small functional and logic programming languages. Many of the same issues arise for realistic languages, and many of the same solutions still apply.

The specification of an operational semantics rarely corresponds to an efficient language implementation, since it is designed primarily to be easy to reason about. Thus we also study compilation, the translation from a source language to a target language which can be executed more efficiently by an abstract machine. Of course we want to show that compilation preserves the observable behavior of programs. Another important set of questions is whether programs satisfy some abstract specification, for example, if a particular function really computes the integer logarithm. Similarly, we may ask if two programs compute the same function, even though they may implement different algorithms and thus may differ operationally. These questions lead to general type theory and denotational semantics, which we consider only superficially in these notes. We concentrate on type systems and the operational behavior of programs, since they determine programming style and are closest to the programmer’s intuition about a language. They are also amenable to immediate implementation, which is not so direct, for example, for denotational semantics.

The principal novel aspect of these notes is that the operational perspective is not limited to the programming languages we study (the object language), but encompasses the meta-language, that is, the framework in which we carry out our investigation. Informally, the meta-language is centered on the notions of judgment and deductive system explained below. They have been formalized in a logical framework (LF) [HHP93] in which judgments can be specified at a high level of abstraction, consistent with informal practice in computer science. LF has been given an operational interpretation in the Elf meta-programming language [Pfe91a], thus providing means for a computational meta-theory. Elf implementations of the languages we discuss and proofs of the meta-theorems (except where explicitly noted) are available electronically and constitute an important supplement to these notes. They provide the basis for hands-on experimentation with language variants, extensions, proofs of exercises, and projects related to the formalization and implementation of other topics in the theory of programming languages.

1.2 Deductive Systems

In logic, deductive systems are often introduced as a syntactic device for establishing semantic properties. We are given a language and a semantics assigning meaning to expressions in the language, in particular to a category of expressions called formulas. Furthermore, we have a distinguished semantic property, such as truth in a particular model. A deductive system is then defined through a set of axioms (all of which are true formulas) and rules of inference which yield true formulas when given true formulas. A deduction can be viewed as a tree labelled with formulas, where the axioms are leaves and inference rules are interior nodes, and the label of the root is the formula whose truth is established by the deduction. This naturally leads to a number of meta-theoretic questions concerning a deductive system. Perhaps the most immediate are soundness: “Are the axioms true, and is truth preserved by the inference rules?” and completeness: “Can every true formula be deduced?”. A difficulty with this general approach is that it requires the mathematical notion of a model, which is complex and not immediately computational.

An alternative is provided by Martin-Löf [ML85a, ML85b] who introduces the notion of a judgment (such as “A is true”) as something we may know by virtue of a proof. For him the notions of judgment and proof are thus more basic than the notions of proposition and truth. The meaning of propositions is explained via the rules we may use to establish their truth. In Martin-Löf’s work these notions are mostly informal, intended as a philosophical foundation for constructive mathematics and computer science. Here we are concerned with actual implementation and also the meta-theory of deductive systems. Thus, when we refer to judgments we mean formal judgments and we substitute the synonyms deduction and derivation for formal proof. The term proof is reserved for proofs in the meta-theory. We call a judgment derivable if (and only if) it can be established by a deduction, using the given axioms and inference rules. Thus the derivable judgments are defined inductively. Alternatively we might say that the set of derivable judgments is the least set of judgments containing the axioms and closed under the rules of inference. The underlying view that axioms and inference rules provide a semantic definition for a language was also advanced by Gentzen [Gen35] and is sometimes referred to as proof-theoretic semantics. A study of deductive systems is then a semantic investigation with syntactic means. The investigation of a theory of deductions often gives rise to constructive proofs of properties such as consistency (not every formula is provable), which was one of Gentzen’s primary motivations. This is also an important reason for the relevance of deductive systems in computer science.

The study of deductive systems since the pioneering work of Gentzen has arrived at various styles of calculi, each with its own concepts and methods independent of any particular logical interpretation of the formalism. Systems in the style of Hilbert [HB34] have a close connection to combinatory calculi [CF58]. They are characterized by many axioms and a small number of inference rules. Systems of natural deduction [Gen35, Pra65] are most relevant to these notes, since they directly define the meaning of logical symbols via inference rules. They are also closely related to typed λ-calculi and thus programming languages via the so-called Curry-Howard isomorphism [How69]. Gentzen’s sequent calculus can be considered a calculus of proof search and is thus relevant to logic programming, where computation is realized as proof search according to a fixed strategy.

In these notes we concentrate on calculi of natural deduction, investigating methods for

1. the definition of judgments,

2. the implementation of algorithms for deriving judgments and manipulating deductions, and

3. proving properties of deductive systems.

As an example of these three tasks, we show what they might mean in the context of the description of a programming language.

Let e range over expressions of a statically typed programming language, τ range over types, and v over those expressions which are values. The relevant judgments are

▷ e : τ  e has type τ
e ↪ v  e evaluates to v

1. The deductive systems which define these judgments fix the type system and the operational semantics of our programming language.

2. An implementation of these judgments provides a program for type inference and an interpreter for expressions in the language.

3. A typical meta-theorem is type preservation, which expresses that the type system and the operational semantics are compatible:

If ▷ e : τ is derivable and e ↪ v is derivable, then ▷ v : τ is derivable.

In this context the deductive systems define the judgments under considerations: there simply exists no external, semantical notion against which our inference rules should be measured. Different inference systems lead to different notions of evaluation and thus to different programming languages.

We use standard notation for judgments and deductions. Given a judgment J with derivation D we write

\begin{matrix}D \\ J \end{matrix}

or, because of its typographic simplicity, D :: J. An application of a rule of inference with conclusion J and premisses J_{1}, …, J_{n} has the general form

J_{1} … J_{n} ; rule name ; J

An axiom is simply a special case with no premisses (n = 0) and we still show the horizontal line. We use script letters D, E, P, Q, … to range over deductions. Inference rules are almost always schematic, that is, they contain meta-variables. A schematic inference rule stands for all its instances which can be obtained by replacing the meta-variables by expressions in the appropriate syntactic category. We usually drop the byword “schematic” for the sake of simplicity.

Deductive systems are intended to provide an explicit calculus of evidence for judgments. Sometimes complex side conditions restrict the set of possible instances of an inference rule. This can easily destroy the character of the inference rules in that much of the evidence for a judgment now is implicit in the side conditions. We therefore limit ourselves to side conditions regarding legal occurrences of variables in the premisses. It is no accident that our formalization techniques directly account for such side conditions. Other side conditions as they may be found in the literature can often be converted into explicit premisses involving auxiliary judgments. There are a few standard means to combine judgments to form new ones. In particular, we employ parametric and hypothetical judgments. Briefly, a hypothetical judgment expresses that a judgment J may be derived under the assumption or hypothesis J′. If we succeed in constructing a deduction D′ of J′ we can substitute D′ in every place where J′ was used in the original, hypothetical deduction of J to obtain unconditional evidence for J. A parametric judgment J is a judgment containing a meta-variable x ranging over some syntactic category. It is judged evident if we can provide a deduction D of J such that we can replace x in D by any expression in the appropriate syntactic category and obtain a deduction for the resulting instance of J.

In the statements of meta-theorems we generally refer to a judgment J as derivable or not derivable. This is because judgments and deductions have now become objects of investigation and are the subjects of meta-judgments. However, using the meta-judgment is derivable pervasively tends to be verbose, and we will take the liberty of using “J” to stand for “J is derivable” when no confusion can arise.

1.3 Goals and Approach

We pursue several goals with these notes. First of all, we would like to convey a certain style of language definition using deductive systems. This style is standard practice in modern computer science and students of the theory of programming languages should understand it thoroughly.

Secondly, we would like to impart the main techniques for proving properties of programming languages defined in this style. Meta-theory based on deductive systems requires surprisingly few principles: induction over the structure of derivations is by far the most common technique.

Thirdly, we would like the reader to understand how to employ the LF logical framework [HHP93] and its implementation in Elf [Pfe91a] in order to implement these definitions and related algorithms. This serves several purposes. Perhaps the most important is that it allows hands-on experimentation with otherwise dry definitions and theorems. Students can get immediate feedback on their understanding of the course material and their ideas about exercises. Furthermore, using a logical framework deepens one’s understanding of the methodology of deductive systems, since the framework provides an immediate, formal account of informal explanations and practice in computer science.

Finally, we would like students to develop an understanding of the subject matter, that is, functional and logic programming. This includes an understanding of various type systems, operational semantics for functional languages, high-level compilation techniques, abstract machines, constructive logic, the connection between constructive proofs and functional programs, and the view of goal-directed proof search as the foundation for logic programming. Much of this understanding, as well as the analysis and implementation techniques employed here, apply to other paradigms and more realistic, practical languages.

The notes begin with the theory of Mini-ML, a small functional language including recursion and polymorphism (Chapter 2). We informally discuss the language specification and its meta-theory culminating in a proof of type preservation, always employing deductive systems. This exercise allows us to identify common concepts of deductive systems which drive the design of a logical framework. In Chapter 3 we then incrementally introduce features of the logical framework LF, which is our formal meta-language. Next we show how LF is implemented in the Elf programming language (Chapter 4). Elf endows LF with an operational interpretation in the style of logic programming, thus providing a programming language for meta-programs such as interpreters or type inference procedures. Our meta-theory will always be constructive and we observe that meta-theoretic proofs can also be implemented and executed in Elf, although at present they cannot be verified completely. Next we introduce the important concepts of parametric and hypothetical judgments (Chapter 5) and develop the implementation of the proof of type preservation. At this point the basic techniques have been established, and we devote the remaining chapters to case studies: compilation and compiler correctness (Chapter 6), natural deduction and the connection between constructive proofs and functional programs (Chapter 7), the theory of logic programming (Chapter 8), and advanced type systems (Chapter 9).^{1}

1[One or two additional topics may be added.]