SlideShare a Scribd company logo
Read Anytime Anywhere Easy Ebook Downloads at ebookmeta.com
Essentials of Compilation An Incremental Approach
in Python 1st Edition Jeremy G. Siek
https://ebookmeta.com/product/essentials-of-compilation-an-
incremental-approach-in-python-1st-edition-jeremy-g-siek/
OR CLICK HERE
DOWLOAD EBOOK
Visit and Get More Ebook Downloads Instantly at https://ebookmeta.com
Essentials of Compilation
Essentials of Compilation An Incremental Approach in Python 1st Edition Jeremy G. Siek
Essentials of Compilation
An Incremental Approach in Python
Jeremy G. Siek
The MIT Press
Cambridge, Massachusetts
London, England
© 2023 Jeremy G. Siek
This work is subject to a Creative Commons CC-BY-ND-NC license.
Subject to such license, all rights are reserved.
The MIT Press would like to thank the anonymous peer reviewers who provided comments on
drafts of this book. The generous work of academic experts is essential for establishing the
authority and quality of our publications. We acknowledge with gratitude the contributions of
these otherwise uncredited readers.
This book was set in Times LT Std Roman by the author. Printed and bound in the United
States of America.
Library of Congress Cataloging-in-Publication Data
Names: Siek, Jeremy, author.
Title: Essentials of compilation : an incremental approach in Python / Jeremy G. Siek.
Description: Cambridge, Massachusetts : The MIT Press, [2023] | Includes bibliographical
references and index.
Identifiers: LCCN 2022043053 (print) | LCCN 2022043054 (ebook) | ISBN 9780262048248 |
ISBN 9780262375542 (epub) | ISBN 9780262375559 (pdf)
Subjects: LCSH: Compilers (Computer programs) | Python (Computer program language) |
Programming languages (Electronic computers) | Computer programming.
Classification: LCC QA76.76.C65 S54 2023 (print) | LCC QA76.76.C65 (ebook) | DDC
005.4/53–dc23/eng/20221117
LC record available at https://lccn.loc.gov/2022043053
LC ebook record available at https://lccn.loc.gov/2022043054
10 9 8 7 6 5 4 3 2 1
This book is dedicated to Katie, my partner in everything, my children, who grew
up during the writing of this book, and the programming language students at
Indiana University, whose thoughtful questions made this a better book.
Essentials of Compilation An Incremental Approach in Python 1st Edition Jeremy G. Siek
Contents
Preface xi
1 Preliminaries 1
1.1 Abstract Syntax Trees 1
1.2 Grammars 3
1.3 Pattern Matching 5
1.4 Recursive Functions 6
1.5 Interpreters 8
1.6 Example Compiler: A Partial Evaluator 10
2 Integers and Variables 13
2.1 The LVar Language 13
2.2 The x86Int Assembly Language 16
2.3 Planning the Trip to x86 21
2.4 Remove Complex Operands 23
2.5 Select Instructions 25
2.6 Assign Homes 26
2.7 Patch Instructions 27
2.8 Generate Prelude and Conclusion 27
2.9 Challenge: Partial Evaluator for LVar 28
3 Parsing 29
3.1 Lexical Analysis and Regular Expressions 29
3.2 Grammars and Parse Trees 31
3.3 Ambiguous Grammars 33
3.4 From Parse Trees to Abstract Syntax Trees 34
3.5 Earley’s Algorithm 36
3.6 The LALR(1) Algorithm 40
3.7 Further Reading 43
4 Register Allocation 45
4.1 Registers and Calling Conventions 46
4.2 Liveness Analysis 49
4.3 Build the Interference Graph 51
viii Contents
4.4 Graph Coloring via Sudoku 52
4.5 Patch Instructions 58
4.6 Generate Prelude and Conclusion 58
4.7 Challenge: Move Biasing 59
4.8 Further Reading 62
5 Booleans and Conditionals 65
5.1 The LIf Language 66
5.2 Type Checking LIf Programs 66
5.3 The CIf Intermediate Language 72
5.4 The x86If Language 72
5.5 Shrink the LIf Language 75
5.6 Remove Complex Operands 75
5.7 Explicate Control 76
5.8 Select Instructions 82
5.9 Register Allocation 83
5.10 Patch Instructions 84
5.11 Generate Prelude and Conclusion 84
5.12 Challenge: Optimize Blocks and Remove Jumps 85
5.13 Further Reading 88
6 Loops and Dataflow Analysis 91
6.1 The LWhile Language 91
6.2 Cyclic Control Flow and Dataflow Analysis 91
6.3 Remove Complex Operands 96
6.4 Explicate Control 96
6.5 Register Allocation 96
7 Tuples and Garbage Collection 99
7.1 The LTup Language 99
7.2 Garbage Collection 102
7.3 Expose Allocation 109
7.4 Remove Complex Operands 110
7.5 Explicate Control and the CTup Language 110
7.6 Select Instructions and the x86Global Language 111
7.7 Register Allocation 116
7.8 Generate Prelude and Conclusion 116
7.9 Challenge: Arrays 118
7.10 Further Reading 123
8 Functions 125
8.1 The LFun Language 125
8.2 Functions in x86 130
8.3 Shrink LFun 133
8.4 Reveal Functions and the LFunRef Language 133
Contents ix
8.5 Limit Functions 133
8.6 Remove Complex Operands 134
8.7 Explicate Control and the CFun Language 135
8.8 Select Instructions and the x86Def
callq∗ Language 136
8.9 Register Allocation 138
8.10 Patch Instructions 139
8.11 Generate Prelude and Conclusion 139
8.12 An Example Translation 141
9 Lexically Scoped Functions 143
9.1 The Lλ Language 145
9.2 Assignment and Lexically Scoped Functions 150
9.3 Uniquify Variables 151
9.4 Assignment Conversion 151
9.5 Closure Conversion 153
9.6 Expose Allocation 156
9.7 Explicate Control and CClos 156
9.8 Select Instructions 157
9.9 Challenge: Optimize Closures 158
9.10 Further Reading 160
10 Dynamic Typing 161
10.1 The LDyn Language 161
10.2 Representation of Tagged Values 165
10.3 The LAny Language 166
10.4 Cast Insertion: Compiling LDyn to LAny 170
10.5 Reveal Casts 170
10.6 Assignment Conversion 171
10.7 Closure Conversion 171
10.8 Remove Complex Operands 172
10.9 Explicate Control and CAny 172
10.10 Select Instructions 172
10.11 Register Allocation for LAny 174
11 Gradual Typing 177
11.1 Type Checking L? 177
11.2 Interpreting LCast 183
11.3 Overload Resolution 184
11.4 Cast Insertion 185
11.5 Lower Casts 187
11.6 Differentiate Proxies 188
11.7 Reveal Casts 190
11.8 Closure Conversion 191
11.9 Select Instructions 191
11.10 Further Reading 193
x Contents
12 Generics 195
12.1 Compiling Generics 201
12.2 Resolve Instantiation 202
12.3 Erase Generic Types 202
A Appendix 207
A.1 x86 Instruction Set Quick Reference 207
References 209
Index 217
Preface
There is a magical moment when a programmer presses the run button and the
software begins to execute. Somehow a program written in a high-level language is
running on a computer that is capable only of shuffling bits. Here we reveal the wiz-
ardry that makes that moment possible. Beginning with the groundbreaking work
of Backus and colleagues in the 1950s, computer scientists developed techniques
for constructing programs called compilers that automatically translate high-level
programs into machine code.
We take you on a journey through constructing your own compiler for a small
but powerful language. Along the way we explain the essential concepts, algorithms,
and data structures that underlie compilers. We develop your understanding of how
programs are mapped onto computer hardware, which is helpful in reasoning about
properties at the junction of hardware and software, such as execution time, soft-
ware errors, and security vulnerabilities. For those interested in pursuing compiler
construction as a career, our goal is to provide a stepping-stone to advanced topics
such as just-in-time compilation, program analysis, and program optimization. For
those interested in designing and implementing programming languages, we connect
language design choices to their impact on the compiler and the generated code.
A compiler is typically organized as a sequence of stages that progressively trans-
late a program to the code that runs on hardware. We take this approach to the
extreme by partitioning our compiler into a large number of nanopasses, each of
which performs a single task. This enables the testing of each pass in isolation and
focuses our attention, making the compiler far easier to understand.
The most familiar approach to describing compilers is to dedicate each chapter
to one pass. The problem with that approach is that it obfuscates how language
features motivate design choices in a compiler. We instead take an incremental
approach in which we build a complete compiler in each chapter, starting with
a small input language that includes only arithmetic and variables. We add new
language features in subsequent chapters, extending the compiler as necessary.
Our choice of language features is designed to elicit fundamental concepts and
algorithms used in compilers.
• We begin with integer arithmetic and local variables in chapters 1 and 2, where
we introduce the fundamental tools of compiler construction: abstract syntax trees
and recursive functions.
xii Preface
• In chapter 3 we learn how to use the Lark parser framework to create a parser
for the language of integer arithmetic and local variables. We learn about the
parsing algorithms inside Lark, including Earley and LALR(1).
• In chapter 4 we apply graph coloring to assign variables to machine registers.
• Chapter 5 adds conditional expressions, which motivates an elegant recursive
algorithm for translating them into conditional goto statements.
• Chapter 6 adds loops. This elicits the need for dataflow analysis in the register
allocator.
• Chapter 7 adds heap-allocated tuples, motivating garbage collection.
• Chapter 8 adds functions as first-class values without lexical scoping, similar to
functions in the C programming language (Kernighan and Ritchie 1988). The
reader learns about the procedure call stack and calling conventions and how
they interact with register allocation and garbage collection. The chapter also
describes how to generate efficient tail calls.
• Chapter 9 adds anonymous functions with lexical scoping, that is, lambda
expressions. The reader learns about closure conversion, in which lambdas are
translated into a combination of functions and tuples.
• Chapter 10 adds dynamic typing. Prior to this point the input languages are
statically typed. The reader extends the statically typed language with an Any
type that serves as a target for compiling the dynamically typed language.
• Chapter 11 uses the Any type introduced in chapter 10 to implement a gradually
typed language in which different regions of a program may be static or dynami-
cally typed. The reader implements runtime support for proxies that allow values
to safely move between regions.
• Chapter 12 adds generics with autoboxing, leveraging the Any type and type
casts developed in chapters 10 and 11.
There are many language features that we do not include. Our choices balance the
incidental complexity of a feature versus the fundamental concepts that it exposes.
For example, we include tuples and not records because although they both elicit the
study of heap allocation and garbage collection, records come with more incidental
complexity.
Since 2009, drafts of this book have served as the textbook for sixteen-week
compiler courses for upper-level undergraduates and first-year graduate students at
the University of Colorado and Indiana University. Students come into the course
having learned the basics of programming, data structures and algorithms, and
discrete mathematics. At the beginning of the course, students form groups of two
to four people. The groups complete approximately one chapter every two weeks,
starting with chapter 2 and including chapters according to the students interests
while respecting the dependencies between chapters shown in figure 0.1. Chapter 8
(functions) depends on chapter 7 (tuples) only in the implementation of efficient
tail calls. The last two weeks of the course involve a final project in which students
design and implement a compiler extension of their choosing. The last few chapters
can be used in support of these projects. Many chapters include a challenge problem
that we assign to the graduate students.
Preface xiii
Ch. 1 Preliminaries Ch. 2 Variables Ch. 3 Parsing
Ch. 4 Registers Ch. 5 Conditionals Ch. 6 Loops
Ch. 8 Functions Ch. 7 Tuples Ch. 10 Dynamic
Ch. 9 Lambda Ch. 11 Gradual Typing Ch. 12 Generics
Figure 0.1
Diagram of chapter dependencies.
For compiler courses at universities on the quarter system (about ten weeks in
length), we recommend completing the course through chapter 7 or chapter 8 and
providing some scaffolding code to the students for each compiler pass. The course
can be adapted to emphasize functional languages by skipping chapter 6 (loops)
and including chapter 9 (lambda). The course can be adapted to dynamically typed
languages by including chapter 10.
This book has been used in compiler courses at California Polytechnic State Uni-
versity, Portland State University, Rose–Hulman Institute of Technology, University
of Freiburg, University of Massachusetts Lowell, and the University of Vermont.
This edition of the book uses Python both for the implementation of the compiler
and for the input language, so the reader should be proficient with Python. There
are many excellent resources for learning Python (Lutz 2013; Barry 2016; Sweigart
2019; Matthes 2019).The support code for this book is in the GitHub repository at
the following location:
https://github.com/IUCompilerCourse/
The compiler targets x86 assembly language (Intel 2015), so it is helpful but
not necessary for the reader to have taken a computer systems course (Bryant
and O’Hallaron 2010). We introduce the parts of x86-64 assembly language that
are needed in the compiler. We follow the System V calling conventions (Bryant
and O’Hallaron 2005; Matz et al. 2013), so the assembly code that we gener-
ate works with the runtime system (written in C) when it is compiled using the
GNU C compiler (gcc) on Linux and MacOS operating systems on Intel hardware.
On the Windows operating system, gcc uses the Microsoft x64 calling conven-
tion (Microsoft 2018, 2020). So the assembly code that we generate does not work
with the runtime system on Windows. One workaround is to use a virtual machine
with Linux as the guest operating system.
xiv Preface
Acknowledgments
The tradition of compiler construction at Indiana University goes back to research
and courses on programming languages by Daniel Friedman in the 1970s and 1980s.
One of his students, Kent Dybvig, implemented Chez Scheme (Dybvig 2006), an
efficient, production-quality compiler for Scheme. Throughout the 1990s and 2000s,
Dybvig taught the compiler course and continued the development of Chez Scheme.
The compiler course evolved to incorporate novel pedagogical ideas while also
including elements of real-world compilers. One of Friedman’s ideas was to split
the compiler into many small passes. Another idea, called “the game,” was to test
the code generated by each pass using interpreters.
Dybvig, with help from his students Dipanwita Sarkar and Andrew Keep, devel-
oped infrastructure to support this approach and evolved the course to use even
smaller nanopasses (Sarkar, Waddell, and Dybvig 2004; Keep 2012). Many of the
compiler design decisions in this book are inspired by the assignment descriptions
of Dybvig and Keep (2010). In the mid 2000s, a student of Dybvig named Abdu-
laziz Ghuloum observed that the front-to-back organization of the course made it
difficult for students to understand the rationale for the compiler design. Ghuloum
proposed the incremental approach (Ghuloum 2006) on which this book is based.
I thank the many students who served as teaching assistants for the compiler
course at IU including Carl Factora, Ryan Scott, Cameron Swords, and Chris
Wailes. I thank Andre Kuhlenschmidt for work on the garbage collector and x86
interpreter, Michael Vollmer for work on efficient tail calls, and Michael Vitousek
for help with the first offering of the incremental compiler course at IU.
I thank professors Bor-Yuh Chang, John Clements, Jay McCarthy, Joseph Near,
Ryan Newton, Nate Nystrom, Peter Thiemann, Andrew Tolmach, and Michael
Wollowski for teaching courses based on drafts of this book and for their feedback.
I thank the National Science Foundation for the grants that helped to support this
work: Grant Numbers 1518844, 1763922, and 1814460.
I thank Ronald Garcia for helping me survive Dybvig’s compiler course in the
early 2000s and especially for finding the bug that sent our garbage collector on a
wild goose chase!
Jeremy G. Siek
Bloomington, Indiana
1 Preliminaries
In this chapter we introduce the basic tools needed to implement a compiler. Pro-
grams are typically input by a programmer as text, that is, a sequence of characters.
The program-as-text representation is called concrete syntax. We use concrete syn-
tax to concisely write down and talk about programs. Inside the compiler, we use
abstract syntax trees (ASTs) to represent programs in a way that efficiently sup-
ports the operations that the compiler needs to perform. The process of translating
concrete syntax to abstract syntax is called parsing and is studied in chapter 3. For
now we use the parse function in Python’s ast module to translate from concrete
to abstract syntax.
ASTs can be represented inside the compiler in many different ways, depending
on the programming language used to write the compiler. We use Python classes
and objects to represent ASTs, especially the classes defined in the standard ast
module for the Python source language. We use grammars to define the abstract
syntax of programming languages (section 1.2) and pattern matching to inspect
individual nodes in an AST (section 1.3). We use recursive functions to construct
and deconstruct ASTs (section 1.4). This chapter provides a brief introduction to
these components.
1.1 Abstract Syntax Trees
Compilers use abstract syntax trees to represent programs because they often need
to ask questions such as, for a given part of a program, what kind of language feature
is it? What are its subparts? Consider the program on the left and the diagram
of its AST on the right (1.1). This program is an addition operation that has two
subparts, a input operation and a negation. The negation has another subpart, the
integer constant 8. By using a tree to represent the program, we can easily follow
the links to go from one part of a program to its subparts.
input_int() + -8
+
input_int() -
8 (1.1)
2 Chapter 1
We use the standard terminology for trees to describe ASTs: each rectangle above
is called a node. The arrows connect a node to its children, which are also nodes.
The top-most node is the root. Every node except for the root has a parent (the
node of which it is the child). If a node has no children, it is a leaf node; otherwise
it is an internal node.
We use a Python class for each kind of node. The following is the class definition
for constants (aka literals) from the Python ast module.
class Constant:
def __init__(self, value):
self.value = value
An integer constant node includes just one thing: the integer value. To create an
AST node for the integer 8, we write Constant(8).
eight = Constant(8)
We say that the value created by Constant(8) is an instance of the Constant class.
The following is the class definition for unary operators.
class UnaryOp:
def __init__(self, op, operand):
self.op = op
self.operand = operand
The specific operation is specified by the op parameter. For example, the class USub
is for unary subtraction. (More unary operators are introduced in later chapters.)
To create an AST that negates the number 8, we write the following.
neg_eight = UnaryOp(USub(), eight)
The call to the input_int function is represented by the Call and Name classes.
class Call:
def __init__(self, func, args):
self.func = func
self.args = args
class Name:
def __init__(self, id):
self.id = id
To create an AST node that calls input_int, we write
read = Call(Name('input_int'), [])
Finally, to represent the addition in (1.1), we use the BinOp class for binary
operators.
class BinOp:
def __init__(self, left, op, right):
self.op = op
self.left = left
self.right = right
Preliminaries 3
Similar to UnaryOp, the specific operation is specified by the op parameter, which
for now is just an instance of the Add class. So to create the AST node that adds
negative eight to some user input, we write the following.
ast1_1 = BinOp(read, Add(), neg_eight)
To compile a program such as (1.1), we need to know that the operation associ-
ated with the root node is addition and we need to be able to access its two children.
Python provides pattern matching to support these kinds of queries, as we see in
section 1.3.
We often write down the concrete syntax of a program even when we actually
have in mind the AST, because the concrete syntax is more concise. We recommend
that you always think of programs as abstract syntax trees.
1.2 Grammars
A programming language can be thought of as a set of programs. The set is infinite
(that is, one can always create larger programs), so one cannot simply describe
a language by listing all the programs in the language. Instead we write down a
set of rules, a context-free grammar, for building programs. Grammars are often
used to define the concrete syntax of a language, but they can also be used to
describe the abstract syntax. We write our rules in a variant of Backus-Naur form
(BNF) (Backus et al. 1960; Knuth 1964). As an example, we describe a small
language, named LInt, that consists of integers and arithmetic operations.
The first grammar rule for the abstract syntax of LInt says that an instance of
the Constant class is an expression:
exp ::= Constant(int) (1.2)
Each rule has a left-hand side and a right-hand side. If you have an AST node that
matches the right-hand side, then you can categorize it according to the left-hand
side. Symbols in typewriter font, such as Constant, are terminal symbols and must
literally appear in the program for the rule to be applicable. Our grammars do
not mention white space, that is, delimiter characters like spaces, tabs, and new
lines. White space may be inserted between symbols for disambiguation and to
improve readability. A name such as exp that is defined by the grammar rules is
a nonterminal. The name int is also a nonterminal, but instead of defining it with
a grammar rule, we define it with the following explanation. An int is a sequence
of decimals (0 to 9), possibly starting with – (for negative integers), such that
the sequence of decimals represents an integer in the range –263
to 263
– 1. This
enables the representation of integers using 64 bits, which simplifies several aspects
of compilation. In contrast, integers in Python have unlimited precision, but the
techniques needed to handle unlimited precision fall outside the scope of this book.
The second grammar rule is the input_int operation, which receives an input
integer from the user of the program.
exp ::= Call(Name('input_int'),[]) (1.3)
4 Chapter 1
The third rule categorizes the negation of an exp node as an exp.
exp ::= UnaryOp(USub(),exp) (1.4)
We can apply these rules to categorize the ASTs that are in the LInt language. For
example, by rule (1.2), Constant(8) is an exp, and then by rule (1.4) the following
AST is an exp.
UnaryOp(USub(), Constant(8))
–
8
(1.5)
The next two grammar rules are for addition and subtraction expressions:
exp ::= BinOp(exp,Add(),exp) (1.6)
exp ::= BinOp(exp,Sub(),exp) (1.7)
We can now justify that the AST (1.1) is an exp in LInt. We know that
Call(Name('input_int'),[]) is an exp by rule (1.3), and we have already cat-
egorized UnaryOp(USub(), Constant(8)) as an exp, so we apply rule (1.6) to show
that
BinOp(Call(Name('input_int'),[]),Add(),UnaryOp(USub(),Constant(8)))
is an exp in the LInt language.
If you have an AST for which these rules do not apply, then the AST is not in
LInt. For example, the program input_int() * 8 is not in LInt because there is
no rule for the * operator. Whenever we define a language with a grammar, the
language includes only those programs that are justified by the grammar rules.
The language LInt includes a second nonterminal stmt for statements. There is a
statement for printing the value of an expression
stmt ::= Expr(Call(Name('print'),[exp]))
and a statement that evaluates an expression but ignores the result.
stmt ::= Expr(exp)
The last grammar rule for LInt states that there is a Module node to mark the
top of the whole program:
LInt ::= Module(stmt∗
)
The asterisk ∗ indicates a list of the preceding grammar item, in this case a list of
statements. The Module class is defined as follows:
class Module:
def __init__(self, body):
self.body = body
where body is a list of statements.
Preliminaries 5
exp ::= int | input_int() | - exp | exp + exp | exp - exp | (exp)
stmt ::= print(exp) | exp
LInt ::= stmt∗
Figure 1.1
The concrete syntax of LInt.
exp ::= Constant(int) | Call(Name('input_int'),[])
| UnaryOp(USub(),exp) | BinOp(exp,Add(),exp)
| BinOp(exp,Sub(),exp)
stmt ::= Expr(Call(Name('print'),[exp])) | Expr(exp)
LInt ::= Module(stmt∗
)
Figure 1.2
The abstract syntax of LInt.
It is common to have many grammar rules with the same left-hand side but
different right-hand sides, such as the rules for exp in the grammar of LInt. As
shorthand, a vertical bar can be used to combine several right-hand sides into a
single rule.
The concrete syntax for LInt is shown in figure 1.1 and the abstract syntax for
LInt is shown in figure 1.2. We recommend using the parse function in Python’s
ast module to convert the concrete syntax into an abstract syntax tree.
1.3 Pattern Matching
As mentioned in section 1.1, compilers often need to access the parts of an AST
node. As of version 3.10, Python provides the match feature to access the parts of
a value. Consider the following example:
match ast1_1:
case BinOp(child1, op, child2):
print(op)
In the example above, the match form checks whether the AST (1.1) is a binary
operator and binds its parts to the three pattern variables (child1, op, and child2).
In general, each case consists of a pattern and a body. Patterns are recursively
defined to be one of the following: a pattern variable, a class name followed by
a pattern for each of its constructor’s arguments, or other literals such as strings
or lists. The body of each case may contain arbitrary Python code. The pattern
variables can be used in the body, such as op in print(op).
6 Chapter 1
A match form may contain several clauses, as in the following function leaf that
recognizes when an LInt node is a leaf in the AST. The match proceeds through the
clauses in order, checking whether the pattern can match the input AST. The body
of the first clause that matches is executed. The output of leaf for several ASTs
is shown on the right side of the following:
def leaf(arith):
match arith:
case Constant(n):
return True
case Call(Name('input_int'), []):
return True
case UnaryOp(USub(), e1):
return False
case BinOp(e1, Add(), e2):
return False
case BinOp(e1, Sub(), e2):
return False
print(leaf(Call(Name('input_int'), [])))
print(leaf(UnaryOp(USub(), eight)))
print(leaf(Constant(8)))
True
False
True
When constructing a match expression, we refer to the grammar definition to
identify which nonterminal we are expecting to match against, and then we make
sure that (1) we have one case for each alternative of that nonterminal and (2)
the pattern in each case corresponds to the corresponding right-hand side of a
grammar rule. For the match in the leaf function, we refer to the grammar for LInt
shown in figure 1.2. The exp nonterminal has five alternatives, so the match has five
cases. The pattern in each case corresponds to the right-hand side of a grammar
rule. For example, the pattern BinOp(e1,Add(),e2) corresponds to the right-hand
side BinOp(exp,Add(),exp). When translating from grammars to patterns, replace
nonterminals such as exp with pattern variables of your choice (such as e1 and e2).
1.4 Recursive Functions
Programs are inherently recursive. For example, an expression is often made of
smaller expressions. Thus, the natural way to process an entire program is to use
a recursive function. As a first example of such a recursive function, we define the
function is_exp as shown in figure 1.3, to take an arbitrary value and determine
whether or not it is an expression in LInt. We say that a function is defined by
structural recursion if it is defined using a sequence of match cases that correspond
to a grammar and the body of each case makes a recursive call on each child node.1
We define a second function, named is_stmt, that recognizes whether a value is
1. This principle of structuring code according to the data definition is advocated in the book
How to Design Programs by Felleisen et al. (2001).
Preliminaries 7
def is_exp(e):
match e:
case Constant(n):
return True
case Call(Name('input_int'), []):
return True
case UnaryOp(USub(), e1):
return is_exp(e1)
case BinOp(e1, Add(), e2):
return is_exp(e1) and is_exp(e2)
case BinOp(e1, Sub(), e2):
return is_exp(e1) and is_exp(e2)
case _:
return False
def is_stmt(s):
match s:
case Expr(Call(Name('print'), [e])):
return is_exp(e)
case Expr(e):
return is_exp(e)
case _:
return False
def is_Lint(p):
match p:
case Module(body):
return all([is_stmt(s) for s in body])
case _:
return False
print(is_Lint(Module([Expr(ast1_1)])))
print(is_Lint(Module([Expr(BinOp(read, Sub(),
UnaryOp(Add(), Constant(8))))])))
Figure 1.3
Example of recursive functions for LInt. These functions recognize whether an AST is in LInt.
a LInt statement. Finally, figure 1.3 contains the definition of is_Lint, which
determines whether an AST is a program in LInt. In general, we can write one
recursive function to handle each nonterminal in a grammar. Of the two examples
at the bottom of the figure, the first is in LInt and the second is not.
8 Chapter 1
1.5 Interpreters
The behavior of a program is defined by the specification of the programming
language. For example, the Python language is defined in the Python language ref-
erence (Python Software Foundation 2021b) and the CPython interpreter (Python
Software Foundation 2021a). In this book we use interpreters to specify each lan-
guage that we consider. An interpreter that is designated as the definition of a
language is called a definitional interpreter (Reynolds 1972). We warm up by cre-
ating a definitional interpreter for the LInt language. This interpreter serves as a
second example of structural recursion. The definition of the interp_Lint function
is shown in figure 1.4. The body of the function matches on the Module AST node
and then invokes interp_stmt on each statement in the module. The interp_stmt
function includes a case for each grammar rule of the stmt nonterminal, and it calls
interp_exp on each subexpression. The interp_exp function includes a case for
each grammar rule of the exp nonterminal. We use several auxiliary functions such
as add64 and input_int that are defined in the support code for this book.
Let us consider the result of interpreting a few LInt programs. The following
program adds two integers:
print(10 + 32)
The result is 42, the answer to life, the universe, and everything: 42!2
We wrote
this program in concrete syntax, whereas the parsed abstract syntax is
Module([Expr(Call(Name('print'),
[BinOp(Constant(10), Add(), Constant(32))]))])
The following program demonstrates that expressions may be nested within each
other, in this case nesting several additions and negations.
print(10 + -(12 + 20))
What is the result of this program?
The last feature of the LInt language, the input_int operation, prompts the user
of the program for an integer. Recall that program (1.1) requests an integer input
and then subtracts 8. So, if we run
interp_Lint(Module([Expr(Call(Name('print'), [ast1_1]))]))
and if the input is 50, the result is 42.
We include the input_int operation in LInt so that a clever student cannot
implement a compiler for LInt that simply runs the interpreter during compilation
to obtain the output and then generates the trivial code to produce the output.3
The job of a compiler is to translate a program in one language into a program
in another language so that the output program behaves the same way as the
input program. This idea is depicted in the following diagram. Suppose we have
2. The Hitchhiker’s Guide to the Galaxy by Douglas Adams.
3. Yes, a clever student did this in the first instance of this course!
Preliminaries 9
def interp_exp(e):
match e:
case BinOp(left, Add(), right):
l = interp_exp(left); r = interp_exp(right)
return add64(l, r)
case BinOp(left, Sub(), right):
l = interp_exp(left); r = interp_exp(right)
return sub64(l, r)
case UnaryOp(USub(), v):
return neg64(interp_exp(v))
case Constant(value):
return value
case Call(Name('input_int'), []):
return input_int()
def interp_stmt(s):
match s:
case Expr(Call(Name('print'), [arg])):
print(interp_exp(arg))
case Expr(value):
interp_exp(value)
def interp_Lint(p):
match p:
case Module(body):
for s in body:
interp_stmt(s)
Figure 1.4
Interpreter for the LInt language.
two languages, L1 and L2, and a definitional interpreter for each language. Given a
compiler that translates from language L1 to L2 and given any program P1 in L1,
the compiler must translate it into some program P2 such that interpreting P1 and
P2 on their respective interpreters with same input i yields the same output o.
P1 P2
o
compile
interp_L2(i)
interp_L1(i)
(1.8)
We establish the convention that if running the definitional interpreter on a pro-
gram produces an error, then the meaning of that program is unspecified unless
the exception raised is a TrappedError. A compiler for the language is under no
10 Chapter 1
obligation regarding programs with unspecified behavior; it does not have to pro-
duce an executable, and if it does, that executable can do anything. On the other
hand, if the error is a TrappedError, then the compiler must produce an executable
and it is required to report that an error occurred. To signal an error, exit with a
return code of 255. The interpreters in chapters 10 and 11 and in section 7.9 use
TrappedError.
In the next section we see our first example of a compiler.
1.6 Example Compiler: A Partial Evaluator
In this section we consider a compiler that translates LInt programs into LInt
programs that may be more efficient. The compiler eagerly computes the parts
of the program that do not depend on any inputs, a process known as partial
evaluation (Jones, Gomard, and Sestoft 1993). For example, given the following
program
print(input_int() + -(5 + 3) )
our compiler translates it into the program
print(input_int() + -8)
Figure 1.5 gives the code for a simple partial evaluator for the LInt language. The
output of the partial evaluator is a program in LInt. In figure 1.5, the structural
recursion over exp is captured in the pe_exp function, whereas the code for partially
evaluating the negation and addition operations is factored into three auxiliary
functions: pe_neg, pe_add and pe_sub. The input to these functions is the output
of partially evaluating the children. The pe_neg, pe_add and pe_sub functions
check whether their arguments are integers and if they are, perform the appropriate
arithmetic. Otherwise, they create an AST node for the arithmetic operation.
To gain some confidence that the partial evaluator is correct, we can test whether
it produces programs that produce the same result as the input programs. That is,
we can test whether it satisfies the diagram of (1.8).
Exercise 1.1 Create three programs in the LInt language and test whether partially
evaluating them with pe_Lint and then interpreting them with interp_Lint gives
the same result as directly interpreting them with interp_Lint.
Preliminaries 11
def pe_neg(r):
match r:
case Constant(n):
return Constant(neg64(n))
case _:
return UnaryOp(USub(), r)
def pe_add(r1, r2):
match (r1, r2):
case (Constant(n1), Constant(n2)):
return Constant(add64(n1, n2))
case _:
return BinOp(r1, Add(), r2)
def pe_sub(r1, r2):
match (r1, r2):
case (Constant(n1), Constant(n2)):
return Constant(sub64(n1, n2))
case _:
return BinOp(r1, Sub(), r2)
def pe_exp(e):
match e:
case BinOp(left, Add(), right):
return pe_add(pe_exp(left), pe_exp(right))
case BinOp(left, Sub(), right):
return pe_sub(pe_exp(left), pe_exp(right))
case UnaryOp(USub(), v):
return pe_neg(pe_exp(v))
case Constant(value):
return e
case Call(Name('input_int'), []):
return e
def pe_stmt(s):
match s:
case Expr(Call(Name('print'), [arg])):
return Expr(Call(Name('print'), [pe_exp(arg)]))
case Expr(value):
return Expr(pe_exp(value))
def pe_P_int(p):
match p:
case Module(body):
new_body = [pe_stmt(s) for s in body]
return Module(new_body)
Figure 1.5
A partial evaluator for LInt.
Essentials of Compilation An Incremental Approach in Python 1st Edition Jeremy G. Siek
2 Integers and Variables
This chapter covers compiling a subset of Python to x86-64 assembly code (Intel
2015). The subset, named LVar, includes integer arithmetic and local variables. We
often refer to x86-64 simply as x86. The chapter first describes the LVar language
(section 2.1) and then introduces x86 assembly (section 2.2). Because x86 assembly
language is large, we discuss only the instructions needed for compiling LVar. We
introduce more x86 instructions in subsequent chapters. After introducing LVar and
x86, we reflect on their differences and create a plan to break down the translation
from LVar to x86 into a handful of steps (section 2.3). The rest of the chapter gives
detailed hints regarding each step. We aim to give enough hints that the well-
prepared reader, together with a few friends, can implement a compiler from LVar
to x86 in a short time. To suggest the scale of this first compiler, we note that the
instructor solution for the LVar compiler is approximately 300 lines of code.
2.1 The LVar Language
The LVar language extends the LInt language with variables. The concrete syntax
of the LVar language is defined by the grammar presented in figure 2.1, and the
abstract syntax is presented in figure 2.2. The nonterminal var may be any Python
identifier. As in LInt, input_int is a nullary operator, - is a unary operator, and
+ is a binary operator. Similarly to LInt, the abstract syntax of LVar includes the
Module instance to mark the top of the program. Despite the simplicity of the LVar
language, it is rich enough to exhibit several compilation techniques.
The LVar language includes an assignment statement, which defines a variable for
use in later statements and initializes the variable with the value of an expression.
The abstract syntax for assignment is defined in figure 2.2. The concrete syntax for
assignment is
var = exp
For example, the following program initializes the variable x to 32 and then prints
the result of 10 + x, producing 42.
x = 12 + 20
print(10 + x)
14 Chapter 2
exp ::= int | input_int() | - exp | exp + exp | exp - exp | (exp)
stmt ::= print(exp) | exp
exp ::= var
stmt ::= var = exp
LVar ::= stmt∗
Figure 2.1
The concrete syntax of LVar.
exp ::= Constant(int) | Call(Name('input_int'),[])
| UnaryOp(USub(),exp) | BinOp(exp,Add(),exp)
| BinOp(exp,Sub(),exp)
stmt ::= Expr(Call(Name('print'),[exp])) | Expr(exp)
exp ::= Name(var)
stmt ::= Assign([Name(var)], exp)
LVar ::= Module(stmt∗
)
Figure 2.2
The abstract syntax of LVar.
2.1.1 Extensible Interpreters via Method Overriding
To prepare for discussing the interpreter of LVar, we explain why we implement it
in an object-oriented style. Throughout this book we define many interpreters, one
for each language that we study. Because each language builds on the prior one,
there is a lot of commonality between these interpreters. We want to write down the
common parts just once instead of many times. A naive interpreter for LVar would
handle the case for variables but dispatch to an interpreter for LInt in the rest of
the cases. The following code sketches this idea. (We explain the env parameter in
section 2.1.2.)
def interp_Lint(e, env):
match e:
case UnaryOp(USub(), e1):
return - interp_Lint(e1, env)
...
def interp_Lvar(e, env):
match e:
case Name(id):
return env[id]
case _:
return interp_Lint(e, env)
The problem with this naive approach is that it does not handle situations in which
an LVar feature, such as a variable, is nested inside an LInt feature, such as the -
operator, as in the following program.
y = 10
print(-y)
Integers and Variables 15
If we invoke interp_Lvar on this program, it dispatches to interp_Lint to handle
the - operator, but then it recursively calls interp_Lint again on its argument.
Because there is no case for Name in interp_Lint, we get an error!
To make our interpreters extensible we need something called open recursion, in
which the tying of the recursive knot is delayed until the functions are composed.
Object-oriented languages provide open recursion via method overriding. The fol-
lowing code uses method overriding to interpret LInt and LVar using Python class
definitions. We define one class for each language and define a method for inter-
preting expressions inside each class. The class for LVar inherits from the class for
LInt, and the method interp_exp in LVar overrides the interp_exp in LInt. Note
that the default case of interp_exp in LVar uses super to invoke interp_exp, and
because LVar inherits from LInt, that dispatches to the interp_exp in LInt.
class InterpLint:
def interp_exp(e):
match e:
case UnaryOp(USub(), e1):
return neg64(self.interp_exp(e1))
...
...
def InterpLvar(InterpLint):
def interp_exp(e):
match e:
case Name(id):
return env[id]
case _:
return super().interp_exp(e)
...
We return to the troublesome example, repeated here:
y = 10
print(-y)
We can invoke the interp_exp method for LVar on the -y expression, which we call
e0, by creating an object of the LVar class and calling the interp_exp method
InterpLvar().interp_exp(e0)
To process the - operator, the default case of interp_exp in LVar dispatches to the
interp_exp method in LInt. But then for the recursive method call, it dispatches
to interp_exp in LVar, where the Name node is handled correctly. Thus, method
overriding gives us the open recursion that we need to implement our interpreters
in an extensible way.
2.1.2 Definitional Interpreter for LVar
Having justified the use of classes and methods to implement interpreters, we revisit
the definitional interpreter for LInt shown in figure 2.3 and then extend it to create
an interpreter for LVar, shown in figure 2.4. We change the interp_stmt method in
the interpreter for LInt to take two extra parameters named env, which we discuss
in the next paragraph, and cont for continuation, which is the technical name for
what comes after a particular point in a program. The cont parameter is the list
of statements that follow the current statement. Note that interp_stmts invokes
interp_stmt on the first statement and passes the rest of the statements as the
argument for cont. This organization enables each statement to decide what if
16 Chapter 2
anything should be evaluated after it, for example, allowing a return statement to
exit early from a function (see Chapter 8).
The interpreter for LVar adds two new cases for variables and assignment. For
assignment, we need a way to communicate the value bound to a variable to all the
uses of the variable. To accomplish this, we maintain a mapping from variables to
values called an environment. We use a Python dictionary to represent the environ-
ment. The interp_exp function takes the current environment, env, as an extra
parameter. When the interpreter encounters a variable, it looks up the correspond-
ing value in the environment. If the variable is not in the environment (because
the variable was not defined) then the lookup will fail and the interpreter will halt
with an error. Recall that the compiler is not obligated to compile such programs
(Section 1.5).1
When the interpreter encounters an assignment, it evaluates the
initializing expression and then associates the resulting value with the variable in
the environment.
The goal for this chapter is to implement a compiler that translates any program
P1 written in the LVar language into an x86 assembly program P2 such that P2
exhibits the same behavior when run on a computer as the P1 program interpreted
by interp_Lvar. That is, they output the same integer n. We depict this correctness
criteria in the following diagram:
P1 P2
n
compile
interp_Lvar interp_x86int
Next we introduce the x86Int subset of x86 that suffices for compiling LVar.
2.2 The x86Int Assembly Language
Figure 2.5 defines the concrete syntax for x86Int. We use the AT&T syntax expected
by the GNU assembler. A program begins with a main label followed by a sequence
of instructions. The globl directive makes the main procedure externally visible so
that the operating system can call it. An x86 program is stored in the computer’s
memory. For our purposes, the computer’s memory is a mapping of 64-bit addresses
to 64-bit values. The computer has a program counter (PC) stored in the rip
register that points to the address of the next instruction to be executed. For most
instructions, the program counter is incremented after the instruction is executed
so that it points to the next instruction in memory. Most x86 instructions take
two operands, each of which is an integer constant (called an immediate value), a
register, or a memory location.
1. In Chapter 5 we introduce type checking rules that prohibit access to undefined variables.
Integers and Variables 17
class InterpLint:
def interp_exp(self, e, env):
match e:
case BinOp(left, Add(), right):
l = self.interp_exp(left, env)
r = self.interp_exp(right, env)
return add64(l, r)
case BinOp(left, Sub(), right):
l = self.interp_exp(left, env)
r = self.interp_exp(right, env)
return sub64(l, r)
case UnaryOp(USub(), v):
return neg64(self.interp_exp(v, env))
case Constant(value):
return value
case Call(Name('input_int'), []):
return int(input())
def interp_stmt(self, s, env, cont):
match s:
case Expr(Call(Name('print'), [arg])):
val = self.interp_exp(arg, env)
print(val, end='')
return self.interp_stmts(cont, env)
case Expr(value):
self.interp_exp(value, env)
return self.interp_stmts(cont, env)
case _:
raise Exception('error in interp_stmt, unexpected ' + repr(s))
def interp_stmts(self, ss, env):
match ss:
case []:
return 0
case [s, *ss]:
return self.interp_stmt(s, env, ss)
def interp(self, p):
match p:
case Module(body):
self.interp_stmts(body, {})
def interp_Lint(p):
return InterpLint().interp(p)
Figure 2.3
Interpreter for LInt as a class.
18 Chapter 2
class InterpLvar(InterpLint):
def interp_exp(self, e, env):
match e:
case Name(id):
return env[id]
case _:
return super().interp_exp(e, env)
def interp_stmt(self, s, env, cont):
match s:
case Assign([Name(id)], value):
env[id] = self.interp_exp(value, env)
return self.interp_stmts(cont, env)
case _:
return super().interp_stmt(s, env, cont)
def interp_Lvar(p):
return InterpLvar().interp(p)
Figure 2.4
Interpreter for the LVar language.
reg ::= rsp | rbp | rax | rbx | rcx | rdx | rsi | rdi |
r8 | r9 | r10 | r11 | r12 | r13 | r14 | r15
arg ::= $int | %reg | int(%reg)
instr ::= addq arg,arg | subq arg,arg | negq arg | movq arg,arg |
callq label | pushq arg | popq arg | retq
x86Int ::= .globl main
main: instr∗
Figure 2.5
The syntax of the x86Int assembly language (AT&T syntax).
A register is a special kind of variable that holds a 64-bit value. There are 16
general-purpose registers in the computer; their names are given in figure 2.5. A
register is written with a percent sign, %, followed by its name, for example, %rax.
An immediate value is written using the notation $n where n is an integer. An
access to memory is specified using the syntax n(%r), which obtains the address
stored in register r and then adds n bytes to the address. The resulting address is
used to load or to store to memory depending on whether it occurs as a source or
destination argument of an instruction.
An arithmetic instruction such as addq s, d reads from the source s and des-
tination d, applies the arithmetic operation, and then writes the result to the
Integers and Variables 19
.globl main
main:
movq $10, %rax
addq $32, %rax
retq
Figure 2.6
An x86 program that computes 10 + 32.
destination d. The move instruction movq s, d reads from s and stores the result
in d. The callq label instruction jumps to the procedure specified by the label, and
retq returns from a procedure to its caller. We discuss procedure calls in more
detail further in this chapter and in chapter 8. The last letter q indicates that these
instructions operate on quadwords, which are 64-bit values.
Appendix A.1 contains a reference for all the x86 instructions used in this book.
Figure 2.6 depicts an x86 program that computes 10 + 32. The instruction
movq $10, %rax puts 10 into register rax, and then addq $32, %rax adds 32 to the
10 in rax and puts the result, 42, into rax. The last instruction retq finishes the
main function by returning the integer in rax to the operating system. The oper-
ating system interprets this integer as the program’s exit code. By convention, an
exit code of 0 indicates that a program has completed successfully, and all other
exit codes indicate various errors.
We exhibit the use of memory for storing intermediate results in the next example.
Figure 2.7 lists an x86 program that computes 52 + -10. This program uses a region
of memory called the procedure call stack (stack for short). The stack consists of a
separate frame for each procedure call. The memory layout for an individual frame
is shown in figure 2.8. The register rsp is called the stack pointer and contains the
address of the item at the top of the stack. In general, we use the term pointer
for something that contains an address. The stack grows downward in memory,
so we increase the size of the stack by subtracting from the stack pointer. In the
context of a procedure call, the return address is the location of the instruction
that immediately follows the call instruction on the caller side. The function call
instruction, callq, pushes the return address onto the stack prior to jumping to
the procedure. The register rbp is the base pointer and is used to access variables
that are stored in the frame of the current procedure call. The base pointer of the
caller is stored immediately after the return address. Figure 2.8 shows the memory
layout of a frame with storage for n variables, which are numbered from 1 to n.
Variable 1 is stored at address –8(%rbp), variable 2 at –16(%rbp), and so on.
In the program shown in figure 2.7, consider how control is transferred from the
operating system to the main function. The operating system issues a callq main
instruction that pushes its return address on the stack and then jumps to main. In
x86-64, the stack pointer rsp must be divisible by 16 bytes prior to the execution
of any callq instruction, so that when control arrives at main, the rsp is 8 bytes
20 Chapter 2
.globl main
main:
pushq %rbp
movq %rsp, %rbp
subq $16, %rsp
movq $10, -8(%rbp)
negq -8(%rbp)
movq -8(%rbp), %rax
addq $52, %rax
addq $16, %rsp
popq %rbp
retq
Figure 2.7
An x86 program that computes 52 + -10.
Position Contents
8(%rbp) return address
0(%rbp) old rbp
–8(%rbp) variable 1
–16(%rbp) variable 2
… …
0(%rsp) variable n
Figure 2.8
Memory layout of a frame.
out of alignment (because the callq pushed the return address). The first three
instructions are the typical prelude for a procedure. The instruction pushq %rbp
first subtracts 8 from the stack pointer rsp and then saves the base pointer of the
caller at address rsp on the stack. The next instruction movq %rsp, %rbp sets the
base pointer to the current stack pointer, which is pointing to the location of the
old base pointer. The instruction subq $16, %rsp moves the stack pointer down to
make enough room for storing variables. This program needs one variable (8 bytes),
but we round up to 16 bytes so that rsp is 16-byte-aligned, and then we are ready
to make calls to other functions.
The first instruction after the prelude is movq $10, -8(%rbp), which stores 10
in variable 1. The instruction negq -8(%rbp) changes the contents of variable 1
to –10. The next instruction moves the –10 from variable 1 into the rax register.
Finally, addq $52, %rax adds 52 to the value in rax, updating its contents to 42.
The conclusion of the main function consists of the last three instructions. The
first two restore the rsp and rbp registers to their states at the beginning of the
procedure. In particular, addq $16, %rsp moves the stack pointer to point to the
old base pointer. Then popq %rbp restores the old base pointer to rbp and adds 8
Integers and Variables 21
reg ::= 'rsp' | 'rbp' | 'rax' | 'rbx' | 'rcx' | 'rdx' | 'rsi' | 'rdi' |
'r8' | 'r9' | 'r10' | 'r11' | 'r12' | 'r13' | 'r14' | 'r15'
arg ::= Immediate(int) | Reg(reg) | Deref(reg,int)
instr ::= Instr('addq',[arg,arg]) | Instr('subq',[arg,arg])
| Instr('movq',[arg,arg]) | Instr('negq',[arg])
| Instr('pushq',[arg]) | Instr('popq',[arg])
| Callq(label,int) | Retq() | Jump(label)
x86Int ::= X86Program(instr∗
)
Figure 2.9
The abstract syntax of x86Int assembly.
to the stack pointer. The last instruction, retq, jumps back to the procedure that
called this one and adds 8 to the stack pointer.
Our compiler needs a convenient representation for manipulating x86 programs,
so we define an abstract syntax for x86, shown in figure 2.9. We refer to this lan-
guage as x86Int. The main difference between this and the concrete syntax of x86Int
(figure 2.5) is that labels, instruction names, and register names are explicitly rep-
resented by strings. Regarding the abstract syntax for callq, the Callq AST node
includes an integer for representing the arity of the function, that is, the number
of arguments, which is helpful to know during register allocation (chapter 4).
2.3 Planning the Trip to x86
To compile one language to another, it helps to focus on the differences between
the two languages because the compiler will need to bridge those differences. What
are the differences between LVar and x86 assembly? Here are some of the most
important ones:
1. x86 arithmetic instructions typically have two arguments and update the second
argument in place. In contrast, LVar arithmetic operations take two arguments
and produce a new value. An x86 instruction may have at most one memory-
accessing argument. Furthermore, some x86 instructions place special restrictions
on their arguments.
2. An argument of an LVar operator can be a deeply nested expression, whereas
x86 instructions restrict their arguments to be integer constants, registers, and
memory locations.
3. A program in LVar can have any number of variables, whereas x86 has 16 registers
and the procedure call stack.
We ease the challenge of compiling from LVar to x86 by breaking down the problem
into several steps, which deal with these differences one at a time. Each of these steps
is called a pass of the compiler. This term indicates that each step passes over, or
traverses, the AST of the program. Furthermore, we follow the nanopass approach,
22 Chapter 2
which means that we strive for each pass to accomplish one clear objective rather
than two or three at the same time. We begin by sketching how we might implement
each pass and give each pass a name. We then figure out an ordering of the passes
and the input/output language for each pass. The very first pass has LVar as its
input language, and the last pass has x86Int as its output language. In between these
two passes, we can choose whichever language is most convenient for expressing the
output of each pass, whether that be LVar, x86Int, or a new intermediate language
of our own design. Finally, to implement each pass we write one recursive function
per nonterminal in the grammar of the input language of the pass.
Our compiler for LVar consists of the following passes:
remove_complex_operands ensures that each subexpression of a primitive opera-
tion or function call is a variable or integer, that is, an atomic expression. We refer
to nonatomic expressions as complex. This pass introduces temporary variables
to hold the results of complex subexpressions.
select_instructions handles the difference between LVar operations and x86
instructions. This pass converts each LVar operation to a short sequence of
instructions that accomplishes the same task.
assign_homes replaces variables with registers or stack locations.
The next question is, in what order should we apply these passes? This question
can be challenging because it is difficult to know ahead of time which orderings will
be better (that is, will be easier to implement, produce more efficient code, and so
on), and therefore ordering often involves trial and error. Nevertheless, we can plan
ahead and make educated choices regarding the ordering.
The select_instructions and assign_homes passes are intertwined. In chap-
ter 8 we learn that in x86, registers are used for passing arguments to functions
and that it is preferable to assign parameters to their corresponding registers. This
suggests that it would be better to start with the select_instructions pass,
which generates the instructions for argument passing, before performing register
allocation. On the other hand, by selecting instructions first we may run into a
dead end in assign_homes. Recall that only one argument of an x86 instruction
may be a memory access, but assign_homes might be forced to assign both argu-
ments to memory locations. A sophisticated approach is to repeat the two passes
until a solution is found. However, to reduce implementation complexity we rec-
ommend placing select_instructions first, followed by the assign_homes, and
then a third pass named patch_instructions that uses a reserved register to fix
outstanding problems.
Figure 2.10 presents the ordering of the compiler passes and identifies the input
and output language of each pass. The output of the select_instructions pass is
the x86Var language, which extends x86Int with an unbounded number of program-
scope variables and removes the restrictions regarding instruction arguments. The
last pass, prelude_and_conclusion, places the program instructions inside a main
function with instructions for the prelude and conclusion. The remainder of this
chapter provides guidance on the implementation of each of the compiler passes
represented in figure 2.10.
Integers and Variables 23
LVar Lmon
Var
x86Var x86Var x86Int x86Int
remove_complex_operands
select_instructions
assign_homes
patch_instructions
prelude_and_conclusion
Figure 2.10
Diagram of the passes for compiling LVar.
atm ::= Constant(int) | Name(var)
exp ::= atm | Call(Name('input_int'),[])
| UnaryOp(USub(),atm) | BinOp(atm,Add(),atm)
| BinOp(atm,Sub(),atm)
stmt ::= Expr(Call(Name('print'),[atm])) | Expr(exp)
| Assign([Name(var)], exp)
Lmon
Var ::= Module(stmt∗
)
Figure 2.11
Lmon
Var
is LVar with operands restricted to atomic expressions.
2.4 Remove Complex Operands
The remove_complex_operands pass compiles LVar programs into a restricted form
in which the arguments of operations are atomic expressions. Put another way,
this pass removes complex operands, such as the expression -10 in the following
program. This is accomplished by introducing a new temporary variable, assigning
the complex operand to the new variable, and then using the new variable in place
of the complex operand, as shown in the output of remove_complex_operands on
the right.
x = 42 + -10
print(x + 10)
⇒
tmp_0 = -10
x = 42 + tmp_0
tmp_1 = x + 10
print(tmp_1)
Figure 2.11 presents the grammar for the output of this pass, the language Lmon
Var .
The only difference is that operator arguments are restricted to be atomic expres-
sions that are defined by the atm nonterminal. In particular, integer constants and
variables are atomic.
The atomic expressions are pure (they do not cause or depend on
side effects) whereas complex expressions may have side effects, such as
24 Chapter 2
Call(Name('input_int'),[]). A language with this separation between pure
expressions versus expressions with side effects is said to be in monadic normal
form (Moggi 1991; Danvy 2003), which explains the mon in the name Lmon
Var . An
important invariant of the remove_complex_operands pass is that the relative
ordering among complex expressions is not changed, but the relative ordering
between atomic expressions and complex expressions can change and often does.
These changes are behavior preserving because atomic expressions are pure.
We recommend implementing this pass with an auxiliary method named rco_exp
with two parameters: an LVar expression and a Boolean that specifies whether the
expression needs to become atomic or not. The rco_exp method should return a
pair consisting of the new expression and a list of pairs, associating new temporary
variables with their initializing expressions.
Returning to the example program with the expression 42 + -10, the subexpres-
sion -10 should be processed using the rco_exp function with True as the second
argument, because -10 is an argument of the + operator and therefore needs to
become atomic. The output of rco_exp applied to -10 is as follows.
-10 ⇒
tmp_1
[(tmp_1, -10)]
Take special care of programs, such as the following, that assign an atomic
expression to a variable. You should leave such assignments unchanged, as shown
in the program on the right:
a = 42
b = a
print(b)
⇒
a = 42
b = a
print(b)
A careless implementation might produce the following output with unnecessary
temporary variables.
tmp_1 = 42
a = tmp_1
tmp_2 = a
b = tmp_2
print(b)
Exercise 2.1 Implement the remove_complex_operands pass in compiler.py, cre-
ating auxiliary functions for each nonterminal in the grammar, that is, rco_exp
and rco_stmt. We recommend that you use the function utils.generate_name()
to generate fresh names from a stub string.
Exercise 2.2 Create five LVar programs that exercise the most interesting parts of
the remove_complex_operands pass. The five programs should be placed in the
subdirectory tests/var, and the file names should end with the file extension .py.
Integers and Variables 25
Run the run-tests.py script in the support code to check whether the output
programs produce the same result as the input programs.
2.5 Select Instructions
In the select_instructions pass we begin the work of translating to x86Var.
The target language of this pass is a variant of x86 that still uses variables, so we
add an AST node of the form Variable(var) to the arg nonterminal of the x86Int
abstract syntax (figure 2.9). We recommend implementing an auxiliary function
named select_stmt for the stmt nonterminal.
Next consider the cases for the stmt nonterminal, starting with arithmetic opera-
tions. For example, consider the following addition operation, on the left side. (Let
arg1 and arg2 be the translations of atm1 and atm2, respectively.) There is an addq
instruction in x86, but it performs an in-place update. So, we could move arg1 into
the rax register, then add arg2 to rax, and then finally move rax into var.
var = atm1 + atm2 ⇒
movq arg1, %rax
addq arg2, %rax
movq %rax, var
However, with some care we can generate shorter sequences of instructions. Suppose
that one or more of the arguments of the addition is the same variable as the left-
hand side of the assignment. Then the assignment statement can be translated into
a single addq instruction, as follows.
var = atm1 + var ⇒ addq arg1, var
On the other hand, if atm2 is not the same variable as the left-hand side, then we
can move arg1 into the left-hand var and then add arg2 to var.
var = atm1 + atm2 ⇒
movq arg1, var
addq arg2, var
The input_int operation does not have a direct counterpart in x86 assembly,
so we provide this functionality with the function read_int in the file runtime.c,
written in C (Kernighan and Ritchie 1988). In general, we refer to all the func-
tionality in this file as the runtime system, or simply the runtime for short. When
compiling your generated x86 assembly code, you need to compile runtime.c to
runtime.o (an object file, using gcc with option -c) and link it into the executable.
For our purposes of code generation, all you need to do is translate an assignment
of input_int into a call to the read_int function followed by a move from rax to
the left-hand side variable. (The return value of a function is placed in rax.)
var = input_int(); ⇒
callq read_int
movq %rax, var
26 Chapter 2
Similarly, we translate the print operation, shown below, into a call to the
print_int function defined in runtime.c. In x86, the first six arguments to func-
tions are passed in registers, with the first argument passed in register rdi. So we
move the arg into rdi and then call print_int using the callq instruction.
print(atm) ⇒
movq arg, %rdi
callq print_int
We recommend that you use the function utils.label_name to transform strings
into labels, for example, in the target of the callq instruction. This practice makes
your compiler portable across Linux and Mac OS X, which requires an underscore
prefixed to all labels.
Exercise 2.3 Implement the select_instructions pass in compiler.py. Create
three new example programs that are designed to exercise all the interesting cases
in this pass. Run the run-tests.py script to check whether the output programs
produce the same result as the input programs.
2.6 Assign Homes
The assign_homes pass compiles x86Var programs to x86Var programs that no longer
use program variables. Thus, the assign_homes pass is responsible for placing all
the program variables in registers or on the stack. For runtime efficiency, it is better
to place variables in registers, but because there are only sixteen registers, some
programs must necessarily resort to placing some variables on the stack. In this
chapter we focus on the mechanics of placing variables on the stack. We study an
algorithm for placing variables in registers in chapter 4.
Consider again the following LVar program from section 2.4:
a = 42
b = a
print(b)
The output of select_instructions is shown next, on the left, and the output
of assign_homes is on the right. In this example, we assign variable a to stack
location -8(%rbp) and variable b to location -16(%rbp).
movq $42, a
movq a, b
movq b, %rax
⇒
movq $42, -8(%rbp)
movq -8(%rbp), -16(%rbp)
movq -16(%rbp), %rax
The assign_homes pass should replace all uses of variables with stack locations.
In the process of assigning variables to stack locations, it is convenient for you to
compute and store the size of the frame (in bytes) in the field stack_space of
the X86Program node, which is needed later to generate the conclusion of the main
procedure. The x86-64 standard requires the frame size to be a multiple of 16 bytes.
Random documents with unrelated
content Scribd suggests to you:
A
Scotland under Charles the Second.
t the death of Cromwell there was not, in the general aspect of
political matters, any definite forecast of what twelve months
after would be the form of government; certainly an easy and
unopposed restoration of the Stuart monarchy was about the last
idea, warranted by the history of the previous fifteen years. But one
man, the still-tongued, close-minded General Monk, solved the
question. By his influence as head of the army, and his tact and
sagacity in party wire-pulling, he so managed that within eight
months of the Protector’s death, Charles II. was quietly proclaimed
King of Great Britain and Ireland. It was a twenty-seven years of as
mean rule, as has ever darkened the pages of British history.
Retaliations and persecutions—one long attempt to turn back the
stream of progress—a corrupt court, leavening the national life with
foulness and frivolity, such might be the general headings of the
chapters chronicling the reign of the “Merry Monarch.”
The restoration was in England baptized in blood. Ten “regicides”
were hanged at Charing Cross. This was harsh—revengeful; but not
despicable or unprecedented. But it is with disgust, with shame for
our common humanity, that we learn that the bodies of Cromwell,
Ireton, and Bradshaw were taken from their graves in Westminster
Abbey, and on the death anniversary (30th January) of “King Charles
the Martyr,” drawn on hurdles to Tyburn, and there hung on the
gallows; then the heads cut off and fixed on Westminster Hall.
And Scotland must not be left without examples of severity. The
Marquis of Argyle was the first victim. At the coronation of Charles at
Scone, he was the noble who placed the crown on the king’s head.
But Charles hated him as a leader of the presbyterians, who then
held him in irksome tutelage. After a most unfair trial, nothing
tangible being found against him except some private letters to
General Monk, in which he expressed himself favourable to
Cromwell, he was found guilty, and condemned to death. He met his
fate with great firmness, saying that if he could not brave death like
a Roman, he could submit to it like a Christian.
Other victims followed. Swinburne has said of Mary of Scotland, “A
kinder or more faithful friend, a deadlier or more dangerous enemy,
it would be impossible to dread or to desire.” Mary’s descendants
were noways remarkable for fidelity in friendship, but they were
implacable in their hatreds. When he was in the over-careful hands
of the Covenanters, Charles had treasured up against a day of
vengeance, many affronts, brow-beatings, and intimidations, and
now he meant, in his stubborn way, to demand payment, with heavy
interest, of the old debts.
And so Charles, the Covenanted King of Scotland, and in whose
cause its best blood had been shed, had nothing but hatred for the
land of his fathers, and for its presbyterian faith. A packed and
subservient Scottish Parliament proceeded to pass, first a Rescissory
Act, rescinding all statutes, good and bad, which had been passed
since the commencement of the civil wars; and next, an Act of
Supremacy, making the king supreme judge in all matters, both civil
and ecclesiastical. Charles soon made it evident that he meant to
establish episcopacy. James Sharpe, minister of the little Fifeshire
town of Crail, was sent to London to look after presbyterian
interests; he was got at on the selfish side, and made archbishop of
St. Andrews. Nine other pliant Scottish ministers received episcopal
ordination in Westminster Abbey.
On the third anniversary of the Restoration, 29th May, 1662,
copies of the Covenants were in Edinburgh publicly torn to pieces by
the common hangman. The ministers were ordered to attend
diocesan meetings, and to acknowledge the authority of their
bishops. The majority acquiesced; but it is pleasing to learn that
nearly four hundred resigned their livings, rather than submit to the
prelatic yoke. To take the places of the recusants, a hosts of curates,
often persons of mean character and culture, were ordained. The
people did not like the men thus thrust upon them as ministers, and
they still sought the services of their old pastors; hence originated
the “conventicles,” a contemptuous title for a meeting-place of
dissenters.
And now began, chiefly in the west and south of Scotland, those
field meetings which afterwards became so notable. At first they
were simply assemblies for worship, no arms were worn; after
service a quiet dispersal. But, as signifying nonconformity to
prescribed forms, they gave great offence. A new Act forbade, under
punishment for sedition, any preaching without the sanction of the
bishops; and inflicting pains and penalties on all persons absenting
themselves from their parish churches. If fines were not paid,
soldiers were quartered on the recusants, and their cattle, furniture,
and very clothing were sold. It was even accounted seditious to give
sustenance to the ejected ministers.
It can be easily asked, why did this Scottish people, with the
memory of their past, submit to these things? There was, as in
England, a reaction to an extreme of loyalty; there was the
satisfaction of finding themselves freed from English domination in
its tangible form of Cromwell’s troops and garrisons; there was the
pleasure of once more seeing a Parliament in Edinburgh, even
though it merely registered and gave legal form to the king’s
decrees. They were told that the advantage of being governed by
their own native prince implied as its price the establishment of that
prince’s form of religious faith. Their own nobles and many of their
ministers had conformed; and thus bereft of their natural leaders,
there was weakness and division. Despite of all these
discouragements, they were often goaded into insurrections; which
were cruelly suppressed, and made the excuses for further
intolerance, and still harsher persecutions.
The field conventicles continued. In the solitudes of nature, in
lonely glens, or on pine-shaded hillsides, with sentinels posted on
the heights, arose the solemn psalm, and the preachers prayer and
exhortation. And men now came armed to these gatherings, the
women had to be defended, force was to be met by force. To
suppress such meetings, troops were sent into the insubordinate
districts, under a wild fanatical Royalist, General Dalziel, and had
free quarters on the inhabitants. By 1666, a reign of terror was fully
inaugurated; Dalziel flared like a baleful meteor over the West of
Scotland. In November of this year, without concert or
premeditation, an open insurrection broke out. At Dalry, in Ayrshire,
four soldiers were grossly maltreating an aged man, and common
humanity could not stand by and look on with indifference or mere
sympathy. The people rescued the old man, disarmed the soldiers,
and took their officer prisoner to Dumfries. A resolution was
suddenly taken to march on Edinburgh. They gathered in a
fortnight’s march to barely 2000 men, and wearied and worn out,
encamped on a plateau, called Rullion Green, on the Pentland hills, a
few miles south of Edinburgh. Here they were attacked by double
their numbers under Dalziel, and, after a gallant resistance,
considering their inferior arms and discipline, were put to flight.
Some fifty were killed on the field, one hundred and thirty were
taken prisoners, thirty-four of whom were, chiefly at the instigation
of Archbishop Sharpe, hanged as rebels, and the rest banished.
THUMBIKINS.
(From the Scottish Antiquarian Museum.)
And tortures—such as have had no place in modern history since
the palmy days of the Spanish Inquisition were inflicted to extort
confessions of complicity in a rising, which was really the offspring of
momentary excitement. Thumbikins squeezed the fingers by iron
screws. These tortures were generally borne with heroic patience
and resolution. One young minister, Hugh McKail, comely in person,
well educated, an enthusiast in his covenanting faith, was subjected
to the torture of the boot. His leg was crushed, but he uttered no
cry, only moving his lips in silent prayer. He had taken very little part
in the insurrection, but was condemned to death. On the gallows-
ladder his last words were:—“Farewell father, mother, and all my
friends in life, farewell earth and all its delights, farewell sun, moon,
and stars, welcome death, glory, and eternal life.” Seeing what
impressions such words made on the listeners, in after executions
drums were beaten to drown the voices of the sufferers.
A weary ten years ensued of alternate “indulgence,” and renewed
intolerance. In 1667, the Duke of Lauderdale was placed at the head
of Scottish affairs. He had subscribed to the covenant, and had been
a Presbyterian representative at the Westminster Assembly. He was
now a subservient courtier, but did not at first assume the role of a
persecutor. He disbanded the army, and proclaimed an indemnity to
those who had fought at Rullion Green, on their signing a bond of
peace. The ministers ousted from their parishes were permitted to
return, but on conditions which the strict consciences of many could
not accept; and those who did accept were placed under close
surveillance, and under severe penalties forbidden to take part in
any field meetings. Some of the bishops were good men, striving
earnestly to make peace within the church. One of these, Leighton,
Bishop of Dunblane, made an attempt to reconcile Presbyterianism
with a modified episcopacy. The bishops were merely to sit as
chairmen, or moderators, in the diocesan convocations, and to have
no veto on the proceedings, but the Covenanters thought this a
snare for entrapping them into an acknowledgment of prelacy, and
the idea was abandoned.
And Lauderdale who had begun his rule leniently, now afraid of
being represented to the King as lukewarm in his service, blossomed
out into a cruel persecutor, forcibly suppressing field meetings, and
enforcing extreme penalties on nonconformists. It has been
estimated that up to this date seventeen thousand persons had
suffered in fine, imprisonment, and death. It was said that fines
extorted for non-attendance at the parish churches, were applied to
supply the extravagance of Lady Lauderdale,—a rapacious, bad,
clever woman. Landowners were required under penalties to become
bound for their tenants, that they would attend their parish
churches, take no part in conventicles, and not relieve outlawed
persons.
The gentry generally refused to enter into such bonds; and
Lauderdale wrote to the King that the country was in a state of
incipient rebellion, and required reduction by force of arms. He
treated the whole of the west country as if in open revolt. Not only
did he send ordinary troops with field artillery into the devoted
districts, but he brought down from the hills a Highland host of 9000
men to live upon, and with every encouragement to plunder and
oppress, the poor people. Speaking an unknown tongue, strange in
manners and attire, they were to the lowlanders a veritable plague
of human locusts. When, after a few months of free quarterage,
they went back to their hills, themselves and a number of horses
were loaded with booty, as if from the sack of a rich town. But so far
as we can learn they were not guilty of personal violence upon those
they were sent to despoil; perhaps in this respect hardly coming up
to the wishes and expectations of their employers.
In May, 1679, occurred a deed of blood which widened the gulf
between the Covenanters and the government, and gave legal
colouring to harshness and persecution. In Fifeshire, one Carmichael
had become especially obnoxious as a cruel persecutor, and an
active commissioner for receiving the fines laid upon the
malcontents. On 3rd May, a party of twelve men, chiefly small
farmers in the district, with David Hackston of Rathillet and John
Balfour of Burley as the leaders, lay in wait for Carmichael, with full
purpose to slay him. It appears he had received some warning, and
kept out of the way. After waiting long, the band were, in sullen
disappointment, preparing to separate, when the carriage of Sharpe,
the Archbishop, appeared unexpectedly, conveying him and his
daughter home to St. Andrews. To these superstitious men, nursed
under persecution by old biblical texts into religious fanaticism, it
appeared as if an act of necessary vengeance was here thrust upon
them, that instead of an inferior agent, a foremost persecutor, who
had hounded to the death many of their brethren, was now
delivered into their hands. They took him from his carriage, and
there on Magus Muir—suing upon his knees for mercy, his grey
hairs, and his daughter’s anguished cries, also pleading for his life—
they slew him with many sword thrusts.
A general cry of horror and repudiation rang through the land. It
was a savage murder; but so had been the deaths of hundreds of
persons more innocent than he of offences against justice and
common right. More severe measures of repression were taken; new
troops were raised, and the officers instructed to act with the utmost
rigour. And the Covenanters grew desperate; they assembled in
greater numbers, were more fully armed, and more defiant in their
language. On 29th May, the anniversary of the Restoration, a
mounted party entered the village of Rutherglen, about two miles
from Glasgow. They extinguished the festive bonfire, held a service
of denunciatory psalms, prayers, and exhortations in the market
place, and burned the Acts which had been issued against the
Covenant. In quest of the insurgents, and to avenge the affront on
the government, a body of cavalry rode out of Glasgow barracks, on
the 1st of June. Their leader was a distinguished soldier—a man of
courage and gallant bearing, John Graham of Claverhouse—
afterwards, for his services in the royal cause, created Viscount
Dundee.
In the annals of Scotland there is no name amongst the
unworthiest of her sons,—Monteith the betrayer of Wallace, Cardinal
Beaton, the ruthless persecutor, Dalziel, with a monomania for
murder and oppression,—so utterly detestable as that of the dashing
cavalier, Claverhouse. His portrait is that of a haughty, self-centred
man; one would think too proud for the meanly savage work he was
set to do, but which, with fell intensity, he seemed to revel in doing.
In the conflict, he appeared to have a charmed life, and in these
superstitious times he was believed to have made a paction with
Satan:—for doing the fiend’s work he was to have so many years
immunity from death: neither lead nor steel could harm him. It was
said that his mortal wound, received in the moment of victory at
Killiecrankie, was from being shot by a silver bullet.
Claverhouse, in quest of the demonstrators at Rutherglen, came,
at Drumclog, about twenty miles south of Glasgow, on the body of
insurgents; about fifty horsemen fairly well appointed, as many
infantry with fire-arms, and a number armed with pikes, scythes,
and pitch-forks. The Covenanters had skilfully posted themselves; a
morass and broad ditch in front, the infantry in the centre, a troop of
horse on each flank. Claverhouse’s call to surrender was answered
by the singing of a verse of a warlike psalm. The troops gave a loud
cheer, and rode into the morass; they found it impassable and
themselves under a steady fire from the Covenanters. Claverhouse
sent flanking parties to right and left. These were boldly met before
they had time to form after crossing the ditch, and nearly cut to
pieces. And then the Covenanters made a sudden rush, and after a
desperate defence by Claverhouse, they utterly routed him,—the
only battle he ever lost.
This victory of the Covenanters over regular troops, ably
commanded, was a general surprise, and it found the victors ill-
prepared to follow it up to advantage. They next day occupied
Hamilton, and, reinforced by numbers, proceeded to attack Glasgow.
They were at first beaten back by Claverhouse, but he thought it
advisable to retreat to Edinburgh; and then the insurgents occupied
Glasgow. The King meanwhile had sent the Duke of Monmouth—a
courteous and courageous gentleman,—albeit the bar sinister ran
through his escutcheon—to collect an army to quell the rebellion. On
21st June the Covenanters—who had now their headquarters near
Hamilton, on the south-western bank of the Clyde, learned that the
Duke, at the head of a powerful army, was advancing towards
Bothwell Bridge—crossing which he would be upon them.
In the face of the common enemy, polemical disputes between the
different presbyterian parties brought confusion into their councils.
The moderate party drew up a supplication to the Duke, describing
their many grievances, and asking that they be submitted to a free
parliament. The Duke sent a courteous reply, expressing sympathy,
and offering to intercede for them with the King,—but they must first
lay down their arms. This condition the extreme party would not
listen to, and at this most unsuitable moment, they nominated fresh
officers—men indisposed to acknowledge any allegiance to the King,
or, in matters appertaining to religion, to submit to the civil power.
Under Rathillet, Burley and other irreconcilables, 300 men were
posted to hold the bridge; they made a stout defence; but it was
forced at the point of the bayonet. Bishop Burnet says,—“The main
body of the insurgents had not the grace to submit, the courage to
fight, nor the sense to run away.” But when the cannon began to
make havoc in their ranks, and they saw the deadly array of
horsemen, and the serried ranks of disciplined infantry preparing to
charge—they threw down their arms, and became a mob of
fugitives.
And now Claverhouse had to avenge Drumclog. His war-cry on
that day had been “No Quarter,” and this was his intention at
Bothwell Bridge. Four hundred were killed on the field and in the
flight, but the strict orders of the Duke were “Give quarter to all who
surrender—make prisoners, but spare life;” and thus the relentless
swords of Claverhouse and Dalziel were stayed. With the indignation
of a true soldier, Monmouth rejected a proposal to burn Hamilton
and to devastate the surrounding country; and he issued a
proclamation promising pardon to all who made their submission by
a certain day.
But the milder spirit of Monmouth found no place in the treatment
of the prisoners taken at Bothwell. They were marched to
Edinburgh, suffering much on the way; there, 1200 men were
huddled together without shelter in the Greyfriars churchyard—
sleeping amongst the tombs upon the bare ground. Several
supposed leaders were executed, some escaped further misery by
death from exposure, others were set free on signing a declaration
never to take arms against the King, and 257 were sent as slaves to
Barbadoes.
And meantime Claverhouse was passing as a destroying angel
through the western shires. Making little distinction between those
who had, and those who had not, taken part in the late insurrection
—he seized the property, and imprisoned or put to death, all against
whom any charge of contumacy could be laid. The hunted
Covenanters were driven into wilder seclusions, and their barbarous
treatment naturally made them more aggressive and extravagant in
their language. Useless to talk to men frenzied to despair of loyalty
to a King, who, in his life of unhallowed pleasure in distant London,
heard not, or cared not, for the bitter cry of the people whose rights
he had sworn to protect. When they met at midnight in lonely glen
or trackless moor, the leaders, Cameron, Cargill, Renwick, and
others, would, like the Hebrew Prophets of old, mingle prophecy
with denunciation; their high-strung enthusiasm bordered on
insanity.
Cameron and Cargill published a declaration denouncing Charles,
calling on all true sons of the Covenant to throw off their allegiance,
and take up arms against him. And government had now a pretext
for putting Scotland under what was really martial law. The common
soldiers were authorised to put to death, without any pretence of
trial, all who refused to take the prescribed oath, or to answer all
interrogations. It was a capital crime to have any intercourse with
prescribed persons; and torture was inflicted, even on women, to
extort the whereabouts of these persons. At Wigtown, Margaret
McLauchlan, a widow of sixty-three years, and Margaret Wilson, a
girl of eighteen, were drowned by being bound to stakes within
flood-mark.
Amongst many murders perpetrated at this time, that of John
Brown, the Ayrshire carrier, stands out conspicuous in horror. He was
a quiet, sedate man, leading a blameless life; his only offence was
that he did not on Sundays attend the parish church, but either read
his bible at home, or, with a few like-minded, met in a quiet place for
a little service of praise and prayer. One morning, whilst digging
peats for the house fire, he was surrounded by Claverhouse’s
dragoons, and brought to his own door. Here, his wife and children
being by—a baby in its mother’s arms—Claverhouse asked him why
he did not attend on the King’s curate; and John, answering that he
had to obey his conscience rather than the King, Claverhouse told
him to prepare for death. He said he had long been so prepared. He
prayed with fervour, until interrupted by Claverhouse, who saw his
wild dragoons beginning to shew tokens of sympathy; Brown kissed
his wife and little ones, and he was then shot dead. “What do you
think of your bonnie man now?” the devil-hearted slayer asked of
the newly-made widow. “I aye thocht muckle o’ him, but never sae
muckle as I do this day.” She laid her infant on the ground, tied up
the poor shattered head in her kerchief, composed the limbs,
covered the body with a plaid, and then she sat down beside it, and,
in heart-rending sobs and tears, gave full course to natural sorrow.
The tragedy enacted on Magus Moor was a cruel murder, but if there
are degrees of guilt in such an awful crime, that committed at the
cottage door in Ayrshire was surely the more heinous and atrocious
of the two.
Monmouth remained only a short time in Scotland; Lauderdale
was still nominally at the head of affairs. But in November, 1679, the
King sent his brother James to Edinburgh, partly to keep him out of
sight from the people of England. As a rigid Roman Catholic,
standing next in succession to the throne, he was very unpopular. A
cry of popish plots had been got up, and an Exclusion Bill would
have been carried in Parliament,[4]
but Charles dissolved it, and he
never called another; for the last four years of his life he reigned as
an absolute monarch.
James, a royal Stuart, residing in long untenanted Holyrood, was
made much of by the Scottish nobility and gentry, and to conciliate
them he so far unbent his generally sombre and unamiable
demeanour. He paid particular attention to the Highland chieftains,
and thus laid a foundation for that loyalty to himself and his
descendants, so costly to the clansmen. But his presence and his
influence in public affairs did no good to the poor Covenanters.
Against nonconformity of every shade his only remedies were
persecution and suppression. The poor wanderers of the Covenant
were hunted as wild beasts. Richard Cameron was slain at Aire
Moss. Hackston and Cargill were hanged. It is said that James often
amused his leisure hours by witnessing the tortures of the boot and
the thumb-screw.
And not the common people only were thus vexed and harassed.
Strangely-worded oaths, acknowledging the laws and statutes, and
also the King’s supremacy, were administered to all holding official
positions. When, as a privy counsellor, the oath was tendered to the
Earl of Argyle—son of the Marquis who was beheaded at the
commencement of the reign—he declared he took it so far as it was
consistent with itself, and with the Protestant religion. For adding
this qualification, he was tried for, and found guilty of, high treason.
He contrived to escape from Edinburgh Castle in the disguise of a
page holding up his step-daughter’s train. He reached Holland, a
sentence of death hanging over him.
And in England, after dismissing the Oxford parliament, the King
was despotic. If he had any religious faith at all, it was towards
Catholicism, and thus he took up his brother’s quarrel. In the
administration of justice, juries were packed, and judges were venal.
London was adjudged to have illegally extended its political powers,
was fined heavily, and condemned to lose its charters. Breaches of
their charters by provincial towns were looked for, and something
was generally found sufficient to raise prosecutions upon, the award
being always given for the Crown. Fines were levied for the King’s
private advantage, and by his veto in the election of magistrates he
held in his hand Parliamentary elections. The university of Oxford
issued a solemn decree, affirming unlimited submission to the Royal
authority; and the most detestable of the very few judges whose
names are a stain upon the history of English jurisprudence—
Jeffreys—was the very incarnation of venality and injustice; he was a
vulgar bully, ever finding a demoniacal pleasure in cruelty and
wrong-doing.
The country had been sickened of civil war, and public spirit
seemed to have deserted the land. Still the Whig leaders of the late
majority in Parliament made some attempts at organizing resistance.
Shaftesbury was for immediate rebellion; but Lords Essex, Howard,
and William Russell, and Algernon Sidney, more cautiously resolved
to wait the course of events, and act when an opportunity arose.
They certainly meant an insurrection in London, to be supported by
a rising in the West of England, and another in Scotland under the
Earl of Argyle.
But a conspiracy in a lower stratum of political influence, called
the Ryehouse Plot, which proposed the deaths of the King and his
brother, having been divulged to the Government, and certain
arrests made, the prisoners, to save themselves, declared that Lords
Howard and Russell, and Sidney, Hampden (a grandson of the John
Hampden of ship-money fame), and others were implicated. Howard
—recreant to the traditions of his name—turned approver. Lord
William Russell was tried for treason—nobly supported by his wife—
and although the evidence against him was weak, a packed jury
convicted him, and he was beheaded at Lincoln’s Inn Fields. Sidney
was tried by Judge Jeffreys. Howard was the only witness against
him, and for a conviction of treason the law required at least two
witnesses; but a manuscript treatise on Government had been found
amongst Sidney’s papers; certain passages on political liberty would
nowadays be considered as mere truisms, but Jeffreys ruled that
they were equal to two-and-twenty adverse witnesses. He also was
found guilty, and was beheaded on Tower Hill. Shaftesbury fled to
Holland. Lord Essex—a true nobleman—blaming himself for having
put it into Howard’s power to injure Lord Russell, committed suicide.
And some Scottish gentlemen were also implicated in the Whig
plot. Bailie, of Jerviswood, had been in correspondence with Lord
Russell, and was asked to give evidence against him. On his refusal,
he was himself tried for treason,—condemned and executed. Many
were fined and imprisoned; many left the country, or otherwise
could not be found, but were tried in their absence—outlawed, and
their estates forfeited.
James returned to London: he feared the influence of the Duke of
Monmouth, who, trading on his father’s favour and his own
handsome face and genial manners, posed as an ultra-Protestant,
and, in spite of his illegitimate birth, aspired to the succession.
James had Monmouth sent to Holland—then, under the Prince of
Orange, the refuge for English and Scottish exiles.
But for Charles the world of time was now at its vanishing point.
He was only in his fifty-fifth year when, in the midst of his sensuous
pleasures, apoplexy seized him, and Bishop Ken had to tell him his
hours were numbered. Certain religious exercises were gone
through, and the sacramental elements being brought in, the bishop
proposed their administration. The King put this off, and the bishop
retired. And now James looked up a Catholic priest, and had him
smuggled in by a private door to the King’s chamber. The King made
confession, and had the last rites of the Church administered. Thus
made safe by a Romish passport into heaven—the dying King no
doubt enjoyed as a good joke the prayers and admonitions of the
Protestant prelates, who, with the lords-in-waiting, were afterwards
ushered into his chamber. He died February 6th, 1684-5.
Essentials of Compilation An Incremental Approach in Python 1st Edition Jeremy G. Siek
Scotland under James the Second.
Within half-an-hour of his brother’s death, James was seated as
the King in Council. He declared that he would govern by the laws,
and maintain the established church. Loyal addresses from all parts
of his dominions were poured in upon him; and the commencement
of his reign gave promise of stability and popularity. In a lesser
degree he had his brothers vices; but he had shewn considerable
aptitude for public business, and was not deficient in personal
courage. In 1665, he had, in a war with Holland, taken the
command of the Channel fleet. On the 3rd of June a great battle was
fought off the Norfolk coast, within sight of Lowestoft. When the
fight was at its hottest, the Dutch admiral’s ship blew up, and a
Dutch fire-ship grappled with and destroyed an English ship. James
had twice to shift his flag, as his ships were successively disabled.
After an obstinate contest the Dutch ships sailed for the Texel;
James pursued for a time,—eighteen of the enemy’s ships being
taken or destroyed.
But his accession to the throne was not to be unchallenged. The
Duke of Monmouth and the Earl of Argyle met in Holland, and
concerted simultaneous insurrections in England and Scotland.
Monmouth landed at Lyme, in Dorsetshire, on 11th June, and
marched to Taunton, in Somersetshire, at the head of 5,000
irregularly armed troops. He had married the heiress of Buccleuch,
and in other ways became associated with the nobility; stories had
been set afloat of a marriage between his father and his Welsh
mother, Lucy Walters, and he was looked on by many as the true
heir to the throne. At Taunton he was received with acclamations;
twenty young ladies presented him with a pocket-bible, a flag, and a
naked sword. He had himself proclaimed King. After a good deal of
tentative marching through the western counties, he fell back on
Bridgewater, and three miles from this town, at Sedgemoor, a battle
was fought, in which he was utterly defeated. He himself fled before
the close of the fight; and was afterwards captured hiding in a bean-
field.
He was taken to London, and at his own solicitation had an
interview with the King. A larger-minded man than James would
have been moved to generosity, at the sight of his brother’s son
grovelling on his knees before him, and humbly suing for mercy; but
generosity towards fallen enemies was not a distinguishing trait in
the Stuart character. And this young man had long been a thorn in
James’s path; so now no mercy for him—his doom was immediate
execution.
And terrible was the vengeance of the King on not only the
leaders of the insurrection, but on inferior participants, and on all
who had given aid or countenance thereto. There were a number of
military executions; and then Jeffreys was let loose upon the
western counties. His “bloody assize” was a very devil’s carnival of
barbarity and death. The campaign was opened at Winchester with
the trial of Alice Lisle, the aged widow of one of Cromwell’s
lieutenants, for affording food and shelter to two of the fugitive
insurgents. Jeffreys bullied the jury into a verdict of guilty, and then
he sentenced her to be burned alive that same afternoon. Horror-
stricken, the clergy of the cathedral obtained a respite for three
days. Noble ladies, whom she had befriended in the time of the
Commonwealth, solicited her pardon from the King. Her son in the
army had served against Monmouth. And James was actually moved
to change her sentence from burning alive to beheading! And so it
was executed. In this judicial massacre, more than three hundred
persons were put to death, and very many who escaped death,
suffered mutilation, imprisonment, or exile. Hundreds of the
prisoners were presented to the courtiers,—to be sold for ten years
as slaves in the West Indies. The twenty young ladies of Taunton,
who had figured in the ovation to Monmouth, were assigned to the
Queen’s maids-of-honour, and they sold pardons to the girls at the
rate of a hundred pounds a head!
The accession of James brought no relaxation in the oppressive
laws bearing upon Scottish presbyterianism. It was still in the power
of the military to apprehend and interrogate, to torture, to confiscate
the goods, and even to take the lives of those suspected of
nonconformity, or of assisting outlawed persons. It was therefore to
be expected that any attempt to throw off the galling yoke would
have general sympathy and support. Argyle had himself been the
victim of unjust persecution; and yet his invasion of Scotland was as
futile and disastrous as that of Monmouth was of England.
Argyle was a Highland chief, influenced by his old family feuds;
and his foremost idea was to fight the clans which were the
hereditary enemies of his house, and also loyal Jacobites. So with
about three hundred men he landed on the western peninsula of
Cantyre, and was joined by about a thousand of his Campbell
clansmen. He proposed marching to Inverary; but the other leaders
were afraid of their little army being shut up in the highlands, and
thought that the western shires—in which the covenanters were
numerically strong, and where they had already boldly faced the
government troops—would be a better field for operations. There
was as usual in such differences, much wordy recrimination; time
was lost; and when at length a movement was made into
Lanarkshire, long, weary marches, with mistakes in the route,
disheartened and demoralized the insurgents. The royal troops, in
superior numbers, were fast closing in on Argyle, and, without a
battle, his following fell to pieces, and himself was made prisoner. He
was taken with disgraceful indignities to Edinburgh, and his old,
most iniquitous sentence was carried out. Like his father, he met his
fate with firmness; he said the grim instrument of death was “a
sweet Maiden, whose embrace would waft his soul into heaven.”
Upwards of twenty of the more considerable of his followers also
suffered death.
EXECUTION OF THE EARL OF ARGYLE.
As shewing the mean and cruel spirit of James, we may mention
that on medals which he had struck, commemorative of his triumphs
over Monmouth and Argyle, one side bore two severed heads, and
the reverse two headless trunks.
And now in his plenitude of power, James began to shew openly
what was his great intention, namely, the subversion of the
Protestant faith, and the restitution of papal sway in Britain. His
brother had so far paved the way for such a change, that he had
taken advantage of the reaction of loyalty at the Restoration, of the
general disgust at that detestable imposture, the Titus Oates’
“popish plot,” and of the discovery of the atrocious Rye House plot,
to make his government despotic. He had, by his foul example, sown
the seeds of immorality and corruption broadcast through the
national life. Religious fervour and high political principle seemed to
have vanished from the land,—servile submission to kingly authority
was preached by divines, sung by poets, and practised by
statesmen,—as the only safeguard against sombre puritanism,
political strife, and the misrule of the mob.
And now here was a zealot,—seeing sycophants all around him;
men of position hasting to gain his favour through the Romish
confessional; a servile parliament granting him bountiful supplies;
and a powerful French king sending him subsidies,—with the
property, the liberties, the very lives of his subjects at his disposal,—
can we wonder that he thought that his authority could be stretched
to lording it also over their consciences?
A century and a half previously, Henry VIII. had abrogated the
authority of the Pope in England, and James may have believed that
what one despotic king could do, another could undo. Of three
things we hardly know which most to wonder at:—the daring of the
attempt—or, how nearly he succeeded in his designs—or, that amidst
so much apathy, servility, and corruption, he did not, for a time at
least, accomplish his ends. But the Reformation was, on the face of
it, a natural outcome of a new dawn, after the long night of the dark
ages in Europe. It was, with the revival of letters, the new
geographical and scientific discoveries, and the general spirit of
adventure and research, a stepping-stone towards progress and
enlarged political and intellectual freedom; whilst the proposed
retrocession to Rome meant going backwards, and a wilful surrender
to the old bondage and authority.
James publicly attended the rites of his church; he surrounded
himself by Catholic priests, a leading Jesuit, Father Petre, being his
political confidant; he entertained at his court—for the first time in
England since the days of Queen Mary—a papal nuncio. He placed
the Church under the control of a High Commission of seven
members, Jeffreys, now Lord Chancellor, at the head. In chartered
towns, Catholics were to be eligible to serve as mayors and
aldermen. He began the formation of a large standing army, and, in
defiance of the Test Act, and in assertion of his dispensing power, he
largely officered this army by Catholics. The university of Oxford
had, in the previous reign, declared that in no case was resistance to
the royal authority justifiable, and it had now to reap the bitter fruits
of its servile declaration. The King appointed a Roman Catholic to
the deanery of Christ Church; another to the presidency of Magdalen
College, and twelve Catholic fellows were appointed in one day.
Oxford now began to see that passive obedience might well stop
short of a surrender of religious principles; it resisted the royal
mandates; and it would not submit, although twenty-five of its
fellows were expelled.
And a contagion of conversion broke out in the higher social
ranks. Noble lords and ladies of fashion went to mass and
confession; processions of Catholic priests were daily met in the
streets of London; Catholic chapels and monasteries were becoming
numerous, their service bells ringing perpetually.
In Scotland, the Chancellorship was bestowed on one of the King’s
time-serving converts, Drummond, Earl of Perth. He co-operated
with the Earl of Sunderland in England, in driving on James to the
most extravagant reactionary measures. By a new court order all
persons holding civil offices in Scotland were ordered to resign, and
to resume their offices without taking the test oath, ordered in 1681,
they taking, for thus breaking the law, a remission of penalties from
the Crown; all not obtaining such remission to be subjected to the
said penalties. That is,—all officials were ordered to break the law,
and were to be subject to penalties for such infringement,—unless
by getting the King’s pardon they acknowledged his power to
abrogate the law! And this test oath had been the contrivance of
James himself when in Scotland,—forced upon Presbyterians at the
sword’s point, and held so sacred that Argyle had been condemned
to death for taking it with a slight qualification.
The short reign of James was one of the saddest periods in
Scottish history. He had refused to take the usual coronation oath,
which included the maintenance of the established church. In spite
of this refusal—which impaired the validity of his right to rule—a
weakly compliant parliament expressed the loyalty of absolute
submission. The law against conventicles was extended to the
presence of five persons, besides the family attending domestic
worship. If the meeting was held outside the house—even on the
door-step—it was to be considered a field-conventicle punishable by
death. But on the question of repealing the penal acts against
Catholics, Parliament proved refractory, and it was forthwith
dissolved.
The King issued a proclamation depriving the burghs of the right
of electing their own magistrates. When, to favour Roman
Catholicism, he issued his Declaration of Indulgence, by which there
was to be general liberty of worship; yet—strange anomaly—the
laws against field-preaching continued in full force. Under these
laws, James Renwick, a delicate, but enthusiastic field-preacher, was
executed in Edinburgh in February, 1688. He was the last in the
fearfully long roll of covenanting martyrs.
THE COVENANTERS’ MONUMENT IN THE
GREYFRIARS’ CHURCHYARD, EDINBURGH
The Declaration of Indulgence, permitting all professions of
religion to worship in their own ways, was published by James—
solely on his own authority—in April, 1687. At the first blush we may
be inclined to call this general indulgence a step in the right
direction,—even although we know that under the cloak of toleration
to all forms of faith, the King’s main object was to legalise Catholic
worship and ritual. We now say, from the more liberal stand-point of
the nineteenth century, that the penal laws against the exercise of
Catholic rites were tyrannical and unjust. But we have to consider
the times in which these laws were introduced, when after a long
and bitter struggle the papal yoke had been thrown off,—when the
severities of Rome against those she termed heretics were fresh in
the memory,—and that she never abates one jot of her assumption
to be the one authoritative church—claiming the entire submission of
Christendom. And Dissenters knew that the King was here bidding
for their support against the established church. They saw that
Tyrconnel, the King’s Viceroy in Ireland—a country where James did
not require to keep up appearances—was fast arming the Catholics,
preparatory to a total subversion of Protestantism; and thus the
Presbyterian and other dissenters saw in the Episcopal Church the
rallying point of religious freedom; they overlooked its past
subserviency to power and its harshness to themselves, in
consideration of its present danger, and the stand it was now
preparing to make in the common cause.
In April, 1688, the king ordered his Declaration to be read in all
the churches. The London clergy met and signed a refusal to comply
with the order, and the primate, Sancroft, and six other bishops,
presented a petition to the king against being compelled to read a
document which assumed the legality of the dispensing power. Only
in seven of the London churches, and a few in the country, was the
Declaration read. The king was furious, and summoned the bishops
before the privy council; on their acknowledging their signatures to
the petition, they were committed to the Tower. Their passage down
the Thames was a public ovation; from crowded quays, bridges, and
barges arose enthusiastic shouts of encouragement; the very officers
of the Tower went on their knees for the episcopal blessing. In their
imprisonment, the bishops were visited daily by nobles and leading
men; and—which irritated James most of all—a deputation of
dissenting ministers went and thanked them in the name of their
common Protestantism.
And just at this time an event occurred which had a remarkable
bearing on the history of the period. On June 10th, 1688, James’s
queen gave birth to a son. The news had been circulated that a child
was expected; the faithful ventured to prophesy a prince; a blessing
vouchsafed by the intervention of the Virgin Mary, in response to
prayers and pilgrimages. But Protestant England had both feared
and doubted. The Court and its household were, almost exclusively,
composed of Catholics, and when the birth of a prince was
announced, it was generally believed that a strange child had been
smuggled into the palace, and was then being passed off as the
king’s son. There now seems little doubt but that the infant was
really the offspring of the king and queen. Thus, to his father’s joy,
and to Catholic anticipations of the throne being after him still
occupied by a king of the old faith—but with general doubts and
misgivings—with repudiation instead of welcome, came into the
world the ill-fated prince, known in our history as James the
Pretender.
On June 20th, the trial of the bishops took place before the Court
of King’s Bench. They were charged with having “published a false,
malicious, and seditious libel.” Of the four judges, two were for the
petition being a libel, and two were against. The jury had to decide
the question, and were locked up during the night. At ten o’clock
next morning, when the Court again met, there was a silence of
deep suspense before the verdict was pronounced. When the words
“not guilty” fell from the foreman’s lips, a great cheer arose, which
penetrated into the crowded street, and was speedily wafted over
London, extending even to the troops on parade at Blackheath. It
was a day of general congratulation and rejoicing; and bonfires and
illuminations went far into the summer night.
Essentials of Compilation An Incremental Approach in Python 1st Edition Jeremy G. Siek
B
The Revolution of 1688.
efore the birth of the prince, the general idea had been that the
country should tide over James’s misgovernment as best it could,
and wait patiently for the succession to the throne in natural course
of Mary, Princess of Orange, the elder daughter of the king by his
first marriage. But the situation was now altogether changed; and
on the very day of the acquittal of the bishops, there was sent—
signed by the bishop of London, several noblemen, and others—an
invitation to William to come over with an army to the relief of the
country: and the prince at once commenced his preparations.
And meantime, James, his purposes and hopes of success
strengthened by the birth of a son, was indignant at his defeat in the
trial of the bishops, and, goaded on by the French minister and his
inner circle of advisers, he resolved to crush the spirit of the nation
by force of arms. He brought over several regiments of Tyrconnel’s
Irish troops, and their menacing presence, as strangers and
Catholics, was hateful to the English people. A derisive doggrel
ballad, called from its burden Lilliburelo, was sung and whistled all
over the land.
And now the king was told that his Dutch son-in-law was making
great preparations for invasion. He knew that he had lost the best
safeguard of his throne—the confidence and affection of his subjects
—and whilst adopting means for defence, he hastened to retract all
the measures which had made him unpopular. He threw himself in
feigned repentance on the advice of the bishops, and they, in plain
words, like the prophets of old, told him of his injustice and
oppression, and advised him at once to call a Parliament. He
dismissed his priestly adviser Father Petre, and the renegade Lord
Sunderland. He restored its fellows to Oxford, and their franchises to
the corporations. But the precipitation of fear was so evident in his
concessions, that there was no reaction of confidence. The people
were watching the weathercocks, and praying for a north-east, or, as
it was called “a Protestant” wind.
After waiting some weeks for a favourable wind, and with an
after-delay from storms, by the end of October, William was fairly at
sea. He first sailed up the North Sea, as if he intended a landing on
the Yorkshire coast; but changed his course for the Channel. The
wind and tide prevented the royal fleet from attacking him in the
Straits of Dover. From the opposite coasts his fleet presented a
magnificent sight. There were sixty men-of-war and seven hundred
transports, extending twenty miles in length.
It was just a hundred years since such another magnificent
spectacle had been seen in the Channel—the Spanish Armada—also
bent upon the invasion of England. Then, the great fleet meant
papal aggression, and priestly domination; now, it meant deliverance
from this aggression, and freedom of the conscience; then, beacon
fires on mount and headland flashed danger to the lives and liberties
of Englishmen; now the tidings that a foreign fleet was skirting the
coast were of glad and hopeful assurance.
On the 5th of November—the anniversary of the Gunpowder Plot
—the fleet anchored at Torbay, in Devonshire. With his army of
fifteen thousand men, William marched to Exeter, where he was
enthusiastically received. But the memory of Jeffreys’ “bloody assize”
was still fresh in the western shires, and for several days there were
few signs of encouragement; it is said that he even meditated
returning to Holland. But bye-and-bye one nobleman after another,
and several officers of James’s army, entered the camp. The north of
England began to stir in raising and disciplining revolutionary troops,
and the Earl of Bath put Plymouth into William’s hands.
The King hastened down to Salisbury, resolved to stake his
kingdom on the issue of a battle; but William, although a thorough
captain in war, wished to avoid bloodshed; he trusted to the
increasing stream of desertion from the king rendering a great battle
unnecessary. And so it turned out. The sagacious lieutenant-general
of the king’s army, Lord Churchill, the Dukes of Grafton and Ormond,
even the king’s younger daughter Anne, with her husband, Prince
George of Denmark, and many other persons of note, joined the
Prince of Orange.
James went back to London, and sent away the queen and her
five-months’ old child to France. When he knew of their safety he
left London at night, by the river. He threw the great seal into the
Thames, and proceeded to Sheerness, where a small vessel was
waiting for him. Boarding the vessel he attracted the attention of
some Kentish fishermen, who, in hopes of reward, made him
prisoner. Released, by an order of the Lords, he returned to London,
and passed thence to Rochester. William wanted him out of the
country; so facilities were made for his escape, and he was soon at
St. Germains, where Louis gave him a friendly reception; and at St.
Germains he made his home. Assisted by Louis, he made, next year,
an attempt for the recovery of Ireland. In that essentially Catholic
country, it seemed at first that he would there be able to retain one
of the three kingdoms, but his defeat by William, at the Boyne,
compelled his return to France. He died September 16th, 1701, aged
68 years.
The King, having fled, and no parliament sitting, William was
advised to claim the kingdom by right of conquest. But both from
principle and sound policy he held that this would be a less secure
right of possession than would be the choice—as formal as under
the circumstances it could be made—of the English people. So he
summoned a Convention of the States of the Realm,—irregularly
convoked in the emergency, but elected in the usual manner. The
Convention met on 22nd February—six weeks after the King’s flight.
The debates were long and stormy; the two Houses disagreed,—
the Lords could hardly bring themselves to declare for the deposition
of the King; but the Commons were firm, and at length this
resolution was passed in both houses: “That James, having violated
the fundamental laws, and withdrawn himself from the kingdom, has
broken the original contract between king and people, has abdicated
the government, and therefore the throne has become vacant.”
And then came the questions,—Who was to reign? and what was
to be the order of succession? Here there was a division of opinion.
Was James’s infant son to be acknowledged as King—with William as
Regent? or, Should the crown be conferred on Mary in her own
right? William was not a man of many words, but he now got
together a few of the leading men, and to them he spoke very
plainly: he would not interfere with the right of the Convention to
settle its own affairs as it thought best; but for himself he would not
accept any regency, nor—much as he loved his wife—would he
remain in England as her gentleman-usher. In a few hours his words
were all over London, and it was known that he would be King.
So the Convention passed a number of resolutions, embodied in
what was termed a Declaration of Rights,—defining the royal
prerogative, and the powers of parliament; and the Prince and
Princess, having signified their adhesion thereto, it was resolved that
William and Mary be jointly King and Queen of England, Ireland, and
the dominions belonging thereto; the administration to rest in
William. The crown was settled,—first on the survivor of the royal
pair,—then on the children of Mary, then on those of her sister Anne,
and next on the children of William by any other wife. The son of
James and his posterity were thus shut out entirely from the
succession.
The Scottish Convention of Estates passed resolutions nearly
similar to those in the English Declaration of Rights, closing with a
declaration against Prelacy, asserting that there was no higher office
in the Church than presbyter.
On the leading question then before the country, their resolution
had a more decided tone than that of the English Convention. They
declared that James had assumed the throne without taking the
oaths prescribed by law, that he had proceeded to subvert the
constitution of the country from a limited monarchy to an absolute
despotism; that he had employed the powers thus usurped for
violating the laws and liberties, and altering the religion of Scotland;
for doing these things he had forfeited his right to the crown, and
the throne had thereby become vacant. The Scottish royalty was
conferred on William and Mary, in like terms as that of the English
Convention.
Battle of Killiecrankie.
In the crisis of his affairs, James had summoned his Scottish
troops to England. Their commander, Lord Douglas, went over to
William; but the second in command, John Graham of Claverhouse—
now Viscount Dundee—had an interview with the King—assured him
of the loyalty of his troops, about 6,500 well disciplined men,
advised the King either to hazard a battle, or to fall back with these
troops into Scotland. On the King declining both propositions, Lord
Dundee took up a position at Watford, about eighteen miles north-
west of London, expecting an attack by William. But Dundee had
served his early campaigns under the Prince, having in one
engagement rescued him from imminent danger. So the Prince now
sent him a message that he had no quarrel with him. Then came
James’s flight, and the Prince’s entry into London; and Dundee
seeing he could do nothing more to help James in England, rode
back with about twenty-five of his dragoons into Scotland. The
Scottish army was placed under General Mackay, one of William’s
adherents, and he was shortly after sent as commander of the royal
forces into Scotland.
Lord Dundee came to Edinburgh, for some time hovering like a
hawk over the then sitting Convention. The Duke of Gordon still held
the Castle for King James; Dundee had an interview with the Duke
and advised “no surrender,” he then, with a few horsemen, left the
city. (We all know the ringing song in which Sir Walter Scott narrates
his departure.) Like a fiery-cross he went through the highlands,
rousing the clansman to battle for the fallen Stuart King. The man
must have had a dominating personality; in a short time he had
assembled an army, feeble in discipline and cohesion no doubt; but,
as it proved, good for the kind of work it befell them to do.
The highlanders were posted on an open slope at the head of the
pass of Killiecrankie in the north Perthshire hills. To give them battle,
Mackay, on 17th June, 1689, advanced up the pass. When the royal
troops entered the defile, no enemy was to be seen,—only the pines
towering high upon the cliffs on either hand, and the river Garry
rushing swiftly by the narrow pathway through the pass. To the
Lowland and Dutch soldiers, who composed the royal army, it was a
scene novel and magnificent, but bewildering, awe-inspiring.
Dundee allowed the whole of Mackay’s army to emerge from the
pass, and even to form in order of battle, before he began the
attack. It was an hour before sunset that the highlanders advanced.
They fired their muskets only once, and throwing them away, with
fierce shouts they rushed down with broadsword and target.
Mackay’s line was broken by the onset. When it came to disordered
ranks, and the clash of hand to hand combats, the superior discipline
of the royal troops was of no account. Agility, hardihood, and the
confidence of assured victory were on the side of the clansmen. It
was soon a rout; but with such a narrow gorge for retreat it became
a massacre. Two thousand of Mackay’s troops were slain. The
highlanders’ loss was eight hundred; but amongst these was their
gallant leader. Near the end of the battle, Dundee, on horseback,
was extending his right arm to the clan Macdonald, as directing their
movements, when he was struck by a bullet under the arm-pit,
where he was unprotected by his cuirass. With him perished the
cause of King James in Scotland. After his death his army melted
away, and both highlands and lowlands submitted to the
Government of William.
General lenity and toleration were the watchwords of William’s
policy. The episcopal church was to be maintained in England, and
the presbyterian in Scotland; but neither were to ride rough-shod
over dissenters. In Scotland, much against the desires of the more
rigid, as the Cameronians, there were to be no reprisals for former
persecution and oppression. Even obnoxious officials were
maintained in their old places. When the Jacobite rising in Ireland
was quelled by the surrender of Limerick, a treaty was there made
by which Catholics were to be allowed the free exercise of their
religion. William endeavoured to get parliament to ratify this treaty,
but two months after it had been entered into, the English
Parliament imposed a declaration against Transubstantiation on
members of the Irish parliament, and this parliament, entirely
composed of Protestants, whilst giving nominal confirmation, really
put the Catholics in a worse condition than they were before. The
Irish Catholics have since then called Limerick, “the town of the
broken treaty.”

More Related Content

PDF
CD NOTErvvtvvevbvtgv4tgtgtgtgtvefeveS.pdf
PDF
Download full ebook of Computer Systems 5th Edition (eBook PDF) instant downl...
PPTX
Lecture 1 introduction to language processors
PDF
COMPUTER SCIENCE COURSE 204 COMPILER CONSTRUCTION,.pdf
PPT
Chapter One
PDF
Compiler design
PDF
Compiler construction lecture 01 .pptx.pdf
CD NOTErvvtvvevbvtgv4tgtgtgtgtvefeveS.pdf
Download full ebook of Computer Systems 5th Edition (eBook PDF) instant downl...
Lecture 1 introduction to language processors
COMPUTER SCIENCE COURSE 204 COMPILER CONSTRUCTION,.pdf
Chapter One
Compiler design
Compiler construction lecture 01 .pptx.pdf

Similar to Essentials of Compilation An Incremental Approach in Python 1st Edition Jeremy G. Siek (20)

PPTX
ppt_cd.pptx ppt on phases of compiler of jntuk syllabus
DOCX
Compiler Design Material
PPT
Compiler Construction introduction
PPTX
Chapter 2 Program language translation.pptx
PPT
Compiler Design Basics
PPTX
Compiler Design Slides for Third Year Computer Science and Engineering
PDF
C++ book
PDF
C progrmming
PDF
6272 cnote
PPT
Compiler Design Basics
PDF
Chapter1pdf__2021_11_23_10_53_20.pdf
PPTX
Compiler Construction from very basic start
PPT
introduction of compiler unit 1 phases of compiler
PDF
Phases of Compiler
PDF
unit1pdf__2021_12_14_12_37_34.pdf
DOCX
Dineshmaterial1 091225091539-phpapp02
PPT
Introduction to compiler design and phases of compiler
PPTX
Chapter 1.pptx
PDF
Prof. Chethan Raj C, BE, M.Tech (Ph.D) Dept. of CSE. System Software & Operat...
PPTX
Ch 1.pptx
ppt_cd.pptx ppt on phases of compiler of jntuk syllabus
Compiler Design Material
Compiler Construction introduction
Chapter 2 Program language translation.pptx
Compiler Design Basics
Compiler Design Slides for Third Year Computer Science and Engineering
C++ book
C progrmming
6272 cnote
Compiler Design Basics
Chapter1pdf__2021_11_23_10_53_20.pdf
Compiler Construction from very basic start
introduction of compiler unit 1 phases of compiler
Phases of Compiler
unit1pdf__2021_12_14_12_37_34.pdf
Dineshmaterial1 091225091539-phpapp02
Introduction to compiler design and phases of compiler
Chapter 1.pptx
Prof. Chethan Raj C, BE, M.Tech (Ph.D) Dept. of CSE. System Software & Operat...
Ch 1.pptx
Ad

Recently uploaded (20)

PPTX
Cell Structure & Organelles in detailed.
PPTX
1st Inaugural Professorial Lecture held on 19th February 2020 (Governance and...
PDF
VCE English Exam - Section C Student Revision Booklet
PDF
3rd Neelam Sanjeevareddy Memorial Lecture.pdf
PPTX
Pharma ospi slides which help in ospi learning
PDF
O5-L3 Freight Transport Ops (International) V1.pdf
PDF
ANTIBIOTICS.pptx.pdf………………… xxxxxxxxxxxxx
PPTX
Introduction_to_Human_Anatomy_and_Physiology_for_B.Pharm.pptx
PDF
Abdominal Access Techniques with Prof. Dr. R K Mishra
PDF
Microbial disease of the cardiovascular and lymphatic systems
PPTX
Pharmacology of Heart Failure /Pharmacotherapy of CHF
PPTX
Renaissance Architecture: A Journey from Faith to Humanism
PPTX
master seminar digital applications in india
PPTX
Cell Types and Its function , kingdom of life
PDF
Chapter 2 Heredity, Prenatal Development, and Birth.pdf
PDF
Physiotherapy_for_Respiratory_and_Cardiac_Problems WEBBER.pdf
PDF
102 student loan defaulters named and shamed – Is someone you know on the list?
PDF
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
PDF
Classroom Observation Tools for Teachers
PDF
O7-L3 Supply Chain Operations - ICLT Program
Cell Structure & Organelles in detailed.
1st Inaugural Professorial Lecture held on 19th February 2020 (Governance and...
VCE English Exam - Section C Student Revision Booklet
3rd Neelam Sanjeevareddy Memorial Lecture.pdf
Pharma ospi slides which help in ospi learning
O5-L3 Freight Transport Ops (International) V1.pdf
ANTIBIOTICS.pptx.pdf………………… xxxxxxxxxxxxx
Introduction_to_Human_Anatomy_and_Physiology_for_B.Pharm.pptx
Abdominal Access Techniques with Prof. Dr. R K Mishra
Microbial disease of the cardiovascular and lymphatic systems
Pharmacology of Heart Failure /Pharmacotherapy of CHF
Renaissance Architecture: A Journey from Faith to Humanism
master seminar digital applications in india
Cell Types and Its function , kingdom of life
Chapter 2 Heredity, Prenatal Development, and Birth.pdf
Physiotherapy_for_Respiratory_and_Cardiac_Problems WEBBER.pdf
102 student loan defaulters named and shamed – Is someone you know on the list?
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
Classroom Observation Tools for Teachers
O7-L3 Supply Chain Operations - ICLT Program
Ad

Essentials of Compilation An Incremental Approach in Python 1st Edition Jeremy G. Siek

  • 1. Read Anytime Anywhere Easy Ebook Downloads at ebookmeta.com Essentials of Compilation An Incremental Approach in Python 1st Edition Jeremy G. Siek https://ebookmeta.com/product/essentials-of-compilation-an- incremental-approach-in-python-1st-edition-jeremy-g-siek/ OR CLICK HERE DOWLOAD EBOOK Visit and Get More Ebook Downloads Instantly at https://ebookmeta.com
  • 4. Essentials of Compilation An Incremental Approach in Python Jeremy G. Siek The MIT Press Cambridge, Massachusetts London, England
  • 5. © 2023 Jeremy G. Siek This work is subject to a Creative Commons CC-BY-ND-NC license. Subject to such license, all rights are reserved. The MIT Press would like to thank the anonymous peer reviewers who provided comments on drafts of this book. The generous work of academic experts is essential for establishing the authority and quality of our publications. We acknowledge with gratitude the contributions of these otherwise uncredited readers. This book was set in Times LT Std Roman by the author. Printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Names: Siek, Jeremy, author. Title: Essentials of compilation : an incremental approach in Python / Jeremy G. Siek. Description: Cambridge, Massachusetts : The MIT Press, [2023] | Includes bibliographical references and index. Identifiers: LCCN 2022043053 (print) | LCCN 2022043054 (ebook) | ISBN 9780262048248 | ISBN 9780262375542 (epub) | ISBN 9780262375559 (pdf) Subjects: LCSH: Compilers (Computer programs) | Python (Computer program language) | Programming languages (Electronic computers) | Computer programming. Classification: LCC QA76.76.C65 S54 2023 (print) | LCC QA76.76.C65 (ebook) | DDC 005.4/53–dc23/eng/20221117 LC record available at https://lccn.loc.gov/2022043053 LC ebook record available at https://lccn.loc.gov/2022043054 10 9 8 7 6 5 4 3 2 1
  • 6. This book is dedicated to Katie, my partner in everything, my children, who grew up during the writing of this book, and the programming language students at Indiana University, whose thoughtful questions made this a better book.
  • 8. Contents Preface xi 1 Preliminaries 1 1.1 Abstract Syntax Trees 1 1.2 Grammars 3 1.3 Pattern Matching 5 1.4 Recursive Functions 6 1.5 Interpreters 8 1.6 Example Compiler: A Partial Evaluator 10 2 Integers and Variables 13 2.1 The LVar Language 13 2.2 The x86Int Assembly Language 16 2.3 Planning the Trip to x86 21 2.4 Remove Complex Operands 23 2.5 Select Instructions 25 2.6 Assign Homes 26 2.7 Patch Instructions 27 2.8 Generate Prelude and Conclusion 27 2.9 Challenge: Partial Evaluator for LVar 28 3 Parsing 29 3.1 Lexical Analysis and Regular Expressions 29 3.2 Grammars and Parse Trees 31 3.3 Ambiguous Grammars 33 3.4 From Parse Trees to Abstract Syntax Trees 34 3.5 Earley’s Algorithm 36 3.6 The LALR(1) Algorithm 40 3.7 Further Reading 43 4 Register Allocation 45 4.1 Registers and Calling Conventions 46 4.2 Liveness Analysis 49 4.3 Build the Interference Graph 51
  • 9. viii Contents 4.4 Graph Coloring via Sudoku 52 4.5 Patch Instructions 58 4.6 Generate Prelude and Conclusion 58 4.7 Challenge: Move Biasing 59 4.8 Further Reading 62 5 Booleans and Conditionals 65 5.1 The LIf Language 66 5.2 Type Checking LIf Programs 66 5.3 The CIf Intermediate Language 72 5.4 The x86If Language 72 5.5 Shrink the LIf Language 75 5.6 Remove Complex Operands 75 5.7 Explicate Control 76 5.8 Select Instructions 82 5.9 Register Allocation 83 5.10 Patch Instructions 84 5.11 Generate Prelude and Conclusion 84 5.12 Challenge: Optimize Blocks and Remove Jumps 85 5.13 Further Reading 88 6 Loops and Dataflow Analysis 91 6.1 The LWhile Language 91 6.2 Cyclic Control Flow and Dataflow Analysis 91 6.3 Remove Complex Operands 96 6.4 Explicate Control 96 6.5 Register Allocation 96 7 Tuples and Garbage Collection 99 7.1 The LTup Language 99 7.2 Garbage Collection 102 7.3 Expose Allocation 109 7.4 Remove Complex Operands 110 7.5 Explicate Control and the CTup Language 110 7.6 Select Instructions and the x86Global Language 111 7.7 Register Allocation 116 7.8 Generate Prelude and Conclusion 116 7.9 Challenge: Arrays 118 7.10 Further Reading 123 8 Functions 125 8.1 The LFun Language 125 8.2 Functions in x86 130 8.3 Shrink LFun 133 8.4 Reveal Functions and the LFunRef Language 133
  • 10. Contents ix 8.5 Limit Functions 133 8.6 Remove Complex Operands 134 8.7 Explicate Control and the CFun Language 135 8.8 Select Instructions and the x86Def callq∗ Language 136 8.9 Register Allocation 138 8.10 Patch Instructions 139 8.11 Generate Prelude and Conclusion 139 8.12 An Example Translation 141 9 Lexically Scoped Functions 143 9.1 The Lλ Language 145 9.2 Assignment and Lexically Scoped Functions 150 9.3 Uniquify Variables 151 9.4 Assignment Conversion 151 9.5 Closure Conversion 153 9.6 Expose Allocation 156 9.7 Explicate Control and CClos 156 9.8 Select Instructions 157 9.9 Challenge: Optimize Closures 158 9.10 Further Reading 160 10 Dynamic Typing 161 10.1 The LDyn Language 161 10.2 Representation of Tagged Values 165 10.3 The LAny Language 166 10.4 Cast Insertion: Compiling LDyn to LAny 170 10.5 Reveal Casts 170 10.6 Assignment Conversion 171 10.7 Closure Conversion 171 10.8 Remove Complex Operands 172 10.9 Explicate Control and CAny 172 10.10 Select Instructions 172 10.11 Register Allocation for LAny 174 11 Gradual Typing 177 11.1 Type Checking L? 177 11.2 Interpreting LCast 183 11.3 Overload Resolution 184 11.4 Cast Insertion 185 11.5 Lower Casts 187 11.6 Differentiate Proxies 188 11.7 Reveal Casts 190 11.8 Closure Conversion 191 11.9 Select Instructions 191 11.10 Further Reading 193
  • 11. x Contents 12 Generics 195 12.1 Compiling Generics 201 12.2 Resolve Instantiation 202 12.3 Erase Generic Types 202 A Appendix 207 A.1 x86 Instruction Set Quick Reference 207 References 209 Index 217
  • 12. Preface There is a magical moment when a programmer presses the run button and the software begins to execute. Somehow a program written in a high-level language is running on a computer that is capable only of shuffling bits. Here we reveal the wiz- ardry that makes that moment possible. Beginning with the groundbreaking work of Backus and colleagues in the 1950s, computer scientists developed techniques for constructing programs called compilers that automatically translate high-level programs into machine code. We take you on a journey through constructing your own compiler for a small but powerful language. Along the way we explain the essential concepts, algorithms, and data structures that underlie compilers. We develop your understanding of how programs are mapped onto computer hardware, which is helpful in reasoning about properties at the junction of hardware and software, such as execution time, soft- ware errors, and security vulnerabilities. For those interested in pursuing compiler construction as a career, our goal is to provide a stepping-stone to advanced topics such as just-in-time compilation, program analysis, and program optimization. For those interested in designing and implementing programming languages, we connect language design choices to their impact on the compiler and the generated code. A compiler is typically organized as a sequence of stages that progressively trans- late a program to the code that runs on hardware. We take this approach to the extreme by partitioning our compiler into a large number of nanopasses, each of which performs a single task. This enables the testing of each pass in isolation and focuses our attention, making the compiler far easier to understand. The most familiar approach to describing compilers is to dedicate each chapter to one pass. The problem with that approach is that it obfuscates how language features motivate design choices in a compiler. We instead take an incremental approach in which we build a complete compiler in each chapter, starting with a small input language that includes only arithmetic and variables. We add new language features in subsequent chapters, extending the compiler as necessary. Our choice of language features is designed to elicit fundamental concepts and algorithms used in compilers. • We begin with integer arithmetic and local variables in chapters 1 and 2, where we introduce the fundamental tools of compiler construction: abstract syntax trees and recursive functions.
  • 13. xii Preface • In chapter 3 we learn how to use the Lark parser framework to create a parser for the language of integer arithmetic and local variables. We learn about the parsing algorithms inside Lark, including Earley and LALR(1). • In chapter 4 we apply graph coloring to assign variables to machine registers. • Chapter 5 adds conditional expressions, which motivates an elegant recursive algorithm for translating them into conditional goto statements. • Chapter 6 adds loops. This elicits the need for dataflow analysis in the register allocator. • Chapter 7 adds heap-allocated tuples, motivating garbage collection. • Chapter 8 adds functions as first-class values without lexical scoping, similar to functions in the C programming language (Kernighan and Ritchie 1988). The reader learns about the procedure call stack and calling conventions and how they interact with register allocation and garbage collection. The chapter also describes how to generate efficient tail calls. • Chapter 9 adds anonymous functions with lexical scoping, that is, lambda expressions. The reader learns about closure conversion, in which lambdas are translated into a combination of functions and tuples. • Chapter 10 adds dynamic typing. Prior to this point the input languages are statically typed. The reader extends the statically typed language with an Any type that serves as a target for compiling the dynamically typed language. • Chapter 11 uses the Any type introduced in chapter 10 to implement a gradually typed language in which different regions of a program may be static or dynami- cally typed. The reader implements runtime support for proxies that allow values to safely move between regions. • Chapter 12 adds generics with autoboxing, leveraging the Any type and type casts developed in chapters 10 and 11. There are many language features that we do not include. Our choices balance the incidental complexity of a feature versus the fundamental concepts that it exposes. For example, we include tuples and not records because although they both elicit the study of heap allocation and garbage collection, records come with more incidental complexity. Since 2009, drafts of this book have served as the textbook for sixteen-week compiler courses for upper-level undergraduates and first-year graduate students at the University of Colorado and Indiana University. Students come into the course having learned the basics of programming, data structures and algorithms, and discrete mathematics. At the beginning of the course, students form groups of two to four people. The groups complete approximately one chapter every two weeks, starting with chapter 2 and including chapters according to the students interests while respecting the dependencies between chapters shown in figure 0.1. Chapter 8 (functions) depends on chapter 7 (tuples) only in the implementation of efficient tail calls. The last two weeks of the course involve a final project in which students design and implement a compiler extension of their choosing. The last few chapters can be used in support of these projects. Many chapters include a challenge problem that we assign to the graduate students.
  • 14. Preface xiii Ch. 1 Preliminaries Ch. 2 Variables Ch. 3 Parsing Ch. 4 Registers Ch. 5 Conditionals Ch. 6 Loops Ch. 8 Functions Ch. 7 Tuples Ch. 10 Dynamic Ch. 9 Lambda Ch. 11 Gradual Typing Ch. 12 Generics Figure 0.1 Diagram of chapter dependencies. For compiler courses at universities on the quarter system (about ten weeks in length), we recommend completing the course through chapter 7 or chapter 8 and providing some scaffolding code to the students for each compiler pass. The course can be adapted to emphasize functional languages by skipping chapter 6 (loops) and including chapter 9 (lambda). The course can be adapted to dynamically typed languages by including chapter 10. This book has been used in compiler courses at California Polytechnic State Uni- versity, Portland State University, Rose–Hulman Institute of Technology, University of Freiburg, University of Massachusetts Lowell, and the University of Vermont. This edition of the book uses Python both for the implementation of the compiler and for the input language, so the reader should be proficient with Python. There are many excellent resources for learning Python (Lutz 2013; Barry 2016; Sweigart 2019; Matthes 2019).The support code for this book is in the GitHub repository at the following location: https://github.com/IUCompilerCourse/ The compiler targets x86 assembly language (Intel 2015), so it is helpful but not necessary for the reader to have taken a computer systems course (Bryant and O’Hallaron 2010). We introduce the parts of x86-64 assembly language that are needed in the compiler. We follow the System V calling conventions (Bryant and O’Hallaron 2005; Matz et al. 2013), so the assembly code that we gener- ate works with the runtime system (written in C) when it is compiled using the GNU C compiler (gcc) on Linux and MacOS operating systems on Intel hardware. On the Windows operating system, gcc uses the Microsoft x64 calling conven- tion (Microsoft 2018, 2020). So the assembly code that we generate does not work with the runtime system on Windows. One workaround is to use a virtual machine with Linux as the guest operating system.
  • 15. xiv Preface Acknowledgments The tradition of compiler construction at Indiana University goes back to research and courses on programming languages by Daniel Friedman in the 1970s and 1980s. One of his students, Kent Dybvig, implemented Chez Scheme (Dybvig 2006), an efficient, production-quality compiler for Scheme. Throughout the 1990s and 2000s, Dybvig taught the compiler course and continued the development of Chez Scheme. The compiler course evolved to incorporate novel pedagogical ideas while also including elements of real-world compilers. One of Friedman’s ideas was to split the compiler into many small passes. Another idea, called “the game,” was to test the code generated by each pass using interpreters. Dybvig, with help from his students Dipanwita Sarkar and Andrew Keep, devel- oped infrastructure to support this approach and evolved the course to use even smaller nanopasses (Sarkar, Waddell, and Dybvig 2004; Keep 2012). Many of the compiler design decisions in this book are inspired by the assignment descriptions of Dybvig and Keep (2010). In the mid 2000s, a student of Dybvig named Abdu- laziz Ghuloum observed that the front-to-back organization of the course made it difficult for students to understand the rationale for the compiler design. Ghuloum proposed the incremental approach (Ghuloum 2006) on which this book is based. I thank the many students who served as teaching assistants for the compiler course at IU including Carl Factora, Ryan Scott, Cameron Swords, and Chris Wailes. I thank Andre Kuhlenschmidt for work on the garbage collector and x86 interpreter, Michael Vollmer for work on efficient tail calls, and Michael Vitousek for help with the first offering of the incremental compiler course at IU. I thank professors Bor-Yuh Chang, John Clements, Jay McCarthy, Joseph Near, Ryan Newton, Nate Nystrom, Peter Thiemann, Andrew Tolmach, and Michael Wollowski for teaching courses based on drafts of this book and for their feedback. I thank the National Science Foundation for the grants that helped to support this work: Grant Numbers 1518844, 1763922, and 1814460. I thank Ronald Garcia for helping me survive Dybvig’s compiler course in the early 2000s and especially for finding the bug that sent our garbage collector on a wild goose chase! Jeremy G. Siek Bloomington, Indiana
  • 16. 1 Preliminaries In this chapter we introduce the basic tools needed to implement a compiler. Pro- grams are typically input by a programmer as text, that is, a sequence of characters. The program-as-text representation is called concrete syntax. We use concrete syn- tax to concisely write down and talk about programs. Inside the compiler, we use abstract syntax trees (ASTs) to represent programs in a way that efficiently sup- ports the operations that the compiler needs to perform. The process of translating concrete syntax to abstract syntax is called parsing and is studied in chapter 3. For now we use the parse function in Python’s ast module to translate from concrete to abstract syntax. ASTs can be represented inside the compiler in many different ways, depending on the programming language used to write the compiler. We use Python classes and objects to represent ASTs, especially the classes defined in the standard ast module for the Python source language. We use grammars to define the abstract syntax of programming languages (section 1.2) and pattern matching to inspect individual nodes in an AST (section 1.3). We use recursive functions to construct and deconstruct ASTs (section 1.4). This chapter provides a brief introduction to these components. 1.1 Abstract Syntax Trees Compilers use abstract syntax trees to represent programs because they often need to ask questions such as, for a given part of a program, what kind of language feature is it? What are its subparts? Consider the program on the left and the diagram of its AST on the right (1.1). This program is an addition operation that has two subparts, a input operation and a negation. The negation has another subpart, the integer constant 8. By using a tree to represent the program, we can easily follow the links to go from one part of a program to its subparts. input_int() + -8 + input_int() - 8 (1.1)
  • 17. 2 Chapter 1 We use the standard terminology for trees to describe ASTs: each rectangle above is called a node. The arrows connect a node to its children, which are also nodes. The top-most node is the root. Every node except for the root has a parent (the node of which it is the child). If a node has no children, it is a leaf node; otherwise it is an internal node. We use a Python class for each kind of node. The following is the class definition for constants (aka literals) from the Python ast module. class Constant: def __init__(self, value): self.value = value An integer constant node includes just one thing: the integer value. To create an AST node for the integer 8, we write Constant(8). eight = Constant(8) We say that the value created by Constant(8) is an instance of the Constant class. The following is the class definition for unary operators. class UnaryOp: def __init__(self, op, operand): self.op = op self.operand = operand The specific operation is specified by the op parameter. For example, the class USub is for unary subtraction. (More unary operators are introduced in later chapters.) To create an AST that negates the number 8, we write the following. neg_eight = UnaryOp(USub(), eight) The call to the input_int function is represented by the Call and Name classes. class Call: def __init__(self, func, args): self.func = func self.args = args class Name: def __init__(self, id): self.id = id To create an AST node that calls input_int, we write read = Call(Name('input_int'), []) Finally, to represent the addition in (1.1), we use the BinOp class for binary operators. class BinOp: def __init__(self, left, op, right): self.op = op self.left = left self.right = right
  • 18. Preliminaries 3 Similar to UnaryOp, the specific operation is specified by the op parameter, which for now is just an instance of the Add class. So to create the AST node that adds negative eight to some user input, we write the following. ast1_1 = BinOp(read, Add(), neg_eight) To compile a program such as (1.1), we need to know that the operation associ- ated with the root node is addition and we need to be able to access its two children. Python provides pattern matching to support these kinds of queries, as we see in section 1.3. We often write down the concrete syntax of a program even when we actually have in mind the AST, because the concrete syntax is more concise. We recommend that you always think of programs as abstract syntax trees. 1.2 Grammars A programming language can be thought of as a set of programs. The set is infinite (that is, one can always create larger programs), so one cannot simply describe a language by listing all the programs in the language. Instead we write down a set of rules, a context-free grammar, for building programs. Grammars are often used to define the concrete syntax of a language, but they can also be used to describe the abstract syntax. We write our rules in a variant of Backus-Naur form (BNF) (Backus et al. 1960; Knuth 1964). As an example, we describe a small language, named LInt, that consists of integers and arithmetic operations. The first grammar rule for the abstract syntax of LInt says that an instance of the Constant class is an expression: exp ::= Constant(int) (1.2) Each rule has a left-hand side and a right-hand side. If you have an AST node that matches the right-hand side, then you can categorize it according to the left-hand side. Symbols in typewriter font, such as Constant, are terminal symbols and must literally appear in the program for the rule to be applicable. Our grammars do not mention white space, that is, delimiter characters like spaces, tabs, and new lines. White space may be inserted between symbols for disambiguation and to improve readability. A name such as exp that is defined by the grammar rules is a nonterminal. The name int is also a nonterminal, but instead of defining it with a grammar rule, we define it with the following explanation. An int is a sequence of decimals (0 to 9), possibly starting with – (for negative integers), such that the sequence of decimals represents an integer in the range –263 to 263 – 1. This enables the representation of integers using 64 bits, which simplifies several aspects of compilation. In contrast, integers in Python have unlimited precision, but the techniques needed to handle unlimited precision fall outside the scope of this book. The second grammar rule is the input_int operation, which receives an input integer from the user of the program. exp ::= Call(Name('input_int'),[]) (1.3)
  • 19. 4 Chapter 1 The third rule categorizes the negation of an exp node as an exp. exp ::= UnaryOp(USub(),exp) (1.4) We can apply these rules to categorize the ASTs that are in the LInt language. For example, by rule (1.2), Constant(8) is an exp, and then by rule (1.4) the following AST is an exp. UnaryOp(USub(), Constant(8)) – 8 (1.5) The next two grammar rules are for addition and subtraction expressions: exp ::= BinOp(exp,Add(),exp) (1.6) exp ::= BinOp(exp,Sub(),exp) (1.7) We can now justify that the AST (1.1) is an exp in LInt. We know that Call(Name('input_int'),[]) is an exp by rule (1.3), and we have already cat- egorized UnaryOp(USub(), Constant(8)) as an exp, so we apply rule (1.6) to show that BinOp(Call(Name('input_int'),[]),Add(),UnaryOp(USub(),Constant(8))) is an exp in the LInt language. If you have an AST for which these rules do not apply, then the AST is not in LInt. For example, the program input_int() * 8 is not in LInt because there is no rule for the * operator. Whenever we define a language with a grammar, the language includes only those programs that are justified by the grammar rules. The language LInt includes a second nonterminal stmt for statements. There is a statement for printing the value of an expression stmt ::= Expr(Call(Name('print'),[exp])) and a statement that evaluates an expression but ignores the result. stmt ::= Expr(exp) The last grammar rule for LInt states that there is a Module node to mark the top of the whole program: LInt ::= Module(stmt∗ ) The asterisk ∗ indicates a list of the preceding grammar item, in this case a list of statements. The Module class is defined as follows: class Module: def __init__(self, body): self.body = body where body is a list of statements.
  • 20. Preliminaries 5 exp ::= int | input_int() | - exp | exp + exp | exp - exp | (exp) stmt ::= print(exp) | exp LInt ::= stmt∗ Figure 1.1 The concrete syntax of LInt. exp ::= Constant(int) | Call(Name('input_int'),[]) | UnaryOp(USub(),exp) | BinOp(exp,Add(),exp) | BinOp(exp,Sub(),exp) stmt ::= Expr(Call(Name('print'),[exp])) | Expr(exp) LInt ::= Module(stmt∗ ) Figure 1.2 The abstract syntax of LInt. It is common to have many grammar rules with the same left-hand side but different right-hand sides, such as the rules for exp in the grammar of LInt. As shorthand, a vertical bar can be used to combine several right-hand sides into a single rule. The concrete syntax for LInt is shown in figure 1.1 and the abstract syntax for LInt is shown in figure 1.2. We recommend using the parse function in Python’s ast module to convert the concrete syntax into an abstract syntax tree. 1.3 Pattern Matching As mentioned in section 1.1, compilers often need to access the parts of an AST node. As of version 3.10, Python provides the match feature to access the parts of a value. Consider the following example: match ast1_1: case BinOp(child1, op, child2): print(op) In the example above, the match form checks whether the AST (1.1) is a binary operator and binds its parts to the three pattern variables (child1, op, and child2). In general, each case consists of a pattern and a body. Patterns are recursively defined to be one of the following: a pattern variable, a class name followed by a pattern for each of its constructor’s arguments, or other literals such as strings or lists. The body of each case may contain arbitrary Python code. The pattern variables can be used in the body, such as op in print(op).
  • 21. 6 Chapter 1 A match form may contain several clauses, as in the following function leaf that recognizes when an LInt node is a leaf in the AST. The match proceeds through the clauses in order, checking whether the pattern can match the input AST. The body of the first clause that matches is executed. The output of leaf for several ASTs is shown on the right side of the following: def leaf(arith): match arith: case Constant(n): return True case Call(Name('input_int'), []): return True case UnaryOp(USub(), e1): return False case BinOp(e1, Add(), e2): return False case BinOp(e1, Sub(), e2): return False print(leaf(Call(Name('input_int'), []))) print(leaf(UnaryOp(USub(), eight))) print(leaf(Constant(8))) True False True When constructing a match expression, we refer to the grammar definition to identify which nonterminal we are expecting to match against, and then we make sure that (1) we have one case for each alternative of that nonterminal and (2) the pattern in each case corresponds to the corresponding right-hand side of a grammar rule. For the match in the leaf function, we refer to the grammar for LInt shown in figure 1.2. The exp nonterminal has five alternatives, so the match has five cases. The pattern in each case corresponds to the right-hand side of a grammar rule. For example, the pattern BinOp(e1,Add(),e2) corresponds to the right-hand side BinOp(exp,Add(),exp). When translating from grammars to patterns, replace nonterminals such as exp with pattern variables of your choice (such as e1 and e2). 1.4 Recursive Functions Programs are inherently recursive. For example, an expression is often made of smaller expressions. Thus, the natural way to process an entire program is to use a recursive function. As a first example of such a recursive function, we define the function is_exp as shown in figure 1.3, to take an arbitrary value and determine whether or not it is an expression in LInt. We say that a function is defined by structural recursion if it is defined using a sequence of match cases that correspond to a grammar and the body of each case makes a recursive call on each child node.1 We define a second function, named is_stmt, that recognizes whether a value is 1. This principle of structuring code according to the data definition is advocated in the book How to Design Programs by Felleisen et al. (2001).
  • 22. Preliminaries 7 def is_exp(e): match e: case Constant(n): return True case Call(Name('input_int'), []): return True case UnaryOp(USub(), e1): return is_exp(e1) case BinOp(e1, Add(), e2): return is_exp(e1) and is_exp(e2) case BinOp(e1, Sub(), e2): return is_exp(e1) and is_exp(e2) case _: return False def is_stmt(s): match s: case Expr(Call(Name('print'), [e])): return is_exp(e) case Expr(e): return is_exp(e) case _: return False def is_Lint(p): match p: case Module(body): return all([is_stmt(s) for s in body]) case _: return False print(is_Lint(Module([Expr(ast1_1)]))) print(is_Lint(Module([Expr(BinOp(read, Sub(), UnaryOp(Add(), Constant(8))))]))) Figure 1.3 Example of recursive functions for LInt. These functions recognize whether an AST is in LInt. a LInt statement. Finally, figure 1.3 contains the definition of is_Lint, which determines whether an AST is a program in LInt. In general, we can write one recursive function to handle each nonterminal in a grammar. Of the two examples at the bottom of the figure, the first is in LInt and the second is not.
  • 23. 8 Chapter 1 1.5 Interpreters The behavior of a program is defined by the specification of the programming language. For example, the Python language is defined in the Python language ref- erence (Python Software Foundation 2021b) and the CPython interpreter (Python Software Foundation 2021a). In this book we use interpreters to specify each lan- guage that we consider. An interpreter that is designated as the definition of a language is called a definitional interpreter (Reynolds 1972). We warm up by cre- ating a definitional interpreter for the LInt language. This interpreter serves as a second example of structural recursion. The definition of the interp_Lint function is shown in figure 1.4. The body of the function matches on the Module AST node and then invokes interp_stmt on each statement in the module. The interp_stmt function includes a case for each grammar rule of the stmt nonterminal, and it calls interp_exp on each subexpression. The interp_exp function includes a case for each grammar rule of the exp nonterminal. We use several auxiliary functions such as add64 and input_int that are defined in the support code for this book. Let us consider the result of interpreting a few LInt programs. The following program adds two integers: print(10 + 32) The result is 42, the answer to life, the universe, and everything: 42!2 We wrote this program in concrete syntax, whereas the parsed abstract syntax is Module([Expr(Call(Name('print'), [BinOp(Constant(10), Add(), Constant(32))]))]) The following program demonstrates that expressions may be nested within each other, in this case nesting several additions and negations. print(10 + -(12 + 20)) What is the result of this program? The last feature of the LInt language, the input_int operation, prompts the user of the program for an integer. Recall that program (1.1) requests an integer input and then subtracts 8. So, if we run interp_Lint(Module([Expr(Call(Name('print'), [ast1_1]))])) and if the input is 50, the result is 42. We include the input_int operation in LInt so that a clever student cannot implement a compiler for LInt that simply runs the interpreter during compilation to obtain the output and then generates the trivial code to produce the output.3 The job of a compiler is to translate a program in one language into a program in another language so that the output program behaves the same way as the input program. This idea is depicted in the following diagram. Suppose we have 2. The Hitchhiker’s Guide to the Galaxy by Douglas Adams. 3. Yes, a clever student did this in the first instance of this course!
  • 24. Preliminaries 9 def interp_exp(e): match e: case BinOp(left, Add(), right): l = interp_exp(left); r = interp_exp(right) return add64(l, r) case BinOp(left, Sub(), right): l = interp_exp(left); r = interp_exp(right) return sub64(l, r) case UnaryOp(USub(), v): return neg64(interp_exp(v)) case Constant(value): return value case Call(Name('input_int'), []): return input_int() def interp_stmt(s): match s: case Expr(Call(Name('print'), [arg])): print(interp_exp(arg)) case Expr(value): interp_exp(value) def interp_Lint(p): match p: case Module(body): for s in body: interp_stmt(s) Figure 1.4 Interpreter for the LInt language. two languages, L1 and L2, and a definitional interpreter for each language. Given a compiler that translates from language L1 to L2 and given any program P1 in L1, the compiler must translate it into some program P2 such that interpreting P1 and P2 on their respective interpreters with same input i yields the same output o. P1 P2 o compile interp_L2(i) interp_L1(i) (1.8) We establish the convention that if running the definitional interpreter on a pro- gram produces an error, then the meaning of that program is unspecified unless the exception raised is a TrappedError. A compiler for the language is under no
  • 25. 10 Chapter 1 obligation regarding programs with unspecified behavior; it does not have to pro- duce an executable, and if it does, that executable can do anything. On the other hand, if the error is a TrappedError, then the compiler must produce an executable and it is required to report that an error occurred. To signal an error, exit with a return code of 255. The interpreters in chapters 10 and 11 and in section 7.9 use TrappedError. In the next section we see our first example of a compiler. 1.6 Example Compiler: A Partial Evaluator In this section we consider a compiler that translates LInt programs into LInt programs that may be more efficient. The compiler eagerly computes the parts of the program that do not depend on any inputs, a process known as partial evaluation (Jones, Gomard, and Sestoft 1993). For example, given the following program print(input_int() + -(5 + 3) ) our compiler translates it into the program print(input_int() + -8) Figure 1.5 gives the code for a simple partial evaluator for the LInt language. The output of the partial evaluator is a program in LInt. In figure 1.5, the structural recursion over exp is captured in the pe_exp function, whereas the code for partially evaluating the negation and addition operations is factored into three auxiliary functions: pe_neg, pe_add and pe_sub. The input to these functions is the output of partially evaluating the children. The pe_neg, pe_add and pe_sub functions check whether their arguments are integers and if they are, perform the appropriate arithmetic. Otherwise, they create an AST node for the arithmetic operation. To gain some confidence that the partial evaluator is correct, we can test whether it produces programs that produce the same result as the input programs. That is, we can test whether it satisfies the diagram of (1.8). Exercise 1.1 Create three programs in the LInt language and test whether partially evaluating them with pe_Lint and then interpreting them with interp_Lint gives the same result as directly interpreting them with interp_Lint.
  • 26. Preliminaries 11 def pe_neg(r): match r: case Constant(n): return Constant(neg64(n)) case _: return UnaryOp(USub(), r) def pe_add(r1, r2): match (r1, r2): case (Constant(n1), Constant(n2)): return Constant(add64(n1, n2)) case _: return BinOp(r1, Add(), r2) def pe_sub(r1, r2): match (r1, r2): case (Constant(n1), Constant(n2)): return Constant(sub64(n1, n2)) case _: return BinOp(r1, Sub(), r2) def pe_exp(e): match e: case BinOp(left, Add(), right): return pe_add(pe_exp(left), pe_exp(right)) case BinOp(left, Sub(), right): return pe_sub(pe_exp(left), pe_exp(right)) case UnaryOp(USub(), v): return pe_neg(pe_exp(v)) case Constant(value): return e case Call(Name('input_int'), []): return e def pe_stmt(s): match s: case Expr(Call(Name('print'), [arg])): return Expr(Call(Name('print'), [pe_exp(arg)])) case Expr(value): return Expr(pe_exp(value)) def pe_P_int(p): match p: case Module(body): new_body = [pe_stmt(s) for s in body] return Module(new_body) Figure 1.5 A partial evaluator for LInt.
  • 28. 2 Integers and Variables This chapter covers compiling a subset of Python to x86-64 assembly code (Intel 2015). The subset, named LVar, includes integer arithmetic and local variables. We often refer to x86-64 simply as x86. The chapter first describes the LVar language (section 2.1) and then introduces x86 assembly (section 2.2). Because x86 assembly language is large, we discuss only the instructions needed for compiling LVar. We introduce more x86 instructions in subsequent chapters. After introducing LVar and x86, we reflect on their differences and create a plan to break down the translation from LVar to x86 into a handful of steps (section 2.3). The rest of the chapter gives detailed hints regarding each step. We aim to give enough hints that the well- prepared reader, together with a few friends, can implement a compiler from LVar to x86 in a short time. To suggest the scale of this first compiler, we note that the instructor solution for the LVar compiler is approximately 300 lines of code. 2.1 The LVar Language The LVar language extends the LInt language with variables. The concrete syntax of the LVar language is defined by the grammar presented in figure 2.1, and the abstract syntax is presented in figure 2.2. The nonterminal var may be any Python identifier. As in LInt, input_int is a nullary operator, - is a unary operator, and + is a binary operator. Similarly to LInt, the abstract syntax of LVar includes the Module instance to mark the top of the program. Despite the simplicity of the LVar language, it is rich enough to exhibit several compilation techniques. The LVar language includes an assignment statement, which defines a variable for use in later statements and initializes the variable with the value of an expression. The abstract syntax for assignment is defined in figure 2.2. The concrete syntax for assignment is var = exp For example, the following program initializes the variable x to 32 and then prints the result of 10 + x, producing 42. x = 12 + 20 print(10 + x)
  • 29. 14 Chapter 2 exp ::= int | input_int() | - exp | exp + exp | exp - exp | (exp) stmt ::= print(exp) | exp exp ::= var stmt ::= var = exp LVar ::= stmt∗ Figure 2.1 The concrete syntax of LVar. exp ::= Constant(int) | Call(Name('input_int'),[]) | UnaryOp(USub(),exp) | BinOp(exp,Add(),exp) | BinOp(exp,Sub(),exp) stmt ::= Expr(Call(Name('print'),[exp])) | Expr(exp) exp ::= Name(var) stmt ::= Assign([Name(var)], exp) LVar ::= Module(stmt∗ ) Figure 2.2 The abstract syntax of LVar. 2.1.1 Extensible Interpreters via Method Overriding To prepare for discussing the interpreter of LVar, we explain why we implement it in an object-oriented style. Throughout this book we define many interpreters, one for each language that we study. Because each language builds on the prior one, there is a lot of commonality between these interpreters. We want to write down the common parts just once instead of many times. A naive interpreter for LVar would handle the case for variables but dispatch to an interpreter for LInt in the rest of the cases. The following code sketches this idea. (We explain the env parameter in section 2.1.2.) def interp_Lint(e, env): match e: case UnaryOp(USub(), e1): return - interp_Lint(e1, env) ... def interp_Lvar(e, env): match e: case Name(id): return env[id] case _: return interp_Lint(e, env) The problem with this naive approach is that it does not handle situations in which an LVar feature, such as a variable, is nested inside an LInt feature, such as the - operator, as in the following program. y = 10 print(-y)
  • 30. Integers and Variables 15 If we invoke interp_Lvar on this program, it dispatches to interp_Lint to handle the - operator, but then it recursively calls interp_Lint again on its argument. Because there is no case for Name in interp_Lint, we get an error! To make our interpreters extensible we need something called open recursion, in which the tying of the recursive knot is delayed until the functions are composed. Object-oriented languages provide open recursion via method overriding. The fol- lowing code uses method overriding to interpret LInt and LVar using Python class definitions. We define one class for each language and define a method for inter- preting expressions inside each class. The class for LVar inherits from the class for LInt, and the method interp_exp in LVar overrides the interp_exp in LInt. Note that the default case of interp_exp in LVar uses super to invoke interp_exp, and because LVar inherits from LInt, that dispatches to the interp_exp in LInt. class InterpLint: def interp_exp(e): match e: case UnaryOp(USub(), e1): return neg64(self.interp_exp(e1)) ... ... def InterpLvar(InterpLint): def interp_exp(e): match e: case Name(id): return env[id] case _: return super().interp_exp(e) ... We return to the troublesome example, repeated here: y = 10 print(-y) We can invoke the interp_exp method for LVar on the -y expression, which we call e0, by creating an object of the LVar class and calling the interp_exp method InterpLvar().interp_exp(e0) To process the - operator, the default case of interp_exp in LVar dispatches to the interp_exp method in LInt. But then for the recursive method call, it dispatches to interp_exp in LVar, where the Name node is handled correctly. Thus, method overriding gives us the open recursion that we need to implement our interpreters in an extensible way. 2.1.2 Definitional Interpreter for LVar Having justified the use of classes and methods to implement interpreters, we revisit the definitional interpreter for LInt shown in figure 2.3 and then extend it to create an interpreter for LVar, shown in figure 2.4. We change the interp_stmt method in the interpreter for LInt to take two extra parameters named env, which we discuss in the next paragraph, and cont for continuation, which is the technical name for what comes after a particular point in a program. The cont parameter is the list of statements that follow the current statement. Note that interp_stmts invokes interp_stmt on the first statement and passes the rest of the statements as the argument for cont. This organization enables each statement to decide what if
  • 31. 16 Chapter 2 anything should be evaluated after it, for example, allowing a return statement to exit early from a function (see Chapter 8). The interpreter for LVar adds two new cases for variables and assignment. For assignment, we need a way to communicate the value bound to a variable to all the uses of the variable. To accomplish this, we maintain a mapping from variables to values called an environment. We use a Python dictionary to represent the environ- ment. The interp_exp function takes the current environment, env, as an extra parameter. When the interpreter encounters a variable, it looks up the correspond- ing value in the environment. If the variable is not in the environment (because the variable was not defined) then the lookup will fail and the interpreter will halt with an error. Recall that the compiler is not obligated to compile such programs (Section 1.5).1 When the interpreter encounters an assignment, it evaluates the initializing expression and then associates the resulting value with the variable in the environment. The goal for this chapter is to implement a compiler that translates any program P1 written in the LVar language into an x86 assembly program P2 such that P2 exhibits the same behavior when run on a computer as the P1 program interpreted by interp_Lvar. That is, they output the same integer n. We depict this correctness criteria in the following diagram: P1 P2 n compile interp_Lvar interp_x86int Next we introduce the x86Int subset of x86 that suffices for compiling LVar. 2.2 The x86Int Assembly Language Figure 2.5 defines the concrete syntax for x86Int. We use the AT&T syntax expected by the GNU assembler. A program begins with a main label followed by a sequence of instructions. The globl directive makes the main procedure externally visible so that the operating system can call it. An x86 program is stored in the computer’s memory. For our purposes, the computer’s memory is a mapping of 64-bit addresses to 64-bit values. The computer has a program counter (PC) stored in the rip register that points to the address of the next instruction to be executed. For most instructions, the program counter is incremented after the instruction is executed so that it points to the next instruction in memory. Most x86 instructions take two operands, each of which is an integer constant (called an immediate value), a register, or a memory location. 1. In Chapter 5 we introduce type checking rules that prohibit access to undefined variables.
  • 32. Integers and Variables 17 class InterpLint: def interp_exp(self, e, env): match e: case BinOp(left, Add(), right): l = self.interp_exp(left, env) r = self.interp_exp(right, env) return add64(l, r) case BinOp(left, Sub(), right): l = self.interp_exp(left, env) r = self.interp_exp(right, env) return sub64(l, r) case UnaryOp(USub(), v): return neg64(self.interp_exp(v, env)) case Constant(value): return value case Call(Name('input_int'), []): return int(input()) def interp_stmt(self, s, env, cont): match s: case Expr(Call(Name('print'), [arg])): val = self.interp_exp(arg, env) print(val, end='') return self.interp_stmts(cont, env) case Expr(value): self.interp_exp(value, env) return self.interp_stmts(cont, env) case _: raise Exception('error in interp_stmt, unexpected ' + repr(s)) def interp_stmts(self, ss, env): match ss: case []: return 0 case [s, *ss]: return self.interp_stmt(s, env, ss) def interp(self, p): match p: case Module(body): self.interp_stmts(body, {}) def interp_Lint(p): return InterpLint().interp(p) Figure 2.3 Interpreter for LInt as a class.
  • 33. 18 Chapter 2 class InterpLvar(InterpLint): def interp_exp(self, e, env): match e: case Name(id): return env[id] case _: return super().interp_exp(e, env) def interp_stmt(self, s, env, cont): match s: case Assign([Name(id)], value): env[id] = self.interp_exp(value, env) return self.interp_stmts(cont, env) case _: return super().interp_stmt(s, env, cont) def interp_Lvar(p): return InterpLvar().interp(p) Figure 2.4 Interpreter for the LVar language. reg ::= rsp | rbp | rax | rbx | rcx | rdx | rsi | rdi | r8 | r9 | r10 | r11 | r12 | r13 | r14 | r15 arg ::= $int | %reg | int(%reg) instr ::= addq arg,arg | subq arg,arg | negq arg | movq arg,arg | callq label | pushq arg | popq arg | retq x86Int ::= .globl main main: instr∗ Figure 2.5 The syntax of the x86Int assembly language (AT&T syntax). A register is a special kind of variable that holds a 64-bit value. There are 16 general-purpose registers in the computer; their names are given in figure 2.5. A register is written with a percent sign, %, followed by its name, for example, %rax. An immediate value is written using the notation $n where n is an integer. An access to memory is specified using the syntax n(%r), which obtains the address stored in register r and then adds n bytes to the address. The resulting address is used to load or to store to memory depending on whether it occurs as a source or destination argument of an instruction. An arithmetic instruction such as addq s, d reads from the source s and des- tination d, applies the arithmetic operation, and then writes the result to the
  • 34. Integers and Variables 19 .globl main main: movq $10, %rax addq $32, %rax retq Figure 2.6 An x86 program that computes 10 + 32. destination d. The move instruction movq s, d reads from s and stores the result in d. The callq label instruction jumps to the procedure specified by the label, and retq returns from a procedure to its caller. We discuss procedure calls in more detail further in this chapter and in chapter 8. The last letter q indicates that these instructions operate on quadwords, which are 64-bit values. Appendix A.1 contains a reference for all the x86 instructions used in this book. Figure 2.6 depicts an x86 program that computes 10 + 32. The instruction movq $10, %rax puts 10 into register rax, and then addq $32, %rax adds 32 to the 10 in rax and puts the result, 42, into rax. The last instruction retq finishes the main function by returning the integer in rax to the operating system. The oper- ating system interprets this integer as the program’s exit code. By convention, an exit code of 0 indicates that a program has completed successfully, and all other exit codes indicate various errors. We exhibit the use of memory for storing intermediate results in the next example. Figure 2.7 lists an x86 program that computes 52 + -10. This program uses a region of memory called the procedure call stack (stack for short). The stack consists of a separate frame for each procedure call. The memory layout for an individual frame is shown in figure 2.8. The register rsp is called the stack pointer and contains the address of the item at the top of the stack. In general, we use the term pointer for something that contains an address. The stack grows downward in memory, so we increase the size of the stack by subtracting from the stack pointer. In the context of a procedure call, the return address is the location of the instruction that immediately follows the call instruction on the caller side. The function call instruction, callq, pushes the return address onto the stack prior to jumping to the procedure. The register rbp is the base pointer and is used to access variables that are stored in the frame of the current procedure call. The base pointer of the caller is stored immediately after the return address. Figure 2.8 shows the memory layout of a frame with storage for n variables, which are numbered from 1 to n. Variable 1 is stored at address –8(%rbp), variable 2 at –16(%rbp), and so on. In the program shown in figure 2.7, consider how control is transferred from the operating system to the main function. The operating system issues a callq main instruction that pushes its return address on the stack and then jumps to main. In x86-64, the stack pointer rsp must be divisible by 16 bytes prior to the execution of any callq instruction, so that when control arrives at main, the rsp is 8 bytes
  • 35. 20 Chapter 2 .globl main main: pushq %rbp movq %rsp, %rbp subq $16, %rsp movq $10, -8(%rbp) negq -8(%rbp) movq -8(%rbp), %rax addq $52, %rax addq $16, %rsp popq %rbp retq Figure 2.7 An x86 program that computes 52 + -10. Position Contents 8(%rbp) return address 0(%rbp) old rbp –8(%rbp) variable 1 –16(%rbp) variable 2 … … 0(%rsp) variable n Figure 2.8 Memory layout of a frame. out of alignment (because the callq pushed the return address). The first three instructions are the typical prelude for a procedure. The instruction pushq %rbp first subtracts 8 from the stack pointer rsp and then saves the base pointer of the caller at address rsp on the stack. The next instruction movq %rsp, %rbp sets the base pointer to the current stack pointer, which is pointing to the location of the old base pointer. The instruction subq $16, %rsp moves the stack pointer down to make enough room for storing variables. This program needs one variable (8 bytes), but we round up to 16 bytes so that rsp is 16-byte-aligned, and then we are ready to make calls to other functions. The first instruction after the prelude is movq $10, -8(%rbp), which stores 10 in variable 1. The instruction negq -8(%rbp) changes the contents of variable 1 to –10. The next instruction moves the –10 from variable 1 into the rax register. Finally, addq $52, %rax adds 52 to the value in rax, updating its contents to 42. The conclusion of the main function consists of the last three instructions. The first two restore the rsp and rbp registers to their states at the beginning of the procedure. In particular, addq $16, %rsp moves the stack pointer to point to the old base pointer. Then popq %rbp restores the old base pointer to rbp and adds 8
  • 36. Integers and Variables 21 reg ::= 'rsp' | 'rbp' | 'rax' | 'rbx' | 'rcx' | 'rdx' | 'rsi' | 'rdi' | 'r8' | 'r9' | 'r10' | 'r11' | 'r12' | 'r13' | 'r14' | 'r15' arg ::= Immediate(int) | Reg(reg) | Deref(reg,int) instr ::= Instr('addq',[arg,arg]) | Instr('subq',[arg,arg]) | Instr('movq',[arg,arg]) | Instr('negq',[arg]) | Instr('pushq',[arg]) | Instr('popq',[arg]) | Callq(label,int) | Retq() | Jump(label) x86Int ::= X86Program(instr∗ ) Figure 2.9 The abstract syntax of x86Int assembly. to the stack pointer. The last instruction, retq, jumps back to the procedure that called this one and adds 8 to the stack pointer. Our compiler needs a convenient representation for manipulating x86 programs, so we define an abstract syntax for x86, shown in figure 2.9. We refer to this lan- guage as x86Int. The main difference between this and the concrete syntax of x86Int (figure 2.5) is that labels, instruction names, and register names are explicitly rep- resented by strings. Regarding the abstract syntax for callq, the Callq AST node includes an integer for representing the arity of the function, that is, the number of arguments, which is helpful to know during register allocation (chapter 4). 2.3 Planning the Trip to x86 To compile one language to another, it helps to focus on the differences between the two languages because the compiler will need to bridge those differences. What are the differences between LVar and x86 assembly? Here are some of the most important ones: 1. x86 arithmetic instructions typically have two arguments and update the second argument in place. In contrast, LVar arithmetic operations take two arguments and produce a new value. An x86 instruction may have at most one memory- accessing argument. Furthermore, some x86 instructions place special restrictions on their arguments. 2. An argument of an LVar operator can be a deeply nested expression, whereas x86 instructions restrict their arguments to be integer constants, registers, and memory locations. 3. A program in LVar can have any number of variables, whereas x86 has 16 registers and the procedure call stack. We ease the challenge of compiling from LVar to x86 by breaking down the problem into several steps, which deal with these differences one at a time. Each of these steps is called a pass of the compiler. This term indicates that each step passes over, or traverses, the AST of the program. Furthermore, we follow the nanopass approach,
  • 37. 22 Chapter 2 which means that we strive for each pass to accomplish one clear objective rather than two or three at the same time. We begin by sketching how we might implement each pass and give each pass a name. We then figure out an ordering of the passes and the input/output language for each pass. The very first pass has LVar as its input language, and the last pass has x86Int as its output language. In between these two passes, we can choose whichever language is most convenient for expressing the output of each pass, whether that be LVar, x86Int, or a new intermediate language of our own design. Finally, to implement each pass we write one recursive function per nonterminal in the grammar of the input language of the pass. Our compiler for LVar consists of the following passes: remove_complex_operands ensures that each subexpression of a primitive opera- tion or function call is a variable or integer, that is, an atomic expression. We refer to nonatomic expressions as complex. This pass introduces temporary variables to hold the results of complex subexpressions. select_instructions handles the difference between LVar operations and x86 instructions. This pass converts each LVar operation to a short sequence of instructions that accomplishes the same task. assign_homes replaces variables with registers or stack locations. The next question is, in what order should we apply these passes? This question can be challenging because it is difficult to know ahead of time which orderings will be better (that is, will be easier to implement, produce more efficient code, and so on), and therefore ordering often involves trial and error. Nevertheless, we can plan ahead and make educated choices regarding the ordering. The select_instructions and assign_homes passes are intertwined. In chap- ter 8 we learn that in x86, registers are used for passing arguments to functions and that it is preferable to assign parameters to their corresponding registers. This suggests that it would be better to start with the select_instructions pass, which generates the instructions for argument passing, before performing register allocation. On the other hand, by selecting instructions first we may run into a dead end in assign_homes. Recall that only one argument of an x86 instruction may be a memory access, but assign_homes might be forced to assign both argu- ments to memory locations. A sophisticated approach is to repeat the two passes until a solution is found. However, to reduce implementation complexity we rec- ommend placing select_instructions first, followed by the assign_homes, and then a third pass named patch_instructions that uses a reserved register to fix outstanding problems. Figure 2.10 presents the ordering of the compiler passes and identifies the input and output language of each pass. The output of the select_instructions pass is the x86Var language, which extends x86Int with an unbounded number of program- scope variables and removes the restrictions regarding instruction arguments. The last pass, prelude_and_conclusion, places the program instructions inside a main function with instructions for the prelude and conclusion. The remainder of this chapter provides guidance on the implementation of each of the compiler passes represented in figure 2.10.
  • 38. Integers and Variables 23 LVar Lmon Var x86Var x86Var x86Int x86Int remove_complex_operands select_instructions assign_homes patch_instructions prelude_and_conclusion Figure 2.10 Diagram of the passes for compiling LVar. atm ::= Constant(int) | Name(var) exp ::= atm | Call(Name('input_int'),[]) | UnaryOp(USub(),atm) | BinOp(atm,Add(),atm) | BinOp(atm,Sub(),atm) stmt ::= Expr(Call(Name('print'),[atm])) | Expr(exp) | Assign([Name(var)], exp) Lmon Var ::= Module(stmt∗ ) Figure 2.11 Lmon Var is LVar with operands restricted to atomic expressions. 2.4 Remove Complex Operands The remove_complex_operands pass compiles LVar programs into a restricted form in which the arguments of operations are atomic expressions. Put another way, this pass removes complex operands, such as the expression -10 in the following program. This is accomplished by introducing a new temporary variable, assigning the complex operand to the new variable, and then using the new variable in place of the complex operand, as shown in the output of remove_complex_operands on the right. x = 42 + -10 print(x + 10) ⇒ tmp_0 = -10 x = 42 + tmp_0 tmp_1 = x + 10 print(tmp_1) Figure 2.11 presents the grammar for the output of this pass, the language Lmon Var . The only difference is that operator arguments are restricted to be atomic expres- sions that are defined by the atm nonterminal. In particular, integer constants and variables are atomic. The atomic expressions are pure (they do not cause or depend on side effects) whereas complex expressions may have side effects, such as
  • 39. 24 Chapter 2 Call(Name('input_int'),[]). A language with this separation between pure expressions versus expressions with side effects is said to be in monadic normal form (Moggi 1991; Danvy 2003), which explains the mon in the name Lmon Var . An important invariant of the remove_complex_operands pass is that the relative ordering among complex expressions is not changed, but the relative ordering between atomic expressions and complex expressions can change and often does. These changes are behavior preserving because atomic expressions are pure. We recommend implementing this pass with an auxiliary method named rco_exp with two parameters: an LVar expression and a Boolean that specifies whether the expression needs to become atomic or not. The rco_exp method should return a pair consisting of the new expression and a list of pairs, associating new temporary variables with their initializing expressions. Returning to the example program with the expression 42 + -10, the subexpres- sion -10 should be processed using the rco_exp function with True as the second argument, because -10 is an argument of the + operator and therefore needs to become atomic. The output of rco_exp applied to -10 is as follows. -10 ⇒ tmp_1 [(tmp_1, -10)] Take special care of programs, such as the following, that assign an atomic expression to a variable. You should leave such assignments unchanged, as shown in the program on the right: a = 42 b = a print(b) ⇒ a = 42 b = a print(b) A careless implementation might produce the following output with unnecessary temporary variables. tmp_1 = 42 a = tmp_1 tmp_2 = a b = tmp_2 print(b) Exercise 2.1 Implement the remove_complex_operands pass in compiler.py, cre- ating auxiliary functions for each nonterminal in the grammar, that is, rco_exp and rco_stmt. We recommend that you use the function utils.generate_name() to generate fresh names from a stub string. Exercise 2.2 Create five LVar programs that exercise the most interesting parts of the remove_complex_operands pass. The five programs should be placed in the subdirectory tests/var, and the file names should end with the file extension .py.
  • 40. Integers and Variables 25 Run the run-tests.py script in the support code to check whether the output programs produce the same result as the input programs. 2.5 Select Instructions In the select_instructions pass we begin the work of translating to x86Var. The target language of this pass is a variant of x86 that still uses variables, so we add an AST node of the form Variable(var) to the arg nonterminal of the x86Int abstract syntax (figure 2.9). We recommend implementing an auxiliary function named select_stmt for the stmt nonterminal. Next consider the cases for the stmt nonterminal, starting with arithmetic opera- tions. For example, consider the following addition operation, on the left side. (Let arg1 and arg2 be the translations of atm1 and atm2, respectively.) There is an addq instruction in x86, but it performs an in-place update. So, we could move arg1 into the rax register, then add arg2 to rax, and then finally move rax into var. var = atm1 + atm2 ⇒ movq arg1, %rax addq arg2, %rax movq %rax, var However, with some care we can generate shorter sequences of instructions. Suppose that one or more of the arguments of the addition is the same variable as the left- hand side of the assignment. Then the assignment statement can be translated into a single addq instruction, as follows. var = atm1 + var ⇒ addq arg1, var On the other hand, if atm2 is not the same variable as the left-hand side, then we can move arg1 into the left-hand var and then add arg2 to var. var = atm1 + atm2 ⇒ movq arg1, var addq arg2, var The input_int operation does not have a direct counterpart in x86 assembly, so we provide this functionality with the function read_int in the file runtime.c, written in C (Kernighan and Ritchie 1988). In general, we refer to all the func- tionality in this file as the runtime system, or simply the runtime for short. When compiling your generated x86 assembly code, you need to compile runtime.c to runtime.o (an object file, using gcc with option -c) and link it into the executable. For our purposes of code generation, all you need to do is translate an assignment of input_int into a call to the read_int function followed by a move from rax to the left-hand side variable. (The return value of a function is placed in rax.) var = input_int(); ⇒ callq read_int movq %rax, var
  • 41. 26 Chapter 2 Similarly, we translate the print operation, shown below, into a call to the print_int function defined in runtime.c. In x86, the first six arguments to func- tions are passed in registers, with the first argument passed in register rdi. So we move the arg into rdi and then call print_int using the callq instruction. print(atm) ⇒ movq arg, %rdi callq print_int We recommend that you use the function utils.label_name to transform strings into labels, for example, in the target of the callq instruction. This practice makes your compiler portable across Linux and Mac OS X, which requires an underscore prefixed to all labels. Exercise 2.3 Implement the select_instructions pass in compiler.py. Create three new example programs that are designed to exercise all the interesting cases in this pass. Run the run-tests.py script to check whether the output programs produce the same result as the input programs. 2.6 Assign Homes The assign_homes pass compiles x86Var programs to x86Var programs that no longer use program variables. Thus, the assign_homes pass is responsible for placing all the program variables in registers or on the stack. For runtime efficiency, it is better to place variables in registers, but because there are only sixteen registers, some programs must necessarily resort to placing some variables on the stack. In this chapter we focus on the mechanics of placing variables on the stack. We study an algorithm for placing variables in registers in chapter 4. Consider again the following LVar program from section 2.4: a = 42 b = a print(b) The output of select_instructions is shown next, on the left, and the output of assign_homes is on the right. In this example, we assign variable a to stack location -8(%rbp) and variable b to location -16(%rbp). movq $42, a movq a, b movq b, %rax ⇒ movq $42, -8(%rbp) movq -8(%rbp), -16(%rbp) movq -16(%rbp), %rax The assign_homes pass should replace all uses of variables with stack locations. In the process of assigning variables to stack locations, it is convenient for you to compute and store the size of the frame (in bytes) in the field stack_space of the X86Program node, which is needed later to generate the conclusion of the main procedure. The x86-64 standard requires the frame size to be a multiple of 16 bytes.
  • 42. Random documents with unrelated content Scribd suggests to you:
  • 43. A Scotland under Charles the Second. t the death of Cromwell there was not, in the general aspect of political matters, any definite forecast of what twelve months after would be the form of government; certainly an easy and unopposed restoration of the Stuart monarchy was about the last idea, warranted by the history of the previous fifteen years. But one man, the still-tongued, close-minded General Monk, solved the question. By his influence as head of the army, and his tact and sagacity in party wire-pulling, he so managed that within eight months of the Protector’s death, Charles II. was quietly proclaimed King of Great Britain and Ireland. It was a twenty-seven years of as mean rule, as has ever darkened the pages of British history. Retaliations and persecutions—one long attempt to turn back the stream of progress—a corrupt court, leavening the national life with foulness and frivolity, such might be the general headings of the chapters chronicling the reign of the “Merry Monarch.” The restoration was in England baptized in blood. Ten “regicides” were hanged at Charing Cross. This was harsh—revengeful; but not despicable or unprecedented. But it is with disgust, with shame for our common humanity, that we learn that the bodies of Cromwell, Ireton, and Bradshaw were taken from their graves in Westminster Abbey, and on the death anniversary (30th January) of “King Charles the Martyr,” drawn on hurdles to Tyburn, and there hung on the gallows; then the heads cut off and fixed on Westminster Hall. And Scotland must not be left without examples of severity. The Marquis of Argyle was the first victim. At the coronation of Charles at Scone, he was the noble who placed the crown on the king’s head. But Charles hated him as a leader of the presbyterians, who then
  • 44. held him in irksome tutelage. After a most unfair trial, nothing tangible being found against him except some private letters to General Monk, in which he expressed himself favourable to Cromwell, he was found guilty, and condemned to death. He met his fate with great firmness, saying that if he could not brave death like a Roman, he could submit to it like a Christian. Other victims followed. Swinburne has said of Mary of Scotland, “A kinder or more faithful friend, a deadlier or more dangerous enemy, it would be impossible to dread or to desire.” Mary’s descendants were noways remarkable for fidelity in friendship, but they were implacable in their hatreds. When he was in the over-careful hands of the Covenanters, Charles had treasured up against a day of vengeance, many affronts, brow-beatings, and intimidations, and now he meant, in his stubborn way, to demand payment, with heavy interest, of the old debts. And so Charles, the Covenanted King of Scotland, and in whose cause its best blood had been shed, had nothing but hatred for the land of his fathers, and for its presbyterian faith. A packed and subservient Scottish Parliament proceeded to pass, first a Rescissory Act, rescinding all statutes, good and bad, which had been passed since the commencement of the civil wars; and next, an Act of Supremacy, making the king supreme judge in all matters, both civil and ecclesiastical. Charles soon made it evident that he meant to establish episcopacy. James Sharpe, minister of the little Fifeshire town of Crail, was sent to London to look after presbyterian interests; he was got at on the selfish side, and made archbishop of St. Andrews. Nine other pliant Scottish ministers received episcopal ordination in Westminster Abbey. On the third anniversary of the Restoration, 29th May, 1662, copies of the Covenants were in Edinburgh publicly torn to pieces by the common hangman. The ministers were ordered to attend diocesan meetings, and to acknowledge the authority of their bishops. The majority acquiesced; but it is pleasing to learn that nearly four hundred resigned their livings, rather than submit to the prelatic yoke. To take the places of the recusants, a hosts of curates,
  • 45. often persons of mean character and culture, were ordained. The people did not like the men thus thrust upon them as ministers, and they still sought the services of their old pastors; hence originated the “conventicles,” a contemptuous title for a meeting-place of dissenters. And now began, chiefly in the west and south of Scotland, those field meetings which afterwards became so notable. At first they were simply assemblies for worship, no arms were worn; after service a quiet dispersal. But, as signifying nonconformity to prescribed forms, they gave great offence. A new Act forbade, under punishment for sedition, any preaching without the sanction of the bishops; and inflicting pains and penalties on all persons absenting themselves from their parish churches. If fines were not paid, soldiers were quartered on the recusants, and their cattle, furniture, and very clothing were sold. It was even accounted seditious to give sustenance to the ejected ministers. It can be easily asked, why did this Scottish people, with the memory of their past, submit to these things? There was, as in England, a reaction to an extreme of loyalty; there was the satisfaction of finding themselves freed from English domination in its tangible form of Cromwell’s troops and garrisons; there was the pleasure of once more seeing a Parliament in Edinburgh, even though it merely registered and gave legal form to the king’s decrees. They were told that the advantage of being governed by their own native prince implied as its price the establishment of that prince’s form of religious faith. Their own nobles and many of their ministers had conformed; and thus bereft of their natural leaders, there was weakness and division. Despite of all these discouragements, they were often goaded into insurrections; which were cruelly suppressed, and made the excuses for further intolerance, and still harsher persecutions. The field conventicles continued. In the solitudes of nature, in lonely glens, or on pine-shaded hillsides, with sentinels posted on the heights, arose the solemn psalm, and the preachers prayer and exhortation. And men now came armed to these gatherings, the
  • 46. women had to be defended, force was to be met by force. To suppress such meetings, troops were sent into the insubordinate districts, under a wild fanatical Royalist, General Dalziel, and had free quarters on the inhabitants. By 1666, a reign of terror was fully inaugurated; Dalziel flared like a baleful meteor over the West of Scotland. In November of this year, without concert or premeditation, an open insurrection broke out. At Dalry, in Ayrshire, four soldiers were grossly maltreating an aged man, and common humanity could not stand by and look on with indifference or mere sympathy. The people rescued the old man, disarmed the soldiers, and took their officer prisoner to Dumfries. A resolution was suddenly taken to march on Edinburgh. They gathered in a fortnight’s march to barely 2000 men, and wearied and worn out, encamped on a plateau, called Rullion Green, on the Pentland hills, a few miles south of Edinburgh. Here they were attacked by double their numbers under Dalziel, and, after a gallant resistance, considering their inferior arms and discipline, were put to flight. Some fifty were killed on the field, one hundred and thirty were taken prisoners, thirty-four of whom were, chiefly at the instigation of Archbishop Sharpe, hanged as rebels, and the rest banished. THUMBIKINS. (From the Scottish Antiquarian Museum.)
  • 47. And tortures—such as have had no place in modern history since the palmy days of the Spanish Inquisition were inflicted to extort confessions of complicity in a rising, which was really the offspring of momentary excitement. Thumbikins squeezed the fingers by iron screws. These tortures were generally borne with heroic patience and resolution. One young minister, Hugh McKail, comely in person, well educated, an enthusiast in his covenanting faith, was subjected to the torture of the boot. His leg was crushed, but he uttered no cry, only moving his lips in silent prayer. He had taken very little part in the insurrection, but was condemned to death. On the gallows- ladder his last words were:—“Farewell father, mother, and all my friends in life, farewell earth and all its delights, farewell sun, moon, and stars, welcome death, glory, and eternal life.” Seeing what impressions such words made on the listeners, in after executions drums were beaten to drown the voices of the sufferers. A weary ten years ensued of alternate “indulgence,” and renewed intolerance. In 1667, the Duke of Lauderdale was placed at the head of Scottish affairs. He had subscribed to the covenant, and had been a Presbyterian representative at the Westminster Assembly. He was now a subservient courtier, but did not at first assume the role of a persecutor. He disbanded the army, and proclaimed an indemnity to those who had fought at Rullion Green, on their signing a bond of peace. The ministers ousted from their parishes were permitted to return, but on conditions which the strict consciences of many could not accept; and those who did accept were placed under close surveillance, and under severe penalties forbidden to take part in any field meetings. Some of the bishops were good men, striving earnestly to make peace within the church. One of these, Leighton, Bishop of Dunblane, made an attempt to reconcile Presbyterianism with a modified episcopacy. The bishops were merely to sit as chairmen, or moderators, in the diocesan convocations, and to have no veto on the proceedings, but the Covenanters thought this a snare for entrapping them into an acknowledgment of prelacy, and the idea was abandoned.
  • 48. And Lauderdale who had begun his rule leniently, now afraid of being represented to the King as lukewarm in his service, blossomed out into a cruel persecutor, forcibly suppressing field meetings, and enforcing extreme penalties on nonconformists. It has been estimated that up to this date seventeen thousand persons had suffered in fine, imprisonment, and death. It was said that fines extorted for non-attendance at the parish churches, were applied to supply the extravagance of Lady Lauderdale,—a rapacious, bad, clever woman. Landowners were required under penalties to become bound for their tenants, that they would attend their parish churches, take no part in conventicles, and not relieve outlawed persons. The gentry generally refused to enter into such bonds; and Lauderdale wrote to the King that the country was in a state of incipient rebellion, and required reduction by force of arms. He treated the whole of the west country as if in open revolt. Not only did he send ordinary troops with field artillery into the devoted districts, but he brought down from the hills a Highland host of 9000 men to live upon, and with every encouragement to plunder and oppress, the poor people. Speaking an unknown tongue, strange in manners and attire, they were to the lowlanders a veritable plague of human locusts. When, after a few months of free quarterage, they went back to their hills, themselves and a number of horses were loaded with booty, as if from the sack of a rich town. But so far as we can learn they were not guilty of personal violence upon those they were sent to despoil; perhaps in this respect hardly coming up to the wishes and expectations of their employers. In May, 1679, occurred a deed of blood which widened the gulf between the Covenanters and the government, and gave legal colouring to harshness and persecution. In Fifeshire, one Carmichael had become especially obnoxious as a cruel persecutor, and an active commissioner for receiving the fines laid upon the malcontents. On 3rd May, a party of twelve men, chiefly small farmers in the district, with David Hackston of Rathillet and John Balfour of Burley as the leaders, lay in wait for Carmichael, with full
  • 49. purpose to slay him. It appears he had received some warning, and kept out of the way. After waiting long, the band were, in sullen disappointment, preparing to separate, when the carriage of Sharpe, the Archbishop, appeared unexpectedly, conveying him and his daughter home to St. Andrews. To these superstitious men, nursed under persecution by old biblical texts into religious fanaticism, it appeared as if an act of necessary vengeance was here thrust upon them, that instead of an inferior agent, a foremost persecutor, who had hounded to the death many of their brethren, was now delivered into their hands. They took him from his carriage, and there on Magus Muir—suing upon his knees for mercy, his grey hairs, and his daughter’s anguished cries, also pleading for his life— they slew him with many sword thrusts. A general cry of horror and repudiation rang through the land. It was a savage murder; but so had been the deaths of hundreds of persons more innocent than he of offences against justice and common right. More severe measures of repression were taken; new troops were raised, and the officers instructed to act with the utmost rigour. And the Covenanters grew desperate; they assembled in greater numbers, were more fully armed, and more defiant in their language. On 29th May, the anniversary of the Restoration, a mounted party entered the village of Rutherglen, about two miles from Glasgow. They extinguished the festive bonfire, held a service of denunciatory psalms, prayers, and exhortations in the market place, and burned the Acts which had been issued against the Covenant. In quest of the insurgents, and to avenge the affront on the government, a body of cavalry rode out of Glasgow barracks, on the 1st of June. Their leader was a distinguished soldier—a man of courage and gallant bearing, John Graham of Claverhouse— afterwards, for his services in the royal cause, created Viscount Dundee. In the annals of Scotland there is no name amongst the unworthiest of her sons,—Monteith the betrayer of Wallace, Cardinal Beaton, the ruthless persecutor, Dalziel, with a monomania for murder and oppression,—so utterly detestable as that of the dashing
  • 50. cavalier, Claverhouse. His portrait is that of a haughty, self-centred man; one would think too proud for the meanly savage work he was set to do, but which, with fell intensity, he seemed to revel in doing. In the conflict, he appeared to have a charmed life, and in these superstitious times he was believed to have made a paction with Satan:—for doing the fiend’s work he was to have so many years immunity from death: neither lead nor steel could harm him. It was said that his mortal wound, received in the moment of victory at Killiecrankie, was from being shot by a silver bullet. Claverhouse, in quest of the demonstrators at Rutherglen, came, at Drumclog, about twenty miles south of Glasgow, on the body of insurgents; about fifty horsemen fairly well appointed, as many infantry with fire-arms, and a number armed with pikes, scythes, and pitch-forks. The Covenanters had skilfully posted themselves; a morass and broad ditch in front, the infantry in the centre, a troop of horse on each flank. Claverhouse’s call to surrender was answered by the singing of a verse of a warlike psalm. The troops gave a loud cheer, and rode into the morass; they found it impassable and themselves under a steady fire from the Covenanters. Claverhouse sent flanking parties to right and left. These were boldly met before they had time to form after crossing the ditch, and nearly cut to pieces. And then the Covenanters made a sudden rush, and after a desperate defence by Claverhouse, they utterly routed him,—the only battle he ever lost. This victory of the Covenanters over regular troops, ably commanded, was a general surprise, and it found the victors ill- prepared to follow it up to advantage. They next day occupied Hamilton, and, reinforced by numbers, proceeded to attack Glasgow. They were at first beaten back by Claverhouse, but he thought it advisable to retreat to Edinburgh; and then the insurgents occupied Glasgow. The King meanwhile had sent the Duke of Monmouth—a courteous and courageous gentleman,—albeit the bar sinister ran through his escutcheon—to collect an army to quell the rebellion. On 21st June the Covenanters—who had now their headquarters near Hamilton, on the south-western bank of the Clyde, learned that the
  • 51. Duke, at the head of a powerful army, was advancing towards Bothwell Bridge—crossing which he would be upon them. In the face of the common enemy, polemical disputes between the different presbyterian parties brought confusion into their councils. The moderate party drew up a supplication to the Duke, describing their many grievances, and asking that they be submitted to a free parliament. The Duke sent a courteous reply, expressing sympathy, and offering to intercede for them with the King,—but they must first lay down their arms. This condition the extreme party would not listen to, and at this most unsuitable moment, they nominated fresh officers—men indisposed to acknowledge any allegiance to the King, or, in matters appertaining to religion, to submit to the civil power. Under Rathillet, Burley and other irreconcilables, 300 men were posted to hold the bridge; they made a stout defence; but it was forced at the point of the bayonet. Bishop Burnet says,—“The main body of the insurgents had not the grace to submit, the courage to fight, nor the sense to run away.” But when the cannon began to make havoc in their ranks, and they saw the deadly array of horsemen, and the serried ranks of disciplined infantry preparing to charge—they threw down their arms, and became a mob of fugitives. And now Claverhouse had to avenge Drumclog. His war-cry on that day had been “No Quarter,” and this was his intention at Bothwell Bridge. Four hundred were killed on the field and in the flight, but the strict orders of the Duke were “Give quarter to all who surrender—make prisoners, but spare life;” and thus the relentless swords of Claverhouse and Dalziel were stayed. With the indignation of a true soldier, Monmouth rejected a proposal to burn Hamilton and to devastate the surrounding country; and he issued a proclamation promising pardon to all who made their submission by a certain day. But the milder spirit of Monmouth found no place in the treatment of the prisoners taken at Bothwell. They were marched to Edinburgh, suffering much on the way; there, 1200 men were huddled together without shelter in the Greyfriars churchyard—
  • 52. sleeping amongst the tombs upon the bare ground. Several supposed leaders were executed, some escaped further misery by death from exposure, others were set free on signing a declaration never to take arms against the King, and 257 were sent as slaves to Barbadoes. And meantime Claverhouse was passing as a destroying angel through the western shires. Making little distinction between those who had, and those who had not, taken part in the late insurrection —he seized the property, and imprisoned or put to death, all against whom any charge of contumacy could be laid. The hunted Covenanters were driven into wilder seclusions, and their barbarous treatment naturally made them more aggressive and extravagant in their language. Useless to talk to men frenzied to despair of loyalty to a King, who, in his life of unhallowed pleasure in distant London, heard not, or cared not, for the bitter cry of the people whose rights he had sworn to protect. When they met at midnight in lonely glen or trackless moor, the leaders, Cameron, Cargill, Renwick, and others, would, like the Hebrew Prophets of old, mingle prophecy with denunciation; their high-strung enthusiasm bordered on insanity. Cameron and Cargill published a declaration denouncing Charles, calling on all true sons of the Covenant to throw off their allegiance, and take up arms against him. And government had now a pretext for putting Scotland under what was really martial law. The common soldiers were authorised to put to death, without any pretence of trial, all who refused to take the prescribed oath, or to answer all interrogations. It was a capital crime to have any intercourse with prescribed persons; and torture was inflicted, even on women, to extort the whereabouts of these persons. At Wigtown, Margaret McLauchlan, a widow of sixty-three years, and Margaret Wilson, a girl of eighteen, were drowned by being bound to stakes within flood-mark. Amongst many murders perpetrated at this time, that of John Brown, the Ayrshire carrier, stands out conspicuous in horror. He was a quiet, sedate man, leading a blameless life; his only offence was
  • 53. that he did not on Sundays attend the parish church, but either read his bible at home, or, with a few like-minded, met in a quiet place for a little service of praise and prayer. One morning, whilst digging peats for the house fire, he was surrounded by Claverhouse’s dragoons, and brought to his own door. Here, his wife and children being by—a baby in its mother’s arms—Claverhouse asked him why he did not attend on the King’s curate; and John, answering that he had to obey his conscience rather than the King, Claverhouse told him to prepare for death. He said he had long been so prepared. He prayed with fervour, until interrupted by Claverhouse, who saw his wild dragoons beginning to shew tokens of sympathy; Brown kissed his wife and little ones, and he was then shot dead. “What do you think of your bonnie man now?” the devil-hearted slayer asked of the newly-made widow. “I aye thocht muckle o’ him, but never sae muckle as I do this day.” She laid her infant on the ground, tied up the poor shattered head in her kerchief, composed the limbs, covered the body with a plaid, and then she sat down beside it, and, in heart-rending sobs and tears, gave full course to natural sorrow. The tragedy enacted on Magus Moor was a cruel murder, but if there are degrees of guilt in such an awful crime, that committed at the cottage door in Ayrshire was surely the more heinous and atrocious of the two. Monmouth remained only a short time in Scotland; Lauderdale was still nominally at the head of affairs. But in November, 1679, the King sent his brother James to Edinburgh, partly to keep him out of sight from the people of England. As a rigid Roman Catholic, standing next in succession to the throne, he was very unpopular. A cry of popish plots had been got up, and an Exclusion Bill would have been carried in Parliament,[4] but Charles dissolved it, and he never called another; for the last four years of his life he reigned as an absolute monarch. James, a royal Stuart, residing in long untenanted Holyrood, was made much of by the Scottish nobility and gentry, and to conciliate them he so far unbent his generally sombre and unamiable demeanour. He paid particular attention to the Highland chieftains,
  • 54. and thus laid a foundation for that loyalty to himself and his descendants, so costly to the clansmen. But his presence and his influence in public affairs did no good to the poor Covenanters. Against nonconformity of every shade his only remedies were persecution and suppression. The poor wanderers of the Covenant were hunted as wild beasts. Richard Cameron was slain at Aire Moss. Hackston and Cargill were hanged. It is said that James often amused his leisure hours by witnessing the tortures of the boot and the thumb-screw. And not the common people only were thus vexed and harassed. Strangely-worded oaths, acknowledging the laws and statutes, and also the King’s supremacy, were administered to all holding official positions. When, as a privy counsellor, the oath was tendered to the Earl of Argyle—son of the Marquis who was beheaded at the commencement of the reign—he declared he took it so far as it was consistent with itself, and with the Protestant religion. For adding this qualification, he was tried for, and found guilty of, high treason. He contrived to escape from Edinburgh Castle in the disguise of a page holding up his step-daughter’s train. He reached Holland, a sentence of death hanging over him. And in England, after dismissing the Oxford parliament, the King was despotic. If he had any religious faith at all, it was towards Catholicism, and thus he took up his brother’s quarrel. In the administration of justice, juries were packed, and judges were venal. London was adjudged to have illegally extended its political powers, was fined heavily, and condemned to lose its charters. Breaches of their charters by provincial towns were looked for, and something was generally found sufficient to raise prosecutions upon, the award being always given for the Crown. Fines were levied for the King’s private advantage, and by his veto in the election of magistrates he held in his hand Parliamentary elections. The university of Oxford issued a solemn decree, affirming unlimited submission to the Royal authority; and the most detestable of the very few judges whose names are a stain upon the history of English jurisprudence— Jeffreys—was the very incarnation of venality and injustice; he was a
  • 55. vulgar bully, ever finding a demoniacal pleasure in cruelty and wrong-doing. The country had been sickened of civil war, and public spirit seemed to have deserted the land. Still the Whig leaders of the late majority in Parliament made some attempts at organizing resistance. Shaftesbury was for immediate rebellion; but Lords Essex, Howard, and William Russell, and Algernon Sidney, more cautiously resolved to wait the course of events, and act when an opportunity arose. They certainly meant an insurrection in London, to be supported by a rising in the West of England, and another in Scotland under the Earl of Argyle. But a conspiracy in a lower stratum of political influence, called the Ryehouse Plot, which proposed the deaths of the King and his brother, having been divulged to the Government, and certain arrests made, the prisoners, to save themselves, declared that Lords Howard and Russell, and Sidney, Hampden (a grandson of the John Hampden of ship-money fame), and others were implicated. Howard —recreant to the traditions of his name—turned approver. Lord William Russell was tried for treason—nobly supported by his wife— and although the evidence against him was weak, a packed jury convicted him, and he was beheaded at Lincoln’s Inn Fields. Sidney was tried by Judge Jeffreys. Howard was the only witness against him, and for a conviction of treason the law required at least two witnesses; but a manuscript treatise on Government had been found amongst Sidney’s papers; certain passages on political liberty would nowadays be considered as mere truisms, but Jeffreys ruled that they were equal to two-and-twenty adverse witnesses. He also was found guilty, and was beheaded on Tower Hill. Shaftesbury fled to Holland. Lord Essex—a true nobleman—blaming himself for having put it into Howard’s power to injure Lord Russell, committed suicide. And some Scottish gentlemen were also implicated in the Whig plot. Bailie, of Jerviswood, had been in correspondence with Lord Russell, and was asked to give evidence against him. On his refusal, he was himself tried for treason,—condemned and executed. Many were fined and imprisoned; many left the country, or otherwise
  • 56. could not be found, but were tried in their absence—outlawed, and their estates forfeited. James returned to London: he feared the influence of the Duke of Monmouth, who, trading on his father’s favour and his own handsome face and genial manners, posed as an ultra-Protestant, and, in spite of his illegitimate birth, aspired to the succession. James had Monmouth sent to Holland—then, under the Prince of Orange, the refuge for English and Scottish exiles. But for Charles the world of time was now at its vanishing point. He was only in his fifty-fifth year when, in the midst of his sensuous pleasures, apoplexy seized him, and Bishop Ken had to tell him his hours were numbered. Certain religious exercises were gone through, and the sacramental elements being brought in, the bishop proposed their administration. The King put this off, and the bishop retired. And now James looked up a Catholic priest, and had him smuggled in by a private door to the King’s chamber. The King made confession, and had the last rites of the Church administered. Thus made safe by a Romish passport into heaven—the dying King no doubt enjoyed as a good joke the prayers and admonitions of the Protestant prelates, who, with the lords-in-waiting, were afterwards ushered into his chamber. He died February 6th, 1684-5.
  • 58. Scotland under James the Second. Within half-an-hour of his brother’s death, James was seated as the King in Council. He declared that he would govern by the laws, and maintain the established church. Loyal addresses from all parts of his dominions were poured in upon him; and the commencement of his reign gave promise of stability and popularity. In a lesser degree he had his brothers vices; but he had shewn considerable aptitude for public business, and was not deficient in personal courage. In 1665, he had, in a war with Holland, taken the command of the Channel fleet. On the 3rd of June a great battle was fought off the Norfolk coast, within sight of Lowestoft. When the fight was at its hottest, the Dutch admiral’s ship blew up, and a Dutch fire-ship grappled with and destroyed an English ship. James had twice to shift his flag, as his ships were successively disabled. After an obstinate contest the Dutch ships sailed for the Texel; James pursued for a time,—eighteen of the enemy’s ships being taken or destroyed. But his accession to the throne was not to be unchallenged. The Duke of Monmouth and the Earl of Argyle met in Holland, and concerted simultaneous insurrections in England and Scotland. Monmouth landed at Lyme, in Dorsetshire, on 11th June, and marched to Taunton, in Somersetshire, at the head of 5,000 irregularly armed troops. He had married the heiress of Buccleuch, and in other ways became associated with the nobility; stories had been set afloat of a marriage between his father and his Welsh mother, Lucy Walters, and he was looked on by many as the true heir to the throne. At Taunton he was received with acclamations; twenty young ladies presented him with a pocket-bible, a flag, and a
  • 59. naked sword. He had himself proclaimed King. After a good deal of tentative marching through the western counties, he fell back on Bridgewater, and three miles from this town, at Sedgemoor, a battle was fought, in which he was utterly defeated. He himself fled before the close of the fight; and was afterwards captured hiding in a bean- field. He was taken to London, and at his own solicitation had an interview with the King. A larger-minded man than James would have been moved to generosity, at the sight of his brother’s son grovelling on his knees before him, and humbly suing for mercy; but generosity towards fallen enemies was not a distinguishing trait in the Stuart character. And this young man had long been a thorn in James’s path; so now no mercy for him—his doom was immediate execution. And terrible was the vengeance of the King on not only the leaders of the insurrection, but on inferior participants, and on all who had given aid or countenance thereto. There were a number of military executions; and then Jeffreys was let loose upon the western counties. His “bloody assize” was a very devil’s carnival of barbarity and death. The campaign was opened at Winchester with the trial of Alice Lisle, the aged widow of one of Cromwell’s lieutenants, for affording food and shelter to two of the fugitive insurgents. Jeffreys bullied the jury into a verdict of guilty, and then he sentenced her to be burned alive that same afternoon. Horror- stricken, the clergy of the cathedral obtained a respite for three days. Noble ladies, whom she had befriended in the time of the Commonwealth, solicited her pardon from the King. Her son in the army had served against Monmouth. And James was actually moved to change her sentence from burning alive to beheading! And so it was executed. In this judicial massacre, more than three hundred persons were put to death, and very many who escaped death, suffered mutilation, imprisonment, or exile. Hundreds of the prisoners were presented to the courtiers,—to be sold for ten years as slaves in the West Indies. The twenty young ladies of Taunton, who had figured in the ovation to Monmouth, were assigned to the
  • 60. Queen’s maids-of-honour, and they sold pardons to the girls at the rate of a hundred pounds a head! The accession of James brought no relaxation in the oppressive laws bearing upon Scottish presbyterianism. It was still in the power of the military to apprehend and interrogate, to torture, to confiscate the goods, and even to take the lives of those suspected of nonconformity, or of assisting outlawed persons. It was therefore to be expected that any attempt to throw off the galling yoke would have general sympathy and support. Argyle had himself been the victim of unjust persecution; and yet his invasion of Scotland was as futile and disastrous as that of Monmouth was of England. Argyle was a Highland chief, influenced by his old family feuds; and his foremost idea was to fight the clans which were the hereditary enemies of his house, and also loyal Jacobites. So with about three hundred men he landed on the western peninsula of Cantyre, and was joined by about a thousand of his Campbell clansmen. He proposed marching to Inverary; but the other leaders were afraid of their little army being shut up in the highlands, and thought that the western shires—in which the covenanters were numerically strong, and where they had already boldly faced the government troops—would be a better field for operations. There was as usual in such differences, much wordy recrimination; time was lost; and when at length a movement was made into Lanarkshire, long, weary marches, with mistakes in the route, disheartened and demoralized the insurgents. The royal troops, in superior numbers, were fast closing in on Argyle, and, without a battle, his following fell to pieces, and himself was made prisoner. He was taken with disgraceful indignities to Edinburgh, and his old, most iniquitous sentence was carried out. Like his father, he met his fate with firmness; he said the grim instrument of death was “a sweet Maiden, whose embrace would waft his soul into heaven.” Upwards of twenty of the more considerable of his followers also suffered death.
  • 61. EXECUTION OF THE EARL OF ARGYLE. As shewing the mean and cruel spirit of James, we may mention that on medals which he had struck, commemorative of his triumphs over Monmouth and Argyle, one side bore two severed heads, and the reverse two headless trunks. And now in his plenitude of power, James began to shew openly what was his great intention, namely, the subversion of the Protestant faith, and the restitution of papal sway in Britain. His brother had so far paved the way for such a change, that he had taken advantage of the reaction of loyalty at the Restoration, of the general disgust at that detestable imposture, the Titus Oates’ “popish plot,” and of the discovery of the atrocious Rye House plot, to make his government despotic. He had, by his foul example, sown the seeds of immorality and corruption broadcast through the national life. Religious fervour and high political principle seemed to have vanished from the land,—servile submission to kingly authority was preached by divines, sung by poets, and practised by statesmen,—as the only safeguard against sombre puritanism, political strife, and the misrule of the mob.
  • 62. And now here was a zealot,—seeing sycophants all around him; men of position hasting to gain his favour through the Romish confessional; a servile parliament granting him bountiful supplies; and a powerful French king sending him subsidies,—with the property, the liberties, the very lives of his subjects at his disposal,— can we wonder that he thought that his authority could be stretched to lording it also over their consciences? A century and a half previously, Henry VIII. had abrogated the authority of the Pope in England, and James may have believed that what one despotic king could do, another could undo. Of three things we hardly know which most to wonder at:—the daring of the attempt—or, how nearly he succeeded in his designs—or, that amidst so much apathy, servility, and corruption, he did not, for a time at least, accomplish his ends. But the Reformation was, on the face of it, a natural outcome of a new dawn, after the long night of the dark ages in Europe. It was, with the revival of letters, the new geographical and scientific discoveries, and the general spirit of adventure and research, a stepping-stone towards progress and enlarged political and intellectual freedom; whilst the proposed retrocession to Rome meant going backwards, and a wilful surrender to the old bondage and authority. James publicly attended the rites of his church; he surrounded himself by Catholic priests, a leading Jesuit, Father Petre, being his political confidant; he entertained at his court—for the first time in England since the days of Queen Mary—a papal nuncio. He placed the Church under the control of a High Commission of seven members, Jeffreys, now Lord Chancellor, at the head. In chartered towns, Catholics were to be eligible to serve as mayors and aldermen. He began the formation of a large standing army, and, in defiance of the Test Act, and in assertion of his dispensing power, he largely officered this army by Catholics. The university of Oxford had, in the previous reign, declared that in no case was resistance to the royal authority justifiable, and it had now to reap the bitter fruits of its servile declaration. The King appointed a Roman Catholic to the deanery of Christ Church; another to the presidency of Magdalen
  • 63. College, and twelve Catholic fellows were appointed in one day. Oxford now began to see that passive obedience might well stop short of a surrender of religious principles; it resisted the royal mandates; and it would not submit, although twenty-five of its fellows were expelled. And a contagion of conversion broke out in the higher social ranks. Noble lords and ladies of fashion went to mass and confession; processions of Catholic priests were daily met in the streets of London; Catholic chapels and monasteries were becoming numerous, their service bells ringing perpetually. In Scotland, the Chancellorship was bestowed on one of the King’s time-serving converts, Drummond, Earl of Perth. He co-operated with the Earl of Sunderland in England, in driving on James to the most extravagant reactionary measures. By a new court order all persons holding civil offices in Scotland were ordered to resign, and to resume their offices without taking the test oath, ordered in 1681, they taking, for thus breaking the law, a remission of penalties from the Crown; all not obtaining such remission to be subjected to the said penalties. That is,—all officials were ordered to break the law, and were to be subject to penalties for such infringement,—unless by getting the King’s pardon they acknowledged his power to abrogate the law! And this test oath had been the contrivance of James himself when in Scotland,—forced upon Presbyterians at the sword’s point, and held so sacred that Argyle had been condemned to death for taking it with a slight qualification. The short reign of James was one of the saddest periods in Scottish history. He had refused to take the usual coronation oath, which included the maintenance of the established church. In spite of this refusal—which impaired the validity of his right to rule—a weakly compliant parliament expressed the loyalty of absolute submission. The law against conventicles was extended to the presence of five persons, besides the family attending domestic worship. If the meeting was held outside the house—even on the door-step—it was to be considered a field-conventicle punishable by death. But on the question of repealing the penal acts against
  • 64. Catholics, Parliament proved refractory, and it was forthwith dissolved. The King issued a proclamation depriving the burghs of the right of electing their own magistrates. When, to favour Roman Catholicism, he issued his Declaration of Indulgence, by which there was to be general liberty of worship; yet—strange anomaly—the laws against field-preaching continued in full force. Under these laws, James Renwick, a delicate, but enthusiastic field-preacher, was executed in Edinburgh in February, 1688. He was the last in the fearfully long roll of covenanting martyrs.
  • 65. THE COVENANTERS’ MONUMENT IN THE GREYFRIARS’ CHURCHYARD, EDINBURGH
  • 66. The Declaration of Indulgence, permitting all professions of religion to worship in their own ways, was published by James— solely on his own authority—in April, 1687. At the first blush we may be inclined to call this general indulgence a step in the right direction,—even although we know that under the cloak of toleration to all forms of faith, the King’s main object was to legalise Catholic worship and ritual. We now say, from the more liberal stand-point of the nineteenth century, that the penal laws against the exercise of Catholic rites were tyrannical and unjust. But we have to consider the times in which these laws were introduced, when after a long and bitter struggle the papal yoke had been thrown off,—when the severities of Rome against those she termed heretics were fresh in the memory,—and that she never abates one jot of her assumption to be the one authoritative church—claiming the entire submission of Christendom. And Dissenters knew that the King was here bidding for their support against the established church. They saw that Tyrconnel, the King’s Viceroy in Ireland—a country where James did not require to keep up appearances—was fast arming the Catholics, preparatory to a total subversion of Protestantism; and thus the Presbyterian and other dissenters saw in the Episcopal Church the rallying point of religious freedom; they overlooked its past subserviency to power and its harshness to themselves, in consideration of its present danger, and the stand it was now preparing to make in the common cause. In April, 1688, the king ordered his Declaration to be read in all the churches. The London clergy met and signed a refusal to comply with the order, and the primate, Sancroft, and six other bishops, presented a petition to the king against being compelled to read a document which assumed the legality of the dispensing power. Only in seven of the London churches, and a few in the country, was the Declaration read. The king was furious, and summoned the bishops before the privy council; on their acknowledging their signatures to the petition, they were committed to the Tower. Their passage down the Thames was a public ovation; from crowded quays, bridges, and
  • 67. barges arose enthusiastic shouts of encouragement; the very officers of the Tower went on their knees for the episcopal blessing. In their imprisonment, the bishops were visited daily by nobles and leading men; and—which irritated James most of all—a deputation of dissenting ministers went and thanked them in the name of their common Protestantism. And just at this time an event occurred which had a remarkable bearing on the history of the period. On June 10th, 1688, James’s queen gave birth to a son. The news had been circulated that a child was expected; the faithful ventured to prophesy a prince; a blessing vouchsafed by the intervention of the Virgin Mary, in response to prayers and pilgrimages. But Protestant England had both feared and doubted. The Court and its household were, almost exclusively, composed of Catholics, and when the birth of a prince was announced, it was generally believed that a strange child had been smuggled into the palace, and was then being passed off as the king’s son. There now seems little doubt but that the infant was really the offspring of the king and queen. Thus, to his father’s joy, and to Catholic anticipations of the throne being after him still occupied by a king of the old faith—but with general doubts and misgivings—with repudiation instead of welcome, came into the world the ill-fated prince, known in our history as James the Pretender. On June 20th, the trial of the bishops took place before the Court of King’s Bench. They were charged with having “published a false, malicious, and seditious libel.” Of the four judges, two were for the petition being a libel, and two were against. The jury had to decide the question, and were locked up during the night. At ten o’clock next morning, when the Court again met, there was a silence of deep suspense before the verdict was pronounced. When the words “not guilty” fell from the foreman’s lips, a great cheer arose, which penetrated into the crowded street, and was speedily wafted over London, extending even to the troops on parade at Blackheath. It was a day of general congratulation and rejoicing; and bonfires and illuminations went far into the summer night.
  • 69. B The Revolution of 1688. efore the birth of the prince, the general idea had been that the country should tide over James’s misgovernment as best it could, and wait patiently for the succession to the throne in natural course of Mary, Princess of Orange, the elder daughter of the king by his first marriage. But the situation was now altogether changed; and on the very day of the acquittal of the bishops, there was sent— signed by the bishop of London, several noblemen, and others—an invitation to William to come over with an army to the relief of the country: and the prince at once commenced his preparations. And meantime, James, his purposes and hopes of success strengthened by the birth of a son, was indignant at his defeat in the trial of the bishops, and, goaded on by the French minister and his inner circle of advisers, he resolved to crush the spirit of the nation by force of arms. He brought over several regiments of Tyrconnel’s Irish troops, and their menacing presence, as strangers and Catholics, was hateful to the English people. A derisive doggrel ballad, called from its burden Lilliburelo, was sung and whistled all over the land. And now the king was told that his Dutch son-in-law was making great preparations for invasion. He knew that he had lost the best safeguard of his throne—the confidence and affection of his subjects —and whilst adopting means for defence, he hastened to retract all the measures which had made him unpopular. He threw himself in feigned repentance on the advice of the bishops, and they, in plain words, like the prophets of old, told him of his injustice and oppression, and advised him at once to call a Parliament. He dismissed his priestly adviser Father Petre, and the renegade Lord
  • 70. Sunderland. He restored its fellows to Oxford, and their franchises to the corporations. But the precipitation of fear was so evident in his concessions, that there was no reaction of confidence. The people were watching the weathercocks, and praying for a north-east, or, as it was called “a Protestant” wind. After waiting some weeks for a favourable wind, and with an after-delay from storms, by the end of October, William was fairly at sea. He first sailed up the North Sea, as if he intended a landing on the Yorkshire coast; but changed his course for the Channel. The wind and tide prevented the royal fleet from attacking him in the Straits of Dover. From the opposite coasts his fleet presented a magnificent sight. There were sixty men-of-war and seven hundred transports, extending twenty miles in length. It was just a hundred years since such another magnificent spectacle had been seen in the Channel—the Spanish Armada—also bent upon the invasion of England. Then, the great fleet meant papal aggression, and priestly domination; now, it meant deliverance from this aggression, and freedom of the conscience; then, beacon fires on mount and headland flashed danger to the lives and liberties of Englishmen; now the tidings that a foreign fleet was skirting the coast were of glad and hopeful assurance. On the 5th of November—the anniversary of the Gunpowder Plot —the fleet anchored at Torbay, in Devonshire. With his army of fifteen thousand men, William marched to Exeter, where he was enthusiastically received. But the memory of Jeffreys’ “bloody assize” was still fresh in the western shires, and for several days there were few signs of encouragement; it is said that he even meditated returning to Holland. But bye-and-bye one nobleman after another, and several officers of James’s army, entered the camp. The north of England began to stir in raising and disciplining revolutionary troops, and the Earl of Bath put Plymouth into William’s hands. The King hastened down to Salisbury, resolved to stake his kingdom on the issue of a battle; but William, although a thorough captain in war, wished to avoid bloodshed; he trusted to the increasing stream of desertion from the king rendering a great battle
  • 71. unnecessary. And so it turned out. The sagacious lieutenant-general of the king’s army, Lord Churchill, the Dukes of Grafton and Ormond, even the king’s younger daughter Anne, with her husband, Prince George of Denmark, and many other persons of note, joined the Prince of Orange. James went back to London, and sent away the queen and her five-months’ old child to France. When he knew of their safety he left London at night, by the river. He threw the great seal into the Thames, and proceeded to Sheerness, where a small vessel was waiting for him. Boarding the vessel he attracted the attention of some Kentish fishermen, who, in hopes of reward, made him prisoner. Released, by an order of the Lords, he returned to London, and passed thence to Rochester. William wanted him out of the country; so facilities were made for his escape, and he was soon at St. Germains, where Louis gave him a friendly reception; and at St. Germains he made his home. Assisted by Louis, he made, next year, an attempt for the recovery of Ireland. In that essentially Catholic country, it seemed at first that he would there be able to retain one of the three kingdoms, but his defeat by William, at the Boyne, compelled his return to France. He died September 16th, 1701, aged 68 years. The King, having fled, and no parliament sitting, William was advised to claim the kingdom by right of conquest. But both from principle and sound policy he held that this would be a less secure right of possession than would be the choice—as formal as under the circumstances it could be made—of the English people. So he summoned a Convention of the States of the Realm,—irregularly convoked in the emergency, but elected in the usual manner. The Convention met on 22nd February—six weeks after the King’s flight. The debates were long and stormy; the two Houses disagreed,— the Lords could hardly bring themselves to declare for the deposition of the King; but the Commons were firm, and at length this resolution was passed in both houses: “That James, having violated the fundamental laws, and withdrawn himself from the kingdom, has
  • 72. broken the original contract between king and people, has abdicated the government, and therefore the throne has become vacant.” And then came the questions,—Who was to reign? and what was to be the order of succession? Here there was a division of opinion. Was James’s infant son to be acknowledged as King—with William as Regent? or, Should the crown be conferred on Mary in her own right? William was not a man of many words, but he now got together a few of the leading men, and to them he spoke very plainly: he would not interfere with the right of the Convention to settle its own affairs as it thought best; but for himself he would not accept any regency, nor—much as he loved his wife—would he remain in England as her gentleman-usher. In a few hours his words were all over London, and it was known that he would be King. So the Convention passed a number of resolutions, embodied in what was termed a Declaration of Rights,—defining the royal prerogative, and the powers of parliament; and the Prince and Princess, having signified their adhesion thereto, it was resolved that William and Mary be jointly King and Queen of England, Ireland, and the dominions belonging thereto; the administration to rest in William. The crown was settled,—first on the survivor of the royal pair,—then on the children of Mary, then on those of her sister Anne, and next on the children of William by any other wife. The son of James and his posterity were thus shut out entirely from the succession. The Scottish Convention of Estates passed resolutions nearly similar to those in the English Declaration of Rights, closing with a declaration against Prelacy, asserting that there was no higher office in the Church than presbyter. On the leading question then before the country, their resolution had a more decided tone than that of the English Convention. They declared that James had assumed the throne without taking the oaths prescribed by law, that he had proceeded to subvert the constitution of the country from a limited monarchy to an absolute despotism; that he had employed the powers thus usurped for violating the laws and liberties, and altering the religion of Scotland;
  • 73. for doing these things he had forfeited his right to the crown, and the throne had thereby become vacant. The Scottish royalty was conferred on William and Mary, in like terms as that of the English Convention. Battle of Killiecrankie. In the crisis of his affairs, James had summoned his Scottish troops to England. Their commander, Lord Douglas, went over to William; but the second in command, John Graham of Claverhouse— now Viscount Dundee—had an interview with the King—assured him of the loyalty of his troops, about 6,500 well disciplined men, advised the King either to hazard a battle, or to fall back with these troops into Scotland. On the King declining both propositions, Lord Dundee took up a position at Watford, about eighteen miles north- west of London, expecting an attack by William. But Dundee had served his early campaigns under the Prince, having in one engagement rescued him from imminent danger. So the Prince now sent him a message that he had no quarrel with him. Then came James’s flight, and the Prince’s entry into London; and Dundee seeing he could do nothing more to help James in England, rode back with about twenty-five of his dragoons into Scotland. The Scottish army was placed under General Mackay, one of William’s adherents, and he was shortly after sent as commander of the royal forces into Scotland. Lord Dundee came to Edinburgh, for some time hovering like a hawk over the then sitting Convention. The Duke of Gordon still held the Castle for King James; Dundee had an interview with the Duke and advised “no surrender,” he then, with a few horsemen, left the city. (We all know the ringing song in which Sir Walter Scott narrates his departure.) Like a fiery-cross he went through the highlands, rousing the clansman to battle for the fallen Stuart King. The man must have had a dominating personality; in a short time he had
  • 74. assembled an army, feeble in discipline and cohesion no doubt; but, as it proved, good for the kind of work it befell them to do. The highlanders were posted on an open slope at the head of the pass of Killiecrankie in the north Perthshire hills. To give them battle, Mackay, on 17th June, 1689, advanced up the pass. When the royal troops entered the defile, no enemy was to be seen,—only the pines towering high upon the cliffs on either hand, and the river Garry rushing swiftly by the narrow pathway through the pass. To the Lowland and Dutch soldiers, who composed the royal army, it was a scene novel and magnificent, but bewildering, awe-inspiring. Dundee allowed the whole of Mackay’s army to emerge from the pass, and even to form in order of battle, before he began the attack. It was an hour before sunset that the highlanders advanced. They fired their muskets only once, and throwing them away, with fierce shouts they rushed down with broadsword and target. Mackay’s line was broken by the onset. When it came to disordered ranks, and the clash of hand to hand combats, the superior discipline of the royal troops was of no account. Agility, hardihood, and the confidence of assured victory were on the side of the clansmen. It was soon a rout; but with such a narrow gorge for retreat it became a massacre. Two thousand of Mackay’s troops were slain. The highlanders’ loss was eight hundred; but amongst these was their gallant leader. Near the end of the battle, Dundee, on horseback, was extending his right arm to the clan Macdonald, as directing their movements, when he was struck by a bullet under the arm-pit, where he was unprotected by his cuirass. With him perished the cause of King James in Scotland. After his death his army melted away, and both highlands and lowlands submitted to the Government of William. General lenity and toleration were the watchwords of William’s policy. The episcopal church was to be maintained in England, and the presbyterian in Scotland; but neither were to ride rough-shod over dissenters. In Scotland, much against the desires of the more rigid, as the Cameronians, there were to be no reprisals for former persecution and oppression. Even obnoxious officials were
  • 75. maintained in their old places. When the Jacobite rising in Ireland was quelled by the surrender of Limerick, a treaty was there made by which Catholics were to be allowed the free exercise of their religion. William endeavoured to get parliament to ratify this treaty, but two months after it had been entered into, the English Parliament imposed a declaration against Transubstantiation on members of the Irish parliament, and this parliament, entirely composed of Protestants, whilst giving nominal confirmation, really put the Catholics in a worse condition than they were before. The Irish Catholics have since then called Limerick, “the town of the broken treaty.”