Wolfram 3

For the vast majority of rules written down at random, such problems do indeed occur.
But it is possible to find rules in which they do not, and the pictures on the previous two
pages [129, 130] show a few examples I have found of such rules. In cases (a) and (b),
the behavior is fairly simple. But in the other cases, it is considerably more complicated.

There is a steady overall increase, but superimposed on this increase are fluctuations, as
shown in the pictures on the facing page.

In cases (c) and (d), these fluctuations turn out to have a very regular nested form. But in
the other cases, the fluctuations seem instead in many respects random. Thus in case (f),
for example, the number of positive and negative fluctuations appears on average to be
equal even after a million steps.

But in a sense one of the most surprising features of the facing page is that the
fluctuations it shows are so violent. One might have thought that in going say from
f[2000] to f[2001] there would only ever be a small change. After all, between n=2000
and 2001 there is only a 0.05% change in the size of n.

But much as we saw in the previous section it turns out that it is not so much the size of n
that seems to matter as various aspects of its representation. And indeed, in cases (c) and
(d), for example, it so happens that there is a direct relationship between the fluctuations
in f[n] and the base 2 digit sequence of n.

In case (d), the fluctuation in each f[n] turns out to be essentially just the number of 1's
that occur in the base 2 digit sequence for n. And in case (c), the fluctuations are
determined by the total number of 1's that occur in the digit sequences of all numbers less
than n.

There are no such simple relationships for the other rules shown on the facing page. But
in general one suspects that all these rules can be thought of as being like simple
computer programs that take some representation of n as their input.

And what we have discovered in this section is that even though the rules ultimately
involve only addition and subtraction, they nevertheless correspond to programs that are
capable of producing behavior of great complexity.

888888

The Sequence of Primes

In the sequence of all possible numbers 1, 2, 3, 4, 5, 6, 7, 8, ... most are divisible by
others--so that for example 6 is divisible by 2 and 3. But this is not true of every number.
And so for example 5 and 7 are not divisible by any other numbers (except trivially by 1).
And in fact it has been known for more than two thousand years that there are an infinite

sequence of so-called prime numbers which are not divisible by other numbers, the first
few being 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, ...

The picture below shows a simple rule by which such primes can be obtained. The idea is
to start out on the top line with all possible numbers. Then on the second line, one
removes all numbers larger than 2 that are divisible by 2. On the third line one removes
numbers divisible by 3, and so on. As one goes on, fewer and fewer numbers remain. But
some numbers always remain, and these numbers are exactly the primes.

Given the simplicity of this rule, one might imagine that the sequence of primes it
generates would also be correspondingly simple. But just as in so many other examples in
this book, in fact it is not. And indeed the plots on the facing page show various features
of this sequence which indicate that it is in many respects quite random.

Captions on this page:

A filtering process that yields the prime numbers. One starts on the top line with all
numbers between 1 and 100. Then on the second line, one removes numbers larger than 2
that are divisible by 2--as indicated by the gray dots. On the third line, one removes
numbers larger than 3 that are divisible by 3. If one then continues forever, there are
some numbers that always remain, and these are exactly the primes. The process shown is
essentially the sieve of Eratosthenes, already known in 200 BC.

888888

[No text on this page]


Features of the sequence of primes. Despite the simplicity of the rule on the facing page
that generates the primes, the actual sequence of primes that is obtained seems in many
respects remarkably random.

88888

The examples of complexity that I have shown so far in this book are almost all
completely new. But the first few hundred primes were no doubt known even in

antiquity, and it must have been evident that there was at least some complexity in their
distribution.

However, without the whole intellectual structure that I have developed in this book, the
implications of this observation--and its potential connection, for example, to phenomena
in nature--were not recognized. And even though there has been a vast amount of
mathematical work done on the sequence of primes over the course of many centuries,
almost without exception it has been concerned not with basic issues of complexity but
instead with trying to find specific kinds of regularities.

Yet as it turns out, few regularities have in fact been found, and often the results that have
been established tend only to support the idea that the sequence has many features of
randomness. And so, as one example, it might appear from the pictures on the previous
page that (c), (d) and (e) always stay systematically above the axis. But in fact with
considerable effort it has been proved that all of them are in a sense more random--and
eventually cross the axis an infinite number of times, and indeed go any distance up or
down.

So is the complexity that we have seen in the sequence of primes somehow unusual
among sequences based on numbers? The pictures on the facing page show a few other
examples of sequences generated according to simple rules based on properties of
numbers.

And in each case we again see a remarkable level of complexity.

Some of this complexity can be understood if we look at each number not in terms of its
overall size, but rather in terms of its digit sequence or set of possible divisors. But in
most cases--often despite centuries of work in number theory--considerable complexity
remains.

And indeed the only reasonable conclusion seems to be that just as in so many other
systems in this book, such sequences of numbers exhibit complexity that somehow arises
as a fundamental consequence of the rules by which the sequences are generated.

88888



Sequences based on various simple properties of numbers. Extensive work in number
theory has managed to establish only a few properties of these. It is for example known

that (d) never reaches zero, while curve (c) reaches zero only for numbers of the form 4^r
(8s + 7). Sequence (b) is zero at so-called perfect numbers. Even perfect numbers always
have a known form, but whether any odd perfect numbers exist is a question that has
remained unresolved for more than two thousand years. The claim that sequence (e)
never reaches zero is known as Goldbach's Conjecture. It was made in 1742 but no proof
or counterexample has ever been found.

88888

Mathematical Constants

The last few sections [2, 3, 4] have shown that one can set up all sorts of systems based
on numbers in which great complexity can occur. But it turns out that the possibility of
such complexity is already suggested by some well-known facts in elementary
mathematics.

The facts in question concern the sequences of digits in numbers like π (pi). To a very
rough approximation, π is 3.14. A more accurate approximation is
3.14159265358979323846264338327950288.

But how does this sequence of digits continue?

One might suppose that at some level it must be quite simple and regular. For the value of
π is specified by the simple definition of being the ratio of the circumference of any circle
to its diameter.

But it turns out that even though this definition is simple, the digit sequence of π is not
simple at all. The facing page shows the first 4000 digits in the sequence, both in the
usual case of base 10, and in base 2. And the picture below shows a pictorial
representation of the first 20,000 digits in the sequence.


A pictorial representation of the first 20,000 digits of π in base 2. The curve drawn goes
up every time a digit is 1, and down every time it is 0. Great complexity is evident. If the
curve were continued further, it would spend more time above the axis, and no aspect of
what is seen provides any evidence that the digit sequence is anything but perfectly
random.

888888



The first 4000 digits of π in bases 10 and 2. Despite the simple definition of π as the ratio
of the circumference to the diameter of a circle, its digit sequence is sufficiently
complicated as to seem for practical purposes random.

88888

In no case are there any obvious regularities. Indeed, in all the more than two hundred
billion digits of π that have so far been computed, no significant regularity of any kind
has ever been found. Despite the simplicity of its definition, the digit sequence of π seems
for practical purposes completely random.

But what about other numbers? Is π a special case, or are there other familiar
mathematical constants that have complicated digit sequences? There are some numbers
whose digit sequences effectively have limited length. Thus, for example, the digit
sequence of 3/8 in base 10 is 0.375. (Strictly, the digit sequence is 0.3750000000..., but
the 0's do not affect the value of the number, so are normally suppressed.)

It is however easy to find numbers whose digit sequences do not terminate. Thus, for
example, the exact value of 1/3 in base 10 is 0.3333333333333..., where the 3's repeat
forever. And similarly, 1/7 is 0.142857142857142857142857142857..., where now the
block of digits 142857 repeats forever. The table below gives the digit sequences for
several rational numbers obtained by dividing pairs of whole numbers. In all cases what
we see is that the digit sequences of such numbers have a simple repetitive form. And in
fact, it turns out that absolutely all rational numbers have digit sequences that eventually
repeat.

We can get some understanding of why this is so by looking at the details of how
processes for performing division work. The pictures


Digit sequences for various rational numbers, given in base 10 (above) and base 2
(below). For a number of the form p/q, the digit sequence always repeats with a period of
at most q-1 steps.

88888

below show successive steps in a particular method for computing the base 2 digit
sequence for the rational numbers p/q.

The method is essentially standard long division, although it is somewhat simpler in base
2 than in the usual case of base 10. The idea is to have a number r which essentially keeps
track of the remainder at each step in the division. One starts by setting r equal to p. Then
at each step, one compares the values of 2r and q. If 2r is less than q, the digit generated
at that step is 0, and r is replaced by 2r. Otherwise, r is replaced by 2r - q. With this
procedure, the value of r is always less than q. And as a result, the digit sequence
obtained always repeats at most every q-1 steps.

It turns out, however, that rational numbers are very unusual in having such simple digit
sequences. And indeed, if one looks for example at square roots the story is completely
different.

Perfect squares such as 4 = 2×2 and 9 = 3×3 are specifically set up to have square roots
that are just whole numbers. But as the table at the top of the next page shows, other
square roots have much more complicated digit sequences. In fact, so far as one can tell,
all whole numbers other than perfect squares have square roots whose digit sequences
appear completely random.


Successive steps in the computation of various rational numbers. In each case, the
column on the right shows the sequence of base 2 digits in the number, while the box on
the left shows the remainder at each of the steps in the computation.

88888

But how is such randomness produced? The picture at the top of the facing page shows
an example of a procedure for generating the base 2 digit sequence for the square root of
a given number n.

The procedure is only slightly more complicated than the one for division discussed
above. It involves two numbers r and s, which are initially set to be n and 0, respectively.
At each step it compares the values of r and s, and if r is larger than s it replaces r and s
by 4(r-s-1) and 2(s+2) respectively; otherwise it replaces them just by 4r and 2s. And it
then turns out that the base 2 digits of s correspond exactly to the base 2 digits of
Sqrt[n]--with one new digit being generated at each step.

As the picture shows, the results of the procedure exhibit considerable complexity. And
indeed, it seems that just like so many other examples that we have discussed in this
book, the procedure for generating square roots is based on simple rules but nevertheless
yields behavior of great complexity.


Digit sequences for various square roots, given at the top in base 10 and at the bottom in
base 2. Despite their simple definition, all these sequences seem for practical purposes
random.

888888

It turns out that square roots are certainly not alone in having apparently random digit
sequences. As an example, the table on the next page gives the digit sequences for some
cube roots and fourth roots, as well as for some logarithms and exponentials. And so far
as one can tell, almost all these kinds of numbers also have apparently random digit
sequences.

In fact, rational numbers turn out to be the only kinds of numbers that have repetitive
digit sequences. And at least in square roots, cube roots, and so on, it is known that no
nested digit sequences


A procedure for generating the digit sequences of square roots. Two numbers, r and s, are
involved. To find Sqrt[n] one starts by setting r=n and s=0. Then at each step one applies
the rule {r, s} -> If[r >= s+1, {4(r-s-1), 2(s+2)}, {4r, 2s}]. The result is that the digits of s
in base 2 turn out to correspond exactly to the digits of Sqrt[n]. Note that if n is not
between 1 and 4, it must be multiplied or divided by an appropriate power of 4 before
starting this procedure.

888888

ever occur. It is straightforward to construct a nested digit sequence using for example
the substitution systems on page 83, but the point is that such a digit sequence never
corresponds to a number that can be obtained by the mathematical operation of taking
roots.

So far in this chapter we have always used digit sequences as our way of representing
numbers. But one might imagine that perhaps this representation is somehow perverse,
and that if we were just to choose another one, then numbers generated by simple
mathematical operations would no longer seem complex.

Any representation for a number can in a sense be thought of as specifying a procedure
for constructing that number. Thus, for example, the pictures at the top of the facing page
show how the base 10 and base 2 digit sequence representations of π can be used to
construct the number π.


Digit sequences for cube roots, fourth roots, logarithms and exponentials, given at the top
in base 10 and the bottom in base 2. Once again, these sequences seem for practical
purposes random.

888888

By replacing the addition and multiplication that appear above by other operations one
can then get other representations for numbers. A common example are so-called
continued fraction representations, in which the operations of addition and division are
used, as shown below.

The table on the next page gives the continued fraction representations for various
numbers. In the case of rational numbers, the results are always of limited length. But for
other numbers, they go on forever. Square roots turn out to have purely repetitive
continued fraction representations. And the representations of E 2.718 and all its roots
also show definite regularity. But for π, as well as for cube roots, fourth roots, and so on,
the continued fraction representations one gets seem essentially random.

What about other representations of numbers? At some level, one can always use
symbolic expressions like Sqrt[2] + Exp[Sqrt[3]] to represent numbers. And almost by
definition, numbers that can be obtained by simple mathematical operations will
correspond to simple such expressions. But the problem is that there is no telling how
difficult it may be to compute the actual value of a number from the symbolic expression
that is used to represent it.

And in thinking about representations of numbers, it seems appropriate to restrict oneself
to cases where the effort required to find the value of a number from its representation is
essentially the same for


Procedures for building up π from its base 10 and base 2 digit sequence representations.

The continued fraction representation of π. In this representation the value of π is built up
by successive additions and divisions, rather than successive additions and
multiplications.

88888

all numbers. If one does this, then the typical experience is that in any particular
representation, some class of numbers will have simple forms. But other numbers, even
though they may be the result of simple mathematical operations, tend to have seemingly
random forms.

And from this it seems appropriate to conclude that numbers generated by simple
mathematical operations are often in some intrinsic sense complex, independent of the
particular representation that one uses to look at them.


Continued fraction representations for several numbers. Square roots yield repetitive
sequences in this representation, but cube roots and higher roots yield seemingly random
sequences.

88888

Mathematical Functions

The last section showed that individual numbers obtained by applying various simple
mathematical functions can have features that are quite complex. But what about the
functions themselves?

The pictures below show curves obtained by plotting standard mathematical functions.
All of these curves have fairly simple, essentially repetitive forms. And indeed it turns
out that almost all the standard mathematical functions that are defined, for example, in
Mathematica, yield similarly simple curves.

But if one looks at combinations of these standard functions, it is fairly easy to get more
complicated results. The pictures on the next page show what happens, for example, if
one adds together various sine functions. In the first picture, the curve one gets has a
fairly simple repetitive structure. In the second picture, the curve is more complicated,
but still has an overall repetitive structure. But in the third and fourth pictures, there is no
such repetitive structure, and indeed the curves look in many respects random.


Plots of some standard mathematical functions. The top row shows three trigonometric
functions. The bottom row shows three so-called special functions that are commonly
encountered in mathematical physics and other areas of traditional science. In all cases
the curves shown have fairly simple repetitive forms.

888888

In the third picture, however, the points where the curve crosses the axis come in two
regularly spaced families. And as the pictures on the facing page indicate, for any curve
like Sin[x] + Sin[α x] the relative arrangements of these crossing points turn out to be
related to the output of a generalized substitution system in which the rule at each step is
obtained from a term in the continued fraction representation of (α-1)/(α+1).

When α is a square root, then as discussed in the previous section, the continued fraction
representation is purely repetitive,


Curves obtained by adding together various sine functions. In the first two cases, the
curves are ultimately repetitive; in the second two cases they are not. If viewed as
waveforms for sounds, then these curves correspond to chords. The first curve yields a
perfect fifth, while the third curve yields a diminished fifth (or tritone) in an equal
temperament scale.

888888



Curves obtained by adding or subtracting exactly two sine or cosine functions turn out to
have a pattern of axis crossings that can be reproduced by a generalized substitution
system. In general there is an axis crossing within an interval when the corresponding
element in the generalized substitution system is black, and there is not when the element
is white. In the case of Cos[x] - Cos[α x] each step in the generalized substitution system
has a rule determined as shown on the left from a term in the continued fraction
representation of (α-1)/(α+1). In the first two examples shown α is a quadratic irrational,
so that the continued fraction is repetitive, and the pattern obtained is purely nested. (The
second example is analogous to the Fibonacci substitution system on page 83.) In the last
two examples, however, there is no such regularity. Note that successive terms in each
continued fraction are shown alongside successive steps in the substitution system going
up the page.

88888

making the generated pattern nested. But when α is not a square root the pattern can be
more complicated. And if more than two sine functions are involved there no longer
seems to be any particular connection to generalized substitution systems or continued
fractions.

Among all the various mathematical functions defined, say, in Mathematica it turns out
that there are also a few--not traditionally common in natural science--which yield
complex curves but which do not appear to have any explicit dependence on
representations of individual numbers. Many of these are related to the so-called
Riemann zeta function, a version of which is shown in the picture below.

The basic definition of this function is fairly simple. But in the end the function turns out
to be related to the distribution of primes--and the curve it generates is quite complicated.
Indeed, despite immense mathematical effort for over a century, it has so far been
impossible even to establish for example the so-called Riemann Hypothesis, which in
effect just states that all the peaks in the curve lie above the axis, and all the valleys
below.


A curve associated with the so-called Riemann zeta function. The zeta function Zeta[s] is
defined as Sum[1/k^s, {k, ∞}]. The curve shown here is the so-called Riemann-Siegel Z

function, which is essentially Zeta[1/2 + I t]. The celebrated Riemann Hypothesis in
effect states that all peaks after the first one in this curve must lie above the axis.

8888

Iterated Maps and the Chaos Phenomenon

The basic idea of an iterated map is to take a number between 0 and 1, and then in a
sequence of steps to update this number according to a fixed rule or "map". Many of the
maps I will consider can be expressed in terms of standard mathematical functions, but in
general all that is needed is that the map take any possible number between 0 and 1 and
yield some definite number that is also between 0 and 1.

The pictures on the next two pages [150, 151] show examples of behavior obtained with
four different possible choices of maps.

Cases (a) and (b) on the first page show much the same kind of complexity that we have
seen in many other systems in this chapter--in both digit sequences and sizes of numbers.
Case (c) shows complexity in digit sequences, but the sizes of the numbers it generates
rapidly tend to 0. Case (d), however, seems essentially trivial--and shows no complexity
in either digit sequences or sizes of numbers.

On the first of the next two pages all the examples start with the number 1/2--which has a
simple digit sequence. But the examples on the second of the next two pages instead start
with the number π/4--which has a seemingly random digit sequence.

Cases (a), (b) and (c) look very similar on both pages [150, 151], particularly in terms of
sizes of numbers. But case (d) looks quite different. For on the first page it just yields 0's.
But on the second page, it yields numbers whose sizes continually vary in a seemingly
random way.

If one looks at digit sequences, it is rather clear why this happens. For as the picture
illustrates, the so-called shift map used in case (d) simply serves to shift all digits one
position to the left at each step. And this means that over the course of the evolution of
the system, digits further to the right in the original number will progressively end up all
the way to the left--so that insofar as these digits show randomness, this will lead to
randomness in the sizes of the numbers generated.

It is important to realize, however, that in no real sense is any randomness actually being
generated by the evolution of this system. Instead, it is just that randomness that was
inserted in the digit sequence of the original number shows up in the results one gets.

888888



Examples of iterated maps starting from simple initial conditions. At each step there is a
number x between 0 and 1 that is updated by applying a fixed mapping. The four
mappings considered here are given above both as formulas and in terms of plots. The
pictures at the top of the page show the base 2 digit sequences of successive numbers
obtained by iterating this mapping, while the pictures in the middle of the page plot the
sizes of these numbers. In all cases, the initial conditions consist of the number 1/2--
which has a very simple digit sequence. Yet despite this simplicity, cases (a) and (b)
show considerable complexity in both the digit sequences and the sizes of the numbers
produced (compare page 122). In case (c), the digit sequences are complicated but the
sizes of the numbers tend rapidly to zero. And finally, in case (d), neither the digit
sequences nor the sizes of numbers are anything but trivial. Note that in the pictures
above each horizontal row of digits corresponds to a number, and that digits further to the
left contribute progressively more to the size of this number.

88888



The same iterated maps as on the facing page, but now started from the initial condition
π/4--a number with a seemingly random digit sequence. After fairly few steps, cases (a)
and (b) yield behavior that is almost indistinguishable from what was seen with simple
initial conditions on the facing page. And in case (c), the same exponential decay in the
sizes of numbers occurs as before. But in case (d), the behavior is much more
complicated. Indeed, if one just looked at the sizes of numbers produced, then one sees
the same kind of complexity as in cases (a) and (b). But looking at digit sequences one
realizes that this complexity is actually just a direct transcription of complexity
introduced by giving an initial condition with a seemingly random digit sequence. Case
(d) is the so-called shift map--a classic example of a system that exhibits the sensitive
dependence on initial conditions often known as chaos.

88888

This is very different from what happens in cases (a) and (b). For in these cases complex
and seemingly random results are obtained even on the first of the previous two pages
[150, 151]--when the original number has a very simple digit sequence. And the point is
that these maps actually do intrinsically generate complexity and randomness; they do not
just transcribe it when it is inserted in their initial conditions.

In the context of the approach I have developed in this book this distinction is easy to
understand. But with the traditional mathematical approach, things can get quite
confused. The main issue--already mentioned at the beginning of this chapter--is that in
this approach the only attribute of numbers that is usually considered significant is their
size. And this means that any issue based on discussing explicit digit sequences for
numbers--and whether for example they are simple or complicated--tends to seem at best
bizarre.

Indeed, thinking about numbers purely in terms of size, one might imagine that as soon as
any two numbers are sufficiently close in size they would inevitably lead to results that
are somehow also close. And in fact this is for example the basis for much of the
formalism of calculus in traditional mathematics.

But the essence of the so-called chaos phenomenon is that there are some systems where
arbitrarily small changes in the size of a number can end up having large effects on the
results that are produced. And the shift map shown as case (d) on the previous two pages
[150, 151] turns out to be a classic example of this.

The pictures at the top of the facing page show what happens if one uses as the initial
conditions for this system two numbers whose sizes differ by just one part in a billion
billion. And looking at the plots of sizes of numbers produced, one sees that for quite a
while these two different initial conditions lead to results that are indistinguishably close.
But at some point they diverge and soon become quite different.

And at least if one looks only at the sizes of numbers, this seems rather mysterious. But
as soon as one looks at digit sequences, it immediately becomes much clearer. For as the
pictures at the top of the facing page show, the fact that the numbers which are used as
initial conditions differ only by a very small amount in size just means that their first
several digits are the same. And for a while these digits are

888888

what is important. But since the evolution of the system continually shifts digits to the
left, it is inevitable that the differences that exist in later digits will eventually become
important.

The fact that small changes in initial conditions can lead to large changes in results is a
somewhat interesting phenomenon. But as I will discuss at length in Chapter 7 one must
realize that on its own this cannot explain why randomness--or complexity--should occur

in any particular case. And indeed, for the shift map what we have seen is that
randomness will occur only when the initial conditions that are given happen to be a
number whose digit sequence is random.

But in the past what has often been confusing is that traditional mathematics implicitly
tends to assume that initial conditions of this kind are in some sense inevitable. For if one
thinks about numbers


The effect of making a small change in the initial conditions for the shift map--shown as
case (d) on pages 150 and 151. The first picture shows results for the same initial
condition as on page 151. The second picture shows what happens if one changes the size
of the number in this initial condition by just one part in a billion billion. The plots to the
left indicate that for a while the sizes of numbers obtained by the evolution of the system
in these two cases are indistinguishable. But suddenly the results diverge and become
completely different. Looking at the digit sequences above shows why this happens. The
point is that a small change in the size of the number in the initial conditions corresponds
to a change in digits far to the right. But the evolution of the system progressively shifts
digits to the left, so that the digits which differ eventually become important. The much-
investigated chaos phenomenon consists essentially of this effect.

88888

purely in terms of size, one should make no distinction between numbers that are
sufficiently close in size. And this implies that in choosing initial conditions for a system
like the shift map, one should therefore make no distinction between the exact number
1/2 and numbers that are sufficiently close in size to 1/2.

But it turns out that if one picks a number at random subject only to the constraint that its
size be in a certain range, then it is overwhelmingly likely that the number one gets will
have a digit sequence that is essentially random. And if one then uses this number as the
initial condition for a shift map, the results will also be correspondingly random--just like
those on the previous page.

In the past this fact has sometimes been taken to indicate that the shift map somehow
fundamentally produces randomness. But as I have discussed above, the only randomness
that can actually come out of such a system is randomness that was explicitly put in
through the details of its initial conditions. And this means that any claim that the system
produces randomness must really be a claim about the details of what initial conditions
are typically given for it.

I suppose in principle it could be that nature would effectively follow the same
idealization as in traditional mathematics, and would end up picking numbers purely
according to their size. And if this were so, then it would mean that the initial conditions
for systems like the shift map would naturally have digit sequences that are almost
always random.

But this line of reasoning can ultimately never be too useful. For what it says is that the
randomness we see somehow comes from randomness that is already present--but it does
not explain where that randomness comes from. And indeed--as I will discuss in Chapter
7--if one looks only at systems like the shift map then it is not clear any new randomness
can ever actually be generated.

But a crucial discovery in this book is that systems like (a) and (b) on pages 150 and 151
can show behavior that seems in many respects random even when their initial conditions
show no sign of randomness and are in fact extremely simple.

Yet the fact that systems like (a) and (b) can intrinsically generate randomness even from
simple initial conditions does not mean that they

888888

do not also show sensitive dependence on initial conditions. And indeed the pictures
below illustrate that even in such cases changes in digit sequences are progressively
amplified--just like in the shift map case (d).

But the crucial point that I will discuss more in Chapter 7 is that the presence of sensitive
dependence on initial conditions in systems like (a) and (b) in no way implies that it is
what is responsible for the randomness and complexity we see in these systems. And
indeed, what looking at the shift map in terms of digit sequences shows us is that this
phenomenon on its own can make no contribution at all to what we can reasonably
consider the ultimate production of randomness.

Continuous Cellular Automata

Despite all their differences, the various kinds of programs discussed in the previous
chapter have one thing in common: they are all based on elements that can take on only a
discrete set of possible forms, typically just colors black and white. And in this chapter,
we have introduced a similar kind of discreteness into our study of systems based on
numbers


Differences in digit sequences produced by a small change in initial conditions for the
four iterated maps discussed in this section. Cases (a), (b) and (d) exhibit sensitive
dependence on initial conditions, in the sense that a change in insignificant digits far to
the right eventually grows to affect all digits. Case (c) does not show such sensitivity to
initial conditions, but instead always evolves to 0, independent of its initial conditions.

88888

by considering digit sequences in which each digit can again have only a discrete set of
possible values, typically just 0 and 1.

So now a question that arises is whether all the complexity we have seen in the past three
chapters [2, 3, 4] somehow depends on the discreteness of the elements in the systems we
have looked at.

And to address this question, what I will do in this section is to consider a generalization
of cellular automata in which each cell is not just black or white, but instead can have any
of a continuous range of possible levels of gray. One can update the gray level of each
cell by using rules that are in a sense a cross between the totalistic cellular automaton
rules that we discussed at the beginning of the last chapter and the iterated maps that we
just discussed in the previous section.

The idea is to look at the average gray level of a cell and its immediate neighbors, and
then to get the gray level for that cell at the next step by applying a fixed mapping to the
result. The picture below shows a very simple case in which the new gray level of each
cell is exactly the average of the one for that cell and its immediate neighbors. Starting
from a single black cell, what happens in this case is that the gray essentially just diffuses
away, leaving in the end a uniform pattern.

The picture on the facing page shows what happens with a slightly more complicated rule
in which the average gray level is multiplied by 3/2, and then only the fractional part is
kept if the result of this is greater than 1.


A continuous cellular automaton in which each cell can have any level of gray between
white (0) and black (1). The rule shown here takes the new gray level of each cell to be
the average of its own gray level and those of its immediate neighbors.

88888

And what we see is that despite the presence of continuous gray levels, the behavior that
is produced exhibits the same kind of complexity that we have seen in many ordinary
cellular automata and other systems with discrete underlying elements.


A continuous cellular automaton with a slightly more complicated rule. The rule takes the
new gray level of each cell to be the fractional part of the average gray level of the cell
and its neighbors multiplied by 3/2. The picture shows that starting from a single black
cell, this rule yields behavior of considerable complexity. Note that the operation
performed on individual average gray levels is exactly iterated map (a) from page 150.

88888

In fact, it turns out that in continuous cellular automata it takes only extremely simple
rules to generate behavior of considerable complexity. So as an example the picture
below shows a rule that determines the new gray level for a cell by just adding the
constant 1/4 to the average gray level for the cell and its immediate neighbors, and then
taking the fractional part of the result.

The facing page and the one after show what happens when one chooses different values
for the constant that is added. A remarkable diversity of behavior is seen. Sometimes the
behavior is purely repetitive, but often it has features that seem effectively random.

And in fact, as the picture in the middle of page 160 shows, it is even possible to find
cases that exhibit localized structures very much like those occasionally seen in ordinary
cellular automata.


A continuous cellular automaton whose rule adds the constant 1/4 to the average gray
level for a cell and its immediate neighbors, and takes the fractional part of the result. The
background simply repeats every 4 steps, but the main pattern has a complex and in many
respects random form.

Continuous cellular automata with the same kind of rules as in the picture above, but with
a variety of different constants being added. Note that it is not so much the size of the
constant as properties like its digit sequence that seem to determine the overall form of
behavior produced in each case.

8888


88888888



More steps in the evolution of continuous cellular automata with the same kind of rules
as on the previous page. In order to remove the uniform stripes, the picture in the middle
shows the difference between the gray level of each cell and its immediate neighbor.
Note the presence of discrete localized structures even though the underlying rules for the
system involve continuous gray levels.

8888888

Partial Differential Equations

By introducing continuous cellular automata with a continuous range of gray levels, we
have successfully removed some of the discreteness that exists in ordinary cellular
automata. But there is nevertheless much discreteness that remains: for a continuous
cellular automaton is still made up of discrete cells that are updated in discrete time steps.

So can one in fact construct systems in which there is absolutely no such discreteness?
The answer, it turns out, is that at least in principle one can, although to do so requires a
somewhat higher level of mathematical abstraction than has so far been necessary in this
book.

The basic idea is to imagine that a quantity such as gray level can be set up to vary
continuously in space and time. And what this means is that instead of just having gray
levels in discrete cells at discrete time steps, one supposes that there exists a definite gray
level at absolutely every point in space and every moment in time--as if one took the limit
of an infinite collection of cells and time steps, with each cell being an infinitesimal size,
and each time step lasting an infinitesimal time.

But how does one give rules for the evolution of such a system? Having no explicit time
steps to work with, one must instead just specify the rate at which the gray level changes
with time at every point in space. And typically one gives this rate as a simple formula

that depends on the gray level at each point in space, and on the rate at which that gray
level changes with position.

Such rules are known in mathematics as partial differential equations, and in fact they
have been widely studied for about two hundred years. Indeed, it turns out that almost all
the traditional mathematical models that have been used in physics and other areas of
science are ultimately based on partial differential equations. Thus, for example,
Maxwell's equations for electromagnetism, Einstein's equations for gravity, Schrödinger's
equation for quantum mechanics and the Hodgkin-Huxley equation for the
electrochemistry of nerve cells are all examples of partial differential equations.

It is in a sense surprising that systems which involve such a high level of mathematical
abstraction should have become so widely used

888888

in practice. For as we shall see later in this book, it is certainly not that nature
fundamentally follows these abstractions.

And I suspect that in fact the current predominance of partial differential equations is in
many respects a historical accident--and that had computer technology been developed
earlier in the history of mathematics, the situation would probably now be very different.

But particularly before computers, the great attraction of partial differential equations was
that at least in simple cases explicit mathematical formulas could be found for their
behavior. And this meant that it was possible to work out, for example, the gray level at a
particular point in space and time just by evaluating a single mathematical formula,
without having in a sense to follow the complete evolution of the partial differential
equation.

The pictures on the facing page show three common partial differential equations that
have been studied over the years.

The first picture shows the diffusion equation, which can be viewed as a limiting case of
the continuous cellular automaton on page 156. Its behavior is always very simple: any
initial gray progressively diffuses away, so that in the end only uniform white is left.

The second picture shows the wave equation. And with this equation, the initial lump of
gray shown just breaks into two identical pieces which propagate to the left and right
without change.

The third picture shows the sine-Gordon equation. This leads to slightly more
complicated behavior than the other equations--though the pattern it generates still has a
simple repetitive form.

Considering the amount of mathematical work that has been done on partial differential
equations, one might have thought that a vast range of different equations would by now
have been studied. But in fact almost all the work--at least in one dimension--has
concentrated on just the three specific equations on the facing page, together with a few
others that are essentially equivalent to them.

And as we have seen, these equations yield only simple behavior.

So is it in fact possible to get more complicated behavior in partial differential equations?
The results in this book on other kinds of systems strongly suggest that it should be. But
traditional

88888



Three partial differential equations that have historically been studied extensively. Just
like in other pictures in this book, position goes across the page, and time down the page.
In each equation u is the gray level at a particular point, D[u, t] is the rate of change
(derivative) of the gray level with time, and D[u,t,t] is the rate of change of that rate of
change (second derivative). Similarly, D[u,x] is the rate of change with position in space,
and D[u,x,x] is the rate of change of that rate of change. On this page and the ones that
follow [165, 166] the initial conditions used are u=Exp[-x^2], D[u,t]=0.

88888

mathematical methods give very little guidance about how to find such behavior. Indeed,
it seems that the best approach is essentially just to search through many different partial
differential equations, looking for ones that turn out to show complex behavior.

But an immediate difficulty is that there is no obvious way to sample possible partial
differential equations. In discrete systems such as cellular automata there are always a
discrete set of possible rules. But in partial differential equations any mathematical
formula can appear.

Nevertheless, by representing formulas as symbolic expressions with discrete sets of
possible components, one can devise at least some schemes for sampling partial
differential equations.

But even given a particular partial differential equation, there is no guarantee that the
equation will yield self-consistent results. Indeed, for a very large fraction of randomly
chosen partial differential equations what one finds is that after just a small amount of
time, the gray level one gets either becomes infinitely large or starts to vary infinitely
quickly in space or time. And whenever such phenomena occur, the original equation can
no longer be used to determine future behavior.

But despite these difficulties I was eventually able to find the partial differential
equations shown on the next two pages [165, 166].

The mathematical statement of these equations is fairly simple. But as the pictures show,
their behavior is highly complex.

Indeed, strangely enough, even though the underlying equations are continuous, the
patterns they produce seem to involve patches that have a somewhat discrete structure.

But the main point that the pictures on the next two pages [165, 166] make is that the
kind of complex behavior that we have seen in this book is in no way restricted to
systems that are based on discrete elements. It is certainly much easier to find and to
study such behavior in these discrete systems, but from what we have learned in this
section, we now know that the same kind of behavior can also occur in completely
continuous systems such as partial differential equations.

88888



Examples of partial differential equations I have found that have more complicated
behavior. The background in each case is purely repetitive, but the main part of the
pattern is complex, and reminiscent of what is produced by continuous cellular automata
and many other kinds of systems discussed in this book.

8888888


888888

Continuous Versus Discrete Systems

One of the most obvious differences between my approach to science based on simple
programs and the traditional approach based on mathematical equations is that programs
tend to involve discrete elements while equations tend to involve continuous quantities.

But how significant is this difference in the end?

One might have thought that perhaps the basic phenomenon of complexity that I have
identified could only occur in discrete systems. But from the results of the last few
sections [8, 9], we know that this is not the case.

What is true, however, is that the phenomenon was immensely easier to discover in
discrete systems than it would have been in continuous ones. Probably complexity is not
in any fundamental sense rarer in continuous systems than in discrete ones. But the point
is that discrete systems can typically be investigated in a much more direct way than
continuous ones.

Indeed, given the rules for a discrete system, it is usually a rather straightforward matter
to do a computer experiment to find out how the system will behave. But given an
equation for a continuous system, it often requires considerable analysis to work out even
approximately how the system will behave. And in fact, in the end one typically has
rather little idea which aspects of what one sees are actually genuine features of the
system, and which are just artifacts of the particular methods and approximations that one
is using to study it.

With all the work that was done on continuous systems in the history of traditional
science and mathematics, there were undoubtedly many cases in which effects related to
the phenomenon of complexity were seen. But because the basic phenomenon of
complexity was not known and was not expected, such effects were probably always
dismissed as somehow not being genuine features of the systems being studied. Yet when
I came to investigate discrete systems there was no


Solutions to the same equations as on the previous page over a longer period of time.
Note the appearance of discrete structures. Particularly in the last picture some details are
sensitive to the numerical approximation scheme used in computing the solution to the
equation.

888888

possibility of dismissing what I saw in such a way. And as a result I was in a sense forced
into recognizing the basic phenomenon of complexity.

But now, armed with the knowledge that this phenomenon exists, it is possible to go back
and look again at continuous systems.

And although there are significant technical difficulties, one finds as the last few sections
[8, 9] have shown that the phenomenon of complexity can occur in continuous systems
just as it does in discrete ones.

It remains much easier to be sure of what is going on in a discrete system than in a
continuous one. But I suspect that essentially all of the various phenomena that we have
observed in discrete systems in the past several chapters can in fact also be found even in
continuous systems with fairly simple rules.

888888

Two Dimensions and Beyond

Introduction

The physical world in which we live involves three dimensions of space. Yet so far in
this book all the systems we have discussed have effectively been limited to just one
dimension.

The purpose of this chapter, therefore, is to see how much of a difference it makes to
allow more than one dimension.

At least in simple cases, the basic idea--as illustrated in the pictures below--is to consider
systems whose elements do not just lie along a one-dimensional line, but instead are
arranged for example on a two-dimensional grid.


Examples of simple arrangements of elements in one, two and three dimensions. In two
dimensions, what is shown is a square grid; triangular and hexagonal grids are also
possible. In three dimensions, what is shown is a cubic lattice; various other lattices,
analogous to those for regular crystals, are also possible--as are arrangements that are not
repetitive.

888888

Traditional science tends to suggest that allowing more than one dimension will have
very important consequences. Indeed, it turns out that many of the phenomena that have
been most studied in traditional science simply do not occur in just one dimension.

Phenomena that involve geometrical shapes, for example, usually require at least two
dimensions, while phenomena that rely on the existence of knotted structures require
three dimensions. But what about the phenomenon of complexity? How much does it
depend on dimension?

It could be that in going beyond one dimension the character of the behavior that we
would see would immediately change. And indeed in the course of this chapter, we will
come across many examples of specific effects that depend on having more than one
dimension.

But what we will discover in the end is that at an overall level the behavior we see is not
fundamentally much different in two or more dimensions than in one dimension. Indeed,
despite what we might expect from traditional science, adding more dimensions does not
ultimately seem to have much effect on the occurrence of behavior of any significant
complexity.

Cellular Automata

The cellular automata that we have discussed so far in this book are all purely one-
dimensional, so that at each step, they involve only a single line of cells. But one can also
consider two-dimensional cellular automata that involve a whole grid of cells, with the
color of each cell being updated according to a rule that depends on its neighbors in all
four directions on the grid, as in the picture below.


The form of the rule for a typical two-dimensional cellular automaton. In the cases
discussed in this section, each cell is either black or white. Usually I consider so-called
totalistic rules in which the new color of the center cell depends only on the average of
the previous colors of its four neighbors, as well as on its own previous color.

888888

TEXT FROM PAGE

The pictures below show what happens with an especially simple rule in which a
particular cell is taken to become black if any of its four neighbors were black on the
previous step.

Starting from a single black cell, this rule just yields a uniformly expanding diamond-
shaped region of black cells. But by changing the rule slightly, one can obtain more
complicated patterns of growth. The pictures below show what happens, for example,
with a rule in which each cell becomes black if just one or all four of its neighbors were
black on the previous step, but otherwise stays the same color as it was before.


Successive steps in the evolution of a two-dimensional cellular automaton whose rule
specifies that a particular cell should become black if any of its neighbors were black on
the previous step. (In the numbering scheme described on page 173 this rule is code
1022.)

Steps in the evolution of a two-dimensional cellular automaton whose rule specifies that a
particular cell should become black if exactly one or all four of its neighbors were black
on the previous step, but should otherwise stay the same color. Starting with a single
black cell, this rule yields an intricate, if very regular, pattern of growth. (In the
numbering scheme on page 173, the rule is code 942.)

8888888

The patterns produced in this case no longer have a simple geometrical form, but instead
often exhibit an intricate structure somewhat reminiscent of a snowflake. Yet despite this
intricacy, the patterns still show great regularity. And indeed, if one takes the patterns
from successive steps and stacks them on top of each other to form a three-dimensional
object, as in the picture below, then this object has a very regular nested structure.

But what about other rules? The facing page and the one that follows show patterns
produced by two-dimensional cellular automata with a sequence of different rules. Within
each pattern there is often considerable complexity. But this complexity turns out to be
very similar to the complexity we have already seen in one-dimensional


A three-dimensional object formed by stacking the two-dimensional patterns from the
bottom of the previous page. Such pictures are the analogs for two-dimensional cellular
automata of the two-dimensional pictures that I often generate for one-dimensional
cellular automata.

88888



Patterns generated by a sequence of two-dimensional cellular automaton rules. The
patterns are produced by starting from a single black square and then running for 22
steps. In each case the base 2 digit sequence for the code number specifies the rule as
follows. The last digit specifies what color the center cell should be if all its neighbors
were white on the previous step, and it too was white. The second-to-last digit specifies
what happens if all the neighbors are white, but the center cell itself is black. And each
earlier digit then specifies what should happen if progressively more neighbors are black.
(Compare page 60.)

888888



Patterns generated by two-dimensional cellular automata from the previous page, but now
after twice as many steps.

888888



Evolution of one-dimensional slices through some of the two-dimensional cellular
automata from the previous two pages [173, 174]. Each picture shows the colors of cells
that lie on the one-dimensional line that goes through the middle of each two-dimensional
pattern. The results are strikingly similar to ones we saw in previous chapters [2, 3] in
purely one-dimensional cellular automata.

888888

cellular automata. And indeed the previous page shows that if one looks at the evolution
of a one-dimensional slice through each two-dimensional pattern the results one gets are
strikingly similar to what we have seen in ordinary one-dimensional cellular automata.

But looking at such slices cannot reveal much about the overall shapes of the two-
dimensional patterns. And in fact it turns out that for all the two-dimensional cellular
automata shown on the last few pages [173, 174, 175], these shapes are always very
regular.

But it is nevertheless possible to find two-dimensional cellular automata that yield less
regular shapes. And as a first example, the picture on the facing page shows a rule that
produces a pattern whose surface has seemingly random irregularities, at least on a small
scale.

In this particular case, however, it turns out that on a larger scale the surface follows a
rather smooth curve. And indeed, as the picture on page 178 shows, it is even possible to
find cellular automata that yield overall shapes that closely approximate perfect circles.

But it is certainly not the case that all two-dimensional cellular automata produce only
simple overall shapes. The pictures on pages 179-181 show one rule, for example, that
does not. The rule is actually rather simple: it just states that a particular cell should
become black whenever exactly three of its eight neighbors--including diagonals--are
black, and otherwise it should stay the same color as it was before.

In order to get any kind of growth with this rule one must start with at least three black
cells. The picture at the top of page 179 shows what happens with various numbers of
black cells. In some cases the patterns produced are fairly simple--and typically stop
growing after just a few steps. But in other cases, much more complicated patterns are
produced, which often apparently go on growing forever.

The pictures on page 181 show the behavior produced by starting from a row of eleven
black cells, and then evolving for several hundred steps. The shapes obtained seem
continually to go on changing, with no simple overall form ever being produced.

And so it seems that there can be great complexity not only in the detailed arrangement of
black and white cells in a two-dimensional cellular automaton pattern, but also in the
overall shape of the pattern.

888888



A two-dimensional cellular automaton that yields a pattern with a rough surface. The rule
used here includes diagonal neighbors, and so involves a total of 8 neighbors for each
cell, as indicated in the icon on the left. The rule specifies that the center cell should
become black if either 3 or 5 of its 8 neighbors were black on the step before, and should
otherwise stay the same color as it was before. The initial condition in the case shown
consists of a row of 7 black cells. In an extension to 8 neighbors of the scheme used in
the pictures a few pages back, the rule has code number 175850.

888888



A cellular automaton that yields a pattern whose shape closely approximates a circle. The
rule used is of the same kind as on the previous page, but now takes the center cell to
become black only if it has exactly 3 black neighbors. If it has 1, 2 or 4 black neighbors
then it stays the same color as it was before, and if it has 5 or more black neighbors, then
it becomes white on the next step (code number 746). The initial condition consists of a
row of 7 black cells, just as in the picture on the previous page. The pattern shown here is
the result of 400 steps in the evolution of the system. After t steps, the radius of the
approximate circle is about 0.37t.

888888

So what about three-dimensional cellular automata? It is straightforward to generalize the
setup for two-dimensional rules to the three-dimensional case. But particularly on a
printed page it is fairly difficult to display the evolution of a three-dimensional cellular
automaton in a way that can readily be assimilated.

Pages 182 and 183 do however show a few examples of three-dimensional cellular
automata. And just as in the two-dimensional case, there are some specific new
phenomena that can be seen. But overall it seems that the basic kinds of behavior
produced are just the same as in one and two dimensions. And in particular, the basic
phenomenon of complexity does not seem to depend in any crucial way on the
dimensionality of the system one looks at.


Patterns produced by evolution according to a simple two-dimensional cellular automaton
rule starting from rows of black cells of various lengths. The rule used specifies that a
particular cell should become black if exactly three out of its eight neighbors (with
diagonal neighbors included) are black (code number 174826). The patterns in the picture
are obtained by 60 steps of evolution according to this rule. The smaller patterns above
have all stopped growing after this number of steps, but many of the other patterns
apparently go on growing forever.

888888



Three-dimensional objects formed by stacking successive two-dimensional patterns
produced in the evolution of the cellular automaton from the previous page. The large
picture on the right shows 200 steps of evolution.

888888



Stages in the evolution of the cellular automaton from the facing page, starting with an
initial condition consisting of a row of 11 black cells.

888888



Examples of three-dimensional cellular automata. In the top set of pictures, the rule
specifies that a cell should become black whenever any of the six neighbors with which it
shares a face were black on the step before. In the bottom pictures, the rule specifies that
a cell should become black only when exactly one of its six neighbors was black on the
step before. In both cases, the initial condition contains a single black cell. In the top
pictures, the limiting shape obtained is a regular octahedron. In the bottom pictures, it is a
nested pattern analogous to the two-dimensional one on page 171.

888888



Further examples of three-dimensional cellular automata, but now with rules that depend
on all 26 neighbors that share either a face or a corner with a particular cell. In the top
pictures, the rule specifies that a cell should become black when exactly one of its 26
neighbors was black on the step before. In the bottom pictures, the rule specifies that a
cell should become black only when exactly two of its 26 neighbors were black on the
step before. In the top pictures, the initial condition contains a single black cell; in the
bottom pictures, it contains a line of three black cells.

88888

Turing Machines

Much as for cellular automata, it is straightforward to generalize Turing machines to two
dimensions. The basic idea--shown in the picture below--is to allow the head of the

Turing machine to move around on a two-dimensional grid rather than just going
backwards and forwards on a one-dimensional tape.

When we looked at one-dimensional Turing machines earlier in this book, we found that
it was possible for them to exhibit complex behavior, but that such behavior was rather
rare.

In going to two dimensions we might expect that complex behavior would somehow
immediately become more common. But in fact what we find is that the situation is
remarkably similar to one dimension.

For Turing machines with two or three possible states, only repetitive and nested
behavior normally seem to occur. With four states, more complex behavior is possible,
but it is still rather rare.

The facing page shows some examples of two-dimensional Turing machines with four
states. Simple behavior is overwhelmingly the most common. But out of a million
randomly chosen rules, there will typically be a few that show complex behavior. Page
186 shows one example where the behavior seems in many respects completely random.


An example of a two-dimensional Turing machine whose head has three possible states.
The black dot represents the position of the head at each step, and the three possible
orientations of the arrow on this dot correspond to the three possible states of the head.
The rule specifies in which of the four possible directions the head should move at each
step. Note that the orientation of the arrow representing the state of the head has no direct
relationship to directions on the grid--or to which way the head will move at the next
step.

888888



Examples of patterns produced by two-dimensional Turing machines whose heads have
four possible states. In each case, all cells are initially white, and one of the rules given

on the left is applied for the specified number of steps. Note that in the later cases shown,
the head often visits the same position on the grid many times.

888888



The path traced out by the head of the two-dimensional Turing machine with rule (e)
from the previous page. There are many seemingly random fluctuations in this path,
though in general it tends to grow to the right.

888888

Substitution Systems and Fractals

One-dimensional substitution systems of the kind we discussed on page 82 can be
thought of as working by progressively subdividing each element they contain into
several smaller elements.

One can construct two-dimensional substitution systems that work in essentially the same
way, as shown in the pictures below.

The next page gives some more examples of two-dimensional substitution systems. The
patterns that are produced are certainly quite intricate. But there is nevertheless great
regularity in their overall forms. Indeed, just like patterns produced by one-dimensional
substitution systems on page 83, all the patterns shown here ultimately have a simple
nested structure.

Why does such nesting occur? The basic reason is that at every step the rules for the
substitution system simply replace each black square with several smaller black squares.
And on subsequent steps, each of these new black squares is then in turn replaced in
exactly the


A two-dimensional substitution system in which each square is replaced by four smaller
squares at every step according to the rule shown on the left. The pattern generated has a
nested form.

888888



Patterns from various two-dimensional substitution systems. In each case what is shown
is the pattern obtained after five steps of evolution according to the rules on the right,
starting with a single black square.

8888888

same way, so that it ultimately evolves to produce an identical copy of the whole pattern.

But in fact there is nothing about this basic process that depends on the squares being
arranged in any kind of rigid grid. And the picture below shows what happens if one just
uses a simple geometrical rule to replace each black square by two smaller black squares.
The result, once again, is that one gets an intricate but highly regular nested pattern.

In a substitution system where black squares are arranged on a grid, one can be sure that
different squares will never overlap. But if there is just a geometrical rule that is used to
replace each black square, then it is possible for the squares produced to overlap, as in the
picture on the next page. Yet at least in this example, the overall pattern that is ultimately
obtained still has a purely nested structure.

The general idea of building up patterns by repeatedly applying geometrical rules is at the
heart of so-called fractal geometry. And the


The pattern obtained by starting with a single black square and then at every step
replacing each black cell with two smaller black cells according to the simple geometrical
rule shown on the left. Note that in applying the rule to a particular square, one must take

account of the orientation of that square. The final pattern obtained has an intricate nested
structure.

88888

pictures on the facing page show several more examples of fractal patterns produced in
this way.

The details of the geometrical rules used are different in each case. But what all the rules
have in common is that they involve replacing one black square by two or more smaller
black squares. And with this kind of setup, it is ultimately inevitable that all the patterns
produced must have a completely regular nested structure.

So what does it take to get patterns with more complicated structure? The basic answer,
much as we saw in one-dimensional substitution systems on page 85, is some form of
interaction between different elements--so that the replacement for a particular element at
a given step can depend not only on the characteristics of that element itself, but also on
the characteristics of other neighboring elements.

But with geometrical replacement rules of the kind shown on the facing page there is a
problem with this. For elements can end up anywhere in the plane, making it difficult to
define an obvious notion of neighbors. And the result of this has been that in traditional
fractal geometry the idea of interaction between elements is not considered--so that all
patterns that are produced have a purely nested form.


The pattern obtained by repeatedly applying the simple geometrical rule shown on the
right. Even though this basic rule does not involve overlapping squares, the pattern
obtained even by step 3 already has squares that overlap. But the overall pattern obtained
after a large number of steps still has a nested form.

88888

Yet if one sets up elements on a grid it is straightforward to allow the replacements for a
given element to depend on its neighbors, as in the picture at the top of the next page.
And if one does this, one immediately gets all sorts of fairly complicated patterns that are
often not just purely nested--as illustrated in the pictures on the next page.

In Chapter 3 we discussed both ordinary one-dimensional substitution systems, in which
every element is replaced at each step, and sequential substitution systems, in which just

a single block of elements are replaced at each step. And what we did to find which block
of elements should be replaced at a given step was to scan the whole sequence of
elements from left to right.


Examples of fractal patterns produced by repeatedly applying the geometrical rules
shown for a total of 12 steps. The details of each pattern are different, but in all cases the
patterns have a nested overall structure. The presence of this nested structure is an
inevitable consequence of the fact that the rule for replacing an element at a particular
position does not depend in any way on other elements.

888888

So how can this be generalized to higher dimensions? On a two-dimensional grid one can
certainly imagine snaking backwards and forwards or spiralling outwards to scan all the
elements. But as soon as one defines any particular order for elements--however they
may be laid out--this in effect reduces one to dealing with a one-dimensional system.

And indeed there seems to be no immediate way to generalize sequential substitution
systems to two or more dimensions. In Chapter 9, however, we will see that with more
sophisticated ideas it is in fact possible in any number of dimensions to set up
substitution systems in which elements are scanned in order--but whatever order is used,
the results are in some sense always the same.


A two-dimensional neighbor-dependent substitution system. The grid of cells is assumed
to wrap around in both its dimensions.

Patterns generated by 8 steps of evolution in various two-dimensional neighbor-
dependent substitution systems.

88888

One feature of systems like cellular automata is that their elements are always set up in a
regular array that remains the same from one step to the next. In substitution systems with

geometrical replacement rules there is slightly more freedom, but still the elements are
ultimately constrained to lie in a two-dimensional plane.

Indeed, in all the systems that we have discussed so far there is in effect always a fixed
underlying geometrical structure which remains unchanged throughout the evolution of
the system.

It turns out, however, that it is possible to construct systems in which there is no such
invariance in basic structure, and in this section I discuss as an example one version of
what I will call network systems.

A network system is fundamentally just a collection of nodes with various connections
between these nodes, and rules that specify how these connections should change from
one step to the next.

At any particular step in its evolution, a network system can be thought of a little like an
electric circuit, with the nodes of the network corresponding to the components in the
circuit, and the connections to the wires joining these components together.

And as in an electric circuit, the properties of the system depend only on the way in
which the nodes are connected together, and not on any specific layout for the nodes that
may happen to be used.

Of course, to make a picture of a network system, one has to choose particular positions
for each of its nodes. But the crucial point is that these positions have no fundamental
significance: they are introduced solely for the purpose of visual representation.

In constructing network systems one could in general allow each node to have any
number of connections coming from it. But at least for the purposes of this section
nothing fundamental turns out to be lost if one restricts oneself to the case in which every
node has exactly two outgoing connections--each of which can then either go to another
node, or can loop back to the original node itself.

With this setup the very simplest possible network consists of just one node, with both
connections from the node looping back, as

88888

in the top picture below. With two nodes, there are already three possible patterns of
connections, as shown on the second line below. And as the number of nodes increases,
the number of possible different networks grows very rapidly.

For most of these networks there is no way of laying out their nodes so as to get a picture
that looks like anything much more than a random jumble of wires. But it is nevertheless

possible to construct many specific networks that have easily recognizable forms, as
shown in the pictures on the facing page.

Each of the networks illustrated at the top of the facing page consists at the lowest level
of a collection of identical nodes. But the remarkable fact that we see is that just by
changing the pattern of


Possible networks formed by having one, two or three nodes, with two connections
coming out of each node. The picture shows all inequivalent cases ignoring labels, but
excludes networks in which there are nodes which cannot be reached by connections
from other nodes.

888888

connections between these nodes it is possible to get structures that effectively
correspond to arrays with different numbers of dimensions.

Example (a) shows a network that is effectively one-dimensional. The network consists
of pairs of nodes that can be arranged in a sequence in which each pair is connected to
one other pair on the left and another pair on the right.

But there is nothing intrinsically one-dimensional about the structure of network systems.
And as example (b) demonstrates, it is just a matter of rearranging connections to get a
network that looks like a two-dimensional rather than a one-dimensional array. Each
individual node in example (b) still has exactly two connections coming out of it, but now
the overall pattern of connections is such that every block of nodes is connected to four
rather than two neighboring blocks, so that the network effectively forms a two-
dimensional square grid.


Examples of networks that correspond to arrays in one, two and three dimensions. At an
underlying level, each network consists just of a collection of nodes with two connections
coming from each node. But by setting up appropriate patterns for these connections, one
can get networks with very different effective geometrical structures.

88888

Example (c) then shows that with appropriate connections, it is also possible to get a
three-dimensional array, and indeed using the same principles an array with any number
of dimensions can easily be obtained.

The pictures below show examples of networks that form infinite trees rather than arrays.
Notice that the first and last networks shown actually have an identical pattern of
connections, but they look different here because the nodes are arranged in a different
way on the page.


Examples of networks that correspond to infinite trees. Note that networks (a) and (c) are
identical, though they look different because the nodes are laid out differently on the
page. All the networks shown are truncated at the leaves of each tree.

888888

In general, there is great variety in the possible structures that can be set up in network
systems, and as one further example the picture below shows a network that forms a
nested pattern.

In the pictures above we have seen various examples of individual networks that might
exist at a particular step in the evolution of a network system. But now we must consider
how such networks are transformed from one step in evolution to the next.

The basic idea is to have rules that specify how the connections coming out of each node
should be rerouted on the basis of the local structure of the network around that node.

But to see the effect of any such rules, one must first find a uniform way of displaying the
networks that can be produced. The pictures at the top of the next page show one possible
approach based on always arranging the nodes in each network in a line across the page.
And although this representation can obscure the geometrical structure


An example of a network that forms a nested geometrical structure. As in all the other
networks shown, each node here is identical, and has just two connections coming out of
it.

8888888

of a particular network, as in the second and third cases above, it more readily allows
comparison between different networks.

In setting up rules for network systems, it is convenient to distinguish the two
connections that come out of each node. And in the pictures above one connection is
therefore always shown going above the line of nodes, while the other is always shown
going below.

The pictures on the facing page show examples of evolution obtained with four different
choices of underlying rules. In the first case, the rule specifies that the "above"
connection from each node should be rerouted so that it leads to the node obtained by
following the "below" connection and then the "above" connection from that node. The
"below" connection is left unchanged.

The other rules shown are similar in structure, except that in cases (c) and (d), the
"above" connection from each node is rerouted so that it simply loops back to the node
itself.

In case (d), the result of this is that the network breaks up into several disconnected
pieces. And it turns out that none of the rules I consider here can ever reconnect these
pieces again. So as a consequence, what I do in the remainder of this section is to track
only the piece that includes the first node shown in pictures such as those


Networks from previous pictures laid out in a uniform way. Network (a) corresponds to a
one-dimensional array, (b) to a two-dimensional array, and (c) to a tree. In the layout
shown here, all the networks have their nodes arranged along a line. Note that in cases (a)
and (b) the connections are arranged so that the arrays effectively wrap around; in case
(c) the leaves of the tree are taken to have connections that loop back to themselves.

8888888

above. And in effect, this then means that other nodes are dropped from the network, so
that the total size of the network decreases.

By changing the underlying rules, however, the number of nodes in a network can also be
made to increase. The basic way this can be done is by breaking a connection coming
from a particular node by inserting a new node and then connecting that new node to
nodes obtained by following connections from the original node.

The pictures on the next page show examples of behavior produced by two rules that use
this mechanism. In both cases, a new node is inserted in the "above" connection from
each existing node in


The evolution of network systems with four different choices of underlying rules.
Successive steps in the evolution are shown on successive lines down the page. In case
(a), the "above" connection of each node is rerouted at each step to lead to the node
reached by following first the below connection and then the above connection from that
node; the below connection is left unchanged. In case (b), the above connection of each
node is rerouted to the node reached by following the above connection and then the
above connection again; the below connection is left unchanged. In case (c), the above
connection of each node is rerouted so as to loop back to the node itself, while the below
connection is left unchanged. And in case (d), the above connection is rerouted so as to
loop back, while the below connection is rerouted to lead to the node reached by
following the above connection. With the "above" connection labelled as 1 and the
"below" connection as 2, these rules correspond to replacing connections {{1}, {2}} at
each node by (a) {{2, 1}, {2}}, (b) {{1, 1}, {2}}, (c) {{}, {2}}, and (d) {{}, {1}}.

888888

the network. In the first case, the connections from the new node are exactly the same as
the connections from the existing node, while in the second case, the "above" and
"below" connections are reversed.

But in both cases the behavior obtained is quite simple. Yet much like neighbor-
independent substitution systems these network systems have the property that exactly
the same operation is always performed at each node on every step.

In general, however, one can set up network systems that have rules in which different
operations are performed at different nodes, depending on the local structure of the
network near each node.

One simple scheme for doing this is based on looking at the two connections that come
out of each node, and then performing one operation if these two connections lead to the
same node, and another if the connections lead to different nodes.

The pictures on the facing page show some examples of what can happen with this
scheme. And again it turns out that the behavior is always quite simple--with the network
having a structure that inevitably grows in an essentially repetitive way.

But as soon as one allows dependence on slightly longer-range features of the network,
much more complicated behavior immediately


Evolution of network systems whose rules involve the addition of new nodes. In both
cases, the new nodes are inserted in the "above" connection from each node. In case (a),
the connections from the new node lead to the same nodes as the connections from the
original node. In case (b), the above and below connections for the new node are
reversed. In the pictures above, new nodes are placed immediately after the nodes that
give rise to them, and gray lines are used to indicate the origin of each node. Note that the
initial conditions consist of a network that contains only a single node.

888888

becomes possible. And indeed, the pictures on the next two pages [202, 203] show
examples of what can happen if the rules are allowed to depend on the number of distinct
nodes reached by following not just one but up to two successive connections from each
node.

With such rules, the sequence of networks obtained no longer needs to form any kind of
simple progression, and indeed one finds that even the total number of nodes at each step
can vary in a way that seems in many respects completely random.

When we discuss issues of fundamental physics in Chapter 9 we will encounter a variety
of other types of network systems--and I suspect that some of these systems will in the
end turn out to be closely related to the basic structure of space and spacetime in our
universe.


Examples of network systems with rules that cause different operations to be performed
at different nodes. Each rule contains two cases, as shown above. The first case specifies
what to do if both connections from a particular node lead to the same node; the second
case specifies what to do when they lead to different nodes. In the rules shown, the

connections from a particular node (indicated by a solid circle) and from new nodes
created from this node always go to the nodes indicated by open circles that are reached
by following just a single above or below connection from the original node. Even if this
restriction is removed, however, more complicated behavior does not appear to be seen.

888888



Network systems in which the rule depends on the number of distinct nodes reached by
going up to distance two away from each node. The plots show the total number of nodes
obtained at each step. In cases (a) and (b), the behavior of the system is eventually
repetitive. In case (c), it is nested--the size of the network at step t is related to the
number of 1's in the base 2 digit sequence of t.

888888



Network systems in which the total number of nodes obtained on successive steps
appears to vary in a largely random way forever. About one in 10,000 randomly chosen
network systems seem to exhibit the kind of behavior shown here.

88888

Multiway Systems

The network systems that we discussed in the previous section do not have any
underlying grid of elements in space. But they still in a sense have a simple one-
dimensional arrangement of states in time. And in fact, all the systems that we have
considered so far in this book can be thought of as having the same simple structure in
time. For all of them are ultimately set up just to evolve progressively from one state to
the next.

Multiway systems, however, are defined so that they can have not just a single state, but a
whole collection of possible states at any given step.

The picture below shows a very simple example of such a system.

Each state in the system consists of a sequence of elements, and in the particular case of
the picture above, the rule specifies that at each step each of these elements either
remains the same or is replaced by a pair of elements. Starting with a single state
consisting of one element, the picture then shows that applying these rules immediately
gives two possible states: one with a single element, and the other with two.

Multiway systems can in general use any sets of rules that define replacements for blocks
of elements in sequences. We already saw exactly these kinds of rules when we discussed
sequential substitution systems on page 88. But in sequential substitution systems the
idea was to do just one replacement at each step. In multiway systems, however,


A very simple multiway system in which one element in each sequence is replaced at
each step by either one or two elements. The main feature of multiway systems is that all
the distinct sequences that result are kept.

888888

the idea is to do all possible replacements at each step--and then to keep all the possible
different sequences that are generated.

The pictures below show what happens with some very simple rules. In each of these
examples the behavior turns out to be rather simple--with for example the number of
possible sequences always increasing uniformly from one step to the next.

In general, however, this number need not exhibit such uniform growth, and the pictures
below show examples where fluctuations occur.


Examples of simple multiway systems. The number of distinct sequences at step t in
these three systems is respectively Ceiling[t/2], t and Fibonacci[t+1] (which increases
approximately like 1.618^t).

Examples of multiway systems with slightly more complicated behavior. The plots on the
right show the total number of possible states obtained at each step, and the differences of
these numbers from one step to the next. In both cases, essentially repetitive behavior is
seen, every 40 and 161 steps respectively. Note that in case (a), the total number of
possible states at step t increases roughly like t^2, while in case (b) it increases only like t.

88888

But in both these cases it turns out to be not too long before these fluctuations essentially
repeat. The picture below shows an example where a larger amount of apparent
randomness is seen. Yet even in this case one finds that there ends up again being
essential repetition--although now only every 1071 steps.


A multiway system with behavior that shows some signs of apparent randomness. The
rule for this system involves three possible replacements. Note that the first replacement
only removes elements and does not insert new ones. In the pictures sequences containing
zero elements therefore sometimes appear. At least with the initial condition used here,
despite considerable early apparent randomness, the differences in number of elements do
repeat (shifted by 1) every 1071 steps.

8888888

If one looks at many multiway systems, most either grow exponentially quickly, or not at
all; slow growth of the kind seen on the facing page is rather rare. And indeed even when
such growth leads to a certain amount of apparent randomness it typically in the end
seems to exhibit some form of repetition. If one allows more rapid growth, however, then
there presumably start to be all sorts of multiway systems that never show any such
regularity. But in practice it tends to be rather difficult to study these kinds of multiway
systems--since the number of states they generate quickly becomes too large to handle.

One can get some idea about how such systems behave, however, just by looking at the
states that occur at early steps. The picture below shows an example--with ultimately
fairly simple nested behavior.

The pictures on the next page show some more examples. Sometimes the set of states that
get generated at a particular step show essential repetition--though often with a long
period. Sometimes this set in effect includes a large fraction of the possible digit
sequences of a given length--and so essentially shows nesting. But in other cases there is

at least a hint of considerably more complexity--even though the total number of states
may still end up growing quite smoothly.


The collections of states generated on successive steps by a simple multiway system with
rapid growth shown on page 205. The particular rule used here eventually generates all
states beginning with a white cell. At step t there are Fibonacci[t+1] states; a given state
with m white cells and n black cells appears at step 2m+n-1.

888888

Looking carefully at the pictures of multiway system evolution on previous pages [204,
205, 206, 207], a feature one notices is that the same sequences often occur on several
different steps. Yet it is a consequence of the basic setup for multiway systems that
whenever any particular sequence occurs, it must always lead to exactly the same
behavior.

So this means that the complete evolution can be represented as in the picture at the top
of the facing page, with each sequence shown explicitly only once, and any sequence
generated more than once indicated just by an arrow going back to its first occurrence.


Collections of states generated at particular steps in the evolution of various multiway
systems. Rule (k) was shown on the previous page; rules (d) and (f) on page 205.

888888

But there is no need to arrange the picture like this: for the whole behavior of the
multiway system can in a sense be captured just by giving the network of what sequence
leads to what other. The picture below shows stages in building up such a network. And
what we see is that just as the network systems that we discussed in the previous section
can build up their own pattern of connections in space, so also multiway systems can in
effect build up their own pattern of connections in time--and this pattern can often be
quite complicated.


The evolution of a multiway system, first with every sequence explicitly shown at each
step, and then with every sequence only ever shown once.

The network built up by the evolution of the multiway system from the top of the page.
This network in effect represents a network of connections in time between states of the
multiway system.

88888

Systems Based on Constraints

In the course of this book we have looked at many different kinds of systems. But in one
respect all these systems have ultimately been set up in the same basic way: they are all
based on explicit rules that specify how the system evolves from step to step.

In traditional science, however, it is common to consider systems that are set up in a
rather different way: instead of having explicit rules for evolution, the systems are just
given constraints to satisfy.

As a simple example, consider a line of cells in which each cell is colored black or white,
and in which the arrangement of colors is subject to the constraint that every cell should
have exactly one black and one white neighbor. Knowing only this constraint gives no
explicit procedure for working out the color of each cell. And in fact it may at first not be
clear that there will be any arrangement of colors that can satisfy the constraint. But it
turns out that there is--as shown below.

And having seen this picture, one might then imagine that there must be many other
patterns that would also satisfy the constraint. After all, the constraint is local to
neighboring cells, so one might suppose that parts of the pattern sufficiently far apart
should always be independent. But in fact this is not true, and instead the system works a
bit like a puzzle in which there is only one way to fit in each piece. And in the end it is
only the perfectly repetitive pattern shown above that can satisfy the required constraint
at every cell.

Other constraints, however, can allow more freedom. Thus, for example, with the
constraint that every cell must have at least one neighbor whose color is different from its
own, any of the patterns in the picture at the top of the facing page are allowed, as indeed
is any pattern that involves no more than two successive cells of the same color.


A system consisting of a line of black and white cells whose form is defined by the
constraint that every cell should have exactly one black and one white neighbor. The
pattern shown is the only possible one that satisfies this constraint. The idea of implicitly
determining the behavior of a system by giving constraints that it must satisfy is common
in traditional science and mathematics.

888888

But while the first arrangement of colors shown above looks somewhat random, the last
two are simple and purely repetitive.

So what about other choices of constraints? We have seen in this book many examples of
systems where simple sets of rules give rise to highly complex behavior. But what about
systems based on constraints? Are there simple sets of constraints that can force complex
patterns?

It turns out that in one-dimensional systems there are not. For in one dimension it is
possible to prove that any local set of constraints that can be satisfied at all can always be
satisfied by some simple and purely repetitive arrangement of colors.

But what about two dimensions? The proof for one dimension breaks down in two
dimensions, and so it becomes at least conceivable that a simple set of constraints could
force a complex pattern to occur.

As a first example of a two-dimensional system, consider an array of black and white
cells in which the constraint is imposed that every black cell should have exactly one
black neighbor, and every white cell should have exactly two white neighbors.


A system consisting of a line of black and white cells whose form is defined by the
constraint that every cell should have at least one neighbor whose color is different from
its own. There are many possible arrangements of colors that satisfy this constraint.
Some, like the first arrangement above, look quite random. But others, like the second
two arrangements above, are simple and repetitive. It turns out that in a one-dimensional
system no set of local constraints can force arrangements of more complicated types.

A system consisting of a grid of black and white cells defined by the constraint that every
black cell should have exactly one black neighbor among its four neighbors, and every
white cell should have exactly two white neighbors. The infinite repetitive pattern shown
here, together with its rotations and reflections, is the only one that satisfies this
constraint. (The picture is assumed to wrap around at each edge.) The pattern can be
viewed as a tessellation of 5×5 blocks of cells.

888888

As in one dimension, knowing the constraint does not immediately provide a procedure
for finding a pattern which satisfies it. But a little experimentation reveals that the simple
repetitive pattern above satisfies the constraint, and in fact it is the only pattern to do so.


Patterns satisfying constraints which specify that every black cell and every white cell
must have a certain fixed number of black and white neighbors. The blank rectangles in
the upper right indicate constraints that cannot be satisfied by any pattern whatsoever.
Most of the constraints are satisfied by a single pattern, together with its rotations and
reflections. In some cases, two distinct patterns are possible, and in a few cases, an
infinite set of patterns are possible. In all cases where the constraints can be satisfied at
all, a simple repetitive pattern nevertheless suffices.

888888

What about other constraints? The pictures on the facing page show schematically what
happens with constraints that require each cell to have various numbers of black and
white neighbors.

Several kinds of results are seen. In the two cases shown as blank rectangles on the upper
right, there are no patterns at all that satisfy the constraints. But in every other case the
constraints can be satisfied, though typically by just one or sometimes two simple infinite
repetitive patterns. In the three cases shown in the center a whole range of mixtures of
different repetitive patterns are possible. But ultimately, in every case where some pattern
can work, a simple repetitive pattern is all that is needed.

So what about more complicated constraints? The pictures below show examples based
on constraints that require the local arrangement of colors around every cell to match a
fixed set of possible templates.

There are a total of 4,294,967,296 possible sets of such templates. And of these,
766,979,044 lead to constraints that cannot be satisfied by any pattern. But among the
3,527,988,252 that remain, it turns out that every single one can be satisfied by a simple
repetitive pattern. In fact the number of different repetitive patterns that are ever needed
is quite small: if a particular constraint can be satisfied by any pattern, then one of the set
of 171 repetitive patterns on the next two pages [214, 215] is always sufficient.


Systems specified by the constraint that the local arrangement of colors around every cell
must match the fixed set of possible templates shown. Note that these templates apply to
every cell, with templates of neighboring cells overlapping. Pattern (a) can be viewed as
formed from a tessellation of 5×10 blocks of cells; pattern (b) from a tessellation of
24×24 blocks. With the numbering scheme for constraints used on the next two pages
[214, 215] the cases shown here correspond to 1384774 and 328778790.

888888


888888



The complete collection of all 171 patterns needed to satisfy constraints of the type
shown on the previous page. If none of these 171 patterns satisfy a particular constraint,
then it follows that no pattern at all will satisfy the constraint. The patterns are labelled by
numbers which specify the minimal constraint which requires the given pattern. Patterns
differing by overall reflection, rotation or interchange of black and white are not shown.

88888

So how can one force more complex patterns to occur?

The basic answer is that one must extend at least slightly the kinds of constraints that one
considers. And one way to do this is to require not only that the colors around each cell
match a set of templates, but also that a particular template from this set must appear at
least somewhere in the array of cells.

The pictures below show a few examples of patterns determined by constraints of this
kind. A typical feature is that the patterns are divided into several separate regions, often
emanating from some kind of center. But at least in all the examples below, the patterns
that occur in each individual region are still simple and repetitive.

So how can one find constraints that force more complex patterns? To do so has been
fairly difficult, and in fact has taken almost as much computational effort as any other
single result in this book.

The basic problem is that given a constraint it can be extremely difficult to find out what
pattern--if any--will satisfy the constraint.

In a system like a cellular automaton that is based on explicit rules, it is always
straightforward to take the rule and apply it to see


Examples of patterns produced by systems in which not only must the arrangement of
colors in each neighborhood match one of a fixed set of templates, but also a certain
template from this set must occur at least once in the pattern. The constraints are
numbered as before, and in each picture the template that must occur is shown at the
center. Constraint 1125528937 leads to a pattern that repeats in 98×98 blocks. The last
pattern shown is also repetitive, repeating every 56 cells on the diagonal.

88888

what pattern is produced. But in a system that is based on constraints, there is no such
direct procedure, and instead one must in effect always go outside of the system to work
out what patterns can occur.

The most straightforward approach might just be to enumerate every single possible
pattern and then see which, if any, of them satisfy a particular constraint. But in systems
containing more than just a few cells, the total number of possible patterns is absolutely
astronomical, and so enumerating them becomes completely impractical.

A more practical alternative is to build up patterns iteratively, starting with a small
region, and then adding new cells in essentially all possible ways, at each stage
backtracking if the constraint for the system does not end up being satisfied.

The pictures on the next page show a few sequences of patterns produced by this method.
In some cases, there emerge quite quickly simple repetitive patterns that satisfy the
constraint. But in other cases, a huge number of possibilities have to be examined in
order to find any suitable pattern.

And what if there is no pattern at all that can satisfy a particular constraint? One might
think that to demonstrate this would effectively require examining every conceivable
pattern on the infinite grid of cells. But in fact, if one can show that there is no pattern
that satisfies the constraint in a limited region, then this proves that no pattern can satisfy
the constraint on the whole grid. And indeed for many constraints, there are already quite
small regions for which it is possible to establish that no pattern can be found.

But occasionally, as in the third picture on the next page, one runs into constraints that
can be satisfied for regions containing thousands of cells, but not for the whole grid. And
to analyze such cases inevitably requires examining huge numbers of possible patterns.

But with an appropriate collection of tricks, it is in the end feasible to take almost any
system of the type discussed here, and determine what pattern, if any, satisfies its
constraint.

So what kinds of patterns can be needed? In the vast majority of cases, simple repetitive
patterns, or mixtures of such patterns, are the only ones that are needed.

88888

But if one systematically examines possible constraints in the order shown on pages 214
and 215, then it turns out that after examining more than 18 million of them, one finally
discovers the system shown on the facing page. And in this system, unlike all others
before it, no repetitive pattern is possible; the only pattern that satisfies the constraint is
the non-repetitive nested pattern shown in the picture.

After testing millions of constraints, and tens of billions of candidate patterns, therefore,
it is finally possible to establish that a system based on simple constraints of the type
discussed here can be forced to exhibit behavior more complex than pure repetition.


Stages in finding patterns that satisfy constraints (a) 4670324, (b) 373384574, and (c)
387520105. Gray is used to indicate cells whose colors have not yet been determined.
The first stage shown in each case corresponds to cells whose colors can be deduced
immediately from the presence of a particular template at the center. In case (a) choices
for additional cells can be made straightforwardly, and an infinite regular pattern can be
built up without any backtracking. In case (b), many choices for additional cells have to
be tried, with much backtracking, and in the end the automatic procedure fails to find a
repetitive pattern. Nevertheless, as the last stage demonstrates, a repetitive pattern does in
fact exist. In case (c), the automatic procedure finds a fairly large and almost regular
pattern that satisfies the constraints, but in this case it turns out that no infinite pattern
exists.

888888



The simplest system based on constraints that is forced to exhibit a non-repetitive pattern.
The constraint requires that the arrangement of colors around each cell must match one of
the 12 templates shown, and that at least somewhere in the pattern a template containing
a pair of stacked black cells must occur. In the numbering scheme used on preceding
pages, the constraint is number 18762389. The pattern shown is unique, in that no
variations of it, except for trivial translations, will satisfy the constraints. The nested
structure on the diagonal essentially corresponds to a progression of base 2 digit
sequences for positive and negative numbers.

888888

What about still more complex behavior?

There are altogether 137,438,953,472 constraints of the type shown on page 216. And of
the millions of these that I have tested, none have forced anything more complicated than
the kind of nested behavior seen on the previous page. But if one extends again the type
of constraints one considers, it turns out to become possible to construct examples that
force more complex behavior.

The idea is to set up templates that involve complete 3×3 blocks of cells, including
diagonal neighbors. The picture below then shows an example of such a system, in which
by allowing only a specific set of 33 templates, a nested pattern is forced to occur.

What about more complex patterns? Searches have not succeeded in finding anything.
But explicit construction, based on correspondence with one-dimensional cellular
automata, leads to the example shown at the top of the facing page: a system with 56
allowed templates in which the only pattern satisfying the constraint is a complex and
largely random one, derived from the rule 30 cellular automaton.


An example of a system based on a constraint involving 3×3 templates of cells. In this
particular system, only the 33 templates shown above (out of the 512 possible ones) are
allowed to occur. This constraint, together with the requirement that the first template
must appear at least somewhere, then turns out to force a nested pattern to occur. The
system shown was specifically constructed in correspondence with the rule 60 elementary
one-dimensional cellular automaton.

888888

So finally this shows that it is indeed possible to force complex behavior to occur in
systems based on constraints. But from what we have seen in this section such behavior
appears to be quite rare: unlike many of the simple rules that we have discussed in this
book, it seems that almost all simple constraints lead only to fairly simple patterns.

Any phenomenon based on rules can always ultimately also be described in terms of
constraints. But the results of this section indicate that these descriptions can have to be
fairly complicated for complex behavior to occur. So the fact that traditional science and
mathematics tends to concentrate on equations that operate like constraints provides yet
another reason for their failure to identify the fundamental phenomenon of complexity
that I discuss in this book.


A system based on a constraint, in which a complex and largely random pattern is forced
to occur. The constraint specifies that only the 56 3×3 templates shown at left can occur
anywhere in the pattern, with the first template appearing at least once. The pattern
required to satisfy this constraint corresponds to a shifted version of the one generated by
the evolution of the rule 30 elementary one-dimensional cellular automaton.

88888

Starting from Randomness

The Emergence of Order

In the past several chapters, we have seen many examples of behavior that simple
programs can produce. But while we have discussed a whole range of different kinds of
underlying rules, we have for the most part considered only the simplest possible initial
conditions--so that for example we have usually started with just a single black cell.

My purpose in this chapter is to go to the opposite extreme, and to consider completely
random initial conditions, in which, for example, every cell is chosen to be black or white
at random.

One might think that starting from such randomness no order would ever emerge. But in
fact what we will find in this chapter is that many systems spontaneously tend to organize
themselves, so that even with completely random initial conditions they end up producing
behavior that has many features that are not at all random.

The picture at the top of the next page shows as a simple first example a cellular
automaton which starts from a typical random initial condition, then evolves down the
page according to the very simple rule that a cell becomes black if either of its neighbors
are black.

What the picture then shows is that every region of white that exists in the initial
conditions progressively gets filled in with black, so that in the end all that remains is a
uniform state with every cell black.

88888

The pictures below show examples of other cellular automata that exhibit the same basic
phenomenon. In each case the initial conditions are random, but the system nevertheless
quickly organizes itself to become either uniformly white or uniformly black.

The facing page shows cellular automata that exhibit slightly more complicated behavior.
Starting from random initial conditions, these cellular automata again quickly settle down
to stable states. But now these stable states are not just uniform in color, but instead
involve a collection of definite structures that either remain fixed on successive steps, or
repeat periodically.

So if they have simple underlying rules, do all cellular automata started from random
initial conditions eventually settle down to give stable states that somehow look simple?


A cellular automaton that evolves to a simple uniform state when started from any
random initial condition. The rule in this case was first shown on page 24, and is number
254 in the scheme described on page 53. It specifies that a cell should become black
whenever either of its neighbors is already black.

Four more examples of cellular automata that evolve from random initial conditions to
completely uniform states. The rules shown here correspond to numbers 0, 32, 160 and
250.

888888

It turns out that they do not. And indeed the picture on the next page shows one of many
examples in which starting from random initial conditions there continues to be very
complicated behavior forever. And indeed the behavior that is produced appears in many
respects completely random. But dotted around the picture one sees many definite white
triangles and other small structures that indicate at least a certain degree of organization.


Examples of cellular automata that evolve from random initial conditions to produce a
definite set of simple structures. For any particular rule, the form of these structures is
always the same. But their positions depend on the details of the initial conditions given,
and in many cases the final arrangement of structures can be thought of as a kind of
filtered version of the initial conditions. Thus for example in the first rule shown here a
structure consisting of a black cell occurs wherever there was an isolated black cell in the
initial conditions. The rules shown are numbers 4, 108, 218 and 232.

88888



A cellular automaton that never settles down to a stable state, but instead continues to
show behavior that seems in many respects random. The rule is number 126.

88888



Other examples of cellular automata that never settle down to stable states when started
from random initial conditions. Each picture is a total of 300 cells across. Note the
presence of triangles and other small structures dotted throughout all of the pictures.

88888

The pictures above and on the previous page show more examples of cellular automata
with similar behavior. There is considerable randomness in the patterns produced in each
case. But despite this randomness there are always triangles and other small structures
that emerge in the evolution of the system.

So just how complex can the behavior of a cellular automaton that starts from random
initial conditions be? We have seen some examples where the behavior quickly stabilizes,
and others where it continues to be quite random forever. But in a sense the greatest
complexity lies between these extremes--in systems that neither stabilize completely, nor
exhibit close to uniform randomness forever.

The facing page and the one that follows show as an example the cellular automaton that
we first discussed on page 32. The initial conditions used are again completely random.
But the cellular automaton quickly organizes itself into a set of definite localized
structures. Yet now these structures do not just remain fixed, but instead move around
and interact with each other in complicated ways. And the result of this is an elaborate
pattern that mixes order and randomness--and is as complex as anything we have seen in
this book.


Two more cellular automata that generate various small structures but continue to show
seemingly quite random behavior forever.

888888



Complex behavior in the rule 110 cellular automaton starting from a random initial
condition. The system quickly organizes itself to produce a set of definite localized
structures, which then move around and interact with each other in complicated ways.

88888



A continuation of the pattern from the previous page. Each page shows 700 steps in the
evolution of the cellular automaton.

8888888

Four Classes of Behavior

In the previous section we saw what a number of specific cellular automata do if one
starts them from random initial conditions. But in this section I want to ask the more
general question of what arbitrary cellular automata do when started from random initial
conditions.

One might at first assume that such a general question could never have a useful answer.
For every single cellular automaton after all ultimately has a different underlying rule,
with different properties and potentially different consequences.

But the next few pages [232, 233, 234] show various sequences of cellular automata, all
starting from random initial conditions.

And while it is indeed true that for almost every rule the specific pattern produced is at
least somewhat different, when one looks at all the rules together, one sees something
quite remarkable: that even though each pattern is different in detail, the number of
fundamentally different types of patterns is very limited.

Indeed, among all kinds of cellular automata, it seems that the patterns which arise can
almost always be assigned quite easily to one of just four basic classes illustrated below.

These classes are conveniently numbered in order of increasing complexity, and each one
has certain immediate distinctive features.

In class 1, the behavior is very simple, and almost all initial conditions lead to exactly the
same uniform final state.


Examples of the four basic classes of behavior seen in the evolution of cellular automata
from random initial conditions. I first developed this classification in 1983.

8888888



The behavior of all cellular automata that involve only nearest neighbors in a
symmetrical way, have two possible colors for each cell, and leave states consisting only
of white cells unchanged.

888888



Totalistic cellular automata whose rules involve nearest and next-nearest neighbors, and
where each cell has two possible colors.

88888



A sequence of totalistic cellular automata with rules that involve only nearest neighbors,
but where each cell can have three possible colors.

888888

In class 2, there are many different possible final states, but all of them consist just of a
certain set of simple structures that either remain the same forever or repeat every few
steps.

In class 3, the behavior is more complicated, and seems in many respects random,
although triangles and other small-scale structures are essentially always at some level
seen.

And finally, as illustrated on the next few pages [236, 237, 238, 239], class 4 involves a
mixture of order and randomness: localized structures are produced which on their own
are fairly simple, but these structures move around and interact with each other in very
complicated ways.

I originally discovered these four classes of behavior some nineteen years ago by looking
at thousands of pictures similar to those on the last few pages [232, 233, 234]. And at
first, much as I have done here, I based my classification purely on the general visual
appearance of the patterns I saw.

But when I studied more detailed properties of cellular automata, what I found was that
most of these properties were closely correlated with the classes that I had already
identified. Indeed, in trying to predict detailed properties of a particular cellular
automaton, it was often enough just to know what class the cellular automaton was in.

And in a sense the situation was similar to what is seen, say, with the classification of
materials into solids, liquids and gases, or of living organisms into plants and animals. At
first, a classification is made purely on the basis of general appearance. But later, when
more detailed properties become known, these properties turn out to be correlated with
the classes that have already been identified.

Often it is possible to use such detailed properties to make more precise definitions of the
original classes. And typically all reasonable definitions will then assign any particular
system to the same class.


Examples of class 4 cellular automata with totalistic rules involving nearest neighbors
and three possible colors for each cell. Each picture shows 1500 steps of evolution from
random initial conditions.

888888


88888


888888


88888


88888

But with almost any general classification scheme there are inevitably borderline cases
which get assigned to one class by one definition and another class by another definition.
And so it is with cellular automata: there are occasionally rules like those in the pictures
below that show some features of one class and some of another.

But such rules are quite unusual, and in most cases the behavior one sees instead falls
squarely into one of the four classes described above.

So given the underlying rule for a particular cellular automaton, can one tell what class of
behavior the cellular automaton will produce?

In most cases there is no easy way to do this, and in fact there is little choice but just to
run the cellular automaton and see what it does.

But sometimes one can tell at least a certain amount simply from the form of the
underlying rule. And so for example all rules that lie in the first two columns on page 232
can be shown to be unable ever to produce anything besides class 1 or class 2 behavior.

In addition, even when one can tell rather little from a single rule, it is often the case that
rules which occur next to each other in some sequence have similar behavior. This can be
seen for example in the pictures on the facing page. The top row of rules all have class 1
behavior. But then class 2 behavior is seen, followed by class 4 and then class 3. And
after that, the remainder of the rules are mostly class 3.

The fact that class 4 appears between class 2 and class 3 in the pictures on the facing page
is not uncommon. For while class 4 is above class 3 in terms of apparent complexity, it is
in a sense intermediate


Rare examples of borderline cellular automata that do not fit squarely into any one of the
four basic classes described in the text. Different definitions based on different specific
properties will place these cellular automata into different classes. The rules shown are
totalistic ones involving nearest neighbors and three possible colors for each cell. The
first rule can be either class 2 or class 4, the second class 3 or 4, the third class 2 or 3 and
the fourth class 1, 2 or 3.

8888888



A sequence of totalistic rules involving nearest neighbors and four possible colors for
each cell chosen to show transitions between rules with different classes of behavior.
Note that class 4 seems to occur between class 2 and class 3.

888888

between class 2 and class 3 in terms of what one might think of as overall activity.

The point is that class 1 and 2 systems rapidly settle down to states in which there is
essentially no further activity. But class 3 systems continue to have many cells that
change at every step, so that they in a sense maintain a high level of activity forever.
Class 4 systems are then in the middle: for the activity that they show neither dies out
completely, as in class 2, nor remains at the high level seen in class 3.

And indeed when one looks at a particular class 4 system, it often seems to waver
between class 2 and class 3 behavior, never firmly settling on either of them.

In some respects it is not surprising that among all possible cellular automata one can
identify some that are effectively on the boundary between class 2 and class 3. But what
is remarkable about actual class 4 systems that one finds in practice is that they have
definite characteristics of their own--most notably the presence of localized structures--
that seem to have no direct relation to being somehow on the boundary between class 2
and class 3.

And it turns out that class 4 systems with the same general characteristics are seen for
example not only in ordinary cellular automata but also in such systems as continuous
cellular automata.

The facing page shows a sequence of continuous cellular automata of the kind we
discussed on page 155. The underlying rules in such systems involve a parameter that can
vary smoothly from 0 to 1.

For different values of this parameter, the behavior one sees is different. But it seems that
this behavior falls into essentially the same four classes that we have already seen in
ordinary cellular automata. And indeed there are even quite direct analogs of for example
the triangle structures that we saw in ordinary class 3 cellular automata.

But since continuous cellular automata have underlying rules based on a continuous
parameter, one can ask what happens if one smoothly varies this parameter--and in
particular one can ask what sequence of classes of behavior one ends up seeing.

The answer is that there are normally some stretches of class 1 or 2 behavior, and some
stretches of class 3 behavior. But at the transitions

888888



Examples of the evolution of continuous cellular automata from random initial
conditions. As discussed on page 155, each cell here can have any gray level between 0
and 1, and at each step the gray level of a given cell is determined by averaging the gray
levels of the cell and its two neighbors, adding the specified constant, and then keeping
only the fractional part of the result. The behavior produced once again falls into distinct
classes that correspond well to the four classes seen on previous pages in ordinary
cellular automata.

888888



Examples of continuous cellular automata that exhibit class 4 behavior. The rules are of
the same kind as in the previous picture, except that in the third case shown here, the gray
level of each neighboring cell is multiplied by 1.13 before the average is done. In
addition, the actual gray levels in these pictures are obtained by taking the difference
between the gray level of each cell and its neighbor, thus removing the uniform stripes
visible in the previous picture. It is remarkable that class 4 behavior with discrete
localized structures can still occur in the continuous systems shown here.

888888

it turns out that class 4 behavior is typically seen--as illustrated on the facing page. And
what is particularly remarkable is that this behavior involves the same kinds of localized
structures and other features that we saw in ordinary discrete class 4 cellular automata.

So what about two-dimensional cellular automata? Do these also exhibit the same four
classes of behavior that we have seen in one dimension? The pictures on the next two
pages [246, 247] show various steps in the evolution of some simple two-dimensional
cellular automata starting from random initial conditions. And just as in one dimension a
few distinct classes of behavior can immediately be seen.

But the correspondence with one dimension becomes much more obvious if one looks
not at the complete state of a two-dimensional cellular automaton at a few specific steps,
but rather at a one-dimensional slice through the system for a whole sequence of steps.

The pictures on page 248 show examples of such slices. And what we see is that the
patterns in these slices look remarkably similar to the patterns we already saw in ordinary

one-dimensional cellular automata. Indeed, by looking at such slices one can readily
identify the very same four classes of behavior as in one-dimensional cellular automata.

So in particular one sees class 4 behavior. In the examples on page 248, however, such
behavior always seems to occur superimposed on some kind of repetitive background--
much as in the case of the rule 110 one-dimensional cellular automaton on page 229.

So can one get class 4 behavior with a simple white background? Much as in one
dimension this does not seem to happen with the very simplest possible kinds of rules.
But as soon as one goes to slightly more complicated rules--though still very simple--one
can find examples.

And so as one example page 249 shows a two-dimensional cellular automaton often
called the Game of Life in which all sorts of localized structures occur even on a white
background. If one watches a movie of the behavior of this cellular automaton its
correspondence to a one-dimensional class 4 system is not particularly obvious. But as
soon as one looks at a one-dimensional slice--as on page 249--what one sees is
immediately strikingly similar to what we have seen in many one-dimensional class 4
cellular automata.

888888



Examples of the evolution of two-dimensional cellular automata with various totalistic
rules starting from random initial conditions. The rules involve a cell and its four
immediate neighbors. Each successive base 2 digit in the code number for the rule gives
the outcome when the total of the cell and its four neighbors runs from 5 down to 0.

888888



Patterns produced after 500 steps in the evolution of a sequence of two-dimensional
cellular automata starting from random initial conditions. The rules shown are of the
same kind as on the facing page, and include most of the 64 possibilities that leave a state
that contains only white cells unchanged.

888888



One-dimensional slices through the evolution of various two-dimensional cellular
automata. In each picture black cells further back from the position of the slice are shown
in progressively lighter shades of gray, as if they were receding into a kind of fog. Note
the presence of examples of both class 3 and class 4 behavior that look strikingly similar
to examples in one dimension.

8888888



The behavior of a class 4 two-dimensional cellular automaton often known in recreational
computing as the Game of Life. Localized structures that move (so-called gliders) show
up as streaks in the pictures given here. The rule for this cellular automaton considers the
8 neighbors of a cell (including diagonals): if two of these neighbors are black, then the
cell stays the same color as before; if three are black, then the cell becomes black; and if
any other number of neighbors are black, then the cell becomes white. This rule is outer
totalistic 9-neighbor code 224. The pictures on the right show cells that were black on
preceding steps in progressively lighter shades of gray.

888888

Sensitivity to Initial Conditions

In the previous section we identified four basic classes of cellular automata by looking at
the overall appearance of patterns they produce. But these four classes also have other
significant distinguishing features--and one important example of these is their sensitivity
to small changes in initial conditions.

The pictures below show the effect of changing the initial color of a single cell in a
typical cellular automaton from each of the four classes of cellular automata identified in
the previous section.

The results are rather different for each class.

In class 1, changes always die out, and in fact exactly the same final state is reached
regardless of what initial conditions were used. In class 2, changes may persist, but they
always remain localized in a small region of the system. In class 3, however, the behavior
is quite different. For as the facing page shows, any change that is made


The effect of changing the color of a single cell in the initial conditions for typical
cellular automata from each of the four classes identified in the previous section. The
black dots indicate all the cells that change. The way that such changes behave is
characteristically different for each of the four classes of systems.

888888



The effect of changing the color of a single initial cell in three typical class 3 cellular
automata.

888888

typically spreads at a uniform rate, eventually affecting every part of the system. In class
4, changes can also spread, but only in a sporadic way--as illustrated on the facing page
and the one that follows.

So what is the real significance of these different responses to changes in initial
conditions? In a sense what they reveal are basic differences in the way that each class of
systems handles information.

In class 1, information about initial conditions is always rapidly forgotten--for whatever
the initial conditions were, the system quickly evolves to a single final state that shows no
trace of them.

In class 2, some information about initial conditions is retained in the final configuration
of structures, but this information always remains completely localized, and is never in
any way communicated from one part of the system to another.

A characteristic feature of class 3 systems, on the other hand, is that they show long-
range communication of information--so that any change made anywhere in the system
will almost always eventually be communicated even to the most distant parts of the
system.

Class 4 systems are once again somewhat intermediate between class 2 and class 3.
Long-range communication of information is in principle possible, but it does not always
occur--for any particular change is only communicated to other parts of the system if it
happens to affect one of the localized structures that moves across the system.

There are many characteristic differences between the four classes of systems that we
identified in the previous section. But their differences in the handling of information are
in some respects particularly fundamental. And indeed, as we will see later in this book, it
is often possible to understand some of the most important features of systems that occur
in nature just by looking at how their handling of information corresponds to what we
have seen in the basic classes of systems that we have identified here.


The effect of small changes in initial conditions in the rule 110 class 4 cellular
automaton. The changes spread only when they are in effect carried by localized
structures that propagates across the system.

888888


88888


888888

Systems of Limited Size and Class 2 Behavior

In the past two sections [2, 3] we have seen two important features of class 2 systems:
first, that their behavior is always eventually repetitive, and second, that they do not
support any kind of long-range communication.

So what is the connection between these two features?

The answer is that the absence of long-range communication effectively forces each part
of a class 2 system to behave as if it were a system of limited size. And it is then a
general result that any system of limited size that involves discrete elements and follows
definite rules must always eventually exhibit repetitive behavior. Indeed, as we will
discuss in the next chapter, it is this phenomenon that is ultimately responsible for much
of the repetitive behavior that we see in nature.

The pictures below show a very simple example of the basic phenomenon. In each case
there is a dot that can be in one of six possible positions. And at every step the dot moves
a fixed number of positions to the right, wrapping around as soon as it reaches the right-
hand end.


A simple system that contains a single dot which can be in one of six possible positions.
At each step, the dot moves some number of positions to the right, wrapping around as
soon as it reaches the right-hand end. The behavior of this system, like other systems of
limited size, is always repetitive.

888888

Looking at the pictures we then see that the behavior which results is always purely
repetitive--though the period of repetition is different in different cases. And the basic
reason for the repetitive behavior is that whenever the dot ends up in a particular position,
it must always repeat whatever it did when it was last in that position.

But since there are only six possible positions in all, it is inevitable that after at most six
steps the dot will always get to a position where it has been before. And this means that
the behavior must repeat with a period of at most six steps.

The pictures below show more examples of the same setup, where now the number of
possible positions is 10 and 11. In all cases, the behavior is repetitive, and the maximum
repetition period is equal to the number of possible positions.


More examples of the type of system shown on the previous page, but now with 10 and
11 possible positions for the dot. The behavior always repeats itself in at most 10 or 11
steps. But the exact number of steps in each case depends on the prime factors of the
numbers that define the system.

888888

Sometimes the actual repetition period is equal to this maximum value. But often it is
smaller. And indeed it is a common feature of systems of limited size that the repetition
period one sees can depend greatly on the exact size of the system and the exact rule that
it follows.

In the type of system shown on the facing page, it turns out that the repetition period is
maximal whenever the number of positions moved at each step shares no common factor
with the total number of possible positions--and this is achieved for example whenever
either of these quantities is a prime number.

The pictures below show another example of a system of limited size based on a simple
rule. The particular rule is at each step to double the number that represents the position
of the dot, wrapping around as soon as this goes past the right-hand end.


A system where the number that represents the position of the dot doubles at each step,
wrapping around whenever it reaches the right-hand end. (After t steps the dot is thus at
position Mod[2^t, n] in a size n system.) The plot at left gives the repetition period for
this system as a function of its size; for odd n this period is equal to
MultiplicativeOrder[2, n].

888888

Once again, the behavior that results is always repetitive, and the repetition period can
never be greater than the total number of possible positions for the dot. But as the picture
shows, the actual repetition period jumps around considerably as the size of the system is
changed. And as it turns out, the repetition period is again related to the factors of the
number of possible positions for the dot--and tends to be maximal in those cases where
this number is prime.

So what happens in systems like cellular automata?

The pictures on the facing page show some examples of cellular automata that have a
limited number of cells. In each case the cells are in effect arranged around a circle, so
that the right neighbor of the rightmost cell is the leftmost cell and vice versa.

And once again, the behavior of these systems is ultimately repetitive. But the period of
repetition is often quite large.

The maximum possible repetition period for any system is always equal to the total
number of possible states of the system.

For the systems involving a single dot that we discussed above, the possible states
correspond just to possible positions for the dot, and the number of states is therefore
equal to the size of the system.

But in a cellular automaton, every possible arrangement of black and white cells
corresponds to a possible state of the system. With n cells there are thus 2^n possible
states. And this number increases very rapidly with the size n: for 5 cells there are already
32 states, for 10 cells 1024 states, for 20 cells 1,048,576 states, and for 30 cells
1,073,741,824 states.

The pictures on the next page show the actual repetition periods for various cellular
automata. In general, a rapid increase with size is characteristic of class 3 behavior. Of
the elementary rules, however, only rule 45 seems to yield periods that always stay close
to the maximum of 2^n. And in all cases, there are considerable fluctuations in the periods
that occur as the size changes.

So how does all of this relate to class 2 behavior? In the examples we have just discussed,
we have explicitly set up systems that have limited size. But even when a system in
principle contains an infinite number of cells it is still possible that a particular pattern in
that system will only grow to occupy a limited number of cells. And in any

8888888



The behavior of cellular automata with a limited number of cells. In each case the right
neighbor of the rightmost cell is taken to be the leftmost cell and vice versa. The pattern
produced always eventually repeats, but the period of repetition can increase rapidly with
the size of the system.

888888

such case, the pattern must repeat itself with a period of at most 2^n steps, where n is the
size of the pattern.

In a class 2 system with random initial conditions, a similar thing happens: since different
parts of the system do not communicate with each other, they all behave like separate
patterns of limited size. And in fact in most class 2 cellular automata these patterns are
effectively only a few cells across, so that their repetition periods are necessarily quite
short.


Repetition periods for various cellular automata as a function of size. The initial
conditions used in each case consist of a single black cell, as in the pictures on the
previous page. The dashed gray line indicates the maximum possible repetition period of
2^n. The maximum repetition period for rule 90 is 2^((n-1)/2) - 1. For rule 30, the peak
repetition periods are of order 2^(0.63 n), while for rule 45, they are close to 2^n (for n =
29, for example, the period is 463,347,935, which is 86% of the maximum possible). For
rule 110, the peaks seem to increase roughly like n^3.

888888

Randomness in Class 3 Systems

When one looks at class 3 systems the most obvious feature of their behavior is its
apparent randomness. But where does this randomness ultimately come from? And is it
perhaps all somehow just a reflection of randomness that was inserted in the initial
conditions?

The presence of randomness in initial conditions--together with sensitive dependence on
initial conditions--does imply at least some degree of randomness in the behavior of any
class 3 system. And indeed when I first saw class 3 cellular automata I assumed that this
was the basic origin of their randomness.

But the crucial point that I discovered only some time later is that random behavior can
also occur even when there is no randomness in initial conditions. And indeed, in earlier
chapters of this book we have already seen many examples of this fundamental
phenomenon.

The pictures below now compare what happens in the rule 30 cellular automaton from
page 27 if one starts from random initial conditions and from initial conditions involving
just a single black cell.


Comparison of the patterns produced by the rule 30 cellular automaton starting from
random initial conditions and from simple initial conditions involving just a single black
cell. Away from the edge of the second picture, the patterns look remarkably similar.

8888888

The behavior we see in the two cases rapidly becomes almost indistinguishable. In the
first picture the random initial conditions certainly affect the detailed pattern that is
obtained. But the crucial point is that even without any initial randomness much of what
we see in the second picture still looks like typical random class 3 behavior.

So what about other class 3 cellular automata? Do such systems always produce
randomness even with simple initial conditions?

The pictures below show an example in which random class 3 behavior is obtained when
the initial conditions are random, but where the pattern produced by starting with a single
black cell has just a simple nested form.

Nevertheless, the pictures on the facing page demonstrate that if one uses initial
conditions that are slightly different--though still simple--then one can still see
randomness in the behavior of this particular cellular automaton.


Patterns produced by the rule 22 cellular automaton starting from random initial
conditions and from an initial condition containing a single black cell. With random
initial conditions typical class 3 behavior is seen. But with the specific initial condition
shown on the right, a simple nested pattern is produced.

88888



Rule 22 with various different simple initial conditions. In the top four cases, the pattern
produced ultimately has a simple nested form. But in the bottom case, it is instead in
many respects random, much like rule 30.

888888

There are however a few cellular automata in which class 3 behavior is obtained with
random initial conditions, but in which no significant randomness is ever produced with
simple initial conditions.

The pictures below show one example. And in this case it turns out that all patterns are in
effect just simple superpositions of the basic nested pattern that is obtained by starting
with a single black cell.

As a result, when the initial conditions involve only a limited region of black cells, the
overall pattern produced always ultimately has a simple nested form. Indeed, at each of
the steps where a new white triangle starts in the center, the whole pattern consists just of
two copies of the region of black cells from the initial conditions.

The only way to get a random pattern therefore is to have an infinite number of randomly
placed black cells in the initial conditions.


Patterns generated by rule 90 with various initial conditions. This particular cellular
automaton rule has the special property of additivity which implies that with any initial
conditions the patterns that it produces can be obtained as simple superpositions of the

first pattern shown above. Any initial condition that contains black cells only in a limited
region will thus lead to a pattern that ultimately has a simple nested form. Unlike rule 30
or rule 22 therefore, rule 90 cannot intrinsically generate randomness starting from
simple initial conditions. The randomness in the last picture shown here is thus purely a
consequence of the randomness in its initial conditions. Note that the pictures above show
only half as many steps of evolution as the corresponding pictures of rule 22 on the
previous page.

888888

And indeed when random initial conditions are used, rule 90 does manage to produce
random behavior of the kind expected in class 3.

But if there are deviations from perfect randomness in the initial conditions, then these
will almost inevitably show up in the evolution of the system. And thus, for example, if
the initial density of black cells is low, then correspondingly low densities will occur
again at various later steps, as in the second picture below.

With rule 22, on the other hand, there is no such effect, and instead after just a few steps
no visible trace remains of the low density of initial black cells.


Examples of evolution from random initial conditions with a low density of black cells.
In rule 22 the low initial density has no long-term effect. But in rule 90 its effect
continues forever. The reason for this difference is that in rule 22 the randomness we see
is intrinsically generated by the evolution of the system, while in rule 90 it comes from
randomness in the initial conditions.

8888888

A couple of sections ago we saw that all class 3 systems have the property that the
detailed patterns they produce are highly sensitive to detailed changes in initial
conditions. But despite this sensitivity at the level of details, the point is that any system
like rule 22 or rule 30 yields patterns whose overall properties depend very little on the
form of the initial conditions that are given.

By intrinsically generating randomness such systems in a sense have a certain
fundamental stability: for whatever is done to their initial conditions, they still give the
same overall random behavior, with the same large-scale properties. And as we shall see

in the next few chapters, there are in fact many systems in nature whose apparent stability
is ultimately a consequence of just this kind of phenomenon.

Special Initial Conditions

We have seen that cellular automata such as rule 30 generate seemingly random behavior
when they are started both from random initial conditions and from simple ones. So one
may wonder whether there are in fact any initial conditions that make rule 30 behave in a
simple way.

As a rather trivial example, one certainly knows that if its initial state is uniformly white,
then rule 30 will just yield uniform white forever. But as the pictures below demonstrate,
it is also possible to find less trivial initial conditions that still make rule 30 behave in a
simple way.


Examples of special initial conditions that make the rule 30 cellular automaton yield
simple repetitive behavior. Small patches with the same structures as shown here can be
seen embedded in typical random patterns produced by rule 30. At left is a representation
of rule 30. Finding initial conditions that make cellular automata yield behavior with
certain repetition periods is closely related to the problem of satisfying constraints
discussed on page 210.

8888888

In fact, it turns out that in any cellular automaton it is inevitable that initial conditions
which consist just of a fixed block of cells repeated forever will lead to simple repetitive
behavior.

For what happens is that each block in effect independently acts like a system of limited
size. The right-hand neighbor of the rightmost cell in any particular block is the leftmost
cell in the next block, but since all the blocks are identical, this cell always has the same
color as the leftmost cell in the block itself. And as a result, the block evolves just like
one of the systems of limited size that we discussed on page 255. So this means that
given a block that is n cells wide, the repetition period that is obtained must be at most
2^n steps.

But if one wants a short repetition period, then there is a question of whether there is a
block of any size which can produce it. The pictures on the next page show the blocks
that are needed to get repetition periods of up to ten steps in rule 30. It turns out that no

block of any size gives a period of exactly two steps, but blocks can be found for all
larger periods at least up to 15 steps.

But what about initial conditions that do not just consist of a single block repeated
forever? It turns out that for rule 30, no other kind of initial conditions can ever yield
repetitive behavior.

But for many rules--including a fair number of class 3 ones--the situation is different.
And as one example the picture on the right below shows an initial condition for rule 126
that involves two different blocks but which nevertheless yields repetitive behavior.


Rule 126 with a typical random initial condition, and with an initial condition that
consists of a random sequence of the blocks and . Rule 126 in general shows class
3 behavior, as on the left. But with the special initial condition on the right it acts like a
simple class 2 rule. Note the patches of class 2 behavior even in the picture on the left.

8888888



All patterns that repeat in 10 or less steps under evolution according to rule 30. In each
case the initial conditions consist of a fixed block of cells that is repeated over and over
again. Note that there are no initial conditions that yield a repetition period of exactly 2
steps. To get period 11, a block that contains 275 cells is required.

888888

In a sense what is happening here is that even though rule 126 usually shows class 3
behavior, it is possible to find special initial conditions that make it behave like a simple
class 2 rule.

And in fact it turns out to be quite common for there to exist special initial conditions for
one cellular automaton that make it behave just like some other cellular automaton.

Rule 126 will for example behave just like rule 90 if one starts it from special initial
conditions that contain only blocks consisting of pairs of black and white cells.

The pictures below show how this works: on alternate steps the arrangement of blocks in
rule 126 corresponds exactly to the arrangement of individual cells in rule 90. And
among other things this explains why it is that with simple initial conditions rule 126
produces exactly the same kind of nested pattern as rule 90.


Two examples of the fact that with special initial conditions rule 126 behaves exactly like
rule 90. The initial conditions that are used consist of blocks of cells where each block
contains either two black cells or two white cells. If one looks only on every other step,
then the blocks behave exactly like individual cells in rule 90. This correspondence is the
basic reason that rule 126 produces the same kind of nested patterns as rule 90 when it is
started from simple initial conditions.

888888

The point is that these initial conditions in effect contain only blocks for which rule 126
behaves like rule 90. And as a result, the overall patterns produced by rule 126 in this
case are inevitably exactly like those produced by rule 90.

So what about other cellular automata that can yield similar patterns? In every example in
this book where nested patterns like those from rule 90 are obtained it turns out that the
underlying rules that are responsible can be set up to behave exactly like rule 90.
Sometimes this will happen, say, for any initial condition that has black cells only in a
limited region. But in other cases--like the example of rule 22 on page 263--rule 90
behavior is obtained only with rather specific initial conditions.

So what about rule 90 itself? Why does it yield nested patterns?

The basic reason can be thought of as being that just as other rules can emulate rule 90
when their initial conditions contain only certain blocks, so also rule 90 is able to emulate
itself in this way.

The picture below shows how this works. The idea is to consider the initial conditions not
as a sequence of individual cells, but rather as a sequence of blocks each containing two
adjacent cells. And with an appropriate form for these blocks what one finds is that the
configuration of blocks evolves exactly according to rule 90.

The fact that both individual cells and whole blocks of cells evolve according to the same
rule then means that whatever pattern is


A demonstration of the fact that in rule 90 blocks of cells can behave just like individual
cells. One consequence of this is that the patterns produced by rule 90 have a nested or
self-similar form.

888888

produced must have exactly the same structure whether it is looked at in terms of
individual cells or in terms of blocks of cells. And this can be achieved in only two ways:
either the pattern must be essentially uniform, or it must have a nested structure--just like
we see in rule 90.

So what happens with other rules? It turns out that the property of self-emulation is rather
rare among cellular automaton rules. But one other example is rule 150--as illustrated in
the picture below.

So what else is there in common between rule 90 and rule 150? It turns out that they are
both additive rules, implying that the patterns they produce can be superimposed in the
way we discussed on page 264. And in fact one can show that any rule that is additive
will be able to emulate itself and will thus yield nested patterns. But there are rather few
additive rules, and indeed with two colors and nearest neighbors the only fundamentally
different ones are precisely rules 90 and 150.

Ultimately, however, additive rules are not the only ones that can emulate themselves. An
example of another kind is rule 184, in which blocks of three cells can act like a single
cell, as shown below.


Another example of a rule in which blocks of cells can behave just like individual cells.
Rule 90 and rule 150 are also essentially the only fundamentally different elementary
cellular automaton rules that have the property of being additive (see page 264).

A rule that is not additive, but in which blocks of cells can again behave just like
individual cells.

8888888

With simple initial conditions of the type we have used so far this rule will always
produce essentially trivial behavior. But one way to see the properties of the rule is to use
nested initial conditions, obtained for example from substitution systems of the kind we
discussed on page 82.

With most rules, including 90 and 150, such nested initial conditions typically yield
results that are ultimately indistinguishable from those obtained with typical random
initial conditions. But for rule 184, an appropriate choice of nested initial conditions
yields the highly regular pattern shown below.


The pattern produced by rule 184 (shown at left) evolving from a nested initial condition.
The particular initial condition shown can be obtained by applying the substitution
system -> , -> , starting from a single black element (see page 83). With this
initial condition, rule 184 exhibits an equal number of black and white stripes, which
annihilate in pairs so as to yield a regular nested pattern.

888888

The nested structure seen in this pattern can then be viewed as a consequence of the fact
that rule 184 is able to emulate itself. And the picture below shows that rule 184--unlike
any of the additive rules--still produces recognizably nested patterns even when the initial
conditions that are used are random.

As we will see on page 338 the presence of such patterns is particularly clear when there
are equal numbers of black and white cells in the initial conditions--but how these cells
are arranged does not usually matter much at all. And in general it is possible to find
quite a few cellular automata that yield nested patterns like rule 184 even from random
initial conditions. The picture on the next page shows a particularly striking example in
which explicit regions are formed that contain patterns with the same overall structure as
rule 90.


Rule 184 evolving from a random initial condition. Nested structure similar to what we
saw in the previous picture is still visible. The presence of such structure is most obvious
when there are equal numbers of black and white cells in the initial conditions, but it does
not rely on any regularity in the arrangement of these cells.

888888



Another example of a cellular automaton that produces a nested pattern even from
random initial conditions. The particular rule shown involves next-nearest as well as
nearest neighbors and has rule number 4067213884. As in rule 184, the nested behavior
seen here is most obvious when the density of black and white cells in the initial
conditions is equal.

888888

The Notion of Attractors

In this chapter we have seen many examples of patterns that can be produced by starting
from random initial conditions and then following the evolution of cellular automata for
many steps.

But what can be said about the individual configurations of black and white cells that
appear at each step? In random initial conditions, absolutely any sequence of black and
white cells can be present. But it is a feature of most cellular automata that on subsequent
steps the sequences that can be produced become progressively more restricted.

The first picture below shows an extreme example of a class 1 cellular automaton in
which after just one step the only sequences that can occur are those that contain only
black cells.

The resulting configuration can be thought of as a so-called attractor for the cellular
automaton evolution. It does not matter what initial conditions one starts from: one
always reaches the same all-black attractor in the end. The situation is somewhat similar
to what happens in a mechanical system like a physical pendulum. One can start the
pendulum swinging in any configuration, but it will always tend to evolve to the
configuration in which it is hanging straight down.

The second picture above shows a class 2 cellular automaton that once again evolves to
an attractor after just one step. But now the attractor does not just consist of a single
configuration, but instead


Examples of simple cellular automata that evolve after just one step to attractors in which
only certain sequences of black and white cells can occur. In the first case, the sequences
that can occur are ones that involve only black cells. In the second case, the sequences are
ones in which every black cell is surrounded by white cells. The rules shown are numbers
255 and 4.

888888

consists of all configurations in which black cells occur only when they are surrounded
on each side by at least one white cell.

The picture below shows that for any particular configuration of this kind, there are in
general many different initial conditions that can lead to it. In a mechanical analogy each
possible final configuration is like the lowest point in a basin--and a ball started anywhere
in the basin will then always roll to that lowest point.

For one-dimensional cellular automata, it turns out that there is a rather compact way to
summarize all the possible sequences of black and white cells that can occur at any given
step in their evolution.

The basic idea is to construct a network in which each such sequence of black and white
cells corresponds to a possible path.

In the pictures at the top of the facing page, the first network in each case represents
random initial conditions in which any possible sequence of black and white cells can
occur. Starting from the node in the middle, one can go around either the left or the right
loop in the network any number of times in any order--representing the fact that black
and white cells can appear any number of times in any order.

At step 2 in the rule 255 example on the facing page, however, the network has only one
loop--representing the fact that at this step the only sequences which can occur with this
rule are ones that consist purely of black cells, just as we saw on the previous page.

The case of rule 4 is slightly more complicated: at step 2, the possible sequences that can
occur are now represented by a network with two nodes. Starting at the right-hand node

one can go around the loop to the right any number of times, corresponding to sequences
of


Four different initial conditions that all lead to the same final state in the rule 4 cellular
automaton shown on the previous page. The final state can be thought of as one of the
possible attractors for the evolution of the cellular automaton; the initial conditions
shown then represent different elements in the basin of attraction for this attractor.

88888888

any number of white cells. At any point one can follow the arrow to the left to get a black
cell, but the form of the network implies that this black cell must always be followed by
at least one white cell.

The pictures on the next page show more examples of class 1 and 2 cellular automata.
Unlike in the picture above, these rules do not reach their final states after one step, but
instead just progressively evolve towards these states. And in the course of this evolution,
the set of sequences that can occur becomes progressively smaller.

In rule 128, for example, the fact that regions of black shrink by one cell on each side at
each step means that any region of black that exists after t steps must have at least t white
cells on either side of it.

The networks shown on the next page capture all effects like this. And to do this we see
that on successive steps they become somewhat more complicated. But at least for these
class 1 and 2 examples, the progression of networks always continues to have a fairly
simple form.


Networks representing possible sequences of black and white cells that can occur at
successive steps in the evolution of the two cellular automata shown on the left. In each
case the possible sequences correspond to possible paths through the network. Both rules
start on step 1 from random initial conditions in which all sequences of black and white
cells are allowed. On subsequent steps, rule 255 allows only sequences containing just
black cells, while rule 4 allows sequences that contain both black and white cells, but
requires that every black cell be surrounded by white cells.

8888888

TEXT FROM PAGE

So what happens with class 3 and 4 systems? The pictures on the facing page show a
couple of examples. In rule 126, the only effect at step 2 is that black cells can no longer
appear on their own: they must always be in groups of two or more. By step 3, it becomes
difficult to see any change if one just looks at an explicit picture of the cellular automaton
evolution. But from the network, one finds that now an infinite collection of other blocks
are forbidden, beginning with the length 12 block . And on later steps, the
set of sequences that are allowed rapidly becomes more complicated--as reflected in a
rapid increase in the complexity of the corresponding networks.


successive steps in the evolution of several class 1 and 2 cellular automata. These
networks never have more than about t^2 nodes after t steps.

88888

Indeed, this kind of rapid increase in network complexity is a general characteristic of
most class 3 and 4 rules. But it turns out that there are a few rules which at first appear to
be exceptions.

The pictures at the top of the next page show four different rules that each have the
property that if started from initial conditions in which all possible sequences of cells are
allowed, these same sequences can all still occur at any subsequent step in the evolution.

The first two rules that are shown exhibit very simple class 2 behavior. But the last two
show typical class 3 behavior.

What is going on, however, is that in a sense the particular initial conditions that allow all
possible sequences are special for these rules.


successive steps in the evolution of typical class 3 and 4 cellular automata. The number
of nodes in these networks seems to increase at a rate that is at least exponential.

888888

And indeed if one starts with almost any other initial conditions--say for example ones
that do not allow any pair of black cells together, then as the pictures below illustrate,
rapidly increasing complexity in the sets of sequences that are allowed is again observed.


Examples of cellular automata which continue to allow all possible sequences of black
and white cells at any step in their evolution. Such cellular automata in effect define what
are known as surjective or onto mappings.

Networks representing possible sequences that can occur in the evolution of the cellular
automata at the top of the page, starting from initial conditions in which black cells are
only allowed to appear in pairs.

888888

Structures in Class 4 Systems

The next page shows three typical examples of class 4 cellular automata. In each case the
initial conditions that are used are completely random. But after just a few steps, the
systems organize themselves to the point where definite structures become visible.

Most of these structures eventually die out, sometimes in rather complicated ways. But a
crucial feature of any class 4 systems is that there must always be certain structures that
can persist forever in it.

So how can one find out what these structures are for a particular cellular automaton?
One approach is just to try each possible initial condition in turn, looking to see whether
it leads to a new persistent structure. And taking the code 20 cellular automaton from the
top of the next page, the page that follows shows what happens in this system with each
of the first couple of hundred possible initial conditions.

In most cases everything just dies out. But when we reach initial condition number 151
we finally see a structure that persists.

This particular structure is fairly simple: it just remains fixed in position and repeats
every two steps. But not all persistent structures are that simple. And indeed at initial
condition 187 we see a considerably more complicated structure, that instead of staying
still moves systematically to the right, repeating its basic form only every 9 steps.

The existence of structures that move is a fundamental feature of class 4 systems. For as
we discussed on page 252, it is these kinds of structures that make it possible for
information to be communicated from one part of a class 4 system to another--and that
ultimately allow the complex behavior characteristic of class 4 to occur.

But having now seen the structure obtained with initial condition 187, we might assume
that all subsequent structures that arise in the code 20 cellular automaton must be at least
as complicated. It turns out, however, that initial condition 189 suddenly yields a much
simpler structure--that just stays unchanged in one position at every step.

But going on to initial condition 195, we again find a more complicated structure--this
time one that repeats only every 22 steps.

88888



Three typical examples of class 4 cellular automata. In each case various kinds of
persistent structures are seen.

88888



The behavior of the code 20 cellular automaton from the top of the facing page for all
initial conditions with black cells in a region of size less than nine. In most cases the
patterns produced simply die out. But with some initial conditions, persistent structures
are formed. Each initial condition is assigned a number whose base 2 digit sequence
gives the configuration of black and white cells in that initial condition. Note that initial

conditions 195 and 219 both yield the period 22 persistent structure shown on the next
page.

888888

So just what set of structures does the code 20 cellular automaton ultimately support?
There seems to be no easy way to tell, but the picture below shows all the structures that I
found by explicitly looking at evolution from the first twenty-five billion possible initial
conditions.

Are other structures possible? The largest structure in the picture above starts from a
block that is 30 cells wide. And with the more than ten billion blocks between 30 and 34
cells wide, no new structures at all appear. Yet in fact other structures are possible. And
the way to tell this is that for small repetition periods there is a systematic procedure that
allows one to find absolutely all structures with a given period.

The picture on the facing page shows the results of using this procedure for repetition
periods up to 15. And for all repetition periods up to 10--with the exception of 7--at least
one fixed or moving structure ultimately turns out to exist. Often, however, the smallest
structures for a given period are quite large, so that for example in the case of period 6
the smallest possible structure is 64 cells wide.


Persistent structures found by testing the first twenty-five billion possible initial
conditions for the code 20 cellular automaton shown on the previous page. Note that
reflected versions of the structures shown are also possible. The base 2 digit sequences of
the numbers given correspond to the initial conditions in each case, as on the previous
page.

8888888

So what about other class 4 cellular automata--like the ones I showed at the beginning of
this section? Do they also end up having complicated sets of possible persistent
structures?


All the persistent structures with repetition periods up to 15 steps in the code 20 cellular
automaton. The structures shown were found by a systematic method similar to the one
used to find all sequences that satisfy the constraints on page 268.

8888888

The picture below shows the structures one finds by explicitly testing the first two billion
possible initial conditions for the code 357 cellular automaton from page 282.

Already with initial condition number 28 a fairly complicated structure with repetition
period 48 is seen. But with all the first million initial conditions, only one other structure
is produced, and this structure is again one that does not move.

So are moving structures in fact possible in the code 357 cellular automaton? My
experience with many different rules is that whenever sufficiently complicated persistent
structures occur, structures that move can eventually be found. And indeed with code
357, initial condition 4,803,890 yields just such a structure.


Persistent structures in the code 357 cellular automaton from page 282 obtained by
testing the first two billion possible initial conditions. This cellular automaton allows
three possible colors for each cell; the initial conditions thus correspond to the base 3
digits of the numbers given. No persistent structures of any size exist in this cellular
automaton with repetition periods of less than 5 steps.

888888

So if moving structures are inevitable in class 4 systems, what other fundamentally
different kinds of structures might one see if one were to look at sufficiently many large
initial conditions?

The picture below shows the first few persistent structures found in the code 1329
cellular automaton from the bottom of page 282. The smallest structures are stationary,
but at initial condition 916 a structure is found that moves--all much the same as in the
two other class 4 cellular automata that we have just discussed.

But when initial condition 54,889 is reached, one suddenly sees the rather different kind
of structure shown on the next page. The right-hand part of this structure just repeats with
a period of 256 steps, but as this part moves, it leaves behind a sequence of other

persistent structures. And the result is that the whole structure continues to grow forever,
adding progressively more and more cells.


Persistent structures in the code 1329 cellular automaton shown on page 282.

8888888

Yet looking at the picture above, one might suppose that when unlimited growth occurs,
the pattern produced must be fairly complicated. But once again code 1329 has a surprise
in store. For the facing page shows that when one reaches initial condition 97,439 there is
again unlimited growth--but now the pattern that is produced is very simple. And in fact
if one were just to see this pattern, one would probably assume that it came from a rule
whose typical behavior is vastly simpler than code 1329.


Unbounded growth in code 1329. The initial condition contains a block of 10 cells. The
right-hand side of the pattern repeats every 256 steps, and as it moves it leaves behind an
infinite sequence of persistent structures.

888888



Further examples of unbounded growth in code 1329. Most of the patterns produced are
complex--but some are simple.

888888



A typical example of the behavior of the rule 110 cellular automaton with random initial
conditions. The background pattern consists of blocks of 14 cells that repeat every 7
steps.

888888

Indeed, it is a general feature of class 4 cellular automata that with appropriate initial
conditions they can mimic the behavior of all sorts of other systems. And when we
discuss computation and the notion of universality in Chapter 11 we will see the
fundamental reason this ends up being so. But for now the main point is just how diverse
and complex the behavior of class 4 cellular automata can be--even when their
underlying rules are very simple.

And perhaps the most striking example is the rule 110 cellular automaton that we first
saw on page 32. Its rule is extremely simple--involving just nearest neighbors and two
colors of cells. But its overall behavior is as complex as any system we have seen.

The facing page shows a typical example with random initial conditions. And one
immediate slight difference from other class 4 rules that we have discussed is that
structures in rule 110 do not exist on a blank background: instead, they appear as
disruptions in a regular repetitive pattern that consists of blocks of 14 cells repeating
every 7 steps.

The next page shows the kinds of persistent structures that can be generated in rule 110
from blocks less than 40 cells wide. And just like in other class 4 rules, there are
stationary structures and moving structures--as well as structures that can be extended by
repeating blocks they contain.

So are there also structures in rule 110 that exhibit unbounded growth? It is certainly not
easy to find them. But if one looks at blocks of width 41, then such structures do
eventually show up, as the picture on page 293 demonstrates.

So how do the various structures in rule 110 interact? The answer, as pages 294-296
demonstrate, can be very complicated.

In some cases, one structure essentially just passes through another with a slight delay.
But often a collision between two structures produces a whole cascade of new structures.
Sometimes the outcome of a collision is evident after a few steps. But quite often it takes
a very large number of steps before one can tell for sure what is going to happen.

So even though the individual structures in class 4 systems like rule 110 may behave in
fairly repetitive ways, interactions between these structures can lead to behavior of
immense complexity.

888888



Persistent structures found in rule 110. Extended versions exist of all but structures (a)
and (j). Structures (m) and (n) also exist in alternate forms shifted with respect to the
background.

888888



An example of unbounded growth in rule 110. The initial condition consists of a block of
length 41 inserted between blocks of the background. New structures on both left and
right are produced every 77 steps; the central structure moves 20 cells to the left during
each cycle so that the structures on the left are separated by 37 steps while those on the
right are separated by 107 steps.

8888888



Collisions between persistent structures (o) and (j) from page 292. (The first structure is
actually an extended form containing four copies of structure (o) from page 292.) Each

successive picture shows what happens when the original structures are started
progressively further apart.

8888888



Collisions between structures (o) and (e) from page 292.

888888



A collision between structures (l) and (i) from page 292. It takes more than 4000 steps for
the final outcome involving 8 separate structures to become clear. The height of the
picture corresponds to 2000 steps, and the third picture ends at step 4300.

8888888

Mechanisms in
Programs and Nature

Universality of Behavior

In the past several chapters my main purpose has been to address the fundamental
question of how simple programs behave. In this chapter my purpose is now to take what
we have learned and begin applying it to the study of actual phenomena in nature.

At the outset one might have thought this would never work. For one might have
assumed that any program based on simple rules would always lead to behavior that was
much too simple to be relevant to most of what we see in nature. But one of the main
discoveries of this book is that programs based on simple rules do not always produce
simple behavior.

And indeed in the past several chapters we have seen many examples where remarkably
simple rules give rise to behavior of great complexity. But to what extent is the behavior
obtained from simple programs similar to behavior we see in nature?

One way to get some idea of this is just to look at pictures of natural systems and
compare them with pictures of simple programs.

At the level of details there are certainly differences. But at an overall level there are
striking similarities. And indeed it is quite remarkable just how often systems in nature
end up showing behavior that looks almost identical to what we have seen in some simple
program or another somewhere in this book.

888888

So why might this be? It is not, I believe, any kind of coincidence, or trick of perception.
And instead what I suspect is that it reflects a deep correspondence between simple
programs and systems in nature.

When one looks at systems in nature, one of the striking things one notices is that even
when systems have quite different underlying physical, biological or other components
their overall patterns of behavior can often seem remarkably similar.

And in my study of simple programs I have seen essentially the same phenomenon: that
even when programs have quite different underlying rules, their overall behavior can be
remarkably similar.

So this suggests that a kind of universality exists in the types of behavior that can occur,
independent of the details of underlying rules.

And the crucial point is that I believe that this universality extends not only across simple
programs, but also to systems in nature. So this means that it should not matter much
whether the components of a system are real molecules or idealized black and white cells;
the overall behavior produced should show the same universal features.

And if this is the case, then it means that one can indeed expect to get insight into the
behavior of natural systems by studying the behavior of simple programs. For it suggests
that the basic mechanisms responsible for phenomena that we see in nature are somehow
the same as those responsible for phenomena that we see in simple programs.

In this chapter my purpose is to discuss some of the most common phenomena that we
see in nature, and to study how they correspond with phenomena that occur in simple
programs.

Some of the phenomena I discuss have at least to some extent already been analyzed by
traditional science. But we will find that by thinking in terms of simple programs it

usually becomes possible to see the basic mechanisms at work with much greater clarity
than before.

And more important, many of the phenomena that I consider--particularly those that
involve significant complexity--have never been satisfactorily explained in the context of
traditional science. But what we will find in this chapter is that by making use of my
discoveries about simple programs a great many of these phenomena can now for the first
time successfully be explained.

888888

Three Mechanisms for Randomness

In nature one of the single most common things one sees is apparent randomness. And
indeed, there are a great many different kinds of systems that all exhibit randomness. And
it could be that in each case the cause of randomness is different. But from my
investigations of simple programs I have come to the conclusion that one can in fact
identify just three basic mechanisms for randomness, as illustrated in the pictures below.

In the first mechanism, randomness is explicitly introduced into the underlying rules for
the system, so that a random color is chosen for every cell at each step.

This mechanism is the one most commonly considered in the traditional sciences. It
corresponds essentially to assuming that there is a random external environment which
continually affects the system one is looking at, and continually injects randomness into
it.

In the second mechanism shown above, there is no such interaction with the environment.
The initial conditions for the system are chosen randomly, but then the subsequent
evolution of the system is assumed to follow definite rules that involve no randomness.


Three possible mechanisms that can be responsible for randomness. The diagonal arrows
represent external input. In the first case, there is random input from the environment at
every step. In the second case, there is random input only in the initial conditions. And in
the third case, there is effectively no random input at all. Yet despite their different
underlying structure, each of these mechanisms leads to randomness in the column shown
at the left. The first mechanism corresponds to randomness produced by external noise,
as captured in so-called stochastic models. The second mechanism is essentially the one
suggested by chaos theory. The third mechanism is new, and is suggested by the results

on the behavior of simple programs in this book. I will give evidence that this third
mechanism is the most common one in nature.

888888

A crucial feature of these rules, however, is that they make the system behave in a way
that depends sensitively on the details of its initial conditions. In the particular case
shown, the rules are simply set up to shift every color one position to the left at each step.

And what this does is to make the sequence of colors taken on by any particular cell
depend on the colors of cells progressively further and further to the right in the initial
conditions. Insofar as the initial conditions are random, therefore, so also will the
sequence of colors of any particular cell be correspondingly random.

In general, the rules can be more complicated than those shown in the example on the
previous page. But the basic idea of this mechanism for randomness is that the
randomness one sees arises from some kind of transcription of randomness that is present
in the initial conditions.

The two mechanisms for randomness just discussed have one important feature in
common: they both assume that the randomness one sees in any particular system must
ultimately come from outside of that system. In a sense, therefore, neither of these
mechanisms takes any real responsibility for explaining the origins of randomness: they
both in the end just say that randomness comes from outside whatever system one
happens to be looking at.

Yet for quite a few years, this rather unsatisfactory type of statement has been the best
that one could make. But the discoveries about simple programs in this book finally allow
new progress to be made.

The crucial point that we first saw on page 27 is that simple programs can produce
apparently random behavior even when they are given no random input whatsoever. And
what this means is that there is a third possible mechanism for randomness, which this
time does not rely in any way on randomness already being present outside the system
one is looking at.

If we had found only a few examples of programs that could generate randomness in this
way, then we might think that this third mechanism was a rare and special one. But in
fact over the past few chapters we have seen that practically every kind of simple
program that we can construct is capable of generating such randomness.

Wolfram 3

More Related Content

Similar to Wolfram 3 (20)

More from Sabiq Hafidz (20)

Recently uploaded (20)

Wolfram 3