Solving recurrences

CS103A Handout 23
Winter 2002 February 22, 2002
Solving Recurrence Relations
Introduction
A wide variety of recurrence problems occur in models. Some of these recurrence relations can be solved
using iteration or some other ad hoc technique. However, one very important class of recurrence relations
can be explicitly solved in a systematic way. These are the recurrence relations that express the terms of a
sequence as a linear combination of previous terms.
Definition: A linear homogeneous recurrence relation of degree k with constant coefficients
is a recurrence relation of the form:
an = c1an− 1 + c2an−2 + L+ ckan−k
where c1,c 2 ,c 3 ,K,c k are real numbers, and ck ≠ 0 .
The recurrence relation in the definition is linear since the right-hand side is a sum of multiple of the
previous terms of the sequence. The recurrence relation is homogeneous since no terms of the recurrence
relation fail to involve a previous term of the sequence in some way. The coefficients of the terms of the
sequence are all constants, rather than functions that depend on n. The degree is k because an is expressed
in terms of the previous k terms of the sequence. A consequence of strong induction is that a sequence
satisfying the recurrence relation in the definition is uniquely determined by this recurrence relation and
the k initial condition:
an = c1an− 1 + c2an−2 + L+ ckan−k
a0 = C0
a1 = C1
KK
ak −1 = Ck −1
Solving the Little Monsters
The basic approach for solving linear homogeneous recurrence relations is to look for solutions of the form
an = r
n
, wherer is a constant. Note that an = r
n
is a solution to the recurrence if and only if:
r
n
= c1r
n−1
+ c2r
n− 2
+ c3r
n−3
+ L+ ckr
n−k
When both sides of the equation are divided by r
n−k
and the right-hand side is subtracted from the left,
we obtain the equivalent equation:
r
k
− c1r
k− 1
− c 2r
k − 2
− c3r
k − 3
−L − ck −1r − ck = 0

Consequently, the sequence with an = r
n
is a solution if and only if r is a solution to this last equation,
which is called the characteristic equation of the recurrence relation. The solutions of this equation are
called characteristic roots of the recurrence relation. As we’ll see, these characteristic roots can be used to
give an explicit formula for all the solutions of the recurrence relation. First, we should develop results
that deal with linear homogeneous recurrence relations with constant coefficients of degree two. The
results for second order relations can be extended to solve higher-order equations. The mathematics
involved in proving everything is really messy, so even though I’ll refer to it in lecture, I don’t want to
place it in here, because you might incorrectly think you’re responsible for knowing the proof, and you’re
not.
Let’s turn our undivided attention to linear homogeneous recurrence relations of degree two. First, consider
the case where there are two distinct characteristic roots.
A Fact Without Proof: Let c1 and c2 be real numbers. Suppose that r
2
− rc1 − c2 = 0 has two
distinct roots r1 and r2 . Then the sequence an{ } is a solution of the
recurrence relation an = c1an− 1 + c2an−2 if and only if an = b1r1
2
+ b2r2
2
for n = 0,1,2,3,4,5,K where b1 and b2 are constants determined by
the initial conditions of the recurrence relation.
Problem: Find an explicit solution to the tiling recurrence relation we developed
last time. You may recall that it went a-something like this:
Tn = Tn− 1 + Tn− 2
T0 = 1
T1 = 1
Well, we surmise that the solution will be a linear combination or terms having the form Tn = r
n
.
Following the rule from above, we just plug in Tn = r
n
into out recurrence relation and see what constraints
are placed on r . Let’s listen in:
Tn = Tn− 1 + Tn− 2
guess that Tn = r
n ⇒
r
n
= r
n− 1
+ r
n− 2
r
2
= r + 1
r2
− r − 1= 0
r1 =
1+ 5
2
;r2 =
1 − 5
2
Well, so the guess that Tn = r
n
is a good one provided that r be equal to one of the two roots above. In
fact, any linear combination of
1+ 5
2






n
and
1− 5
2






n
will also satisfy the recurrence if you ignore the
initial conditions for a moment—that is, Tn = b1
1+ 5
2






n
+ b2
1− 5
2






n
for any constant coefficients as
factors. It’s only when you supply the initial conditions that b1 and b2 are required to adopt a specific
value. In fact, the initial conditions here dictate that:
2

b1 + b2 = 1
b1
1 + 5
2
+ b2
1− 5
2
= 1
And after solving for b1 and b2 do we finally arrive at the unique solution to our recurrence problem:
Tn =
5 + 5
10
1 + 5
2






n
+
5 − 5
10
1 − 5
2






n
How very satisfying.
Problem: Find a closed form solution to the bit string problem modeled in our last
handout.
Welllllll, the recurrence relation, save the initial conditions, is exactly the same as it that for the tiling
problem; that translates to a solution of the same basic form. But the fact that the initial conditions are
different hints that the constants multiplying the individual terms:
note same form, but possibly
different constants
The different initial conditions place different constraint on the values of b1 and b2 do we. Now the
following system of equations must be solved:
b1 + b2 = 1
b1
1 + 5
2
+ b2
1− 5
2
= 2
We end up with something like this as our solution in this case:
Bn =
5 + 3 5
10
1 + 5
2






n
+
5− 3 5
10
1− 5
2






n
Problem: Solve the recurrence relation given below
Jn = Jn−2
J0 = 3J1 = 5
You may be asking, “Jerry, where in the world is the Jn− 1 term?” Relax—it’s in there—it’s just that its
multiplying coefficient is zero, so I didn’t bother writing it in. It’s still a homogeneous equation of degree
two, so I solve it like any other recurrence of this type.
Bn = b1
1+ 5
2






n
+ b2
1− 5
2






n
3

Jn = Jn−2
J0 = 3J1 = 5
⇒
Jn = Jn−2
r2
= 1
r
2
− 1 = 0
r1 = 1;r2 = − 1
Jn = b1 1( )n
+ b2 −1( )n
= b1 + b2 −1( )n
Funny little twist—dropping the 1( )n
, since it’s transparent to us anyway.
b1 + b2 = 5
b1 − b2 =
5
3
b1 =
10
3
,b 2 =
5
3
⇒ Jn =
10
3
+
5
3
−1( )n
Solving Coupled Recurrence Relations
The first recurrences handout included examples where solutions involved coupled recurrence relations.
The circular Tower of Hanoi Problem, the n × 3 tiling problem, the 2 × 2 × npillar problem, and the
Martian DNA problem were all complex enough to require (or at least benefit from) the invention of a
second counting problem and a second recurrence variable. Remember the 2 × 2 × n pillar recurrence? If not,
here it is once more:
Sn =
1
2
2Sn−1 + Sn− 2 + 4Tn−1
n = 0
n = 1
n ≥ 2





Tn =
0
1
Sn−1 + Tn−1
n = 0
n = 1
n ≥ 2





Because S and T are defined in terms of one another, it’s possible to use the second recurrence of the two to
eliminate all occurrences or T from the first. It takes a algebra and a little ingenuity, but when we rid of
the second variable, we are often left with a single recurrence relation which can be solved like other
recurrences we’ve seen before. The above system of recurrences reduces to a single linear, homogeneous,
constant-coefficient equation once we get rid of T. Don’t believe me? Read on, Thomas.
Problem: Solve the above system of recurrences for Sn by eliminating Tn and
solving the linear equation that results.
I want to completely eliminate all traces ofT and arrive at (what will turn out to
be) a (cubic) recurrence relation for just Sn , and then solve it. First things first: We
want to get rid of theTn−1 term from the recurrence relation for Sn , and to do that, I make the neat little
observation that the second equality below follows from the first by replacing n by n-1:
Sn = 2Sn−1 + Sn− 2 + 4Tn−1
Sn−1 = 2Sn− 2 + Sn−3 + 4Tn−2
Subtracting the second equation from the first, we get:
4

Sn − Sn−1 = 2Sn− 1 + Sn− 2 + 4Tn−1( ) − 2Sn−2 + Sn− 3 + 4Tn− 2( )
= 2Sn−1 − Sn−2 − Sn− 3 + 4Tn− 1 − 4Tn− 2
Sn = 3Sn−1 − Sn−2 − Sn− 3 + 4Tn−1 − 4Tn− 2
= 3Sn−1 − Sn−2 − Sn− 3 + 4 Tn− 1 − Tn− 2( )
Believe it or not, this is progress, because I have something very interesting to say about Tn−1 − Tn− 2 —it’s
always equal to Sn− 2 . Just look at the crafty manipulations I work up from theTn recurrence relation:
Tn = Sn−1 + Tn−1
Tn−1 = Sn− 2 + Tn− 2
Tn−1 − Tn− 2 = Sn− 2 + Tn− 2 − Tn−2
= Sn− 2
How crafty! Now I can eradicate any mention ofT from the Sn recurrence relation, and I do so like-a this:
Sn = 3Sn−1 − Sn− 2 − Sn−3 + 4 Tn−1 − Tn− 2( )
= 3Sn−1 − Sn− 2 − Sn−3 + 4Sn−2
= 3Sn−1 + 3Sn− 2 − Sn−3
Therefore, an uncoupled recurrence relation for Sn can be expressed as follows:
Sn =
1
2
2S1 + S0 + 4T1 = 9
3Sn−1 + 3Sn− 2 − Sn−3
n = 0
n = 1
n = 2
n ≥ 3







Admittedly, that was a lot of work, but this is pretty exciting, because we’re on the verge of taking what
is clearly a homogeneous, constant-coefficient equation and coming up with a closed form solution. As
always, guess a solution of the form Sn = a
n
and substitute to see what values of a make everything work
out regardless of the initial conditions. More algebra:
Sn S n = a
n = 3Sn−1 + 3Sn− 2 − Sn−3 Sn =a n
a
n
= 3a
n−1
+ 3a
n− 2
− a
n−3
a3
= 3a2
+ 3a − 1
0 = a
3
− 3a
2
− 3a + 1
0 = a + 1( ) a
2
− 4a + 1( )
0 = a + 1( ) a − 2 + 3( ) a − 2 − 3( )
a = − 1, 2 ± 3
That means that the general solution to our recurrence (ignoring the initial conditions) is
Sn = x 2 + 3( )
n
+ y 2 − 3( )
n
+ z −1( )n
, where x, y, and z can be any real numbers. Boundary conditions
require that:
5

S0 = x + y+ z = 1
S1 = 2 + 3( )x + 2 − 3( )y − z = 2
S2 = 7 + 4 3( )x + 1 − 4 3( )y + z = 9
This is a system of three linear equations for three unknowns, and it admits exactly one solution. Solving
for x, y, and z is a matter of algebra, and after all that algebra is over, we converge on a closed formula of
Sn =
1
6
2 + 3( )
n+1
+
1
6
2 − 3( )
n+1
+
1
3
−1( )n
, which is
1
6
2 + 3( )
n+1
rounded to the nearest integer. Neat!
Problem: Revisit the Martian DNA problem, and show that the number of valid
Martian DNA strands of length n is given as an + bn = F3n + 2 (yes, the Fibonacci
number!)
Let’s bring back the recurrence that defined a and b.
an =
0 n = 0
2 n = 1
an−1 + 2bn−1 n ≥ 2





bn =
1 n = 0
3 n = 1
2an− 1 + 3bn−1 n ≥ 2





Eliminate one of the recurrence terms in order to get a closed solution (though this time we’ll ultimately
need to solve for both variables, because you’re interested in their sum.)
an = an−1 + 2bn− 1
bn = 2an−1 + 3bn−1
Notice that the first one tells us something about an − an−1 , so let’s subtract a shifted version of the second
equation from the original to get at something that’s all b and no a.
bn = 2an− 1 + 3bn−1
bn−1 = 2an− 2 + 3bn−2
bn − bn−1 = 2 an−1 − an− 2( ) + 3bn−1 − 3bn− 2
bn = bn−1 + 2 an−1 − an− 2( )+ 3bn−1 − 3bn− 2
The first equation tells us that an−1 − an− 2 = 2bn− 2 . Substitution yields
bn = bn−1 + 4bn− 2 + 3bn−1 − 3bn− 2
= 4bn−1 + bn− 2
The characteristic equation here is r
2
− 4r − 1 = 0 ; bn = c1 2 + 5( )
n
+ c2 2 − 5( )
n
. Because b0 = 1 and
b1 = 3 , we determine c1 , c2 by solving the following two equations:
6

c1 + c2 = 1
2 + 5( )c1 + 2 − 5( )c2 = 3
⇒ c1 =
5 + 5
10
, c2 =
5 − 5
10
.
bn =
5 + 5
10
2 + 5( )
n
+
5 − 5
10
2 − 5( )
n
.
You might think you have to do the same thing for a, and in theory you’re right in that you need to solve
it, but we more or less have. Because
bn = 2an−1 + 3bn−1
we actually know that
bn = 2an−1 + 3bn−1
2an−1 = bn − 3bn−1
an−1 =
1
2
bn −
3
2
bn− 1
an =
1
2
bn+1 −
3
2
bn
an + bn =
1
2
bn+1 −
3
2
bn + bn
=
1
2
bn+1 −
1
2
bn
Recall that we’re really just interested in an + bn , and since it can be defined just in terms of the bn, we get:
an + bn =
1
2
bn+1 −
3
2
bn + bn
=
1
2
bn+1 −
1
2
bn
=
1
2
5 + 5
10
2 + 5( )
n+1
+
5 − 5
10
2 − 5( )
n+1




 −
1
2
5 + 5
10
2 + 5( )
n
+
5 − 5
10
2 − 5( )
n





=
1
2
5 + 5
10
2 + 5( )− 1( )2 + 5( )
n
+
1
2
5 − 5
10
2 − 5( )− 1( ) 2 − 5( )
n
=
1
2
5 + 5
10
1 + 5( ) 2 + 5( )
n
+
1
2
5 − 5
10
1 − 5( ) 2 − 5( )
n
=
1
2
10 + 6 5
10
2 + 5( )
n
+
1
2
10 − 6 5
10
2 − 5( )
n
=
5 + 3 5
10
2 + 5( )
n
+
5 − 3 5
10
2 − 5( )
n
That’s typically the closed-form you’d leave it in, but we also want to show that an + bn = F3n + 2 . Here
goes:
7

F3n + 2 =
1
5
1+ 5
2






3n +2
−
1
5
1− 5
2






3n + 2
=
1
5
1+ 5
2






2
1+ 5
2






3n
−
1
5
1− 5
2






2
1− 5
2






3n
=
1
5
6 + 2 5
4
1+ 5
2






3







n
−
1
5
6 − 2 5
4
1− 5
2






3







n
=
6 + 2 5
4 5
2 + 5( )
n
−
6 − 2 5
4 5
2 + 5( )
n
=
5
5
6 + 2 5
4 5
2 + 5( )
n
−
5
5
6 − 2 5
4 5
2 + 5( )
n
=
10 + 6 5
20
2 + 5( )
n
+
10 − 6 5
20
2 + 5( )
n
=
5 + 3 5
10
2 + 5( )
n
+
5 − 3 5
20
2 + 5( )
n
Therefore, an + bn = F3n + 2 , and we rejoice.
Problem: What about the Circular Tower of Hanoi recurrence? Why can’t we solve
that one as easily?
Well, you probably already noticed that both the pillar and the DNA recurrences
didn’t have any inhomogeneous terms anywhere. That’s not the case with the
Circular Tower of Hanoi recurrence, so while we are certainly invited to eliminate one of the variables
from the set of recurrences, there’s not much hope for solving it like we did for Martian DNA.
Qn =
0;
2Rn− 1 + 1
if n = 0
if n > 0



Rn =
0;
Qn + Qn−1 + 1
if n = 0
if n > 0



Clearly, it’s a piece of cake to define Qn in terms of previous Qs, but these 1s just aren’t going to go away.
Qn = 2Rn− 1 + 1
= 2 Qn−1 + Qn− 2 + 1( )+ 1
= 2Qn− 1 + 2Qn− 2 + 3
so that
Qn =
0
1
2Qn + 2Qn−1 + 3
if n = 0
if n = 1
if n > 1





We’re sorta bumming because of that inhomogeneous 3.
There is a trick to solving this one, but it’s not something I’m formally going to require you to know, since
there’s little pedagogical value in memorizing tricks. For those of you with nothing to do on a Friday
8

night, try substituting Qn = An−1 − 1 into the above system (making sure to adjust those base cases as
well.) Solve for An , then take whatever you got there and subtract 1 from it to get Qn .
General Approach to Solving all Linear First-Order Recurrences
There is a general technique that can reduce virtually any recurrence of the form
anTn = bnTn− 1 + cn
to a sum. The idea is to multiply both sides by a summation factor, sn , to arrive at
snanTn = snbnTn− 1 + sncn . This factor sn is chosen to make snbn = sn− 1an−1 . Then if we write Sn = snanTn
we have a sum-recurrence for Sn as:
Sn = Sn−1 + sncn
Hence Sn = s0a0T0 + sk ck
1≤ k ≤n
∑ = s1b1T0 + sk ck
1≤ k ≤n
∑ and the solution to the original recurrence becomes
Tn =
1
snan
s1b1T0 + skck
1≤k ≤ n
∑





 .
So, you ask: How can we be clever enough to find the perfect sn ? Well, the relation sn = sn− 1an−1 bn can
be unfolded by repeated substitution for the si to tell us that the fraction:
sn =
an− 1an− 2an− 3 Ka1
bnbn− 1bn− 2 Kb2
(or any convenient constant multiple of this value) will be a suitable summation factor.
Problem: Solve Vn =
5 n =0
nVn− 1 + 3⋅n! n ≥ 1



by finding the appropriate summation
factor.
Derivation of the solution just follows protocol. You're specifically told to use
summation factors here, so we should do that. Notice here that an = 1 and bn = n, so that the summation
factor should be chosen as:
sn =
1n− 1
n!
=
1
n!
V0 = 5 for sure, but the recurrence formula Vn = nVn− 1 + 3⋅n! becomes
1
n!
Vn =
1
n − 1( )!
Vn−1 + 3. Let
Tn =
1
n!
Vn for the time being, just so we can solve an easier recurrence relation. We tackle Tn = Tn− 1 + 3,
and wouldn't you know it: Tn = 3n + T0 = 3n +
1
0!
V0 = 3n + 5 . Tn =
1
n!
Vn . We need Vn = n!Tn , so therefore
we have that Vn = n! 3n + 5( ). Woo!
9

Problem: Solve Cn =
0 n =0
n + 1 +
2
n
Ck
0≤k ≤n− 1
∑ n ≥ 1




by finding the appropriate
summation factor.
Let’s write out the recurrences for Cn and Cn−1 a little more explicitly.
Cn = n + 1 +
2
n
Ck
0≤k ≤n− 1
∑
Cn−1 = n +
2
n − 1
Ck
0≤k ≤n− 2
∑
Subtracting the second one from the first one is a good idea, but only after the multiply through by a factor
that’ll make each of the summations equal to each other:
Cn = n + 1 +
2
n
Ck
0≤k ≤n− 1
∑
n − 1
n
Cn−1 =
n − 1
n
n +
n − 1
n
2
n − 1
Ck
0≤k ≤n− 2
∑
= n − 1 +
2
n
Ck
0≤k ≤n− 2
∑
Subtracting the second one from the first, we arrive at:
Cn −
n − 1
n
Cn− 1 = n + 1 +
2
n
Ck
0≤ k≤ n−1
∑





 − n − 1+
2
n
Ck
0 ≤k ≤n− 2
∑






= 2 +
2
n
Cn−1
Cn =
n + 1
n
Cn−1 + 2
nCn = n + 1( )Cn− 1 + 2n
If we choose a summation factor of sn =
an− 1an− 2an− 3 Ka1
bnbn− 1bn− 2 Kb2
=
n − 1( ) n − 2( ) n − 3( )K1
n + 1( )n n − 1( )K3
=
2
n n + 1( )
and multiply
through, we arrive at:
2
n n + 1( )
nCn =
2
n n + 1( )
n + 1( )Cn−1 +
2
n n + 1( )
2n
2
n + 1( )
Cn =
2
n
Cn− 1 +
4
n + 1( )
Cn
n + 1
=
Cn−1
n
+
2
n + 1
10

If we let n + 1( )Dn = Cn, then we finally arrive at an equation that looks reasonable: Dn = Dn−1 +
2
n + 1
.
Repeated substitution yields the following:
Dn = Dn−1 +
2
n + 1
= Dn−2 +
2
n





 +
2
n + 1
= Dn−3 +
2
n − 1





 +
2
n
+
2
n + 1
= Dn−4 +
2
n − 2





 +
2
n − 1
+
2
n
+
2
n + 1
M
= D0 + 2
1
k + 1
1≤k ≤n
∑
= 2 Hn+1 − 1( )
= 2 Hn +
1
n + 1
− 1





 = 2 Hn −
n
n + 1






Recall that Cn = n + 1( )Dn, so that Cn = 2 n + 1( ) Hn −
n
n + 1





 = 2 n + 1( )Hn − 2n.
11

Solving recurrences

More Related Content

What's hot (20)

Similar to Solving recurrences (20)

Recently uploaded (20)

Solving recurrences