Sums ADA

1. 2 Sums SUMS ARE EVERYWHERE in mathematics, so we need basic tools to handle them. This chapter develops the notation and general techniques that make summation user-friendly. 2.1 NOTATION In Chapter 1 we encountered the sum of the

2. rst n integers, which we wrote out as 1 + 2 + 3 + · · · + (n − 1) + n. The `· · · ' in such formulas tells us to complete the pattern established by the surrounding terms. Of course we have to watch out for sums like 1 + 7 + · · · + 41.7, which are meaningless without a mitigating context. On the other hand, the inclusion of terms like 3 and (n − 1) was a bit of overkill; the pattern would presumably have been clear if we had written simply 1 + 2 + · · · + n. Sometimes we might even be so bold as to write just 1 + · · · + n. We'll be working with sums of the general form a1 + a2 + · · · + an , (2.1) where each ak is a number that has been de

3. ned somehow. This notation has the advantage that we can see" the whole sum, almost as if it were written out in full, if we have a good enough imagination. Each element ak of a sum is called a term. The terms are often speci

4. ed A term is how long this course lasts. implicitly as formulas that follow a readily perceived pattern, and in such cases we must sometimes write them in an expanded form so that the meaning is clear. For example, if 1 + 2 + · · · + 2n−1 is supposed to denote a sum of n terms, not of 2n−1 , we should write it more explicitly as 20 + 21 + · · · + 2n−1 . 21

5. 22 SUMS The three-dots notation has many uses, but it can be ambiguous and a Le signe Pi=∞ i=1 in- dique que l'on doit donner au nombre entier i toutes ses valeurs 1, 2, 3, . . . , et prendre la somme des termes." |J. Fourier [127] bit long-winded. Other alternatives are available, notably the delimited form n X k=1 ak , (2.2) which is called Sigma-notation because it uses the Greek letter P (uppercase sigma). This notation tells us to include in the sum precisely those terms ak whose index k is an integer that lies between the lower and upper limits 1 and n, inclusive. In words, we sum over k, from 1 to n." Joseph Fourier introduced this delimited P -notation in 1820, and it soon took the mathematical world by storm. Incidentally, the quantity after P (here ak) is called the summand. The index variable k is said to be bound to the P sign in (2.2), because the k in ak is unrelated to appearances of k outside the Sigma-notation. Any other letter could be substituted for k here without changing the meaning of Well, I wouldn't want to use a or n as the index variable instead of k in (2.2); those letters are free variables" that do have meaning outside the P here. (2.2). The letter i is often used (perhaps because it stands for index"), but we'll generally sum on k since it's wise to keep i for √ −1. It turns out that a generalized Sigma-notation is even more useful than the delimited form: We simply write one or more conditions under the P , to specify the set of indices over which summation should take place. For example, the sums in (2.1) and (2.2) can also be written as X 1kn ak . (2.3) In this particular example there isn't much dierence between the new form and (2.2), but the general form allows us to take sums over index sets that aren't restricted to consecutive integers. For example, we can express the sum of the squares of all odd positive integers below 100 as follows: X 1k100 k odd k2 . The delimited equivalent of this sum, 49 X k=0 (2k + 1)2 , is more cumbersome and less clear. Similarly, the sum of reciprocals of all prime numbers between 1 and N is X pN p prime 1 p ;

6. 2.1 NOTATION 23 the delimited form would require us to write π(N) X k=1 1 pk , where pk denotes the kth prime and π(N) is the number of primes N. (Incidentally, this sum gives the approximate average number of distinct prime factors of a random integer near N, since about 1/p of those integers are divisible by p. Its value for large N is approximately lnlnN + M, where M ≈ 0.2614972128476427837554268386086958590515666 is Mertens's constant [271]; lnx stands for the natural logarithm of x, and lnlnx stands for ln(lnx).) The biggest advantage of general Sigma-notation is that we can manipulate it more easily than the delimited form. For example, suppose we want The summation symbol looks like a distorted pacman. to change the index variable k to k + 1. With the general form, we have X 1kn ak = X 1k+1n ak+1 ; it's easy to see what's going on, and we can do the substitution almost without thinking. But with the delimited form, we have n X k=1 ak = n−1 X k=0 ak+1 ; it's harder to see what's happened, and we're more likely to make a mistake. On the other hand, the delimited form isn't completely useless. It's nice and tidy, and we can write it quickly because (2.2) has seven symbols A tidy sum. compared with (2.3)'s eight. Therefore we'll often use P with upper and lower delimiters when we state a problem or present a result, but we'll prefer to work with relations-under- P when we're manipulating a sum whose index variables need to be transformed. The P sign occurs more than 1000 times in this book, so we should be That's nothing. You should see how many times appears in The Iliad. sure that we know exactly what it means. Formally, we write X P(k) ak (2.4) as an abbreviation for the sum of all terms ak such that k is an integer satisfying a given property P(k). (A property P(k) is any statement about k that can be either true or false.) For the time being, we'll assume that only

7. nitely many integers k satisfying P(k) have ak 6= 0; otherwise in

8. nitely many nonzero numbers are being added together, and things can get a bit

9. 24 SUMS tricky. At the other extreme, if P(k) is false for all integers k, we have an empty sum; the value of an empty sum is de

10. ned to be zero. A slightly modi

11. ed form of (2.4) is used when a sum appears within the text of a paragraph rather than in a displayed equation: We write ` P P(k) ak', attaching property P(k) as a subscript of P , so that the formula won't stick out too much. Similarly, ` Pn k=1 ak' is a convenient alternative to (2.2) when we want to con

12. ne the notation to a single line. People are often tempted to write n−1 X k=2 k(k − 1)(n − k) instead of n X k=0 k(k − 1)(n − k) because the terms for k = 0, 1, and n in this sum are zero. Somehow it seems more ecient to add up n − 2 terms instead of n + 1 terms. But such temptations should be resisted; eciency of computation is not the same as eciency of understanding! We will

13. nd it advantageous to keep upper and lower bounds on an index of summation as simple as possible, because sums can be manipulated much more easily when the bounds are simple. Indeed, the form Pn−1 k=2 can even be dangerously ambiguous, because its meaning is not at all clear when n = 0 or n = 1 (see exercise 1). Zero-valued terms cause no harm, and they often save a lot of trouble. So far the notations we've been discussing are quite standard, but now we are about to make a radical departure from tradition. Kenneth E. Iverson introduced a wonderful idea in his programming language APL [191, page 11; see also 220], and we'll see that it greatly simpli

14. es many of the things we want to do in this book. The idea is simply to enclose a true-or-false statement in brackets, and to say that the result is 1 if the statement is true, 0 if the Hey: The Kro- necker delta that I've seen in other books (I mean δkn , which is 1 if k = n, 0 otherwise) is just a special case of Iverson's convention: We can write [ k = n ] instead. statement is false. For example, [p prime] = ¯ 1, if p is a prime number; 0, if p is not a prime number. Iverson's convention allows us to express sums with no constraints whatever on the index of summation, because we can rewrite (2.4) in the form X k ak £ P(k) ¤ . (2.5) If P(k) is false, the term ak £ P(k) ¤ is zero, so we can safely include it among the terms being summed. This makes it easy to manipulate the index of summation, because we don't have to fuss with boundary conditions. I am often surprised by new, important applications [of this notation]. |B. de Finetti [123] A slight technicality needs to be mentioned: Sometimes ak isn't de

15. ned for all integers k. We get around this diculty by assuming that £ P(k) ¤ is very strongly zero when P(k) is false; it's so much zero, it makes ak £ P(k) ¤

16. 2.1 NOTATION 25 equal to zero even when ak is unde

17. ned. For example, if we use Iverson's convention to write the sum of reciprocal primes N as X p [p prime][p N]/p , there's no problem of division by zero when p = 0, because our convention tells us that [0 prime][0 N]/0 = 0. Let's sum up what we've discussed so far about sums. There are two good ways to express a sum of terms: One way uses `· · · ', the other uses ` P '. The three-dots form often suggests useful manipulations, particularly the combination of adjacent terms, since we might be able to spot a simplifying pattern if we let the whole sum hang out before our eyes. But too much detail can also be overwhelming. Sigma-notation is compact, impressive to family and friends, and often suggestive of manipulations that are not obvious in . . . and it's less likely to lose points on an exam for lack of rigor. three-dots form. When we work with Sigma-notation, zero terms are not generally harmful; in fact, zeros often make P -manipulation easier. 2.2 SUMS AND RECURRENCES OK, we understand now how to express sums with fancy notation. But how does a person actually go about

18. nding the value of a sum? One way is to observe that there's an intimate relation between sums and recurrences. The sum Sn = n X k=0 ak is equivalent to the recurrence (Think of Sn as not just a single number, but as a sequence de

19. ned for all n 0.) S0 = a0 ; Sn = Sn−1 + an , for n 0. (2.6) Therefore we can evaluate sums in closed form by using the methods we learned in Chapter 1 to solve recurrences in closed form. For example, if an is equal to a constant plus a multiple of n, the sum- recurrence (2.6) takes the following general form: R0 = α ; Rn = Rn−1 + β + γn , for n 0. (2.7) Proceeding as in Chapter 1, we

20. nd R1 = α + β + γ, R2 = α + 2β + 3γ, and so on; in general the solution can be written in the form Rn = A(n)α + B(n)β + C(n)γ , (2.8)

21. 26 SUMS where A(n), B(n), and C(n) are the coecients of dependence on the general parameters α, β, and γ. The repertoire method tells us to try plugging in simple functions of n for Rn, hoping to

22. nd constant parameters α, β, and γ where the solution is especially simple. Setting Rn = 1 implies α = 1, β = 0, γ = 0; hence A(n) = 1 . Setting Rn = n implies α = 0, β = 1, γ = 0; hence B(n) = n . Setting Rn = n2 implies α = 0, β = −1, γ = 2; hence 2C(n) − B(n) = n2 and we have C(n) = (n2 + n)/2. Easy as pie. Actually easier; π = P n0 8 (4n+1)(4n+3) . Therefore if we wish to evaluate n X k=0 (a + bk) , the sum-recurrence (2.6) boils down to (2.7) with α = β = a, γ = b, and the answer is aA(n) + aB(n) + bC(n) = a(n + 1) + b(n + 1)n/2. Conversely, many recurrences can be reduced to sums; therefore the special methods for evaluating sums that we'll be learning later in this chapter will help us solve recurrences that might otherwise be dicult. The Tower of Hanoi recurrence is a case in point: T0 = 0 ; Tn = 2Tn−1 + 1 , for n 0. It can be put into the special form (2.6) if we divide both sides by 2n : T0/20 = 0; Tn/2n = Tn−1/2n−1 + 1/2n , for n 0. Now we can set Sn = Tn/2n , and we have S0 = 0; Sn = Sn−1 + 2−n , for n 0. It follows that Sn = n X k=1 2−k .

23. 2.2 SUMS AND RECURRENCES 27 (Notice that we've left the term for k = 0 out of this sum.) The sum of the geometric series 2−1 +2−2 +· · ·+2−n = (1 2 )1 +(1 2 )2 +· · ·+(1 2 )n will be derived later in this chapter; it turns out to be 1 − (1 2 )n . Hence Tn = 2n Sn = 2n − 1. We have converted Tn to Sn in this derivation by noticing that the recurrence could be divided by 2n . This trick is a special case of a general technique that can reduce virtually any recurrence of the form anTn = bnTn−1 + cn (2.9) to a sum. The idea is to multiply both sides by a summation factor, sn: snanTn = snbnTn−1 + sncn . This factor sn is cleverly chosen to make snbn = sn−1an−1 . Then if we write Sn = snanTn we have a sum-recurrence, Sn = Sn−1 + sncn . Hence Sn = s0a0T0 + n X k=1 skck = s1b1T0 + n X k=1 skck , and the solution to the original recurrence (2.9) is Tn = 1 snan µ s1b1T0 + n X k=1 skck ¶ . (2.10) For example, when n = 1 we get T1 = (s1b1T0 +s1c1)/s1a1 = (b1T0 +c1)/a1. (The value of s1 cancels out, so it can be anything but zero.) But how can we be clever enough to

24. nd the right sn? No problem: The relation sn = sn−1an−1/bn can be unfolded to tell us that the fraction sn = an−1an−2 . . . a1 bnbn−1 . . . b2 , (2.11) or any convenient constant multiple of this value, will be a suitable summation factor. For example, the Tower of Hanoi recurrence has an = 1 and bn = 2; the general method we've just derived says that sn = 2−n is a good thing to multiply by, if we want to reduce the recurrence to a sum. We don't need a brilliant ash of inspiration to discover this multiplier. We must be careful, as always, not to divide by zero. The summation- factor method works whenever all the a's and all the b's are nonzero.

25. 28 SUMS Let's apply these ideas to a recurrence that arises in the study of quicksort, one of the most important methods for sorting data inside a computer. (Quicksort was invented by Hoare in 1962 [189].) The average number of comparison steps made by quicksort when it is applied to n items in random order satis

26. es the recurrence C0 = 0 ; Cn = n + 1 + 2 n n−1 X k=0 Ck , for n 0. (2.12) Hmmm. This looks much scarier than the recurrences we've seen before; it includes a sum over all previous values, and a division by n. Trying small cases gives us some data (C1 = 2, C2 = 5, C3 = 26 3 ) but doesn't do anything to quell our fears. We can, however, reduce the complexity of (2.12) systematically, by

27. rst getting rid of the division and then getting rid of the P sign. The idea is to multiply both sides by n, obtaining the relation nCn = n2 + n + 2 n−1 X k=0 Ck , for n 0; hence, if we replace n by n − 1, (n − 1)Cn−1 = (n − 1)2 + (n − 1) + 2 n−2 X k=0 Ck , for n − 1 0. We can now subtract the second equation from the

28. rst, and the P sign disappears: nCn − (n − 1)Cn−1 = 2n + 2Cn−1 , for n 1. It turns out that this relation also holds when n = 1, because C1 = 2. Therefore the original recurrence for Cn reduces to a much simpler one: C0 = 0 ; nCn = (n + 1)Cn−1 + 2n , for n 0. Progress. We're now in a position to apply a summation factor, since this recurrence has the form of (2.9) with an = n, bn = n + 1, and cn = 2n. The general method described on the preceding page tells us to multiply the recurrence through by some multiple of sn = an−1an−2 . . . a1 bnbn−1 . . . b2 = (n − 1) · (n − 2) · . . . · 1 (n + 1) · n · . . . · 3 = 2 (n + 1)n .

29. 2.2 SUMS AND RECURRENCES 29 The solution, according to (2.10), is therefore We started with a P in the recurrence, and worked hard to get rid of it. But then after applying a summation factor, we came up with another P . Are sums good, or bad, or what? Cn = 2(n + 1) n X k=1 1 k + 1 . The sum that remains is very similar to a quantity that arises frequently in applications. It arises so often, in fact, that we give it a special name and a special notation: Hn = 1 + 1 2 + · · · + 1 n = n X k=1 1 k . (2.13) The letter H stands for harmonic; Hn is a harmonic number, so called because the kth harmonic produced by a violin string is the fundamental tone produced by a string that is 1/k times as long. We can complete our study of the quicksort recurrence (2.12) by putting Cn into closed form; this will be possible if we can express Cn in terms of Hn. The sum in our formula for Cn is n X k=1 1 k + 1 = X 1kn 1 k + 1 . We can relate this to Hn without much diculty by changing k to k − 1 and revising the boundary conditions: X 1kn 1 k + 1 = X 1k−1n 1 k = X 2kn+1 1 k = µ X 1kn 1 k ¶ − 1 1 + 1 n + 1 = Hn − n n + 1 . Alright! We have found the sum needed to complete the solution to (2.12): But your spelling is alwrong. The average number of comparisons made by quicksort when it is applied to n randomly ordered items of data is Cn = 2(n + 1)Hn − 2n . (2.14) As usual, we check that small cases are correct: C0 = 0, C1 = 2, C2 = 5.

30. 30 SUMS 2.3 MANIPULATION OF SUMS The key to success with sums is an ability to change one P into another that is simpler or closer to some goal. And it's easy to do this by Not to be confused with

31. nance. learning a few basic rules of transformation and by practicing their use. Let K be any

32. nite set of integers. Sums over the elements of K can be transformed by using three simple rules: X k∈K cak = c X k∈K ak ; (distributive law) (2.15) X k∈K (ak + bk) = X k∈K ak + X k∈K bk ; (associative law) (2.16) X k∈K ak = X p(k)∈K ap(k) . (commutative law) (2.17) The distributive law allows us to move constants in and out of a P . The associative law allows us to break a P into two parts, or to combine two P 's into one. The commutative law says that we can reorder the terms in any way we please; here p(k) is any permutation of the set of all integers. For example, Why not call it permutative instead of commutative? if K = f−1, 0, +1g and if p(k) = −k, these three laws tell us respectively that ca−1 + ca0 + ca1 = c(a−1 + a0 + a1) ; (distributive law) (a−1 + b−1) + (a0 + b0) + (a1 + b1) = (a−1 + a0 + a1) + (b−1 + b0 + b1) ; (associative law) a−1 + a0 + a1 = a1 + a0 + a−1 . (commutative law) Gauss's trick in Chapter 1 can be viewed as an application of these three basic laws. Suppose we want to compute the general sum of an arithmetic progression, S = X 0kn (a + bk) . By the commutative law we can replace k by n − k, obtaining This is something like changing variables inside an integral, but easier. S = X 0n−kn ¡ a + b(n − k) ¢ = X 0kn (a + bn − bk) . These two equations can be added by using the associative law: 2S = X 0kn ¡ (a + bk) + (a + bn − bk) ¢ = X 0kn (2a + bn) .

33. 2.3 MANIPULATION OF SUMS 31 And we can now apply the distributive law and evaluate a trivial sum: What's one and one and one and one and one and one and one and one and one and one? I don't know, said Alice. I lost count. She can't do Addition. |Lewis Carroll [50] 2S = (2a + bn) X 0kn 1 = (2a + bn)(n + 1) . Dividing by 2, we have proved that n X k=0 (a + bk) = (a + 1 2 bn)(n + 1) . (2.18) The right-hand side can be remembered as the average of the

34. rst and last terms, namely 1 2 ¡ a + (a + bn) ¢ , times the number of terms, namely (n + 1). It's important to bear in mind that the function p(k) in the general commutative law (2.17) is supposed to be a permutation of all the integers. In other words, for every integer n there should be exactly one integer k such that p(k) = n. Otherwise the commutative law might fail; exercise 3 illustrates this with a vengeance. Transformations like p(k) = k + c or p(k) = c − k, where c is an integer constant, are always permutations, so they always work. On the other hand, we can relax the permutation restriction a little bit: We need to require only that there be exactly one integer k with p(k) = n when n is an element of the index set K. If n / ∈ K (that is, if n is not in K), it doesn't matter how often p(k) = n occurs, because such k don't take part in the sum. Thus, for example, we can argue that X k∈K k even ak = X n∈K n even an = X 2k∈K 2k even a2k = X 2k∈K a2k , (2.19) since there's exactly one k such that 2k = n when n ∈ K and n is even. Iverson's convention, which allows us to obtain the values 0 or 1 from logical statements in the middle of a formula, can be used together with the distributive, associative, and commutative laws to deduce additional proper- Additional, eh? ties of sums. For example, here is an important rule for combining dierent sets of indices: If K and K0 are any sets of integers, then X k∈K ak + X k∈K0 ak = X k∈K∩K0 ak + X k∈K∪K0 ak . (2.20) This follows from the general formulas X k∈K ak = X k ak [k ∈ K] (2.21) and [k ∈ K] + [k ∈ K0 ] = [k ∈ K ∩ K0 ] + [k ∈ K ∪ K0 ] . (2.22)

35. 32 SUMS Typically we use rule (2.20) either to combine two almost-disjoint index sets, as in m X k=1 ak + n X k=m ak = am + n X k=1 ak , for 1 m n; or to split o a single term from a sum, as in (The two sides of (2.20) have been switched here.) X 0kn ak = a0 + X 1kn ak , for n 0. (2.23) This operation of splitting o a term is the basis of a perturbation method that often allows us to evaluate a sum in closed form. The idea is to start with an unknown sum and call it Sn: Sn = X 0kn ak . (Name and conquer.) Then we rewrite Sn+1 in two ways, by splitting o both its last term and its

36. rst term: Sn + an+1 = X 0kn+1 ak = a0 + X 1kn+1 ak = a0 + X 1k+1n+1 ak+1 = a0 + X 0kn ak+1 . (2.24) Now we can work on this last sum and try to express it in terms of Sn. If we succeed, we obtain an equation whose solution is the sum we seek. For example, let's use this approach to

37. nd the sum of a general geomet- If it's geometric, there should be a geometric proof. ³ ³ ³³³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³³³ ³ ³B B B B B B B B B B B B B B BB B B B B BB B B B B B B B B B B £ £ £ £ £ £ P P P P P P P P P P P P P P ric progression, Sn = X 0kn axk . The general perturbation scheme in (2.24) tells us that Sn + axn+1 = ax0 + X 0kn axk+1 , and the sum on the right is x P 0kn axk = xSn by the distributive law. Therefore Sn + axn+1 = a + xSn, and we can solve for Sn to obtain n X k=0 axk = a − axn+1 1 − x , for x 6= 1. (2.25)

38. 2.3 MANIPULATION OF SUMS 33 (When x = 1, the sum is of course simply (n + 1)a.) The right-hand side can be remembered as the

39. rst term included in the sum minus the

40. rst term Ah yes, this formula was drilled into me in high school. excluded (the term after the last), divided by 1 minus the term ratio. That was almost too easy. Let's try the perturbation technique on a slightly more dicult sum, Sn = X 0kn k 2k . In this case we have S0 = 0, S1 = 2, S2 = 10, S3 = 34, S4 = 98; what is the general formula? According to (2.24) we have Sn + (n + 1)2n+1 = X 0kn (k + 1)2k+1 ; so we want to express the right-hand sum in terms of Sn. Well, we can break it into two sums with the help of the associative law, X 0kn k 2k+1 + X 0kn 2k+1 , and the

41. rst of the remaining sums is 2Sn. The other sum is a geometric progression, which equals (2 − 2n+2 )/(1 − 2) = 2n+2 − 2 by (2.25). Therefore we have Sn + (n + 1)2n+1 = 2Sn + 2n+2 − 2, and algebra yields X 0kn k 2k = (n − 1)2n+1 + 2 . Now we understand why S3 = 34: It's 32 + 2, not 2·17. A similar derivation with x in place of 2 would have given us the equation Sn + (n + 1)xn+1 = xSn + (x − xn+2 )/(1 − x); hence we can deduce that n X k=0 kxk = x − (n + 1)xn+1 + nxn+2 (1 − x)2 , for x 6= 1. (2.26) It's interesting to note that we could have derived this closed form in a completely dierent way, by using elementary techniques of dierential calculus. If we start with the equation n X k=0 xk = 1 − xn+1 1 − x and take the derivative of both sides with respect to x, we get n X k=0 kxk−1 = (1−x) ¡ −(n+1)xn ¢ + 1−xn+1 (1 − x)2 = 1 − (n+1)xn + nxn+1 (1 − x)2 ,

42. 34 SUMS because the derivative of a sum is the sum of the derivatives of its terms. We will see many more connections between calculus and discrete mathematics in later chapters. 2.4 MULTIPLE SUMS The terms of a sum might be speci

43. ed by two or more indices, not just by one. For example, here's a double sum of nine terms, governed by two Oh no, a nine-term governor. indices j and k: Notice that this doesn't mean to sum over all j 1 and all k 3. X 1j,k3 ajbk = a1b1 + a1b2 + a1b3 + a2b1 + a2b2 + a2b3 + a3b1 + a3b2 + a3b3 . We use the same notations and methods for such sums as we do for sums with a single index. Thus, if P(j, k) is a property of j and k, the sum of all terms aj,k such that P(j, k) is true can be written in two ways, one of which uses Iverson's convention and sums over all pairs of integers j and k: X P(j,k) aj,k = X j,k aj,k £ P(j, k) ¤ . Only one P sign is needed, although there is more than one index of summation; P denotes a sum over all combinations of indices that apply. We also have occasion to use two P 's, when we're talking about a sum of sums. For example, X j X k aj,k £ P(j, k) ¤ is an abbreviation for X j µX k aj,k £ P(j, k) ¤ ¶ , which is the sum, over all integers j, of P k aj,k £ P(j, k) ¤ , the latter being the Multiple 's are evaluated right to left (inside-out). sum over all integers k of all terms aj,k for which P(j, k) is true. In such cases we say that the double sum is summed

44. rst on k. A sum that depends on more than one index can be summed

45. rst on any one of its indices. In this regard we have a basic law called interchanging the order of summation, which generalizes the associative law (2.16) we saw earlier: X j X k aj,k £ P(j, k) ¤ = X P(j,k) aj,k = X k X j aj,k £ P(j, k) ¤ . (2.27)

46. 2.4 MULTIPLE SUMS 35 The middle term of this law is a sum over two indices. On the left, P j P k stands for summing

47. rst on k, then on j. On the right, P k P j stands for summing

48. rst on j, then on k. In practice when we want to evaluate a double sum in closed form, it's usually easier to sum it

49. rst on one index rather than on the other; we get to choose whichever is more convenient. Sums of sums are no reason to panic, but they can appear confusing to Who's panicking? I think this rule is fairly obvious compared to some of the stu in Chapter 1. a beginner, so let's do some more examples. The nine-term sum we began with provides a good illustration of the manipulation of double sums, because that sum can actually be simpli

50. ed, and the simpli

51. cation process is typical of what we can do with P P 's: X 1j,k3 ajbk = X j,k ajbk[1 j, k 3] = X j,k ajbk[1 j 3][1 k 3] = X j X k ajbk[1 j 3][1 k 3] = X j aj[1 j 3] X k bk[1 k 3] = X j aj[1 j 3] µX k bk[1 k 3] ¶ = µX j aj[1 j 3] ¶µX k bk[1 k 3] ¶ = µ 3 X j=1 aj ¶µ 3 X k=1 bk ¶ . The

52. rst line here denotes a sum of nine terms in no particular order. The second line groups them in threes, (a1b1 + a1b2 + a1b3) + (a2b1 + a2b2 + a2b3) + (a3b1 + a3b2 + a3b3). The third line uses the distributive law to factor out the a's, since aj and [1 j 3] do not depend on k; this gives a1(b1 + b2 + b3) + a2(b1 + b2 + b3) + a3(b1 + b2 + b3). The fourth line is the same as the third, but with a redundant pair of parentheses thrown in so that the

53. fth line won't look so mysterious. The

54. fth line factors out the (b1 + b2 + b3) that occurs for each value of j: (a1 + a2 + a3)(b1 + b2 + b3). The last line is just another way to write the previous line. This method of derivation can be used to prove a general distributive law, X j∈J k∈K ajbk = µ X j∈J aj ¶µ X k∈K bk ¶ , (2.28) valid for all sets of indices J and K. The basic law (2.27) for interchanging the order of summation has many variations, which arise when we want to restrict the ranges of the indices

55. 36 SUMS instead of summing over all integers j and k. These variations come in two avors, vanilla and rocky road. First, the vanilla version: X j∈J X k∈K aj,k = X j∈J k∈K aj,k = X k∈K X j∈J aj,k . (2.29) This is just another way to write (2.27), since the Iversonian [j ∈ J, k ∈ K] factors into [j ∈ J][k ∈ K]. The vanilla- avored law applies whenever the ranges of j and k are independent of each other. The rocky-road formula for interchange is a little trickier. It applies when the range of an inner sum depends on the index variable of the outer sum: X j∈J X k∈K(j) aj,k = X k∈K0 X j∈J0(k) aj,k . (2.30) Here the sets J, K(j), K0 , and J0 (k) must be related in such a way that [j ∈ J] £ k ∈ K(j) ¤ = [k ∈ K0 ] £ j ∈ J0 (k) ¤ . A factorization like this is always possible in principle, because we can let J = K0 be the set of all integers and K(j) = J0 (k) be the basic property P(j, k) that governs a double sum. But there are important special cases where the sets J, K(j), K0 , and J0 (k) have a simple form. These arise frequently in applications. For example, here's a particularly useful factorization: [1 j n][j k n] = [1 j k n] = [1 k n][1 j k] . (2.31) This Iversonian equation allows us to write n X j=1 n X k=j aj,k = X 1jkn aj,k = n X k=1 k X j=1 aj,k . (2.32) One of these two sums of sums is usually easier to evaluate than the other; (Now is a good time to do warmup exercises 4 and 6.) (Or to check out the Snickers bar languishing in the freezer.) we can use (2.32) to switch from the hard one to the easy one. Let's apply these ideas to a useful example. Consider the array       a1a1 a1a2 a1a3 . . . a1an a2a1 a2a2 a2a3 . . . a2an a3a1 a3a2 a3a3 . . . a3an . . . . . . . . . ... . . . ana1 ana2 ana3 . . . anan       of n2 products ajak. Our goal will be to

56. nd a simple formula for S@ = X 1jkn ajak ,

57. 2.4 MULTIPLE SUMS 37 the sum of all elements on or above the main diagonal of this array. Because ajak = akaj, the array is symmetrical about its main diagonal; therefore S@ will be approximately half the sum of all the elements (except for a fudge factor that takes account of the main diagonal). Does rocky road have fudge in it? Such considerations motivate the following manipulations. We have S@ = X 1jkn ajak = X 1kjn akaj = X 1kjn ajak = S@ , because we can rename (j, k) as (k, j). Furthermore, since [1 j k n] + [1 k j n] = [1 j, k n] + [1 j = k n] , we have 2S@ = S@ + S@ = X 1j,kn ajak + X 1j=kn ajak . The

58. rst sum is ¡Pn j=1 aj ¢¡Pn k=1 ak ¢ = ¡Pn k=1 ak ¢2 , by the general distributive law (2.28). The second sum is Pn k=1 a2 k. Therefore we have S@ = X 1jkn ajak = 1 2 Ãµ n X k=1 ak ¶2 + n X k=1 a2 k ! , (2.33) an expression for the upper triangular sum in terms of simpler single sums. Encouraged by such success, let's look at another double sum: S = X 1jkn (ak − aj)(bk − bj) . Again we have symmetry when j and k are interchanged: S = X 1kjn (aj − ak)(bj − bk) = X 1kjn (ak − aj)(bk − bj) . So we can add S to itself, making use of the identity [1 j k n] + [1 k j n] = [1 j, k n] − [1 j = k n] to conclude that 2S = X 1j,kn (aj − ak)(bj − bk) − X 1j=kn (aj − ak)(bj − bk) .

59. 38 SUMS The second sum here is zero; what about the

60. rst? It expands into four separate sums, each of which is vanilla avored: X 1j,kn ajbj − X 1j,kn ajbk − X 1j,kn akbj + X 1j,kn akbk = 2 X 1j,kn akbk − 2 X 1j,kn ajbk = 2n X 1kn akbk − 2 µ n X k=1 ak ¶µ n X k=1 bk ¶ . In the last step both sums have been simpli

61. ed according to the general distributive law (2.28). If the manipulation of the

62. rst sum seems mysterious, here it is again in slow motion: 2 X 1j,kn akbk = 2 X 1kn X 1jn akbk = 2 X 1kn akbk X 1jn 1 = 2 X 1kn akbkn = 2n X 1kn akbk . An index variable that doesn't appear in the summand (here j) can simply be eliminated if we multiply what's left by the size of that variable's index set (here n). Returning to where we left o, we can now divide everything by 2 and rearrange things to obtain an interesting formula: µ n X k=1 ak ¶µ n X k=1 bk ¶ = n n X k=1 akbk − X 1jkn (ak − aj)(bk − bj) . (2.34) This identity yields Chebyshev's monotonic inequalities as a special case: (Chebyshev [58] actually proved the analogous result for integrals instead of sums, ( Rb a f(x) dx) · ( Rb a g(x) dx) (b − a) · ( Rb a f(x)g(x) dx), if f(x) and g(x) are monotone nondecreasing functions.) µ n X k=1 ak ¶µ n X k=1 bk ¶ n n X k=1 akbk , if a1 · · · an and b1 · · · bn; µ n X k=1 ak ¶µ n X k=1 bk ¶ n n X k=1 akbk , if a1 · · · an and b1 · · · bn. (In general, if a1 · · · an and if p is a permutation of f1, . . . , ng, it's not dicult to prove that the largest value of Pn k=1 akbp(k) occurs when bp(1) · · · bp(n), and the smallest value occurs when bp(1) · · · bp(n).)

63. 2.4 MULTIPLE SUMS 39 Multiple summation has an interesting connection with the general operation of changing the index of summation in single sums. We know by the commutative law that X k∈K ak = X p(k)∈K ap(k) , if p(k) is any permutation of the integers. But what happens when we replace k by f(j), where f is an arbitrary function f : J → K that takes an integer j ∈ J into an integer f(j) ∈ K? The general formula for index replacement is X j∈J af(j) = X k∈K ak #f− (k) , (2.35) where #f− (k) stands for the number of elements in the set f− (k) = © j ¯ ¯ f(j) = k ª , that is, the number of values of j ∈ J such that f(j) equals k. It's easy to prove (2.35) by interchanging the order of summation, X j∈J af(j) = X j∈J k∈K ak £ f(j) = k ¤ = X k∈K ak X j∈J £ f(j) = k ¤ , since P j∈J £ f(j) = k ¤ = #f− (k). In the special case that f is a one-to-one correspondence between J and K, we have #f− (k) = 1 for all k, and the My other math teacher calls this a bijection; maybe I'll learn to love that word some day. And then again. . . general formula (2.35) reduces to X j∈J af(j) = X f(j)∈K af(j) = X k∈K ak . This is the commutative law (2.17) we had before, slightly disguised. Our examples of multiple sums so far have all involved general terms like ak or bk. But this book is supposed to be concrete, so let's take a look at a multiple sum that involves actual numbers: Sn = X 1jkn 1 k − j . For example, S1 = 0; S2 = 1; S3 = 1 2−1 + 1 3−1 + 1 3−2 = 5 2 . Watch out| the authors seem to think that j, k, and n are actual numbers.

64. 40 SUMS The normal way to evaluate a double sum is to sum

65. rst on j or

66. rst on k, so let's explore both options. Sn = X 1kn X 1jk 1 k − j summing

67. rst on j = X 1kn X 1k−jk 1 j replacing j by k − j = X 1kn X 0jk−1 1 j simplifying the bounds on j = X 1kn Hk−1 by (2.13), the de

68. nition of Hk−1 = X 1k+1n Hk replacing k by k + 1 = X 0kn Hk . simplifying the bounds on k Alas! We don't know how to get a sum of harmonic numbers into closed form. Get out the whip. If we try summing

69. rst the other way, we get Sn = X 1jn X jkn 1 k − j summing

70. rst on k = X 1jn X jk+jn 1 k replacing k by k + j = X 1jn X 0kn−j 1 k simplifying the bounds on k = X 1jn Hn−j by (2.13), the de

71. nition of Hn−j = X 1n−jn Hj replacing j by n − j = X 0jn Hj . simplifying the bounds on j We're back at the same impasse. But there's another way to proceed, if we replace k by k + j before deciding to reduce Sn to a sum of sums: Sn = X 1jkn 1 k − j recopying the given sum = X 1jk+jn 1 k replacing k by k + j

72. 2.4 MULTIPLE SUMS 41 = X 1kn X 1jn−k 1 k summing

73. rst on j = X 1kn n − k k the sum on j is trivial = X 1kn n k − X 1kn 1 by the associative law = n µ X 1kn 1 k ¶ − n by gosh = nHn − n . by (2.13), the de

74. nition of Hn Aha! We've found Sn. Combining this with the false starts we made gives us It's smart to say k n instead of k n − 1 here. Simple bounds save energy. a further identity as a bonus: X 0kn Hk = nHn − n . (2.36) We can understand the trick that worked here in two ways, one algebraic and one geometric. (1) Algebraically, if we have a double sum whose terms in- volve k+f(j), where f is an arbitrary function, this example indicates that it's a good idea to try replacing k by k−f(j) and summing on j. (2) Geometrically, we can look at this particular sum Sn as follows, in the case n = 4: k = 1 k = 2 k = 3 k = 4 j = 1 1 1 + 1 2 + 1 3 j = 2 1 1 + 1 2 j = 3 1 1 j = 4 Our

75. rst attempts, summing

76. rst on j (by columns) or on k (by rows), gave us H1 + H2 + H3 = H3 + H2 + H1. The winning idea was essentially to sum by diagonals, getting 3 1 + 2 2 + 1 3 . 2.5 GENERAL METHODS Now let's consolidate what we've learned, by looking at a single example from several dierent angles. On the next few pages we're going to try to

77. nd a closed form for the sum of the

78. rst n squares, which we'll call n: n = X 0kn k2 , for n 0. (2.37) We'll see that there are at least seven dierent ways to solve this problem, and in the process we'll learn useful strategies for attacking sums in general.

79. 42 SUMS First, as usual, we look at some small cases. n 0 1 2 3 4 5 6 7 8 9 10 11 12 n2 0 1 4 9 16 25 36 49 64 81 100 121 144 n 0 1 5 14 30 55 91 140 204 285 385 506 650 No closed form for n is immediately evident; but when we do

80. nd one, we can use these values as a check. Method 0: You could look it up. A problem like the sum of the

81. rst n squares has probably been solved before, so we can most likely

82. nd the solution in a handy reference book. Sure enough, page 36 of the CRC Standard Mathematical Tables [28] has the answer: n = n(n + 1)(2n + 1) 6 , for n 0. (2.38) Just to make sure we haven't misread it, we check that this formula correctly gives 5 = 5 · 6 · 11/6 = 55. Incidentally, page 36 of the CRC Tables has further information about the sums of cubes, . . . , tenth powers. The de

83. nitive reference for mathematical formulas is the Handbook of Mathematical Functions, edited by Abramowitz and Stegun [2]. Pages 813{ (Harder sums can be found in Hansen's comprehensive table [178].) 814 of that book list the values of n for n 100; and pages 804 and 809 exhibit formulas equivalent to (2.38), together with the analogous formulas for sums of cubes, . . . ,

84. fteenth powers, with or without alternating signs. But the best source for answers to questions about sequences is an amaz- ing little book called the Handbook of Integer Sequences, by Sloane [330], which lists thousands of sequences by their numerical values. If you come up with a recurrence that you suspect has already been studied, all you have to do is compute enough terms to distinguish your recurrence from other famous ones; then chances are you'll

85. nd a pointer to the relevant literature in Sloane's Handbook. For example, 1, 5, 14, 30, . . . turns out to be Sloane's sequence number 1574, and it's called the sequence of square pyramidal numbers (because there are n balls in a pyramid that has a square base of n2 balls). Sloane gives three references, one of which is to the handbook of Abramowitz and Stegun that we've already mentioned. Still another way to probe the world's store of accumulated mathematical wisdom is to use a computer program (such as Axiom, MACSYMA, Maple, or Mathematica) that provides tools for symbolic manipulation. Such programs are indispensable, especially for people who need to deal with large formulas. It's good to be familiar with standard sources of information, because they can be extremely helpful. But Method 0 isn't really consistent with the spirit of this book, because we want to know how to

86. gure out the answers

87. 2.5 GENERAL METHODS 43 by ourselves. The look-up method is limited to problems that other people Or, at least to problems having the same answers as problems that other people have decided to consider. have decided are worth considering; a new problem won't be there. Method 1: Guess the answer, prove it by induction. Perhaps a little bird has told us the answer to a problem, or we have arrived at a closed form by some other less-than-rigorous means. Then we merely have to prove that it is correct. We might, for example, have noticed that the values of n have rather small prime factors, so we may have come up with formula (2.38) as something that works for all small values of n. We might also have conjectured the equivalent formula n = n(n + 1 2 )(n + 1) 3 , for n 0, (2.39) which is nicer because it's easier to remember. The preponderance of the evidence supports (2.39), but we must prove our conjectures beyond all reasonable doubt. Mathematical induction was invented for this purpose. Well, Your Honor, we know that 0 = 0 = 0(0+ 1 2 )(0+1)/3, so the basis is easy. For the induction, suppose that n 0, and assume that (2.39) holds when n is replaced by n − 1. Since n = n−1 + n2 , we have 3 n = (n − 1)(n − 1 2 )(n) + 3n2 = (n3 − 3 2 n2 + 1 2 n) + 3n2 = (n3 + 3 2 n2 + 1 2 n) = n(n + 1 2 )(n + 1) . Therefore (2.39) indeed holds, beyond a reasonable doubt, for all n 0. Judge Wapner, in his in

88. nite wisdom, agrees. Induction has its place, and it is somewhat more defensible than trying to look up the answer. But it's still not really what we're seeking. All of the other sums we have evaluated so far in this chapter have been conquered without induction; we should likewise be able to determine a sum like n from scratch. Flashes of inspiration should not be necessary. We should be able to do sums even on our less creative days. Method 2: Perturb the sum. So let's go back to the perturbation method that worked so well for the geometric progression (2.25). We extract the

89. rst and last terms of n+1 in

90. 44 SUMS order to get an equation for n: n + (n + 1)2 = X 0kn (k + 1)2 = X 0kn (k2 + 2k + 1) = X 0kn k2 + 2 X 0kn k + X 0kn 1 = n + 2 X 0kn k + (n + 1) . Oops|the n's cancel each other. Occasionally, despite our best eorts, the perturbation method produces something like n = n, so we lose. Seems more like a draw. On the other hand, this derivation is not a total loss; it does reveal a way to sum the

91. rst n integers in closed form, 2 X 0kn k = (n + 1)2 − (n + 1) , even though we'd hoped to discover the sum of

92. rst integers squared. Could it be that if we start with the sum of the integers cubed, which we might call n, we will get an expression for the integers squared? Let's try it. n + (n + 1)3 = X 0kn (k + 1)3 = X 0kn (k3 + 3k2 + 3k + 1) = n + 3 n + 3 (n+1)n 2 + (n+1) . Sure enough, the n's cancel, and we have enough information to determine Method 2 0 : Perturb your TA. n without relying on induction: 3 n = (n + 1)3 − 3(n + 1)n/2 − (n + 1) = (n + 1)(n2 + 2n + 1 − 3 2 n − 1) = (n + 1)(n + 1 2 )n . Method 3: Build a repertoire. A slight generalization of the recurrence (2.7) will also suce for sum- mands involving n2 . The solution to R0 = α ; Rn = Rn−1 + β + γn + δn2 , for n 0, (2.40) will be of the general form Rn = A(n)α + B(n)β + C(n)γ + D(n)δ ; (2.41) and we have already determined A(n), B(n), and C(n), because (2.40) is the same as (2.7) when δ = 0. If we now plug in Rn = n3 , we

93. nd that n3 is the

94. 2.5 GENERAL METHODS 45 solution when α = 0, β = 1, γ = −3, δ = 3. Hence 3D(n) − 3C(n) + B(n) = n3 ; this determines D(n). We're interested in the sum n, which equals n−1 + n2 ; thus we get n = Rn if we set α = β = γ = 0 and δ = 1 in (2.41). Consequently n = D(n). We needn't do the algebra to compute D(n) from B(n) and C(n), since we already know what the answer will be; but doubters among us should be reassured to

95. nd that 3D(n) = n3 + 3C(n) − B(n) = n3 + 3 (n+1)n 2 − n = n(n+1 2 )(n+1) . Method 4: Replace sums by integrals. People who have been raised on calculus instead of discrete mathematics tend to be more familiar with R than with P , so they

96. nd it natural to try changing P to R . One of our goals in this book is to become so comfortable with P that we'll think R is more dicult than P (at least for exact results). But still, it's a good idea to explore the relation between P and R , since summation and integration are based on very similar ideas. In calculus, an integral can be regarded as the area under a curve, and we can approximate this area by adding up the areas of long, skinny rectangles that touch the curve. We can also go the other way if a collection of long, skinny rectangles is given: Since n is the sum of the areas of rectangles whose sizes are 1 × 1, 1 × 4, . . . , 1 × n2 , it is approximately equal to the area under the curve f(x) = x2 between 0 and n. The horizontal scale here is ten times the vertical scale. 6 - f(x) x 1 2 3 . . . n f(x) = x2 qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq The area under this curve is Rn 0 x2 dx = n3 /3; therefore we know that n is approximately 1 3 n3 .

97. 46 SUMS One way to use this fact is to examine the error in the approximation, En = n − 1 3 n3 . Since n satis

98. es the recurrence n = n−1 + n2 , we

99. nd that En satis

100. es the simpler recurrence En = n − 1 3 n3 = n−1 + n2 − 1 3 n3 = En−1 + 1 3 (n−1)3 + n2 − 1 3 n3 = En−1 + n − 1 3 . Another way to pursue the integral approach is to

101. nd a formula for En by summing the areas of the wedge-shaped error terms. We have This is for people addicted to calculus. n − Zn 0 x2 dx = n X k=1 µ k2 − Zk k−1 x2 dx ¶ = n X k=1 µ k2 − k3 − (k − 1)3 3 ¶ = n X k=1 ¡ k − 1 3 ¢ . Either way, we could

102. nd En and then n. Method 5: Expand and contract. Yet another way to discover a closed form for n is to replace the original sum by a seemingly more complicated double sum that can actually be simpli

103. ed if we massage it properly: n = X 1kn k2 = X 1jkn k = X 1jn X jkn k = X 1jn µ j + n 2 ¶ (n − j + 1) = 1 2 X 1jn ¡ n(n + 1) + j − j2 ¢ = 1 2 n2 (n + 1) + 1 4 n(n + 1) − 1 2 n = 1 2 n(n + 1 2 )(n + 1) − 1 2 n . Going from a single sum to a double sum may appear at

104. rst to be a backward (The last step here is something like the last step of the perturbation method, because we get an equation with the unknown quantity on both sides.) step, but it's actually progress, because it produces sums that are easier to work with. We can't expect to solve every problem by continually simplifying, simplifying, and simplifying: You can't scale the highest mountain peaks by climbing only uphill. Method 6: Use finite calculus. Method 7: Use generating functions. Stay tuned for still more exciting calculations of n = Pn k=0 k2 , as we learn further techniques in the next section and in later chapters.

105. 2.6 FINITE AND INFINITE CALCULUS 47 2.6 FINITE AND INFINITE CALCULUS We've learned a variety of ways to deal with sums directly. Now it's time to acquire a broader perspective, by looking at the problem of summation from a higher level. Mathematicians have developed a

106. nite calculus, analogous to the more traditional in

107. nite calculus, by which it's possible to approach summation in a nice, systematic fashion. In

108. nite calculus is based on the properties of the derivative operator D, de

109. ned by Df(x) = lim h→0 f(x + h) − f(x) h . Finite calculus is based on the properties of the dierence operator ∆, de

110. ned by ∆f(x) = f(x + 1) − f(x) . (2.42) This is the

111. nite analog of the derivative in which we restrict ourselves to positive integer values of h. Thus, h = 1 is the closest we can get to the limit as h → 0, and ∆f(x) is the value of ¡ f(x + h) − f(x) ¢ /h when h = 1. The symbols D and ∆ are called operators because they operate on functions to give new functions; they are functions of functions that produce functions. If f is a suitably smooth function of real numbers to real numbers, then Df is also a function from reals to reals. And if f is any real-to-real As opposed to a cassette function. function, so is ∆f. The values of the functions Df and ∆f at a point x are given by the de

112. nitions above. Early on in calculus we learn how D operates on the powers f(x) = xm . In such cases Df(x) = mxm−1 . We can write this informally with f omitted, D(xm ) = mxm−1 . It would be nice if the ∆ operator would produce an equally elegant result; unfortunately it doesn't. We have, for example, ∆(x3 ) = (x + 1)3 − x3 = 3x2 + 3x + 1 . But there is a type of mth power that does transform nicely under ∆, Math power. and this is what makes

113. nite calculus interesting. Such newfangled mth powers are de

114. ned by the rule xm = m factors z }| { x(x − 1) . . . (x − m + 1) , integer m 0. (2.43) Notice the little straight line under the m; this implies that the m factors are supposed to go down and down, stepwise. There's also a corresponding

115. 48 SUMS de

116. nition where the factors go up and up: xm = m factors z }| { x(x + 1) . . . (x + m − 1) , integer m 0. (2.44) When m = 0, we have x0 = x0 = 1, because a product of no factors is conventionally taken to be 1 (just as a sum of no terms is conventionally 0). The quantity xm is called x to the m falling, if we have to read it aloud; similarly, xm is x to the m rising. These functions are also called falling factorial powers and rising factorial powers, since they are closely related to the factorial function n! = n(n − 1) . . . (1). In fact, n! = nn = 1n . Several other notations for factorial powers appear in the mathematical literature, notably Pochhammer's symbol (x)m for xm or xm ; notations Mathematical terminology is sometimes crazy: Pochhammer [293] actually used the notation (x)m for the binomial coecient ąx m ć , not for factorial powers. like x(m) or x(m) are also seen for xm . But the underline/overline convention is catching on, because it's easy to write, easy to remember, and free of redundant parentheses. Falling powers xm are especially nice with respect to ∆. We have ∆(xm ) = (x + 1)m − xm = (x + 1)x . . . (x − m + 2) − x . . . (x − m + 2)(x − m + 1) = m x(x − 1) . . . (x − m + 2) , hence the

117. nite calculus has a handy law to match D(xm ) = mxm−1 : ∆(xm ) = mxm−1 . (2.45) This is the basic factorial fact. The operator D of in

118. nite calculus has an inverse, the anti-derivative (or integration) operator R . The Fundamental Theorem of Calculus relates D to R : g(x) = Df(x) if and only if Z g(x) dx = f(x) + C . Here R g(x) dx, the inde

119. nite integral of g(x), is the class of functions whose derivative is g(x). Analogously, ∆ has as an inverse, the anti-dierence (or Quemadmodum ad dierentiam denotandam usi sumus signo ∆, ita summam indi- cabimus signo Σ. . . . ex quo quatio z = ∆y, si inver- tatur, dabit quoque y = Σz + C. |L. Euler [110] summation) operator P ; and there's another Fundamental Theorem: g(x) = ∆f(x) if and only if X g(x) δx = f(x) + C . (2.46) Here P g(x) δx, the inde

120. nite sum of g(x), is the class of functions whose dierence is g(x). (Notice that the lowercase δ relates to uppercase ∆ as d relates to D.) The C for inde

121. nite integrals is an arbitrary constant; the C for inde

122. nite sums is any function p(x) such that p(x + 1) = p(x). For

123. 2.6 FINITE AND INFINITE CALCULUS 49 example, C might be the periodic function a + b sin2πx; such functions get washed out when we take dierences, just as constants get washed out when we take derivatives. At integer values of x, the function C is constant. Now we're almost ready for the punch line. In

124. nite calculus also has de

125. nite integrals: If g(x) = Df(x), then Zb a g(x) dx = f(x) ¯ ¯ ¯ b a = f(b) − f(a) . Therefore

126. nite calculus|ever mimicking its more famous cousin|has def- inite sums: If g(x) = ∆f(x), then Xb a g(x) δx = f(x) ¯ ¯ ¯ b a = f(b) − f(a) . (2.47) This formula gives a meaning to the notation Pb a g(x) δx, just as the previous formula de

127. nes Rb a g(x) dx. But what does Pb a g(x) δx really mean, intuitively? We've de

128. ned it by analogy, not by necessity. We want the analogy to hold, so that we can easily remember the rules of

129. nite calculus; but the notation will be useless if we don't understand its signi

130. cance. Let's try to deduce its meaning by looking

131. rst at some special cases, assuming that g(x) = ∆f(x) = f(x + 1) − f(x). If b = a, we have Xa a g(x) δx = f(a) − f(a) = 0 . Next, if b = a + 1, the result is Xa+1 a g(x) δx = f(a + 1) − f(a) = g(a) . More generally, if b increases by 1, we have Xb+1 a g(x) δx − Xb a g(x) δx = ¡ f(b + 1) − f(a) ¢ − ¡ f(b) − f(a) ¢ = f(b + 1) − f(b) = g(b) . These observations, and mathematical induction, allow us to deduce exactly what Pb a g(x) δx means in general, when a and b are integers with b a: Xb a g(x) δx = b−1 X k=a g(k) = X akb g(k) , for integers b a. (2.48) In other words, the de

132. nite sum is the same as an ordinary sum with limits, You call this a punch line? but excluding the value at the upper limit.

133. 50 SUMS Let's try to recap this in a slightly dierent way. Suppose we've been given an unknown sum that's supposed to be evaluated in closed form, and suppose we can write it in the form P akb g(k) = Pb a g(x) δx. The theory of

134. nite calculus tells us that we can express the answer as f(b) − f(a), if we can only

135. nd an inde

136. nite sum or anti-dierence function f such that g(x) = f(x + 1) − f(x). One way to understand this principle is to write P akb g(k) out in full, using the three-dots notation: X akb ¡ f(k + 1) − f(k) ¢ = ¡ f(a+1) − f(a) ¢ + ¡ f(a+2) − f(a+1) ¢ + · · · + ¡ f(b−1) − f(b−2) ¢ + ¡ f(b) − f(b−1) ¢ . Everything on the right-hand side cancels, except f(b) − f(a); so f(b) − f(a) is the value of the sum. (Sums of the form P akb ¡ f(k + 1) − f(k) ¢ are often called telescoping, by analogy with a collapsed telescope, because the thickness of a collapsed telescope is determined solely by the outer radius of And all this time I thought it was telescoping because it collapsed from a very long expression to a very short one. the outermost tube and the inner radius of the innermost tube.) But rule (2.48) applies only when b a; what happens if b a? Well, (2.47) says that we must have Xb a g(x) δx = f(b) − f(a) = − ¡ f(a) − f(b) ¢ = − Xa b g(x) δx . This is analogous to the corresponding equation for de

137. nite integration. A similar argument proves Pb a + Pc b = Pc a, the summation analog of the identity Rb a + Rc b = Rc a . In full garb, Xb a g(x) δx + Xc b g(x) δx = Xc a g(x) δx , (2.49) for all integers a, b, and c. At this point a few of us are probably starting to wonder what all these parallels and analogies buy us. Well for one, de

138. nite summation gives us a Others have been wondering this for some time now. simple way to compute sums of falling powers: The basic laws (2.45), (2.47), and (2.48) imply the general law X 0kn km = km+1 m + 1 ¯ ¯ ¯ ¯ n 0 = nm+1 m + 1 , for integers m, n 0. (2.50) This formula is easy to remember because it's so much like the familiar Rn 0 xm dx = nm+1 /(m + 1).

139. 2.6 FINITE AND INFINITE CALCULUS 51 In particular, when m = 1 we have k1 = k, so the principles of

140. nite calculus give us an easy way to remember the fact that X 0kn k = n2 2 = n(n − 1)/2 . The de

141. nite-sum method also gives us an inkling that sums over the range 0 k n often turn out to be simpler than sums over 1 k n; the former are just f(n) − f(0), while the latter must be evaluated as f(n + 1) − f(1). Ordinary powers can also be summed in this new way, if we

142. rst express them in terms of falling powers. For example, k2 = k2 + k1 , hence X 0kn k2 = n3 3 + n2 2 = 1 3 n(n − 1)(n − 2 + 3 2 ) = 1 3 n(n − 1 2 )(n − 1) . Replacing n by n + 1 gives us yet another way to compute the value of our old friend n = P 0kn k2 in closed form. With friends like this. . . Gee, that was pretty easy. In fact, it was easier than any of the umpteen other ways that beat this formula to death in the previous section. So let's try to go up a notch, from squares to cubes: A simple calculation shows that k3 = k3 + 3k2 + k1 . (It's always possible to convert between ordinary powers and factorial powers by using Stirling numbers, which we will study in Chapter 6.) Thus X akb k3 = k4 4 + k3 + k2 2 ¯ ¯ ¯ ¯ b a . Falling powers are therefore very nice for sums. But do they have any other redeeming features? Must we convert our old friendly ordinary powers to falling powers before summing, but then convert back before we can do anything else? Well, no, it's often possible to work directly with factorial powers, because they have additional properties. For example, just as we have (x + y)2 = x2 + 2xy + y2 , it turns out that (x + y)2 = x2 + 2x1 y1 + y2 , and the same analogy holds between (x + y)m and (x + y)m . (This factorial binomial theorem is proved in exercise 5.37.) So far we've considered only falling powers that have nonnegative exponents. To extend the analogies with ordinary powers to negative exponents,

143. 52 SUMS we need an appropriate de

144. nition of xm for m 0. Looking at the sequence x3 = x(x − 1)(x − 2) , x2 = x(x − 1) , x1 = x , x0 = 1 , we notice that to get from x3 to x2 to x1 to x0 we divide by x − 2, then by x − 1, then by x. It seems reasonable (if not imperative) that we should divide by x + 1 next, to get from x0 to x−1 , thereby making x−1 = 1/(x + 1). Continuing, the

145. rst few negative-exponent falling powers are x−1 = 1 x + 1 , x−2 = 1 (x + 1)(x + 2) , x−3 = 1 (x + 1)(x + 2)(x + 3) , and our general de

146. nition for negative falling powers is x−m = 1 (x + 1)(x + 2) . . . (x + m) , for m 0. (2.51) (It's also possible to de

147. ne falling powers for real or even complex m, but we How can a complex number be even? will defer that until Chapter 5.) With this de

148. nition, falling powers have additional nice properties. Per- haps the most important is a general law of exponents, analogous to the law xm+n = xm xn for ordinary powers. The falling-power version is xm+n = xm (x − m)n , integers m and n. (2.52) For example, x2+3 = x2 (x − 2)3 ; and with a negative n we have x2−3 = x2 (x − 2)−3 = x(x − 1) 1 (x − 1)x(x + 1) = 1 x + 1 = x−1 . If we had chosen to de

149. ne x−1 as 1/x instead of as 1/(x + 1), the law of exponents (2.52) would have failed in cases like m = −1 and n = 1. In fact, we could have used (2.52) to tell us exactly how falling powers ought to be de

150. ned in the case of negative exponents, by setting m = −n. When an Laws have their exponents and their detractors. existing notation is being extended to cover more cases, it's always best to formulate de

151. nitions in such a way that general laws continue to hold.

152. 2.6 FINITE AND INFINITE CALCULUS 53 Now let's make sure that the crucial dierence property holds for our newly de

153. ned falling powers. Does ∆xm = mxm−1 when m 0? If m = −2, for example, the dierence is ∆x−2 = 1 (x + 2)(x + 3) − 1 (x + 1)(x + 2) = (x + 1) − (x + 3) (x + 1)(x + 2)(x + 3) = −2x−3 . Yes|it works! A similar argument applies for all m 0. Therefore the summation property (2.50) holds for negative falling powers as well as positive ones, as long as no division by zero occurs: Xb a xm δx = xm+1 m + 1 ¯ ¯ ¯ ¯ b a , for m 6= −1. But what about when m = −1? Recall that for integration we use Zb a x−1 dx = lnx ¯ ¯ ¯ b a when m = −1. We'd like to have a

154. nite analog of lnx; in other words, we seek a function f(x) such that x−1 = 1 x + 1 = ∆f(x) = f(x + 1) − f(x) . It's not too hard to see that f(x) = 1 1 + 1 2 + · · · + 1 x is such a function, when x is an integer, and this quantity is just the harmonic number Hx of (2.13). Thus Hx is the discrete analog of the continuous lnx. (We will de

155. ne Hx for noninteger x in Chapter 6, but integer values are good enough for present purposes. We'll also see in Chapter 9 that, for large x, the value of Hx − lnx is approximately 0.577 + 1/(2x). Hence Hx and lnx are not 0.577 exactly? Maybe they mean 1/ √ 3. Then again, maybe not. only analogous, their values usually dier by less than 1.) We can now give a complete description of the sums of falling powers: Xb a xm δx =        xm+1 m + 1 ¯ ¯ ¯ ¯ b a , if m 6= −1; Hx ¯ ¯ ¯ b a , if m = −1. (2.53)

156. 54 SUMS This formula indicates why harmonic numbers tend to pop up in the solutions to discrete problems like the analysis of quicksort, just as so-called natural logarithms arise naturally in the solutions to continuous problems. Now that we've found an analog for lnx, let's see if there's one for ex . What function f(x) has the property that ∆f(x) = f(x), corresponding to the identity Dex = ex ? Easy: f(x + 1) − f(x) = f(x) ⇐⇒ f(x + 1) = 2f(x) ; so we're dealing with a simple recurrence, and we can take f(x) = 2x as the discrete exponential function. The dierence of cx is also quite simple, for arbitrary c, namely ∆(cx ) = cx+1 − cx = (c − 1)cx . Hence the anti-dierence of cx is cx /(c − 1), if c 6= 1. This fact, together with the fundamental laws (2.47) and (2.48), gives us a tidy way to understand the general formula for the sum of a geometric progression: X akb ck = Xb a cx δx = cx c − 1 ¯ ¯ ¯ ¯ b a = cb − ca c − 1 , for c 6= 1. Every time we encounter a function f that might be useful as a closed form, we can compute its dierence ∆f = g; then we have a function g whose inde

157. nite sum P g(x) δx is known. Table 55 is the beginning of a table of `Table 55' is on page 55. Get it? dierence/anti-dierence pairs useful for summation. Despite all the parallels between continuous and discrete math, some continuous notions have no discrete analog. For example, the chain rule of in

158. nite calculus is a handy rule for the derivative of a function of a function; but there's no corresponding chain rule of

159. nite calculus, because there's no nice form for ∆f ¡ g(x) ¢ . Discrete change-of-variables is hard, except in certain cases like the replacement of x by c ± x. However, ∆ ¡ f(x) g(x) ¢ does have a fairly nice form, and it provides us with a rule for summation by parts, the

160. nite analog of what in

161. nite calculus calls integration by parts. Let's recall that the formula D(uv) = u Dv + v Du of in

162. nite calculus leads to the rule for integration by parts, Z u Dv = uv − Z v Du ,

163. 2.6 FINITE AND INFINITE CALCULUS 55 Table 55 What's the dierence? f = Σg ∆f = g f = Σg ∆f = g x0 = 1 0 2x 2x x1 = x 1 cx (c − 1)cx x2 = x(x − 1) 2x cx /(c − 1) cx xm mxm−1 cf c∆f xm+1 /(m + 1) xm f + g ∆f + ∆g Hx x−1 = 1/(x + 1) fg f∆g + Eg∆f after integration and rearranging terms; we can do a similar thing in

164. nite calculus. We start by applying the dierence operator to the product of two functions u(x) and v(x): ∆ ¡ u(x) v(x) ¢ = u(x+1) v(x+1) − u(x) v(x) = u(x+1) v(x+1) − u(x) v(x+1) + u(x) v(x+1) − u(x) v(x) = u(x) ∆v(x) + v(x+1) ∆u(x) . (2.54) This formula can be put into a convenient form using the shift operator E, de

165. ned by Ef(x) = f(x + 1) . Substituting Ev(x) for v(x+1) yields a compact rule for the dierence of a product: ∆(uv) = u ∆v + Ev ∆u . (2.55) (The E is a bit of a nuisance, but it makes the equation correct.) Taking In

166. nite calculus avoids E here by letting 1 → 0. the inde

167. nite sum on both sides of this equation, and rearranging its terms, yields the advertised rule for summation by parts: X u ∆v = uv − X Ev ∆u . (2.56) As with in

168. nite calculus, limits can be placed on all three terms, making the inde

169. nite sums de

170. nite. This rule is useful when the sum on the left is harder to evaluate than the one on the right. Let's look at an example. The function R xex dx is typically integrated by parts; its discrete analog is P x2x δx, which we encountered I guess ex = 2x , for small values of 1. earlier this chapter in the form Pn k=0 k 2k . To sum this by parts, we let

171. 56 SUMS u(x) = x and ∆v(x) = 2x ; hence ∆u(x) = 1, v(x) = 2x , and Ev(x) = 2x+1 . Plugging into (2.56) gives X x2x δx = x2x − X 2x+1 δx = x2x − 2x+1 + C . And we can use this to evaluate the sum we did before, by attaching limits: n X k=0 k2k = Xn+1 0 x2x δx = x2x − 2x+1 ¯ ¯ ¯ n+1 0 = ¡ (n + 1)2n+1 − 2n+2 ¢ − (0·20 − 21 ) = (n − 1)2n+1 + 2 . It's easier to

172. nd the sum this way than to use the perturbation method, because we don't have to think. The ultimate goal of mathematics is to eliminate all need for intelligent thought. We stumbled across a formula for P 0kn Hk earlier in this chapter, and counted ourselves lucky. But we could have found our formula (2.36) systematically, if we had known about summation by parts. Let's demonstrate this assertion by tackling a sum that looks even harder, P 0kn kHk. The solution is not dicult if we are guided by analogy with R x lnx dx: We take u(x) = Hx and ∆v(x) = x = x1 , hence ∆u(x) = x−1 , v(x) = x2 /2, Ev(x) = (x + 1)2 /2, and we have X xHx δx = x2 2 Hx − X (x + 1)2 2 x−1 δx = x2 2 Hx − 1 2 X x1 δx = x2 2 Hx − x2 4 + C . (In going from the

173. rst line to the second, we've combined two falling powers (x+1)2 x−1 by using the law of exponents (2.52) with m = −1 and n = 2.) Now we can attach limits and conclude that X 0kn kHk = Xn 0 xHx δx = n2 2 ³ Hn − 1 2 ´ . (2.57) 2.7 INFINITE SUMS When we de

174. ned P -notation at the beginning of this chapter, we

175. nessed the question of in

176. nite sums by saying, in essence, Wait until later. This is

177. nesse? For now, we can assume that all the sums we meet have only

178. nitely many nonzero terms. But the time of reckoning has

179. nally arrived; we must face

180. 2.7 INFINITE SUMS 57 the fact that sums can be in

181. nite. And the truth is that in

182. nite sums are bearers of both good news and bad news. First, the bad news: It turns out that the methods we've used for manipulating P 's are not always valid when in

183. nite sums are involved. But next, the good news: There is a large, easily understood class of in

184. nite sums for which all the operations we've been performing are perfectly legitimate. The reasons underlying both these news items will be clear after we have looked more closely at the underlying meaning of summation. Everybody knows what a

185. nite sum is: We add up a bunch of terms, one by one, until they've all been added. But an in

186. nite sum needs to be de

187. ned more carefully, lest we get into paradoxical situations. For example, it seems natural to de

188. ne things so that the in

189. nite sum S = 1 + 1 2 + 1 4 + 1 8 + 1 16 + 1 32 + · · · is equal to 2, because if we double it we get 2S = 2 + 1 + 1 2 + 1 4 + 1 8 + 1 16 + · · · = 2 + S . On the other hand, this same reasoning suggests that we ought to de

190. ne T = 1 + 2 + 4 + 8 + 16 + 32 + · · · to be −1, for if we double it we get Sure: 1 + 2 + 4 + 8 + · · · is the in

191. nite precision representation of the number −1, in a binary computer with in

192. nite word size. 2T = 2 + 4 + 8 + 16 + 32 + 64 + · · · = T − 1. Something funny is going on; how can we get a negative number by summing positive quantities? It seems better to leave T unde

193. ned; or perhaps we should say that T = ∞, since the terms being added in T become larger than any

194. xed,

195. nite number. (Notice that ∞ is another solution to the equation 2T = T − 1; it also solves the equation 2S = 2 + S.) Let's try to formulate a good de

196. nition for the value of a general sum P k∈K ak, where K might be in

197. nite. For starters, let's assume that all the terms ak are nonnegative. Then a suitable de

198. nition is not hard to

199. nd: If there's a bounding constant A such that X k∈F ak A for all

200. nite subsets F ⊂ K, then we de

201. ne P k∈K ak to be the least such A. (It follows from well-known properties of the real numbers that the set of all such A always contains a smallest element.) But if there's no bounding constant A, we say that P k∈K ak = ∞; this means that if A is any real number, there's a set of

202. nitely many terms ak whose sum exceeds A.

203. 58 SUMS The de

204. nition in the previous paragraph has been formulated carefully so that it doesn't depend on any order that might exist in the index set K. Therefore the arguments we are about to make will apply to multiple sums with many indices k1, k2, . . . , not just to sums over the set of integers. The set K might even be uncount- able. But only a countable number of terms can be nonzero, if a bounding constant A exists, because at most nA terms are 1/n. In the special case that K is the set of nonnegative integers, our de

205. nition for nonnegative terms ak implies that X k0 ak = lim n→∞ n X k=0 ak . Here's why: Any nondecreasing sequence of real numbers has a limit (possi- bly ∞). If the limit is A, and if F is any

206. nite set of nonnegative integers whose elements are all n, we have P k∈F ak Pn k=0 ak A; hence A = ∞ or A is a bounding constant. And if A0 is any number less than the stated limit A, then there's an n such that Pn k=0 ak A0 ; hence the

207. nite set F = f0, 1, . . . , ng witnesses to the fact that A0 is not a bounding constant. We can now easily compute the value of certain in

208. nite sums, according to the de

209. nition just given. For example, if ak = xk , we have X k0 xk = lim n→∞ 1 − xn+1 1 − x = ¯ 1/(1 − x), if 0 x 1; ∞, if x 1. In particular, the in

210. nite sums S and T considered a minute ago have the re- spective values 2 and ∞, just as we suspected. Another interesting example is X k0 1 (k + 1)(k + 2) = X k0 k−2 = lim n→∞ n X k=0 k−2 = lim n→∞ k−1 −1 ¯ ¯ ¯ ¯ n 0 = 1 . Now let's consider the case that the sum might have negative terms as well as nonnegative ones. What, for example, should be the value of X k0 (−1)k = 1 − 1 + 1 − 1 + 1 − 1 + · · · ? If we group the terms in pairs, we get Aggregatum quan- titatum a − a + a − a + a − a etc. nunc est = a, nunc = 0, adeoque continuata in in

211. ni- tum serie ponendus = a/2, fateor acumen et veritatem animadversionis tu. |G. Grandi [163] (1 − 1) + (1 − 1) + (1 − 1) + · · · = 0 + 0 + 0 + · · · , so the sum comes out zero; but if we start the pairing one step later, we get 1 − (1 − 1) − (1 − 1) − (1 − 1) − · · · = 1 − 0 − 0 − 0 − · · · ; the sum is 1.

212. 2.7 INFINITE SUMS 59 We might also try setting x = −1 in the formula P k0 xk = 1/(1 − x), since we've proved that this formula holds when 0 x 1; but then we are forced to conclude that the in

213. nite sum is 1 2 , although it's a sum of integers! Another interesting example is the doubly in

214. nite P k ak where ak = 1/(k + 1) for k 0 and ak = 1/(k − 1) for k 0. We can write this as · · · + (−1 4 ) + (−1 3 ) + (−1 2 ) + 1 + 1 2 + 1 3 + 1 4 + · · · . (2.58) If we evaluate this sum by starting at the center element and working outward, · · · + ³ −1 4 + ¡ −1 3 + (−1 2 + (1) + 1 2 ) + 1 3 ¢ + 1 4 ´ + · · · , we get the value 1; and we obtain the same value 1 if we shift all the parentheses one step to the left, · · · + ³ −1 5 + ¡ −1 4 + (−1 3 + (−1 2 ) + 1) + 1 2 ¢ + 1 3 ´ + · · · , because the sum of all numbers inside the innermost n parentheses is − 1 n + 1 − 1 n − · · · − 1 2 + 1 + 1 2 + · · · + 1 n − 1 = 1 − 1 n − 1 n + 1 . A similar argument shows that the value is 1 if these parentheses are shifted any

215. xed amount to the left or right; this encourages us to believe that the sum is indeed 1. On the other hand, if we group terms in the following way, · · · + ³ −1 4 + ¡ −1 3 + (−1 2 + 1 + 1 2 ) + 1 3 + 1 4 ¢ + 1 5 + 1 6 ´ + · · · , the nth pair of parentheses from inside out contains the numbers − 1 n+1 − 1 n − · · · − 1 2 + 1 + 1 2 + · · · + 1 2n−1 + 1 2n = 1 + H2n − Hn+1 . We'll prove in Chapter 9 that limn→∞(H2n −Hn+1) = ln2; hence this group- ing suggests that the doubly in

216. nite sum should really be equal to 1 + ln2. There's something aky about a sum that gives dierent values when its terms are added up in dierent ways. Advanced texts on analysis have a variety of de

217. nitions by which meaningful values can be assigned to such pathological sums; but if we adopt those de

218. nitions, we cannot operate with P -notation as freely as we have been doing. We don't need the delicate re

219. ne- ments of conditional convergence for the purposes of this book; therefore we'll stick to a de

220. nition of in

221. nite sums that preserves the validity of all the Is this the

222. rst page with no grati? operations we've been doing in this chapter.

223. 60 SUMS In fact, our de

224. nition of in

225. nite sums is quite simple. Let K be any set, and let ak be a real-valued term de

226. ned for each k ∈ K. (Here `k' might actually stand for several indices k1, k2, . . . , and K might therefore be multidimensional.) Any real number x can be written as the dierence of its positive and negative parts, x = x+ − x− , where x+ = x·[x 0] and x− = −x·[x 0]. (Either x+ = 0 or x− = 0.) We've already explained how to de

227. ne values for the in

228. nite sums P k∈K a+ k and P k∈K a− k , because a+ k and a− k are nonnegative. Therefore our general de

229. nition is X k∈K ak = X k∈K a+ k − X k∈K a− k , (2.59) unless the right-hand sums are both equal to ∞. In the latter case, we leave P k∈K ak unde

230. ned. Let A+ = P k∈K a+ k and A− = P k∈K a− k . If A+ and A− are both

231. nite, the sum P k∈K ak is said to converge absolutely to the value A = A+ − A− . In other words, absolute convergence means that the sum of absolute values converges. If A+ = ∞ but A− is

232. nite, the sum P k∈K ak is said to diverge to +∞. Similarly, if A− = ∞ but A+ is

233. nite, P k∈K ak is said to diverge to −∞. If A+ = A− = ∞, all bets are o. We started with a de

234. nition that worked for nonnegative terms, then we extended it to real-valued terms. If the terms ak are complex numbers, we can extend the de

235. nition once again, in the obvious way: The sum P k∈K ak is de

236. ned to be P k∈K ak + i P k∈K =ak, where ak and =ak are the real and imaginary parts of ak |provided that both of those sums are de

237. ned. Otherwise P k∈K ak is unde

238. ned. (See exercise 18.) The bad news, as stated earlier, is that some in

239. nite sums must be left unde

240. ned, because the manipulations we've been doing can produce inconsis- tencies in all such cases. (See exercise 34.) The good news is that all of the manipulations of this chapter are perfectly valid whenever we're dealing with sums that converge absolutely, as just de

241. ned. We can verify the good news by showing that each of our transformation rules preserves the value of all absolutely convergent sums. This means, more explicitly, that we must prove the distributive, associative, and commutative laws, plus the rule for summing

242. rst on one index variable; everything else we've done has been derived from those four basic operations on sums. The distributive law (2.15) can be formulated more precisely as follows: If P k∈K ak converges absolutely to A and if c is any complex number, then P k∈K cak converges absolutely to cA. We can prove this by breaking the sum into real and imaginary, positive and negative parts as above, and by proving the special case in which c 0 and each term ak is nonnegative. The

243. 2.7 INFINITE SUMS 61 proof in this special case works because P k∈F cak = c P k∈F ak for all

244. nite sets F; the latter fact follows by induction on the size of F. The associative law (2.16) can be stated as follows: If P k∈K ak and P k∈K bk converge absolutely to A and B, respectively, then P k∈K(ak + bk) converges absolutely to A + B. This turns out to be a special case of a more general theorem that we will prove shortly. The commutative law (2.17) doesn't really need to be proved, because we have shown in the discussion following (2.35) how to derive it as a special case of a general rule for interchanging the order of summation. The main result we need to prove is the fundamental principle of multiple sums: Absolutely convergent sums over two or more indices can always be summed

245. rst with respect to any one of those indices. Formally, we shall prove that if J and the elements of fKj | j ∈ Jg are any sets of indices such that Best to skim this page the

246. rst time you get here. |Your friendly TA X j∈J k∈Kj aj,k converges absolutely to A , then there exist complex numbers Aj for each j ∈ J such that X k∈Kj aj,k converges absolutely to Aj, and X j∈J Aj converges absolutely to A . It suces to prove this assertion when all terms are nonnegative, because we can prove the general case by breaking everything into real and imaginary, positive and negative parts as before. Let's assume therefore that aj,k 0 for all pairs (j, k) ∈ M, where M is the master index set f(j, k) | j ∈ J, k ∈ Kjg. We are given that P (j,k)∈M aj,k is

247. nite, namely that X (j,k)∈F aj,k A for all

248. nite subsets F ⊆ M, and that A is the least such upper bound. If j is any element of J, each sum of the form P k∈Fj aj,k where Fj is a

249. nite subset of Kj is bounded above by A. Hence these

250. nite sums have a least upper bound Aj 0, and P k∈Kj aj,k = Aj by de

251. nition. We still need to prove that A is the least upper bound of P j∈G Aj, for all

252. nite subsets G ⊆ J. Suppose that G is a

253. nite subset of J with P j∈G Aj = A0 A. We can

254. nd

255. nite subsets Fj ⊆ Kj such that P k∈Fj aj,k (A/A0 )Aj for each j ∈ G with Aj 0. There is at least one such j. But then P j∈G,k∈Fj aj,k (A/A0 ) P j∈G Aj = A, contradicting the fact that we have

256. 62 SUMS P (j,k)∈F aj,k A for all

257. nite subsets F ⊆ M. Hence P j∈G Aj A, for all

258. nite subsets G ⊆ J. Finally, let A0 be any real number less than A. Our proof will be complete if we can

259. nd a

260. nite set G ⊆ J such that P j∈G Aj A0 . We know that there's a

261. nite set F ⊆ M such that P (j,k)∈F aj,k A0 ; let G be the set of j's in this F, and let Fj = fk | (j, k) ∈ Fg. Then P j∈G Aj P j∈G P k∈Fj aj,k = P (j,k)∈F aj,k A0 ; QED. OK, we're now legitimate! Everything we've been doing with in

262. nite sums is justi

263. ed, as long as there's a

264. nite bound on all

265. nite sums of the absolute values of the terms. Since the doubly in

266. nite sum (2.58) gave us two dierent answers when we evaluated it in two dierent ways, its positive terms 1 + 1 2 + 1 3 + · · · must diverge to ∞; otherwise we would have gotten the So why have I been hearing a lot lately about harmonic convergence? same answer no matter how we grouped the terms. Exercises Warmups 1 What does the notation 0 X k=4 qk mean? 2 Simplify the expression x · ¡ [x 0] − [x 0] ¢ . 3 Demonstrate your understanding of P -notation by writing out the sums X 0k5 ak and X 0k25 ak2 in full. (Watch out|the second sum is a bit tricky.) 4 Express the triple sum X 1ijk4 aijk as a three-fold summation (with three P 's), a summing

267. rst on k, then j, then i; b summing

268. rst on i, then j, then k. Also write your triple sums out in full without the P -notation, using parentheses to show what is being added together

269. rst.

270. 2 EXERCISES 63 5 What's wrong with the following derivation? µ n X j=1 aj ¶µ n X k=1 1 ak ¶ = n X j=1 n X k=1 aj ak = n X k=1 n X k=1 ak ak = n X k=1 n = n2 . 6 What is the value of P k[1 j k n], as a function of j and n? 7 Let ∇f(x) = f(x) − f(x−1). What is ∇(xm )? Yield to the rising power. 8 What is the value of 0m , when m is a given integer? 9 What is the law of exponents for rising factorial powers, analogous to (2.52)? Use this to de

271. ne x−n . 10 The text derives the following formula for the dierence of a product: ∆(uv) = u ∆v + Ev ∆u . How can this formula be correct, when the left-hand side is symmetric with respect to u and v but the right-hand side is not? Basics 11 The general rule (2.56) for summation by parts is equivalent to X 0kn (ak+1 − ak)bk = anbn − a0b0 − X 0kn ak+1(bk+1 − bk), for n 0. Prove this formula directly by using the distributive, associative, and commutative laws. 12 Show that the function p(k) = k + (−1)k c is a permutation of the set of all integers, whenever c is an integer. 13 Use the repertoire method to

272. nd a closed form for Pn k=0(−1)k k2 . 14 Evaluate Pn k=1 k2k by rewriting it as the multiple sum P 1jkn 2k . 15 Evaluate n = Pn k=1 k3 by the text's Method 5 as follows: First write n + n = 2 P 1jkn jk; then apply (2.33). 16 Prove that xm /(x − n)m = xn /(x − m)n , unless one of the denominators is zero. 17 Show that the following formulas can be used to convert between rising and falling factorial powers, for all integers m: xm = (−1)m (−x)m = (x + m − 1)m = 1/(x − 1)−m ; xm = (−1)m (−x)m = (x − m + 1)m = 1/(x + 1)−m . (The answer to exercise 9 de

273. nes x−m .)

274. 64 SUMS 18 Let z and =z be the real and imaginary parts of the complex number z. The absolute value jzj is p (z)2 + (=z)2. A sum P k∈K ak of complex terms ak is said to converge absolutely when the real-valued sums P k∈K ak and P k∈K =ak both converge absolutely. Prove that P k∈K ak converges absolutely if and only if there is a bounding constant B such that P k∈F jakj B for all

275. nite subsets F ⊆ K. Homework exercises 19 Use a summation factor to solve the recurrence T0 = 5 ; 2Tn = nTn−1 + 3 · n! , for n 0. 20 Try to evaluate Pn k=0 kHk by the perturbation method, but deduce the value of Pn k=0 Hk instead. 21 Evaluate the sums Sn = Pn k=0(−1)n−k , Tn = Pn k=0(−1)n−k k, and Un = Pn k=0(−1)n−k k2 by the perturbation method, assuming that n 0. 22 Prove Lagrange's identity (without using induction): It's hard to prove the identity of somebody who's been dead for 175 years. X 1jkn (ajbk − akbj)2 = µ n X k=1 a2 k ¶µ n X k=1 b2 k ¶ − µ n X k=1 akbk ¶2 . Prove, in fact, an identity for the more general double sum X 1jkn (ajbk − akbj)(AjBk − AkBj) . 23 Evaluate the sum Pn k=1(2k + 1)/k(k + 1) in two ways: a Replace 1/k(k + 1) by the partial fractions 1/k − 1/(k + 1). b Sum by parts. 24 What is P 0kn Hk/(k + 1)(k + 2)? Hint: Generalize the derivation of (2.57). 25 The notation Q k∈K ak means the product of the numbers ak for all This notation was introduced by Jacobi in 1829 [192]. k ∈ K. Assume for simplicity that ak 6= 1 for only

276. nitely many k; hence in

277. nite products need not be de

278. ned. What laws does this Q -notation satisfy, analogous to the distributive, associative, and commutative laws that hold for P ? 26 Express the double product Q 1jkn ajak in terms of the single product Qn k=1 ak by manipulating Q -notation. ¡ This exercise gives us a product analog of the upper-triangle identity (2.33). ¢

279. 2 EXERCISES 65 27 Compute ∆(cx ), and use it to deduce the value of Pn k=1(−2)k /k. 28 At what point does the following derivation go astray? 1 = X k1 1 k(k + 1) = X k1 µ k k + 1 − k − 1 k ¶ = X k1 X j1 µ k j [j = k + 1] − j k [j = k − 1] ¶ = X j1 X k1 µ k j [j = k + 1] − j k [j = k − 1] ¶ = X j1 X k1 µ k j [k = j − 1] − j k [k = j + 1] ¶ = X j1 µ j − 1 j − j j + 1 ¶ = X j1 −1 j(j + 1) = −1 . Exam problems 29 Evaluate the sum Pn k=1(−1)k k/(4k2 − 1). 30 Cribbage players have long been aware that 15 = 7 + 8 = 4 + 5 + 6 = 1 + 2 + 3 + 4 + 5. Find the number of ways to represent 1050 as a sum of consecutive positive integers. (The trivial representation `1050' by itself counts as one way; thus there are four, not three, ways to represent 15 as a sum of consecutive positive integers. Incidentally, a knowledge of cribbage rules is of no use in this problem.) 31 Riemann's zeta function ζ(k) is de

280. ned to be the in

281. nite sum 1 + 1 2k + 1 3k + · · · = X j1 1 jk . Prove that P k2 ¡ ζ(k) − 1 ¢ = 1. What is the value of P k1 ¡ ζ(2k) − 1 ¢ ? 32 Let a . − b = max(0, a − b). Prove that X k0 min(k, x . − k) = X k0 ¡ x . − (2k + 1) ¢ for all real x 0, and evaluate the sums in closed form. Bonus problems 33 Let V k∈K ak denote the minimum of the numbers ak (or their greatest lower bound, if K is in

282. nite), assuming that each ak is either real or ±∞. What laws are valid for V -notation, analogous to those that work for P The laws of the jungle. and Q ? (See exercise 25.)

283. 66 SUMS 34 Prove that if the sum P k∈K ak is unde

284. ned according to (2.59), then it is extremely aky in the following sense: If A− and A+ are any given real numbers, it's possible to

285. nd a sequence of

286. nite subsets F1 ⊂ F2 ⊂ F3 ⊂ · · · of K such that X k∈Fn ak A− , when n is odd; X k∈Fn ak A+ , when n is even. 35 Prove Goldbach's theorem 1 = 1 3 + 1 7 + 1 8 + 1 15 + 1 24 + 1 26 + 1 31 + 1 35 + · · · = X k∈P 1 k − 1 , where P is the set of perfect powers de

287. ned recursively as follows: Perfect power corrupts perfectly. P = fmn | m 2, n 2, m / ∈ P g. 36 Solomon Golomb's self-describing sequence hf(1), f(2), f(3), . . . i is the only nondecreasing sequence of positive integers with the property that it contains exactly f(k) occurrences of k for each k. A few moments' thought reveals that the sequence must begin as follows: n 1 2 3 4 5 6 7 8 9 10 11 12 f(n) 1 2 2 3 3 4 4 4 5 5 5 6 Let g(n) be the largest integer m such that f(m) = n. Show that a g(n) = Pn k=1 f(k). b g ¡ g(n) ¢ = Pn k=1 kf(k). c g ¡ g(g(n)) ¢ = 1 2 ng(n) ¡ g(n) + 1 ¢ − 1 2 Pn−1 k=1 g(k) ¡ g(k) + 1 ¢ . Research problem 37 Will all the 1/k by 1/(k + 1) rectangles, for k 1,

288. t together inside a 1 by 1 square? (Recall that their areas sum to 1.) 1 1 1 2 1 2 1 3 1 3 1 4 1 4 1 5 etc. © ©© ©©© © ©© ©© © ©©© ©©© © © © © © © ©© © ©©© ©© ©A A A A A A A A A A A A ¦¦¦¦¦¦¦¦ ³ ³ ³ ³ ³ ³ ³ ³ © ©

Sums ADA

More Related Content

What's hot (11)

Similar to Sums ADA (20)

More from Spacetoshare (20)

Recently uploaded (20)

Sums ADA