Hard work beats talent when talent doesn’t work hard.
--Your friends at LectureNotes

Note for Design And Analysis Of Algorithm - DAA by Aman Kumar

  • Design And Analysis Of Algorithm - DAA
  • Note
  • Dr. A PJ Abdul Kalam Tech University Lucknow - AKTU
  • Computer Science Engineering
  • B.Tech
  • Uploaded 9 months ago
0 User(s)
Download PDFOrder Printed Copy

Share it with your friends

Leave your Comments

Text from page-1

Algorithms Appendix II: Solving Recurrences [Fa’13] Change is certain. Peace is followed by disturbances; departure of evil men by their return. Such recurrences should not constitute occasions for sadness but realities for awareness, so that one may be happy in the interim. — I Ching [The Book of Changes] (c. 1100 BC) To endure the idea of the recurrence one needs: freedom from morality; new means against the fact of pain (pain conceived as a tool, as the father of pleasure; there is no cumulative consciousness of displeasure); the enjoyment of all kinds of uncertainty, experimentalism, as a counterweight to this extreme fatalism; abolition of the concept of necessity; abolition of the “will”; abolition of “knowledge-in-itself.” — Friedrich Nietzsche The Will to Power (1884) [translated by Walter Kaufmann] Wil Wheaton: Embrace the dark side! Sheldon: That’s not even from your franchise! — “The Wheaton Recurrence”, Bing Bang Theory, April 12, 2010 Solving Recurrences 1 Introduction A recurrence is a recursive description of a function, or in other words, a description of a function in terms of itself. Like all recursive structures, a recurrence consists of one or more base cases and one or more recursive cases. Each of these cases is an equation or inequality, with some function value f (n) on the left side. The base cases give explicit values for a (typically finite, typically small) subset of the possible values of n. The recursive cases relate the function value f (n) to function value f (k) for one or more integers k < n; typically, each recursive case applies to an infinite number of possible values of n. For example, the following recurrence (written in two different but standard ways) describes the identity function f (n) = n: ¨ f (0) = 0 0 if n = 0 f (n) = f (n) = f (n − 1) + 1 for all n > 0 f (n − 1) + 1 otherwise In both presentations, the first line is the only base case, and the second line is the only recursive case. The same function can satisfy many different recurrences; for example, both of the following recurrences also describe the identity function:     0 if n = 0 if n = 0  0 f (n) = 1 f (n) = 2 · f (n/2) if n = 1 if n is even and n > 0    f (bn/2c) + f (dn/2e) otherwise  f (n − 1) + 1 if n is odd We say that a particular function satisfies a recurrence, or is the solution to a recurrence, if each of the statements in the recurrence is true. Most recurrences—at least, those that we will encounter in this class—have a solution; moreover, if every case of the recurrence is an equation, that solution is unique. Specifically, if we transform the recursive formula into a recursive algorithm, the solution to the recurrence is the function computed by that algorithm! © Copyright 2014 Jeff Erickson. This work is licensed under a Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/4.0/). Free distribution is strongly encouraged; commercial distribution is expressly forbidden. See http://www.cs.uiuc.edu/~jeffe/teaching/algorithms/ for the most recent revision. 1

Text from page-2

Algorithms Appendix II: Solving Recurrences [Fa’13] Recurrences arise naturally in the analysis of algorithms, especially recursive algorithms. In many cases, we can express the running time of an algorithm as a recurrence, where the recursive cases of the recurrence correspond exactly to the recursive cases of the algorithm. Recurrences are also useful tools for solving counting problems—How many objects of a particular kind exist? By itself, a recurrence is not a satisfying description of the running time of an algorithm or a bound on the number of widgets. Instead, we need a closed-form solution to the recurrence; this is a non-recursive description of a function that satisfies the recurrence. For recurrence equations, we sometimes prefer an exact closed-form solution, but such a solution may not exist, or may be too complex to be useful. Thus, for most recurrences, especially those arising in algorithm analysis, we are satisfied with an asymptotic solution of the form Θ(g(n)), for some explicit (non-recursive) function g(n). For recursive inequalities, we prefer a tight solution; this is a function that would still satisfy the recurrence if all the inequalities were replaced with the corresponding equations. Again, exactly tight solutions may not exist, or may be too complex to be useful, in which case we seek either a looser bound or an asymptotic solution of the form O(g(n)) or Ω(g(n)). 2 The Ultimate Method: Guess and Confirm Ultimately, there is only one fail-safe method to solve any recurrence: Guess the answer, and then prove it correct by induction. Later sections of these notes describe techniques to generate guesses that are guaranteed to be correct, provided you use them correctly. But if you’re faced with a recurrence that doesn’t seem to fit any of these methods, or if you’ve forgotten how those techniques work, don’t despair! If you guess a closed-form solution and then try to verify your guess inductively, usually either the proof will succeed, in which case you’re done, or the proof will fail, in which case your failure will help you refine your guess. Where you get your initial guess is utterly irrelevant¹—from a classmate, from a textbook, on the web, from the answer to a different problem, scrawled on a bathroom wall in Siebel, included in a care package from your mom, dictated by the machine elves, whatever. If you can prove that the answer is correct, then it’s correct! 2.1 Tower of Hanoi The classical Tower of Hanoi problem gives us the recurrence T (n) = 2T (n − 1) + 1 with base case T (0) = 0. Just looking at the recurrence we can guess that T (n) is something like 2n . If we write out the first few values of T (n), we discover that they are each one less than a power of two. T (0) = 0, T (1) = 1, T (2) = 3, T (3) = 7, T (4) = 15, T (5) = 31, T (6) = 63, It looks like T (n) = 2n − 1 might be the right answer. Let’s check. T (0) = 0 = 20 − 1 Ø T (n) = 2T (n − 1) + 1 = 2(2n−1 − 1) + 1 [induction hypothesis] n =2 −1 Ø [algebra] ¹. . . except of course during exams, where you aren’t supposed to use any outside sources 2 ...,

Text from page-3

Algorithms Appendix II: Solving Recurrences [Fa’13] We were right! Hooray, we’re done! Another way we can guess the solution is by unrolling the recurrence, by substituting it into itself: T (n) = 2T (n − 1) + 1 = 2 (2T (n − 2) + 1) + 1 = 4T (n − 2) + 3 = 4 (2T (n − 3) + 1) + 3 = 8T (n − 3) + 7 = ··· It looks like unrolling the initial Hanoi recurrence k times, for any non-negative integer k, will give us the new recurrence T (n) = 2k T (n − k) + (2k − 1). Let’s prove this by induction: T (n) = 2T (n − 1) + 1 Ø [k = 0, by definition] T (n) = 2k−1 T (n − (k − 1)) + (2k−1 − 1)  = 2k−1 2T (n − k) + 1 + (2k−1 − 1) [inductive hypothesis] [initial recurrence for T (n − (k − 1))] = 2k T (n − k) + (2k − 1) Ø [algebra] Our guess was correct! In particular, unrolling the recurrence n times give us the recurrence T (n) = 2n T (0) + (2n − 1). Plugging in the base case T (0) = 0 give us the closed-form solution T (n) = 2n − 1. 2.2 Fibonacci numbers Let’s try a less trivial example: the Fibonacci numbers Fn = Fn−1 + Fn−2 with base cases F0 = 0 and F1 = 1. There is no obvious pattern in the first several values (aside from the recurrence itself), but we can reasonably guess that Fn is exponential in n. Let’s try to prove inductively that Fn ≤ α · c n for some constants a > 0 and c > 1 and see how far we get. Fn = Fn−1 + Fn−2 ≤ α · c n−1 + α · c n−2 [“induction hypothesis”] ≤ α · c n ??? The last inequality is satisfied if c np≥ c n−1 + c n−2 , or more simply, if c 2 − c − 1 ≥ 0. The smallest value of c that works is φ = (1 + 5)/2 ≈ 1.618034; the other root of the quadratic equation has smaller absolute value, so we can ignore it. So we have most of an inductive proof that Fn ≤ α · φ n for some constant α. All that we’re missing are the base cases, which (we can easily guess) must determine the value of the coefficient a. We quickly compute F0 0 = =0 0 φ 1 and F1 1 = ≈ 0.618034 > 0, 1 φ φ so the base cases of our induction proof are correct as long as α ≥ 1/φ. It follows that Fn ≤ φ n−1 for all n ≥ 0. What about a matching lower bound? Essentially the same inductive proof implies that Fn ≥ β · φ n for some constant β, but the only value of β that works for all n is the trivial β = 0! 3

Text from page-4

Algorithms Appendix II: Solving Recurrences [Fa’13] We could try to find some lower-order term that makes the base case non-trivial, but an easier approach is to recall that asymptotic Ω( ) bounds only have to work for sufficiently large n. So let’s ignore the trivial base case F0 = 0 and assume that F2 = 1 is a base case instead. Some more easy calculation gives us F2 1 1 = 2 ≈ 0.381966 < . 2 φ φ φ Thus, the new base cases of our induction proof are correct as long as β ≤ 1/φ 2 , which implies that Fn ≥ φ n−2 for all n ≥ 1. Putting the upper and lower bounds together, we obtain the tight asymptotic bound Fn = Θ(φ n ). It is possible to get a more exact solution by speculatively refining and conforming our current bounds, but it’s not easy. Fortunately, if we really need it, we can get an exact solution using the annihilator method, which we’ll see later in these notes. 2.3 Mergesort Mergesort is a classical recursive divide-and-conquer algorithm for sorting an array. The algorithm splits the array in half, recursively sorts the two halves, and then merges the two sorted subarrays into the final sorted array. Merge(A[1 .. n], m): i ← 1; j ← m + 1 for k ← 1 to n if j > n B[k] ← A[i]; else if i > m B[k] ← A[ j]; else if A[i] < A[ j] B[k] ← A[i]; else B[k] ← A[ j]; MergeSort(A[1 .. n]): if (n > 1) m ← bn/2c MergeSort(A[1 .. m]) MergeSort(A[m + 1 .. n]) Merge(A[1 .. n], m) i ← i+1 j ← j+1 i ← i+1 j ← j+1 for k ← 1 to n A[k] ← B[k] Let T (n) denote the worst-case running time of MergeSort when the input array has size n. The Merge subroutine clearly runs in Θ(n) time, so the function T (n) satisfies the following recurrence: ( Θ(1) if n = 1, T (n) =   T dn/2e + T bn/2c + Θ(n) otherwise. For now, let’s consider the special case where n is a power of 2; this assumption allows us to take the floors and ceilings out of the recurrence. (We’ll see how to deal with the floors and ceilings later; the short version is that they don’t matter.) Because the recurrence itself is given only asymptotically—in terms of Θ( ) expressions—we can’t hope for anything but an asymptotic solution. So we can safely simplify the recurrence further by removing the Θ’s; any asymptotic solution to the simplified recurrence will also satisfy the original recurrence. (This simplification is actually important for another reason; if we kept the asymptotic expressions, we might be tempted to simplify them inappropriately.) Our simplified recurrence now looks like this: ( 1 if n = 1, T (n) = 2T (n/2) + n otherwise. 4

Lecture Notes