×
Hard work beats talent when talent doesn’t work hard.

# Note for Design And Analysis Of Algorithm - DAA by Aman Kumar

• Design And Analysis Of Algorithm - DAA
• Note
• Dr. A PJ Abdul Kalam Tech University Lucknow - AKTU
• Computer Science Engineering
• B.Tech
• 15 Views
Page-1

#### Note for Design And Analysis Of Algorithm - DAA by Aman Kumar

Topic: / 63

0 User(s)

#### Text from page-1

Algorithms Appendix II: Solving Recurrences [Fa’13] Change is certain. Peace is followed by disturbances; departure of evil men by their return. Such recurrences should not constitute occasions for sadness but realities for awareness, so that one may be happy in the interim. — I Ching [The Book of Changes] (c. 1100 BC) To endure the idea of the recurrence one needs: freedom from morality; new means against the fact of pain (pain conceived as a tool, as the father of pleasure; there is no cumulative consciousness of displeasure); the enjoyment of all kinds of uncertainty, experimentalism, as a counterweight to this extreme fatalism; abolition of the concept of necessity; abolition of the “will”; abolition of “knowledge-in-itself.” — Friedrich Nietzsche The Will to Power (1884) [translated by Walter Kaufmann] Wil Wheaton: Embrace the dark side! Sheldon: That’s not even from your franchise! — “The Wheaton Recurrence”, Bing Bang Theory, April 12, 2010 Solving Recurrences 1 Introduction A recurrence is a recursive description of a function, or in other words, a description of a function in terms of itself. Like all recursive structures, a recurrence consists of one or more base cases and one or more recursive cases. Each of these cases is an equation or inequality, with some function value f (n) on the left side. The base cases give explicit values for a (typically finite, typically small) subset of the possible values of n. The recursive cases relate the function value f (n) to function value f (k) for one or more integers k < n; typically, each recursive case applies to an infinite number of possible values of n. For example, the following recurrence (written in two different but standard ways) describes the identity function f (n) = n: ¨ f (0) = 0 0 if n = 0 f (n) = f (n) = f (n − 1) + 1 for all n > 0 f (n − 1) + 1 otherwise In both presentations, the first line is the only base case, and the second line is the only recursive case. The same function can satisfy many different recurrences; for example, both of the following recurrences also describe the identity function:     0 if n = 0 if n = 0  0 f (n) = 1 f (n) = 2 · f (n/2) if n = 1 if n is even and n > 0    f (bn/2c) + f (dn/2e) otherwise  f (n − 1) + 1 if n is odd We say that a particular function satisfies a recurrence, or is the solution to a recurrence, if each of the statements in the recurrence is true. Most recurrences—at least, those that we will encounter in this class—have a solution; moreover, if every case of the recurrence is an equation, that solution is unique. Specifically, if we transform the recursive formula into a recursive algorithm, the solution to the recurrence is the function computed by that algorithm! © Copyright 2014 Jeff Erickson. This work is licensed under a Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/4.0/). Free distribution is strongly encouraged; commercial distribution is expressly forbidden. See http://www.cs.uiuc.edu/~jeffe/teaching/algorithms/ for the most recent revision. 1

#### Text from page-3

Algorithms Appendix II: Solving Recurrences [Fa’13] We were right! Hooray, we’re done! Another way we can guess the solution is by unrolling the recurrence, by substituting it into itself: T (n) = 2T (n − 1) + 1 = 2 (2T (n − 2) + 1) + 1 = 4T (n − 2) + 3 = 4 (2T (n − 3) + 1) + 3 = 8T (n − 3) + 7 = ··· It looks like unrolling the initial Hanoi recurrence k times, for any non-negative integer k, will give us the new recurrence T (n) = 2k T (n − k) + (2k − 1). Let’s prove this by induction: T (n) = 2T (n − 1) + 1 Ø [k = 0, by definition] T (n) = 2k−1 T (n − (k − 1)) + (2k−1 − 1)  = 2k−1 2T (n − k) + 1 + (2k−1 − 1) [inductive hypothesis] [initial recurrence for T (n − (k − 1))] = 2k T (n − k) + (2k − 1) Ø [algebra] Our guess was correct! In particular, unrolling the recurrence n times give us the recurrence T (n) = 2n T (0) + (2n − 1). Plugging in the base case T (0) = 0 give us the closed-form solution T (n) = 2n − 1. 2.2 Fibonacci numbers Let’s try a less trivial example: the Fibonacci numbers Fn = Fn−1 + Fn−2 with base cases F0 = 0 and F1 = 1. There is no obvious pattern in the first several values (aside from the recurrence itself), but we can reasonably guess that Fn is exponential in n. Let’s try to prove inductively that Fn ≤ α · c n for some constants a > 0 and c > 1 and see how far we get. Fn = Fn−1 + Fn−2 ≤ α · c n−1 + α · c n−2 [“induction hypothesis”] ≤ α · c n ??? The last inequality is satisfied if c np≥ c n−1 + c n−2 , or more simply, if c 2 − c − 1 ≥ 0. The smallest value of c that works is φ = (1 + 5)/2 ≈ 1.618034; the other root of the quadratic equation has smaller absolute value, so we can ignore it. So we have most of an inductive proof that Fn ≤ α · φ n for some constant α. All that we’re missing are the base cases, which (we can easily guess) must determine the value of the coefficient a. We quickly compute F0 0 = =0 0 φ 1 and F1 1 = ≈ 0.618034 > 0, 1 φ φ so the base cases of our induction proof are correct as long as α ≥ 1/φ. It follows that Fn ≤ φ n−1 for all n ≥ 0. What about a matching lower bound? Essentially the same inductive proof implies that Fn ≥ β · φ n for some constant β, but the only value of β that works for all n is the trivial β = 0! 3

#### Text from page-4

Algorithms Appendix II: Solving Recurrences [Fa’13] We could try to find some lower-order term that makes the base case non-trivial, but an easier approach is to recall that asymptotic Ω( ) bounds only have to work for sufficiently large n. So let’s ignore the trivial base case F0 = 0 and assume that F2 = 1 is a base case instead. Some more easy calculation gives us F2 1 1 = 2 ≈ 0.381966 < . 2 φ φ φ Thus, the new base cases of our induction proof are correct as long as β ≤ 1/φ 2 , which implies that Fn ≥ φ n−2 for all n ≥ 1. Putting the upper and lower bounds together, we obtain the tight asymptotic bound Fn = Θ(φ n ). It is possible to get a more exact solution by speculatively refining and conforming our current bounds, but it’s not easy. Fortunately, if we really need it, we can get an exact solution using the annihilator method, which we’ll see later in these notes. 2.3 Mergesort Mergesort is a classical recursive divide-and-conquer algorithm for sorting an array. The algorithm splits the array in half, recursively sorts the two halves, and then merges the two sorted subarrays into the final sorted array. Merge(A[1 .. n], m): i ← 1; j ← m + 1 for k ← 1 to n if j > n B[k] ← A[i]; else if i > m B[k] ← A[ j]; else if A[i] < A[ j] B[k] ← A[i]; else B[k] ← A[ j]; MergeSort(A[1 .. n]): if (n > 1) m ← bn/2c MergeSort(A[1 .. m]) MergeSort(A[m + 1 .. n]) Merge(A[1 .. n], m) i ← i+1 j ← j+1 i ← i+1 j ← j+1 for k ← 1 to n A[k] ← B[k] Let T (n) denote the worst-case running time of MergeSort when the input array has size n. The Merge subroutine clearly runs in Θ(n) time, so the function T (n) satisfies the following recurrence: ( Θ(1) if n = 1, T (n) =   T dn/2e + T bn/2c + Θ(n) otherwise. For now, let’s consider the special case where n is a power of 2; this assumption allows us to take the floors and ceilings out of the recurrence. (We’ll see how to deal with the floors and ceilings later; the short version is that they don’t matter.) Because the recurrence itself is given only asymptotically—in terms of Θ( ) expressions—we can’t hope for anything but an asymptotic solution. So we can safely simplify the recurrence further by removing the Θ’s; any asymptotic solution to the simplified recurrence will also satisfy the original recurrence. (This simplification is actually important for another reason; if we kept the asymptotic expressions, we might be tempted to simplify them inappropriately.) Our simplified recurrence now looks like this: ( 1 if n = 1, T (n) = 2T (n/2) + n otherwise. 4