Divide and Conquer to Multiply and Sort

We begin looking at algorithms, sorted by the techniques they use.

The first is divide and conquer:

Divide problem into smaller problems.
Recursively solve these.
Combine solutions to find answer.

Sorting

Problem class SORT:

Input: an array a of integers.
Output: an array with all elements of a in increasing order.

example:

        +----+----+----+----+----+----+----+----+
    a = | 19 |  1 | 29 | 30 |  6 | 15 |  2 |  5 |
        +----+----+----+----+----+----+----+----+

        +----+----+----+----+----+----+----+----+
 return |  1 |  2 |  5 |  6 | 15 | 19 | 29 | 30 |
        +----+----+----+----+----+----+----+----+

MergeSort

Algorithm Merge-Sort(a)
if a's length > 1 then
  x <- first half of a
  y <- second half of a
  Merge-Sort(x)
  Merge-Sort(y)
  a <- Merge(x, y)
fi

example:

                           |  +----+----+----+----+----+----+----+----+
                           |  |  1 |  2 |  5 |  6 | 15 | 19 | 29 | 30 |
                           |  +----+----+----+----+----+----+----+----+
                           |
        +----+----+----+----+----+----+----+----+
        | 19 |  1 | 29 | 30 |  6 | 15 |  2 |  5 |
        +----+----+----+----+----+----+----+----+
                      |           |
+----+----+----+----+ |           |  +----+----+----+----+
|  1 | 19 | 29 | 30 | |           |  |  2 |  5 |  6 | 15 |
+----+----+----+----+ |           |  +----+----+----+----+
                      |           |
     +----+----+----+----+     +----+----+----+----+
     | 19 |  1 | 29 | 30 |     |  6 | 15 |  2 |  5 |
     +----+----+----+----+     +----+----+----+----+
            |    |                          |   |            
+----+----+ |    | +----+----+  +----+----+ |   | +----+----+
| 19 |  1 | |    | | 29 | 30 |  |  6 | 15 | |   | |  2 |  5 |
+----+----+ |    | +----+----+  +----+----+ |   | +----+----+
            |    |                          |   |            
  +----+----+    +----+----+      +----+----+   +----+----+
  | 19 |  1 |    | 29 | 30 |      |  6 | 15 |   |  2 |  5 |
  +----+----+    +----+----+      +----+----+   +----+----+
     /   \         /     \         /      \        |      \
+----+  +----+  +----+  +----+  +----+  +----+   +----+  +----+
| 19 |  |  1 |  | 29 |  | 30 |  |  6 |  | 15 |   |  2 |  |  5 |
+----+  +----+  +----+  +----+  +----+  +----+   +----+  +----+

Merging

Algorithm Merge(x, y)
x_pos <- 0, y_pos <- 0, a_pos <- 0
while x_pos < x.length and y_pos < y.length do
  if x[x_pos] < y[y_pos] then
    a[a_pos] <- x[x_pos]
    x_pos <- x_pos + 1
  else
    a[a_pos] <- y[y_pos]
    y_pos <- y_pos + 1
  fi
  a_pos <- a_pos + 1
od
append x[x_pos...x.length] to a
append y[y_pos...y.length] to a
return a

example:

 +----+----+----+----+
 |  1 | 19 | 29 | 30 |
 +----+----+----+----+\    +----+----+----+----+----+----+----+----+
                       >-->|  1 |  2 |  5 |  6 | 15 | 19 | 29 | 30 |
 +----+----+----+----+/    +----+----+----+----+----+----+----+----+
 |  2 |  5 |  6 | 15 |
 +----+----+----+----+


How Much Time?

Write a recurrence.
   T(n) = 2 T(n / 2) + O(n)


The Master Theorem (p 73) solves this. For T(n) = a T(n / b) + f(n):


 if f(n) = O(n^(log_b a - eps)) for some eps > 0, then T(n) =
O(n^(log_b a))

 if f(n) = O(n^(log_b a)), then T(n) = O(n^(log_b a) log n)



For Merge-Sort, then, where a = 2 and b = 2, T(n) = O(n log n).

Multiplication

Problem class MULTIPLICATION

Input: two n-digit integers a and b
Output: product of a and b


example:
              1980 = a
         x    2315 = b
         ---------
              9900
             1980
            5940
         + 3960
         ---------
           4573700 = a x b

This is the algorithm you learned in grade school. Notice it takes
O(n^2) time.

Dividing the Problem

We divide each integer into two halves.
          aL = 19  |  80 = aR
                   |
          bL = 23  |  15 = bR


               aL          aR
    x          bL          bR
  -----------------------------
             aL bR       aR bR
+ aL bL      aR bL
  -----------------------------
  aL bL   aL bR + aR bL   aR bR

So our algorithm is to compute aL bL, aL bR, aR bL, and aR bR, and add.
    T(n) <= 4 T(n / 2) + O(n)

    T(n) = O(n^2)


Being Clever

We can actually get away with just three
multiplications!
     x1 = aL bL
     x2 = aR bR
     x3 = (aL + aR) (bL + bR)

               aL          aR
    x          bL          bR
  -----------------------------
  aL bL   aL bR + aR bL   aR bR
    x1     x3 - x1 - x2     x2

Now our algorithm is to compute x1, x2, and x3, find x3 - x1 - x2, and
add.
     T(n) <= 3 T(n / 2) + O(n)

     T(n) = O(n^(log_2 3)) = O(n^1.59)


An Example

 IN Multiply(1980, 2315)
  19 80
  23 15
     IN Multiply(19, 23)
      1 9
      2 3
         IN Multiply(1, 2)
         return 2
         IN Multiply(9, 3)
         return 27
         IN Multiply(10, 5)
          1 0
          0 5
             IN Multiply(1, 0)
             return 0
             IN Multiply(0, 5)
             return 0
             IN Multiply(1, 5)
             return 5
          5 - 0 - 0 = 5
          0
           5
            0
          ---
           50
         return 50
      50 - 27 - 2 = 21
      2
      21
       27
      ---
      437
     return 437
     IN Multiply(80, 15)
      8 0
      1 5
         IN Multiply(8, 1)
         return 8
         IN Multiply(0, 5)
         return 0
         IN Multiply(8, 6)
         return 48
      48 - 8 - 0 = 40
       8
       40
         0
      ----
      1200
     return 1200
     IN Multiply(99, 38)
      9 9
      3 8
         IN Multiply(9, 3)
         return 27
         IN Multiply(9, 8)
         return 72
         IN Multiply(18, 11)
          1 8
          1 1
             IN Multiply(1, 1)
             return 1
             IN Multiply(8, 1)
             return 8
             IN Multiply(9, 2)
             return 18
          18 - 8 - 1 = 9
          1
           9
            8
          ---
          198
         return 198
       198 - 27 - 72 = 99
       27
        99
         72
       ----
       3762
     return 3762
 3762 - 437 - 1200 = 2125

 437
  2125
    1200
 -------
 4583700
return 4583700


How Do Multiplication Algorithms Compare in Practice?

       |
       |
    600+
       |                                       -
 t     |                             Grade
 i  500+                            School
 m     |
 e     |                                   -
    400+
(ms)   |
       |                               -
    300+
       |                           -
       |
    200+                       -               x
       |                                   x
       |                   -       x   x
    100+               -       x     Divide and
       |           -   *   x           Conquer
       |   *   *   x
      0+-------+-------+-------+-------+-------+
       0      200     400     600     800    1000

                          n

times are the average of 50 random trials on a Sun SparcStation 4