# Divide and Conquer to Multiply and Sort

PGSS Computer Science Core Slides

We begin looking at algorithms, sorted by the techniques they use.

The first is divide and conquer:

• Divide problem into smaller problems.
• Recursively solve these.
• Combine solutions to find answer.

### Sorting

Problem class SORT:

Input: an array a of integers.
Output: an array with all elements of a in increasing order.

example:

+----+----+----+----+----+----+----+----+
a = | 19 |  1 | 29 | 30 |  6 | 15 |  2 |  5 |
+----+----+----+----+----+----+----+----+

+----+----+----+----+----+----+----+----+
return |  1 |  2 |  5 |  6 | 15 | 19 | 29 | 30 |
+----+----+----+----+----+----+----+----+

### MergeSort

Algorithm Merge-Sort(a)
if a's length > 1 then
x <- first half of a
y <- second half of a
Merge-Sort(x)
Merge-Sort(y)
a <- Merge(x, y)
fi

example:

|  +----+----+----+----+----+----+----+----+
|  |  1 |  2 |  5 |  6 | 15 | 19 | 29 | 30 |
|  +----+----+----+----+----+----+----+----+
|
+----+----+----+----+----+----+----+----+
| 19 |  1 | 29 | 30 |  6 | 15 |  2 |  5 |
+----+----+----+----+----+----+----+----+
|           |
+----+----+----+----+ |           |  +----+----+----+----+
|  1 | 19 | 29 | 30 | |           |  |  2 |  5 |  6 | 15 |
+----+----+----+----+ |           |  +----+----+----+----+
|           |
+----+----+----+----+     +----+----+----+----+
| 19 |  1 | 29 | 30 |     |  6 | 15 |  2 |  5 |
+----+----+----+----+     +----+----+----+----+
|    |                          |   |
+----+----+ |    | +----+----+  +----+----+ |   | +----+----+
| 19 |  1 | |    | | 29 | 30 |  |  6 | 15 | |   | |  2 |  5 |
+----+----+ |    | +----+----+  +----+----+ |   | +----+----+
|    |                          |   |
+----+----+    +----+----+      +----+----+   +----+----+
| 19 |  1 |    | 29 | 30 |      |  6 | 15 |   |  2 |  5 |
+----+----+    +----+----+      +----+----+   +----+----+
/   \         /     \         /      \        |      \
+----+  +----+  +----+  +----+  +----+  +----+   +----+  +----+
| 19 |  |  1 |  | 29 |  | 30 |  |  6 |  | 15 |   |  2 |  |  5 |
+----+  +----+  +----+  +----+  +----+  +----+   +----+  +----+

### Merging

Algorithm Merge(x, y)
x_pos <- 0, y_pos <- 0, a_pos <- 0
while x_pos < x.length and y_pos < y.length do
if x[x_pos] < y[y_pos] then
a[a_pos] <- x[x_pos]
x_pos <- x_pos + 1
else
a[a_pos] <- y[y_pos]
y_pos <- y_pos + 1
fi
a_pos <- a_pos + 1
od
append x[x_pos...x.length] to a
append y[y_pos...y.length] to a
return a

example:

+----+----+----+----+
|  1 | 19 | 29 | 30 |
+----+----+----+----+\    +----+----+----+----+----+----+----+----+
>-->|  1 |  2 |  5 |  6 | 15 | 19 | 29 | 30 |
+----+----+----+----+/    +----+----+----+----+----+----+----+----+
|  2 |  5 |  6 | 15 |
+----+----+----+----+

### How Much Time?

Write a recurrence.

T(n) = 2 T(n / 2) + O(n)

The Master Theorem (p 73) solves this. For T(n) = a T(n / b) + f(n):

• if f(n) = O(n^(log_b a - eps)) for some eps > 0, then T(n) = O(n^(log_b a))
• if f(n) = O(n^(log_b a)), then T(n) = O(n^(log_b a) log n)

For Merge-Sort, then, where a = 2 and b = 2, T(n) = O(n log n).

### Multiplication

Problem class MULTIPLICATION

Input: two n-digit integers a and b
Output: product of a and b

example:

1980 = a
x    2315 = b
---------
9900
1980
5940
+ 3960
---------
4573700 = a x b
This is the algorithm you learned in grade school. Notice it takes O(n^2) time.

### Dividing the Problem

We divide each integer into two halves.

aL = 19  |  80 = aR
|
bL = 23  |  15 = bR

aL          aR
x          bL          bR
-----------------------------
aL bR       aR bR
+ aL bL      aR bL
-----------------------------
aL bL   aL bR + aR bL   aR bR
So our algorithm is to compute aL bL, aL bR, aR bL, and aR bR, and add.
T(n) <= 4 T(n / 2) + O(n)

T(n) = O(n^2)

### Being Clever

We can actually get away with just three multiplications!

x1 = aL bL
x2 = aR bR
x3 = (aL + aR) (bL + bR)

aL          aR
x          bL          bR
-----------------------------
aL bL   aL bR + aR bL   aR bR
x1     x3 - x1 - x2     x2
Now our algorithm is to compute x1, x2, and x3, find x3 - x1 - x2, and add.
T(n) <= 3 T(n / 2) + O(n)

T(n) = O(n^(log_2 3)) = O(n^1.59)

### An Example

IN Multiply(1980, 2315)
19 80
23 15
IN Multiply(19, 23)
1 9
2 3
IN Multiply(1, 2)
return 2
IN Multiply(9, 3)
return 27
IN Multiply(10, 5)
1 0
0 5
IN Multiply(1, 0)
return 0
IN Multiply(0, 5)
return 0
IN Multiply(1, 5)
return 5
5 - 0 - 0 = 5
0
5
0
---
50
return 50
50 - 27 - 2 = 21
2
21
27
---
437
return 437
IN Multiply(80, 15)
8 0
1 5
IN Multiply(8, 1)
return 8
IN Multiply(0, 5)
return 0
IN Multiply(8, 6)
return 48
48 - 8 - 0 = 40
8
40
0
----
1200
return 1200
IN Multiply(99, 38)
9 9
3 8
IN Multiply(9, 3)
return 27
IN Multiply(9, 8)
return 72
IN Multiply(18, 11)
1 8
1 1
IN Multiply(1, 1)
return 1
IN Multiply(8, 1)
return 8
IN Multiply(9, 2)
return 18
18 - 8 - 1 = 9
1
9
8
---
198
return 198
198 - 27 - 72 = 99
27
99
72
----
3762
return 3762
3762 - 437 - 1200 = 2125

437
2125
1200
-------
4583700
return 4583700

### How Do Multiplication Algorithms Compare in Practice?

|
|
600+
|                                       -
i  500+                            School
m     |
e     |                                   -
400+
(ms)   |
|                               -
300+
|                           -
|
200+                       -               x
|                                   x
|                   -       x   x
100+               -       x     Divide and
|           -   *   x           Conquer
|   *   *   x
0+-------+-------+-------+-------+-------+
0      200     400     600     800    1000

n

times are the average of 50 random trials on a Sun SparcStation 4