Dynamic Programming

PGSS Computer Science Core Slides

Our final algorithmic technique is dynamic programming.

Alice: Looking at problems upside-down can help!
Bob: (But be careful with your hat!)

Dynamic Programming

Two steps:

1. Find a recursive solution that involves solving the same problems many times.
2. Calculate bottom up to avoid recalculation.

Fibonacci Numbers

```1 1 2 3 5 8 13 21 34 55 89...
```
(Each number is the sum of the previous two.)

```Algorithm Fibonacci(n)
if n <= 1 then
return 1
else
return Fibonacci(n - 1) + Fibonacci(n - 2)
fi
```
```      __5__
/     \
4       3
/ \     / \
3   2   2   1
/ \ / \ / \
2 1 1 0 1 0
/ \
1 0
```

We can approximate the running time with the number of additions, which we can write as a recurrence.

```adds(0) = 0
||
\/
~= 0.7236 * 1.618^n - 1
```

A real timing:

```to compute      takes
Fibonacci(40)  75.22 seconds
Fibonacci(70)   4.43 years
```

The Dynamic Programming Approach

Dynamic programming suggests we start at the bottom and work up.

```Algorithm Fast-Fibonacci(n)
fib[0] <- 1, fib[1] <- 1
for i <- 2 to n do
fib[i] <- fib[i - 2] + fib[i - 1]
od
return fib[n]
```

The number of additions is only n - 1!

```to compute      took           now takes
Fibonacci(40)  75.22 seconds   2 microseconds
Fibonacci(70)   4.43 years     3 microseconds
```

A New Problem

Problem class MAKE-CHANGE:

Input: coin denominations d[0], d[1],..., d[n - 1], an amount a
Output: the number of coins needed to total a exactly

example:

input: the 1-cent, 3-cent, and 4-cent denominations, the amount a=6
output: 2 (we can use two 3-cent pieces, but there is no one-coin solution)

Formulating a Solution

1. Think of a recursive solution.

```Make-Change(a) = 1 +    min     Make-Change(a - d[i])
0 <= i < n
```
```          ___6___
/    \  \
5__    3  2
/ \ \  / \ |
_4   2 1 2 0 1
/ |\  | | |   |
3  1 0 1 0 1   0
/ \ |   |   |
2  0 0   0   0
/ \
2 0
|
1
|
0
```

2. Compute bottom up.

```Algorithm Make-Change(amt)
coins[0] <- 0
for a <- 1 to amt do
coins[a] <- infinity
for i <- 0 to n - 1 do
if d[i] <= a and 1 + coins[a - d[i]] < coins[a] then
coins[a] <- 1 + coins[a - d[i]]
fi
od
od
returns coins[amt]
```

This takes amt * n iterations.

Homework 2

Homework 2 could be approach with dynamic programming.

1. Think of a recursive solution.

```Algorithm Calc-Spreadsheet(box)
if left side of box's formula is constant then
left <- left hand constant
else
fi

if right side of box's formula is constant then
right <- right hand constant
else
fi

if left and right are defined then
return operation on left and right
else
return undefined
fi
```

2. Compute bottom up.

```Algorithm Calc-Spreadsheet
for each box i do result[i] <- undefined od
while changes are still being made do
for each box i do
if result[i] = undefined then
left <- current left side
right <- current right side
if left != undefined and right != undefined then
result[i] <- operation on left and right
fi
fi
od
od
```

Graph Paths

Problem class ALL-PAIRS-PATHS:

Input: graph (V, E), edge distances d: E->R+.
Output: length p[s,t] of shortest path from s to t, for all pairs of vertices.

example:

```    1
2---4
3 |\_ | 1
| 6\|
1---3
2
```
output:
```        to
1 2 3 4
+-------
1|0 3 2 3
from 2|3 0 2 1
3|2 2 0 1
4|3 1 1 0
```

The Recursive Solution

We define the following quantity:

``` (k)    length of shortest path between s and t only passing through
p     = vertices 1, 2,..., k in between.
s, t
```
We can calculate p[s,t]^(k) by recursive calls to compute p[u,v]^(k-1):
``` (k)           (k - 1)   (k - 1)    (k - 1)
p     = min { p       , p        + p        }
s, t          s, t      s, k       k, t
```
This is because the shortest path only through 1...k either passes through k or it doesn't. If the path doesn't, the first term will be the minimum. If it does, then the path will go from s to k only through 1...k-1 and from k to t only through 1...k-1, and so the second term will hold. The path will never go through k more than once, since then we could remove the loop involving k.

The Bottom-Up Solution

``` (0)       d(s, t)   if (s, t) is an edge in the graph
p     <- {
s, t      infinity  otherwise
for k <- 1 to n do
for s <- 1 to n do
for t <- 1 to n do
(k)           (k - 1)   (k - 1)    (k - 1)
p     = min { p       , p        + p        }
s, t          s, t      s, k       k, t
od
od
od
(n)
return p
```

Example: (@ denotes infinity here.)

```    1
2---4
3 |\_ | 1
| 6\|
1---3
2

0 3 2 @
(0)  3 0 6 1
p   : 2 6 0 1
@ 1 1 0

0 3 2 @
(1)  3 0 5 1
p   : 2 5 0 1
@ 1 1 0

0 3 2 4
(2)  3 0 5 1
p   : 2 5 0 1
4 1 1 0

0 3 2 3
(3)  3 0 5 1
p   : 2 5 0 1
3 1 1 0

0 3 2 3
(4)  3 0 2 1
p   : 2 2 0 1
3 1 1 0
```