16-745: Dynamic Optimization
Instructor: Chris Atkeson, cga at cmu
MW 3-4:20 NSH 3002
Events of Interest
Resources and Readings
Jan 10: Introduction to the course.
Jan 12: Formulating trajectory optimization as function optimization.
Examples of formulating a trajectory optimization problem
as a function optimization problem:
Case Studies In Trajectory Optimization: Trains, Planes, And Other
Robert J. Vanderbei
Example use of AMPL
A free trial version of AMPL is available from here.
AMPL is also available for remote use through the Neos Server.
Click on SNOPT/[AMPL Input] under Nonlinearly Constrained Optimization.
Example use of Matlab
Jan 17: MLK holiday.
Function optimization using
order gradient methods.
A nice chapter on function optimization techniques:
Numerical Recipes in C, chapter 10
(2nd or 3rd edition, 2nd edition is electronically available for free
under Obsolete Versions):
Minimization or Maximization of Functions,
This material from any other numerical methods book is also fine.
software list 1,
conjugate gradient v2,
quasi-Newton/variable metric methods, and
Jan 19: Ways to robustify function optimization:
Problems: local minima, discontinuities, redundant/rank deficient constraints,
bad scaling, no formulas for derivatives,
Techniques: Levenberg Marquardt,
scaling and preconditioning, regularize parameters, soft constraints,
line search, sparse methods,
Jan 24: Constraints.
Jan 24: Non-gradient optimization methods:
local unimodal sampling,
Nelder Mead/Simplex/Amoeba method,
fit surfaces (for example
Response Surface Methodology (RSM),
Memory-based Stochastic Optimization, and
Jan 26: Use of splines in trajectory optimization.
Cubic Hermite spline.
Need paper reference.
Jan 26: Sequential quadratic programming.
Witkin paper text
Witkin paper figures
Jan 26: Covariance Matrix Adaptation Evolution Strategy.
See also Hansen web page.
Example of use.
Jan 31 - Feb. 9: Policy optimization I: Use function optimization.
Known in machine learning/reinforcement learning as policy search, policy refinement, policy gradient, ...
Feb. 7: No class
Feb. 9-23: Dynamic Programming.
Linear Quadratic Regulator,
Differential Dynamic Programming,
Feb. 16: Anca Dragan CHOMP and goal sets.
Feb. 28: trajectory optimization based on integrating the dynamics:
calculus of variations,
Pontryagin's minimum principle,
multiple shooting methods,
Mar. 2: SeungJoon Lee Airfoil optimization.
Mar. 2: Model Predictive Control (MPC), (a.k.a. receding horizon control).
Mar. 7. 9: No class
Mar. 14: Robustness
Be robust to random disturbances, varying initial conditions, parametric
model error, high frequency unmodelled dynamics,
and model jumps (touchdown and liftoff during walking, for example).
Monte Carlo trajectory/policy optimization.
Mar. 16: Dual Control.
Information state DP.
Mar. 21: Uncertainty Propagation
Gaussian Propagation (like Kalman Filter),
Unscented (like Unscented Filter), Second Order Kalman Filter (See Kendrick).
Mar. 23: Local Approaches to Dual Control/Stochastic DDP
Information state trajectory optimization.
Stochastic Control for Economic Models,
David Kendrick, Second Edition 2002.
Mar. 23: Erhan Arisoy
Mar. 28: Modeling Techniques and Modeling Error
Mar. 30: High Frequency Unmodeled Dynamics
How model-based techniques can fail. Why policy X techniques also fail.
What reinforcement learning (RL) can learn from adaptive control theory.
Mar. 30: Kyle Strabala
Apr. 4: Monte-Carlo, DP, and DDP approaches to Multiple Models
Apr. 6: Learning From Demonstration
Apr. 11: X. Xinjilefu
Apr. 11: Optimizing similar tasks
Apr. 13: Feng Zhou
Apr. 13: A*-like algorithms
Apr. 18: Meta-optimization
Apr. 20: Jiuguang Wang
Apr. 25: Project presentations
Apr. 27: Project presentations