Function optimization using
order gradient methods
Goal: Review gradient descent approaches.
A nice chapter on function optimization techniques:
Numerical Recipes in C, chapter 10
(2nd or 3rd edition, 2nd edition is electronically available for free
under Obsolete Versions):
Minimization or Maximization of Functions,
This material from any other numerical methods book is also fine.
software list 1,
conjugate gradient v2,
quasi-Newton/variable metric methods,
Reduced dimensionality second order methods.
Stanford MSandE 311;
U. Stuttgart: Toussaint
Optimization Methods for Large-Scale Machine Learning;
Identifying and attacking the saddle point problem in
high-dimensional non-convex optimization