dlr::optimization::OptimizerBFGS< Functor > Class Template Reference

OptimizerBFGS implements the Quasi-Newton method of Broyden, Fletcher, Goldfarb, and Shanno, as described in [1] (and possibly in [2]). More...

#include <optimizerBFGS.h>

Inheritance diagram for dlr::optimization::OptimizerBFGS< Functor >:

[legend]Collaboration diagram for dlr::optimization::OptimizerBFGS< Functor >:

[legend]List of all members.


Public Types
typedef Functor::argument_type	argument_type
	This is the Type of the objective function argument.
typedef Functor::result_type	result_type
	This is the Type of the objective function return value.
Public Member Functions
	OptimizerBFGS ()
	The default constructor sets parameters to reasonable values for functions which take values and arguments in the "normal" range of 0 to 100 or so.
	OptimizerBFGS (const Functor &functor)
	This constructor specifies the specific Functor instance to use.
	OptimizerBFGS (const OptimizerBFGS &source)
	Copy constructor.
virtual	~OptimizerBFGS ()
	The destructor destroys the class instance and deallocates any associated storage.
virtual std::vector< size_t >	getNumberOfFunctionCalls ()
	This method returns the number of function calls required to complete the previous minimization.
virtual std::vector< size_t >	getNumberOfGradientCalls ()
	This method returns the number of gradient calls required to complete the previous minimization.
virtual std::vector< size_t >	getNumberOfIterations ()
	This method returns the number of iterations required to complete the previous minimization.
virtual void	setParameters (size_t iterationLimit=500, size_t numberOfRestarts=1, double argumentTolerance=1.2E-7, double gradientTolerance=0.00001, double lineSearchAlpha=1.0E-4, double lineSearchArgumentTolerance=1.0e-7, double numericEpsilon=3.0E-8, double maximumStepMagnitudeFactor=100.0)
	This method sets minimization parameters.
virtual void	setIterationLimit (size_t iterationLimit)
	This method sets the optimization parameter controlling the maximum number of iterations, without affecting any other optimization parameters.
virtual void	setNumberOfRestarts (size_t numberOfRestarts)
	This method sets the optimization parameter controlling the number of restarts, without affecting any other optimization parameters.
virtual void	setStartPoint (const typename Functor::argument_type &startPoint)
	This method sets the initial conditions for the minimization.
virtual void	setVerbosity (int verbosity)
	This method sets the amount of text printed to the standard output during the optimization.
virtual OptimizerBFGS &	operator= (const OptimizerBFGS &source)
	Assignment operator.
Functor	objectiveFunction ()
	This method returns a copy of the Functor instance used for optimization.
result_type	optimalValue ()
	This method finds the optimum of the current Functor, if necessary, and returns the Functor value at that point.
argument_type	optimum ()
	This method finds the optimum of the current Functor, if necessary, and returns the Functor argument which produces that optimum.
void	setObjectiveFunction (const Functor &functor)
	This method specifies the Functor instance to use for the optimization.
Protected Member Functions
double	gradientConvergenceMetric (const argument_type &theta, const result_type &value, const argument_type &gradient)
	This protected member function is used to asses whether the algorithm has reached convergence.
virtual std::pair< typename Functor::argument_type, typename Functor::result_type >	run ()
	Perform the optimization.
Triple< typename Functor::argument_type, typename Functor::result_type, typename Functor::argument_type >	doBfgs (const argument_type &theta, const result_type &startValue, const argument_type &startGradient, size_t &numberOfFunctionCalls, size_t &numberOfGradientCalls, size_t &numberOfIterations)
	Perform one complete BFGS minimization, starting from the specified position.
virtual void	setOptimum (const typename Functor::argument_type &optimum, const typename Functor::result_type &optimalValue, bool needsFurtherOptimization)
	This protected member function provides a way for subclasses to communicate intermediate optimization results outside of the normal "return value of this->run()" method.
Protected Attributes
double	m_argumentTolerance
size_t	m_iterationLimit
double	m_gradientTolerance
double	m_lineSearchAlpha
double	m_lineSearchArgumentTolerance
double	m_maximumStepMagnitudeFactor
size_t	m_numberOfRestarts
double	m_numericEpsilon
OptimizerLineSearch< Functor >	m_optimizerLineSearch
argument_type	m_startPoint
std::vector< size_t >	m_functionCallCount
std::vector< size_t >	m_gradientCallCount
std::vector< size_t >	m_iterationCount
Functor	m_functor
	m_functor->operator()() should compute the objective function.
bool	m_needsOptimization
	Set to false if m_optimum contains a valid optimum, true otherwise.
argument_type	m_optimum
	Caches the result of the most recent optimization.
result_type	m_optimalValue
	Caches the result of the most recent optimization.

Detailed Description

template<class Functor>
class dlr::optimization::OptimizerBFGS< Functor >

OptimizerBFGS implements the Quasi-Newton method of Broyden, Fletcher, Goldfarb, and Shanno, as described in [1] (and possibly in [2]).

This algorithm seeks the parameter value which minimizes the objective function. The template parameter (Functor) defines the type to use as the objective function of the minimization, and must support the GradientFunction interface.

[1] W. H. Press et al., Numerical Recipes in C The Art of Scientific Computing, Cambridge University Press, 1988.

[2] C. G. Broyden, R. Fletcher, M. Goldfarb and Shanno, The convergence of a class of double rank minimization algorithms, J. Inst. Math. Appl., 6:222-231, 1970.

Definition at line 44 of file optimizerBFGS.h.

Member Typedef Documentation

template<class Functor>

typedef Functor::argument_type dlr::optimization::OptimizerBFGS< Functor >::argument_type

This is the Type of the objective function argument.

Reimplemented from dlr::optimization::Optimizer< Functor >.

Definition at line 49 of file optimizerBFGS.h.

template<class Functor>

typedef Functor::result_type dlr::optimization::OptimizerBFGS< Functor >::result_type

This is the Type of the objective function return value.

Reimplemented from dlr::optimization::Optimizer< Functor >.

Definition at line 50 of file optimizerBFGS.h.

Constructor & Destructor Documentation

template<class Functor>

dlr::optimization::OptimizerBFGS< Functor >::OptimizerBFGS ( ) [inline]

The default constructor sets parameters to reasonable values for functions which take values and arguments in the "normal" range of 0 to 100 or so.

Definition at line 363 of file optimizerBFGS.h.

References dlr::optimization::OptimizerBFGS< Functor >::setParameters().

template<class Functor>

dlr::optimization::OptimizerBFGS< Functor >::OptimizerBFGS ( const Functor & functor ) [inline, explicit]

This constructor specifies the specific Functor instance to use.

Using this constructor exclusively avoids the danger of calling optimalValue() or optimum() before a Functor instance has been specified.

Parameters:

functor

A copy of this argument will be stored internally for use in optimization.

Definition at line 384 of file optimizerBFGS.h.

References dlr::optimization::OptimizerBFGS< Functor >::setParameters().

template<class Functor>

dlr::optimization::OptimizerBFGS< Functor >::OptimizerBFGS ( const OptimizerBFGS< Functor > & source ) [inline]

Copy constructor.

This constructor simply copies the source argument.

Parameters:

source

The OptimizerBFGS instance to be copied.

Definition at line 405 of file optimizerBFGS.h.

References dlr::optimization::copyArgumentType(), and dlr::optimization::OptimizerBFGS< Functor >::m_startPoint.

template<class Functor>

dlr::optimization::OptimizerBFGS< Functor >::~OptimizerBFGS ( ) [inline, virtual]

The destructor destroys the class instance and deallocates any associated storage.

Definition at line 426 of file optimizerBFGS.h.

Member Function Documentation

template<class Functor>

virtual std::vector<size_t> dlr::optimization::OptimizerBFGS< Functor >::getNumberOfFunctionCalls ( ) [inline, virtual]

This method returns the number of function calls required to complete the previous minimization.

If the minimization parameter "restarts" is 0, there will be only one element in the returned vector. If restarts is greater than 0, the first element of the return value will reflect the number of function calls in the initial minimization, and subsequent numbers will reflect the number of function calls in the following restarted minimizations. If a valid minimization has not been performed since the last update to startPoint, parameters, etc., then the return value will be an empty vector.

Returns:: a vector of function call counts.

Definition at line 100 of file optimizerBFGS.h.

References dlr::optimization::OptimizerBFGS< Functor >::m_functionCallCount.

template<class Functor>

virtual std::vector<size_t> dlr::optimization::OptimizerBFGS< Functor >::getNumberOfGradientCalls ( ) [inline, virtual]

This method returns the number of gradient calls required to complete the previous minimization.

If the minimization parameter "restarts" is 0, there will be only one element in the returned vector. If restarts is greater than 0, the first element of the return value will reflect the number of gradient calls in the initial minimization, and subsequent numbers will reflect the number of gradient calls in the following restarted minimizations. If a valid minimization has not been performed since the last update to startPoint, parameters, etc., then the return value will be an empty vector.

Returns:: a vector of gradient call counts.

Definition at line 117 of file optimizerBFGS.h.

References dlr::optimization::OptimizerBFGS< Functor >::m_gradientCallCount.

template<class Functor>

virtual std::vector<size_t> dlr::optimization::OptimizerBFGS< Functor >::getNumberOfIterations ( ) [inline, virtual]

This method returns the number of iterations required to complete the previous minimization.

If the minimization parameter "restarts" is 0, there will be only one element in the returned vector. If restarts is greater than 0, the first element of the return value will reflect the number of iterations in the initial minimization, and subsequent numbers will reflect the number of iterations in the following restarted minimizations. If a valid minimization has not been performed since the last update to startPoint, parameters, etc., then the return value will be an empty vector.

Returns:: a vector of iteration counts.

Definition at line 134 of file optimizerBFGS.h.

References dlr::optimization::OptimizerBFGS< Functor >::m_iterationCount.

template<class Functor>

void dlr::optimization::OptimizerBFGS< Functor >::setParameters	(	size_t	iterationLimit = `500`,
		size_t	numberOfRestarts = `1`,
		double	argumentTolerance = `1.2E-7`,
		double	gradientTolerance = `0.00001`,
		double	lineSearchAlpha = `1.0E-4`,
		double	lineSearchArgumentTolerance = `1.0e-7`,
		double	numericEpsilon = `3.0E-8`,
		double	maximumStepMagnitudeFactor = `100.0`
	)			`[inline, virtual]`

This method sets minimization parameters.

Default values are reasonable for functions which take values and arguments in the "normal" range of 0 to 100 or so.

Parameters:

	iterationLimit	Each minimization will terminate after this many iterations.
	numberOfRestarts	Following successful termination, the minimization will be re-run this many times to refine the result accuracy. Generally you should leave this at its default value.
	gradientTolerance	Iteration will terminate when the magnitude of the gradient times the magnitude of the parameter vector becomes smaller than the function value by this factor.
	argumentTolerance	Iteration will terminate when a minimization step moves, along every axis, a distance less than this factor times the corresponding element of the argument vector.
	numericEpsilon	Sets the internal epsilon value of the gradient update routine.
	maximumStepMagnitudeFactor	Sets the maximum step distance for each line minimization in the algorithm.

Definition at line 434 of file optimizerBFGS.h.

Referenced by dlr::optimization::OptimizerBFGS< Functor >::OptimizerBFGS().

template<class Functor>

virtual void dlr::optimization::OptimizerBFGS< Functor >::setIterationLimit ( size_t iterationLimit ) [inline, virtual]

This method sets the optimization parameter controlling the maximum number of iterations, without affecting any other optimization parameters.

Parameters:

iterationLimit

Each minimization will terminate after this many iterations.

Definition at line 184 of file optimizerBFGS.h.

References dlr::optimization::OptimizerBFGS< Functor >::m_iterationLimit.

template<class Functor>

virtual void dlr::optimization::OptimizerBFGS< Functor >::setNumberOfRestarts ( size_t numberOfRestarts ) [inline, virtual]

This method sets the optimization parameter controlling the number of restarts, without affecting any other optimization parameters.

Parameters:

numberOfRestarts

Following successful termination, the minimization will be re-run this many times to refine the result accuracy.

Definition at line 199 of file optimizerBFGS.h.

References dlr::optimization::OptimizerBFGS< Functor >::m_numberOfRestarts.

template<class Functor>

void dlr::optimization::OptimizerBFGS< Functor >::setStartPoint ( const typename Functor::argument_type & startPoint ) [inline, virtual]

This method sets the initial conditions for the minimization.

Gradient based search will start at this location in parameter space.

Parameters:

startPoint

Indicates a point in the parameter space of the objective function.

Definition at line 467 of file optimizerBFGS.h.

References dlr::optimization::copyArgumentType(), dlr::optimization::OptimizerBFGS< Functor >::m_functionCallCount, dlr::optimization::OptimizerBFGS< Functor >::m_gradientCallCount, and dlr::optimization::OptimizerBFGS< Functor >::m_iterationCount.

template<class Functor>

virtual void dlr::optimization::OptimizerBFGS< Functor >::setVerbosity ( int verbosity ) [inline, virtual]

This method sets the amount of text printed to the standard output during the optimization.

Currently this method does nothing, since OptimizerBFGS never generates any standard output.

Parameters:

verbosity

This argument indicates the desired output level. Setting verbosity to zero mean that no standard output should be generated. Higher numbers indicate increasingly more output.

Definition at line 228 of file optimizerBFGS.h.

template<class Functor>

OptimizerBFGS< Functor > & dlr::optimization::OptimizerBFGS< Functor >::operator= ( const OptimizerBFGS< Functor > & source ) [inline, virtual]

Assignment operator.

Parameters:

source

The OptimizerBFGS instance to be copied.

Returns:: Reference to *this.

Definition at line 485 of file optimizerBFGS.h.

template<class Functor>

double dlr::optimization::OptimizerBFGS< Functor >::gradientConvergenceMetric	(	const argument_type &	theta,
		const result_type &	value,
		const argument_type &	gradient
	)			`[inline, protected]`

This protected member function is used to asses whether the algorithm has reached convergence.

Parameters:

	theta	This argument specifies the parameter values (arguments to the objective function) being assessed.
	value	This argument specifies the function value at the point described by theta.
	gradient	This argument specifies the function gradient at the point described by theta.

Returns:: The return value gets progressively smaller as we approach a local minimum.

Definition at line 512 of file optimizerBFGS.h.

template<class Functor>

std::pair< typename Functor::argument_type, typename Functor::result_type > dlr::optimization::OptimizerBFGS< Functor >::run ( ) [inline, protected, virtual]

Perform the optimization.

This virtual function overrides the definition in Optimizer.

Returns:: A std::pair of the vector parameter which brings the specified Functor to an optimum, and the corresponding optimal Functor value.

Implements dlr::optimization::Optimizer< Functor >.

Definition at line 533 of file optimizerBFGS.h.

References dlr::optimization::copyArgumentType(), DLR_THROW3, dlr::optimization::OptimizerBFGS< Functor >::doBfgs(), dlr::optimization::OptimizerBFGS< Functor >::m_functionCallCount, dlr::optimization::Optimizer< Functor >::m_functor, dlr::optimization::OptimizerBFGS< Functor >::m_gradientCallCount, dlr::optimization::OptimizerBFGS< Functor >::m_iterationCount, and dlr::optimization::OptimizerBFGS< Functor >::m_numberOfRestarts.

template<class Functor>

Triple< typename Functor::argument_type, typename Functor::result_type, typename Functor::argument_type > dlr::optimization::OptimizerBFGS< Functor >::doBfgs	(	const argument_type &	theta,
		const result_type &	startValue,
		const argument_type &	startGradient,
		size_t &	numberOfFunctionCalls,
		size_t &	numberOfGradientCalls,
		size_t &	numberOfIterations
	)			`[inline, protected]`

Perform one complete BFGS minimization, starting from the specified position.

Parameters:

	theta	This argument specifies the point at which to start the minimization.
	startValue	This argument should be set to the value of the objective function evaluated at theta.
	startGradient	This argument should be set to the objective function gradient evaluated at theta.
	numberOfFunctionCalls	This parameter is used to return the number of function calls required to perform the minimization.
	numberOfGradientCalls	This parameter is used to return the number of gradient calls required to perform the minimization.
	numberOfIterations	This parameter is used to return the number of BFGS iterations required to perform the minimization.

Returns:: A dlr::triple of the vector parameter which brings the specified Functor to an optimum, and the corresponding optimal Functor value, and the corresponding gradient.

Definition at line 588 of file optimizerBFGS.h.

Referenced by dlr::optimization::OptimizerBFGS< Functor >::run().

template<class Functor>

Functor dlr::optimization::Optimizer< Functor >::objectiveFunction ( ) [inline, inherited]

This method returns a copy of the Functor instance used for optimization.

Returns:: A Functor instance.

Definition at line 91 of file optimizer.h.

References dlr::optimization::Optimizer< Functor >::m_functor.

template<class Functor>

Optimizer< Functor >::result_type dlr::optimization::Optimizer< Functor >::optimalValue ( ) [inline, inherited]

This method finds the optimum of the current Functor, if necessary, and returns the Functor value at that point.

Note that you must have specified an objective function (Functor) before calling this method.

Returns:: The Functor value at it's optimum.

Definition at line 282 of file optimizer.h.

References dlr::optimization::Optimizer< Functor >::m_needsOptimization, dlr::optimization::Optimizer< Functor >::m_optimalValue, dlr::optimization::Optimizer< Functor >::run(), and dlr::optimization::Optimizer< Functor >::setOptimum().

template<class Functor>

Optimizer< Functor >::argument_type dlr::optimization::Optimizer< Functor >::optimum ( ) [inline, inherited]

This method finds the optimum of the current Functor, if necessary, and returns the Functor argument which produces that optimum.

Note that you must have specified an objective function (Functor) before calling this method.

Returns:: The Functor arguments which produce the optimal value.

Definition at line 297 of file optimizer.h.

References dlr::optimization::Optimizer< Functor >::m_needsOptimization, dlr::optimization::Optimizer< Functor >::m_optimum, dlr::optimization::Optimizer< Functor >::run(), and dlr::optimization::Optimizer< Functor >::setOptimum().

template<class Functor>

void dlr::optimization::Optimizer< Functor >::setObjectiveFunction ( const Functor & functor ) [inline, inherited]

This method specifies the Functor instance to use for the optimization.

If this function is overridden by the base class, it should normally either call Optimizer::setObjectiveFunction(), or explicitly set the member variable m_needsOptimization to true.

Parameters:

functor

A copy of this argument will be stored internally for use in optimization.

Definition at line 311 of file optimizer.h.

References dlr::optimization::Optimizer< Functor >::m_functor, and dlr::optimization::Optimizer< Functor >::m_needsOptimization.

template<class Functor>

virtual void dlr::optimization::Optimizer< Functor >::setOptimum	(	const typename Functor::argument_type &	optimum,
		const typename Functor::result_type &	optimalValue,
		bool	needsFurtherOptimization
	)			`[inline, protected, virtual, inherited]`

This protected member function provides a way for subclasses to communicate intermediate optimization results outside of the normal "return value of this->run()" method.

Parameters:

	optimum	This argument will be saved as the current optimum.
	optimalValue	This argument will be saved as the function value a the current optimum.
	needsFurtherOptimization	This argument indicates whether or not further refinement is necessary.

Definition at line 172 of file optimizer.h.

References dlr::optimization::Optimizer< Functor >::m_needsOptimization, dlr::optimization::Optimizer< Functor >::m_optimalValue, and dlr::optimization::Optimizer< Functor >::m_optimum.

Referenced by dlr::optimization::Optimizer< Functor >::optimalValue(), and dlr::optimization::Optimizer< Functor >::optimum().