mic::neural_nets::optimization::OptimizationFunction< eT > Class Template Reference

Abstract class representing interface to optimization function. More...

#include <OptimizationFunction.hpp>

Inheritance diagram for mic::neural_nets::optimization::OptimizationFunction< eT >:

Public Member Functions
	OptimizationFunction ()
virtual	~OptimizationFunction ()
	Virtual destructor - empty. More...
virtual void	update (mic::types::MatrixPtr< eT > p_, mic::types::MatrixPtr< eT > dp_, eT learning_rate_, eT decay_=0.0)
virtual void	update (mic::types::MatrixPtr< eT > p_, mic::types::MatrixPtr< eT > x_, mic::types::MatrixPtr< eT > y_, eT learning_rate_=0.001)
virtual mic::types::MatrixPtr< eT >	calculateUpdate (mic::types::MatrixPtr< eT > x_, mic::types::MatrixPtr< eT > dx_, eT learning_rate_)=0

Detailed Description

template<typename eT = float>
class mic::neural_nets::optimization::OptimizationFunction< eT >

Abstract class representing interface to optimization function.

Author

tkornuta

Template Parameters

eT	Template type (single/double precision)

Definition at line 41 of file OptimizationFunction.hpp.

Constructor & Destructor Documentation

template<typename eT = float>

mic::neural_nets::optimization::OptimizationFunction< eT >::OptimizationFunction ( )

inline

Constructor. Remembers dimensions.

Definition at line 46 of file OptimizationFunction.hpp.

template<typename eT = float>

virtual mic::neural_nets::optimization::OptimizationFunction< eT >::~OptimizationFunction ( )

inlinevirtual

Virtual destructor - empty.

Definition at line 49 of file OptimizationFunction.hpp.

Member Function Documentation

template<typename eT = float>

virtual mic::types::MatrixPtr<eT> mic::neural_nets::optimization::OptimizationFunction< eT >::calculateUpdate ( mic::types::MatrixPtr< eT >  x_, mic::types::MatrixPtr< eT >  dx_, eT  learning_rate_  )

pure virtual

Abstract method responsible for calculating the update.

Parameters

x_	Pointer to the current matrix (parameter) OR Pointer to current input matrix (in Hebbian learning).
dx_	Pointer to current gradient of that matrix (parameter) OR Pointer to current output matrix (in Hebbian learning).
learning_rate_	Learning rate.

Implemented in mic::neural_nets::optimization::AdaGradPID< eT >, mic::neural_nets::learning::NormalizedHebbianRule< eT >, mic::neural_nets::learning::NormalizedZerosumHebbianRule< eT >, mic::neural_nets::optimization::GradPID< eT >, mic::neural_nets::optimization::AdamID< eT >, mic::neural_nets::optimization::Adam< eT >, mic::neural_nets::optimization::AdaDelta< eT >, mic::neural_nets::learning::BinaryCorrelatorLearningRule< eT >, mic::neural_nets::learning::HebbianRule< eT >, mic::neural_nets::optimization::AdaGrad< eT >, mic::neural_nets::optimization::RMSProp< eT >, mic::neural_nets::optimization::GradientDescent< eT >, and mic::neural_nets::optimization::Momentum< eT >.

Referenced by mic::neural_nets::optimization::OptimizationFunction< eT >::update().

template<typename eT = float>

virtual void mic::neural_nets::optimization::OptimizationFunction< eT >::update ( mic::types::MatrixPtr< eT >  p_, mic::types::MatrixPtr< eT >  dp_, eT  learning_rate_, eT  decay_ = 0.0  )

inlinevirtual

Method responsible for performing the update using backpropagation and gradient descent. Calls abstract method calculateUpdate().

Parameters

p_	Pointer to the current parameter (matrix).
dp_	Pointer to current gradient of that parameter (matrix).
learning_rate_	Learning rate.
decay_	Weight decay rate (determining that the "unused/unupdated" weights will decay to 0) (DEFAULT = 0.0 means "no decay").

Definition at line 58 of file OptimizationFunction.hpp.

References mic::neural_nets::optimization::OptimizationFunction< eT >::calculateUpdate().

Referenced by TEST_F().

Here is the call graph for this function:

template<typename eT = float>

virtual void mic::neural_nets::optimization::OptimizationFunction< eT >::update ( mic::types::MatrixPtr< eT >  p_, mic::types::MatrixPtr< eT >  x_, mic::types::MatrixPtr< eT >  y_, eT  learning_rate_ = 0.001  )

inlinevirtual

Updates the weight matrix according to the hebbian rule.

Parameters

p_	Pointer to the parameter (weight) matrix.
x_	Pointer to the input data matrix.
y_	Pointer to the output data matrix.
learning_rate_	Learning rate (default=0.001).

Reimplemented in mic::neural_nets::learning::NormalizedHebbianRule< eT >, and mic::neural_nets::learning::NormalizedZerosumHebbianRule< eT >.

Definition at line 78 of file OptimizationFunction.hpp.

References mic::neural_nets::optimization::OptimizationFunction< eT >::calculateUpdate().

Here is the call graph for this function:

---

The documentation for this class was generated from the following file:

• src/optimization/OptimizationFunction.hpp