neureka.optimization

Interface Optimizer<V>

Type Parameters:

V - The value type parameter of the tensors processed by this optimizer.

All Superinterfaces:

Component<Tensor<V>>, Optimization<V>

All Known Implementing Classes:

AdaGrad, ADAM, Momentum, RMSProp, SGD

public interface Optimizer<V>
extends Component<Tensor<V>>, Optimization<V>

Optimizers are tensor components which implement the Optimization (functional) interface applying various optimization algorithms to the gradients of tensors.
A simple usage example would be the following:

      Optimizer o = new SGD(0.01); // 0.01 learning rate
      Tensor<Float> w = Tensor.of(0f);
      w.set(o);
  

If you want to create custom Optimizer implementations in a functional style take a look at the following example:

      Optimizer.of( t -> {
          Tensor<?> gradient = t.getGradient();
          // ... apply algorithm ...
      })
  

Or consider using the factory method below to process gradients directly:

      Optimizer.ofGradient( gradient -> {
          // ... apply algorithm ...
      })
  

Nested Class Summary

Nested classes/interfaces inherited from interface neureka.common.composition.Component

Component.IsBeing, Component.OwnerChangeRequest<O>

Field Summary

Fields

Modifier and Type	Field and Description
static AdaGradFactory	AdaGrad
static ADAMFactory	ADAM
static MomentumFactory	Momentum
static RMSPropFactory	RMSProp
static SGDFactory	SGD

Method Summary

Modifier and Type	Method and Description
static <T> Optimizer<T>	of(Optimization<T> o)
static <T> Optimizer<T>	ofGradient(Optimization<T> o)

Methods inherited from interface neureka.common.composition.Component

update

Methods inherited from interface neureka.optimization.Optimization

optimize

Field Detail

ADAM

static final ADAMFactory ADAM

AdaGrad

static final AdaGradFactory AdaGrad

Momentum

static final MomentumFactory Momentum

RMSProp

static final RMSPropFactory RMSProp

SGD

static final SGDFactory SGD

Method Detail

of

static <T> Optimizer<T> of(Optimization<T> o)

Type Parameters:

T - The value type parameter of the tensors processed by this optimizer.

Parameters:

o - The Optimization lambda which receives a tensor for optimization.

Returns:

An Optimizer which will process any passed tensor directly (see ofGradient(Optimization) which processes gradients).

ofGradient

static <T> Optimizer<T> ofGradient(Optimization<T> o)

Type Parameters:

T - The value type parameter of the tensors processed by this optimizer.

Parameters:

o - The Optimization lambda which receives the gradient of a tensor for optimization.

Returns:

An Optimizer which will process the gradient of any passed tensor (see of(Optimization) which processes tensors directly).