dlpy.model.NatGradSolver

class dlpy.model.NatGradSolver(approximation_type=1, learning_rate=0.001, learning_rate_policy='fixed', gamma=0.1, step_size=10, power=0.75, use_locking=True, clip_grad_max=None, clip_grad_min=None, steps=None, fcmp_learning_rate=None, lr_scheduler=None)

Bases: dlpy.model.Solver

Natural gradient solver object

Parameters:

approximation_type : int, optional

Specifies the approximate natural gradient type.

learning_rate : double, optional

Specifies the learning rate for the deep learning algorithm.

learning_rate_policy : string, optional

Specifies the learning rate policy for the deep learning algorithm.
Valid Values: FIXED, STEP, POLY, INV, MULTISTEP
Default: FIXED

gamma : double, optional

Specifies the gamma for the learning rate policy.

step_size : int, optional

Specifies the step size when the learning rate policy is set to STEP.

power : double, optional

Specifies the power for the learning rate policy.

use_locking : bool, optional

When it is false, the gradients are computed asynchronously with multiple threads.

clip_grad_max : double, optional

Specifies the maximum gradient value. All gradients that are greater than the specified value are set to the specified value.

clip_grad_min : double, optional

Specifies the minimum gradient value. All gradients that are less than the specified value are set to the specified value.

steps : list-of-ints, optional

specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.

fcmp_learning_rate : string, optional

specifies the FCMP learning rate function.

lr_scheduler : LRScheduler, optional

Specifies learning rate policy DLPy provides you with some predefined learning rate policies. 1. FixedLR 2. StepLR 3. MultiStepLR 4. PolynomialLR 5. ReduceLROnPlateau 6. CyclicLR Besides, you can also customize your own learning rate policy. You can find more examples at DLPy example folder.

Returns:

NatGradSolver

__init__(approximation_type=1, learning_rate=0.001, learning_rate_policy='fixed', gamma=0.1, step_size=10, power=0.75, use_locking=True, clip_grad_max=None, clip_grad_min=None, steps=None, fcmp_learning_rate=None, lr_scheduler=None)

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__([approximation_type, …])	Initialize self.
add_parameter(key, value)	Adds a parameter to the parameter list of a solver.
clear()
get(k[,d])
items()
keys()
pop(k[,d])	If key is not found, d is returned if given, otherwise KeyError is raised.
popitem()	as a 2-tuple; but raise KeyError if D is empty.
set_method(method)	Sets the solver method in the parameters list.
setdefault(k[,d])
update([E, ]**F)	If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
values()