dlpy.model.LBFGSolver

class dlpy.model.LBFGSolver(m, max_line_search_iters, max_iters, backtrack_ratio, learning_rate=0.001, learning_rate_policy='fixed', gamma=0.1, step_size=10, power=0.75, use_locking=True, clip_grad_max=None, clip_grad_min=None, steps=None, fcmp_learning_rate=None, lr_scheduler=None)

Bases: dlpy.model.Solver

LBFG solver object

Parameters:

m : int

Specifies the number of corrections used in the L-BFGS update.

max_line_search_iters : int

Specifies the maximum number of line search iterations for L-BFGS solver.

max_iters : int

Specifies the maximum number of iterations for the L-BFGS solver. When the miniBatchSize option is not specified, each iteration goes through at least one epoch. When the miniBatchSize option is specified, each L-BFGS iteration processes one mini-batch. The L-BFGS solver stops when the iteration number reaches the value of the maxIters= option or the epoch number reaches the value of the maxEpochs= option.

backtrack_ratio : double

Specifies the backtrack ratio of line search iterations for L-BFGS solver.

learning_rate : double, optional

Specifies the learning rate for the deep learning algorithm.

learning_rate_policy : string, optional

Specifies the learning rate policy for the deep learning algorithm.
Valid Values: FIXED, STEP, POLY, INV, MULTISTEP
Default: FIXED

gamma : double, optional

Specifies the gamma for the learning rate policy.

step_size : int, optional

Specifies the step size when the learning rate policy is set to STEP.

power : double, optional

Specifies the power for the learning rate policy.

use_locking : bool, optional

When it is false, the gradients are computed asynchronously with multiple threads.

clip_grad_max : double, optional

Specifies the maximum gradient value. All gradients that are greater than the specified value are set to the specified value.

clip_grad_min : double, optional

Specifies the minimum gradient value. All gradients that are less than the specified value are set to the specified value.

steps : list-of-ints, optional

specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.

fcmp_learning_rate : string, optional

specifies the FCMP learning rate function.

lr_scheduler : LRScheduler, optional

Specifies learning rate policy DLPy provides you with some predefined learning rate policies. 1. FixedLR 2. StepLR 3. MultiStepLR 4. PolynomialLR 5. ReduceLROnPlateau 6. CyclicLR Besides, you can also customize your own learning rate policy. You can find more examples at DLPy example folder.

Returns:

LBFGSolver

__init__(m, max_line_search_iters, max_iters, backtrack_ratio, learning_rate=0.001, learning_rate_policy='fixed', gamma=0.1, step_size=10, power=0.75, use_locking=True, clip_grad_max=None, clip_grad_min=None, steps=None, fcmp_learning_rate=None, lr_scheduler=None)

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__(m, max_line_search_iters, …[, …])	Initialize self.
add_parameter(key, value)	Adds a parameter to the parameter list of a solver.
clear()
get(k[,d])
items()
keys()
pop(k[,d])	If key is not found, d is returned if given, otherwise KeyError is raised.
popitem()	as a 2-tuple; but raise KeyError if D is empty.
set_method(method)	Sets the solver method in the parameters list.
setdefault(k[,d])
update([E, ]**F)	If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
values()