dlpy.model.AdamSolver¶
-
class
dlpy.model.AdamSolver(beta1=0.9, beta2=0.999, learning_rate=0.001, learning_rate_policy='fixed', gamma=0.1, step_size=10, power=0.75, use_locking=True, clip_grad_max=None, clip_grad_min=None, steps=None, fcmp_learning_rate=None, lr_scheduler=None)¶ Bases: dlpy.model.Solver
Adam solver object
Parameters: - beta1 : double, optional
Specifies the exponential decay rate for the first moment in the Adam learning algorithm.
- beta2 : double, optional
Specifies the exponential decay rate for the second moment in the Adam learning algorithm.
- learning_rate : double, optional
Specifies the learning rate for the deep learning algorithm.
- learning_rate_policy : string, optional
Specifies the learning rate policy for the deep learning algorithm.
Valid Values: FIXED, STEP, POLY, INV, MULTISTEP
Default: FIXED- gamma : double, optional
Specifies the gamma for the learning rate policy.
- step_size: int, optional
Specifies the step size when the learning rate policy is set to STEP.
- power : double, optional
Specifies the power for the learning rate policy.
- use_locking : bool, optional
When it is false, the gradients are computed asynchronously with multiple threads.
- clip_grad_max : double, optional
Specifies the maximum gradient value. All gradients that are greater than the specified value are set to the specified value.
- clip_grad_min : double, optional
Specifies the minimum gradient value. All gradients that are less than the specified value are set to the specified value.
- steps : list-of-ints, optional
specifies a list of epoch counts. When the current epoch matches one of the specified steps, the learning rate is multiplied by the value of the gamma parameter. For example, if you specify {5, 9, 13}, then the learning rate is multiplied by gamma after the fifth, ninth, and thirteenth epochs.
- fcmp_learning_rate : string, optional
specifies the FCMP learning rate function.
- lr_scheduler : LRScheduler, optional
Specifies learning rate policy DLPy provides you with some predefined learning rate policies. 1. FixedLR 2. StepLR 3. MultiStepLR 4. PolynomialLR 5. ReduceLROnPlateau 6. CyclicLR Besides, you can also customize your own learning rate policy. You can find more examples at DLPy example folder.
Returns: -
__init__(beta1=0.9, beta2=0.999, learning_rate=0.001, learning_rate_policy='fixed', gamma=0.1, step_size=10, power=0.75, use_locking=True, clip_grad_max=None, clip_grad_min=None, steps=None, fcmp_learning_rate=None, lr_scheduler=None)¶ Initialize self. See help(type(self)) for accurate signature.
Methods
__init__([beta1, beta2, learning_rate, …]) Initialize self. add_parameter(key, value) Adds a parameter to the parameter list of a solver. clear() get(k[,d]) items() keys() pop(k[,d]) If key is not found, d is returned if given, otherwise KeyError is raised. popitem() as a 2-tuple; but raise KeyError if D is empty. set_method(method) Sets the solver method in the parameters list. setdefault(k[,d]) update([E, ]**F) If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v values()