GradientNormClippingTrainingCallback

class GradientNormClippingTrainingCallback(max_norm, norm_type=None)[source]

A callback for gradient clipping before stepping the optimizer with torch.nn.utils.clip_grad_norm_().

Initialize the callback.

Parameters:

max_norm (float) – The maximum gradient norm for use with gradient clipping.
norm_type (Optional[float]) – The gradient norm type to use for maximum gradient norm, cf. torch.nn.utils.clip_grad_norm_()

Methods Summary

Call before the optimizer's step.

Methods Documentation

pre_step(**kwargs)[source]

Call before the optimizer’s step.