CosineDecayer

Contents

CosineDecayer#

class stable_ssl.optim.CosineDecayer(total_steps, n_cycles=3, gamma=0.2)[source]#

Bases: object

Apply cosine decay with multiple cycles for learning rate scheduling.

This class implements a cosine decay function with multiple cycles that can be used as a learning rate scheduler. The decay follows a cosine curve with additional cyclic variations.

Parameters:

  • total_steps (int) – Total number of training steps.

  • n_cycles (int, optional) – Number of cycles in the cosine decay. Defaults to 3.

  • gamma (float, optional) – Gamma parameter for cycle amplitude. Defaults to 0.2.

Example

>>> decayer = CosineDecayer(total_steps=1000, n_cycles=3)
>>> lr_factor = decayer(step=500)
Contents