Skip to content

Commit 93da29e

Browse files
Fix CosineDecay documentation to clarify alpha is a multiplier
The documentation incorrectly stated that learning rate decays 'to alpha', when it actually decays to 'initial_lr * alpha'. Updated the docstring to make it clear that alpha is a fraction/ multiplier, not an absolute target value.
1 parent b4d9c89 commit 93da29e

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

keras/src/optimizers/schedules/learning_rate_schedule.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -584,9 +584,9 @@ class CosineDecay(LearningRateSchedule):
584584
schedule applies a linear increase per optimizer step to our learning rate
585585
from `initial_learning_rate` to `warmup_target` for a duration of
586586
`warmup_steps`. Afterwards, it applies a cosine decay function taking our
587-
learning rate from `warmup_target` to `alpha` for a duration of
587+
learning rate from `warmup_target` to `warmup_target * alpha` for a duration of
588588
`decay_steps`. If `warmup_target` is None we skip warmup and our decay
589-
will take our learning rate from `initial_learning_rate` to `alpha`.
589+
will take our learning rate from `initial_learning_rate` to `initial_learning_rate * alpha`.
590590
It requires a `step` value to compute the learning rate. You can
591591
just pass a backend variable that you increment at each training step.
592592

0 commit comments

Comments
 (0)