Skip to content

Support model "buffers" #52

@Kaixhin

Description

@Kaixhin

Basically these are parameters that aren't updated via gradient descent (but would be serialized - a good example that already exists here is the running mean or running variance in batch normalisation); they could also be completely fixed.

The current working solution is to just add Variables that don't need gradients within initialize_parameters. We need to also make sure optimizers skips anything with requires_grad=false.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions