Skip to content

find_unused_parameters forced to False for ddp_{spawn,fork} #14089

@LoicGrobol

Description

@LoicGrobol

🐛 Bug

Since #13744 , the ddp_fork and ddp_spawn strategies have find_unused_parameters set to False, making them equivalent to ddp_fork_find_unused_parameters_false and ddp_spawn_find_unused_parameters_false. This is caused by a mistaken simplification of register_strategies that removed the distinction between them. Besides being an API break, this can trigger a runtime error when training any model with unused parameters using these strategies

To Reproduce

https://colab.research.google.com/drive/1xxoSnEDc8s6z-j66o1MbtEjVwGJ7K43p

Environment

  • CUDA:
    • GPU:
      • Tesla T4
    • available: True
    • version: 11.3
  • Packages:
    • lightning: None
    • lightning_app: None
    • numpy: 1.21.6
    • pyTorch_debug: False
    • pyTorch_version: 1.12.0+cu113
    • pytorch-lightning: 1.7.0
    • tqdm: 4.64.0
  • System:
    • OS: Linux
    • architecture:
      • 64bit
    • processor: x86_64
    • python: 3.7.13
    • version: 1 SMP Sun Apr 24 10:03:06 PDT 2022

Please tell me if I can just submit a PR (I can probably have it ready today) or if you'd rather have a go at this yourselves.

cc @tchaton @rohitgr7 @justusschock @kaushikb11 @awaelchli @akihironitta

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpriority: 0High priority taskstrategy: ddpDistributedDataParallel

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions