Skip to content

Commit c42e7d6

Browse files
authored
revert meta device for esm2 (#1380)
Reverts meta device init for esm-2 after large-scale convergence tests showed issues Signed-off-by: Peter St. John <[email protected]>
1 parent 0279dcb commit c42e7d6

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

bionemo-recipes/recipes/esm2_native_te/hydra_config/defaults.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
model_tag: ??? # E.g., nvidia/esm2_t6_8M_UR50D, facebook/esm2_t6_8M_UR50D, or a local path (e.g ./example_8m_checkpoint)
33
num_train_steps: ???
44

5-
use_meta_device: true
5+
use_meta_device: false # meta-device init is still not converging
66

77
# Whether to wrap the model in torch.compile. Note, this is currently not supported with mfsdp (BIONEMO-2977).
88
# We leave this off by default since we don't see much of a performance improvement with TE layers.

0 commit comments

Comments
 (0)