Skip to content

Commit 2a79471

Browse files
authored
Fixing Bitnet after use_rms_norm introduction (#38229)
* fix * make style
1 parent 9661896 commit 2a79471

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

src/transformers/integrations/bitnet.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -362,8 +362,8 @@ def _replace_with_bitnet_linear(
362362
bias=module.bias is not None,
363363
device=module.weight.device,
364364
dtype=module.weight.dtype,
365-
use_rms_norm=quantization_config.use_rms_norm,
366-
rms_norm_eps=quantization_config.rms_norm_eps,
365+
use_rms_norm=quantization_config.use_rms_norm if quantization_config else False,
366+
rms_norm_eps=quantization_config.rms_norm_eps if quantization_config else 1e-6,
367367
)
368368
model._modules[name].requires_grad_(False)
369369
has_been_replaced = True

0 commit comments

Comments
 (0)