Fix/starcoder2 conversion dtype

[AzizCode92]

Important

The Update branch button must only be pressed in very rare occassions.
An outdated branch is never blocking the merge of a PR.
Please reach out to the automation team before pressing that button.

What does this PR do ?

Fixes dtype preservation bug in StarCoder2 NeMo to HuggingFace conversion script where --precision bf16 incorrectly saves model config with torch_dtype: "float32".

Collection: NLP

Changelog

• Fixed convert_starcoder2_nemo_to_hf.py to properly preserve the target dtype in model config
• Added dtype parameter to AutoModelForCausalLM.from_config() call to ensure model is created with correct precision
• Moved dtype determination logic from convert() function to main execution block for proper scope access
• Model config now correctly reflects the precision specified via --precision argument

Usage

The conversion script now correctly preserves the specified precision in the output model config:

# Convert with bfloat16 precision
python convert_starcoder2_nemo_to_hf.py \
  --input_name_or_path /path/to/model.nemo \
  --output_path /path/to/output \
  --hf-model-name /path/to/base/hf/model \
  --precision bf16

# Output config.json now correctly shows:
# "torch_dtype": "bfloat16"  (previously would show "float32")

Before this fix:

{"torch_dtype": "float32"}  // Wrong, despite --precision bf16

After this fix:

{"torch_dtype": "bfloat16"}  // Correct, matches --precision bf16

GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.
The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
• Did you write any new necessary tests? (Not required for this simple config fix, existing conversion tests cover functionality)
• Did you add or update any necessary documentation? (No doc changes needed, preserves existing CLI interface)
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • Reviewer: Does the PR have correct import guards for all optional libraries? (No new imports added)

PR Type:

• New Feature
• Bugfix
• Documentation

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

Additional Information

• Root Cause: AutoModelForCausalLM.from_config(config) creates models in float32 by default, ignoring config.torch_dtype
• Impact: Users converting models with --precision bf16/16 get incorrect model weights
• Testing: Verified with StarCoder2-7B conversion using --precision bf16, config now correctly shows "torch_dtype": "bfloat16"
• Backward Compatibility: Fully maintained, only fixes the dtype metadata issue