tiny-random glm4v configuration can't load due to config validation changes

[tomaarsen]

Hello!

I'm getting failures with the following script starting from #41250

from transformers import AutoConfig

config = AutoConfig.from_pretrained("tiny-random/glm-4v")
print(type(config))

Traceback (most recent call last):
  File "[sic]\demo_glm4v_config.py", line 4, in <module>
    config = AutoConfig.from_pretrained("tiny-random/glm-4v")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[sic]\src\transformers\models\auto\configuration_auto.py", line 1484, in from_pretrained
    return config_class.from_dict(config_dict, **unused_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[sic]\src\transformers\configuration_utils.py", line 757, in from_dict
    config = cls(**config_dict)
             ^^^^^^^^^^^^^^^^^^
  File "[sic]\transformers\Lib\site-packages\huggingface_hub\dataclasses.py", line 280, in init_with_validate
    cls.validate(self)  # type: ignore [attr-defined]
    ^^^^^^^^^^^^^^^^^^
  File "[sic]\transformers\Lib\site-packages\huggingface_hub\dataclasses.py", line 255, in validate
    validator(self)
  File "[sic]\src\transformers\modeling_rope_utils.py", line 723, in validate_rope
    validation_fn(rope_parameters, ignore_keys=self.ignore_keys_at_rope_validation)
  File "[sic]\src\transformers\modeling_rope_utils.py", line 733, in _validate_default_rope_parameters
    self._check_received_keys(rope_type, received_keys, required_keys, ignore_keys=ignore_keys)
  File "[sic]\src\transformers\modeling_rope_utils.py", line 921, in _check_received_keys
    raise KeyError(f"Missing required keys in `rope_parameters` for 'rope_type'='{rope_type}': {missing_keys}")
KeyError: "Missing required keys in `rope_parameters` for 'rope_type'='default': {'rope_theta'}"

Note that the remote config uses the old-style top-level rope parameters, including rope_theta: https://huggingface.co/tiny-random/glm-4v/blob/main/config.json#L28-L37
Is this expected to fail now, or is this an issue of the old-style configuration not propagating nicely to this validation check?

cc @zucchini-nlp

• Tom Aarsen

Originally posted by @tomaarsen in #41250 (comment)