I'm trying to understand the reason behind using Hydra as the internal configuration.

[zeocax]

My thinking was, maybe we could use something like a @ dataclass (e.g., Config()) internally to set up the parameters?
Then, based on that dataclass, we could provide a default Hydra YAML file for users. That way, users could just edit that YAML file for their specific training runs.

The way it seems to be set up now leads to a couple of things that feel a bit off to me:

• Long Command Lines: Hydra being used inside, but it looks like users instead need to type out really long command-line arguments to change the parameters. This feels kinda counter-intuitive. ☹️
• Handling kwargs: Using Hydra YAML for all the internal parameters also seems to make using flexible keyword arguments (kwargs) a bit tricky. I saw reward_kwargs, but it looks like the way to add custom keys is with syntax like +reward_model.reward_kwargs.some_custom_keys=value. That + means you're adding a key that wasn't originally there, right? (I'm not familiar with Hydra, so please correct me if I'm wrong here!).

Appreciate any insights or discussion on this!

[vermouth1992]

We are actually thinking replace internal hydra with dataclass. Stay tuned

[liuzhenhai93]

Actually, hydra and dataclass can be combined.
#1941

[eric-haibin-lin]

moving discussions to #2648