Refactor Legacy and DRTulu parsers to use tool_definitions instead of tool_actors

[hamishivi]

Summary

• Legacy parser: Removes the tool_actors code path and uses only OpenAI-format tool_definitions to derive tool names and parameter names for <tool_name>content</tool_name> tag parsing.
• DRTulu parser: Replaces tool_actors dependency with tool_definitions + explicit stop_sequences. Stop sequences are now fetched from the EnvironmentPool during initialization in grpo_fast.py, then plumbed through create_vllm_engines → LLMRayActor → create_tool_parser.
• Removes import ray from parsers.py since neither parser needs direct Ray actor access anymore.
• Removes the blocking error in grpo_fast.py that prevented --tool_parser_type dr_tulu from being used.
• Updates all tests to use tool_definitions instead of mock Ray actors.

Context

After the EnvironmentPool refactor (#1479), the parsers still had code paths using tool_actors (Ray actor handles). For the Legacy parser this was dead code (it already had a tool_definitions path). For the DRTulu parser it was completely broken — there was a hard error at startup blocking dr_tulu parser usage.

Test plan

• Verify Legacy parser tests pass (uses tool_definitions only)
• Verify DRTulu parser tests pass (uses tool_definitions + stop_sequences)
• Verify create_tool_parser factory tests pass for all parser types
• Run dr_tulu debug script (scripts/train/debug/tools/dr_tulu_parser_debug.sh) on GPU
• Run legacy debug script (scripts/train/debug/tools/legacy_parser_debug.sh) on GPU

Made with Cursor

[gemini-code-assist]

Summary of Changes

Hello @hamishivi, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors the tool parsing mechanism within the system, moving away from direct Ray actor dependencies towards a more standardized approach using OpenAI-format tool definitions and explicit stop sequences. This change streamlines the architecture, particularly for the Legacy and DRTulu parsers, and resolves a critical issue that previously blocked the use of the DRTulu parser. The update ensures that tool information is consistently handled across different parser types, improving maintainability and enabling future enhancements.

Highlights

• Legacy Parser Refactor: The Legacy parser was updated to remove the tool_actors code path, now exclusively using OpenAI-format tool_definitions to derive tool and parameter names for tag parsing.
• DRTulu Parser Enhancement: The DRTulu parser now replaces its tool_actors dependency with tool_definitions and explicit stop_sequences. These stop sequences are fetched from the EnvironmentPool during initialization and plumbed through the system.
• Ray Dependency Removal: The direct import ray statement was removed from parsers.py as neither parser requires direct Ray actor access anymore.
• DRTulu Parser Bug Fix: A blocking error in grpo_fast.py that previously prevented the --tool_parser_type dr_tulu from being used has been removed, enabling its functionality.
• Test Suite Update: All relevant tests were updated to reflect these changes, now utilizing tool_definitions instead of mock Ray actors for consistency.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• open_instruct/environments/tools/parsers.py
  • Removed the ray import statement.
  • Modified OpenInstructLegacyToolParser to remove the tool_actors parameter from its constructor and initialization logic, now relying solely on tool_definitions.
  • Updated DRTuluToolParser to accept tool_definitions and stop_sequences instead of tool_actors in its constructor, and adjusted its internal logic accordingly.
  • Refactored create_tool_parser to remove the tool_actors parameter and introduce stop_sequences as an explicit argument for parser creation.
• open_instruct/environments/tools/tests/test_parsers.py
  • Removed the MockTool class and create_mock_tool_actor utility function, which were used for mocking Ray actors.
  • Added a new helper function make_tool_definition to create OpenAI-format tool definitions for testing purposes.
  • Updated all test cases within TestOpenInstructLegacyToolParser and TestDRTuluToolParser to use make_tool_definition and pass tool_definitions and stop_sequences directly, eliminating the need for Ray actor mocks.
  • Removed setUp and tearDown methods that patched the ray module for testing.
• open_instruct/grpo_fast.py
  • Added tool_stop_sequences as a new parameter to the create_model_and_optimizer function signature.
  • Modified the initialize_tools_and_envs function to collect tool_stop_strings from environment pools specifically when the dr_tulu parser type is active.
  • Removed the explicit ValueError that previously blocked the use of the dr_tulu parser type.
  • Ensured tool_stop_sequences are correctly passed through the create_model_and_optimizer function call.
• open_instruct/vllm_utils.py
  • Added tool_stop_sequences as a new parameter to the LLMRayActor.__init__ method.
  • Stored the tool_stop_sequences in the _tool_stop_sequences instance variable within LLMRayActor.
  • Modified the _init_tool_parser method within LLMRayActor to pass self._tool_stop_sequences to the create_tool_parser function.
  • Added tool_stop_sequences as a new parameter to the create_vllm_engines function.

Activity

• No human activity (comments, reviews) has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request is a solid refactoring that removes the dependency on Ray actors from the Legacy and DRTulu parsers, opting to use tool_definitions instead. This change significantly improves modularity and makes the code easier to test, as demonstrated by the cleaner, mock-free test suite. The implementation is consistent and well-executed across all modified files. I have one suggestion to further improve performance by parallelizing some of the Ray calls. Overall, this is an excellent enhancement.

[hamishivi]

/gemini review

[gemini-code-assist]

Code Review

The pull request successfully refactors the Legacy and DRTulu tool parsers to use OpenAI-format tool_definitions instead of direct Ray actor handles. This improves the modularity of the system and fixes the previously broken dr_tulu parser by correctly plumbing stop sequences from the environment pools to the vLLM engines. The removal of the Ray dependency from parsers.py is a good cleanup. I have provided a suggestion to optimize the Ray metadata fetching in grpo_fast.py to improve startup performance by avoiding sequential blocking calls.

[hamishivi]

/gemini review

[gemini-code-assist]

Code Review

The pull request successfully refactors the Legacy and DRTulu tool parsers to use OpenAI-format tool_definitions instead of direct Ray tool_actors. This change improves modularity by removing the direct dependency on Ray within the parser logic and fixes the previously broken DRTuluToolParser. The plumbing of stop_sequences from the EnvironmentPool through to the LLMRayActor is well-implemented, and the parallelization of tool definition fetching in grpo_fast.py is a good performance optimization. The tests have been appropriately updated to reflect these architectural changes.

[hamishivi]

/gemini review

[gemini-code-assist]

Code Review

This is a great refactoring that simplifies the tool parsers by removing the dependency on Ray actors and instead using OpenAI-style tool_definitions. The changes are consistent across the codebase, and the tests have been thoroughly updated to reflect this, which is excellent. The fix for the dr_tulu parser and the performance improvement in initialize_tools_and_envs are also valuable additions. I have one suggestion to improve the robustness of resource handling.

[hamishivi]

Ran and checked that the existing tool and debug scripts for dr tulu, legacy, hermes all worked.

[natolambert]

This one seemed just like a good add!