[ci] chaos: run llm_streaming tests on large instances

[aslonnie]

or they are consistently running out of memory

[gemini-code-assist]

Summary of Changes

Hello @aslonnie, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refactors the CI configuration for llm_streaming tests to ensure they run on larger instances, mitigating persistent out-of-memory errors. This change improves the reliability and stability of our continuous integration system by providing adequate resources for these resource-intensive tests.

Highlights

• CI Configuration Update: Moved the test_streaming_llm workload to a dedicated Buildkite pipeline step configured to run on large instance types.
• Resource Management: Addressed consistent out-of-memory issues for llm_streaming tests by allocating more resources.
• Test Coverage: Ensured test_streaming_llm continues to be tested under various chaos conditions (no fault, network delay, network bandwidth) within its new dedicated step.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request correctly moves the llm_streaming chaos tests to a larger instance type to address out-of-memory issues. However, the new Buildkite step duplicates the key and label of an existing step, which can cause issues with pipeline execution and make the UI confusing. I've suggested making these unique to improve the configuration's correctness and maintainability.

[gemini-code-assist]

This new test step duplicates the label and key from the other kuberay_tests step defined above.

• key: In Buildkite, step keys must be unique across the pipeline. Duplicate keys can lead to unpredictable behavior, especially with step dependencies and artifact handling.
• label: While not a functional error, having identical labels for different test matrices makes it difficult to distinguish them in the Buildkite UI.

To avoid these issues, please provide a unique key and a more descriptive label for this new step.

  - label: ":kubernetes: chaos (large) {{matrix.workload}} under {{matrix.fault}}"
    key: kuberay_tests_large