Skip to content

[CI] Enable UR runs in benchmarking CI #19753

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: sycl
Choose a base branch
from

Conversation

ianayl
Copy link
Contributor

@ianayl ianayl commented Aug 8, 2025

Temporary change enabling UR benchmarking.

This is not the most ideal solution, but currently no benchmarking data is being collected for the UR; this data is needed urgently. A proper solution can be discussed after UR benchmarking has been re-enabled.

Note: The new benchmark CI runs are using a docker environment, there seems to be a very consistent performance penalty of 2 microseconds as a result compared to previous UR benchmark CI runs that are not using a docker environment. Please see https://intel.github.io/llvm/benchmarks/?tags=UR&runs=Baseline_PVC_L0%2CCommit5661e7275b64a6c00a96ae295815f40a9384e958_PVC_L0 : this compares performance of latest commit with commit 5661e72, which was the commit used to benchmark the last run result from July 30th.

I am currently looking into ways to reduce this penalty, for now I am opening this PR as the data is still useful. It's worth noting that SYCL only incurred 0.5-1 microsecond penalty from this move, so there may be room for improvement here.

@ianayl ianayl marked this pull request as ready for review August 11, 2025 20:09
@ianayl ianayl requested a review from a team as a code owner August 11, 2025 20:09
Copy link
Contributor

@sarnex sarnex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fine with me assuming this wont slow down precommit or postcommit

# intermediate files required can be stitched together from the build files.
# However, this is not exactly "clean" or "fun to maintain"...
- name: Build Unified Runtime
shell: bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't run by default in precommit or postcommit right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, this workflow is completely separate from precommit/postcommit.

If we were to do this in precommit in the future, this solution would not suffice. I proposed in the comments that we have UR build workflows produce an UR artifact, but that would involve reorganizing the UR workflows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants