-
Notifications
You must be signed in to change notification settings - Fork 798
[CI] Enable UR runs in benchmarking CI #19753
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: sycl
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fine with me assuming this wont slow down precommit or postcommit
# intermediate files required can be stitched together from the build files. | ||
# However, this is not exactly "clean" or "fun to maintain"... | ||
- name: Build Unified Runtime | ||
shell: bash |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This isn't run by default in precommit or postcommit right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nope, this workflow is completely separate from precommit/postcommit.
If we were to do this in precommit in the future, this solution would not suffice. I proposed in the comments that we have UR build workflows produce an UR artifact, but that would involve reorganizing the UR workflows.
Temporary change enabling UR benchmarking.
This is not the most ideal solution, but currently no benchmarking data is being collected for the UR; this data is needed urgently. A proper solution can be discussed after UR benchmarking has been re-enabled.
Note: The new benchmark CI runs are using a docker environment, there seems to be a very consistent performance penalty of 2 microseconds as a result compared to previous UR benchmark CI runs that are not using a docker environment. Please see https://intel.github.io/llvm/benchmarks/?tags=UR&runs=Baseline_PVC_L0%2CCommit5661e7275b64a6c00a96ae295815f40a9384e958_PVC_L0 : this compares performance of latest commit with commit 5661e72, which was the commit used to benchmark the last run result from July 30th.
I am currently looking into ways to reduce this penalty, for now I am opening this PR as the data is still useful. It's worth noting that SYCL only incurred 0.5-1 microsecond penalty from this move, so there may be room for improvement here.