fix(pipelinerun): fix the issue of massive invalid status updates caused by unordered arrays, which will greatly impact the resource load and stability of the apiserver. #9295
+15
−3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…sed by unordered arrays, which will greatly impact the resource load and stability of the apiserver.
tekton-robot
added
the
release-note-none
tekton-robot
added
the
size/S
Contributor
Author
|
/kind bug |
tekton-robot
added
kind/bug
khrm
reviewed
Jan 19, 2026
Contributor
khrm
left a comment
khrm
left a comment
There was a problem hiding this comment.
Can you remove cmpopts.SortSlices from pkg/reconciler/pipelinerun/pipelinerun_updatestatus_test.go? It's not needed anymore and will act as a test..
…elinerun/pipelinerun_updatestatus_test.go
Contributor
Author
Nice suggestion. I removed the unnecessary sorting in pipelinerun_updatestatus_test.go to ensure the consistency of this modification. |
Contributor
|
/lgtm /assign @waveywaves @vdemeester |
vdemeester
approved these changes
Jan 19, 2026
Member
vdemeester
left a comment
vdemeester
left a comment
There was a problem hiding this comment.
/hold
To let @afrittoli @waveywaves and @twoGiants to look into it. Thank you for this PR !
I initially thought it could be a problem from the API point of view, but as there was no guarantee it was sorted before..
tekton-robot
added
the
do-not-merge/hold
Collaborator
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: vdemeester The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
tekton-robot
added
the
approved
Member
|
/hold cancel |
tekton-robot
removed
the
do-not-merge/hold
Member
|
/cherry-pick release-v1.0.x release-v1.3.x release-v1.6.x |
Collaborator
|
✅ Cherry-pick to A new pull request has been created to cherry-pick this change to Please review and merge the cherry-pick PR. |
This was referenced Jan 27, 2026
Collaborator
|
✅ Cherry-pick to A new pull request has been created to cherry-pick this change to Please review and merge the cherry-pick PR. |
Collaborator
|
✅ Cherry-pick to A new pull request has been created to cherry-pick this change to Please review and merge the cherry-pick PR. |
Member
|
/cherry-pick release-v1.6.x |
Collaborator
|
❌ Cherry-pick to The automatic cherry-pick to Output: Next steps:
|
vdemeester
added a commit
to vdemeester/tektoncd-pipeline
that referenced
this pull request
Feb 4, 2026
The previous cherry-pick (tektoncd#9314) only included the test file changes (removing cmpopts.SortSlices) but not the actual sorting logic in pipelinerun.go. This caused tests to fail because they expected sorted output but the production code wasn't sorting. This commit adds the missing sort.Slice call that sorts childRefs by PipelineTaskName, then Name, then Kind to prevent excessive status updates caused by unordered arrays. Cherry-pick of the missing part from tektoncd#9295. Signed-off-by: Vincent Demeester <[email protected]> Co-Authored-By: Claude <[email protected]>
vdemeester
added a commit
to vdemeester/tektoncd-pipeline
that referenced
this pull request
Feb 4, 2026
The previous cherry-pick (tektoncd#9312) only included the test file changes (removing cmpopts.SortSlices) but not the actual sorting logic in pipelinerun.go. This caused tests to fail because they expected sorted output but the production code wasn't sorting. This commit adds the missing sort.Slice call that sorts childRefs by PipelineTaskName, then Name, then Kind to prevent excessive status updates caused by unordered arrays. Cherry-pick of the missing part from tektoncd#9295. Signed-off-by: Vincent Demeester <[email protected]> Co-Authored-By: Claude <[email protected]>
This was referenced Feb 4, 2026
vdemeester
added a commit
that referenced
this pull request
Feb 4, 2026
The test file changes from #9295 were cherry-picked but the actual sorting logic in pipelinerun.go was missing. This adds the sort.Slice call that ensures childRefs are sorted by PipelineTaskName, then Name, then Kind, preventing excessive apiserver status updates caused by non-deterministic map iteration order. Co-Authored-By: Claude Opus 4.5 <[email protected]>
vdemeester
added a commit
that referenced
this pull request
Feb 4, 2026
The test file changes from #9295 were cherry-picked but the actual sorting logic in pipelinerun.go was missing. This adds the sort.Slice call that ensures childRefs are sorted by PipelineTaskName, then Name, then Kind, preventing excessive apiserver status updates caused by non-deterministic map iteration order. Co-Authored-By: Claude Opus 4.5 <[email protected]>
vdemeester
added a commit
that referenced
this pull request
Feb 4, 2026
The test file changes from #9295 were cherry-picked but the actual sorting logic in pipelinerun.go was missing. This adds the sort.Slice call that ensures childRefs are sorted by PipelineTaskName, then Name, then Kind, preventing excessive apiserver status updates caused by non-deterministic map iteration order. Co-Authored-By: Claude Opus 4.5 <[email protected]>
vdemeester
added a commit
that referenced
this pull request
Feb 4, 2026
The test file changes from #9295 were cherry-picked but the actual sorting logic in pipelinerun.go was missing. This adds the sort.Slice call that ensures childRefs are sorted by PipelineTaskName, then Name, then Kind, preventing excessive apiserver status updates caused by non-deterministic map iteration order. Co-Authored-By: Claude Opus 4.5 <[email protected]>
tekton-robot
pushed a commit
that referenced
this pull request
Feb 4, 2026
The test file changes from #9295 were cherry-picked but the actual sorting logic in pipelinerun.go was missing. This adds the sort.Slice call that ensures childRefs are sorted by PipelineTaskName, then Name, then Kind, preventing excessive apiserver status updates caused by non-deterministic map iteration order. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
We deployed Tekton/Pipeline in a full DevOps environment, creating/deleting approximately four million resources weekly. As business complexity grows, the cluster accumulates large PipelineRun objects (Child TaskRuns > 64), causing severe stability issues with apiserver memory spikes and high write QPS for PipelineRun objects. We identified that the unsorted childRef array in PipelineRun status triggered massive redundant writes. Implementing array sorting significantly mitigates this issue.
Submitter Checklist
As the author of this PR, please check off the items in this checklist:
/kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tepRelease Notes