Use patch instead of update to replace sidecars with nop image

[prad9192]

What

Changed StopSidecars() to use Patch() instead of Update() when stopping sidecar containers by replacing their images with the nop image.

Why

The original implementation used Update(), which requires an exact resourceVersion match. This causes 409 conflicts when the kubelet updates the pod status between our GET and UPDATE calls (which happens frequently as containers terminate).

This causes the below errors on the pod.

error stopping sidecars of Pod: Operation cannot be fulfilled on pods "...": 
the object has been modified; please apply your changes to the latest version

And the task runs fail with TaskRunResolutionFailed

Even though the task succeeded and sidecars eventually stop, the TaskRun gets marked as failed due to this race condition.
The fix uses JSON Patch (same pattern as UpdateReady() and CancelPod() in this file), which only patches the specific container image fields

Release Notes

Does this PR introduce a user-facing change?

Fixed race condition causing TaskRuns to fail with 409 conflict error when stopping sidecars. 
StopSidecars now uses Patch instead of Update to avoid conflicts with concurrent kubelet pod status updates.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: prad9192 / name: Pradeep Ashwathanarayan (ea7d2ca)

[prad9192]

/retest

[vdemeester]

@prad9192 you will need to rebase against main branch. The e2e failure are fixed in the latest main commits.

[prad9192]

/label kind/bug

[tekton-robot]

@prad9192: The label(s) /label kind/bug cannot be applied. These labels are supported: ``

Details

In response to this:

/label kind/bug

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[prad9192]

/retest

[AlanGreene]

/kind bug

[afrittoli]

Thanks for this fix, it's much appreciated!
Could you add some testing for the new functionality? It should be possible to simulate the situation you described in the PR description of the pod being updated.

[prad9192]

Thanks for this fix, it's much appreciated! Could you add some testing for the new functionality? It should be possible to simulate the situation you described in the PR description of the pod being updated.

Thank you for the feedback, as requested the tests are added, kindly review.
cc: @afrittoli

[afrittoli]

Thanks a lot for the update, lgtm!

One last ask before it's approved - we don't use merge commits in Tekton, could you please squash your PR to a single commit? Thank you!

[prad9192]

Thanks a lot for the update, lgtm!

One last ask before it's approved - we don't use merge commits in Tekton, could you please squash your PR to a single commit? Thank you!

Done, could you please review, when you get a chance?

[afrittoli]

/approve

[vdemeester]

/lgtm

[tekton-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: afrittoli, vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [afrittoli,vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[vdemeester]

/retest

[vdemeester]

/retest