clusters temporarily disappear during sequential PlacementDecisions update

[haoqing0110]

Describe the bug
When multiple PlacementDecisions for the same Placement are updated sequentially, clusters can temporarily disappear from all PlacementDecisions during the update process.

This causes the addon-management-controller to delete and recreate ManagedClusterAddOns for those clusters, resulting in potential service disruption.

To Reproduce

1. Create a ClusterManagementAddOn with installStrategy.type: Placements
2. Create a Placement that selects >100 clusters (e.g., 150 clusters), resulting in multiple PlacementDecisions
3. Trigger a change that causes clusters to move between PlacementDecisions (e.g., update placement scores or labels)
4. Observe sequential PlacementDecision updates where:

inital status: decision-1: c1, c2 ... , c100 decision-2: c101, c102, ...
update: decision-1: c0, c1, c2 ... , c99 decision-2: c100, c101, c102, ...

5. During the time window between these two updates, c100 exists in neither PlacementDecision
6. The addon-management-controller reconciles during this window and deletes the ManagedClusterAddOn for c100.
7. When decision-2 is updated to include c100, the ManagedClusterAddOn is re-created.

Expected behavior
The ManagedClusterAddOn for c100 should remain unchanged throughout the PlacementDecision updates. The addon should not be deleted and recreated.

Environment ie: OCM version, Kubernetes version and provider:

Additional context
Add any other context about the problem here.

[dosubot]

This is a known issue caused by the way PlacementDecisions are updated sequentially: when clusters move between PlacementDecisions, there can be a brief window where a cluster is missing from all PlacementDecisions. During this window, the addon-management-controller may interpret the cluster as removed and delete its ManagedClusterAddOn, only to recreate it when the PlacementDecisions settle. This can cause unnecessary service disruption.

A recent fix addresses this by batching PlacementDecision update events in the addon-management-controller: a 100ms delay is added to informer events, so rapid sequential updates are processed together. This prevents the controller from acting on intermediate states where a cluster is missing from all PlacementDecisions, and avoids unnecessary deletion/recreation of ManagedClusterAddOns. Delete events are still processed immediately for timely cleanup. An integration test was added to confirm that the ManagedClusterAddOn is no longer deleted and recreated during these updates. You can find the technical details in PR #1344.

To benefit from this fix, ensure your OCM deployment includes this change. There are no user-facing configuration options for the batching delay; it is hardcoded in the controller. After upgrading, ManagedClusterAddOns should remain stable during PlacementDecision updates, even when clusters move between decisions.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}