cert-manager namespace is missing the control plane label
https://github.com/kubeflow/manifests/blob/011c81ec2be8875d8032b79b62340ab924258230/namespaces/base/namespaces.yaml#L12
This can cause deadlock because cert-manager pods might not get deployed because the KFServing admission webhooks aren't valid because they can't get a certificate because cert-manager isn't running.
I think we haven't observed this because we deploy cert-manager first.
However, when deploying with GitOps (ACM) I hit this problem.
We should cherry-pick the fix onto the 1.1 branch before closing this issue.
cert-manager namespace is missing the control plane label
https://github.com/kubeflow/manifests/blob/011c81ec2be8875d8032b79b62340ab924258230/namespaces/base/namespaces.yaml#L12
This can cause deadlock because cert-manager pods might not get deployed because the KFServing admission webhooks aren't valid because they can't get a certificate because cert-manager isn't running.
I think we haven't observed this because we deploy cert-manager first.
However, when deploying with GitOps (ACM) I hit this problem.
We should cherry-pick the fix onto the 1.1 branch before closing this issue.