Add instructions for deploying with ConfigSync/ACM

[jlewi]

We should add instructions for deploying with ConfigSync/ACM.

ConfigSync can install KCC so we don't have to do that piece.

However, the current version of KCC is too old and incompatible with some of our specs. So we need to wait for the next release of ACM.

[issue-label-bot]

Issue-Label Bot is automatically applying the labels:

Label	Probability
feature	0.95

Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

[jlewi]

Here are some issues I'm running into as I try this out.

#14 - ACM and the iap-enabler pod will conflict trying to set the ingress policy

We will need to create namespace directories in the ACM repo for

• kube-system (some cert-manager resources) get installed there
• istio-system
• kubeflow
• cert-manager

[jlewi]

It looks like we need a tool to chop up the K8s configs and lay them out in the format ACM expects.

[jlewi]

I'm hitting various problems with Kubeflow manifests violating ACM validation requirements.

[2] KNV9999: metadata.annotations must be a map

• This appears to be coming from the manifests generated by ASM; the istio-ingressgateway service in particular appears to be in violation

I'm also seeing errors like the following.

code-intelligence   KNV1052: cluster-scoped resources MUST NOT declare metadata.namespace

source: kubeflow.org_v1beta1_profile_kubeflow-jlewi.yaml
namespace: kubeflow
metadata.name: kubeflow-jlewi
group: kubeflow.org
version: v1beta1
kind: Profile

This appears to be due to kustomize applying the namespace transform even to instances of cluster scoped custom resources. I will need to investigate to see whether there is a way to customize the namespace transform to prevent it from being applied to clusterscoped resources.

[jlewi]

Another alternative would be a kpt transform to fix the resources.

[jlewi]

I didn't see a way to do this with kustomize. I filed
/kubernetes-sigs/kustomize#2498