Skip to content

Fix management blueprint kptfile and stop using namespace mode for CNRM.#1432

Merged
k8s-ci-robot merged 4 commits into
kubeflow:masterfrom
jlewi:fix_management
Jul 30, 2020
Merged

Fix management blueprint kptfile and stop using namespace mode for CNRM.#1432
k8s-ci-robot merged 4 commits into
kubeflow:masterfrom
jlewi:fix_management

Conversation

@jlewi

@jlewi jlewi commented Jul 29, 2020

Copy link
Copy Markdown
Contributor
  • The management blueprint should have its own KptFile

    • Prior to this PR there was only a KptFile at gcp/
    • This doesn't work because for the management cluster we
      only pull the package gcp/v2/management
  • Related to Fix the management blueprint GoogleCloudPlatform/kubeflow-distribution#102

  • Related to Split/Refactor gcp manifests into separate kpt packages GoogleCloudPlatform/kubeflow-distribution#93

  • For CNRM Switch to workload identity and stop using namespace mode for CNRM; Management cluster should not use CNRM in namespaced mode GoogleCloudPlatform/kubeflow-distribution#13

    • Using namespace mode is just extra complexity because we have to install
      a separate copy of the CNRM controller for every project.
      • The only reason to do really do that is if you want to use different
        GCP service accounts to manage different projects. Typically that's
        not what we do.
      • With workload identity we have 1 namespace per project but they
        all use the same GCP SA so the GCP sa can just be authorized to
        access multiple projects or a folder as needed.
  • Update the resources to the v1beta1 spec for use with AnthosCLI

    • It looks like anthoscli requires a NodePool resource
    • With the v1beta1 specs we need to add the annotation gke.cluster.io = "bootstrap://" so that anthoscli is able to probably group the resources.
  • Move cnrm-install iam and services into kustomize packages

    • This way we can hydrate them like we do other manifests
  • Fix the setters and substitutions for CNRM to make them unique per name

    • This way we could potentially have multiple management clusters per project
      which if nothing else will be useful for testing.

* The management blueprint should have its own KptFile
  * Prior to this PR there was only a KptFile at gcp/
  * This doesn't work because for the management cluster we
    only pull the package gcp/v2/management

* Related to GoogleCloudPlatform/kubeflow-distribution#102
* Related to GoogleCloudPlatform/kubeflow-distribution#93

* For CNRM Switch to workload identity and stop using namespace mode for CNRM; GoogleCloudPlatform/kubeflow-distribution#13

  * Using namespace mode is just extra complexity because we have to install
    a separate copy of the CNRM controller for every project.
    * The only reason to do really do that is if you want to use different
      GCP service accounts to manage different projects. Typically that's
      not what we do.
    * With workload identity we have 1 namespace per project but they
      all use the same GCP SA so the GCP sa can just be authorized to
      access multiple projects or a folder as needed.

* Update the resources to the v1beta1 spec for use with AnthosCLI

  * It looks like anthoscli requires a NodePool resource
  * With the v1beta1 specs we need to add the annotation gke.cluster.io = "bootstrap://" so that anthoscli is able to probably group the resources.

* Move cnrm-install iam and services into kustomize packages
  * This way we can hydrate them like we do other manifests

* Fix the setters and substitutions for CNRM to make them unique per name
  * This way we could potentially have multiple management clusters per project
    which if nothing else will be useful for testing.
@kubeflow-bot

Copy link
Copy Markdown
Contributor

This change is Reviewable

@jlewi

jlewi commented Jul 29, 2020

Copy link
Copy Markdown
Contributor Author

/assign @Bobgy

@Bobgy

Bobgy commented Jul 30, 2020

Copy link
Copy Markdown
Contributor

/lgtm
/approve

@jlewi

jlewi commented Jul 30, 2020

Copy link
Copy Markdown
Contributor Author

/approve

@k8s-ci-robot

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Bobgy, jlewi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 4c1221c into kubeflow:master Jul 30, 2020
k8s-ci-robot pushed a commit that referenced this pull request Jul 31, 2020
…stop using namespace #1437: Fix cloudresourcemanager service; missing ApiVersion. Cherry pick of #1432 #1437 on v1.1-branch. #1432: Fix management blueprint kptfile and stop using namespace #1437: Fix cloudresourcemanager service; missing ApiVersion. (#1439)

* Fix management blueprint kptfile and stop using namespace mode for CNRM.

* The management blueprint should have its own KptFile
  * Prior to this PR there was only a KptFile at gcp/
  * This doesn't work because for the management cluster we
    only pull the package gcp/v2/management

* Related to GoogleCloudPlatform/kubeflow-distribution#102
* Related to GoogleCloudPlatform/kubeflow-distribution#93

* For CNRM Switch to workload identity and stop using namespace mode for CNRM; GoogleCloudPlatform/kubeflow-distribution#13

  * Using namespace mode is just extra complexity because we have to install
    a separate copy of the CNRM controller for every project.
    * The only reason to do really do that is if you want to use different
      GCP service accounts to manage different projects. Typically that's
      not what we do.
    * With workload identity we have 1 namespace per project but they
      all use the same GCP SA so the GCP sa can just be authorized to
      access multiple projects or a folder as needed.

* Update the resources to the v1beta1 spec for use with AnthosCLI

  * It looks like anthoscli requires a NodePool resource
  * With the v1beta1 specs we need to add the annotation gke.cluster.io = "bootstrap://" so that anthoscli is able to probably group the resources.

* Move cnrm-install iam and services into kustomize packages
  * This way we can hydrate them like we do other manifests

* Fix the setters and substitutions for CNRM to make them unique per name
  * This way we could potentially have multiple management clusters per project
    which if nothing else will be useful for testing.

* Add workload identity pool to the management cluster.

* Management nodepool should set workloadMetadataConfig so that we run the workload identity servers.

* Fix.

* Fix cloudresourcemanager service; missing ApiVersion.

Related to: GoogleCloudPlatform/kubeflow-distribution#102
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants