feat: KFP multi user mode PR2 - secure KFP with istio mTLS and authz#1368
Merged
Conversation
Contributor
|
This change is |
Bobgy
changed the title
Contributor
Author
|
/assign @jlewi I still have some small changes left, but maybe you can review this first. |
Bobgy
commented
Jul 9, 2020
| - ml-pipeline-ui.kubeflow.svc.cluster.local | ||
| --- | ||
| apiVersion: "rbac.istio.io/v1alpha1" | ||
| kind: ServiceRole |
Contributor
Author
There was a problem hiding this comment.
Note service role is deprecated for istio, but they still work in istio 1.4.x.
I'd prefer leaving this here for a while and migrate when all platforms upgrade to later istio versions.
Bobgy
changed the title
Bobgy
changed the title
Contributor
Author
|
OK, this is ready |
Bobgy
changed the title
Bobgy
changed the title
Bobgy
changed the title
Contributor
|
/lgtm |
Contributor
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: jlewi The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Bobgy
added a commit
to Bobgy/manifests
that referenced
this pull request
Jul 14, 2020
…ubeflow#1368) * Add argo to stacks/generic * Pull pipelines manifest from upstream * Updated kfp * Minio v3 manifests * Rename minio configmap * Add generic minio install * Generate new test data * Mysql kustomize v3 manifest - generic install * Add mysql gcp pd install * Generate test data * Pipelines kustomize v3 manifests * Add kfp ui virtual service * Add metadata deployment to stacks/generic * Use common cluster domain * Deploy metadata writer * Add kfp cache server * Update test data * Enable KFP multi user mode without istio security * Fix persistence agent watch namespace * Fix namespace env for some deployments * Fix cluster roles and bindings * fix rename * Fix pipelines ui role * Updated kfp to rc2 * simplify pipeline v3 manifest using updated kfp rc2 manifest * Fix pipeline-install-config * remove redundant configmap * update tests * updated to kfp 1.0.0-rc.3 * Adapt to kfp 1.0rc3 refactoring * update test snapshots * fix pull kfp script to detect empty dir * fix example ref * update snapshot * fix gcp pd manifest * Update stacks ref * revert alice example to gcp stack * update snapshot * fix profile controller iam binding * Update kfp profile controller can be configured to different images and istio sidecar * add missing viewer controller cluster roles * Use python3 for sync.py * Revert gcp stack back to use non multi user kfp * revert unintended changes * revert upstream changes * Secure kfp multi user mode with istio authorization * patch minio to disable istio sidecar injection * fix cache server istio authz * enable istio sidecar for profiles deploy * enable istio sidecar for centraldashboard * Do not protect profile controller with istio * Allow admission webhook traffic to cache-server * revert gcp stack back to pipeline generic * Reuse minio generic install as base for gcp-pd and ibm * update snapshot
k8s-ci-robot
pushed a commit
that referenced
this pull request
Jul 14, 2020
* feat: KFP multi user mode PR1 - enable multi user mode without istio authorization (#1342) * Add argo to stacks/generic * Pull pipelines manifest from upstream * Updated kfp * Minio v3 manifests * Rename minio configmap * Add generic minio install * Generate new test data * Mysql kustomize v3 manifest - generic install * Add mysql gcp pd install * Generate test data * Pipelines kustomize v3 manifests * Add kfp ui virtual service * Add metadata deployment to stacks/generic * Use common cluster domain * Deploy metadata writer * Add kfp cache server * Update test data * Enable KFP multi user mode without istio security * Fix persistence agent watch namespace * Fix namespace env for some deployments * Fix cluster roles and bindings * fix rename * Fix pipelines ui role * Updated kfp to rc2 * simplify pipeline v3 manifest using updated kfp rc2 manifest * Fix pipeline-install-config * remove redundant configmap * update tests * updated to kfp 1.0.0-rc.3 * Adapt to kfp 1.0rc3 refactoring * update test snapshots * fix pull kfp script to detect empty dir * fix example ref * update snapshot * fix gcp pd manifest * Update stacks ref * revert alice example to gcp stack * update snapshot * fix profile controller iam binding * Update kfp profile controller can be configured to different images and istio sidecar * add missing viewer controller cluster roles * Use python3 for sync.py * Revert gcp stack back to use non multi user kfp * revert unintended changes * revert upstream changes * Use kubeflow userid header and prefix config for KFP servers (#1365) * feat: KFP multi user mode PR2 - secure KFP with istio mTLS and authz (#1368) * Add argo to stacks/generic * Pull pipelines manifest from upstream * Updated kfp * Minio v3 manifests * Rename minio configmap * Add generic minio install * Generate new test data * Mysql kustomize v3 manifest - generic install * Add mysql gcp pd install * Generate test data * Pipelines kustomize v3 manifests * Add kfp ui virtual service * Add metadata deployment to stacks/generic * Use common cluster domain * Deploy metadata writer * Add kfp cache server * Update test data * Enable KFP multi user mode without istio security * Fix persistence agent watch namespace * Fix namespace env for some deployments * Fix cluster roles and bindings * fix rename * Fix pipelines ui role * Updated kfp to rc2 * simplify pipeline v3 manifest using updated kfp rc2 manifest * Fix pipeline-install-config * remove redundant configmap * update tests * updated to kfp 1.0.0-rc.3 * Adapt to kfp 1.0rc3 refactoring * update test snapshots * fix pull kfp script to detect empty dir * fix example ref * update snapshot * fix gcp pd manifest * Update stacks ref * revert alice example to gcp stack * update snapshot * fix profile controller iam binding * Update kfp profile controller can be configured to different images and istio sidecar * add missing viewer controller cluster roles * Use python3 for sync.py * Revert gcp stack back to use non multi user kfp * revert unintended changes * revert upstream changes * Secure kfp multi user mode with istio authorization * patch minio to disable istio sidecar injection * fix cache server istio authz * enable istio sidecar for profiles deploy * enable istio sidecar for centraldashboard * Do not protect profile controller with istio * Allow admission webhook traffic to cache-server * revert gcp stack back to pipeline generic * Reuse minio generic install as base for gcp-pd and ibm * update snapshot * refactor: pipelines profile controller should get minio access keys from the secret (#1372) * refactor: pipelines profile controller should get minio access keys from the secret * do not print secrets in log * feat: Use KFP multi user mode for GCP (#1373) * refactor: pipelines profile controller should get minio access keys from the secret * do not print secrets in log * use kfp multi user mode for gcp stacks * update snapshot * feat: Add application and common labels to KFP and various fixes (#1374) * Add common labels to kfp components * Add KFP application * update snapshot * Use json format for json patch, because yaml will look like a resource and fail tests * Remove part of label * update snapshots * Fix profile controller deployment version * update snapshot * Fix userid-header for gcp * update snapshot * Fix b64encode exception * update snapshot * update snapshot
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue is resolved by this Pull Request:
Resolves kubeflow/pipelines#4160
Description of your changes:
Secure KFP multi user mode with istio mutual TLS and authz.
Note, this doesn't let any stacks use kfp multi user mode yet. I'll update gcp stack in a separate PR.
Limitation: as discussed in kubeflow/kubeflow#4865 (comment), we don't protect kfam with istio sidecar. The implication is any workload in the cluster can send requests to kfam to change contributors for namespaces. The security risk will be removed when kfam's responsibility of managing permissions will be removed. (likely not in KF 1.1)
Checklist:
cd manifests/testsmake generate-changed-onlymake test