pipelines: Add Kubeflow Pipelines Kubernetes Native API mode documentation#4177
Conversation
google-oss-prow
Bot
added
area/website
VaniHaripriya
force-pushed
the
k8s-native-api-doc-update
branch
from
b761465 to
9e4adab
Compare
VaniHaripriya
changed the title
VaniHaripriya
force-pushed
the
k8s-native-api-doc-update
branch
2 times, most recently
from
6e3901b to
ab82d31
Compare
|
|
||
| ## Deploying Kubeflow Pipelines in Kubernetes Native API Mode | ||
|
|
||
| Kubeflow Pipelines can be deployed in Kubernetes Native API mode, which stores pipeline definitions as Kubernetes Custom Resources instead of using external storage. This mode provides better integration with Kubernetes native tooling and GitOps workflows. |
There was a problem hiding this comment.
| Kubeflow Pipelines can be deployed in Kubernetes Native API mode, which stores pipeline definitions as Kubernetes Custom Resources instead of using external storage. This mode provides better integration with Kubernetes native tooling and GitOps workflows. | |
| Kubeflow Pipelines can be deployed in Kubernetes Native API mode, which stores pipeline definitions as Kubernetes Custom Resources instead of using external DB storage. This mode provides better integration with Kubernetes native tooling and GitOps workflows. |
| > | ||
| > This workaround was verified on Minikube using `v2.0.0`, and aligns with community suggestions in [kubeflow/pipelines#9546](https://github.com/kubeflow/pipelines/issues/9546). It may also help users facing similar pod crash issues in other environments or newer versions. | ||
|
|
||
| ## Deploying Kubeflow Pipelines in Kubernetes Native API Mode |
There was a problem hiding this comment.
Could you add a note that cert-manager is required for this deployment style?
| > | ||
| > This workaround was verified on Minikube using `v2.0.0`, and aligns with community suggestions in [kubeflow/pipelines#9546](https://github.com/kubeflow/pipelines/issues/9546). It may also help users facing similar pod crash issues in other environments or newer versions. | ||
|
|
||
| ## Deploying Kubeflow Pipelines in Kubernetes Native API Mode |
There was a problem hiding this comment.
Could you also make a small note saying something like: "If you are upgrading from a previous version not deployed in Kubernetes native API mode, consider leveraging the migration script to export all existing pipelines and pipeline versions as Kubernetes manifests which can be applied after upgrading Kubeflow Pipelines".
| kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/env/cert-manager/platform-agnostic-k8s-native?ref=$PIPELINE_VERSION" | ||
| ``` | ||
|
|
||
| > 💡 **Alternative**: For multi-user environments, you can use the multi-user Kubernetes native mode: |
There was a problem hiding this comment.
Can we provide a brief explanation on why one would do this, and what is the difference with this mode
VaniHaripriya
force-pushed
the
k8s-native-api-doc-update
branch
from
c6ff615 to
09991b2
Compare
|
|
||
| 1. Deploy the Kubeflow Pipelines in Kubernetes Native Mode: | ||
|
|
||
| ```bash |
There was a problem hiding this comment.
@VaniHaripriya can you take a look and see if the kubeflow manifests repo is updated and contains the required manifests so that we can link their instead of the KFP repo? If the user can deploy from there (instead of github.com/kubeflow/pipelines) then that is preferred since the Kubeflow Docs are tied to Kubeflow releases. If it is not updated, we can stick to kfp repo for now.
There was a problem hiding this comment.
@HumairAK That makes sense. Kubeflow manifests repo has the updated manifests, I will update to use them instead of KFP repo.
There was a problem hiding this comment.
@HumairAK I rechecked the kubeflow/manifests repo and found it currently includes Kubeflow Pipelines 2.5.0, since Kubernetes Native API mode requires version 2.14.0 , we’ll need to continue using the KFP repo for now.
alyssacgoins
left a comment
alyssacgoins
left a comment
There was a problem hiding this comment.
From the perspective of someone who is not as familiar with Kubernetes Native API mode, this documentation makes a lot of sense to me - it's easy to follow and to understand exactly how to replicate these steps.
Thank you @alyssacgoins ! |
…cumentation Signed-off-by: VaniHaripriya <vmudadla@redhat.com>
VaniHaripriya
force-pushed
the
k8s-native-api-doc-update
branch
from
09991b2 to
2ecc28e
Compare
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: mprahl The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
| kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/env/cert-manager/platform-agnostic-k8s-native?ref=$PIPELINE_VERSION" | ||
| ``` | ||
|
|
||
| > 💡 **Alternative**: For multi-user environments with multiple teams or users requiring isolation and RBAC controls on who can access what pipelines, you can use the multi-user Kubernetes native mode: |
There was a problem hiding this comment.
Can we just make it the default in kubeflow platform/manifests or is it not tested enough compared to platform-agnostic-multi-user @HumairAK
…ation (kubeflow#4177) Signed-off-by: VaniHaripriya <vmudadla@redhat.com>
5 participants
Description of Changes
This PR adds the documentation for deploying and using Kubeflow Pipelines in Kubernetes Native API mode.
/area website
/area pipelines
Checklist