Add documentation for Kubeflow Pipelines Kubernetes Native API mode

README.md

@@ -406,13 +406,38 @@ kustomize build common/istio/kubeflow-istio-resources/base | kubectl apply -f -
 
 #### Kubeflow Pipelines
 
+Kubeflow Pipelines offers two deployment options to choose from, each designed for different use cases and operational preferences. The traditional database-based approach stores pipeline definitions in an external database, while the Kubernetes native API mode leverages Kubernetes custom resources for pipeline definition storage and management.
+
+
+##### Pipeline Definitions Stored in the Database
+
 Install the [Multi-User Kubeflow Pipelines](https://www.kubeflow.org/docs/components/pipelines/multi-user/) official Kubeflow component:
 
 ```sh
 kustomize build applications/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user | kubectl apply -f -
 ```
 This installs Argo with the runasnonroot emissary executor. Please note that you are still responsible for analyzing the security issues that arise when containers are run with root access and for deciding if the Kubeflow pipeline main containers are run as runasnonroot. It is generally strongly recommended that all user-accessible OCI containers run with Pod Security Standards [restricted](https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted).
 
+##### Pipeline Definitions Stored as Kubernetes Resources
+
+Kubeflow Pipelines can be deployed in Kubernetes native API mode, which stores pipeline definitions as Kubernetes custom resources (`Pipeline` and `PipelineVersion` kinds) instead of using external storage. This mode provides better integration with Kubernetes native tooling and GitOps workflows.
+
+```sh
+kustomize build applications/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user-k8s-native | kubectl apply -f -
+```
+
+**Using the KFP SDK with Kubernetes Native API Mode:**
+
+For detailed pipeline compilation instructions, please refer to the [Kubeflow Pipelines compilation guide](https://www.kubeflow.org/docs/components/pipelines/user-guides/core-functions/compile-a-pipeline/#compiling-for-kubernetes-native-api-mode).
+
+**Differences in Kubernetes Native API Mode:**
+
+- Pipeline definitions are stored as `Pipeline` and `PipelineVersion` custom resources in Kubernetes.
+- Pipeline validation is handled through Kubernetes admission webhooks.
+- The REST API transparently handles the translation to Kubernetes API calls.
+
+**Benefits of Kubernetes Native Mode**: This approach is ideal for organizations that prefer Kubernetes-native workflows and want to manage pipelines using standard Kubernetes tools and practices. Pipeline definitions can be managed through multiple interfaces: direct kubectl commands, the Kubeflow Pipelines REST API, and the KFP UI for user-friendly pipeline management.
+
 #### KServe
 
 KFServing was rebranded to KServe.
@@ -721,6 +746,3 @@ pre-commit run
   **A:** Istio CNI provides better security by eliminating the need for privileged init containers, making it more compatible with Pod Security Standards (PSS). It also enables native sidecars support introduced in Kubernetes 1.28, which helps address issues with init containers and application lifecycle management.
 - **Q:** Why does Istio CNI fail on Google Kubernetes Engine (GKE) with "read-only file system" errors?
   **A:** GKE mounts `/opt/cni/bin` as read-only for security reasons, preventing the Istio CNI installer from writing the CNI binary. Use the GKE-specific overlay: `kubectl apply -k common/istio/istio-install/overlays/gke`. This overlay uses GKE's writable CNI directory at `/home/kubernetes/bin`. For more details, see [Istio CNI Prerequisites](https://istio.io/latest/docs/setup/additional-setup/cni/#prerequisites) and [Platform Prerequisites](https://istio.io/latest/docs/ambient/install/platform-prerequisites/).-`
-
-
-

example/kustomization.yaml

@@ -42,15 +42,15 @@
 - ../common/istio/istio-install/overlays/oauth2-proxy
 # NOTE: For Google Kubernetes Engine (GKE), use:
 # - ../common/istio/istio-install/overlays/gke
 #GKE mounts `/opt/cni/bin` as read-only for security reasons, preventing the Istio CNI installer from writing the CNI binary.
 #Use the GKE-specific overlay: `kubectl apply -k common/istio/istio-install/overlays/gke`.
 #This overlay uses GKE's writable CNI directory at `/home/kubernetes/bin`.
 #For more details, see [Istio CNI Prerequisites](https://istio.io/latest/docs/setup/additional-setup/cni/#prerequisites) and [Platform Prerequisites](https://istio.io/latest/docs/ambient/install/platform-prerequisites/)
 # oauth2-proxy
 # NOTE: only uncomment ONE of the following overlays, depending on your cluster type
 - ../common/oauth2-proxy/overlays/m2m-dex-only     # for all clusters
 #- ../common/oauth2-proxy/overlays/m2m-dex-and-kind # for KIND clusters (allows K8S JWTs for gateway auth)
 #- ../common/oauth2-proxy/overlays/m2m-dex-and-eks  # for EKS clusters (NOTE: requires you to configure issuer, see overlay)
 # Dex
 - ../common/dex/overlays/oauth2-proxy
 # KNative
@@ -67,7 +67,8 @@
 # Kubeflow Istio Resources
 - ../common/istio/kubeflow-istio-resources/base
 # Kubeflow Pipelines
-- ../applications/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user
+- ../applications/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user # Pipeline Definitions Stored in the Database
+# - ../applications/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user-k8s-native # Pipeline Definitions Stored as Kubernetes Resources
 # Katib
 - ../applications/katib/upstream/installs/katib-with-kubeflow
 # Central Dashboard
@@ -79,7 +80,7 @@
 # Notebook Controller
 - ../applications/jupyter/notebook-controller/upstream/overlays/kubeflow
 # Profiles + KFAM with PSS (Pod Security Standards)
 - ../applications/profiles/pss # TODO MUST BE UPSTREAMED
 # PVC Viewer
 - ../applications/pvcviewer-controller/upstream/base
 # Volumes Web App