Simplified the metrics by kunal-511 · Pull Request #3182 · kubeflow/community-distribution

kunal-511 · 2025-07-03T17:46:51Z

Pull Request Template for Kubeflow Manifests

✏️ Summary of Changes

Separated Knative from Istio
Created Storage & Experimental For SeaweedFS, Ray, and experimental components

✅ Contributor Checklist

I have tested these changes with kustomize. See Installation Prerequisites.
All commits are signed-off to satisfy the DCO check.
I have considered adding my company to the adopters page to support Kubeflow and help the community, since I expect help from the community for my issue (see 1. and 2.).

You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

juliusvonkohout · 2025-07-04T14:32:46Z

+    print("- **Other**: Miscellaneous Kubeflow components including:")
+    print("  - PVC Viewer Controller")
+    print("  - Tensorboard Controller and Web App")
+    print("  - Multi-tenancy and RBAC components")
+    print("  - Network policies")
+    print("  - Custom resources and CRDs")
+    print("  - User namespace resources")
+    print("  - Additional Kubeflow utilities and tools")


We have the components here https://github.com/kubeflow/manifests#kubeflow-components-versions . It should be covered in the same detail and not under other. i would even just extend and merge the two tables with more columns and remove https://github.com/kubeflow/manifests#resource-usage-by-components

networkpolicies and other things without CPU/MEMORY/STORAGE you can ignore anyway.

Please update the table in the main readme after fixing the other open issues

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

juliusvonkohout · 2025-07-05T08:55:04Z

+        'keywords': ['pvcviewer']
+    },
+    'Storage & Experimental': {
+        'keywords': ['seaweedfs', 'ray', 'minio-tenant']


What do you mean by minio-tenant? I think you can just ignore the experimental folder for now entirely.

Seaweedfs can also be counted to kfp if needed.

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

juliusvonkohout · 2025-07-08T14:34:31Z

 # Storage fallback values when YAML parsing is unavailable
 STORAGE_FALLBACK = {
    'Katib': 3,
    'Metadata': 40,


Metadata also belongs to KFP. And please check the storage numbers manually. I remember less than 30 GB in total or so. Since it is either minio XOR seawedfs please do not count them twice.

I checked it manually it is 25

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

juliusvonkohout · 2025-07-14T17:55:29Z

-| Istio | 850m | 2464Mi | 0GB |
-| Katib | 4m | 107Mi | 3GB |
-| Kubeflow Core | 17m | 828Mi | 0GB |
+| Training Operator | 2m | 27Mi | 0GB |


As discussed please remove this table and merge the columns with the existing table at the beginning of the document.

Delete this section https://github.com/kubeflow/manifests#resource-usage-by-components and add it instead here in this table https://github.com/kubeflow/manifests#kubeflow-version-master

I think kserve and a few other components are missing numbers in the table and let's merge istio, Knative etc. Into the same table.

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

juliusvonkohout · 2025-07-15T14:30:24Z

please separate out Dex and oauth2-proxy in a follow up PR. I already separated it in the table in the readme

Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

juliusvonkohout · 2025-07-15T14:34:19Z

ah and the numbers for Tensorboards Web Application are missing in the table

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

juliusvonkohout · 2025-07-15T14:40:40Z

+| Profiles + KFAM | applications/profiles/upstream | [v1.10.0](https://github.com/kubeflow/kubeflow/tree/v1.10.0/components/profile-controller/config) | 7m | 129Mi | 0GB |
+| PodDefaults Webhook | applications/admission-webhook/upstream | [v1.10.0](https://github.com/kubeflow/kubeflow/tree/v1.10.0/components/admission-webhook/manifests) | 1m | 14Mi | 0GB |
+| Jupyter Web Application | applications/jupyter/jupyter-web-app/upstream | [v1.10.0](https://github.com/kubeflow/kubeflow/tree/v1.10.0/components/crud-web-apps/jupyter/manifests) | 4m | 231Mi | 0GB |
+| Tensorboards Web Application | applications/tensorboard/tensorboards-web-app/upstream | [v1.10.0](https://github.com/kubeflow/kubeflow/tree/v1.10.0/components/crud-web-apps/tensorboards/manifests) |  |  |  |


where are the values for tensorboards?

juliusvonkohout · 2025-07-15T14:40:57Z

+| Katib | applications/katib/upstream | [v0.18.0](https://github.com/kubeflow/katib/tree/v0.18.0/manifests/v1beta1) | 13m | 476Mi | 13GB |
+| KServe | applications/kserve/kserve | [v0.15.0](https://github.com/kserve/kserve/releases/tag/v0.15.0/install/v0.15.0) | 600m | 1200Mi | 0GB |
+| KServe Models Web Application | applications/kserve/models-web-app | [v0.14.0](https://github.com/kserve/models-web-app/tree/v0.14.0/config) | 6m | 259Mi  | 0GB |
+| Kubeflow Pipelines | applications/pipeline/upstream | [2.5.0](https://github.com/kubeflow/pipelines/tree/2.5.0/manifests/kustomize) | 970m | 3552Mi | 100GB |


100 GB for pipelines does not seem correct

juliusvonkohout · 2025-07-15T14:41:16Z

+| KServe | applications/kserve/kserve | [v0.15.0](https://github.com/kserve/kserve/releases/tag/v0.15.0/install/v0.15.0) | 600m | 1200Mi | 0GB |
+| KServe Models Web Application | applications/kserve/models-web-app | [v0.14.0](https://github.com/kserve/models-web-app/tree/v0.14.0/config) | 6m | 259Mi  | 0GB |
+| Kubeflow Pipelines | applications/pipeline/upstream | [2.5.0](https://github.com/kubeflow/pipelines/tree/2.5.0/manifests/kustomize) | 970m | 3552Mi | 100GB |
+| Kubeflow Model Registry | applications/model-registry/upstream | [v0.2.19](https://github.com/kubeflow/model-registry/tree/v0.2.19/manifests/kustomize) | 510m | 2112Mi | 20GB |


is 20 GB for model registry correct ?

juliusvonkohout · 2025-07-15T14:45:02Z

+                if kind == 'PersistentVolumeClaim':
+                    storage_str = doc.get('spec', {}).get('resources', {}).get('requests', {}).get('storage', '0')
+                    storage_gb = parse_resource_value(storage_str, 'storage')
+                    component_storage[component] += storage_gb


please compare the storage with the live pvcs on the cluster. 100 GB for pipeline is not right.

juliusvonkohout · 2025-07-15T14:45:40Z

/lgtm
/approve

google-oss-prow · 2025-07-15T14:45:48Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: juliusvonkohout

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [juliusvonkohout]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

juliusvonkohout · 2025-07-15T14:46:29Z

For the follow up PR:

please compare the storage with the live pvcs on the cluster. 100 GB for pipeline is not right.
100 GB for pipelines does not seem correct
is 20 GB for model registry correct ?
Total 134 GB doe snot look correct
where are the values for tensorboards?

Simplified the metrics

e5e4c14

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

google-oss-prow Bot requested review from juliusvonkohout and kimwnasptd July 3, 2025 17:46

google-oss-prow Bot added the size/XL label Jul 3, 2025

juliusvonkohout reviewed Jul 4, 2025

View reviewed changes

Updated metrics table

53b21e4

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

juliusvonkohout reviewed Jul 5, 2025

View reviewed changes

Ignoring experimental folder

85bc412

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

juliusvonkohout reviewed Jul 8, 2025

View reviewed changes

kunal-511 added 2 commits July 9, 2025 18:29

Updated storage fallback

60ca6c0

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

Readme updated with new meteics numbers

fce44ce

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

juliusvonkohout reviewed Jul 14, 2025

View reviewed changes

kunal-511 and others added 6 commits July 15, 2025 00:04

Updated the component naming

5f47bec

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

Removed old resources table

6b79ae7

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

Merged resources and kubeflow component version table

0ffb275

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

Merge to single table

a3df80f

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

added kserve web applications resources values

95b3670

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

Update README.md

76e7a3a

Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

juliusvonkohout reviewed Jul 15, 2025

View reviewed changes

Update README.md

842d33b

Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>

Seperate Dex and OAuth

e32588c

Signed-off-by: kunal-511 <yoyokvunal@gmail.com>

juliusvonkohout reviewed Jul 15, 2025

View reviewed changes

google-oss-prow Bot assigned juliusvonkohout Jul 15, 2025

google-oss-prow Bot added the lgtm label Jul 15, 2025

google-oss-prow Bot added the approved label Jul 15, 2025

google-oss-prow Bot merged commit a4bf1ee into kubeflow:master Jul 15, 2025
7 of 8 checks passed

kunal-511 mentioned this pull request Jul 17, 2025

Live PVC storage #3188

Merged

3 tasks

Conversation

kunal-511 commented Jul 3, 2025

Pull Request Template for Kubeflow Manifests

✏️ Summary of Changes

✅ Contributor Checklist

Uh oh!

juliusvonkohout Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kunal-511 Jul 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juliusvonkohout commented Jul 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juliusvonkohout commented Jul 15, 2025

Uh oh!

google-oss-prow Bot commented Jul 15, 2025

Uh oh!

juliusvonkohout commented Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

juliusvonkohout Jul 4, 2025 •

edited

Loading

kunal-511 Jul 9, 2025 •

edited

Loading

juliusvonkohout commented Jul 15, 2025 •

edited

Loading