Explore the effort for CKF 1.10.1 upgrade

[NohaIhab]

Context

Upstream has released a patch for 1.10 version in https://github.com/kubeflow/manifests/releases/tag/v1.10.1. We need to take a good look at the changes and estimate what it will take to upgrade Charmed Kubeflow.

What needs to get done

1. Look into the diff between 1.10 and 1.10.1 upstream
2. Note down the changes in the different components
3. Break down the effort needed to make the upgrade

Definition of Done

Kubeflow 1.10.1 effort is estimated

[syncronize-issues-to-jira]

Thank you for reporting your feedback to us!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-7509.

This message was autogenerated

[NohaIhab]

1.10.1 Changes

Looking at the diff for the manifests, I observed the following changes:

New Component versions

• Training Operator v1.9.2 Synchronize training operator manifests v1.9.2 kubeflow/community-distribution#3126, changelog
• Kubeflow Pipelines v2.5.0 Synchronize kubeflow pipelines manifests 2.5.0 kubeflow/community-distribution#3121, changelog, 2.4.1 vs 2.5.0
• Argo v3.5.14 (done as part of KFP upgrade)
• KServe v0.15.0 Synchronize kserve manifests v0.15.0 and update tests kubeflow/community-distribution#3101, changelog
• Model Registry v0.2.16 Update kubeflow/model-registry manifests from v0.2.16 kubeflow/community-distribution#3087 -> To be done in KF-7319

Integrations

Note: the integrations are just for the record, but not in our current roadmap

• Spark Operator
  • Update to 2.1.1 Update kubeflow/spark-operator manifests to 2.1.1 kubeflow/community-distribution#3102
  • Enabling Spark by default in KF Enable Spark Operator by default kubeflow/community-distribution#3131
• Integrating kfp profile controller with seaweedfs Integrate profiles with seaweedfs kubeflow/community-distribution#3051
  • effect of the change: Use ACLs in seaweedfs to isolate access to artifacts by profile

[NohaIhab]

1.10.1 Effort breakdown

Component updates

Training Operator v1.9.2

• update manifests (0.5 points)
• update rock (0.5 points)
• integrate rock (0.5 points)

Kubeflow Pipelines v2.5.0

• update manifests (4 points for all charms)
• update rocks (0.5 points * 10 rocks = 5 points)
• integrate rocks (0.5 points * 10 rocks = 5 points)

Argo v3.5.14

• update manifests (0.5 points)
• update rocks (0.5 points * 2 rocks = 1 point)
• integrate rocks (0.5 points * 2 rocks = 1 point)

KServe v0.15.0

• update manifests (0.5 points)
• update rocks (0.5 points * 12 rocks = 6 points)
• integrate rocks (0.5 points * 12 rocks = 6 points)
• explore the added huggingface-multinode ClusterServingRuntime, we should check whether we can have a rock, new test, etc. (0.5 points)

Documentation

• Upgrade guide from 1.10 to 1.10.1 (1 point)

Terraform modules update

• Canon run to update the channel used on all branches used by 1.10 (2 points)
• Update terraform module in track/1.10 branch (1 point)

UATs update

• Update the UATs dependencies (including SDKs) to 1.10.1 versions (2 points)

Total estimate

37 points

[kimwnasptd]

Nice presentation of the info! I also slightly edited the massage to link to the changelog also, in case anyone might also want to take a look (of the components).

@NohaIhab my only suggestion would be to allocate a bit more time into KServe. It looks like they have added a new ClusterServingRuntime for huggingface-multinode. Although the OCI image mentions gpu so most probably we won't need a new image, but would allocate half a day for this exploration

[NohaIhab]

thanks for reviewing @kimwnasptd, good point on the added ClusterServingRuntime! I've updated the comment accordingly.