update spark operator#3379
Conversation
Signed-off-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Signed-off-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
|
@tarekabouzeid for lgtm |
juliusvonkohout
changed the title
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: juliusvonkohout The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Pull request overview
Updates the Spark Operator synchronization workflow and bumps the rendered Spark Operator manifests to the v2.5.0-rc.0 upstream revision.
Changes:
- Simplifies
scripts/synchronize-spark-operator-manifests.shto clone the upstream repo and render Helm output directly into the targetresources.yamlpath (removing script-local bootstrap/kustomization generation). - Updates Spark Operator rendered manifests (
resources.yaml) to2.5.0-rc.0output, including updated labels/images and RBAC/controller args from the newer chart. - Updates the top-level
README.mdcomponent version matrix for Spark Operator.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| scripts/synchronize-spark-operator-manifests.sh | Switches to cloning upstream + direct render to applications/spark/spark-operator/base/resources.yaml, and updates README via shared helper. |
| applications/spark/spark-operator/base/resources.yaml | Regenerated Spark Operator manifests for 2.5.0-rc.0. |
| README.md | Bumps Spark Operator upstream revision link/version in the components table. |
| REPOSITORY_NAME="kubeflow/spark-operator" | ||
| COMMIT=${COMMIT:="2.4.0"} | ||
| REPOSITORY_URL="https://github.com/kubeflow/spark-operator.git" | ||
| COMMIT="v2.5.0-rc.0" |
There was a problem hiding this comment.
COMMIT is pinned to a release-candidate tag (v2.5.0-rc.0). If this repo generally tracks stable upstream releases, consider updating to the final v2.5.0 (or add a short comment explaining why an RC is intentionally used) to avoid pulling potentially unstable manifests/images into the main branch.
| COMMIT="v2.5.0-rc.0" | |
| COMMIT="v2.5.0" |
| helm template -n kubeflow --include-crds spark-operator \ | ||
| --set "spark.jobNamespaces={}" \ | ||
| --set webhook.enable=true \ | ||
| --set webhook.port=9443 \ | ||
| --version ${COMMIT} \ | ||
| --repo https://kubeflow.github.io/spark-operator > resources.yaml | ||
| if [[ "$OSTYPE" == "darwin"* ]]; then | ||
| sed -i '' 's/Spark Operator[^|]*|[^|]*applications\/spark\/spark-operator[^|]*|[^|]*\[[0-9]\.[0-9]\.[0-9]\]([^)]*)/Spark Operator | applications\/spark\/spark-operator | ['"${COMMIT}"'](https:\/\/github.com\/kubeflow\/spark-operator\/tree\/v'"${COMMIT}"')/g' "${MANIFESTS_DIRECTORY}/README.md" | ||
| else | ||
| sed -i 's/Spark Operator.*|.*applications\/spark\/spark-operator[^|]*|.*\[[0-9]\.[0-9]\.[0-9]\]([^)]*)/Spark Operator | applications\/spark\/spark-operator | ['"${COMMIT}"'](https:\/\/github.com\/kubeflow\/spark-operator\/tree\/v'"${COMMIT}"')/g' "${MANIFESTS_DIRECTORY}/README.md" | ||
| fi | ||
| "${SOURCE_DIRECTORY}/${REPOSITORY_DIRECTORY}/charts/spark-operator-chart" > "${MANIFESTS_DIRECTORY}/${DESTINATION_MANIFESTS_PATH}/resources.yaml" |
There was a problem hiding this comment.
The helm template invocation places the chart argument after several flags. To match Helm’s documented helm template [NAME] [CHART] [flags] form (and existing repo usage in tests/helm_kustomize_compare.sh), consider moving the chart path to be the second positional argument immediately after the release name to reduce the chance of flag/arg parsing differences across Helm versions.
|
/lgtm |
google-oss-prow
Bot
deleted the
copilot/cleanup-spark-operator-update-script
branch
* Initial plan * Update kubeflow/spark-operator manifests from v2.5.0-rc.0 Signed-off-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> * Update kubeflow/spark-operator manifests from v2.5.0-rc.0 Signed-off-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> --------- Signed-off-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
This change removes script-local bootstrap logic in the Spark Operator synchronization flow and writes rendered manifests directly to the target
resources.yaml. The goal is to keep the sync script focused on rendering/updating manifests without generatingkustomization.yamlor managing an intermediate destination variable.Scope
kustomization.yamlcreation fromscripts/synchronize-spark-operator-manifests.sh.DESTINATION_DIRECTORYassignment andmkdir -pfor that variable.Script changes
cdinto destination, then> resources.yaml) with a direct absolute write to:applications/spark/spark-operator/base/resources.yaml.Resulting command pattern
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.