Skip to content

Nr. 3 - [TEP-0056]: Implement Pipelines-in-Pipelines reconciliation.#8879

Merged
tekton-robot merged 2 commits intotektoncd:mainfrom
twoGiants:issue-8760-tep-0056-pinp-reconciliation
Oct 30, 2025
Merged

Nr. 3 - [TEP-0056]: Implement Pipelines-in-Pipelines reconciliation.#8879
tekton-robot merged 2 commits intotektoncd:mainfrom
twoGiants:issue-8760-tep-0056-pinp-reconciliation

Conversation

@twoGiants
Copy link
Member

@twoGiants twoGiants commented Jul 12, 2025

Important

Pull request number 3.
The numbering Nr. 3 - [TEP-0056] means the PRs must be merged in order. They build on each other.

Changes

[TEP-0056]: Third PR of Pipelines-in-Pipelines feature implementation.

Child PipelineRuns (PinP) are created by the PipelineRun reconciler, equal to the TaskRun/CustomRun implementations.

An event handler for child PipelineRuns is registered in controller entrypoint. This will trigger the reconciliation loop when child PipelineRuns change their state.

Extend resolvePipelineState with getter for child PipelineRuns using lister and extend runNextSchedulableTask with a condition check for a PipelineTask which is a Pipeline and implement the creation of a new PipelineRun from the resolved pipeline state and pipeline facts.

Setting the ChildReferences was extended for child PipelineRuns.

Rename label/annotation factory.

The unit/e2e test framework was refactored and extended to prepare for future tests. The test setup is a parent pipeline with one or more embedded child/grandchild pipelines using the PipelineSpec (alpha) field. It follows the given-when-then test flow arrangement.

The pipeline manifests yaml definitions use variables for every field which is validated. Multiple helper functions were created equal to reconciliation unit tests for TaskRuns/CustomRuns.

Test data factory functions were put in the testing package in the factory.go file.

The unit tests validates:

  • the status and condition of the parent PipelineRuns which should trigger the creation of the child PipelineRuns,
  • the actual created child/grandchild PipelineRuns if they have the correct metadata i.e. name, owner reference, etc. and the embedded pipelines from the PipelineSpec fields in the pipeline tasks of the parent pipeline.

Similar checks are performed in TestReconcile for TaskRun and in TestReconcile_V1Beta1CustomTask for CustomTasks.

The e2e tests validate:

  • parent PipelineRun creation,
  • child PipelineRun creation,
  • successful finish of all resources,
  • correct label and annotation propagation,
  • amount of events published.

Similar checks are performed in
TestPipelineRun|TestPipelineRunStatusSpec|... for TaskRun and in TestCustomTask for CustomTask.

Issues #8760, #7166.

Release notes will be added with the last PR which will make this feature functional for users.

/kind feature

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

  • Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
  • Has Tests included if any functionality added or changed
  • pre-commit Passed
  • Follows the commit message standard
  • Meets the Tekton contributor standards (including functionality, content, code)
  • Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
  • Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
  • Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

A Pipeline can now execute embedded Pipelines (Pipelines-in-Pipelines) using the `PipelineSpec` field under `tasks`. Refer to the TEP-0056 for more details.

@tekton-robot tekton-robot added kind/feature Categorizes issue or PR as related to a new feature. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Jul 12, 2025
@tekton-robot tekton-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Jul 12, 2025
@tekton-robot
Copy link
Collaborator

The following Tekton test failed:

Test name Commit Details Required Rerun command
check-pr-has-kind-label 86cc7e7 link true /test check-pr-has-kind-label

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/reconciler/testing/factory.go Do not exist 0.0%

@waveywaves
Copy link
Member

/kind feature

@waveywaves waveywaves modified the milestones: v1.4.0, v1.5.0 Aug 29, 2025
@vdemeester vdemeester modified the milestones: v1.5.0, v1.6.0 (LTS) Oct 3, 2025
@twoGiants twoGiants force-pushed the issue-8760-tep-0056-pinp-reconciliation branch from 86cc7e7 to 05e788a Compare October 28, 2025 07:24
@twoGiants twoGiants marked this pull request as ready for review October 28, 2025 07:25
@tekton-robot tekton-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 28, 2025
@vdemeester
Copy link
Member

@twoGiants this now can be rebased 👼🏼

@twoGiants twoGiants force-pushed the issue-8760-tep-0056-pinp-reconciliation branch from 05e788a to e2c0619 Compare October 28, 2025 10:39
@vdemeester vdemeester force-pushed the issue-8760-tep-0056-pinp-reconciliation branch 2 times, most recently from 5ac5277 to 18ab6a7 Compare October 30, 2025 11:24
[TEP-0056]: Third PR of Pipelines-in-Pipelines feature implementation.

Child `PipelineRuns` (PinP) are created by the `PipelineRun` reconciler,
equal to the `TaskRun/CustomRun` implementations.

An event handler for child `PipelineRuns` is registered in controller
entrypoint. This will trigger the reconciliation loop when child
`PipelineRuns` change their state.

Extend `resolvePipelineState` with getter for child `PipelineRuns` using
lister and extend `runNextSchedulableTask` with a condition check for a
`PipelineTask` which is a `Pipeline` and implement the creation of
a new `PipelineRun` from the resolved pipeline state and pipeline facts.

Setting the `ChildReferences` was extended for child `PipelineRuns`.

Rename label/annotation factory.

The unit/e2e test framework was refactored and extended to prepare for
future tests. The test setup is a parent pipeline with one or more
embedded child/grandchild pipelines using the `PipelineSpec` (alpha)
field. It follows the given-when-then test flow arrangement.

The pipeline manifests yaml definitions use variables for every field
which is validated. Multiple helper functions were created equal to
reconciliation unit tests for TaskRuns/CustomRuns.

Test data factory functions were put in the `testing` package in the
`factory.go` file.

The unit tests validates:
- the status and condition of the parent PipelineRuns which should
trigger the creation of the child PipelineRuns,
- the actual created child/grandchild PipelineRuns if they have the correct
metadata i.e. name, owner reference, etc. and the embedded pipelines
from the `PipelineSpec` fields in the pipeline tasks of the parent
pipeline.

Similar checks are performed in `TestReconcile` for `TaskRun` and in
`TestReconcile_V1Beta1CustomTask` for `CustomTasks`.

The e2e tests validate:
- parent PipelineRun creation,
- child PipelineRun creation,
- successful finish of all resources,
- correct label and annotation propagation,
- amount of events published.

Similar checks are performed in
`TestPipelineRun|TestPipelineRunStatusSpec|...` for `TaskRun` and in
`TestCustomTask` for `CustomTask`.

Issues tektoncd#8760, tektoncd#7166.

Signed-off-by: Stanislav Jakuschevskij <[email protected]>
Signed-off-by: Vincent Demeester <[email protected]>
@vdemeester vdemeester force-pushed the issue-8760-tep-0056-pinp-reconciliation branch from 18ab6a7 to 7b864b3 Compare October 30, 2025 12:26
@vdemeester
Copy link
Member

2025-10-30T13:34:00.0427657Z examples_test.go:128: exit status 1 Output: Error from server (BadRequest): error when creating "STDIN": admission webhook "validation.webhook.pipeline.tekton.dev" denied the request: validation failed: feature flag enable-cel-in-whenexpression should be set to true to use CEL: '$(params.path)' == 'README.md' in WhenExpression: spec.pipelineSpec.tasks[0].when[0]

@vdemeester
Copy link
Member

There is really something fishy with the example tests and feature-flags

@vdemeester
Copy link
Member

2025-10-30T13:34:03.1437169Z Error from server (BadRequest): error when creating "STDIN": admission webhook "validation.webhook.pipeline.tekton.dev" denied the request: validation failed: feature flag enable-param-enum should be set to true to use Enum: spec.params[message]
2025-10-30T13:34:03.1441196Z Error from server (BadRequest): error when creating "STDIN": admission webhook "validation.webhook.pipeline.tekton.dev" denied the request: validation failed: feature flag enable-param-enum should be set to true to use Enum: spec.finally.params[message], spec.tasks.params[message]

Prior to this, they may affect the other tests (mainly examples one).

Signed-off-by: Vincent Demeester <[email protected]>
@vdemeester
Copy link
Member

/approve
/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Oct 30, 2025
@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 30, 2025
@tekton-robot tekton-robot merged commit b56477c into tektoncd:main Oct 30, 2025
75 of 82 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/feature Categorizes issue or PR as related to a new feature. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants