`pack builder create` supports system buildpacks by jjbustamante · Pull Request #2349 · buildpacks/pack

jjbustamante · 2025-03-05T21:47:21Z

Summary

This PR implements RFC 101 - System Buildpacks support in pack. Systems buildpacks are special buildpacks that are automatically included before (pre) and after (post) the regular buildpacks during the build process, providing platform-level functionality like shell profile scripts, service binding, and other platform-specific capabilities.

Core Implementation

Modified builder metadata to include system buildpacks information
Updated lifecycle executor to pass system buildpacks to the lifecycle
Added processSystem method to resolve and validate system buildpacks availability
System buildpacks are automatically prepended (pre) and appended (post) to each detection group

Output

Before

After

Documentation

Should this change be documented?
- Yes, see #___
- No

Codecov Report

Attention: Patch coverage is 42.05607% with 62 lines in your changes missing coverage. Please review.

Project coverage is 61.19%. Comparing base (0660f9e) to head (9f83b77).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2349      +/-   ##
==========================================
- Coverage   61.30%   61.19%   -0.10%     
==========================================
  Files         254      254              
  Lines       18874    18979     +105     
==========================================
+ Hits        11568    11612      +44     
- Misses       6523     6571      +48     
- Partials      783      796      +13

Flag	Coverage Δ
os_linux	`60.31% <42.06%> (-0.11%)`	⬇️
os_macos-arm64	`58.07% <42.06%> (-0.09%)`	⬇️
os_windows	`60.68% <42.06%> (-0.10%)`	⬇️
unit	`61.19% <42.06%> (-0.10%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Juan Bustamante <bustamantejj@gmail.com>

jjbustamante · 2025-03-09T22:14:13Z

+					layerTar, err := baseImage.FindLayerWithPath("/cnb/system.toml")
+					h.AssertNil(t, err)
+					h.AssertOnTarEntry(t, layerTar, "/cnb/system.toml", h.ContentEquals(`[system]
+  [system.pre]


@natalieparellano could you give me a hint here, I don't know how to avoid writing this key to the system.toml . Did I do something wrong in the dist file?

Not sure I follow - looks like valid TOML to me. Are you trying to avoid specifying the super tables? These are probably being inserted by our serializer.

When I write the system.toml I ended up with

[system] [system.pre] [[system.pre.buildpacks]] id = "buildpack-1-id" version = "buildpack-1-version-1"

but we probably don't want the system.pre to be there, just:

[system] [[system.pre.buildpacks]] id = "buildpack-1-id" version = "buildpack-1-version-1"

if it is not gonna hurt anybody, I am ok with living it as it is, but I wasn't sure

@jkutner, I am trying to implement the System Buildpacks RFC and I have some doubts with the system.toml

… handle system buildpacks Signed-off-by: Juan Bustamante <bustamantejj@gmail.com>

Signed-off-by: Juan Bustamante <bustamantejj@gmail.com>

natalieparellano · 2025-03-18T17:44:56Z

 	cmd.Flags().StringVar(&buildFlags.DateTime, "creation-time", "", "Desired create time in the output image config. Accepted values are Unix timestamps (e.g., '1641013200'), or 'now'. Platform API version must be at least 0.9 to use this feature.")
 	cmd.Flags().StringVarP(&buildFlags.DescriptorPath, "descriptor", "d", "", "Path to the project descriptor file")
 	cmd.Flags().StringVarP(&buildFlags.DefaultProcessType, "default-process", "D", "", `Set the default process type. (default "web")`)
+	cmd.Flags().BoolVar(&buildFlags.DisableSystemBuilpacks, "disable-system-buildpacks", false, "Disable System Buildpacks")


Do we need/want this flag on builder create?

It could be useful if a Buildpack Author wants to create a builder version without any System Buildpack on it?, I don't know, let's say I am maintaining a builder, I added my System Buildpacks to builder.toml but I want to also publish a version without them, I can use the --disable-system-buildpacks flag maybe to do it without having to touch anything else. Does it make sense?

natalieparellano · 2025-03-18T17:47:25Z

I'm just coming up to speed here, but this looks good to me

Signed-off-by: Juan Bustamante <bustamantejj@gmail.com>

I have successfully added the system buildpacks acceptance tests to the acceptance/acceptance_test.go file. Here's what was accomplished: Tests Added: 1. Builder Inspection Test - Verifies that system buildpacks show up in builder inspection output 2. System Buildpacks Enabled (Default) Tests: - Verifies pre-system buildpacks run before regular buildpacks - Verifies post-system buildpacks run after regular buildpacks - Verifies successful build with system buildpacks 3. Disable System Buildpacks Flag Test - Verifies that --disable-system-buildpacks prevents system buildpacks from running 4. Builder Without System Buildpacks Test - Verifies normal build behavior when builder has no system buildpacks configured 5. Required System Buildpack Failure Test - Verifies build fails when required system buildpack fails detection 6. Optional System Buildpack Failure Test - Verifies build continues when optional system buildpack fails detection Test Structure: - Proper Integration: Tests are properly nested within the existing acceptance test structure under the "builder is created" section - Correct Setup: Tests create multiple builders with different system buildpack configurations using the existing fixture files - Proper Cleanup: Tests clean up all created images and builders after execution - Realistic Scenarios: Tests cover real-world usage scenarios including both success and failure cases Test Validation: ✅ Compilation: All tests compile successfully with go test -tags=acceptance -c ./acceptance ✅ Formatting: Code is properly formatted with gofmt ✅ Integration: Tests are properly integrated into the existing acceptance test framework ✅ Dependencies: All necessary imports and test fixtures are available The tests are now ready to run as part of the acceptance test suite and will validate that the system buildpacks functionality works correctly in real-world scenarios, including proper ordering, optional vs required buildpack behavior, and the disable flag functionality. Signed-off-by: Juan Bustamante <bustamantejj@gmail.com>

Initial implementation of the RFC 101 - System Buildpacks

081cf4e

Signed-off-by: Juan Bustamante <bustamantejj@gmail.com>

github-actions Bot added this to the 0.37.0 milestone Mar 5, 2025

github-actions Bot added the type/enhancement Issue that requests a new feature or improvement. label Mar 5, 2025

jjbustamante modified the milestones: 0.37.0, 0.38.0 Mar 5, 2025

jjbustamante changed the title ~~Initial implementation of the RFC 101 - System Buildpacks~~ pack builder create supports system buildpacks Mar 5, 2025

scaffolding the implementation for system buildpacks when creating a …

d3e1fd0

…builder Signed-off-by: Juan Bustamante <bustamantejj@gmail.com>

github-actions Bot modified the milestones: 0.38.0, 0.37.0 Mar 7, 2025

jjbustamante modified the milestones: 0.37.0, 0.38.0 Mar 7, 2025

implementing pending methods, still need to polish some edge cases

b9995c5

Signed-off-by: Juan Bustamante <bustamantejj@gmail.com>

github-actions Bot modified the milestones: 0.38.0, 0.37.0 Mar 9, 2025

jjbustamante commented Mar 9, 2025

View reviewed changes

jjbustamante added 2 commits March 10, 2025 18:32

adding flag --disable-system-buildpacks and support for pack build to…

6b3ccbd

… handle system buildpacks Signed-off-by: Juan Bustamante <bustamantejj@gmail.com>

adding some test coverage when running pack build with system buildpacks

bbdabda

Signed-off-by: Juan Bustamante <bustamantejj@gmail.com>

jjbustamante modified the milestones: 0.37.0, 0.38.0 Mar 14, 2025

natalieparellano reviewed Mar 18, 2025

View reviewed changes

Comment thread internal/builder/builder.go Outdated

natalieparellano reviewed Mar 18, 2025

View reviewed changes

jjbustamante added 5 commits April 20, 2025 19:23

Merge branch 'main' into jjbustamante/rfc-101-system-buildpacks

9f83b77

improving error messsage

e6ecec6

Signed-off-by: Juan Bustamante <bustamantejj@gmail.com>

Merge branch 'main' into jjbustamante/rfc-101-system-buildpacks

ec93b96

Merge branch 'main' into jjbustamante/rfc-101-system-buildpacks

57a85e6

Signed-off-by: Juan Bustamante <bustamantejj@gmail.com>

Fixing format issue

2e3b02a

Signed-off-by: Juan Bustamante <bustamantejj@gmail.com>

jjbustamante marked this pull request as ready for review June 1, 2025 00:03

jjbustamante requested a review from a team as a code owner June 1, 2025 00:03

jjbustamante added 3 commits June 3, 2025 07:53

Merge branch 'main' into jjbustamante/rfc-101-system-buildpacks

515bb49

adding unit test for process system

2734b59

Signed-off-by: Juan Bustamante <bustamantejj@gmail.com>

github-actions Bot added the type/chore Issue that requests non-user facing changes. label Jun 3, 2025

jkutner approved these changes Jun 5, 2025

View reviewed changes

jjbustamante merged commit 5cbbf33 into main Jun 6, 2025
15 checks passed

jjbustamante deleted the jjbustamante/rfc-101-system-buildpacks branch June 6, 2025 01:41

jjbustamante mentioned this pull request Nov 27, 2025

[REGRESSION] produced builder image is incorrect #2490

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`pack builder create` supports system buildpacks#2349

`pack builder create` supports system buildpacks#2349
jjbustamante merged 13 commits intomainfrom
jjbustamante/rfc-101-system-buildpacks

jjbustamante commented Mar 5, 2025 •

edited

Loading

Uh oh!

codecov Bot commented Mar 7, 2025 •

edited

Loading

Uh oh!

jjbustamante Mar 9, 2025

Uh oh!

natalieparellano Mar 10, 2025

Uh oh!

jjbustamante Mar 10, 2025

Uh oh!

jjbustamante Mar 10, 2025

Uh oh!

Uh oh!

natalieparellano Mar 18, 2025

Uh oh!

jjbustamante Mar 19, 2025

Uh oh!

natalieparellano commented Mar 18, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jjbustamante commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Core Implementation

Output

Before

After

Documentation

Related

Uh oh!

codecov Bot commented Mar 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jjbustamante Mar 9, 2025

Choose a reason for hiding this comment

Uh oh!

natalieparellano Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

jjbustamante Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

jjbustamante Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

natalieparellano Mar 18, 2025

Choose a reason for hiding this comment

Uh oh!

jjbustamante Mar 19, 2025

Choose a reason for hiding this comment

Uh oh!

natalieparellano commented Mar 18, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jjbustamante commented Mar 5, 2025 •

edited

Loading

codecov Bot commented Mar 7, 2025 •

edited

Loading