Fixing non-deterministic layer and metadata ordering during pack builder create

[jjbustamante]

Summary

In order to fix the issue, I added a coupled of Sort methods invocation in two places:

• At pkg/client/create_builder.go: I noticed that when we download the buildpacks modules, this process happens asynchronously and the corresponding BuildModule dependencies may have different orders before calling AddBuildpack(module). We added the sort at this point. That fixes the non-deterministic order in the io.buildpacks.builder.metadata labels.

• At internal/builder/builder.go: The order in RootFS.Layers is determined by the order in which the image.AddLayerWithDiffID is called. Sorting the modules before being added into the final image fixes that issue.

Output

Before

Using the reproduction steps described in the issue, this is the output with the current pack version:

Diff .RootFS.Layers

builder git:(main) ✗ diff -U 99 <(docker inspect --format '{{json .RootFS.Layers}}' test-builder-22-a | jq) <(docker inspect --format '{{json .RootFS.Layers}}' test-builder-22-b | jq)
--- /dev/fd/11	2023-01-19 13:16:26
+++ /dev/fd/12	2023-01-19 13:16:26
@@ -1,28 +1,28 @@
 [
   "sha256:6515074984c6f8bb1b8a9962c8fb5f310fc85e70b04c88442a3939c026dbfad3",
   "sha256:6fff76fa4f4f204665575cff7a45b4da99bcc2b7d15a01568264f7145fc889df",
   "sha256:c0e6eab9cfab0dd9b3110b53b45a30742fe00acd190fb588fef0b913e9d1578f",
   "sha256:86192190aac38304dbd2f16c8f239164b340b7da922ac42a8cc6534700946a5b",
   "sha256:6004d330f731b48a8317641954fa3c4af4eb02d3e0fea55869883dc24dd6120d",
   "sha256:ff426b14f1d37e31949cb06b412b7177760248296fcee5635e5a02a459d1ff7a",
   "sha256:0e8f10e5649aa6920f37a92d4166ebf63dd223379a50fef7dfa63082098681cf",
   "sha256:6aa3691a73805f608e5fce69fb6bc89aec8362f58a6b4be2682515e9cfa3cc1a",
   "sha256:74164b18e9bdbbaf956d6f680fa4a32a24a3663196eb9feb78ab817539a53f9c",
-  "sha256:95360acdfd192df2a542339281e8117a572fb78d8261280b532657e5c3a16745",
-  "sha256:c6abc9498653e9ce5634c34000112f0e9aaace5b92b230982b6b6af948547e15",
-  "sha256:ef5b7af3fad6d14dc84df61b90f02f8683eed6f89662b4dc5f28b5e83f49d416",
-  "sha256:5b29645fb516ce1e0190f104accd9481ebb52f065969c290777b4aa6c2149e73",
   "sha256:e3fad32c15d52e651fa41d74903c03965623abc547c49172640b22c2b85368ad",
-  "sha256:796cbf67638e578f1836d09bf5e8c328740b431c1a6af920e871b321b6b926a8",
-  "sha256:32a47c0486c495dcb62a313552faa5084f3f9c0f0fd5217f1ef1e9ab539b2f29",
   "sha256:a6015baa5d504da0db8ac7eb7e12b96d8d49a3451198bcd34a1716c8f00c3769",
-  "sha256:56fab3673be8cfbce5202f329e4669a94938c0f6966698d2f9dc81dbca441717",
-  "sha256:52408e830252ff6d36972236b50afd90cf360bd49b9736dadb06f61ae2a7952e",
-  "sha256:6aac89c2a94af7d16ff7d79f52345fbb0203519b2d9bb1cf0122372a5c67f13a",
   "sha256:bf78add1f473ae2a853b8f5c90199aaf065ac62f3a45f0f6311161a948f14c28",
-  "sha256:5a32dcd05279534592ceeae6b2b25a5307477f604f6b5f96706abad2ea2fab66",
+  "sha256:6aac89c2a94af7d16ff7d79f52345fbb0203519b2d9bb1cf0122372a5c67f13a",
+  "sha256:52408e830252ff6d36972236b50afd90cf360bd49b9736dadb06f61ae2a7952e",
   "sha256:3ed8feb76b680f696263e69dae5233d6ecb5c521650fba950aa3fcfde226416d",
+  "sha256:c6abc9498653e9ce5634c34000112f0e9aaace5b92b230982b6b6af948547e15",
+  "sha256:95360acdfd192df2a542339281e8117a572fb78d8261280b532657e5c3a16745",
+  "sha256:5a32dcd05279534592ceeae6b2b25a5307477f604f6b5f96706abad2ea2fab66",
+  "sha256:5b29645fb516ce1e0190f104accd9481ebb52f065969c290777b4aa6c2149e73",
+  "sha256:56fab3673be8cfbce5202f329e4669a94938c0f6966698d2f9dc81dbca441717",
+  "sha256:796cbf67638e578f1836d09bf5e8c328740b431c1a6af920e871b321b6b926a8",
+  "sha256:32a47c0486c495dcb62a313552faa5084f3f9c0f0fd5217f1ef1e9ab539b2f29",
+  "sha256:ef5b7af3fad6d14dc84df61b90f02f8683eed6f89662b4dc5f28b5e83f49d416",
   "sha256:107dfbc623e09e9294dd00889c4e4e1b1a14ba0e53462d344588c995d3615af1",
   "sha256:34153d9c0c3d4ca784bfe88d1e6c13672e71fb56e8d4fb8dae21d49a6046e4bf",
   "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef"
 ]

Diff "io.buildpacks.builder.metadata"

 builder git:(main) ✗ diff -U 1 <(docker inspect --format '{{index .Config.Labels "io.buildpacks.builder.metadata"}}' test-builder-22-a | jq) <(docker inspect --format '{{index .Config.Labels "io.buildpacks.builder.metadata"}}' test-builder-22-b | jq)
--- /dev/fd/11	2023-01-19 13:17:05
+++ /dev/fd/12	2023-01-19 13:17:05
@@ -94,8 +94,2 @@
     {
-      "id": "heroku/jvm",
-      "name": "Heroku OpenJDK Buildpack",
-      "version": "1.0.6",
-      "homepage": "https://github.com/heroku/buildpacks-jvm"
-    },
-    {
       "id": "heroku/jvm-function-invoker",
@@ -109,2 +103,8 @@
       "version": "1.0.3",
+      "homepage": "https://github.com/heroku/buildpacks-jvm"
+    },
+    {
+      "id": "heroku/jvm",
+      "name": "Heroku OpenJDK Buildpack",
+      "version": "1.0.6",
       "homepage": "https://github.com/heroku/buildpacks-jvm"

After

After building my local pack instance with the changes included in this PR, I deleted the builders, recreated them and execute the same commands showed above.

➜  builder git:(main) ✗ diff -U 99 <(docker inspect --format '{{json .RootFS.Layers}}' test-builder-22-a | jq) <(docker inspect --format '{{json .RootFS.Layers}}' test-builder-22-b | jq)
➜  builder git:(main) ✗ diff -U 1 <(docker inspect --format '{{index .Config.Labels "io.buildpacks.builder.metadata"}}' test-builder-22-a | jq) <(docker inspect --format '{{index .Config.Labels "io.buildpacks.builder.metadata"}}' test-builder-22-b | jq)

No difference were found by the diff command

• Should this change be documented?
  • Yes, see #___
  • [ x] No

Resolves #1453

Signed-off-by: Juan Bustamante jbustamante@vmware.com

[codecov]

Codecov Report

Merging #1600 (ed319e0) into main (6c23a08) will increase coverage by 0.02%.
The diff coverage is 100.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1600      +/-   ##
==========================================
+ Coverage   80.58%   80.59%   +0.02%     
==========================================
  Files         156      156              
  Lines       10480    10489       +9     
==========================================
+ Hits         8444     8453       +9     
  Misses       1534     1534              
  Partials      502      502              

Flag	Coverage Δ
os_linux	79.40% <100.00%> (+0.07%)	⬆️
os_macos	77.42% <100.00%> (+0.02%)	⬆️
os_windows	80.50% <100.00%> (+0.02%)	⬆️
unit	80.59% <100.00%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

[jjbustamante]

@jkutner , @hone or @dfreilich If you can take a look at this PR will be great!