install: store boot assets in ESP for composefs bls install

[p5]

Please note: I do not currently have an environment to test this with GRUB.

The existing code copied the boot entries, kernel and initrd into the rootfs rather than the ESP. This meant systemd-boot was unable to see them, and is not compliant with the BLS spec. From my brief investigation, it seems like grub handles this fine, however systemd-boot does not.

I am aware this is different to how the existing ostree backend works today, however this change makes the implementation more BLS-compliant (which it seems like we're wanting to be for the composefs backend) due to the following requirement:

When Type #1 boot loader menu entry snippets refer to other files (for linux, initrd, efi, devicetree, and devicetree-overlay), those files must be located on the same partition, and the paths must be absolute paths relative to the root of that file system.

In my mind I would wonder, if this was an intentional decision, why we would want bootc to even need to be aware of whether sd-boot or grub is being used after the initial install as my understanding is that grub can act in a BLS-compliant way. Therefore, a single BLS-compliant implementation that works across grub and sd-boot would be simpler overall (though I'm more than happy to be corrected on this).

The majority of this code was created referencing the existing setup_composefs_uki_boot() function.

Also, please note that I don't really know what I'm doing when it comes to bootloaders, so there may be a better approach to solve this problem.

Assisted by: ChatGPT (for conceptual knowledge)
Assisted by: GitHub Copilot gpt-4.1 (for code completion and rust syntax help)

[p5]

Can confirm with this patch set (plus coreos/bootupd#978 & the below), we are able to produce a bootable Chainguard Wolfi disk image with sd-boot.

local-bootc-patch on  local-bootc-patch [!?] 
❯ sudo losetup -Pf --show bootable.img
/dev/loop0

local-bootc-patch on  local-bootc-patch [!?] 
❯ sudo mount /dev/loop0p2 /var/mnt/esp

local-bootc-patch on  local-bootc-patch [!?] 
❯ tree /var/mnt/esp  
/var/mnt/esp
├── EFI
│   ├── BOOT
│   │   └── BOOTX64.EFI
│   ├── Linux
│   │   └── 4b4eb53d1cab36f96d8f10864d98096a8bdabea944abf1fdf6fb02496fcacf88
│   │       ├── initrd
│   │       └── vmlinuz
│   └── systemd
│       └── systemd-bootx64.efi
└── loader
    ├── entries
    │   └── bootc-composefs-1.conf
    ├── entries.srel
    ├── keys
    ├── loader.conf
    └── random-seed

Additional bootc patch, dependent on bootupd:

From 49326238bbc7b3e441050647b023c4032139dde7 Mon Sep 17 00:00:00 2001
From: Robert Sturla <[email protected]>
Date: Sun, 24 Aug 2025 17:49:05 +0100
Subject: [PATCH] install: pass through bootupd bootloader args

Signed-off-by: Robert Sturla <[email protected]>
---
 crates/lib/src/bootloader.rs | 9 +++++++++
 crates/lib/src/install.rs    | 4 ++++
 2 files changed, 13 insertions(+)

diff --git a/crates/lib/src/bootloader.rs b/crates/lib/src/bootloader.rs
index 0f02198a..93fb7a30 100644
--- a/crates/lib/src/bootloader.rs
+++ b/crates/lib/src/bootloader.rs
@@ -28,12 +28,21 @@ pub(crate) fn install_via_bootupd(
     } else {
         vec![]
     };
+
+    // If configopts.bootloader is systemd-boot, pass in the `--bootloader systemd-boot` flag to bootupctl
+    let mut bootloader_args = vec![];
+    if configopts.bootloader == Some("systemd-boot".into()) {
+        bootloader_args.push("--bootloader");
+        bootloader_args.push("systemd-boot");
+    }
+
     let devpath = device.path();
     println!("Installing bootloader via bootupd");
     Command::new("bootupctl")
         .args(["backend", "install", "--write-uuid"])
         .args(verbose)
         .args(bootupd_opts.iter().copied().flatten())
+        .args(bootloader_args)
         .args(src_root_arg)
         .args(["--device", devpath.as_str(), rootfs.as_str()])
         .log_debug()
diff --git a/crates/lib/src/install.rs b/crates/lib/src/install.rs
index c058768f..7e313960 100644
--- a/crates/lib/src/install.rs
+++ b/crates/lib/src/install.rs
@@ -252,6 +252,10 @@ pub(crate) struct InstallConfigOpts {
     /// The stateroot name to use. Defaults to `default`.
     #[clap(long)]
     pub(crate) stateroot: Option<String>,
+
+    /// The bootloader to use
+    #[clap(long)]
+    pub(crate) bootloader: Option<String>,
 }

 #[derive(
--
2.50.1

[p5]

/gemini review

[gemini-code-assist]

Code Review

This pull request correctly refactors setup_composefs_bls_boot to store boot assets in the ESP, which is essential for systemd-boot compatibility. The approach of locating and mounting the ESP partition is sound and aligns with the existing setup_composefs_uki_boot function. However, I've found a critical issue in the implementation of the mounting logic that will cause failures. My review includes a specific code suggestion to fix this.

[p5]

I tried to implement this in a slightly different way - with the vmlinuz and initrd in rootfs, and the boot entries in ESP that point to them. Thinking it might not have been a mistake directing these files to rootfs. But unfortunately this failed to boot.

Unsuccessful patch (applied ontop of fdc7199):

From 346fa791c9b405a0402a79dac22428b50146b4ea Mon Sep 17 00:00:00 2001
From: Robert Sturla <[email protected]>
Date: Sun, 24 Aug 2025 20:44:51 +0100
Subject: [PATCH] store kernel/initrd in rootfs

---
 crates/lib/src/install.rs | 82 +++++++++++++++++++++------------------
 1 file changed, 45 insertions(+), 37 deletions(-)

diff --git a/crates/lib/src/install.rs b/crates/lib/src/install.rs
index 47228e7a..c058768f 100644
--- a/crates/lib/src/install.rs
+++ b/crates/lib/src/install.rs
@@ -1633,47 +1633,49 @@ fn find_vmlinuz_initrd_duplicates(digest: &str) -> Result<Option<String>> {
     Ok(symlink_to)
 }
 
-#[context("Writing BLS entries to disk")]
-fn write_bls_boot_entries_to_disk(
-    boot_dir: &Utf8PathBuf,
+/// Write vmlinuz and initrd to the rootfs boot directory
+#[context("Writing vmlinuz/initrd to rootfs")]
+fn write_vmlinuz_initrd_to_rootfs(
+    rootfs_boot_dir: &Utf8PathBuf,
     deployment_id: &Sha256HashValue,
     entry: &UsrLibModulesVmlinuz<Sha256HashValue>,
     repo: &ComposefsRepository<Sha256HashValue>,
 ) -> Result<()> {
     let id_hex = deployment_id.to_hex();
-
-    // Write the initrd and vmlinuz at /boot/<id>/
-    let path = boot_dir.join(&id_hex);
+    let path = rootfs_boot_dir.join(&id_hex);
     create_dir_all(&path)?;
-
-    let entries_dir = cap_std::fs::Dir::open_ambient_dir(&path, cap_std::ambient_authority())
+    let dir = cap_std::fs::Dir::open_ambient_dir(&path, cap_std::ambient_authority())
         .with_context(|| format!("Opening {path}"))?;
-
-    entries_dir
-        .atomic_write(
-            "vmlinuz",
-            read_file(&entry.vmlinuz, &repo).context("Reading vmlinuz")?,
-        )
-        .context("Writing vmlinuz to path")?;
-
+    dir.atomic_write(
+        "vmlinuz",
+        read_file(&entry.vmlinuz, &repo).context("Reading vmlinuz")?,
+    ).context("Writing vmlinuz to rootfs")?;
     let Some(initramfs) = &entry.initramfs else {
         anyhow::bail!("initramfs not found");
     };
+    dir.atomic_write(
+        "initrd",
+        read_file(initramfs, &repo).context("Reading initrd")?,
+    ).context("Writing initrd to rootfs")?;
+    let owned_fd = dir.reopen_as_ownedfd().context("Reopen as owned fd")?;
+    rustix::fs::fsync(owned_fd).context("fsync rootfs boot dir")?;
+    Ok(())
+}
 
-    entries_dir
-        .atomic_write(
-            "initrd",
-            read_file(initramfs, &repo).context("Reading initrd")?,
-        )
-        .context("Writing initrd to path")?;
-
-    // Can't call fsync on O_PATH fds, so re-open it as a non O_PATH fd
-    let owned_fd = entries_dir
-        .reopen_as_ownedfd()
-        .context("Reopen as owned fd")?;
-
-    rustix::fs::fsync(owned_fd).context("fsync")?;
-
+/// Write BLS entry to ESP, referencing vmlinuz/initrd in rootfs
+#[context("Writing BLS entry to ESP")]
+fn write_bls_entry_to_esp(
+    esp_dir: &Utf8PathBuf,
+    bls_config: &BLSConfig,
+) -> Result<()> {
+    let entries_dir = cap_std::fs::Dir::open_ambient_dir(esp_dir, cap_std::ambient_authority())
+        .with_context(|| format!("Opening {esp_dir}"))?;
+    entries_dir.atomic_write(
+        format!("bootc-composefs-{}.conf", bls_config.sort_key.as_ref().unwrap()),
+        bls_config.to_string().as_bytes(),
+    )?;
+    let owned_fd = entries_dir.reopen_as_ownedfd().context("Reopen as owned fd")?;
+    rustix::fs::fsync(owned_fd).context("fsync ESP dir")?;
     Ok(())
 }
 
@@ -1768,18 +1770,24 @@ pub(crate) fn setup_composefs_bls_boot(
             bls_config.title = Some(id_hex.clone());
             bls_config.sort_key = Some("1".into());
             bls_config.machine_id = None;
-            bls_config.linux = format!("/EFI/Linux/{id_hex}/vmlinuz");
-            bls_config.initrd = vec![format!("/EFI/Linux/{id_hex}/initrd")];
+            bls_config.linux = format!("/boot/{id_hex}/vmlinuz");
+            bls_config.initrd = vec![format!("/boot/{id_hex}/initrd")];
             bls_config.options = Some(cmdline_refs);
             bls_config.extra = HashMap::new();
 
             if let Some(symlink_to) = find_vmlinuz_initrd_duplicates(&boot_digest)? {
-                bls_config.linux = format!("/EFI/Linux/{symlink_to}/vmlinuz");
-                bls_config.initrd = vec![format!("/EFI/Linux/{symlink_to}/initrd")];
+                bls_config.linux = format!("/boot/{symlink_to}/vmlinuz");
+                bls_config.initrd = vec![format!("/boot/{symlink_to}/initrd")];
             } else {
-                let efi_dir_utf8 = Utf8PathBuf::from_path_buf(efi_dir.clone())
-                    .map_err(|_| anyhow::anyhow!("EFI dir is not valid UTF-8"))?;
-                write_bls_boot_entries_to_disk(&efi_dir_utf8, id, usr_lib_modules_vmlinuz, &repo)?;
+                // Write vmlinuz/initrd to rootfs boot dir
+                let rootfs_boot_dir = root_path.join("boot");
+                write_vmlinuz_initrd_to_rootfs(&rootfs_boot_dir, id, usr_lib_modules_vmlinuz, &repo)?;
+                // Write BLS entry to ESP/loader/entries
+                let loader_entries_dir = mounted_esp.join("loader/entries");
+                create_dir_all(&loader_entries_dir).context("Creating loader/entries in ESP")?;
+                let loader_entries_dir_utf8 = Utf8PathBuf::from_path_buf(loader_entries_dir)
+                    .map_err(|_| anyhow::anyhow!("loader/entries dir is not valid UTF-8"))?;
+                write_bls_entry_to_esp(&loader_entries_dir_utf8, &bls_config)?;
             }
 
             (bls_config, boot_digest)
-- 
2.50.1

This does make sense though, as we're aiming for bls-compliance, and the bls specification requires all assets referenced in a entry file to be on the ESP.

When Type #1 boot loader menu entry snippets refer to other files (for linux, initrd, efi, devicetree, and devicetree-overlay), those files must be located on the same partition, and the paths must be absolute paths relative to the root of that file system.

[cgwalters]

In my mind I would wonder, if this was an intentional decision, why we would want bootc to even need to be aware of whether sd-boot or grub is being used after the initial install as my understanding is that grub can act in a BLS-compliant way. Therefore, a single BLS-compliant implementation that works across grub and sd-boot would be simpler overall (though I'm more than happy to be corrected on this).

Yes I think this is the ideal we want to get to, is that grub can be made to just implement BLS.

[cgwalters]

Assisted by: ChatGPT (for conceptual knowledge)
Assisted by: GitHub Copilot gpt-4.1 (for code completion and rust syntax help)

Thank you so much for adding these tags! It's very helpful to know.

[cgwalters]

We're in a state right now where we don't have automated tests on the composefs backend branch at all which we need to fix pretty soon and as we start to think about handling both grub and systemd-boot we'll need to do that. But skimming this looks sane to me.

[cgwalters]

The recommended ESP mount point is /efi, not /esp https://uapi-group.org/specifications/specs/boot_loader_specification/#mount-points

Regardless though it's a bit ugly for us to mutate the target disk image for this; we can mount it to a temporary directory instead.

(Actually though, what would be even nicer is to use the new mount API to get a detached FD instead...hmm, could add a helper that does that but falls back to mkdtemp + mount + openat + umount(MNT_DETACH) if we don't have open_tree or so...)

[p5]

Thanks for the reviews!
All make sense, I'll need to learn how the new mount API works and how to fallback to the mkdtemp + mount + openat + umount(MNT_DETACH) for pre-6.15 kernels.

Yes I think this is the ideal we want to get to, is that grub can be made to just implement BLS.

Perfect! Glad there's not going to need to be much forking out based on the bootloader - the majority of the implementation will be identical between the two.

I wanted to get this confirmation first before I spent too much more time looking at this.

[cgwalters]

All make sense, I'll need to learn how the new mount API works and how to fallback to the mkdtemp + mount + openat + umount(MNT_DETACH) for pre-6.15 kernels.

Let's not block on that, just mounting to a tmpdir I think is totally fine.

[p5]

Have pushed a new commit with the updates.
Applied some of the changes to the pre-existing uki boot function too for consistency.

Will squash after I've confirmed this still works as expected.
Will be able to test this tonight, so please don't merge until after squashing.

---

May follow this up with a separate PR after I learn what the new mount API is.