[do-not-merge] Rebase feature/virtio-mem on top of main #5400
+3,038
−763
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Update the Kconfig to enable support for virtio-mem in the guest. Signed-off-by: Riccardo Mancini <[email protected]>
virtio-mem requires a new Kconfig, so I've rebuilt the artifacts and now I'm making the devctr point to those. Signed-off-by: Riccardo Mancini <[email protected]>
Automatically generate bindings for virtio-mem. Signed-off-by: Riccardo Mancini <[email protected]>
Create the new module for the virtio-mem device. Signed-off-by: Riccardo Mancini <[email protected]>
Allow to configure the virtio-mem device from the VmmConfig and the PUT API to /hotplug/memory. Signed-off-by: Riccardo Mancini <[email protected]>
Test the freshly added PUT API to /hotplug/memory. Signed-off-by: Riccardo Mancini <[email protected]>
Add a dummy virtio-mem device that is detected by the guest driver. The device is configured with total_size, block_size, and slot_size, and uses a fixed address after the MMIO64 memory zone. Signed-off-by: Riccardo Mancini <[email protected]>
Check that the driver correctly detects the virtio-mem device, with the correct parameters. Signed-off-by: Riccardo Mancini <[email protected]>
Add support for GET /hotplug/memory that returns the current status of the virtio-mem device. This API can only be called after boot. Signed-off-by: Riccardo Mancini <[email protected]>
Add new API to swagger and device-api.md Signed-off-by: Riccardo Mancini <[email protected]>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## feature/virtio-mem #5400 +/- ##
======================================================
+ Coverage 82.41% 83.12% +0.71%
======================================================
Files 266 270 +4
Lines 30571 30875 +304
======================================================
+ Hits 25194 25666 +472
+ Misses 5377 5209 -168
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
commit 4ee4c68 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Aug 26 07:06:12 2025 +0000 chore(deps): Bump the firecracker group with 13 updates Bumps the firecracker group with 13 updates: | Package | From | To | | --- | --- | --- | | [thiserror](https://github.com/dtolnay/thiserror) | `2.0.15` | `2.0.16` | | [serde_json](https://github.com/serde-rs/json) | `1.0.142` | `1.0.143` | | [regex](https://github.com/rust-lang/regex) | `1.11.1` | `1.11.2` | | [bitflags](https://github.com/bitflags/bitflags) | `2.9.2` | `2.9.3` | | [cc](https://github.com/rust-lang/cc-rs) | `1.2.33` | `1.2.34` | | [cfg-if](https://github.com/rust-lang/cfg-if) | `1.0.1` | `1.0.3` | | [indexmap](https://github.com/indexmap-rs/indexmap) | `2.10.0` | `2.11.0` | | [jobserver](https://github.com/rust-lang/jobserver-rs) | `0.1.33` | `0.1.34` | | [prettyplease](https://github.com/dtolnay/prettyplease) | `0.2.36` | `0.2.37` | | [regex-automata](https://github.com/rust-lang/regex) | `0.4.9` | `0.4.10` | | [regex-syntax](https://github.com/rust-lang/regex) | `0.8.5` | `0.8.6` | | [winapi-util](https://github.com/BurntSushi/winapi-util) | `0.1.9` | `0.1.10` | | [winnow](https://github.com/winnow-rs/winnow) | `0.7.12` | `0.7.13` | Updates `thiserror` from 2.0.15 to 2.0.16 - [Release notes](https://github.com/dtolnay/thiserror/releases) - [Commits](dtolnay/thiserror@2.0.15...2.0.16) Updates `serde_json` from 1.0.142 to 1.0.143 - [Release notes](https://github.com/serde-rs/json/releases) - [Commits](serde-rs/json@v1.0.142...v1.0.143) Updates `regex` from 1.11.1 to 1.11.2 - [Release notes](https://github.com/rust-lang/regex/releases) - [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md) - [Commits](rust-lang/regex@1.11.1...1.11.2) Updates `bitflags` from 2.9.2 to 2.9.3 - [Release notes](https://github.com/bitflags/bitflags/releases) - [Changelog](https://github.com/bitflags/bitflags/blob/main/CHANGELOG.md) - [Commits](bitflags/bitflags@2.9.2...2.9.3) Updates `cc` from 1.2.33 to 1.2.34 - [Release notes](https://github.com/rust-lang/cc-rs/releases) - [Changelog](https://github.com/rust-lang/cc-rs/blob/main/CHANGELOG.md) - [Commits](rust-lang/cc-rs@cc-v1.2.33...cc-v1.2.34) Updates `cfg-if` from 1.0.1 to 1.0.3 - [Release notes](https://github.com/rust-lang/cfg-if/releases) - [Changelog](https://github.com/rust-lang/cfg-if/blob/main/CHANGELOG.md) - [Commits](rust-lang/cfg-if@v1.0.1...v1.0.3) Updates `indexmap` from 2.10.0 to 2.11.0 - [Changelog](https://github.com/indexmap-rs/indexmap/blob/main/RELEASES.md) - [Commits](indexmap-rs/indexmap@2.10.0...2.11.0) Updates `jobserver` from 0.1.33 to 0.1.34 - [Commits](rust-lang/jobserver-rs@0.1.33...0.1.34) Updates `prettyplease` from 0.2.36 to 0.2.37 - [Release notes](https://github.com/dtolnay/prettyplease/releases) - [Commits](dtolnay/prettyplease@0.2.36...0.2.37) Updates `regex-automata` from 0.4.9 to 0.4.10 - [Release notes](https://github.com/rust-lang/regex/releases) - [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md) - [Commits](rust-lang/regex@regex-automata-0.4.9...regex-automata-0.4.10) Updates `regex-syntax` from 0.8.5 to 0.8.6 - [Release notes](https://github.com/rust-lang/regex/releases) - [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md) - [Commits](rust-lang/regex@regex-syntax-0.8.5...regex-syntax-0.8.6) Updates `winapi-util` from 0.1.9 to 0.1.10 - [Commits](BurntSushi/winapi-util@0.1.9...0.1.10) Updates `winnow` from 0.7.12 to 0.7.13 - [Changelog](https://github.com/winnow-rs/winnow/blob/main/CHANGELOG.md) - [Commits](winnow-rs/winnow@v0.7.12...v0.7.13) --- updated-dependencies: - dependency-name: thiserror dependency-version: 2.0.16 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: serde_json dependency-version: 1.0.143 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: regex dependency-version: 1.11.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: bitflags dependency-version: 2.9.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: cc dependency-version: 1.2.34 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: cfg-if dependency-version: 1.0.3 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: indexmap dependency-version: 2.11.0 dependency-type: indirect update-type: version-update:semver-minor dependency-group: firecracker - dependency-name: jobserver dependency-version: 0.1.34 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: prettyplease dependency-version: 0.2.37 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: regex-automata dependency-version: 0.4.10 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: regex-syntax dependency-version: 0.8.6 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: winapi-util dependency-version: 0.1.10 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker - dependency-name: winnow dependency-version: 0.7.13 dependency-type: indirect update-type: version-update:semver-patch dependency-group: firecracker ... Signed-off-by: dependabot[bot] <[email protected]> commit af1d121 Author: Babis Chalios <[email protected]> Date: Mon Aug 25 16:23:38 2025 +0200 fix: BAR writes through PCI configuration capability PCI configuration capability allows a driver to access a BAR without mapping it in virtual address space. The driver issues reads/writes directly within the PCI configuration space (which should always be addressable either via MMIO or Port IO) which the device translates corresponding BAR accesses. The way this works is that the guests writes the offset and length of a BAR access within the capability structure and then reads/writes data using a 4-bytes dedicated array that also lives in the capability address space. We had a bug in the logic that handles writes where a guest would program a write of a certain length (L) and then try to perform a write using a buffer where buffer.len() < L. Our logic would then try to perform a write using the slice buffer[..L] which would cause Rust to panic with an out of range exception. Fix this by taking into account the buffer's length and using a slice with length min(L, buffer.len()). Signed-off-by: Babis Chalios <[email protected]> commit 92e936d Author: Riccardo Mancini <[email protected]> Date: Fri Aug 8 18:13:55 2025 +0100 refactor(dev-mngr): store vm as Arc in MMIODevManagerConstructorArgs Store the vm type as an Arc so it's going to be possible to pass it to the virtio-mem device. Signed-off-by: Riccardo Mancini <[email protected]> commit 031fb88 Author: Babis Chalios <[email protected]> Date: Fri Aug 22 12:31:14 2025 +0200 pci: add test for device initialization Add a unit test that ensures the device initialization process works properly. Signed-off-by: Babis Chalios <[email protected]> commit 0ca3666 Author: Babis Chalios <[email protected]> Date: Thu Aug 21 16:05:32 2025 +0200 pci: add unit test for properties of Notification capability Add a unit test that ensures that the Notification VirtIO capability is specification compliant. Signed-off-by: Babis Chalios <[email protected]> commit 1dee309 Author: Babis Chalios <[email protected]> Date: Thu Aug 21 15:00:33 2025 +0200 fix(pci): handling of PCI ISR capability We always enable MSI-X interrupts for VirtIO devices when using the PCI transport. Currently, we don't offer a fallback if the driver doesn't want to use MSI-X, but VirtIO drivers we are working with always support that. So this should not be a problem for our use-cases in the kernels we support at the moment. On the other hand, despite not supporting legacy interrupts, we still did some maintenance for the ISR status byte which is not used by MSI-X interrupts at all. Drop all this handling, ensure that reads always return 0 and writes have no effect in the state of the device. Also, add warning messages for the cases where the guest tries to access the ISR parts of the BAR. Signed-off-by: Babis Chalios <[email protected]> commit 54eb06d Author: Babis Chalios <[email protected]> Date: Thu Aug 21 12:52:07 2025 +0200 fix(pci): handling of PCI configuration capability The PCI configuration capability gives a way to the guest to access BAR memory for a device without actually mapping the BAR in guest memory. The way this works is that the guest accesses the capability directly (in PCI configuration space) and it programs reads/writes from/to BAR memory. When such an access is detected we redirect certain PCI configuration space access to VirtIO BAR accesses. These accesses happen in chunks of 4 bytes, however the code that handles BAR accesses expects the length of an access to match exactly the length of the field we are accessing, e.g. accessing a u8 field needs a &[u8] slice with length of 1. The emulation logic was not taking that into account so the mechanism wasn't effectively working. Fix that and also get rid off an unsafe `transmute` in favour of an Le32::into<u32>(). Finally, add unit tests that ensure everything works as expected. Signed-off-by: Babis Chalios <[email protected]> commit 2540713 Author: Babis Chalios <[email protected]> Date: Thu Aug 21 10:28:24 2025 +0200 pci: add unit tests for VirtIO PCI devices Add more unit tests in for VirtIO PCI capabilities other than the common configuration capability. Signed-off-by: Babis Chalios <[email protected]> commit 8ee3af1 Author: Babis Chalios <[email protected]> Date: Thu Aug 21 10:28:55 2025 +0200 fix(pci): fix handling of VirtIO PCI capabilities The VirtioPciCap type had the wrong structure. This wasn't a problem, because the wrong field was actually operating as part of the padding of which it was taking away the space. Also, fix the initialization of the VirtioPciCfgCap so that it reports the correct length for the capability. Signed-off-by: Babis Chalios <[email protected]> commit 6323d07 Author: Babis Chalios <[email protected]> Date: Thu Aug 21 10:10:25 2025 +0200 pci: remove dead code Remove dead code from the logic that handles setting up a PCI VirtIO device transport. This was either code that we pulled from Cloud Hypervisor and we don't need here or code that we are not currently using. Signed-off-by: Babis Chalios <[email protected]> commit c91b8a4 Author: Babis Chalios <[email protected]> Date: Mon Aug 11 10:36:11 2025 +0200 test: add PCI VirtIO transport unit tests Add unit tests that ensure our logic for handling the interaction with guest VirtIO drivers over the PCI transport is correct. Also, drop some dead code and fix some of the handling of various fields. A list of logic fixes we found while writing the tests: * We found some effectively dead code. As VirtIO stipulates that accesses to 64bit configuration fields MUST happen with two individual 32bit reads, so drop the code that was actually handling the 64bit accesses. * There were fields that the code was handling as write-only, when the specification declares them read-write. This was not an issue, since the Linux driver apparently never reads them, but add the read handling, so that we are spec compliant. * The specification mentions that the driver MUST NOT write a value of 0 in the queue_enable field. Enforce it. Signed-off-by: Babis Chalios <[email protected]> commit e9c95b0 Author: Patrick Roy <[email protected]> Date: Thu Aug 21 08:15:56 2025 +0100 refactor: Drop CrcReader While it's a nice, rusty abstraction, it's really overkill here, given that we're using "100 lines to do the work of 5" (as Riccardo aptly put it). Signed-off-by: Patrick Roy <[email protected]> commit 32a77ca Author: Patrick Roy <[email protected]> Date: Tue Aug 19 11:48:32 2025 +0100 test: add unit test for snapshot (de)serialization This is covered by rust intergration tests, but having a unit test that does the snapshot/restore roundtrip is nice, because I can run it on my laptop without needing sudo. Signed-off-by: Patrick Roy <[email protected]> commit feb4bc1 Author: Patrick Roy <[email protected]> Date: Mon Aug 18 15:56:43 2025 +0100 fix: use temporary buffer for loading vmstate file In commit e7504ae ("refactor: cleanup vmm::snapshot module"), firecracker started reading the snapshot vmstate file in a single pass instead of first loading it into a Vec and then deserializing. This seems to have caused some performance regression due to the deserializer doing many successive reads, resulting in many read(2) syscalls. Fix this by going back to first reading the snapshot file into a buffer, and then deserializing from slice instead. Signed-off-by: Patrick Roy <[email protected]> commit 57ea0a3 Author: Egor Lazarchuk <[email protected]> Date: Fri Aug 15 16:40:32 2025 +0100 chore: use GitHub syntax for notes Instead of **Note** syntax we can use GitHub version [!NOTE] to have a better visuals. Signed-off-by: Egor Lazarchuk <[email protected]> commit 92c8e56 Author: Egor Lazarchuk <[email protected]> Date: Mon Aug 18 12:29:48 2025 +0100 chore: clean up x86_64 part of actions.md There is no need to repeat that SendCtrlAltDel is x86_64 only when the header of the section says so already. Signed-off-by: Egor Lazarchuk <[email protected]> commit 1d7431a Author: Egor Lazarchuk <[email protected]> Date: Fri Aug 15 12:59:41 2025 +0100 chore: clarify network naming in network-setup.md Make it more clear that the `iface_id` is only used internally in Firecracker. Move information about the order of network interface creation and network interface names in the guest from FAQ into network-setup.md. Signed-off-by: Egor Lazarchuk <[email protected]> commit f0452e3 Author: Nikita Kalyazin <[email protected]> Date: Wed Aug 20 11:23:45 2025 +0000 test(vulnerabilities): ignore host checks on ubuntu Ubuntu 6.14 kernel is configured with CONFIG_MITIGATION_GDS making /sys/devices/system/cpu/vulnerabilities/gather_data_sampling report Vulnerable. Suppress the check until the distro updates the kernel configuration. Signed-off-by: Nikita Kalyazin <[email protected]> commit de58e7f Author: Nikita Kalyazin <[email protected]> Date: Tue Aug 19 12:28:17 2025 +0000 test(diff_snapshot_works_after_error): disable api timing check Since we deliberately create a condition of no space, it is not guaranteed that Firecracker is able to produce complete log messages. Signed-off-by: Nikita Kalyazin <[email protected]> commit 915899e Author: Nikita Kalyazin <[email protected]> Date: Tue Aug 19 12:26:50 2025 +0000 test(diff_snapshot_works_after_error): do not rely on log Since we deliberately create a condition of no space, it is not guaranteed that Firecracker is able to produce complete log messages. Use the API error message instead. Signed-off-by: Nikita Kalyazin <[email protected]> Signed-off-by: Riccardo Mancini <[email protected]>
Manciukic
force-pushed
the
virtio-mem/rebase
branch
from
c77f267 to
4537170
Compare
Manciukic
force-pushed
the
feature/virtio-mem
branch
from
832e565 to
2798deb
Compare
|
Tests passed, I've rebased the feature branch. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a squashed merge of upstream/main into feature/virtio-mem. No commits were changed.
Changes
...
Reason
...
License Acceptance
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.PR Checklist
tools/devtool checkbuild --allto verify that the PR passesbuild checks on all supported architectures.
tools/devtool checkstyleto verify that the PR passes theautomated style checks.
how they are solving the problem in a clear and encompassing way.
in the PR.
CHANGELOG.md.Runbook for Firecracker API changes.
integration tests.
TODO.rust-vmm.