Skip to content

improve the performance of building page tables #635

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

simongdavies
Copy link
Contributor

This PR reduces the overhead of building page tables and page table entries, especially as the size of a sandboxes memory increases.

This pull request focuses on optimizing memory management in the src/hyperlight_host/src/mem/mgr.rs file, particularly improving the efficiency of page table entry (PTE) handling and memory region lookups. Key changes include pre-allocating buffers for PTEs, reducing shared memory writes, and introducing caching for sequential memory region access patterns.

Memory Management Optimizations:

  • Pre-allocation of PTE buffer: Added a pre-allocated vec to store all page table entries (pte_buffer), minimizing shared memory writes by writing the entire buffer in one operation. [1] [2]
  • Efficient shared memory write: Converted the PTE buffer to bytes and wrote it to shared memory in a single operation using copy_from_slice. This approach avoids repeated conversions and individual writes, improving performance.

Sequential Access Optimization:

  • Caching for memory region lookup: Updated the get_page_flags method to include a cached region index (cached_region_idx) for faster sequential access. The method first checks the cached region, then adjacent regions, and falls back to binary search for non-sequential access patterns. [1] [2]

@simongdavies simongdavies added the kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. label Jun 16, 2025
@ludfjig
Copy link
Contributor

ludfjig commented Jun 16, 2025

I am curious how much this speeds things up. Do you have any numbers?

@simongdavies
Copy link
Contributor Author

I am curious how much this speeds things up. Do you have any numbers?

I'll add some in while, this really only shows up when we start to use larger sandboxes and if we remove all the snapshot code, which needs a separate fix that will take longer

@simongdavies
Copy link
Contributor Author

simongdavies commented Jun 17, 2025

These before and after measurements are done after removing the snapshotting code (which has the largest impact on creation/function call with reset time as sandbox size increases, this will be dealt with in a separate PR. The flamegraph is from a simple sample program that creates a sandbox with a 500MB heap and then drops it.

The benchmarks show the creation of sandboxes with increasing heap size.

Flamegraph

Before

flamegraph-before

After

flamegraph-after

Benchmark

Before

Benchmark-before

After

Benchmark-after

@simongdavies
Copy link
Contributor Author

@ludfjig ☝️

@ludfjig
Copy link
Contributor

ludfjig commented Jun 17, 2025

@ludfjig ☝️

Is that ran with just bench release? Could it be in debug mode? Unrelated to your PR but we should probably prevent benchmarks from ever running in debug-profile

@simongdavies
Copy link
Contributor Author

@ludfjig ☝️

Is that ran with just bench release? Could it be in debug mode? Unrelated to your PR but we should probably prevent benchmarks from ever running in debug-profile

Good catch updated, yes we should, if only to stop me from doing it again #638

Copy link
Contributor

@ludfjig ludfjig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@simongdavies simongdavies merged commit 7d54ffc into hyperlight-dev:main Jun 17, 2025
43 of 60 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement For PRs adding features, improving functionality, docs, tests, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants