Skip to content

[Backport][main to 0.9] | feat(cache): make FileCachePool thread-safe for multi-vCPU access (#1555) - #1571

Merged
lihuiba merged 1 commit into
release/0.9from
backport-pr-eef65fb012f83dcc0fa4aa9ec3d8997131160aaf-0.9
Jul 27, 2026
Merged

[Backport][main to 0.9] | feat(cache): make FileCachePool thread-safe for multi-vCPU access (#1555)#1571
lihuiba merged 1 commit into
release/0.9from
backport-pr-eef65fb012f83dcc0fa4aa9ec3d8997131160aaf-0.9

Conversation

@photonlibos

Copy link
Copy Markdown
Collaborator

feat(cache): make FileCachePool thread-safe for multi-vCPU access (#1555)

  • feat(cache): make FileCachePool thread-safe for multi-vCPU access

Guard all FileCachePool metadata (fileIndex_, lru_, cold tiers, totalUsed_,
tuning state) with a coarse photon::mutex (m_lock_) so one pool can be shared
across multiple photon vCPUs (OS threads).

Invariants:

  • m_lock_ is held only across in-memory ops; never across open()/do_open() or
    forceRecycle()/eviction(), keeping the lock order rw_lock -> m_lock_ one-way
    (avoids ABBA with ObjectCache's per-item mutex and non-reentrant self-deadlock).
  • eviction/evict snapshot a victim under m_lock_, release it, do the I/O
    (open()+WLOCK truncate), then re-lock to finalize.
  • the write path accounts size under the store rw_lock (updateSpace fstats under
    m_lock_) so it can't drift against eviction's WLOCK+truncate; forceRecycle()
    is deferred to do_pwritev2 after rw_lock is released.
  • running_/exit_/isFull_ and LruEntry::truncate_done become std::atomic.

Add a multi-vCPU concurrency stress test (CachePool.concurrent_stress).

QuotaFilePool is left unchanged and documented as not-yet-thread-safe (it is
currently unwired: the factory always builds a plain FileCachePool).

Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com

  • check total used in eviction loop

Co-authored-by: Claude Opus 4.8 noreply@anthropic.com
Generated by Backport Auto PR, by cherry-pick related commits.

Please review and decide whether to merge or close this backport PR.

Conflicts

Cherry-pick produced conflicts. Conflict markers are committed as-is; please resolve them manually before merging.

  • eef65fb: fs/cache/full_file_cache/cache_store.cpp

@photonlibos photonlibos added bugfix A PR that should be back-ported to prior release branches (release/*) needs-manual-merge labels Jul 27, 2026
)

* feat(cache): make FileCachePool thread-safe for multi-vCPU access

Guard all FileCachePool metadata (fileIndex_, lru_, cold tiers, totalUsed_,
tuning state) with a coarse photon::mutex (m_lock_) so one pool can be shared
across multiple photon vCPUs (OS threads).

Invariants:
- m_lock_ is held only across in-memory ops; never across open()/do_open() or
  forceRecycle()/eviction(), keeping the lock order rw_lock -> m_lock_ one-way
  (avoids ABBA with ObjectCache's per-item mutex and non-reentrant self-deadlock).
- eviction/evict snapshot a victim under m_lock_, release it, do the I/O
  (open()+WLOCK truncate), then re-lock to finalize.
- the write path accounts size under the store rw_lock (updateSpace fstats under
  m_lock_) so it can't drift against eviction's WLOCK+truncate; forceRecycle()
  is deferred to do_pwritev2 after rw_lock is released.
- running_/exit_/isFull_ and LruEntry::truncate_done become std::atomic.

Add a multi-vCPU concurrency stress test (CachePool.concurrent_stress).

QuotaFilePool is left unchanged and documented as not-yet-thread-safe (it is
currently unwired: the factory always builds a plain FileCachePool).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* check total used in eviction loop

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@xiaoyang-hhh
xiaoyang-hhh force-pushed the backport-pr-eef65fb012f83dcc0fa4aa9ec3d8997131160aaf-0.9 branch from 76efcad to 5797fa4 Compare July 27, 2026 03:18
@xiaoyang-hhh xiaoyang-hhh removed bugfix A PR that should be back-ported to prior release branches (release/*) needs-manual-merge labels Jul 27, 2026
@xiaoyang-hhh
xiaoyang-hhh marked this pull request as ready for review July 27, 2026 03:23
@xiaoyang-hhh
xiaoyang-hhh requested a review from lihuiba July 27, 2026 05:37
@lihuiba
lihuiba merged commit 212595e into release/0.9 Jul 27, 2026
36 checks passed
@photonlibos
photonlibos deleted the backport-pr-eef65fb012f83dcc0fa4aa9ec3d8997131160aaf-0.9 branch July 27, 2026 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants