Skip to content

fix(spool): Start spooling at 80% memory utilization#5472

Merged
jjbayer merged 5 commits intomasterfrom
fix/unspool-at-80
Dec 12, 2025
Merged

fix(spool): Start spooling at 80% memory utilization#5472
jjbayer merged 5 commits intomasterfrom
fix/unspool-at-80

Conversation

@jjbayer
Copy link
Copy Markdown
Member

@jjbayer jjbayer commented Dec 11, 2025

The current assumption of the spooler is that it starts spooling at 90% memory utilization, and that memory usage will stabilize before it hits 95%, at which point relay starts failing health checks.

In practice, memory usage keeps growing (at a slower pace) and we almost always become unhealthy.

Lower the default threshold to prevent reaching 95% before utilization stabilizes.

ref: INC-1542

/// Default max memory usage for unspooling.
fn spool_max_backpressure_memory_percent() -> f32 {
0.9
0.8
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered not changing the default and only lowering it for SaaS, but

  1. This will only change behavior for users that have spooling configured and run relay at ~80% memory utilization.
  2. It might actually benefit self-hosted / managed relay users to switch to more sensible defaults.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be tricky to get right, but since spooling is never enabled by default, users who opt-in will have to make this decision for themselves and 80% also seems like a better threshold for them.

@jjbayer jjbayer marked this pull request as ready for review December 11, 2025 11:42
@jjbayer jjbayer requested a review from a team as a code owner December 11, 2025 11:42
/// Default max memory usage for unspooling.
fn spool_max_backpressure_memory_percent() -> f32 {
0.9
0.8
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be tricky to get right, but since spooling is never enabled by default, users who opt-in will have to make this decision for themselves and 80% also seems like a better threshold for them.

Co-authored-by: David Herberth <david.herberth@sentry.io>
@jjbayer jjbayer enabled auto-merge December 11, 2025 14:50
@jjbayer jjbayer added this pull request to the merge queue Dec 12, 2025
Merged via the queue into master with commit 90129f4 Dec 12, 2025
46 of 49 checks passed
@jjbayer jjbayer deleted the fix/unspool-at-80 branch December 12, 2025 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants