Skip to content

Add daemon server for persistent RBS environment#2080

Open
rhiroe wants to merge 2 commits intosoutaro:masterfrom
rhiroe:feat/server-daemon
Open

Add daemon server for persistent RBS environment#2080
rhiroe wants to merge 2 commits intosoutaro:masterfrom
rhiroe:feat/server-daemon

Conversation

@rhiroe
Copy link

@rhiroe rhiroe commented Feb 14, 2026

Summary

Introduces a daemon mode for steep check that keeps the LSP server (Master + Workers) running persistently in the background, eliminating the expensive RBS environment loading on every invocation.

  • Add steep server start|stop|restart|status subcommands to manage the daemon lifecycle
  • steep check automatically connects to the daemon when available (opt out with --no-daemon)
  • Daemon communicates with clients via Unix socket, using the existing LSP/Master/Worker infrastructure internally
  • Background file watcher (using listen gem) detects .rb/.rbs changes and notifies the server; .rbs changes additionally trigger pre-warming of the type checker

Motivation

Every steep check invocation currently spawns worker processes that each independently load the full RBS environment (gem signatures, stdlib, project signatures). This is the single largest bottleneck — on the Steep codebase itself, it accounts for most of the ~40s wall-clock time and ~235s cumulative CPU time across workers.

The daemon eliminates this by loading the RBS environment once and reusing it across checks.

Benchmark results

Measured on the Steep codebase itself (steep check --severity-level=error, Ruby 3.4.8, Linux aarch64, 10 CPU cores / 9 workers).

Full project check

Mode Avg wall time Speedup
Non-daemon (--no-daemon) 40.31s baseline
Daemon — cold start (start + check combined) 33.49s 1.2x
Daemon — cold start (check after server ready) 21.67s 1.9x
Daemon — warm (no file changes) 16.53s 2.4x
Daemon — warm (after RBS change) 24.37s 1.7x
Daemon — warm (2nd check after RBS change) 14.75s 2.7x

Single-file check (lib/steep/subtyping/check.rb, 1112 lines)

Mode Avg wall time Speedup
Non-daemon (--no-daemon) 12.46s baseline
Daemon — warm 1.33s 9.4x

Performance characteristics

The daemon's speedup comes from eliminating the fixed overhead (RBS environment loading + worker process startup) that dominates every non-daemon invocation. How much it helps depends on the ratio of fixed overhead to actual type-checking work:

Non-daemon cost  = Fixed overhead (~12s) + Type checking (scales with file count)
Daemon warm cost = Type checking only    + Socket round-trip (~1s)

The fewer files you check, the larger the daemon's relative speedup:

Scenario Non-daemon Daemon warm Speedup Why
Single file 12.5s 1.3s 9.4x Fixed overhead dominates; daemon eliminates it almost entirely
Full project 40.3s 16.5s 2.4x Type checking work is significant; daemon only eliminates the fixed portion

This means the daemon is most impactful for:

  1. Incremental checks during development — editing a file and checking just that file is ~1s instead of ~12s
  2. Repeated full checks — iterating on type errors across the project is 2.4–2.7x faster

The daemon is least impactful (but still faster) when:

  • Running a one-off full project check (cold start: 1.2x)

RBS change impact

When RBS signatures are modified, the daemon detects the change and re-warms incrementally (full project check):

  • Server-side re-warming: ~3s (vs 9.4s for full startup warm-up)
  • First check after change: ~24s (one-time penalty)
  • Subsequent checks: return to normal warm performance (~15s)

Even immediately after an RBS change, the daemon is still 1.7x faster than non-daemon.

No regression in non-daemon mode

Scope master feat/server-daemon (--no-daemon) Diff
Full project 37.30s 36.86s -1.2% (noise)
Single file 11.68s 12.46s +6.7% (noise)

Both are well within natural measurement variance (±3–4s between runs).

How it works

Without daemon (existing behavior):

[Ruby startup] → [fork N workers] → [each worker loads RBS] → [type check] → [exit]
                                    ^^^^^^^^^^^^^^^^^^^^^^^
                                    ~235s CPU, repeated every time

With daemon:

steep server start (once):
  [fork daemon] → [start Master + Workers] → [load RBS] → [listen on Unix socket]

steep check (each invocation):
  [connect to socket] → [send file list as JSON] → [stream diagnostics back]
  (user time: <1s)

CLI changes

  • steep server start — Start the daemon (double-fork, returns immediately)
  • steep server stop — Stop the daemon (SIGTERM with graceful fallback to SIGKILL)
  • steep server restart — Stop + start
  • steep server status — Show daemon status (Ready/Warming up/Starting/Not running)
  • steep check --[no-]daemon — Enable/disable daemon auto-detection (default: enabled)

Note: The daemon loads the Steepfile at startup. If the Steepfile is modified (e.g., adding targets, changing libraries), run steep server restart to pick up the changes.

@rhiroe rhiroe force-pushed the feat/server-daemon branch 6 times, most recently from 6ffa5c3 to f75557a Compare February 15, 2026 08:55
Implement a daemon mode that keeps Steep's LSP Server (Master + Workers)
running persistently so that the expensive RBS environment loading is
performed only once. Subsequent `steep check` invocations connect to the
daemon via Unix socket and skip the startup cost.

- `steep server start/stop/restart/status` subcommands
- `steep check --[no-]daemon` option (daemon enabled by default)
- Background file watcher for automatic RBS re-warming on signature changes
- Fallback to standard mode when daemon is not available
@rhiroe rhiroe force-pushed the feat/server-daemon branch from f75557a to 35f4532 Compare February 15, 2026 09:49
@ParadoxV5
Copy link
Contributor

I wonder if the RBS environment daemon and the language server can be on the same server.
Or even reimplement the RBS environment to use the LSP.

@rhiroe
Copy link
Author

rhiroe commented Feb 16, 2026

For this PR, I'll reimplement the daemon's external protocol to use LSP instead of the current custom JSON protocol.

Regarding full integration of the daemon with the language server (sharing a single Master process for both editor and CLI), there seem to be several challenges to consider:

  • The current Master assumes a single reader/writer pair — supporting multiple clients (editor + CLI) would require multiplexing or a proxy layer
  • LSP initialize can only be called once per session, so connecting a second client (e.g., steep check alongside an editor) needs careful handling
  • Managing interaction_worker lifecycle (start on editor connect, clean up on disconnect)

Does this sound reasonable?

Replace the custom line-delimited JSON protocol between the daemon
server and steep check client with standard LSP JSON-RPC, matching
the protocol already used internally between Master and Workers.
@rhiroe rhiroe marked this pull request as ready for review February 19, 2026 08:34
@soutaro soutaro added this to the Steep 2.0 milestone Mar 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants