deno lsp: 126s → 20s first response, 30+ second -> 300ms go to definition, 10-11GB res -> 3 GB res (6 patches) -- includes comprehensive repro, simulation, and evidence

[alita-moore]

Once again, authored by Claude but I validated everything and am running the patched binary right now in my IDE with noticeably improved performance. I suspect the reason that this one was missed up until now is because deno doesn't ship large package sources in the filesystem. But if deno is going to support mixed repos (please do, thank you!) large uncommitted resources are common place in most languages. I proposed some fixes here with meaningful evidence (defect 1 and 2, note that 3 and 4 are hacks I didn't think very much about) about their efficacy, but to be honest I don't recommend that any of them be merged. My hope is that this (and probably future issues because deno is still eating ~11gigs of memory for unknown reasons and is still painfully but usably slow) will inspire the maintainers to revisit the LSP performance. Most notably the root cause in this issue could probably do with some tinder love and care 🙃.

---

On a 53-project monorepo, one textDocument/documentSymbol took 126.7 s. Enumerating every file in that workspace takes 221 ms, so almost all of that is work nothing asked for.

To be precise about the headline, because two of these numbers mean different things:

• The three mergeable fixes take it to 38.7 s — a 3.3x improvement.
• Adding a fourth change, which is a diagnostic and should not be merged, takes it to 20.0 s — 6.3x. That fourth one skips work rather than removing redundancy; it is included as evidence of where the remaining time goes, not as a proposal.

So: 3.3x from fixes you could take today, and a measured pointer at roughly 20 more seconds that appears to be redundant.

Full write-up, reproductions, patch scripts and raw evidence: https://github.com/alita-moore/deno-lsp-performance

Workspace shape: 73 npm workspace members, 5,781 TS files totalling 6.1 MB of source, ~96k .d.ts in node_modules.

Measurements

One session, one probe, unpatched vs patched builds of v2.9.5, both built with --features lsp-tracing:

	unpatched	+M5+R1+lazy	+export gate
documentSymbol	126,734 ms	38,650 ms	19,989 ms
lsp.did_change_configuration	41,504 ms	1,871 ms	2,123 ms
tsc.op.op_script_names	42,136 ms	35,625 ms	15,906 ms
peak RSS	2,943 MB	2,316 MB	2,298 MB

Directory opens on the same workspace, unpatched vs the two enumeration fixes: 39,014 → 5,910 (−84.9%).

Four defects

1 & 2 — two callers of the same line. Both bottom out at libs/config/glob/collector.rs:178, the fs_read_dir inside FileCollector::collect_file_patterns:

• Workspace member-glob expansion (discovery.rs:898), reached from refresh_config_tree. Expanding "workspaces": ["packages/*"] descends every matched member in full, pruning only node_modules, .git and one exact vendor path. A .venv inside a matched member is walked; the identical tree at the repository root is not — 2,587 opens vs 1, everything else held fixed. No user configuration reaches this walk at all.
• tsconfig root-set collection (compiler_options.rs:87 → collect_specifiers), reached from refresh_compiler_options_resolver. Same walk, and collect_specifiers is called once per distinct FilePatterns, so each member's tsconfig gets its own.

The same mechanism is unsafe at the first and safe at the second, which is why they need different fixes. Honouring the ignore set during member expansion loses workspace members whose directory is gitignored (measured: 2 of 2 in one shape). During root-set collection it does not, because exclude governs seeds rather than reachability — an imported file still enters the graph, verified against the binary.

Fixes: bound member-glob descent by whether the pattern can still match below the directory (the predicate, can_match_under, already exists and is already trusted by split_by_base); honour the ignore set for root-set collection.

3 — eager per-scope npm dependency resolution. ConfiguredDepResolutions is built eagerly for every scope in cli/lsp/resolver.rs. With 73 members that is 73 constructions, most never used. Holding it in a OnceLock per scope takes did_change_configuration from 41,504 ms to 1,871 ms. Related: #36662.

4 — redundant export-key enumeration. Gating the export_keys enumeration behind an env var halves documentSymbol and takes op_script_names from 42 s to 16 s. This is a diagnostic, not a proposed patch — it skips the work rather than removing the redundancy, and should not be merged as-is. It is included as evidence that ~20 s there appears to be redundant, consistent with #36662.

What a reader will assume, which is false

• tsconfig include/exclude do not affect defect 1 at all. Different subsystem. 46 configurations were measured against it; none moves this cost.
• walk_workspace is not the problem. It is capped at 1,000 entries (language_server.rs:1065) and accounted for 91 of 42,666 opens.
• Deleting workspaces from package.json "fixes" it by skipping the expansion entirely. That is why this can look like an npm-workspaces problem rather than an enumeration one.

Every claim above was located by capturing a call stack at each opendir and symbolising it against the binary, not by reading source. The tooling is in the repo and runs against any workspace.

Still unexplained

With all four patches the server reaches 10.6 GB resident in 72 seconds on this workspace, up from ~8 GB unpatched, against 6.1 MB of source. RSS is flat once reached, so it is not a leak — it allocates once and holds. The reproduction never gets near this (peaks ~2.3 GB), so I have not identified the cause. Possibly the patches remove the work that was throttling allocation, but that is unverified.

[alita-moore]

authored by claude again, but I've reviewed it and think it represents my stance on this. I suspect that it's time to clean up the debt for the LSP. I would be surprised if these performance issues didn't make it working with deno in a real workspace meaningfully more painful. The performance gap between the default deno runtime and the LSP is pretty dramatic; and in my opinion, unnecessary. In a deno only workspace these issues are likely mitigated, but in any realistic workspace (next project, python packages, pnpm workspace, mixed source) the current deno LSP performance is painful. I like deno a lot (hence why I spent a day trying to make it usable for my use case), and I have hope that you'll find the time to take a serious look at the subsystems involved here. My gut tells me that with a proper refactor that the deno LSP could be 10x faster and use 10x less memory than even the proposed (6 patch) solution presented here. And this matters for real production workloads. The most damning piece of these performance issues is that it makes deno performance scale with the workspace size NOT the actual code being edited. If you put a hello world deno function in my repo on the pre-patch binaries it'd take 100+ seconds to do a go to definition on the hello function. If deno wants to be integrated into existing systems (which please do 😄 ) these LSP issues make it unusable. I don't know the exact economies, but presumably more code implies a more mature system which implies a business capable and willing to spend more money. So I think taking a few weeks to actually make the LSP and these subsystems in general optimal is worth its weight in gold. But that could just be my over engineering nature talking and I'm sure you all know your customer demographic better than I could.

I've taken about a day on this and my system is usable again, so I'm going to leave it here. I hope to see future versions that make the LSP not only usable but excellent; and if not I'll just have claude patch the binaries until it makes sense on your side to invest the time. Thanks!

---

Update: two more defects, and measurements from a six-patch build.

Where it lands

Same workspace, same probe, one session, unpatched vs patched builds of v2.9.5:

	unpatched	six patches
documentSymbol	120,486 ms	18,587 ms
lsp.did_change_configuration	75,072 ms	1,504 ms
definition	32,226 ms	131 ms
tsc.op.op_script_names	63,190 ms	15,350 ms
peak RSS	2,629 MB	1,861 MB

Subjectively, and this is an impression rather than a measurement: in a real editor the six-patch binary settles at roughly 3 GB resident after warmup against 8–10 GB before, and is noticeably more responsive. It is usable.

I want to be clear that 3 GB is itself poor. This workspace is 6.1 MB of source across 5,781 TypeScript files. Three gigabytes to hold an editing session over six megabytes of code is not a good outcome — it is merely an acceptable one for my purposes, which is why I am going to stop digging here. A genuinely well-tuned language server for a workspace this size should need dramatically less, and I would treat the remaining 3 GB as unexplained rather than as a floor.

The two new defects

5 — deno.json's workspace cannot bound membership. resolve_workspace_for_config_folder fills one member map from two sequential blocks: the deno.json workspace block, and the package.json workspaces block, which inserts over anything with a comment noting the duplicate check is deliberately suppressed. So the field is purely additive. Where an npm glob already covers the same directories it contributes nothing, and there is no supported way to say "this deno.json is the authority". On the workspace here: 69 config folders resolved, 52 declared, 17 that nothing declared.

6 — the npm resolution graph is duplicated per scope and per request. A resolver scope is created for every workspace member, each seeded with every npm specifier in the single deno.lock rather than its own, each materialising a private copy of the resulting 2,567-package graph — and snapshot() then deep-clones all of them on every request, from 35 handlers. A sampling heap profiler puts 88% of live heap on NpmResolutionCell::snapshot after config load. It is glibc malloc, not V8. There is a todo(dsherret): this is pretty terrible on the exact line.

Ablation: enabling one path instead of six changes nothing measurable — 2,016 packuments read either way — so deno.enablePaths narrows walk_workspace and nothing after it. A synthetic workspace of 73 empty members plus this deno.lock reaches 1,113 MB; delete the lockfile and it is 49 MB.

Please read the patches as diagnostics, not proposals

None of these are production patches, and I am not asking for them to be merged. They are the smallest changes that isolate each defect well enough to measure it. Two are explicitly unmergeable: the export-key gate skips work rather than removing the redundancy, and defect 5 changes a tested contract — cargo test -p deno_config goes 168 to 167, and the failing test asserts the npm-union on purpose.

The performance is now acceptable on this workspace, but I would not read that as the problem being solved. Six defects turned up from pulling one thread, and nothing about the way they were found suggests the thread has ended — each one was located by capturing a call stack and symbolising it, and each time the next one was underneath.

What I would actually recommend is a deeper investigation, and ideally a meaningful refactor aimed squarely at scalability in large and, critically, mixed-source workspaces — repositories that are simultaneously a Deno workspace and an npm/pnpm workspace, with vendored trees, generated output and virtualenvs sitting alongside the source.

That shape is where every one of these defects lives, and they share a family resemblance: work is sized by what is present on disk or in a lockfile rather than by what the request actually needs. Enumeration walks directories no configuration asked for; membership is the union of two systems with no way to bound it; the dependency graph is seeded from every specifier in the lockfile and then copied per scope and per request. Each patch here narrows one instance of that. None of them change the underlying assumption, and I do not think this is fixable by accumulating narrow fixes.

Everything — reproductions, patch scripts, raw evidence, the heap profiler — is in https://github.com/alita-moore/deno-lsp-performance

[steve02081504]

hmmm i'd only see ya issue, wheres pr?

[alita-moore]

the relevant patches are contained in the linked repo as string applied patches. I will not be submitting a PR because I don't think that my changes should be merged but rather that the suggested changes should act as proof of the problem and encourage a deeper investigation / refactor. The attached repository contains extensive reproduction, simulation, as well as everything necessary to recreate the mentioned binary. Please let me know if there's anything more I can do to help.

[alita-moore]

it doesn't appear you're a maintainer @steve02081504, if you're mostly interested in checking out the improved binary you can download it here: https://github.com/alita-moore/deno-lsp-node-modules-repro/actions/runs/32644431020/artifacts/9494721360

[bartlomieju]

I'm happy to upstream these changes if you open PRs for them

[alita-moore]

@bartlomieju I don't think that my specific changes should be merged, I think that the subsystems involved / the LSP in general would benefit from some improvements. I don't have the time to do this myself. So I've done what I can to present the necessary reproductions and background so that you or someone from your team with more knowledge about this system can make meaningful improvements hopefully far beyond this work. I suspect this is the tip of the iceberg and can't take it further.

Understandably I'm requesting probably a few weeks or more of meaningful effort from your team to meaningfully refactor the LSP subsystem. So if you / your team don't feel the LSP performance on mixed / larger systems is in the profit center, or you're busy with more important things, then that's okay. But I think it's worth your time to review the evidence and make an informed decision. It's not too much trouble to make the hacky patches so that my personal binary is acceptable.

I personally feel that fixing only the issues I pointed out and especially in the way I fixed them is itself a temporary change, so I'd suggest either leave the performance as it is and defer it entirely or dig deep and make the LSP 20-50x (predicted) faster, improve the abstractions to make similar failures inexpressible, and add extensive test coverage. Basically, I'm urging you consider whether or not the LSP performance is something your team really cares deeply about, which is not a decision I can make.

Thank you for your time!