[Kotlin LSP] Do not download extra Java.#1079
Conversation
| """Provides JAVA_HOME and JVM options for the Kotlin Language Server process.""" | ||
| env: dict[str, str] = {} | ||
|
|
||
| # Only set JAVA_HOME if using bundled Java (not from PATH) |
There was a problem hiding this comment.
Shouldn't we verify that the java found on path is already set in JAVA_HOME?
There was a problem hiding this comment.
Thanks for the quick feedback. Fixed the other issues (were easy).
For this one I need a little more help. With a couple of questions to LLMs, I got to this solution, but not sure if that is what you meant.
Thank you
There was a problem hiding this comment.
I don't know how JAVA_HOME is used by the KotlinLS and whether it's important that the java executable that we find/download coincides with the value of JAVA_HOME. My question was about finding out whether it's necessary and ensuring for the values to coincide for KotlinLS if it is
There was a problem hiding this comment.
One of the first errors I found was kotlin-lsp trying to chmod its own shipped Java. Thus, I suppose we can delete all java management here, and rely solely on kotlin-lsp. I did not change that behavior as I am afraid I am breaking something I am not aware of. If we have currently tests for kotlin, I can try to simplify and try to understand if things still work.
There was a problem hiding this comment.
We have tests for kotlin, they are disabled in CI due to resource reasons. Even locally I can get test failures when running tests simultaneously because of some race conditions and failures to clean up resources. You can run tests with pytest -m kotlin.
I have noticed today that the kotlin LS starts openjdk processes that are never torn down, a real problem. I suspect this might be a reason for the failing tests. If you'd like to look into it, I'd be grateful.
Pinging @mareurs who knows more about Kotlin LS issues than I do
There was a problem hiding this comment.
Investigated the zombie JVM issue. Here's what I found:
TL;DR: The "openjdk processes that are never torn down" are Gradle daemons, not KLS itself.
Evidence
Ran controlled before/after tests (snapshot all Java PIDs, run pytest -m kotlin, snapshot again):
-
KLS cleanup works correctly. Even under SIGKILL (simulated crash), KLS exits naturally because the
--stdiopipe breaks on parent death (EOF on stdin). Thepsutil-based process tree cleanup inls_process.pyalso works for the happy path. Zero leaked KLS processes across multiple test runs. -
Each KLS startup spawns a Gradle daemon (~500MB RSS) that persists independently:
PID 2430590 (GradleDaemon 9.0.0) PWD = .../serena/test/resources/repos/kotlin/test_repo JAVA_HOME = .../KotlinLanguageServer/.../jre/21.0.7-linux-x86_64 Parent = systemd --user (already daemonized) RSS = 555MB, idle timeout = 3 hours (Gradle default) -
Reproduced deterministically: killed all test Gradle daemons, ran test, new one appeared with
PWDpointing at the Kotlin test repo andJAVA_HOMEpointing at KLS's bundled JRE.
Why it accumulates
- Gradle daemons are keyed by Java home + Gradle version + JVM args
- If anything differs between runs (e.g. different
JAVA_TOOL_OPTIONSfrom Serena's JVM options setting), Gradle spawns a new daemon instead of reusing - Default idle timeout is 3 hours, so they pile up during development/testing sessions
Mitigation options
- Run
gradle --stopafter KLS shutdown (orpkill -f GradleDaemonin test teardown) - Set
org.gradle.daemon=falsein the test repo'sgradle.properties - Set shorter daemon idle timeout:
org.gradle.daemon.idletimeout=60000(1 min)
The test repo is at test/resources/repos/kotlin/test_repo — adding gradle.properties with org.gradle.daemon=false there would prevent daemon accumulation during testing.
Code ReviewReviewed the diff. The goal (reuse system Java/KLS) is good, but there are some issues. Bug: JAVA_HOME / PATH version mismatchIn if java_home := os.environ.get("JAVA_HOME"):
self._java_home_path = java_home
else:
self._java_home_path = os.path.dirname(os.path.dirname(java_path))The version check runs Fix: If
|
|
@mareurs thanks for pitching in, I agree with the analysis and the recommendations. Let's not address the gradle daemon related issues here, it's independent. But I suspect they are related to test failures due to failing initialization in CI (and on my laptop) |
I am lucky, I have 128G RAM, 64 cores. It never fails :) |
The upside of a smaller system is that zombies are more apparent ;) |
|
@cx-alberto-simoes please explain why this change is necessary. What is the point of reusing the system JRE? The Kotlin LSP spawns detached daemon processes, and the only way to reliably terminate them is to search for processes that are associated with the JRE. See PR #1082. So unless you have very good reasons for not using the bundled JRE, this is not a good idea. |
|
@opcode81, ok, let me clarify:
|
| if shutil.which(kotlin_script_name): | ||
| log.info(f"Found Kotlin {kotlin_script_name} in PATH. Using it instead of bundled version.") | ||
| return kotlin_script_name |
There was a problem hiding this comment.
We should not do this implicitly. As @mareurs already mentioned, we have an explicit mechanism for the user to overrride the language server installation (ls_path).
There was a problem hiding this comment.
But that is not coherent with other languages. For instance, we do not try to install clangd.
There was a problem hiding this comment.
but ok, will try to use ls_path, and see if I still require changes.
Nevertheless, I suggest removing the download of Java.
There was a problem hiding this comment.
It is consistent with all language servers that we install.
If we can install it, then the version we install is the one that is used unless it is explicitly overridden.
The problem with clangd is that we cannot install it. Therefore, we have to rely on the user installing it. If we could install it, we would - and apply the same mechanisms.
The problem with just using any version the user may have installed is that we cannot ensure that our code is actually compatible. For instance, some language servers change their behaviour in different versions and start returning different types of content (e.g. a different way of returning symbol names) - and this may necessitate workarounds, etc.
So it is actually discouraged to use any other version than the one we have installed and tested.
If a user wants to use a different version, it should be an explicit decision.
There was a problem hiding this comment.
Nevertheless, I suggest removing the download of Java.
Definitely. If that was redundant, then this is an improvement we should keep!
|
@opcode81 I think this is now acceptable by your design. Thanks |
|
Just a poke, as this PR should be quite simple to approve. |
|
Indeed, this looks good now. Merging... |
Changes