make code completions an experimental opt-in before the alpha release

[BurntSushi]

          We could do something similar to what r-a does with their "experimental" capabilities which requires explicit opt-in from client side. Refer to the first paragraph in https://github.com/rust-lang/rust-analyzer/blob/master/docs/book/src/contributing/lsp-extensions.md#lsp-extensions:

All capabilities are enabled via the experimental field of ClientCapabilities or ServerCapabilities.

Specifically, here they send the capabilities that the VS Code extension supports.

So, for us it could look like below which would be sent when during the server initialization process:

"capabilities": {
	"experimental": {
		"completions": {
			"enabled": true
		}
	}
} 

Originally posted by @dhruvmanila in astral-sh/ruff#17744 (comment)

[BurntSushi]

Specifically motivated by astral-sh/ruff#17744 (review), where it's probably a good idea to not have code completion until it means some minimal standard of quality. In astral-sh/ruff#17744, I just built a very small end-to-end MVP to make completions actually work via LSP.

[MichaReiser]

At least, before the LSP alpha, which might not be part of the initial CLI

[MichaReiser]

CC: @charliermarsh in case you're interested in taking this on

[charliermarsh]

Yeah I can do this.

[charliermarsh]

Looking into this more, I think these should just be initializationOptions, maybe? It seems like the whole experimental thing is more for servers to indicate to clients that they support non-standard extensions. \cc @dhruvmanila

[dhruvmanila]

The initializationOptions just says that the server supports completions and what all specific features of completions that the server supports. This is mainly used by the client to make sure it asks for completions at the right time e.g., after the . character in module..

What we want here is that, by default, the completions should be off as it's pretty barebones but we don't want to block users / us from trying out completions. And, we also don't want to be competing with other language servers that will still be required for full completion support. This is why we want it to be an opt-in feature rather than opt-out for now.

Does that make sense?

[dhruvmanila]

We basically want the equivalent of --preview for the language server. If we used the --preview flag from the command-line, that would make it harder for users because it would require them to understand how to override the command used by the editor to start the server. And, that would be different for each editor. Another option if it's possible would be to make it a server option instead of relying on a specific namespace in the capabilities.

[charliermarsh]

Makes sense, I'm just not sure that we want exactly what RA is doing, since experimental is actually part of the LSP spec and has a specific meaning, and it's not clear to me that we actually want that meaning.

[MichaReiser]

I searched the node lsp client code to see how and where to pass the experimental capabilities but I didn't see a way how it could be customized when creating the client:

https://github.com/microsoft/vscode-languageserver-node/blob/c02c91cb63c8524a476e32ff56343c2370881262/client/src/common/client.ts#L1404-L1405

I'd suggest to go with initializationOptions for now. We can migrate away from it in the future.

[MichaReiser]

Nevermind, I think I found the relevant code:

1. https://github.com/rust-lang/rust-analyzer/blob/15d87419f1a123d8f888d608129c3ce3ff8f13d4/editors/code/src/client.ts#L291-L292
2. https://github.com/rust-lang/rust-analyzer/blob/15d87419f1a123d8f888d608129c3ce3ff8f13d4/editors/code/src/client.ts#L297-L340

[charliermarsh]

Thanks, I'll look into it today.