Integration testing setup with server install / launcher tests #312
Conversation
| return None | ||
|
|
||
| def eval(self, what): | ||
| return "/tmp" |
There was a problem hiding this comment.
These were unused at this point, and I think they're most likely to belong in unit tests instead of integration. Stay tuned for some thoughts/work I have toward that.
ches
force-pushed
the
integration-testing
branch
from
7f8d967 to
82bcd6a
Compare
|
Is it hard to add neovim to the base ensime image? ensime-docker v2.x |
It shouldn't be, I just never got around to it in part since I don't think anyone else tried this out until now.
…but that may (or may not?) be a rub. Last I remember there was some sticky situation about rebuilding the branch of the Docker image that has the server pinned at v1.x. I don't think I ever got a complete understanding of the scheme of how the images are built/tagged from that thread (e.g. manual build-and-push vs. automated builds IIRC), which is also probably another reason I didn't mess with the Docker image here yet… |
|
I think the tldr is: send it to the v2.x branch of the docker image. |
@jvican It probably needs a rebase, but it should be a pretty easy one. There was still some fragility in this too (some notes in my hidden comments above, if they make any sense). Mainly it's just that we never got into any discussion about this PR. I whipped up the branch on a whim when I wanted to play with the Neovim embedded possibility and had some time. I still think an integration testing harness could be useful in the long run, though this initial case (launcher) might be less interesting for the future after #354. If we can get it to be stable though it still serves as an example of the framework that others can follow for other functionality (completion sounds like it could be a good one). |
K, I'll give it a try when I can give it a few minutes' attention. Our builds are currently running against |
|
I strongly recommend upgrading to the v2.x-cache images. |
(Note: this will certainly fail the build until I can update the Docker image with a Neovim installation)
This has a few rough edges which I'll point out inline, but it's ready to poke around with and discuss.
I have added a test setup that integrates against a real Neovim instance, with initial tests of the launcher functionality as a proof of concept. @jvican mentioned this before (#136 (comment)) and I think it's a great idea—Gherkin BDD is really optimized for "full-stack" tests (acceptance, end-to-end, whatever). This isn't 100% "black box" since using the Neovim Python client is RPC versus actually driving a UI, but it's close enough for the depth of coverage we get with such an easy setup, and it can run completely headless for CI. Of course this won't cover the VimL code that's specific to standard Vim—as @jvican noted—but that's a thin layer, and as far as we can trust Neovim's compatibility it comes pretty close since the tests exercise things by executing commands (e.g.
:EnInstall), etc.Things like vimrunner that can script Vim execution with
+client-serverseem unlikely to be as robust, and I think it's safe to say we would never want to maintain a CI setup that has X and a virtual framebuffer config, so this is probably the best we're reasonably going to do.One caveat to try to resolve: running the tests sometimes leaves ENSIME server processes running, especially in the headless/embedded mode. I'm not sure why yet.
Probably obvious, but you need Neovim installed with
nvimon your path, and sbt.I made an asciicast, because hey, it's kind of fun to see 😄 Running on a fairly wimpy MacBook Air so the sbt runs are not the fastest.