Running tests through `wasm-pack` doesn't work if JS snippets are used

[koute]

Describe the Bug

Using JS snippets prevents wasm-pack from being able to run tests.

Steps to Reproduce

1. Create a new crate.
2. Paste this into Cargo.toml:

[dependencies]
wasm-bindgen = "0.2"

[dev-dependencies]
wasm-bindgen-test = "0.2"

3. Paste this into src/lib.rs:

extern crate wasm_bindgen;

#[cfg(test)]
extern crate wasm_bindgen_test;

use wasm_bindgen::prelude::*;

#[cfg(test)]
use wasm_bindgen_test::*;

#[cfg(test)]
#[wasm_bindgen_test]
fn foobar() {
    #[wasm_bindgen(inline_js = "export function foo() {}")]
    extern "C" {
        fn foo();
    }

    foo();
}

4. Run:

$ wasm-pack test --chrome --headless

Actual Behavior

$ wasm-pack test --chrome --headless
[INFO]: Checking for the Wasm target...
    Finished dev [unoptimized + debuginfo] target(s) in 0.03s
    Finished dev [unoptimized + debuginfo] target(s) in 0.03s
     Running target/wasm32-unknown-unknown/debug/deps/foobar-27ec9db4a368c0d0.wasm
error: executing `wasm-bindgen` over the wasm file
	caused by: failed to generate bindings for JS import `foo`
	caused by: local JS snippets are not supported with `--target nodejs`; see rustwasm/rfcs#6 for more details, but this restriction will be lifted in the future
error: test failed, to rerun pass '--lib'
Error: Running Wasm tests with wasm-bindgen-test failed
Caused by: failed to execute `cargo test`: exited with exit code: 1

Expected Behavior

I expect the tests to run.

Additional Context

I need this to be able to run stdweb's test suite. I could probably hack around it by having a custom test runner, but I'd rather not do that and just use standard tooling.

Also, it seems somewhat counterintuitive that even though I want to run the tests under headless Chrome they're being built for Node.js by wasm-pack? Is this expected?

$ wasm-pack --version
wasm-pack 0.8.1
$ wasm-bindgen --version
wasm-bindgen 0.2.42

[alexcrichton]

Thanks for the report!

This is actually sort of an extension of #822 and is a relic of how the testing situation still has an unresolve question here between wasm-pack and wasm-bindgen that we haven't gotten around to fixing.

What's happening here under the hood is that each test binary (the wasm file) is either configured to automatically run in node.js or in a browser. This configuration is done in code currently with the wasm_bindgen_test_configure! macro (it's a pretty bad interface). When cargo test "executes" all wasm binaries then wasm-bindgen-test-runner will attempt to execute all tests in the test suite that they expect to be run in. This means that --chrome to wasm-pack is also running Node.js tests. I believe --node also runs headless browser tests. (as you can see, not a great situation!)

The problem here is that the default for all tests is to execute in Node.js. The test above is actually configured to run in Node.js because nothing else was mentioned. As a result, wasm-pack test --chrome is failing because it's attempting to execute the test in Node.js. You can fix this with a call to wasm_bindgen_test::wasm_bindgen_test_configure!(run_in_browser); in the file.

@fitzgen IIRC you were working on this way back when wasm-pack test was first implemented, I forget if you had thoughts of how we might solve this?

[fitzgen]

I never started work on resolving this impedance mismatch.

As a strawperson proposal for an end state resolution:

• Let's make wasm-bindgen-test generate code that does feature detection for where it is running in (don't remember exactly what this would entail)
• Allow both wasm-pack test --node and wasm-pack test --firefox --headless to work if the code doesn't depend on anything only present in one or the other environment
• Allow the source to declare that it only supports node or browsers, in which case attempts to run in any other environment exit with an error code and helpful message

How does this sound?

[alexcrichton]

While wasm-bindgen-test does have code to detect if it's running in node/browsers I think it wouldn't solve this issue because tests in browsers may even fail the wasm-bindgen step when they're generating for node (aka the snippets issue). I'm realizing though we have actually previously talked about this so I'm gonna look into implementing that.