Switch to using globalThis instead of eval?

[vlovich]

Summary

Today in the instrumented code there's new Function('return this') all over the place. This isn't compatible with some environments that disable eval (e.g. Cloudflare workers). Is it possible to leverage globalThis instead (even if there's reason for it not to be a default, maybe for it to be a configurable option?).

Today if you load a Stryker mutation under Workers it fails with

Stryker config

// @ts-check
/** @type {import('@stryker-mutator/api/core').PartialStrykerOptions} */

process.env.STRYKER = '1'

const config = {
  _comment:
    "This config was generated using 'stryker init'. Please take a look at: https://stryker-mutator.io/docs/stryker-js/configuration/ for more information.",
  packageManager: "npm",
  reporters: ["html", "clear-text", "progress"],
  checkers: ["typescript"],
  tsconfigFile: "tests/tsconfig.json",
  coverageAnalysis: "perTest",
  commandRunner: { command: "npm run tests" },
};
export default config;

Test runner config

N/A

Stryker environment

npm ls | grep stryker
├── @stryker-mutator/core@6.4.1
├── @stryker-mutator/typescript-checker@6.4.1

Using mocha 10 but that's not really relevant.

[nicojs]

Hi @vlovich, thanks for opening this issue!

We're indeed using (new Function('return this'))() to discover the global object. The reason for that is that it 'always works'. Using globalThis doesn't 'always work', as you can read on MDN:

[...] — but hosts are allowed to provide a different value for globalThis that's unrelated to the global object.

You're referring to Cloudflare workers here, but AFAIK that is a runtime environment. How exactly are you running your tests? If you run them with npx mocha ..., they should be executed in nodejs, where new Function(...) is allowed.

I've based the implementation of new Function on instanbuljs, but since then, they seem to have made this functionality configurable: coverageGlobalScopeFunc. We could do something similar, or we could use something based on core.js' polyfill implementation of globalThis: https://github.com/zloirock/core-js/blob/master/packages/core-js/internals/global.js

[nicojs]

Ah, are you using the --disallow-code-generation-from-strings cli option?

[vlovich]

No. The way it works is that I take my src folder and bundle it via ESBuild. I then construct a Miniflare instance within the test. Miniflare 3 runs that JS script under workerd which is the OSS version of the Cloudflare Workers runtime you can run locally (i.e. it embeds V8 and runs the code). Workerd is like NodeJS or Deno in that it's it's own JS runtime. Miniflare is the glue code that makes it friendly to simulate the broader Cloudflare product suite. Miniflare2 simulated workerd behavior by forking a new NodeJS VM - it's possible it uses the flag you pointed out.

I figured it's a "it works more reliably" thing for the existing JS ecosystem, but a flag to disable that would be helpful. For now the hack I came up with is to do a global string replace of the generated bundle. I then had to tunnel across the process.env so that stryker would actually pick the right mutants (ideally this would be something Miniflare + workerd could do automatically). My report is claiming 100% coverage though so I don't think I've fully solved the issue. So I think there's work to be done to make this a simpler experience. I only really care about my project at the moment but I'm probably the first person to use it and if I can get this to work it's a good model for getting good test coverage for anyone using Cloudflare Workers

[nicojs]

Ah yes, now I understand.

I think we could safely use a core-js-like workaround. If it's good enough for core-js, it should be good enough for us. That would solve this issue.

My report is claiming 100% coverage though so I don't think I've fully solved the issue.

Do you mean that Stryker is claiming 100% coverage? That is to be expected because the command test runner doesn't support coverage at all (coverageAnalysis: "perTest" is ignored).

I'm curious, you mentioned that you're using mocha under the hood. Maybe we can use the @stryker-mutator/mocha-runner here and configure your ESBuild command using the buildCommand option. That might dramatically improve performance as well as support perTest coverage. What does your "test" script look like to run your tests?