npm test does not honour Jest settings when the CI environment variable is set.

[svdHero]

Describe the bug

npm test does not honour Jest settings when the environment variable CI is set.

Did you try recovering your dependencies?

Yes, I did a

npm install -g npm@latest
npm install

and npm --version says 6.14.6.

The problem still exists.

Environment

W:\GIT_WORKSPACES\react-ci-test>npx create-react-app --info
npx: installed 98 in 17.52s

Environment Info:

  current version of create-react-app: 3.4.1
  running from C:\Users\XXX\AppData\Roaming\npm-cache\_npx\15344\node_modules\create-react-app

  System:
    OS: Windows 8.1 6.3.9600
    CPU: (8) x64 Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz
  Binaries:
    Node: 14.0.0 - C:\Program Files\nodejs\node.EXE
    Yarn: Not Found
    npm: 6.14.6 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Internet Explorer: 11.0.9600.19036
  npmPackages:
    react: ^16.13.1 => 16.13.1
    react-dom: ^16.13.1 => 16.13.1
    react-scripts: 3.4.1 => 3.4.1
  npmGlobalPackages:
    create-react-app: Not Found


W:\GIT_WORKSPACES\react-ci-test>

Steps to reproduce

Have the follwoing jest settings in package.json

  "jest": {
    "watchPathIgnorePatterns": [
      ".*\\.testcases\\.js"
    ]
  }

Run

npm test --ci --coverage

All tests succeed and files like App.testcases.ts are excluded from test.

Run

set CI=true
npm test --ci --coverage

All previous tests succeed, but the overall process fails, because files like App.testcases.ts are NOT excluded from test.

Expected behavior

Independent of the CI environment variable, npm test should honour the jest settings in package.json.

Actual behavior

Test output:

W:\GIT_WORKSPACES\react-ci-test>npm test --ci --coverage

> react-ci-test@0.1.0 test W:\GIT_WORKSPACES\react-ci-test
> react-scripts test

 FAIL  src/__tests__/App.testcases.js
  ? Test suite failed to run

    Your test suite must contain at least one test.

      at W:/GIT_WORKSPACES/react-ci-test/node_modules/@jest/core/build/TestScheduler.js:242:24
      at asyncGeneratorStep (W:/GIT_WORKSPACES/react-ci-test/node_modules/@jest/core/build/TestScheduler.js:131:24)
      at _next (W:/GIT_WORKSPACES/react-ci-test/node_modules/@jest/core/build/TestScheduler.js:151:9)
      at W:/GIT_WORKSPACES/react-ci-test/node_modules/@jest/core/build/TestScheduler.js:156:7
      at W:/GIT_WORKSPACES/react-ci-test/node_modules/@jest/core/build/TestScheduler.js:148:12
      at onResult (W:/GIT_WORKSPACES/react-ci-test/node_modules/@jest/core/build/TestScheduler.js:271:25)

 PASS  src/__tests__/App.test.js

Test Suites: 1 failed, 1 passed, 2 total
Tests:       2 passed, 2 total
Snapshots:   0 total
Time:        5.035s
Ran all test suites.
npm ERR! Test failed.  See above for more details.

I.e., if the CI environment variable is set, npm test ignores the jest settings in package.json.

Reproducible demo

An almost out-of-the-box app:
https://github.com/svdHero/react-ci-test

[eddiemonge]

Files matching watchPathIgnorePatterns should be not ignored when running in CI mode. They are only ignored when running in watch mode so this seems to be running correctly to me. Read the docs in the link to find out more.

[svdHero]

I checked your link. Where does it say anything about CI mode? As far as I see, CI mode is not mentioned at all there.

[eddiemonge]

The link is for the parameter watchPathIgnorePatterns which is not ran in CI mode. That is for local dev

[svdHero]

Ah sorry I was confused, I did not get the meaning of "watch mode" before. I assumed that "watching" meant file watching in general, even when in CI mode. My fault.

I also found testPathIgnorePattern. I guess I should open an issue for supporting this in create-react-app.

Thanks eddiemonge for pointing me in the right direction, despite my ignorance.

[svdHero]

I opened a new issue concerning testPathIgnorePatterns here: #9345