CI is failing because of Descending test id on line: 'ok 2 - should run tests with dependency repeatedly' #54535
Description
In this CI run, the only failure is test-runner-watch-mode, which fails very frequently (see #49605).
It is marked as flaky though so the CI still eventually displays: All tests passed.
The CI run still fails due to the following error:
16:57:05 tap2junit -i test.tap -o js-tests.junit.xml
16:57:08 if errorlevel 1 exit /b
16:57:08 )
16:57:08 Traceback (most recent call last):
16:57:08 File "C:\Python310\lib\runpy.py", line 196, in _run_module_as_main
16:57:08 return _run_code(code, main_globals, None,
16:57:08 File "C:\Python310\lib\runpy.py", line 86, in _run_code
16:57:08 exec(code, run_globals)
16:57:08 File "C:\Python310\Scripts\tap2junit.exe\__main__.py", line 7, in <module>
16:57:08 File "C:\Python310\lib\site-packages\tap2junit\__main__.py", line 94, in main
16:57:08 convert(
16:57:08 File "C:\Python310\lib\site-packages\tap2junit\__main__.py", line 68, in convert
16:57:08 result = parse(name or input_file, data, package)
16:57:08 File "C:\Python310\lib\site-packages\tap2junit\__main__.py", line 58, in parse
16:57:08 tap_parser.parse(data)
16:57:08 File "C:\Python310\lib\site-packages\tap2junit\tap13.py", line 237, in parse
16:57:08 self._parse(io.StringIO(source))
16:57:08 File "C:\Python310\lib\site-packages\tap2junit\tap13.py", line 194, in _parse
16:57:08 raise ValueError(f"Descending test id on line: {line!r}")
16:57:08 ValueError: Descending test id on line: 'not ok 2 - should run tests with dependency repeatedly'
That line appears in the error output of test-runner-watch-mode:
00:29:32 not ok 799 parallel/test-runner-watch-mode # TODO : Fix flaky test
00:29:32 ---
00:29:32 duration_ms: 14971.46800
00:29:32 severity: flaky
00:29:32 exitcode: 1
00:29:32 stack: |-
00:29:32 TAP version 13
00:29:32 # Subtest: test runner watch mode
00:29:32 # Subtest: should run tests repeatedly
00:29:32 not ok 1 - should run tests repeatedly
00:29:32 ---
00:29:32 duration_ms: 3830.556
00:29:32 location: 'C:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\parallel\\test-runner-watch-mode.mjs:136:3'
00:29:32 failureType: 'testCodeFailure'
00:29:32 error: |-
00:29:32 The input did not match the regular expression /# tests 1/. Input:
00:29:32
00:29:32 '# Subtest: test has ran\n' +
00:29:32 'ok 1 - test has ran\n' +
00:29:32 ' ---\n' +
00:29:32 ' duration_ms: 5.2762\n' +
00:29:32 ' ...\n' +
00:29:32 '# Subtest: C:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.684\\\\test.js\n' +
00:29:32 'not ok 2 - C:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.684\\\\test.js\n' +
00:29:32 ' ---\n' +
00:29:32 ' duration_ms: 1068.813\n' +
00:29:32 " location: 'C:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.684\\\\test.js:1:1'\n" +
00:29:32 " failureType: 'testCodeFailure'\n" +
00:29:32 ' exitCode: ~\n' +
00:29:32 " signal: 'SIGTERM'\n" +
00:29:32 " error: 'test failed'\n" +
00:29:32 " code: 'ERR_TEST_FAILURE'\n" +
00:29:32 ' ...\n' +
00:29:32 '# Subtest: test has ran\n' +
00:29:32 'ok 3 - test has ran\n' +
00:29:32 ' ---\n' +
00:29:32 ' duration_ms: 6.1427\n' +
00:29:32 ' ...\n' +
00:29:32 '1..3\n' +
00:29:32 '# tests 3\n' +
00:29:32 '# suites 0\n' +
00:29:32 '# pass 2\n' +
00:29:32 '# fail 1\n' +
00:29:32 '# cancelled 0\n' +
00:29:32 '# skipped 0\n' +
00:29:32 '# todo 0\n' +
00:29:32 '# duration_ms 741.4156\n'
00:29:32
00:29:32 code: 'ERR_ASSERTION'
00:29:32 name: 'AssertionError'
00:29:32 expected:
00:29:32 actual: |-
00:29:32 # Subtest: test has ran
00:29:32 ok 1 - test has ran
00:29:32 ---
00:29:32 duration_ms: 5.2762
00:29:32 ...
00:29:32 # Subtest: C:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.684\\test.js
00:29:32 not ok 2 - C:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.684\\test.js
00:29:32 ---
00:29:32 duration_ms: 1068.813
00:29:32 location: 'C:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.684\\test.js:1:1'
00:29:32 failureType: 'testCodeFailure'
00:29:32 exitCode: ~
00:29:32 signal: 'SIGTERM'
00:29:32 error: 'test failed'
00:29:32 code: 'ERR_TEST_FAILURE'
00:29:32 ...
00:29:32 # Subtest: test has ran
00:29:32 ok 3 - test has ran
00:29:32 ---
00:29:32 duration_ms: 6.1427
00:29:32 ...
00:29:32 1..3
00:29:32 # tests 3
00:29:32 # suites 0
00:29:32 # pass 2
00:29:32 # fail 1
00:29:32 # cancelled 0
00:29:32 # skipped 0
00:29:32 # todo 0
00:29:32 # duration_ms 741.4156
00:29:32
00:29:32 operator: 'match'
00:29:32 stack: |-
00:29:32 testUpdate (file:///C:/workspace/node-test-binary-windows-js-suites/node/test/parallel/test-runner-watch-mode.mjs:74:14)
00:29:32 process.processTicksAndRejections (node:internal/process/task_queues:105:5)
00:29:32 async testWatch (file:///C:/workspace/node-test-binary-windows-js-suites/node/test/parallel/test-runner-watch-mode.mjs:129:26)
00:29:32 async TestContext.<anonymous> (file:///C:/workspace/node-test-binary-windows-js-suites/node/test/parallel/test-runner-watch-mode.mjs:137:5)
00:29:32 async Test.run (node:internal/test_runner/test:879:9)
00:29:32 async Promise.all (index 0)
00:29:32 async Suite.run (node:internal/test_runner/test:1239:7)
00:29:32 async Test.processPendingSubtests (node:internal/test_runner/test:590:7)
00:29:32 ...
00:29:32 # Subtest: should run tests with dependency repeatedly
00:29:32 ok 2 - should run tests with dependency repeatedly
00:29:32 ---
00:29:32 duration_ms: 2051.9995
00:29:32 ...
00:29:32 # Subtest: should run tests with ESM dependency
00:29:32 ok 3 - should run tests with ESM dependency
00:29:32 ---
00:29:32 duration_ms: 1931.9474
00:29:32 ...
00:29:32 # Subtest: should support running tests without a file
00:29:32 ok 4 - should support running tests without a file
00:29:32 ---
00:29:32 duration_ms: 1877.6795
00:29:32 ...
00:29:32 # Subtest: should support a watched test file rename
00:29:32 ok 5 - should support a watched test file rename
00:29:32 ---
00:29:32 duration_ms: 2002.1372
00:29:32 ...
00:29:32 # Subtest: should not throw when delete a watched test file
00:29:32 ok 6 - should not throw when delete a watched test file
00:29:32 ---
00:29:32 duration_ms: 2463.515
00:29:32 ...
00:29:32 1..6
00:29:32 not ok 1 - test runner watch mode
00:29:32 ---
00:29:32 duration_ms: 14164.3195
00:29:32 type: 'suite'
00:29:32 location: 'C:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\parallel\\test-runner-watch-mode.mjs:134:1'
00:29:32 failureType: 'subtestsFailed'
00:29:32 error: '1 subtest failed'
00:29:32 code: 'ERR_TEST_FAILURE'
00:29:32 ...
00:29:32 1..1
00:29:32 # tests 6
00:29:32 # suites 1
00:29:32 # pass 5
00:29:32 # fail 1
00:29:32 # cancelled 0
00:29:32 # skipped 0
00:29:32 # todo 0
00:29:32 # duration_ms 14185.4253
00:29:32 ...
cc @nodejs/test_runner