Skip to content

test(integration): stabilize watch helper using run-completion sync#5879

Open
mostafaNazari702 wants to merge 1 commit into
mochajs:mainfrom
mostafaNazari702:fix/watch-flaky-test-outside-cwd
Open

test(integration): stabilize watch helper using run-completion sync#5879
mostafaNazari702 wants to merge 1 commit into
mochajs:mainfrom
mostafaNazari702:fix/watch-flaky-test-outside-cwd

Conversation

@mostafaNazari702

@mostafaNazari702 mostafaNazari702 commented Apr 10, 2026

Copy link
Copy Markdown
Contributor

PR Checklist

Overview

Replace fixed-sleep timing in runMochaWatchAsync with stdout synchronisation to remove flaky watch integration tests.
old approach relied on sleeps around the "change" callback which could race with incomplete runs on CI causing inconsistent test results.

we buffer instead stdout to reliably detect "stats" run-completion markers across chunk boundaries and replace the multi-waiter loop with a single progress waite, and snapshots the run baseline before triggering "change" to prevent race conditions.
an expectedRuns option is added to avoid unnecessary rerun waits in negative tests and tests are updated to use it where appropriate.

Now we have faster and more stable watch tests (from 120 seconds down to, from my test, 51) with all integration tests still passing.

EDIT:

from 120 seconds down to, from my test, 51
I actually ran a focused test rather than the full integration suite. Time still same. I apologize for the mistake.

@mostafaNazari702

Copy link
Copy Markdown
Contributor Author

Wrong branch.

@codecov

codecov Bot commented Apr 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.00%. Comparing base (6695fba) to head (716b6ec).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5879      +/-   ##
==========================================
+ Coverage   80.89%   81.00%   +0.10%     
==========================================
  Files          64       64              
  Lines        4602     4602              
  Branches      976      976              
==========================================
+ Hits         3723     3728       +5     
+ Misses        879      874       -5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mostafaNazari702

Copy link
Copy Markdown
Contributor Author

I was actually right, thought that i pushed code to the previous active PR that i have. Sorry.
I would also like to mention that the PR body was rewritten from my notes, which normally sound like gibberish. using AI.

@mostafaNazari702 mostafaNazari702 force-pushed the fix/watch-flaky-test-outside-cwd branch from 8bb403d to f755981 Compare April 10, 2026 15:28
@mark-wiemer

Copy link
Copy Markdown
Member

Nice, this could finally close that issue :) updated the branch to get another run to see if there is flakiness left, thank you :)

@JoshuaKGoldberg

Copy link
Copy Markdown
Member

@mostafaNazari702 the most recent CI run has integration failures ☹️ can you investigate? It looks like this might not have actually fixed the flake?

@JoshuaKGoldberg JoshuaKGoldberg added the status: waiting for author waiting on response from OP or other posters - more information needed label May 16, 2026
@mostafaNazari702 mostafaNazari702 force-pushed the fix/watch-flaky-test-outside-cwd branch from 62ee929 to 54dcfda Compare May 16, 2026 15:15
@mostafaNazari702

mostafaNazari702 commented May 16, 2026

Copy link
Copy Markdown
Contributor Author

Accidental push without committing closed the PR, excuse me.
Anyway, looked into and rewrote the helper synchronisation.

@JoshuaKGoldberg JoshuaKGoldberg added status: needs review a maintainer should (re-)review this pull request and removed status: waiting for author waiting on response from OP or other posters - more information needed labels May 18, 2026
@mostafaNazari702 mostafaNazari702 force-pushed the fix/watch-flaky-test-outside-cwd branch from 23e44f8 to 904bd70 Compare May 22, 2026 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: needs review a maintainer should (re-)review this pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🛠️ Repo: CI run failures (test failures)

3 participants