leakcheck: Fix flaky test TestCheck by Pranjali-2501 · Pull Request #8309 · grpc/grpc-go

Pranjali-2501 · 2025-05-09T18:17:20Z

leakcheck.TestCheck and leakcheck.TestCheckRegisterIgnore is flaky due to a gouroutine leak caused by context.WithTimeout().
Flakiness was filed in #8276.

RELEASE NOTES: N/A

linux-foundation-easycla · 2025-05-09T18:17:25Z

The committers listed above are authorized under a signed CLA.

✅ login: Pranjali-2501 (67f7beb, baf4db3, f4a565e, 01e4247, 9af90a0)

codecov · 2025-05-09T18:22:19Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.19%. Comparing base (7fb5738) to head (baf4db3).
Report is 17 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8309      +/-   ##
==========================================
+ Coverage   82.15%   82.19%   +0.04%     
==========================================
  Files         419      419              
  Lines       41904    41990      +86     
==========================================
+ Hits        34426    34515      +89     
+ Misses       6013     6008       -5     
- Partials     1465     1467       +2

see 31 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

dfawley · 2025-05-09T18:19:43Z

internal/leakcheck/leakcheck_test.go

 	defer cancel()
-	CheckGoroutines(ctx, e)
-	if e.errorCount != leakCount {
+	if CheckGoroutines(ctx, e); e.errorCount == 0 {


That works, or you can do

Suggested change

if CheckGoroutines(ctx, e); e.errorCount == 0 {

if CheckGoroutines(ctx, e); ctx.Err() == nil {

Or even:

Suggested change

if CheckGoroutines(ctx, e); e.errorCount == 0 {

if CheckGoroutines(ctx, e); e.errorCount < 3 {

This check expects that the goroutines spawned by time.Sleep() should still be alive.
I have not put e.errorcount == 3, because ctx can cause its goroutine live longer and it will cause a errorcount of 4 (Because of this the test was flaky)

I have not put e.errorcount == 3, because ctx can cause its goroutine live longer and it will cause a errorcount of 4

Right, which is why I suggested changing to failing if it's less than three. We should have at least three. Or we can just confirm the check failed by checking ctx.Err() and then not have to worry about all the complexity to track errorCount at all. Either way is fine.

Right, which is why I suggested changing to failing if it's less than three. We should have at least three. Or we can just confirm the check failed by checking ctx.Err() and then not have to worry about all the complexity to track errorCount at all. Either way is fine.

Right, make sense. Both works fine. I'm going with e.errorcount < 3for now.

dfawley · 2025-05-09T18:23:16Z

internal/leakcheck/leakcheck_test.go

+
+	// "sync"
+


Please revert this and below.

dfawley

Please also change the PR first comment to include "Fixes #". This will ensure the PR and issue are linked and close the issue when the PR is merged.

dfawley · 2025-05-12T15:19:50Z

internal/leakcheck/leakcheck_test.go

 	defer cancel()
-	CheckGoroutines(ctx, e)
-	if e.errorCount != leakCount {
+	if CheckGoroutines(ctx, e); e.errorCount == 0 {


I have not put e.errorcount == 3, because ctx can cause its goroutine live longer and it will cause a errorcount of 4

Right, which is why I suggested changing to failing if it's less than three. We should have at least three. Or we can just confirm the check failed by checking ctx.Err() and then not have to worry about all the complexity to track errorCount at all. Either way is fine.

dfawley · 2025-05-12T15:21:15Z

internal/leakcheck/leakcheck_test.go

@@ -47,16 +47,14 @@ func TestCheck(t *testing.T) {
 	for i := 0; i < leakCount; i++ {
 		go func() { time.Sleep(2 * time.Second) }()


Can you change this to not use Sleep? We should always avoid sleeps for correctness.

Instead do:

ch := make(chan struct{}) for i := 0; i < leakCount; i++ { go func() { <-ch }() } // test interestingGoroutines() and CheckGoroutines() close(ch) // make goroutines exit

dfawley

Looks good after a few minor nits are fixed. Thanks!

internal/leakcheck/leakcheck_test.go

arjan-bal · 2025-05-13T06:47:29Z

internal/leakcheck/leakcheck_test.go

-	if e.errorCount != leakCount {
-		t.Errorf("CheckGoroutines found %v leaks, want %v leaks", e.errorCount, leakCount)
+	if CheckGoroutines(ctx, e); e.errorCount < leakCount {
+		t.Errorf("CheckGoroutines() = %v, want count %v", e.errorCount, leakCount)


"want count < %d" to more accurately describe the assertion.

Checking for e.errorcount >= leakcount, not less than leakcount.

Want e.errorcount >= leakcount because ctx can cause its spawned goroutine to leak, which causes flakiness in the test.

modify test to remove flakyness

f4a565e

Pranjali-2501 added this to the 1.73 Release milestone May 9, 2025

Pranjali-2501 requested review from arjan-bal and dfawley May 9, 2025 18:17

Pranjali-2501 self-assigned this May 9, 2025

Pranjali-2501 added the Type: Bug label May 9, 2025

dfawley reviewed May 9, 2025

View reviewed changes

revert commented lines

67f7beb

Pranjali-2501 requested a review from dfawley May 12, 2025 03:51

dfawley reviewed May 12, 2025

View reviewed changes

replace sleep() with channel

9af90a0

Pranjali-2501 requested a review from dfawley May 12, 2025 17:51

dfawley approved these changes May 12, 2025

View reviewed changes

resolving nits

01e4247

arjan-bal reviewed May 13, 2025

View reviewed changes

internal/leakcheck/leakcheck_test.go Show resolved Hide resolved

arjan-bal reviewed May 13, 2025

View reviewed changes

change TestCheckRegisterIgnore

baf4db3

Pranjali-2501 requested a review from arjan-bal May 13, 2025 07:44

arjan-bal changed the title ~~Modify Flaky test: leakcheck.TestCheck~~ leakcheck: Fix flaky test TestCheck May 13, 2025

arjan-bal merged commit b89909b into grpc:master May 13, 2025
15 of 16 checks passed

github-actions bot locked as resolved and limited conversation to collaborators Nov 10, 2025

	if CheckGoroutines(ctx, e); e.errorCount == 0 {
	if CheckGoroutines(ctx, e); ctx.Err() == nil {

		@@ -47,16 +47,14 @@ func TestCheck(t *testing.T) {
		for i := 0; i < leakCount; i++ {
		go func() { time.Sleep(2 * time.Second) }()

Conversation

Pranjali-2501 commented May 9, 2025 • edited by arjan-bal Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

linux-foundation-easycla bot commented May 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented May 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dfawley left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dfawley left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Pranjali-2501 commented May 9, 2025 •

edited by arjan-bal

Loading

linux-foundation-easycla bot commented May 9, 2025 •

edited

Loading

codecov bot commented May 9, 2025 •

edited

Loading