Skip to content

Pipe: Fix connection leak caused by clients not closed after task dropped (2 situations) #15910

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Jul 11, 2025

Conversation

SteveYurongSu
Copy link
Member

@SteveYurongSu SteveYurongSu commented Jul 11, 2025

This pull request introduces several changes across multiple files to improve connection management, enhance error handling, and ensure proper resource cleanup. The most notable updates include replacing direct clientManager calls with a getClientManager() method for safer access, adding checks to prevent operations after closure, and improving logging for debugging purposes.

Connection Management Improvements:

Resource Cleanup Enhancements:

Error Handling Improvements:

Operational Safety Enhancements:

These changes collectively enhance the robustness of the system by ensuring safer client management, preventing resource leaks, and improving error handling and operational safety.

Adds additional checks and cleanup logic when returning or closing clients, especially when nodes are null or connectors are closed. Enhances error logging and ensures resources are properly invalidated to prevent leaks. Also makes the close() method in AsyncPipeDataTransferServiceClient public for broader access.
@SteveYurongSu SteveYurongSu marked this pull request as ready for review July 11, 2025 03:25
Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
B Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Copy link

codecov bot commented Jul 11, 2025

Codecov Report

Attention: Patch coverage is 8.69565% with 42 lines in your changes missing coverage. Please review.

Project coverage is 39.07%. Comparing base (966cfc7) to head (f934d31).
Report is 8 commits behind head on master.

Files with missing lines Patch % Lines
...hrift/async/handler/PipeTransferTsFileHandler.java 0.00% 12 Missing ⚠️
...nector/client/IoTDBDataNodeAsyncClientManager.java 0.00% 10 Missing ⚠️
...pipe/connector/protocol/IoTDBSslSyncConnector.java 0.00% 6 Missing ⚠️
...org/apache/iotdb/commons/client/ClientManager.java 37.50% 5 Missing ⚠️
...connector/protocol/IoTDBConfigRegionConnector.java 0.00% 4 Missing ⚠️
...ol/thrift/async/IoTDBDataRegionAsyncConnector.java 0.00% 2 Missing ⚠️
...ft/async/handler/PipeTransferTrackableHandler.java 0.00% 2 Missing ⚠️
...t/task/subtask/connector/PipeConnectorSubtask.java 50.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #15910      +/-   ##
============================================
- Coverage     39.09%   39.07%   -0.02%     
  Complexity      198      198              
============================================
  Files          4847     4847              
  Lines        315799   315914     +115     
  Branches      39658    39672      +14     
============================================
- Hits         123448   123434      -14     
- Misses       192351   192480     +129     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@SteveYurongSu SteveYurongSu merged commit bf8329b into master Jul 11, 2025
58 of 60 checks passed
@SteveYurongSu SteveYurongSu deleted the fix-pipe-connection-leak branch July 11, 2025 07:10
JackieTien97 pushed a commit that referenced this pull request Jul 11, 2025
SteveYurongSu added a commit that referenced this pull request Jul 14, 2025
SteveYurongSu added a commit that referenced this pull request Jul 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants