Don't let active logical broker connections hinder new connects (#2266) #2316
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The logical broker connections (such as to the group coordinator) are reserved for specific use and shall not be reused for things like Metadata requests. The code to lookup a usable broker to send Metadata requests understood this, but the code to select a broker to create a new connection to if no usable connections are found (sparse connnections) did not understand this and already thought there was an active connection (the logical broker connection) and refused to set up a new one. This could lead to consumers stalling consumption if it was fetching from a single broker and that broker went down, no new connection to the cluster would be made to refresh metadata.
Closed
7 tasks
rnpridgeon
approved these changes
May 9, 2019
This was referenced May 9, 2019
Closed
|
@edenhill, this seems like a fairly important fix. Are you going to release 1.0.1 with this fix in it soon? Curious if this bug would cause |
Contributor
Author
|
Yes, it is part of the v1.0.1 release, v1.0.1-RC1 was tagged yesterday, we're looking to do the final release later this week. This bug could cause a consumer to not consume any messages, effectively making consumer_poll() block indefinitely |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The logical broker connections (such as to the group coordinator)
are reserved for specific use and shall not be reused for things like
Metadata requests. The code to lookup a usable broker to send
Metadata requests understood this, but the code to select a broker
to create a new connection to if no usable connections are
found (sparse connnections) did not understand this and already
thought there was an active connection (the logical broker connection)
and refused to set up a new one.
This could lead to consumers stalling consumption if it was fetching
from a single broker and that broker went down, no new connection
to the cluster would be made to refresh metadata.
This fixes #2266