Skip to content

encoding: add compressor options in UseCompressor and SetSendCompressor#9036

Open
Dostonlv wants to merge 1 commit intogrpc:masterfrom
delta-team-uz:feature/compressor-options
Open

encoding: add compressor options in UseCompressor and SetSendCompressor#9036
Dostonlv wants to merge 1 commit intogrpc:masterfrom
delta-team-uz:feature/compressor-options

Conversation

@Dostonlv
Copy link
Copy Markdown

@Dostonlv Dostonlv commented Apr 2, 2026

Fixes #7017

RELEASE NOTES:

  • encoding: Add optional compressorOptions to UseCompressor and SetSendCompressor, forwarded to Compress method.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 2, 2026

Codecov Report

❌ Patch coverage is 57.69231% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.25%. Comparing base (2b8b708) to head (6d65379).
⚠️ Report is 31 commits behind head on master.

Files with missing lines Patch % Lines
stream.go 55.55% 2 Missing and 2 partials ⚠️
rpc_util.go 57.14% 2 Missing and 1 partial ⚠️
server.go 60.00% 0 Missing and 2 partials ⚠️
encoding/gzip/gzip.go 0.00% 0 Missing and 1 partial ⚠️
internal/transport/server_stream.go 75.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #9036      +/-   ##
==========================================
- Coverage   83.01%   82.25%   -0.77%     
==========================================
  Files         413      413              
  Lines       33054    33315     +261     
==========================================
- Hits        27441    27403      -38     
- Misses       4205     4254      +49     
- Partials     1408     1658     +250     
Files with missing lines Coverage Δ
encoding/encoding.go 68.96% <ø> (-6.90%) ⬇️
encoding/gzip/gzip.go 60.00% <0.00%> (-2.50%) ⬇️
internal/transport/server_stream.go 83.87% <75.00%> (-12.74%) ⬇️
server.go 82.03% <60.00%> (-0.93%) ⬇️
rpc_util.go 62.43% <57.14%> (-20.55%) ⬇️
stream.go 62.91% <55.55%> (-19.42%) ⬇️

... and 40 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Dostonlv
Copy link
Copy Markdown
Author

Dostonlv commented Apr 7, 2026

@easwars assalamu alaykum could you review my code?

Copy link
Copy Markdown
Contributor

@Pranjali-2501 Pranjali-2501 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Dostonlv , Thanks for raising the PR.

I have added a few comments, PTAL.

Comment thread stream.go
Comment thread server.go Outdated
Comment thread server.go
Comment thread rpc_util.go
@Pranjali-2501 Pranjali-2501 added the Type: Feature New features or improvements in behavior label Apr 7, 2026
@Pranjali-2501 Pranjali-2501 added this to the 1.81 Release milestone Apr 7, 2026
@Dostonlv Dostonlv force-pushed the feature/compressor-options branch from 36a0335 to a572da0 Compare April 9, 2026 15:33
@Dostonlv Dostonlv force-pushed the feature/compressor-options branch from a572da0 to 6d65379 Compare April 9, 2026 15:47
@Dostonlv
Copy link
Copy Markdown
Author

Dostonlv commented Apr 9, 2026

@Pranjali-2501 Assalamu alaykum added tests to verify compressor options are correctly propagated. Please review.

@Pranjali-2501
Copy link
Copy Markdown
Contributor

@arjan-bal, assigning it to you for second review.

@arjan-bal arjan-bal added Type: API Change Breaking API changes (experimental APIs only!) and removed Type: Feature New features or improvements in behavior labels Apr 22, 2026
@arjan-bal arjan-bal changed the title encoding: support compressor options in Compress method encoding: add compressor options in UseCompressor and SetSendCompressor Apr 22, 2026
@arjan-bal
Copy link
Copy Markdown
Contributor

I left a few questions on the proposal: #7017 (comment)

I'll hold off on the PR review until we get confirmation on those points.

type wrapCompressor struct {
encoding.Compressor
compressInvokes int32
mu sync.Mutex
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use the mutex to protect compressInvokes too. Unifying the synchronization mechanisms will make the code much simpler.

Additionally, please use empty lines to group the mutex-guarded fields, similar to this example:

// mu protects the following fields and all fields within balancerCurrent
// and balancerPending. mu does not need to be held when calling into the
// child balancers, as all calls into these children happen only as a direct
// result of a call into the gracefulSwitchBalancer, which are also
// guaranteed to be synchronous. There is one exception: an UpdateState call
// from a child balancer when current and pending are populated can lead to
// calling Close() on the current. To prevent that racing with an
// UpdateSubConnState from the channel, we hold currentMu during Close and
// UpdateSubConnState calls.
mu sync.Mutex
balancerCurrent *balancerWrapper
balancerPending *balancerWrapper
closed bool // set to true when this balancer is closed

Comment on lines +255 to +256
{"unary", testUnarySendCompressorOptionsPropagate},
{"stream", testStreamSendCompressorOptionsPropagate},
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: The helper seems to be abstracting most of the test logic here. I would recommend avoiding the table driven approach and have separate tests for unary and streaming RPCs, inlining the helpers. Same for the TestUseCompressorOptionsPropagate test.

See https://google.github.io/styleguide/go/best-practices#leave-testing-to-the-test-function

if len(wc.receivedOpts) == 0 {
t.Fatal("Compress was not called")
}
if got := wc.receivedOpts[0]; len(got) == 0 || got[0] != wantOpt {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: You can use cmp.Diff to compare the entire slice.

@arjan-bal arjan-bal assigned Dostonlv and unassigned arjan-bal May 6, 2026
@arjan-bal
Copy link
Copy Markdown
Contributor

arjan-bal commented May 8, 2026

After discussing this with the other maintainers, we agreed that we need to provide a migration path for custom compressor implementers rather than breaking the API in a single release. We will need a transitional gRPC version where both the old and new APIs coexist before we fully remove the old one. I am currently evaluating the best approach to achieve this.

#8683

@arjan-bal arjan-bal self-assigned this May 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Type: API Change Breaking API changes (experimental APIs only!)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New Compressor: Support metadata other than Name for trained compression

3 participants