extproc: register filter and parse base and override config by eshitachandwani · Pull Request #9073 · grpc/grpc-go

eshitachandwani · 2026-04-17T08:29:34Z

This PR is part of implementing A93: xds-ext-proc

This PR adds the ext_proc filter and the builder that implements the builder interface. That includes parsing and validating the base and the override config. The registration of the filter is under the GRPC_EXPERIMENTAL_XDS_EXT_PROC_ON_CLIENT flag.

#ext-proc-a93

RELEASE NOTES: None

codecov · 2026-04-17T08:33:51Z

Codecov Report

❌ Patch coverage is 77.37226% with 31 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.03%. Comparing base (06fc26a) to head (db6dac5).
⚠️ Report is 15 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/xds/httpfilter/extproc/ext_proc.go	73.56%	16 Missing and 7 partials ⚠️
internal/xds/httpfilter/extproc/config.go	76.00%	6 Missing ⚠️
internal/xds/httpfilter/extconfig.go	92.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #9073      +/-   ##
==========================================
+ Coverage   80.52%   83.03%   +2.50%     
==========================================
  Files         413      416       +3     
  Lines       33543    33624      +81     
==========================================
+ Hits        27012    27921     +909     
+ Misses       4316     4268      -48     
+ Partials     2215     1435     -780

Files with missing lines	Coverage Δ
internal/xds/httpfilter/extconfig.go	`92.00% <92.00%> (ø)`
internal/xds/httpfilter/extproc/config.go	`76.00% <76.00%> (ø)`
internal/xds/httpfilter/extproc/ext_proc.go	`73.56% <73.56%> (ø)`

... and 47 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

mbissa · 2026-04-17T11:15:41Z

/gemini review

gemini-code-assist

Code Review

This pull request implements the Envoy external processing (ext_proc) HTTP filter for xDS, including configuration parsing for base and override settings. The reviewer noted that the implementation incorrectly uses a non-existent GRPC body processing mode instead of STREAMED and lacks support for TypedStruct types in the configuration parser. As a result, the validation logic and unit tests require updates to correctly handle supported processing modes and additional configuration formats.

mbissa · 2026-04-20T08:47:53Z

LGTM, barring the udpa.type.v1.TypedStruct and xds.type.v3.TypedStruct support comment from gemini. Can go ahead if we get an answer on those.

eshitachandwani · 2026-04-21T03:29:56Z

LGTM, barring the udpa.type.v1.TypedStruct and xds.type.v3.TypedStruct support comment from gemini. Can go ahead if we get an answer on those.

It has been discussed that we only have to support udpa.type.v1.TypedStruct and xds.type.v3.TypedStruct for third-party filters and need not be supported for filters already in xDS

easwars · 2026-04-21T21:25:03Z

+			return nil, fmt.Errorf("ext_proc: %v", err)
+		}
+	}
+	return baseConfig{config: msg}, nil


Why are we storing it in proto form in the internal representation of the config? We should be storing it as a native Go struct that we own.

Basically if we parse and store it in say interceptorConfig we will have difficulty understanding if the override had a nil value and we do not have to override, or we have to override with say false. For example for the FailureModeAllow here . We could change this to constructing a go interceptorConfig struct for base config and using the proto for override to differentiate between no override and override with false and then use the override proto to override the necessary fields. But I think keeping the parsing logic same for both makes more sense to me. WDYT?

I would like to simplify this by saying that if an override config exists, it will have to explicitly set this value to true to allow RPCs to continue upon failure. If this field is not set (or false), we will fail RPCs. If we go with that approach, we don't have to store the proto form. Please do check with other language implementations. I don't exactly remember if their proto API allows for presence checks, and if they plan on using them.

changes to be introduced as per grpc service design warrant changes in this code

easwars · 2026-05-04T23:03:00Z

+	"google.golang.org/grpc/internal/xds/matcher"
+	"google.golang.org/grpc/metadata"
+
+	mutationpb "github.com/envoyproxy/go-control-plane/envoy/config/common/mutation_rules/v3"


Nit: a v3 on this import rename as well to be consistent?

easwars · 2026-05-04T23:16:25Z

+			return nil, fmt.Errorf("ext_proc: %v", err)
+		}
+	}
+	return baseConfig{config: msg}, nil


I would like to simplify this by saying that if an override config exists, it will have to explicitly set this value to true to allow RPCs to continue upon failure. If this field is not set (or false), we will fail RPCs. If we go with that approach, we don't have to store the proto form. Please do check with other language implementations. I don't exactly remember if their proto API allows for presence checks, and if they plan on using them.

easwars · 2026-05-04T23:19:35Z

+type serverConfig struct {
+	// targetURI is the name of the external server.
+	targetURI string
+	// channelCredentials specifies the transport credentials to use to connect to
+	// the external server. Must not be nil.
+	channelCredentials credentials.TransportCredentials
+	// callCredentials specifies the per-RPC credentials to use when making calls
+	// to the external server.
+	callCredentials []credentials.PerRPCCredentials
+	// timeout is the RPC timeout for the call to the external server. If unset,
+	// the timeout depends on the usage of this external server. For example,
+	// cases like ext_authz and ext_proc, where there is a 1:1 mapping between the
+	// data plane RPC and the external server call, the timeout will be capped by
+	// the timeout on the data plane RPC. For cases like RLQS where there is a
+	// side channel to the external server, an unset timeout will result in no
+	// timeout being applied to the external server call.
+	timeout time.Duration
+	// initialMetadata is the additional metadata to include in all RPCs sent to
+	// the external server.
+	initialMetadata metadata.MD
+}


Based on some of our offline discussions, this stuct must be comparable. So, this cannot store the credentials as interfaces, but instead should store them as JSON strings.

easwars · 2026-05-04T23:20:49Z

+	// failureModeAllow specifies the behavior when the RPC to the external
+	// processing server fails. If true, the dataplane RPC will be allowed to
+	// continue. If false, the data plane RPC will be failed with a grpc status
+	// code of UNAVAILABLE.
+	failureModeAllow *bool
+	// processingModes specifies the processing mode for each dataplane event.
+	processingModes *processingModes


Why do these fields have to pointers? I understand they can come either from the base config or the override config. But I don't see the reason for them being pointers.

The reason for them being pointers is that even in override proto , the override needs to happen per field. The fileds of override proto can either be present and need to be overwritten or not be present. And since we construct the interceptorConfig while parsing and merge the interceptorConfigs together, we need a way to have the differentiation between no override vs false override in the override interceptorConfig.

THe other way is what I had earlier, i.e. do the required validation and store it as a proto when parsing and then create the interceptorCOnfig only once when building the interceptor using both the protos.

THe pro of current way is that we discussed that parsing functions will have bootstrap and server info and that is where we will need to call the grpcServiceProtoToInternalRep utility from. WDYT>

easwars · 2026-05-04T23:22:32Z

+	//
+	// mutationRules specifies the rules for what modifications an external
+	// processing server may make to headers/trailers sent to it.
+	mutationRules headerMutationRules


Since this will be used by ext_authz as well, can you please find a common location for this?

Moved this and serverConfig and related functions to a extconfig.go file in httpfilter package so that it can be shared across both ext_proc and ext_authz filters or any other filters in future.

easwars · 2026-05-04T23:29:01Z

+	if pm == nil {
+		return nil
+	}


What is the expected behavior in this case? For the base config, this is a required field.

And in the override config section of the gRFC, it says "Same as in top-level filter config". Does that mean that if the override is specified, this field must be set?

I dont think that should be the case , but let me confirm.

easwars · 2026-05-04T23:30:23Z

+func validateServerConfig(cfg *serverConfig) error {
+	// TODO(https://github.com/grpc/grpc-go/issues/8747): Once we have a common
+	// way to validate a target URI, switch to that.
+	if cfg.targetURI == "" {
+		return fmt.Errorf("extproc: targetURI must be a non-empty string")
+	}
+	if cfg.channelCredentials == nil {
+		return fmt.Errorf("extproc: channelCredentials must be non-nil")
+	}
+	if cfg.timeout < 0 {
+		return fmt.Errorf("extproc: timeout must be non-negative")
+	}
+	for k, vals := range cfg.initialMetadata {
+		if len(k) == 0 || len(k) >= 16384 {
+			return fmt.Errorf("extproc: initialMetadata key %q has invalid length %d; must be in range [1, 16384)", k, len(k))
+		}
+		for _, v := range vals {
+			if len(v) >= 16384 {
+				return fmt.Errorf("extproc: initialMetadata value for key %q has invalid length %d; must be less than 16384", k, len(v))
+			}
+		}
+	}
+	return nil
+}


The common function that takes care of converting the GrpcService proto to its internal representation should take care of these validation checks.

Okay! removing it. I was not sure if that utility would include validation or not and so added it to be safe.

easwars · 2026-05-04T23:31:42Z

+	iCfg := interceptorConfig{
+		processingModes:          processingModesFromProto(msg.GetProcessingMode()),
+		requestAttributes:        msg.GetRequestAttributes(),
+		responseAttributes:       msg.GetResponseAttributes(),
+		disableImmediateResponse: msg.GetDisableImmediateResponse(),
+		observabilityMode:        msg.GetObservabilityMode(),
+	}


Can this literal struct initialization be done later, after all checks that can result in early returns are complete.

easwars · 2026-05-04T23:32:22Z

+	if msg.GetGrpcService() == nil {
+		return nil, fmt.Errorf("extproc: empty grpc_service provided in config %v", cfg)
+	}
+	if msg.GetGrpcService().GetGoogleGrpc() == nil {


This should also be taken care of by the common function, right?

Changed. I was not sure if these validations would be part of the utiltiy or not.

easwars · 2026-05-04T23:35:34Z

+	if allowed := msg.GetForwardRules().GetAllowedHeaders(); allowed != nil {
+		allowedHeaders, err := convertStringMatchers(allowed.GetPatterns())
+		if err != nil {
+			return nil, err
+		}
+		iCfg.allowedHeaders = allowedHeaders
+	}
+
+	if disallowed := msg.GetForwardRules().GetDisallowedHeaders(); disallowed != nil {
+		disallowedHeaders, err := convertStringMatchers(disallowed.GetPatterns())
+		if err != nil {
+			return nil, err
+		}
+		iCfg.disallowedHeaders = disallowedHeaders
+	}


Would it make sense to have a type to represent the "forwarding rules", such that it has a method that accepts a list of headers, and returns a list of headers based on the rules. That way, the filter will just have a single line of code, and all the logic will be inside this type.

I am not sure if I understand your point completely but I planned to have a helper function constructHeaderMap that will take the allowedHeaders, disallowedHeaders and a list of headers and do the computation. Same as we have for ext_authz here

eshitachandwani added 2 commits April 17, 2026 13:44

add ext_proc parsing

11a461a

add env config

acce672

eshitachandwani added this to the 1.82 Release milestone Apr 17, 2026

eshitachandwani added the Type: Feature New features or improvements in behavior label Apr 17, 2026

Merge branch 'master' into parsing_proc

04b7720

formatting

2eac09b

eshitachandwani requested review from easwars and mbissa April 17, 2026 09:07

eshitachandwani assigned easwars and mbissa Apr 17, 2026

gemini-code-assist Bot reviewed Apr 17, 2026

View reviewed changes

check error in tes

4473a5d

eshitachandwani assigned eshitachandwani and unassigned easwars and mbissa Apr 20, 2026

add regex validation and proc per route

ac256fd

eshitachandwani assigned easwars and mbissa and unassigned eshitachandwani Apr 20, 2026

easwars reviewed Apr 21, 2026

View reviewed changes

easwars assigned eshitachandwani and unassigned easwars Apr 21, 2026

review comments

96536f2

eshitachandwani requested a review from easwars April 23, 2026 19:36

eshitachandwani assigned easwars and unassigned eshitachandwani Apr 23, 2026

mbissa previously approved these changes Apr 28, 2026

View reviewed changes

mbissa self-requested a review April 28, 2026 10:40

eshitachandwani assigned eshitachandwani and unassigned easwars and mbissa Apr 28, 2026

change the base and override cfg

2d2895c

eshitachandwani assigned easwars and mbissa and unassigned eshitachandwani Apr 29, 2026

easwars reviewed May 4, 2026

View reviewed changes

easwars assigned eshitachandwani and unassigned easwars and mbissa May 4, 2026

eshitachandwani added 2 commits May 6, 2026 00:26

change creds to json.Rawmsg, move config to new file common for proc

60eff96

remove validation

db6dac5

eshitachandwani requested a review from easwars May 5, 2026 19:34

eshitachandwani assigned easwars and mbissa and unassigned eshitachandwani May 5, 2026

Conversation

eshitachandwani commented Apr 17, 2026

Uh oh!

codecov Bot commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mbissa commented Apr 17, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mbissa commented Apr 20, 2026

Uh oh!

eshitachandwani commented Apr 21, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

codecov Bot commented Apr 17, 2026 •

edited

Loading