spring cloud gateway + python grpc server does not work

[luzengxiangcn]

Describe the bug
I am a new user of spring cloud gateway. Got following warning, when I went through tutorials.

2022-04-28 18:52:48.545  WARN 35850 --- [-worker-ELG-3-1] i.g.n.s.i.grpc.netty.NettyServerHandler  : Expected header TE: trailers, but null is received. This means some intermediate proxy may not support trailers

Next Step: I replaced java grpc-server with python grpc-server, this warning turned into a fatal error.

D0428 19:12:03.422596000 123145451761664 server.cc:1334]               Failed call creation:  
 {"created":"@1651144323.422515000","description":"Missing :authority or    \:path","file":"src/core/lib/surface/server.cc","file_line":1384,"referenced_errors":  
 [{"created":"@1651144323.422509000","description":"Failed processing incoming   headers","file":"src/core/ext/filters/http/server/http_server_filter.cc","file_line":111,"referenced_errors":   
 [{"created":"@1651144323.422504000","description":"Missing  header","file":"src/core/ext/filters/http/server/http_server_filter.cc","file_line":160,"key":"te"}]}]}  

I think both messages are describing one same thing. Could anyone help me have a look?

It seems like following code is not working

	@Bean
	@ConditionalOnProperty(name = "server.http2.enabled", matchIfMissing = true)
	public GRPCRequestHeadersFilter gRPCRequestHeadersFilter() {
		return new GRPCRequestHeadersFilter();
	}

ENV:
MacBook (Retina, 12-inch, Early 2016)

[luzengxiangcn]

@Albertoimpl Could you please have a look?

[Albertoimpl]

Sure thing! Could you please provide me a repo with the client, server, and the configured gateway so I can take a deeper look at it @luzengxiangcn?

[luzengxiangcn]

Sure thing! Could you please provide me a repo with the client, server, and the configured gateway so I can take a deeper look at it @luzengxiangcn?

Sure, a repo is created.
https://github.com/luzengxiangcn/grpc-python-server.git

Thanks a lot. Please let me know, if you need anything else.

[Albertoimpl]

Thanks a lot for filing this issue. It is indeed something we are not doing in spring-cloud-gateway that behaves differently when using a grpc-python-server.

The problem is that we are not passing through some of the pseudo-headers: https://datatracker.ietf.org/doc/html/rfc7540#section-8.1.2.3 and, apparently, the default grpc-python-server implementation is very specific about this, unlike the Java one, which is fine without those pseudo-headers.

If you must use a server implementation that requires these pseudo-headers, the only workaround I can think of is by translating the request and creating a channel that makes the calls like in this example: https://github.com/Albertoimpl/spring-cloud-gateway-grpc/blob/main/grpc-json-gateway/src/main/java/com/example/grpcserver/hello/JSONToGRPCFilterFactory.java#L87-L95

I have a better version of this code that I am working on: https://github.com/Albertoimpl/spring-cloud-gateway-grpc/tree/grpc-json-native-filter/grpc-json-native-filter-gateway and I tested it with the grpc-python-server and it works well.

I will take a deeper look and see if there is something we can do, but I presume it will require work all the way from the reactor-netty project and some changes to our GRPCHeadersFilters to include those pseudo-headers.

[Albertoimpl]

After some extra debug, I found that we are properly passing all the HTTP/2 headers but that we had a problem in the code that was removing an HTTP/1.1 header when we should not.
This should fix the issue @luzengxiangcn.

[spencergibb]

Closed via #2614