Static/Unmanaged Gateway Mode (vs Provisioning)

[shaneutt]

(follow-up from GAMMA meeting 2023-01-31)

In the past we've had discussions about a mode of operation for Gateway API which we'll call "static mode" (previously called "unmanaged mode" in some old issues) wherein a Gateway resource refers to a gateway that has been provisioned by some other means (e.g. via Helm) but the implementation may or may not actually manage the lifecycle of that gateway.

Multiple implementations now have options for such a mode which can be particularly helpful for allowing legacy/pre-operator deployments to have Gateway API functionality. Implementations that do this include (but are not limited to):

• Contour
• Istio
• Kong
• NGinx

Today there are prospective implementations of GAMMA (such as LinkerD) which are in need of such a mode, but currently that mode is a colloquialism, and not properly documented or accounted for here in upstream Gateway API.

TODO:

• add documentation that prescribes implementing "static" or "unmanaged" Gateway mode
• make any provisions necessary in the conformance tests to accommodate such an implementation, and/or potentially add this as a separate conformance category

Prior art: #892, #925

[howardjohn]

Here is how Istio does it: https://istio.io/latest/docs/tasks/traffic-management/ingress/gateway-api/#manual-deployment

The idea was to match how cloud LB do it, where you point to an existing LB by address

[shaneutt]

Thanks @howardjohn!

cc @kflynn

[sunjayBhatia]

similarly here for Contour: https://projectcontour.io/docs/v1.23.2/guides/gateway-api/#option-1-statically-provisioned

[mlavacca]

This is how Kong deals with it: https://docs.konghq.com/kubernetes-ingress-controller/latest/concepts/gateway-api/#binding-kong-gateway-to-a-gateway-resource

Instead of relying on static/unmanaged gateways, we use static/unmanaged gatewayClasses, i.e., a Gateway is considered unmanaged if it uses an unmanaged GatewayClass.

[kate-osborn]

NGINX only supports "static" mode. We support one Gateway resource that is provisioned and managed by the user. We'd love to be able to run the conformance tests against our implementation via a static mode option or something similar.

[kflynn]

I've run into this with both Emissary-ingress and Linkerd. The Emissary-ingress case is simplest to describe, so I'll do that first: like NGINX as described by @kate-osborn, Emissary only supports "static" mode. It would be lovely to be able to have Emissary pass conformance, and considerably less lovely to have to completely change how Emissary works to do so.

The Linkerd case is one I saw firsthand when getting Envoy Gateway and Linkerd to play nicely together, so let me use that experience as an illustration. Please note that I'm not trying to argue that either Envoy Gateway or Linkerd is "wrong" here -- rather, I'm illustrating a point of friction.

The important things to know about Linkerd here are that:

• From Linkerd's perspective, the ingress is just another meshed workload.

• Linkerd requires workloads to have the linkerd.io/inject annotation to be included in the mesh. Typically this gets included in the Pod template for e.g. a Deployment. It can also be applied to a namespace, in which case it will be applied to every Pod created in that namespace.

Using an ingress installed using "static mode" with Linkerd is really easy: you install the ingress using your favorite installation method, make sure its Pods get the linkerd.io/inject annotation, and off you go. How exactly you apply the annotation depends on the installation mechanism, but they're all basically the same in the end.

If your ingress doesn't support "static mode", it's harder. In Envoy Gateway's case, it has Envoy proxy Pods that need the annotation, but that get recreated any time you touch the Gateway resource. Since you don't have control over the life cycle of those Pods, and you also don't have access to the Envoy Gateway Deployment, you have to annotate the envoy-gateway-system namespace to get Envoy Gateway into the mesh. Functionally, this is fine. Operationally, it's extra friction for this particular ingress: it would be simpler for many end users to give them the ability to just install Envoy Gateway statically.