Add API details GEP-3779 - E/W Authorization #3891

LiorLieberman · 2025-06-30T17:09:09Z

Add API details for GEP-3779.

More comprehensive comparison of targetRef and options for L4 and L7 authz apis is available here in https://docs.google.com/document/d/1CeagBnHDPbzpYAxBmtJqTshxAW8l-aRPwvwgAGbnv2I/edit?tab=t.0

/kind gep

Related #3779

Does this PR introduce a user-facing change?:

NONE

robscott

Thanks @LiorLieberman!

geps/gep-3779/index.md

robscott · 2025-07-01T00:41:53Z

geps/gep-3779/index.md

+
+##### **Experimental Pattern**
+
+To mitigate some of the concerns, `LabelSelector` support in policy attachment is designated as an **experimental pattern**.


For the record, I really hate the combination of label selectors and policy attachment because we'd be combining 2 topics that are already quite complicated. With that said, this has been a widely requested feature, and testing it out in a relatively controlled way seems ~ok to me. We need to be ready to pivot to another approach if this doesn't work and/or ends up being too complicated.

I remain convinced that we should start with only label selectors rather than with the combination.

(I should point out that this opinion should not be considered a blocker for experimental. I just don't like the idea. 😂)

robscott · 2025-07-01T00:45:11Z

geps/gep-3779/index.md

+    // While the exact workload identifier structure is implementation-specific,
+    // implementations are encouraged to follow the convention of
+    // `spiffe://<trust_domain>/ns/<namespace>/sa/<serviceaccount>`
+    // when representing Kubernetes workload identities.


Why would someone use this matcher to refer to Kubernetes identities when the serviceAccounts list is below? Is this intended primarily for cross-cluster use cases?

I think cross-cluster or external workloads are reasonable examples which might apply.

It's niche at the moment, but I have seen users express desire to be able to assert policy at a sub-serviceAccount granularity. This seems to leave the door open for an implementation to support that use case.

I'm really torn about this.

Ultimately this GEP boils down to attaching authorization policy to things described at layer 4, and what we're specifically looking at in this clause is the definition of a an authorization principal. Do we really want different stanzas for every possible kind of principal? If we split out ServiceAccounts and SPIFFE IDs, should we also split out JWT IDs? What if a given mesh can support Unix IDs in some way, or, I dunno, PGP keys or the like?

Likewise, having the different stanzas reflect where the implementation is getting the information feels very strange to me – forcing Chihiro to think about the way ServiceAccounts come from the mTLS handshake but oh right the JWT ID happens after that seems kind of surreal.

In a lot of ways, I'd rather just have a single set of principals that uses a URI-style specification. SPIFFE already definies this with the spiffe:// URI scheme, and it's not hard to imagine defining a parallel scheme for ServiceAccounts (k8s-sa:// maybe?). It's true that a given implementation might not support a given scheme, but then it's already true that a given implementation might not support the Identities stanza because it doesn't support SPIFFE IDs...

robscott · 2025-07-01T00:47:56Z

geps/gep-3779/index.md

+
+    // Namespaces specifies a list of Kubernetes Namespaces that are matched
+    // by this rule. A request originating from any pod within one of these
+    // namespaces will match the rule, regardless of its specific Service Account.


So this is basically just another way of saying "match all ServiceAccounts in these namespaces"? Given the proposed format of ServiceAccounts is ns/name above, would ns/* be a reasonable alternative way to express this instead of having a separate field in the API?

Thats valid. Happy to remove and support ns/* if we are fine with supporting "*". I am worried we are opening a door for more use of "*" or any regex in value fields here though.

applied the recommendation - ptal

robscott · 2025-07-01T00:51:27Z

geps/gep-3779/index.md

+
+The `targetRef` of the policy specifies the workload(s) to which the policy applies. Two options are available for `targetRef`:
+
+#### **Option 1: Targeting a Service**


I seem to remember "ServiceAccount" being raised as a potential option at a previous KubeCon when this topic came up. I think it ended up being deemed impractical/poor UX, but could be good to cover it here for completeness.

geps/gep-3779/index.md

aryan16 · 2025-07-01T16:42:55Z

geps/gep-3779/index.md

+// for policies implemented within Gateway API. It is currently not intended for general-purpose
+// use outside of Gateway API resources.
+// +kubebuilder:validation:XValidation:rule="!(has(self.selector)) || (self.kind == 'Pod' && (self.group == '' || self.group == 'core'))",message="Selector may only be set when targeting Pods."
+type PolicyTargetReferenceWithLabelSelectors struct {


will this "PolicyTargetReferenceWithLabelSelectors" ever be applicable for Gateways?
Gateways can be in-cluster and defined via K8s workload (Gateway may have a pod with some labels). If someone uses the pod labels for the gateway, do we select that Gateway with these semantics?

I think that's a reasonable question that is one of the reasons that we haven't done label selecctors for pods before.

Tricky, a gateway implemented by a pod in k8s is both a workload unto itself, and also a proxy for 0 or more other "entities". In some cases it might be handy to be able to express that certain identities can talk to a gateway (and by extension, may be allowed to talk to entities that gateway proxies for) independent of expressing policy against all the disparate entities which that gateway is a proxy for.

This is also probably a complexity to worry about in the Service reference option as well. If you have a gateway selected by a service and attach policy to that service, what does it mean?

I would say YES. With the option to exclude it if necessary (using the right label selector combination ,e.g with NotIn operator). Though as Ian pointed out there are cases where it will be useful to be able to distribute the policies to the gateway (which is also a workload in this case).

added sections about it, PTAL!

youngnick · 2025-07-02T03:01:00Z

geps/gep-3779/index.md

+##### **Enhanced Discoverability with `gwctl`**
+
+A key challenge with `LabelSelector` is the loss of discoverability. It’s easier to see which policies target a `Service` but difficult to determine which policies might affect a specific pod.
+
+To address this, **investment in tooling is required.** Specifically, the `gwctl` CLI tool should be enhanced to provide insights such as:
+
+```sh
+TODO: complete gwctl commands
+```
+
+Without dedicated tooling, the `LabelSelector` approach could significantly degrade the user experience and observability.


This section is critical for making label selectors actually be understandable. As it stands, they are the very definition of spooky action at a distance - if you don't know to go and look for an AuthorizationPolicy, then requests will randomly fail or succeed based on things outside your control.

geps/gep-3779/index.md

karthikbox · 2025-06-30T23:49:57Z

geps/gep-3779/index.md

+    // A request matches if its attributes are present in this list.
+    //
+    // +optional
+    Attributes []AuthorizationTCPAttributes `json:"attributes,omitempty"`


QQ: Should we rename this to DestinationAttributes to be clear that these are destination's attributes?

(changed it to NetworkAttributes, and ApplicationAttributes) but wonder what does destination implies here in your opinion? these are request attributes.

karthikbox · 2025-06-30T23:57:31Z

geps/gep-3779/index.md

+* The presence of any authorization policy causes the system to default to **deny-by-default** for matching workloads.
+* Another bullet to re-clarify the one above - the default behavior when no policies select a target workload is to allow all traffic. However, **as soon as at least one `AuthorizationPolicy` targets a workload, the model becomes implicitly deny-if-not-allowed**.
+


i'm a bit confused by this. Perhaps an example here might help me understand this better.

doesn't this contradict the statement from ALLOW policy above - If no ALLOW policy exists for a workload, traffic is permitted by default, unless any DENY policy applies. ?

ok, i understood this now. but would help if there's an example.

geps/gep-3779/index.md

karthikbox · 2025-07-07T18:00:48Z

geps/gep-3779/index.md

+  Kind Kind `json:"kind"`
+
+  // Selector is the label selector of target objects of the specified kind.
+  Selector *metav1.LabelSelector `json:"selector"`


should we add a Section Name here too? for example - port can be a section name . When specified, the policy would apply to the pod port specifically. This would be useful for this AuthZ Policy or some other future auth or "workload" policy.

While this is not my main point, the case for supporting this in AuthZPolicy is that it would allow the authz policy to be evaluated only for traffic destined to this port. As opposed to the current proposal, where the whole policy is evaluated for all traffic (perhaps implementations will optimize this behavior) which is going to wasteful.

I perceive Port as an attribute of the request in this case, rather than the target. Our targets are workloads and not Workload*Port.

As an example, assuming we support that, what would it mean to have allow policy thats targeting workload foo on port 8080.

If port is an attribute of the request - all requests to this workload in other ports are going to be denied (see the semantics above, if there is ANY allow policy, we deny anything thats not specifically allowed)

However, with port being an attribute of the target - do we still do that? if yes, how?

Mentioning this here too: there are places where we've seen folks want to do different policies on different ports.

You can achieve that with the port attribute, wouldn't this be sufficient? The thread here was about whether port is part of the Target or not and the implication of such decision.

LiorLieberman · 2025-07-09T23:09:45Z

/cc @kflynn @mikemorris

aryan16 · 2025-07-11T05:34:45Z

geps/gep-3779/index.md

+type AuthorizationPolicySpec struct {
+    // TargetRef identifies the resource this policy is attached to.
+    // +kubebuilder:validation:Required
+    TargetRefs []gatewayv1.PolicyTargetReferenceWithLabelSelectors `json:"targetRefs"`


why do we need a list here? I think we only accept one pod right?

TargetRef Pod = name must be empty, selector must be set, length must be 1
Otherwise multiple refs as long as target isn't Pods, selector can't be set

Please add a CEL validaion for the same.

LiorLieberman · 2025-07-11T23:57:30Z

/cc @howardjohn

LiorLieberman · 2025-07-16T18:05:32Z

/label v1.4-release/targeting-experimental

k8s-ci-robot · 2025-07-16T18:05:35Z

@LiorLieberman: The label(s) /label v1.4-release/targeting-experimental cannot be applied. These labels are supported: api-review, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, team/katacoda, refactor, ci-short, ci-extended, ci-full. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

In response to this:

/label v1.4-release/targeting-experimental

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

aryan16 · 2025-08-04T20:55:39Z

geps/gep-3779/index.md

+    // AuthorizationSource specifies who is making the request.
+    // If omitted, matches any source.
+    // +optional
+    AuthorizationSource *AuthorizationSource `json:"authorizationSource,omitempty"`


It's better to keep it just "Source" as it is self explanatory as it is in AuthoriztaionPolicy.

good catch! changed

kflynn

Thanks for continuing to work on this, @LiorLieberman! These are generally small things for internal consistency, though I do have a couple more comments to add to other threads inline. Overall I think this lines up much more closely with the discussions in the meetings we've had, thanks again!

geps/gep-3779/index.md

kflynn · 2025-08-22T20:08:28Z

geps/gep-3779/index.md

+type AuthorizationPolicySpec struct {
+    // TargetRef identifies the resource this policy is attached to.
+    // (will be translated to CEL later):
+    //   - When Kind is Pod, Name MUST be Empty, Selector MUST be set, Max targetRef length is 1. 


See previous comments about name.

I;ve added clarifications as we discussed, anything in particular I am missing here?

kflynn · 2025-08-22T20:09:36Z

geps/gep-3779/index.md

+// NOTE: In the future, if there’s a need to express more complex
+// logical conditions (e.g. requiring a request to match multiple
+// criteria simultaneously—logical AND), we may evolve this API
+// to support richer match expressions or logical operators.


Good candidate for the "don't include this in godocs" marker, I think...

I think I added the right one but not sure (gateway:util:excludeFromCRD)

geps/gep-3779/index.md

kflynn · 2025-08-22T20:20:51Z

geps/gep-3779/index.md

+
+### Feature Names
+
+Two new feature sets will be added - AuthorizationPolicyCoreFeatures, and later, AuthorizationPolicyExtendedFeatures.


If the intention is to add e.g. AuthorizationPolicyDeny, AuthorizationPolicyGRPC, etc., do we really need AuthorizationPolicyExtendedFeatures?

Whether or not, if we're not adding it yet, this should be dropped or moved to Future Enhancements.

These are feature sets, you are right that within those sets we will have AuthorizationPolicyDeny AuthorizationPolicyGRPC etc.

The feature set can be used to say "I support all authzpolicy extended, or all authzpolicy core", these are not supportedFeatures by themselves, more like a container.

kflynn · 2025-08-22T21:05:07Z

geps/gep-3779/index.md

+
+###### Option 3 (Recommended)
+
+An empty pod selector. Kubernetes official docs clarify that the semantics of empty selectors are the decision of the API owner. In fact, many Kubernetes APIs (I know Service API does the opposite :/) using empty selectors as a select-all mechanism. See [NetworkPolicy podSelector](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.33/#networkpolicy-v1-networking-k8s-io), PodDisruptionBudget, ResourceQuota, and more.


Re-asking a question from several rounds ago: so are you suggesting that targeting a namespace means targeting all Pods in that namespace? or something else?

(WRT Gateways, note that it is meaningful in many cases for a workload in a cluster to send a request to a Gateway in the same cluster, so it is definitely meaningful to talk about E/W auth when talking to the Gateway. If you view the Gateway as a point where you cross an administrative boundary, it's not even necessarily all that weird.)

Bringing feedback from slack here for posterity.

we do mean targeting all Pods in that namespace.

targeting all Gateways would have to be in a separate policy that target the gateways. We currently dont support namespace-wide ref for something-other-than-pods so one can just have multiple targetRefs that targets the gateways by name (in a separate policy)

kflynn

!@#&!@#& 😡

I apparently had a review pending for days when I thought I was just submitting comments. 😡 Sorry about that, @LiorLieberman, here are the comments.

kflynn · 2025-08-22T21:16:22Z

geps/gep-3779/index.md

+    // While the exact workload identifier structure is implementation-specific,
+    // implementations are encouraged to follow the convention of
+    // `spiffe://<trust_domain>/ns/<namespace>/sa/<serviceaccount>`
+    // when representing Kubernetes workload identities.


I'm really torn about this.

Ultimately this GEP boils down to attaching authorization policy to things described at layer 4, and what we're specifically looking at in this clause is the definition of a an authorization principal. Do we really want different stanzas for every possible kind of principal? If we split out ServiceAccounts and SPIFFE IDs, should we also split out JWT IDs? What if a given mesh can support Unix IDs in some way, or, I dunno, PGP keys or the like?

Likewise, having the different stanzas reflect where the implementation is getting the information feels very strange to me – forcing Chihiro to think about the way ServiceAccounts come from the mTLS handshake but oh right the JWT ID happens after that seems kind of surreal.

In a lot of ways, I'd rather just have a single set of principals that uses a URI-style specification. SPIFFE already definies this with the spiffe:// URI scheme, and it's not hard to imagine defining a parallel scheme for ServiceAccounts (k8s-sa:// maybe?). It's true that a given implementation might not support a given scheme, but then it's already true that a given implementation might not support the Identities stanza because it doesn't support SPIFFE IDs...

kflynn · 2025-08-29T17:56:16Z

geps/gep-3779/index.md

+
+##### **Experimental Pattern**
+
+To mitigate some of the concerns, `LabelSelector` support in policy attachment is designated as an **experimental pattern**.


(I should point out that this opinion should not be considered a blocker for experimental. I just don't like the idea. 😂)

k8s-ci-robot requested review from gcs278 and kflynn June 30, 2025 17:09

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 30, 2025

LiorLieberman changed the title ~~Add API details GEP-3779~~ Add API details GEP-3779 - E/W Authorization Jun 30, 2025

LiorLieberman force-pushed the gep-3779-apis branch from 2aada9b to a410c68 Compare June 30, 2025 22:14

robscott reviewed Jul 1, 2025

View reviewed changes

aryan16 reviewed Jul 1, 2025

View reviewed changes

youngnick reviewed Jul 2, 2025

View reviewed changes

ilrudie reviewed Jul 2, 2025

View reviewed changes

geps/gep-3779/index.md Outdated Show resolved Hide resolved

ilrudie reviewed Jul 2, 2025

View reviewed changes

geps/gep-3779/index.md Outdated Show resolved Hide resolved

ilrudie reviewed Jul 2, 2025

View reviewed changes

geps/gep-3779/index.md Outdated Show resolved Hide resolved

LiorLieberman force-pushed the gep-3779-apis branch from 12b8a42 to 776e35c Compare July 2, 2025 22:41

karthikbox reviewed Jul 7, 2025

View reviewed changes

LiorLieberman requested review from ilrudie and robscott July 9, 2025 23:09

k8s-ci-robot requested a review from mikemorris July 9, 2025 23:09

aryan16 reviewed Jul 11, 2025

View reviewed changes

LiorLieberman force-pushed the gep-3779-apis branch from b6dcc79 to 6ad28ad Compare July 11, 2025 23:57

k8s-ci-robot requested a review from howardjohn July 11, 2025 23:57

shaneutt added the v1.4-release/targeting-experimental This issue is targeting a transition to EXPERIMENTAL as part of the v1.4 release. label Jul 16, 2025

shaneutt added this to Release v1.4.0 Jul 16, 2025

shaneutt added this to the v1.4.0 milestone Jul 16, 2025

shaneutt removed this from Release v1.4.0 Jul 30, 2025

shaneutt removed this from the v1.4.0 milestone Jul 30, 2025

shaneutt removed their assignment Jul 30, 2025

shaneutt mentioned this pull request Jul 30, 2025

GEP: Support E/W authorization #3779

Open

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 30, 2025

aryan16 reviewed Aug 4, 2025

View reviewed changes

LiorLieberman force-pushed the gep-3779-apis branch from 684faeb to 61da0f2 Compare August 6, 2025 00:43

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 6, 2025

LiorLieberman added 13 commits August 6, 2025 20:47

add api details gep-3779

a3d3ad4

fmting

3b3f688

some-fixes

e373fa0

some-fixes

03399e4

add Namespace wide section and more clarifications

724b41d

add clarifications and link to google doc for targetRef

e99e0d6

more feedback

1d17716

more feedback

7c53429

add feedback and agreements from meeting

876fdd5

formatting

1391dd4

add guardrails

a2499ca

organize the GEP

9aa4019

remove l7 and deny language from the gep

c81d714

LiorLieberman force-pushed the gep-3779-apis branch from 61da0f2 to c81d714 Compare August 6, 2025 20:48

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 6, 2025

LiorLieberman requested a review from mikemorris August 6, 2025 20:48

kflynn suggested changes Aug 22, 2025

View reviewed changes

kflynn reviewed Aug 22, 2025

View reviewed changes

LiorLieberman added 2 commits August 25, 2025 21:20

flynns feedback

e0e06b0

clarify whats being targeted in a namespace-wide policy

950c663

kflynn reviewed Aug 29, 2025

View reviewed changes


		##### Experimental Pattern

		To mitigate some of the concerns, `LabelSelector` support in policy attachment is designated as an experimental pattern.


		The `targetRef` of the policy specifies the workload(s) to which the policy applies. Two options are available for `targetRef`:

		#### Option 1: Targeting a Service

		* The presence of any authorization policy causes the system to default to deny-by-default for matching workloads.
		* Another bullet to re-clarify the one above - the default behavior when no policies select a target workload is to allow all traffic. However, as soon as at least one `AuthorizationPolicy` targets a workload, the model becomes implicitly deny-if-not-allowed.


		### Feature Names

		Two new feature sets will be added - AuthorizationPolicyCoreFeatures, and later, AuthorizationPolicyExtendedFeatures.


		###### Option 3 (Recommended)

		An empty pod selector. Kubernetes official docs clarify that the semantics of empty selectors are the decision of the API owner. In fact, many Kubernetes APIs (I know Service API does the opposite :/) using empty selectors as a select-all mechanism. See [NetworkPolicy podSelector](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.33/#networkpolicy-v1-networking-k8s-io), PodDisruptionBudget, ResourceQuota, and more.

Add API details GEP-3779 - E/W Authorization #3891

Are you sure you want to change the base?

Add API details GEP-3779 - E/W Authorization #3891

Conversation

LiorLieberman commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

robscott left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ilrudie Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kflynn Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LiorLieberman Jul 8, 2025 • edited by robscott Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LiorLieberman Jul 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LiorLieberman commented Jun 30, 2025 •

edited

Loading

ilrudie Jul 2, 2025 •

edited

Loading

kflynn Aug 22, 2025 •

edited

Loading

LiorLieberman Jul 8, 2025 •

edited by robscott

Loading

LiorLieberman Jul 8, 2025 •

edited

Loading

LiorLieberman Aug 25, 2025 •

edited

Loading