KServe Path-Based Routing Implementation

[madmecodes]

✏️ Summary of Changes

Problem Statement
Input: User request to implement path-based routing for KServe inference services
Requirement: Enable custom URL paths like /serving/namespace/service/v1/models/... to route to KServe services

Original
Client → istio-ingressgateway → isvc-sklearn-predictor-ingress VS → Service

New Path-Based Flow
Client → istio-ingressgateway → kserve-path-gateway → kserve-path-routing VS → knative-local-gateway → Service

First Attempt: Direct VirtualService
FAILED APPROACH

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: test-path-routing
spec:
  hosts: ["*"]  # Problem: Too broad
  gateways: ["kubeflow/kubeflow-gateway"]  # Problem: Conflicts with existing

Issues Encountered:

1. VirtualService Conflicts: Existing KServe VirtualServices matched first
2. Authority Matching: isvc-sklearn-predictor-ingress used authority prefix matching that intercepted requests
3. Processing Order: Istio processed VirtualServices alphabetically, causing conflicts

Why New Host Creation?

1. Conflict Avoidance: Existing VirtualServices match specific hosts
2. Clean Separation: Path-based routing isolated from native KServe routing
3. Precedence Control: Dedicated Gateway ensures our rules process first
4. Maintainability: No interference with KServe's auto-generated routing

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: kserve-path-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      protocol: HTTP
    hosts:
    - "kserve-path.${NAMESPACE}.example.com"  # NEW HOST

Step 2: Create Isolated VirtualService

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: kserve-path-routing
spec:
  gateways:
  - kserve-path-gateway  # DEDICATED GATEWAY
  hosts:
  - "kserve-path.${NAMESPACE}.example.com"  # MATCHES GATEWAY
  http:
  - match:
    - uri:
        prefix: /serving/${NAMESPACE}/isvc-sklearn/
    rewrite:
      uri: /  # STRIP PATH PREFIX
    route:
    - destination:
        host: knative-local-gateway.istio-system.svc.cluster.local
      headers:
        request:
          set:
            Host: isvc-sklearn-predictor.${NAMESPACE}.svc.cluster.local

Output: {"predictions":[1]}

📦 Dependencies

#3098

✅ Contributor Checklist

• I have tested these changes with kustomize. See Installation Prerequisites.
• All commits are signed-off to satisfy the DCO check.
• I have considered adding my company to the adopters page to support Kubeflow and help the community, since I expect help from the community for my issue (see 1. and 2.).

---

You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

[juliusvonkohout]

Please no additional gateways. Gateway vreation is not allowed in user namespaces. just a second virtual service should be enough

[juliusvonkohout]

Please rebase to master

[juliusvonkohout]

Please no dedicated gateway.

[juliusvonkohout]

And gateways are also forbidden in user namespaces. It should work without that as shown on slack.

[madmecodes]

same script working locally in kind cluster, but failing in pipeline debugging...

[juliusvonkohout]

Thank you

/lgtm
/approve

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: juliusvonkohout

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [juliusvonkohout]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment