Add VirtualService conflict troubleshooting for KServe path-based Routing

[madmecodes]

✏️ Summary of Changes

Documents the Istio VirtualService conflict issue that occurs when deploying KServe with path-based routing alongside KubeFlow. Adds troubleshooting guidance to resolve 404 errors caused by conflicts between KubeFlow's wildcard VirtualServices and KServe's specific-host VirtualServices. References upstream Istio issue istio/istio#57404 and provides Gateway configuration workaround.

📦 Dependencies

none

🐛 Related Issues

none

✅ Contributor Checklist

• I have tested these changes with kustomize. See Installation Prerequisites.
• All commits are signed-off to satisfy the DCO check.
• I have considered adding my company to the adopters page to support Kubeflow and help the community, since I expect help from the community for my issue (see 1. and 2.).

---

You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

[juliusvonkohout]

Can you list all the tests and explain what they are for? I mean we have already a tests for kserve path based access.

[juliusvonkohout]

or better jus tmerge
kserve_jwt_authentication_test.yaml
kserve_secure_test.yaml
kserve_test.yaml

Into a single file.

[juliusvonkohout]

You have to find out which ordering makes sense.

[juliusvonkohout]

Please also merge
kserve_jwt_authentication_test.yaml
kserve_secure_test.yaml
kserve_test.yaml

Into a single file.

[madmecodes]

/retest

[madmecodes]

/retest

[juliusvonkohout]

/lgtm
/approve
thank you

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: juliusvonkohout

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [juliusvonkohout]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hahahannes]

Hi @juliusvonkohout and @madmecodes,

I just saw this. Thanks a lot for covering it!

We noticed now that this workaround will lead to problems with cert-manager and the created Ingress for HTTP-01 acme challenges. The ingress will get picked up by the default istio-ingressgateway. The problem is that through the workaround, all routing rules for the istio-ingressgateway (with the host that was set in the kubeflow-gateway) seem to be ignored now. I am checking now how to handle this.

[hahahannes]

Just out of curiosity, were you able to reproduce it ? I was trying to get it fixed upstream at Istio, but seems like they were not able to reproduce.

[hahahannes]

Hi @juliusvonkohout and @madmecodes,

I just saw this. Thanks a lot for covering it!

We noticed now that this workaround will lead to problems with cert-manager and the created Ingress for HTTP-01 acme challenges. The ingress will get picked up by the default istio-ingressgateway. The problem is that through the workaround, all routing rules for the istio-ingressgateway (with the host that was set in the kubeflow-gateway) seem to be ignored now. I am checking now how to handle this.

Quick update, another workaround is to use a Kyverno policy to override the hosts field of the VirtualServices created by the KServe InferenceService to be "*". This way, the Ingress from cert-manager will work as well.

[juliusvonkohout]

Hi @juliusvonkohout and @madmecodes,
I just saw this. Thanks a lot for covering it!
We noticed now that this workaround will lead to problems with cert-manager and the created Ingress for HTTP-01 acme challenges. The ingress will get picked up by the default istio-ingressgateway. The problem is that through the workaround, all routing rules for the istio-ingressgateway (with the host that was set in the kubeflow-gateway) seem to be ignored now. I am checking now how to handle this.

Quick update, another workaround is to use a Kyverno policy to override the hosts field of the VirtualServices created by the KServe InferenceService to be "*". This way, the Ingress from cert-manager will work as well.

Hello, please raise a PR to fix or extend the documentation then.

[hahahannes]

Hi @juliusvonkohout, I created #3263 to follow up. Will open the MR soon