-
Notifications
You must be signed in to change notification settings - Fork 433
Description
Describe the bug
When an egressIP is created and applied to pods, it does not work correctly if wireGuard is enabled.
The egressIP works only for the pod that is on the node that takes the egressIP.
The rest of the pods on the nodes that do not have the egressIP lose connectivity to the outside of the cluster.
If wireGuard is disabled, egressIP works correctly for all pods.
To Reproduce
A kubernetes cluster is created with kind.
git clone https://github.com/antrea-io/antrea.git
git checkout release-1.15
docker pull projects.registry.vmware.com/antrea/antrea-ubuntu:v1.15.0
./ci/kind/kind-setup.sh --images projects.registry.vmware.com/antrea/antrea-ubuntu:v1.15.0 create cluster
antrea is installed
kubectl apply -f https://github.com/antrea-io/antrea/releases/download/v1.15.0/antrea.yml
egressIP is enabled (Egress: true)
kubectl edit cm antrea-config -n kube-system
kubectl rollout restart deployment/antrea-controller -n kube-system
kubectl rollout restart daemonset/antrea-agent -n kube-system
A deployment and egressIP are created.
kubectl create -f hello-world-egressIP.yaml
Test connection to an outside service (nginx on laptop).
All 3x pods show egressIP as their source IP.
wireGuard is enabled ( trafficEncryptionMode: "wireGuard" )
kubectl edit cm antrea-config -n kube-system
kubectl rollout restart deployment/antrea-controller -n kube-system
kubectl rollout restart daemonset/antrea-agent -n kube-system
Test connection to an outside service (nginx on laptop).
Only the pod on the node with the egressIP show egressIP as their source IP.
The other 2x pods fail to connect to the outside cluster (Operation timed out).
Expected
It is expected that the 3 pods of the deployment will use the egressIP as the source IP, to a service external to the cluster, even with wireGuard enable.
Actual behavior
Only the pod running on the node that takes the egressIP works correctly.
Pods on different nodes cannot access services outside the cluster.
If wireGuard is disabled, all pods work correctly with the egressIP.
Versions:
Please provide the following information:
-
Antrea version (Docker image tag): v1.15.0
-
Kubernetes version (use
kubectl version). If your Kubernetes components have different versions, please provide the version for all of them.
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"archive", BuildDate:"2022-04-02T14:49:13Z", GoVersion:"go1.18", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.3", GitCommit:"9e644106593f3f4aa98f8a84b23db5fa378900bd", GitTreeState:"clean", BuildDate:"2023-03-30T06:34:50Z", GoVersion:"go1.19.7", Compiler:"gc", Platform:"linux/amd64"} -
Container runtime: which runtime are you using (e.g. containerd, cri-o, docker) and which version are you using?
Kind: kind version 0.18.0, Docker version 20.10.24+dfsg1, build 297e128 -
Linux kernel version on the Kubernetes Nodes (
uname -r).
root@cluster-worker:/# uname -r
6.1.0-18-amd64
Additional context
hello-world-egressIP.yaml.txt