Skip to content

Antrea EgressIP does not work if wireGuard is enabled #6190

@adolfomaltez

Description

@adolfomaltez

Describe the bug

When an egressIP is created and applied to pods, it does not work correctly if wireGuard is enabled.
The egressIP works only for the pod that is on the node that takes the egressIP.
The rest of the pods on the nodes that do not have the egressIP lose connectivity to the outside of the cluster.
If wireGuard is disabled, egressIP works correctly for all pods.

To Reproduce
A kubernetes cluster is created with kind.

git clone https://github.com/antrea-io/antrea.git
git checkout release-1.15
docker pull projects.registry.vmware.com/antrea/antrea-ubuntu:v1.15.0
./ci/kind/kind-setup.sh --images projects.registry.vmware.com/antrea/antrea-ubuntu:v1.15.0 create cluster

antrea is installed
kubectl apply -f https://github.com/antrea-io/antrea/releases/download/v1.15.0/antrea.yml

egressIP is enabled (Egress: true)

kubectl edit cm antrea-config -n kube-system
kubectl rollout restart deployment/antrea-controller -n kube-system
kubectl rollout restart daemonset/antrea-agent -n kube-system

A deployment and egressIP are created.

kubectl create -f hello-world-egressIP.yaml

Test connection to an outside service (nginx on laptop).
All 3x pods show egressIP as their source IP.

wireGuard is enabled ( trafficEncryptionMode: "wireGuard" )

kubectl edit cm antrea-config -n kube-system
kubectl rollout restart deployment/antrea-controller -n kube-system
kubectl rollout restart daemonset/antrea-agent -n kube-system

Test connection to an outside service (nginx on laptop).
Only the pod on the node with the egressIP show egressIP as their source IP.
The other 2x pods fail to connect to the outside cluster (Operation timed out).

Expected
It is expected that the 3 pods of the deployment will use the egressIP as the source IP, to a service external to the cluster, even with wireGuard enable.

Actual behavior
Only the pod running on the node that takes the egressIP works correctly.
Pods on different nodes cannot access services outside the cluster.
If wireGuard is disabled, all pods work correctly with the egressIP.

Versions:

Please provide the following information:

  • Antrea version (Docker image tag): v1.15.0

  • Kubernetes version (use kubectl version). If your Kubernetes components have different versions, please provide the version for all of them.
    Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"archive", BuildDate:"2022-04-02T14:49:13Z", GoVersion:"go1.18", Compiler:"gc", Platform:"linux/amd64"}
    Server Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.3", GitCommit:"9e644106593f3f4aa98f8a84b23db5fa378900bd", GitTreeState:"clean", BuildDate:"2023-03-30T06:34:50Z", GoVersion:"go1.19.7", Compiler:"gc", Platform:"linux/amd64"}

  • Container runtime: which runtime are you using (e.g. containerd, cri-o, docker) and which version are you using?
    Kind: kind version 0.18.0, Docker version 20.10.24+dfsg1, build 297e128

  • Linux kernel version on the Kubernetes Nodes (uname -r).
    root@cluster-worker:/# uname -r
    6.1.0-18-amd64

Additional context
hello-world-egressIP.yaml.txt

Metadata

Metadata

Assignees

Labels

kind/bugCategorizes issue or PR as related to a bug.lifecycle/frozenIndicates that an issue or PR should not be auto-closed due to staleness.

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions