Add init container to apply Antrea-specific sysctl configuration #7651

hongliangl · 2025-12-26T02:42:19Z

Introduce a new antrea-sysctl-init command to run as an init container
before Antrea components start. The command creates a dedicated sysctl.d
configuration file containing Antrea-specific sysctl settings required
by features that rely on policy routing (e.g. Egress).

The sysctl configuration applies only to the interfaces managed by
Antrea. A relatively high filename prefix is used so that the Antrea-
specific sysctl configuration is applied after most default distribution-
or administrator-provided sysctl configuration files, while still allowing
explicit higher-priority overrides when desired.

This logic is intentionally implemented as an init container rather than
as part of antrea-agent, to avoid modifying node-wide sysctl state during
normal agent operation.

build/charts/antrea/README.md

build/charts/antrea/values.yaml

cmd/antrea-sysctl-init/main.go

luolanzone · 2025-12-29T07:25:06Z

build/charts/antrea/values.yaml

+    # -- Resource requests and limits for the antrea-sysctl-init init container.
+    resources:
+      requests:
+        cpu: "100m"


This is too much for placing a file in the folder. Maybe 20m is enough if we'd like to define it. Or just remove this from the manifest.

I think removing this is just okay.

Maybe 100m is harmless. It seems that 100m is a convention.

Even it's harmless, it would make more sense to give it a reasonable value.

50m seems to be a good choice for future using.

luolanzone · 2025-12-30T02:14:30Z

cmd/antrea-sysctl-init/main.go

+
+func run(opts *options) error {
+	filePath := path.Join(opts.sysctlDir, opts.overwriteFile)
+	sysctlConfig := buildAntreaSysctlConfig(opts.hostGatewayName)


It seems systemd-sysctl needs to be restarted? https://github.com/cilium/cilium/pull/20072/files#diff-1cadee1ea10bb25d793baf555b85040a00ff0bc7f049a2542f0c2590ab4e7f0fR132-R136
Can you double check?

In some OS releases, restart is not required. For Ubuntu, we need a test.

The cases are:

On some OS distributions, adding a new file under /etc/sysctl.d is sufficient to ensure that sysctl settings are automatically re-applied to existing network interfaces when an interface is added or updated. In these environments, the newly added interface also receives theenforced settings.

On Ubuntu, sysctl configuration files under /etc/sysctl.d are not automatically re-applied when network interfaces are added or updated. New interfaces inherit values from net.ipv4.conf.default, while existing interfaces retain their current settings unless sysctl is explicitly reloaded.

Executing the command sysctl --system on all tested OS distributions explicitly applies sysctl configuration files under /etc/sysctl.d to all existing network interfaces, regardless of distribution-specific behavior.

luolanzone · 2025-12-30T02:20:42Z

build/charts/antrea/values.yaml

+    # The filename is chosen to sort last in lexicographic order within the sysctl
+    # configuration directory, so that the settings in this file are applied after other
+    # distribution- or administrator- provided sysctl configuration files.
+    overwriteFile: "99-zzzz-antrea.conf"


I am not seeing there is any need to let user to define this. Better to keep the name as a constant.

We'd better keep the parameter. A corner case is that there is an existing file named 99-zzzzzzzz-xxxx.conf. If that really happens, at least we have a chance to modify our file name so that it can be last in lexicographical order.

We are not modifying sysctl settings for interfaces that are not managed by Antrea, correct? If so, it should be fine that this configuration file is not applied last.

If there is another sysctl configuration overriding Antrea’s settings with 99-zzzzzzz.conf, that is presumably intentional. In that case, rather than continually increasing the filename priority by appending more “z”s, it would make more sense to first understand the purpose of the overriding configuration.

We are not modifying sysctl settings for interfaces that are not managed by Antrea, correct? If so, it should be fine that this configuration file is not applied last.

Will update the comment to state that a relatively high filename prefix is used to ensure the file will be applied after most default distribution- or administrator-provided sysctl configuration files.

The comment already explains it. My point is that it seems unnecessary to make it configurable.

Removed the parameter.

luolanzone · 2025-12-30T02:26:08Z

build/charts/antrea/values.yaml

+    # -- Resource requests and limits for the antrea-sysctl-init init container.
+    resources:
+      requests:
+        cpu: "100m"


Even it's harmless, it would make more sense to give it a reasonable value.

build/charts/antrea/README.md

cmd/antrea-sysctl-init/options.go

tnqn · 2025-12-30T07:34:15Z

build/charts/antrea/values.yaml

+    # The filename is chosen to sort last in lexicographic order within the sysctl
+    # configuration directory, so that the settings in this file are applied after other
+    # distribution- or administrator- provided sysctl configuration files.
+    overwriteFile: "99-zzzz-antrea.conf"


We are not modifying sysctl settings for interfaces that are not managed by Antrea, correct? If so, it should be fine that this configuration file is not applied last.

If there is another sysctl configuration overriding Antrea’s settings with 99-zzzzzzz.conf, that is presumably intentional. In that case, rather than continually increasing the filename priority by appending more “z”s, it would make more sense to first understand the purpose of the overriding configuration.

build/charts/antrea/values.yaml

build/charts/antrea/templates/agent/daemonset.yaml

tnqn · 2025-12-30T16:04:37Z

build/charts/antrea/values.yaml

  kubeletRootDir: "/var/lib/kubelet"
+  antreaSysctlInit:
+    # -- Enable the sysctl override. When enabled, the init container will do the process.
+    enable: true


Is it enabled by default intentionally? I thought it's only needed on some platforms?

It's indeed only needed on some OS releases, and is redundant for others. I thought it is generally harmless to enable it by default. An alternative is to keep it disabled by default and clearly document when and why it should be enabled. Any ideas where should we document that?

luolanzone · 2025-12-31T01:31:22Z

cc @hangyan @xliuxu for awareness of manifest changes and the new binary

hongliangl · 2025-12-31T03:19:14Z

/test-all

hongliangl · 2025-12-31T03:23:33Z

/scale-agent

hongliangl · 2025-12-31T05:05:37Z

/test-all
/scale-agent

Introduce a new antrea-sysctl-init command to run as an init container before Antrea components start. The command creates a dedicated sysctl.d configuration file containing Antrea-specific sysctl settings required by features that rely on policy routing (e.g. Egress). The sysctl configuration applies only to the interfaces managed by Antrea. A relatively high filename prefix is used so that the Antrea- specific sysctl configuration is applied after most default distribution- or administrator-provided sysctl configuration files, while still allowing explicit higher-priority overrides when desired. This logic is intentionally implemented as an init container rather than as part of antrea-agent, to avoid modifying node-wide sysctl state during normal agent operation. Signed-off-by: Hongliang Liu <[email protected]>

Signed-off-by: Hongliang Liu <[email protected]>

hongliangl · 2025-12-31T07:03:41Z

/test-all
/scale-agent

hongliangl · 2025-12-31T07:08:25Z

/test-all

hongliangl · 2025-12-31T07:12:42Z

/test-e2e

hongliangl · 2025-12-31T07:19:13Z

/test-e2e

hongliangl force-pushed the 20251224-add-sysctl-init-container branch from 7297d7b to b3c26f0 Compare December 26, 2025 04:48

luolanzone reviewed Dec 26, 2025

View reviewed changes

build/charts/antrea/README.md Outdated Show resolved Hide resolved

build/charts/antrea/values.yaml Show resolved Hide resolved

hongliangl force-pushed the 20251224-add-sysctl-init-container branch 4 times, most recently from 123ed07 to 2326ffd Compare December 29, 2025 03:16

hongliangl marked this pull request as ready for review December 29, 2025 03:29

hongliangl force-pushed the 20251224-add-sysctl-init-container branch 2 times, most recently from 6377498 to e89dd82 Compare December 29, 2025 06:27

luolanzone reviewed Dec 29, 2025

View reviewed changes

luolanzone requested review from antoninbas and tnqn December 29, 2025 07:26

hongliangl force-pushed the 20251224-add-sysctl-init-container branch from e89dd82 to adaadf7 Compare December 29, 2025 11:06

luolanzone reviewed Dec 30, 2025

View reviewed changes

tnqn reviewed Dec 30, 2025

View reviewed changes

hongliangl force-pushed the 20251224-add-sysctl-init-container branch 2 times, most recently from 857ea44 to 61ce01c Compare December 30, 2025 14:47

hongliangl changed the title ~~Add init container to create sysctl rp_filter overwrite config on Node~~ Add init container to apply Antrea-specific sysctl configuration Dec 30, 2025

hongliangl force-pushed the 20251224-add-sysctl-init-container branch 2 times, most recently from 0ff3bd6 to a64f469 Compare December 30, 2025 15:04

tnqn reviewed Dec 30, 2025

View reviewed changes

build/charts/antrea/templates/agent/daemonset.yaml Outdated Show resolved Hide resolved

tnqn reviewed Dec 30, 2025

View reviewed changes

hongliangl force-pushed the 20251224-add-sysctl-init-container branch from a64f469 to 33a45ce Compare December 30, 2025 23:11

hongliangl force-pushed the 20251224-add-sysctl-init-container branch from 33a45ce to 256e051 Compare December 31, 2025 03:18

hongliangl force-pushed the 20251224-add-sysctl-init-container branch from 256e051 to d1c4207 Compare December 31, 2025 05:05

Test only

7933864

Signed-off-by: Hongliang Liu <[email protected]>

hongliangl force-pushed the 20251224-add-sysctl-init-container branch from d1c4207 to 7933864 Compare December 31, 2025 06:14

Add init container to apply Antrea-specific sysctl configuration #7651

Are you sure you want to change the base?

Add init container to apply Antrea-specific sysctl configuration #7651

Conversation

hongliangl commented Dec 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hongliangl Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

luolanzone commented Dec 31, 2025

Uh oh!

hongliangl commented Dec 31, 2025

Uh oh!

hongliangl commented Dec 31, 2025

Uh oh!

hongliangl commented Dec 31, 2025

Uh oh!

hongliangl commented Dec 31, 2025

Uh oh!

hongliangl commented Dec 31, 2025

Uh oh!

hongliangl commented Dec 31, 2025

Uh oh!

hongliangl commented Dec 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

hongliangl commented Dec 26, 2025 •

edited

Loading

hongliangl Dec 29, 2025 •

edited

Loading