go.mod: bump minimum to Go 1.25 by cyphar · Pull Request #5169 · opencontainers/runc

cyphar · 2026-03-12T10:54:22Z

Go 1.24 is no longer supported.

Signed-off-by: Aleksa Sarai cyphar@cyphar.com

rata

LGTM, thanks!

lifubang · 2026-03-12T11:22:58Z

It's very strange, why there are some cgroup fds?

# (in test file tests/integration/create.bats, line 76)
#   `[ "$violation_found" -eq 0 ]' failed
# runc spec (status=0)
#
# runc create --console-socket /tmp/bats-run-INtO9Z/runc.nmjXkE/tty/sock test_busybox (status=0)
#
# runc state test_busybox (status=0)
# {
#   "ociVersion": "1.3.0",
#   "id": "test_busybox",
#   "pid": 132325,
#   "status": "created",
#   "bundle": "/tmp/bats-run-INtO9Z/runc.nmjXkE/bundle",
#   "rootfs": "/tmp/bats-run-INtO9Z/runc.nmjXkE/bundle/rootfs",
#   "created": "2026-03-12T11:12:37.821929398Z",
#   "owner": ""
# }
# Permitted: FD 0 -> '/dev/pts/0'
# Permitted: FD 1 -> '/dev/pts/0'
# Permitted: FD 2 -> '/dev/pts/0'
# Permitted: FD 3 -> '/tmp/bats-run-INtO9Z/runc.nmjXkE/state/test_busybox/exec.fifo'
# Permitted: FD 4 -> 'socket:[223840]'
# Permitted: FD 5 -> '/proc'
# Permitted: FD 7 -> '/runc'
# Permitted: FD 8 -> '/tmp/bats-run-INtO9Z/runc.nmjXkE/state/test_busybox/exec.fifo'
# Violation: FD 9 -> '/system.slice/runc-test_busybox.scope/cpu.cfs_quota_us'
# Violation: FD 10 -> '/system.slice/runc-test_busybox.scope/cpu.cfs_period_us'
# Permitted: FD 11 -> '/proc/1/task/1/fd'
# Permitted: FD 12 -> 'anon_inode:[eventpoll]'
# Permitted: FD 13 -> 'anon_inode:[eventfd]'
# --- teardown ---

cyphar · 2026-03-12T11:51:25Z

Yeah that's very strange, we don't have any go1.25 build tags (which I could see changing this behaviour) and this is only failing in this PR... We can investigate this more once we get rc1 out...

kolyshkin · 2026-03-12T22:26:43Z

It's very strange, why there are some cgroup fds?
...
Violation: FD 9 -> '/system.slice/runc-test_busybox.scope/cpu.cfs_quota_us'
Violation: FD 10 -> '/system.slice/runc-test_busybox.scope/cpu.cfs_period_us'
...

Two questions actually.

Why are they open now?
Why were they ever open? This is a default container configuration, there should be no CpuPeriod/CpuQuota settings (i.e. they are all 0 and thus these files should have never been written to.

kolyshkin · 2026-03-12T22:26:59Z

rebased

kolyshkin

This needs a changelog entry I guess.

We can also switch official builds and some CI to from Go 1.25 to 1.26:

GO_VERSION in Dockerfile for official builds;
GO_VERSION in .github/workflows/validate.yml for misc CI jobs;
GO_VER_PREFIX in .cirrus.yml for cirrus CI jobs;

but it can be done separately later.

kolyshkin · 2026-03-13T00:18:17Z

Violation: FD 9 -> '/system.slice/runc-test_busybox.scope/cpu.cfs_quota_us'
Violation: FD 10 -> '/system.slice/runc-test_busybox.scope/cpu.cfs_period_us'

Interestingly, this failure seems persistent (at least it failed twice here, before and after a rebase).

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>

lifubang · 2026-03-17T09:58:52Z

Still fail here, but I can't reproduce in my local(with almalinux-8):

[root@iZj6ci1wqars3l7c9qqqefZ runc]# make BUILDTAGS="seccomp urfave_cli_no_docs" localintegration RUNC_USE_SYSTEMD=yes
......
ok 56 runc exec [default CPU affinity should reset to cgroup cpuset]
ok 57 runc create[detect fd leak as comprehensively as possible]
ok 58 runc create
ok 59 runc create exec
......

[root@iZj6ci1wqars3l7c9qqqefZ busybox]# /root/go/src/github.com/opencontainers/runc/runc --systemd-cgroup create test
[root@iZj6ci1wqars3l7c9qqqefZ busybox]# /root/go/src/github.com/opencontainers/runc/runc --systemd-cgroup ps test
UID          PID    PPID  C STIME TTY          TIME CMD
root      137303       1  0 17:51 ?        00:00:00 /root/go/src/github.com/opencontainers/runc/runc init
[root@iZj6ci1wqars3l7c9qqqefZ busybox]# ls /proc/137303/fd/ -alh
total 0
dr-x------ 2 root root  0 Mar 17 17:51 .
dr-xr-xr-x 9 root root  0 Mar 17 17:51 ..
lrwx------ 1 root root 64 Mar 17 17:51 0 -> /dev/pts/1
lrwx------ 1 root root 64 Mar 17 17:51 1 -> /dev/pts/1
lrwx------ 1 root root 64 Mar 17 17:52 10 -> 'anon_inode:[eventfd]'
l--------- 1 root root 64 Mar 17 17:52 12 -> /
lrwx------ 1 root root 64 Mar 17 17:51 2 -> /dev/pts/1
lrwx------ 1 root root 64 Mar 17 17:52 3 -> 'socket:[385060]'
l--------- 1 root root 64 Mar 17 17:52 4 -> /proc
l--------- 1 root root 64 Mar 17 17:52 5 -> /proc/1/task/1/fd
l--------- 1 root root 64 Mar 17 17:52 6 -> /runc
l--------- 1 root root 64 Mar 17 17:52 7 -> /run/runc/test/exec.fifo
lrwx------ 1 root root 64 Mar 17 17:52 9 -> 'anon_inode:[eventpoll]'

[root@iZj6ci1wqars3l7c9qqqefZ busybox]# uname -a
Linux iZj6ci1wqars3l7c9qqqefZ 4.18.0-553.104.1.el8_10.x86_64 #1 SMP Mon Feb 9 02:31:11 EST 2026 x86_64 x86_64 x86_64 GNU/Linux
[root@iZj6ci1wqars3l7c9qqqefZ busybox]# systemctl --version
systemd 239 (239-82.el8_10.13)
+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy

kolyshkin · 2026-03-18T00:02:14Z

I was able to reproduce locally using lima and alma-linux-8

[root@lima-alma8 runc]# ssh -tt localhost "make -C /home/kir.linux/git/runc localintegration RUNC_USE_SYSTEMD=yes"
....
not ok 57 runc create[detect fd leak as comprehensively as possible]
# (in test file tests/integration/create.bats, line 76)
#   `[ "$violation_found" -eq 0 ]' failed
# runc spec (status=0)
#
# runc create --console-socket /tmp/bats-run-Jh05w8/runc.bfKEyr/tty/sock test_busybox (status=0)
#
# runc state test_busybox (status=0)
# {
#   "ociVersion": "1.3.0",
#   "id": "test_busybox",
#   "pid": 29815,
#   "status": "created",
#   "bundle": "/tmp/bats-run-Jh05w8/runc.bfKEyr/bundle",
#   "rootfs": "/tmp/bats-run-Jh05w8/runc.bfKEyr/bundle/rootfs",
#   "created": "2026-03-18T00:00:03.058105702Z",
#   "owner": ""
# }
# Permitted: FD 0 -> '/dev/pts/0'
# Permitted: FD 1 -> '/dev/pts/0'
# Permitted: FD 2 -> '/dev/pts/0'
# Permitted: FD 3 -> '/tmp/bats-run-Jh05w8/runc.bfKEyr/state/test_busybox/exec.fifo'
# Permitted: FD 4 -> 'socket:[124368]'
# Permitted: FD 5 -> '/proc'
# Permitted: FD 7 -> '/runc'
# Permitted: FD 8 -> '/tmp/bats-run-Jh05w8/runc.bfKEyr/state/test_busybox/exec.fifo'
# Violation: FD 9 -> '/system.slice/runc-test_busybox.scope/cpu.cfs_quota_us'
# Violation: FD 10 -> '/system.slice/runc-test_busybox.scope/cpu.cfs_period_us'
# Permitted: FD 11 -> '/proc/1/task/1/fd'
# Permitted: FD 12 -> 'anon_inode:[eventpoll]'
# Permitted: FD 13 -> 'anon_inode:[eventfd]'
....

kolyshkin · 2026-03-18T00:31:41Z

I suspect it is golang runtime that opens these files (as oc/cgroups/fs doesn't do it unless quota/period is set).

The code is in https://github.com/golang/go/blob/master/src/internal/runtime/cgroup/cgroup_linux.go and https://github.com/golang/go/blob/master/src/runtime/cgroup_linux.go, and enabled by the fact that we set go .25 in go.mod. It's added by https://go-review.googlesource.com/c/go/+/670497, see also https://go.dev/issue/73193

Looking into it.

kolyshkin · 2026-03-18T00:37:41Z

@lifubang what go version are you using? You need go 1.25.x which is being manually installed as part of CI job, here:

runc/.cirrus.yml

Lines 70 to 74 in 348c766

    
               # Install Go. 
        
               URL_PREFIX="https://go.dev/dl/" 
        
               # Find out the latest minor release URL. 
        
               filename=$(curl -fsSL "${URL_PREFIX}?mode=json&include=all" | jq -r --arg Ver "go$GO_VER_PREFIX" '. | map(select(.version | contains($Ver))) | first | .files[] | select(.os == "linux" and .arch == "amd64" and .kind == "archive") | .filename') 
        
               curl -fsSL "$URL_PREFIX$filename" | tar Cxz /usr/local

If you're using standard Go from Alma Linux 8, which is 1.24.x, this explains why you can't repro.

kolyshkin · 2026-03-18T01:03:37Z

We set GOMAXPROCS=1 for runc init and I would expect this should disable this mechanism but it's not so.

We can add GODEBUG=containermaxprocs=0,updatemaxprocs=0 to disable this entirely (and speed things up for runc init a bit. It might also make sense to set GOMAXPROCS=1 for runc init, not pass the current value from the environment (although I'm not entirely sure about it).

Here's the patch:

diff --git a/libcontainer/container_linux.go b/libcontainer/container_linux.go
index b4368547..2bc48f10 100644
--- a/libcontainer/container_linux.go
+++ b/libcontainer/container_linux.go
@@ -542,7 +542,8 @@ func (c *Container) newParentProcess(p *Process) (parentProcess, error) {
        if cmd.SysProcAttr == nil {
                cmd.SysProcAttr = &unix.SysProcAttr{}
        }
-       cmd.Env = append(cmd.Env, "GOMAXPROCS="+os.Getenv("GOMAXPROCS"))
+       cmd.Env = append(cmd.Env, "GOMAXPROCS=1")
+       cmd.Env = append(cmd.Env, "GODEBUG=containermaxprocs=0,updatemaxprocs=0")
        cmd.ExtraFiles = append(cmd.ExtraFiles, p.ExtraFiles...)
        if p.ConsoleSocket != nil {
                cmd.ExtraFiles = append(cmd.ExtraFiles, p.ConsoleSocket)

lifubang · 2026-03-18T01:04:44Z

what go version are you using?

go version go1.25.8 linux/amd64

kolyshkin · 2026-03-18T06:38:11Z

what go version are you using?

go version go1.25.8 linux/amd64

Hmm, interesting. I can repro it easily with that version.

kolyshkin · 2026-03-18T06:40:39Z

So, we have a few ways to move forward:

Add this patch;
Whitelist these two cgroup files that are known to be opened by Go runtime;
Try to fix it in Go runtime somehow (which they might not like, saying it is fixed by 1 already).

@cyphar what do you think?

lifubang · 2026-03-19T08:15:07Z

2. Whitelist these two cgroup files that are known to be opened by Go runtime;

I think it's safe to add these two cgroup files to the whitelist.

kolyshkin · 2026-03-26T05:09:22Z

Let's decide on this before 1.5.0-rc.2 release. The options we have are listed here: #5169 (comment)

rata · 2026-03-27T14:54:16Z

@kolyshkin IMHO option 2 seems better. The go variables are only temporary (the behavior will be on and the vars removed in the future) and it seems legit for the go runtime to do that.

rata · 2026-03-27T14:54:24Z

@kolyshkin IMHO option 2 seems better. The go variables are only temporary (the behavior will be on and the vars removed in the future) and it seems legit for the go runtime to exhibit this behavior with fds.

kolyshkin · 2026-03-27T19:29:05Z

My bad; hit the wrong button :)

thaJeztah · 2026-03-27T19:33:34Z

FWIW, GODEBUG can also be set through go.mod does that make a difference? https://github.com/docker/buildx/blob/ce058179ed17bf6ff6266814d8b0691e9215196d/go.mod#L244

kolyshkin · 2026-03-27T22:31:54Z

FWIW, GODEBUG can also be set through go.mod does that make a difference? https://github.com/docker/buildx/blob/ce058179ed17bf6ff6266814d8b0691e9215196d/go.mod#L244

Niiice! Maybe we can use it; so now we have four options:

Add this patch;
Add godebug containermaxprocs=0,updatemaxprocs=0 to go.mod;
Whitelist these two cgroup files that are known to be opened by Go runtime;
Try to fix it in Go runtime somehow (which they might not like, saying it is fixed by 1 or 2 already).

I'm not sure though if godebug options are to stay or not (but we can worry about it later), and how do they affect performance (I guess it's within the noise threshold).

thaJeztah · 2026-03-27T23:40:53Z

Yeah, I was mostly wondering if setting it in go.mod would put it earlier in the chain, so .. perhaps apply it at the earliest moment?

I have not looked at the problem at all otherwise, but it caught my eye 🫶

cyphar added this to the 1.5.0 milestone Mar 12, 2026

cyphar added the backport/1.5-todo A PR in main branch which needs to be backported to release-1.5 label Mar 12, 2026

cyphar mentioned this pull request Mar 12, 2026

VERSION: release v1.5.0-rc.1 #5154

Merged

rata approved these changes Mar 12, 2026

View reviewed changes

cyphar force-pushed the bump-go-1.25 branch from 5fbeb2c to 6ad9eba Compare March 12, 2026 11:06

kolyshkin force-pushed the bump-go-1.25 branch from 6ad9eba to c66b6d2 Compare March 12, 2026 22:26

kolyshkin requested changes Mar 13, 2026

View reviewed changes

cyphar added 2 commits March 17, 2026 17:40

go.mod: bump minimum to Go 1.25

8a58fb4

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>

go fix: use (*sync.WaitGroup).Go

77285fa

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>

lifubang force-pushed the bump-go-1.25 branch from c66b6d2 to 77285fa Compare March 17, 2026 09:40

kolyshkin modified the milestones: 1.5.0, 1.5.0-rc.2 Mar 26, 2026

kolyshkin closed this Mar 27, 2026

kolyshkin reopened this Mar 27, 2026

Conversation

cyphar commented Mar 12, 2026

Uh oh!

rata left a comment

Choose a reason for hiding this comment

Uh oh!

lifubang commented Mar 12, 2026

Uh oh!

cyphar commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kolyshkin commented Mar 12, 2026

Uh oh!

kolyshkin commented Mar 12, 2026

Uh oh!

kolyshkin left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kolyshkin commented Mar 13, 2026

Uh oh!

lifubang commented Mar 17, 2026

Uh oh!

kolyshkin commented Mar 18, 2026

Uh oh!

kolyshkin commented Mar 18, 2026

Uh oh!

kolyshkin commented Mar 18, 2026

Uh oh!

kolyshkin commented Mar 18, 2026

Uh oh!

lifubang commented Mar 18, 2026

Uh oh!

kolyshkin commented Mar 18, 2026

Uh oh!

kolyshkin commented Mar 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lifubang commented Mar 19, 2026

Uh oh!

kolyshkin commented Mar 26, 2026

Uh oh!

rata commented Mar 27, 2026

Uh oh!

rata commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kolyshkin commented Mar 27, 2026

Uh oh!

thaJeztah commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kolyshkin commented Mar 27, 2026

Uh oh!

thaJeztah commented Mar 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

cyphar commented Mar 12, 2026 •

edited

Loading

kolyshkin left a comment •

edited

Loading

kolyshkin commented Mar 18, 2026 •

edited

Loading

rata commented Mar 27, 2026 •

edited

Loading

thaJeztah commented Mar 27, 2026 •

edited

Loading