feat: add psutil comparison tests for cpu, mem and load - #2114
Merged
Conversation
Add internal/common/psutiltest, which runs Python psutil and compares its output against gopsutil as a reference implementation. Tests are skipped when psutil is unavailable. Install a pinned psutil in CI so the comparisons run on all runners.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a shared test helper for comparing gopsutil outputs against Python psutil (as a reference implementation), and adds initial cross-implementation comparison tests for CPU, memory, and load metrics. It also updates CI to best-effort install Python + a pinned psutil version so these comparison tests can run in GitHub Actions.
Changes:
- Add
internal/common/psutiltesthelper package to evaluate psutil expressions viapython -c, JSON-decode results, and provide comparison assertions/tolerances. - Add new
_Against_Psutilcomparison tests forcpu,mem, andload, including bracketed assertions for monotonic counters and eventual/tolerant assertions for fluctuating gauges. - Update the main test workflow to install Python and a pinned psutil version (best-effort) to enable these tests in CI.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
internal/common/psutiltest/psutiltest.go |
Adds python/psutil discovery + Eval[T] helper with strict-mode support via env vars. |
internal/common/psutiltest/assert.go |
Adds bracketed and tolerance-based assertion helpers for stable comparison tests. |
internal/common/psutiltest/psutiltest_test.go |
Adds unit tests validating Eval behavior and error propagation. |
cpu/cpu_psutil_test.go |
Adds psutil comparison tests for CPU counts and CPU times (including per-CPU). |
mem/mem_psutil_test.go |
Adds psutil comparison tests for virtual memory and (linux-only) swap behavior. |
load/load_psutil_test.go |
Adds psutil comparison tests for load averages and context switch counters. |
.github/workflows/test.yml |
Installs Python + pinned psutil (best-effort) before running go test to enable comparisons in CI. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
mergify Bot
added a commit
to ArcadeData/arcadedb
that referenced
this pull request
Aug 5, 2026
…p ci] Bumps the go-modules group in /e2e-go with 2 updates: [github.com/moby/go-archive](https://github.com/moby/go-archive) and [github.com/shirou/gopsutil/v4](https://github.com/shirou/gopsutil). Updates `github.com/moby/go-archive` from 0.2.1 to 0.3.2 Release notes *Sourced from [github.com/moby/go-archive's releases](https://github.com/moby/go-archive/releases).* > v0.3.2 > ------ > > What's Changed > -------------- > > Fix a regression introduced in v0.3.0 that caused archive extraction to fail when paths traversed absolute symlinks inside the destination root, such as `var/run -> /run`. Absolute symlink targets are now resolved relative to the extraction root while relative symlink escapes remain rejected. [moby/go-archive#93](https://redirect.github.com/moby/go-archive/pull/93) > > **Full Changelog**: <moby/go-archive@v0.3.1...v0.3.2> > > v0.3.1 > ------ > > Fixes > ----- > > This patch release fixes a regression introduced in v0.2.1 where archive extraction could fail when an archive omitted explicit entries for parent directories. For example, extracting `etc/dnf/` without a preceding `etc/` entry could return `mkdirat etc/dnf: no such file or directory`. > > This prevented affected images from being extracted. Archive extraction now creates implied parent directories for both file and directory entries. > > What's Changed > -------------- > > * archive: create implied parents for directory entries [moby/go-archive#92](https://redirect.github.com/moby/go-archive/pull/92) > * archive: Tarballer.Go: suppress io.ErrClosedPipe logs on close [moby/go-archive#94](https://redirect.github.com/moby/go-archive/pull/94) > > **Full Changelog**: <moby/go-archive@v0.3.0...v0.3.1> > > v0.3.0 > ------ > > Security > -------- > > This release fixes **CVE-2026-17106** / **[GHSA-hfg8-hc9c-6c3h](https://github.com/moby/go-archive/security/advisories/GHSA-hfg8-hc9c-6c3h)**, where a crafted tar archive could use links to cause extraction operations to create or overwrite files outside the intended destination directory. > > The issue affected `Unpack`, `UnpackLayer`, `Untar`, `UntarUncompressed`, and the `ApplyLayer` helpers. Users should upgrade and avoid extracting untrusted archives with earlier versions. > > What's Changed > -------------- > > * archive: harden tar extraction against path traversal [moby/go-archive#45](https://redirect.github.com/moby/go-archive/pull/45) > * archive: do not follow reparse points in chtimes [moby/go-archive#90](https://redirect.github.com/moby/go-archive/pull/90) > * archive: fix creation time updates on Windows [moby/go-archive#79](https://redirect.github.com/moby/go-archive/pull/79) > * archive: minor cleanups and godoc touch-up [moby/go-archive#87](https://redirect.github.com/moby/go-archive/pull/87) > * archive: RebaseArchiveEntries: fix archive path rebasing [moby/go-archive#43](https://redirect.github.com/moby/go-archive/pull/43) > > Test and CI changes > ------------------- > > * ci: enable dependabot for actions [moby/go-archive#81](https://redirect.github.com/moby/go-archive/pull/81) > * archive: make breakoutErr unwrap its cause [moby/go-archive#91](https://redirect.github.com/moby/go-archive/pull/91) > * archive: use filepath for filesystem paths in tests [moby/go-archive#80](https://redirect.github.com/moby/go-archive/pull/80) > * archive: use filepath for filesystem paths in tests [moby/go-archive#80](https://redirect.github.com/moby/go-archive/pull/80) > > **Full Changelog**: <moby/go-archive@v0.2.1...v0.3.0> Commits * [`9e6d2c7`](moby/go-archive@9e6d2c7) Merge pull request [#93](https://redirect.github.com/moby/go-archive/issues/93) from thaJeztah/fix\_absolute\_symlinks * [`4f6cd58`](moby/go-archive@4f6cd58) archive: resolve hardlinks through absolute symlinks * [`e564ecc`](moby/go-archive@e564ecc) archive: resolve absolute symlinks within extraction root * [`5bb8a45`](moby/go-archive@5bb8a45) Merge pull request [#94](https://redirect.github.com/moby/go-archive/issues/94) from thaJeztah/denoise * [`1bec7ec`](moby/go-archive@1bec7ec) archive: Tarballer.Go: suppress io.ErrClosedPipe logs on close * [`279fa6d`](moby/go-archive@279fa6d) Merge pull request [#92](https://redirect.github.com/moby/go-archive/issues/92) from thaJeztah/fix\_implied\_directories * [`517985a`](moby/go-archive@517985a) archive: create implied parents for directory entries * [`1c23372`](moby/go-archive@1c23372) Merge pull request [#43](https://redirect.github.com/moby/go-archive/issues/43) from thaJeztah/fix\_rebase\_from\_root * [`8829a25`](moby/go-archive@8829a25) RebaseArchiveEntries: fix archive path rebasing * [`c583b20`](moby/go-archive@c583b20) Merge pull request [#90](https://redirect.github.com/moby/go-archive/issues/90) from thaJeztah/chtimes\_nofollow * Additional commits viewable in [compare view](moby/go-archive@v0.2.1...v0.3.2) Updates `github.com/shirou/gopsutil/v4` from 4.26.6 to 4.26.7 Release notes *Sourced from [github.com/shirou/gopsutil/v4's releases](https://github.com/shirou/gopsutil/releases).* > v4.26.7 > ------- > > What's Changed > -------------- > > ### cpu > > * fix: harden parsers against malformed/truncated input by [`@shirou`](https://github.com/shirou) in [shirou/gopsutil#2109](https://redirect.github.com/shirou/gopsutil/pull/2109) > * [cpu][windows]: compute cpu-total times from integer ticks by [`@skartikey`](https://github.com/skartikey) in [shirou/gopsutil#2111](https://redirect.github.com/shirou/gopsutil/pull/2111) > * [darwin][process]: fix errno handling and library lifetime on darwin by [`@shirou`](https://github.com/shirou) in [shirou/gopsutil#2119](https://redirect.github.com/shirou/gopsutil/pull/2119) > * [cpu][windows]: compute total counters from individual stats to handle processor groups correctly by [`@srebhan`](https://github.com/srebhan) in [shirou/gopsutil#2125](https://redirect.github.com/shirou/gopsutil/pull/2125) > * [cpu][windows]: harden the cpu-total computation added in [#2125](https://redirect.github.com/shirou/gopsutil/issues/2125) by [`@shirou`](https://github.com/shirou) in [shirou/gopsutil#2128](https://redirect.github.com/shirou/gopsutil/pull/2128) > > ### net > > * fix(net): pad GetExtendedTcpTable buffer to prevent GC thrashing on Windows by [`@HarshalPatel1972`](https://github.com/HarshalPatel1972) in [shirou/gopsutil#2108](https://redirect.github.com/shirou/gopsutil/pull/2108) > > ### process > > * process: implement Darwin IOCounters via proc\_pid\_rusage by [`@DavRack`](https://github.com/DavRack) in [shirou/gopsutil#2117](https://redirect.github.com/shirou/gopsutil/pull/2117) > > ### other > > * feat: add psutil comparison tests for cpu, mem and load by [`@shirou`](https://github.com/shirou) in [shirou/gopsutil#2114](https://redirect.github.com/shirou/gopsutil/pull/2114) > > New Contributors > ---------------- > > * [`@DavRack`](https://github.com/DavRack) made their first contribution in [shirou/gopsutil#2117](https://redirect.github.com/shirou/gopsutil/pull/2117) > * [`@srebhan`](https://github.com/srebhan) made their first contribution in [shirou/gopsutil#2125](https://redirect.github.com/shirou/gopsutil/pull/2125) > > **Full Changelog**: <shirou/gopsutil@v4.26.6...v4.26.7> Commits * [`52a24c8`](shirou/gopsutil@52a24c8) Merge pull request [#2128](https://redirect.github.com/shirou/gopsutil/issues/2128) from shirou/feat/follow-up-2125 * [`268a953`](shirou/gopsutil@268a953) [cpu][windows]: harden the cpu-total computation added in [#2125](https://redirect.github.com/shirou/gopsutil/issues/2125) * [`1e34da6`](shirou/gopsutil@1e34da6) Merge pull request [#2125](https://redirect.github.com/shirou/gopsutil/issues/2125) from srebhan/fix\_cpu\_windows\_total * [`61f8802`](shirou/gopsutil@61f8802) Merge pull request [#2122](https://redirect.github.com/shirou/gopsutil/issues/2122) from shirou/dependabot/github\_actions/actions/checko... * [`7fb4dcf`](shirou/gopsutil@7fb4dcf) Merge pull request [#2123](https://redirect.github.com/shirou/gopsutil/issues/2123) from shirou/dependabot/github\_actions/actions/setup-... * [`ae7d91a`](shirou/gopsutil@ae7d91a) Merge pull request [#2119](https://redirect.github.com/shirou/gopsutil/issues/2119) from shirou/fix/darwin-errno-and-libcache * [`49052a1`](shirou/gopsutil@49052a1) [darwin][process]: use a PID above PID\_MAX in the not-running tests * [`991b238`](shirou/gopsutil@991b238) [darwin]: pass the remaining Go pointers as unsafe.Pointer on darwin * [`b9930e2`](shirou/gopsutil@b9930e2) Merge pull request [#2124](https://redirect.github.com/shirou/gopsutil/issues/2124) from shirou/dependabot/github\_actions/actions/labele... * [`38a01b4`](shirou/gopsutil@38a01b4) [cpu][windows]: compute total counters from individual stats to handle proces... * Additional commits viewable in [compare view](shirou/gopsutil@v4.26.6...v4.26.7) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore ` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore ` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore ` will remove the ignore condition of the specified dependency and ignore conditions
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Compare gopsutil results against Python psutil — the project gopsutil was originally modeled after, and which we continue to look up to as the reference implementation — to catch parser and semantics regressions with real values.
We have already been using this approach in the process package since #1488 (
TestName_Against_Python). This PR expands it toward all packages, but is just the foundation: a shared helper ininternal/common/psutiltestplus the first batch of comparisons, covering cpu, mem and load:Eval[T](t, expr)runspython -cwith psutil pre-imported and JSON-decodes the result (namedtuples → structs). Tests are skipped when python/psutil is not available; settingGOPSUTIL_PSUTIL_TESTturns skips into failures.require.EventuallyWithTwith a relative tolerance plus an absolute floor.All test names end in
_Against_Psutil(go test -run Against_Psutil ./...).Deliberately not compared (documented in code)
VirtualMemoryStat.Used/UsedPercent: gopsutil usesTotal - Available, psutil usestotal - free - buffers - cached.Percent()-style instantaneous values.Freediffers by design (psutil subtracts speculative pages).CI
test.ymlinstalls python and a pinned psutil (the version these comparisons were verified against) on all runners, withcontinue-on-errorso an install failure only skips the comparison tests. Once this proves stable, strict mode (GOPSUTIL_PSUTIL_TEST: require) can be enabled to make silent skips impossible.Follow-ups planned: host, disk, net, and extending the process comparisons.