Skip to content

Improve library stats generation and validation#1620

Merged
jormundur00 merged 4 commits into
masterfrom
jormundur00/gh-1032
Apr 6, 2026
Merged

Improve library stats generation and validation#1620
jormundur00 merged 4 commits into
masterfrom
jormundur00/gh-1032

Conversation

@jormundur00

@jormundur00 jormundur00 commented Apr 3, 2026

Copy link
Copy Markdown
Member

Summary

This PR expands the library stats support to cover all repo-backed libraries, tightens validation around repository metadata alignment, and makes the library stats validation workflow run when a new metadata version is added.

Changes

  • Adds library coverage stats for all repository-covered libraries to stats/stats.json.
  • Renames the library stats schema from library-stats-schema-v1.0.1.json to library-stats-schema-v1.0.2.json and updates references in the docs and build logic.
  • Extends the stats task/build logic to support the updated library stats model and repository-wide coverage data.
  • Updates the library-stats model so VersionStats.dynamicAccess uses a dedicated value wrapper that can serialize either a structured dynamic-access payload or N/A, mirroring how unavailable coverage metrics are already represented.
  • Keeps VersionStats backwards-friendly for producers by retaining the convenience constructor that still accepts DynamicAccessStats directly.
  • Updates LibraryStatsSchemaValidator to validate the repository stats file against the metadata layout more strictly.
  • Adds a remediation hint for missing metadata-version entries so validation failures tell contributors exactly which generateLibraryStats command to run.
  • Updates library-stats-validation.yml so the validation job also runs when files are added under metadata/<groupId>/<artifactId>/<version>/....
  • Refreshes CI and development documentation to describe the new schema version and library stats workflow behavior.
  • Updates the library stats unit tests to cover the revised schema/model behavior and the new validation guidance.

Testing

  • ../../gradlew test --tests org.graalvm.internal.tck.harness.tasks.ValidateLibraryStatsTaskTests from tests/tck-build-logic

Fixes: #1032

@jormundur00 jormundur00 requested a review from vjovanov as a code owner April 3, 2026 08:46
@jormundur00 jormundur00 self-assigned this Apr 3, 2026
@jormundur00 jormundur00 added the enhancement New feature or request label Apr 3, 2026
@jormundur00

Copy link
Copy Markdown
Member Author

To discuss: Should the coverage be 100% when -H:TrackDynamicAccess doesn't find any dynamic access calls, or should it be 0% (like it's currently set by default)?

CC: @vjovanov

@jormundur00 jormundur00 requested a review from kimeta April 3, 2026 09:02
@jormundur00 jormundur00 force-pushed the jormundur00/gh-1032 branch from 414137b to 0a5dc92 Compare April 3, 2026 13:37
@jormundur00 jormundur00 force-pushed the jormundur00/gh-1032 branch from 0a5dc92 to a3211a8 Compare April 6, 2026 09:16

@vjovanov vjovanov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings from review:

  1. Medium: fallback-to-N/A still parses dynamic-access artifacts and can fail anyway.

    • In AbstractLibraryStatsTask.computeVersionStats, we mark dynamic access unavailable when generateDynamicAccessReport fails, but we still call LibraryStatsSupport.buildVersionStats(...), which parses dynamic-access files.
    • If failed runs leave malformed/partial files behind, parsing can still throw, so generation does not reliably degrade to "N/A".
    • References:
      • tests/tck-build-logic/src/main/groovy/org/graalvm/internal/tck/harness/tasks/AbstractLibraryStatsTask.java:204-219
      • tests/tck-build-logic/src/main/java/org/graalvm/internal/tck/stats/LibraryStatsSupport.java:192,504+
  2. Low: CI docs are out of sync with workflow trigger behavior.

    • Workflow now triggers on metadata changes (metadata/*/*/*/**), but docs still state it only triggers on stats/stats.json or schema changes.
    • References:
      • .github/workflows/library-stats-validation.yml:20-22
      • docs/CI.md:38

Validation I ran locally:

  • ../../gradlew test --tests org.graalvm.internal.tck.harness.tasks.ValidateLibraryStatsTaskTests
  • ../../gradlew test --tests org.graalvm.internal.tck.stats.LibraryStatsSupportTests
  • ./gradlew validateLibraryStats
  • ../../gradlew test in tests/tck-build-logic has one unrelated pre-existing failure in ScaffoldTaskTests (reproduces on master).

@jormundur00

Copy link
Copy Markdown
Member Author

Addressed in 40ef815.

Changes made:

  • Updated library stats generation so a failed generateDynamicAccessReport no longer attempts to parse dynamic-access artifacts before falling back to dynamicAccess: "N/A". The fallback now builds coverage-only stats directly from the JaCoCo report.
  • Added a regression test covering the unavailable-dynamic-access path while preserving coverage metrics.
  • Updated the library stats validation workflow/docs so the workflow is scoped to PRs to master, and the CI documentation now matches the metadata mirror change detection behavior.

Verification:

  • ./gradlew :tck-build-logic:test --tests org.graalvm.internal.tck.stats.LibraryStatsSupportTests --tests org.graalvm.internal.tck.harness.tasks.ValidateLibraryStatsTaskTests

@jormundur00 jormundur00 merged commit 4d5e539 into master Apr 6, 2026
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Each library must contain the dynamic-access report

2 participants