Skip to content

Retrying renamedTo operation up to 5 times because it randomly fails …#11216

Open
a-zitzewitz wants to merge 5 commits intojenkinsci:masterfrom
a-zitzewitz:master
Open

Retrying renamedTo operation up to 5 times because it randomly fails …#11216
a-zitzewitz wants to merge 5 commits intojenkinsci:masterfrom
a-zitzewitz:master

Conversation

@a-zitzewitz
Copy link

@a-zitzewitz a-zitzewitz commented Oct 20, 2025

…on slow file systems.

See JENKINS-76192.

While trying to make all test pass for the sonargraph-integration-plugin I found that the tests will often (not always) fail when installing a tool on a windows node. It is not always the same test that fails, but always the same stack trace:

java.io.IOException: Failed to rename Z:\agent\workspace\graph-integration-plugin_develop\target\Tools\SonargraphBuild\15.5.0\SonargraphBuild-15.5.0.159_2025-10-14 to Z:\agent\workspace\graph-integration-plugin_develop\target\Tools\SonargraphBuild\15.5.0\SonargraphBuild-15.5.0.159_2025-10-14.__rename
 at hudson.FilePath$MoveAllChildrenTo.invoke(FilePath.java:2610)
 at hudson.FilePath$MoveAllChildrenTo.invoke(FilePath.java:2595)
 at hudson.FilePath.act(FilePath.java:1214)
 at hudson.FilePath.act(FilePath.java:1197)
 at hudson.FilePath.moveAllChildrenTo(FilePath.java:2592)
 at hudson.tools.DownloadFromUrlInstaller.performInstallation(DownloadFromUrlInstaller.java:81)
 at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:67)
 at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:109)
 at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:221)
 at com.hello2morrow.sonargraph.integration.jenkins.tool.SonargraphBuild.forNode(SonargraphBuild.java:60)
 at com.hello2morrow.sonargraph.integration.jenkins.controller.SonargraphReportBuilder.callSonargraphBuild(SonargraphReportBuilder.java:442)
 at com.hello2morrow.sonargraph.integration.jenkins.controller.SonargraphReportBuilder.perform(SonargraphReportBuilder.java:331)

That implies that it is not a bug in the plugin, but in the tool installation operation. It seems that freshly created files are not visible immediately for other operations like rename. Therefore the proposed changed will retry the rename operation 5 times before giving up and throwing an IOException. That should give the file system enough time to catch up.

Testing done

All tests pass, change is covered by existing tests

Proposed changelog entries

  • Made tool installations more tolerant for slow file systems.

Proposed changelog category

/remove-label major-bug
/label bug

Proposed upgrade guidelines

N/A

Submitter checklist

  • The Jira issue, if it exists, is well-described.
  • The changelog entries and upgrade guidelines are appropriate for the audience affected by the change (users or developers, depending on the change) and are in the imperative mood (see examples). Fill in the Proposed upgrade guidelines section only if there are breaking changes or changes that may require extra steps from users during upgrade.
  • There is automated testing or an explanation as to why this change has no tests.
  • New public classes, fields, and methods are annotated with @Restricted or have @since TODO Javadocs, as appropriate.
  • New deprecations are annotated with @Deprecated(since = "TODO") or @Deprecated(forRemoval = true, since = "TODO"), if applicable.
  • New or substantially changed JavaScript is not defined inline and does not call eval to ease future introduction of Content Security Policy (CSP) directives (see documentation).
  • For dependency updates, there are links to external changelogs and, if possible, full differentials.
  • For new APIs and extension points, there is a link to at least one consumer.

Desired reviewers

@strangelookingnerd
@jenkinsci/core-pr-reviewer

Before the changes are marked as ready-for-merge:

Maintainer checklist

  • There are at least two (2) approvals for the pull request and no outstanding requests for change.
  • Conversations in the pull request are over, or it is explicit that a reviewer is not blocking the change.
  • Changelog entries in the pull request title and/or Proposed changelog entries are accurate, human-readable, and in the imperative mood.
  • Proper changelog labels are set so that the changelog can be generated automatically.
  • If the change needs additional upgrade steps from users, the upgrade-guide-needed label is set and there is a Proposed upgrade guidelines section in the pull request title (see example).
  • If it would make sense to backport the change to LTS, a Jira issue must exist, be a Bug or Improvement, and be labeled as lts-candidate to be considered (see query).

@welcome
Copy link

welcome bot commented Oct 20, 2025

Yay, your first pull request towards Jenkins core was created successfully! Thank you so much!

A contributor will provide feedback soon. Meanwhile, you can join the chats and community forums to connect with other Jenkins users, developers, and maintainers.

@comment-ops-bot comment-ops-bot bot added the major-bug For changelog: Major bug. Will be highlighted on the top of the changelog label Oct 20, 2025
@a-zitzewitz
Copy link
Author

The failing tests are not related to my changes. They only occurred after merging the main branch.

@comment-ops-bot comment-ops-bot bot added bug For changelog: Minor bug. Will be listed after features and removed major-bug For changelog: Major bug. Will be highlighted on the top of the changelog labels Oct 23, 2025
Copy link
Member

@timja timja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine, unlikely to happen in most cases so shouldn't get invoked except in edge cases.

@timja timja requested a review from a team October 23, 2025 16:13
@daniel-beck
Copy link
Member

Could you file an issue in https://github.com/jenkins-infra/helpdesk/ about this? Even if there's no straightforward infra-side fix, it may help to record this problem. This fix would take a while to arrive at your plugin unless you limit who gets to install it by increasing the core dependency to the next weekly release.

@daniel-beck
Copy link
Member

Did you confirm this resolves the issue for you? You can have a plugin PR depend on this core PR by specifying what was uploaded in https://github.com/jenkinsci/jenkins/runs/53292020888 as e.g. jenkins.version.

@a-zitzewitz
Copy link
Author

@daniel-beck Infrastructure issue created: jenkins-infra/helpdesk#4854

@a-zitzewitz
Copy link
Author

@daniel-beck The problem is that the tests are only failing on your build infrastructure. Maybe an issue with file system configuration? I don't know. But these issues are blocking me from creating a new release of the plugin.

@timja
Copy link
Member

timja commented Oct 30, 2025

@daniel-beck The problem is that the tests are only failing on your build infrastructure. Maybe an issue with file system configuration? I don't know. But these issues are blocking me from creating a new release of the plugin.

but in the plugin change its jenkins.version to be the incremental build from this PR, https://github.com/jenkinsci/jenkins/pull/11216/checks?check_run_id=53292020888

@a-zitzewitz
Copy link
Author

@timja Tried your proposal to test it with the Jenkins version created by the PR. It fails, because this version does not include the chmod fix of @strangelookingnerd . So I don't even get to the Windows build, because the Linux build fails due t the chmod issue.

@timja
Copy link
Member

timja commented Oct 30, 2025

Easiest way is to create a separate test PR in draft that includes the other fix.

See this comment for how to have a fast build for just getting the incremental version:
#11071 (comment)

@a-zitzewitz
Copy link
Author

Can we get this merged? Otherwise my tests will not succeed on the Jenkins CI server and I will never be able to update my plugin. For the tests to work I need @strangelookingnerd changes for the chmod fix and this PR integrated. I cannot even test this in isolation, because the version generated by this PR does not include @strangelookingnerd 's changes. So it fails die to chmod and not rename.

@timja
Copy link
Member

timja commented Nov 4, 2025

@strangelookingnerd 's change is merged so you should be able to check that this works.

@a-zitzewitz
Copy link
Author

@strangelookingnerd 's change is merged so you should be able to check that this works.

Sorry for the stupid question, but where do I find the Jenkins version I need to use for the test?

@timja
Copy link
Member

timja commented Nov 4, 2025

Once the build passes there will be a check in here called Incrementals which will have the version:
image

@timja
Copy link
Member

timja commented Nov 5, 2025

Version is 2.536-rc37676.5eb_e6c397d4f

@a-zitzewitz
Copy link
Author

@timja I can confirm that the rename problem is now fixed.

@daniel-beck
Copy link
Member

daniel-beck commented Nov 5, 2025

Noting that jenkinsci/jenkins-infra-test-plugin#195 / jenkinsci/sonargraph-integration-plugin#36 did not fail to download/extract in standalone tests, so it's not quite as easy as "Jenkins infra bad". There may be something wrong with the tests, or the plugins used in the tests. This PR and the other may just hide a real problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug For changelog: Minor bug. Will be listed after features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants