continuous delivery CD pipeline for releases

[monperrus]

Requirements

• a new script for normal releases (shell or python) (spec below)
• add CI job to handle changes in the release spec set up declarative releases for Spoon SpoonLabs/spoon-deploy#6

Normal release is everything which is not a beta

[monperrus]

FYI, CD of gumtree-spoon is also still Travis-based;

[slarse]

I've got this on next week's TODO list.

[slarse]

By next week I of course meant next month when Travis CI has already stopped working.

Looking at this right now. It's of course possible to just port the CD script to some other platform, but while we're at it, we may want to consider simplifying the CD process. The script is fairly complex and fragile. One example of extreme fragility is that it relies on the version number in pom.xml being on line 21. If I were to run the same formatting on the spoon-core pom file as I do on the parent pom file in #3994, then the version would be bumped down to line 22, breaking the CD script. That seems unreasonably fragile to me.

So there are two goals here: make the CD less fragile, and make it work on some other platform. We can deploy either from GitHub Actions or Jenkins. As this is a critical process, I think we should run it on Jenkins for complete control.

@monperrus I suggest the following changes.

Separate the beta releases and normal releases into different scripts/jobs

These are two fundamentally different concerns. We want to do a beta release every X days, while we want to do a normal release on specific commits (bump commits).

Permanently fix the Javadoc plugin problems (actually, it may already be fixed)

Currently, the CD script is injecting stuff into the pom to fix some crash on Travis. I don't see why we can't fix this permanently by just fixing the pom file. According to the linked issue, a workable solution is to just set the maven.compiler.source property to 8. This may clash with Spoon overriding the source version in the parent pom with the custom java.src.version property, but that's fixable.

However, as the source version is already set to 1.8 in the parent pom, this crash may not even be a problem anymore, and the fix in the CD script is just a leftover from before the source version was explicitly set. The docs for the Javadoc plugin say that it uses the source parameter of maven-compiler-plugin to resolve API links, and since it's set to 1.8, it should be all good.

Normal release changes

Currently, the normal release is where we have the severe fragility I mentioned above. We can get rid of that by running the normal release script on every commit, but exiting immediately if the commit in question is not a bump commit. Then, instead of searching for changes on a specific line, we just compare the results of mvn help:evaluate -Dexpression=project.version -q -DforceStdout on the current and previous commit. If they differ, we have a bump!

Instead of semi-manually fiddling around in the pom with xmlstarlet, I also think it would be much better to use the Maven release plugin.

Current priorities

As we don't plan to make a normal release in the near future, setting up a standalone beta release job is the top priority. I think we should deploy from Jenkins for complete control, but using GitHub Actions like Sorald does is actually simpler.

@monperrus Any thoughts before I get to work on this?

[monperrus]

We should update the Continuous Delivery script to create a new release on Github.

[slarse]

I've done that in Sorald's release script, although that's in Python. It's essentially just a single API call, so it's easy enough to do with curl.

[monperrus]

FYI, CD of spoon-maven-plugin is off because we're out of Travis gas again. Starts to be pressing to move CD to Actions or Jenkins.

[slarse]

Okidoki, I think I can spare some time during the weekend to pick up this thread again.

[slarse]

FYI, CD of spoon-maven-plugin is off because we're out of Travis gas again. Starts to be pressing to move CD to Actions or Jenkins.

I'm a bit confused, I can't find that spoon-maven-plugin has any continuous delivery? Did you mean just the CI running on Travis?

[monperrus]

you're right, there is no CD for spoon-maven-plugin.

[slarse]

But there is CI, and I suppose that's what we want to migrate there?

And gumtree-spoon has already been migrated over to GitHub Actions, so I don't really think there's all that much urgency. AFAIK, we have nothing crucial running on Travis anymore.