GitHub Action Commit Queue

[Trott]

My thinking is let's focus on making this a GitHub Action that can be run when Jenkins is green/yellow. Maybe triggered by a Collaborator comment? It can re-run some things as GitHub Actions to make sure tests still pass on the most popular platforms before adding metadata and merging.

Who wants to be involved? We should probably add and remove people from @nodejs/commit-queue as appropriate.

Refs: nodejs/commit-queue#1

[mmarchini]

I'm in (and I agree with the approach described). Maybe we should fork nodejs/node to work on it (since we'll have to test landing things)?

[richardlau]

I'm in (and I agree with the approach described). Maybe we should fork nodejs/node to work on it (since we'll have to test landing things)?

https://github.com/nodejs/node-auto-test ?

[mmarchini]

My suggestion is we start by making an action that lands the PR (without any checks at first), because there are restrictions for PRs coming from forks, and we'll need to work around those restrictions. I can take a stab at drafting something tomorrow or Thursday.

[nschonni]

Here is one that runs off of the PR being labeled https://github.com/pascalgn/automerge-action
There are some others in Marketplace that might be used as a baseline if they don't entirely fit https://github.com/marketplace?utf8=%E2%9C%93&type=actions&query=pull+request

[mmarchini]

@nschonni thanks, that's probably a good place to start. It'll probably not fit us though since we add metadata to every commit, and usually use node-core-utils to land pull requests, but we can base our implementation on that.

pascalgn/automerge-action suffers the fork issues I mentioned above (pascalgn/automerge-action#46). More specifically, for an action make any changes (merge, commit, comment, add labels, etc.) to a Pull Request, it makes a call to the GitHub API using a GITHUB_TOKEN provided to the action. Usually that token has read and write access, but if a Pull Request is coming from a fork, the token is read-only:

The permissions for the GITHUB_TOKEN in forked repositories is read-only.

(https://help.github.com/en/actions/reference/events-that-trigger-workflows#pull-request-events-for-forked-repositories-2)

Looking at the other events available, I don't see any we could use directly, but here are some ideas:

1. triggering externally, which can be used in two different ways:
   • Create a separate interface/ncu command to trigger commit queue on a Pull Request (only slightly better than the current workflow)
   • A comment/label action to make a call to an external API, which will make a call back to GitHub to trigger a repository_dispatch event, which will run appropriate checks and merge the PR if possible (this is the UX we're aiming for, but it sounds like a more complex solution). The external API would be responsible to check that the event was triggered by a Node.js core collaborator (if we can't validate that for sure, this approach is a no-no)
2. Use the scheduler event to iterate over Pull Requests with a specific label (let's say, commit-queue) and merge them
3. Have a separate branch from master where we land things, and when we land there, GitHub triggers an Action to check, add metadata and push to master, or to revert + reopen PR if checks fail

I think 2 and 3 would have less maintenance burden, and would be easier to ensure only collaborators can land things. 3 could cause a lot of confusion though, so maybe we should try something with the scheduler? Or does anyone have other suggestions?

[jasnell]

I'd be generally +1 on this but would definitely like to see it well tested in a separate repo (not core) and accepted before we enable it here.

My chief concern is around commit squashing. Most of the time squashing is straightforward, but there are often fixups that we need to do to fix formatting, subsystem in the title, typos, munging of comments from squashed commits, etc, that often really does need to be done manually.

[mmarchini]

I'd be generally +1 on this but would definitely like to see it well tested in a separate repo (not core) and accepted before we enable it here.

+1

My chief concern is around commit squashing.

We can start forcing only single-commit PRs only

Most of the time squashing is straightforward, but there are often fixups that we need to do to fix formatting, subsystem in the title, typos, munging of comments from squashed commits, etc, that often really does need to be done manually.

I don't think a commit queue is supposed to eliminate manual landing, but rather make it easier to land common cases. It would be the collaborator's discretion to decide if they want to merge via commit queue or manually.

[jasnell]

It would be the collaborator's discretion to decide if they want to merge via commit queue or manually.

Initially yes, but as I understand it, the whole idea behind the commit queue is to remove the collaborator's discretion from the mix as much as possible. I'm definitely not against it; but while at times I start to feel like the Old Man Who Hates Everything New, I'd like to see us take our time to get it right.

[mmarchini]

I think I like the scheduler idea, it fits well with our "wait until land" policy (for example, the second collaborator who approved a PR could add the commit-queue flag, and after the 48 hours passed GH Actions will land it).

[joyeecheung]

I think it's good to have unconditional squashing in the beginning - it seems understandable to me that if you want something more complicated like landing multiple commits with fixups, you'd need to count on a human being to do the chores instead of expecting too much out of automation. If we only ever support single-commit landing in the automation, we also encourage the culture of sending different PRs for different commits if possible - but as always we could leave room for human beings to land things as they wish, instead of making it impossible to land manually.

Regarding commit message for the squashed commit - how about taking the PR message? Then the workflow would be somewhat similar to how chromium's commit queue works - before you commit, you change the only CL description which is eventually used as commit message.

[joyeecheung]

Looking at the other events available, I don't see any we could use directly, but here are some ideas

I think 3 sounds safer - it also sounds oddly like the LKGR + master strategy that chromium does, where they have sheriffs to keep an eye on the breakage and revert ASAP without caring too much about how significant the PR is. If we could also adopt a similar culture (prioritize revert + reland over waiting for a fix) that would be a better choice IMO.

[mmarchini]

I've been playing with the schedule event and while I think it would work, it would have some extra complexity as well (especially because the context GitHub Actions give us on schedule is rather incomplete).

My biggest concern with 3 is:

3 could cause a lot of confusion though

To elaborate on that, if we have two branches (let's say master and commit-queue) and we land things on commit-queue, one of two things would happen:

1. When folks open a Pull Request it would still point to master by default, or
2. The front page of https://github.com/nodejs/node would point to commit-queue instead of master.

Also, automating Revert can get quite complex.

(edit: I just double-checked, there's no way to select different branches for the front page and for default PR target)

[joyeecheung]

When folks open a Pull Request it would still point to master by default, or

Why would this be an issue if the "intend to land" branch is indeed master? It still sends the message that this is not intended to land on e.g. a backport branch. Provided that we still leaves room for people to land commits manually, it still makes sense to point to the master branch and the temporary branch is more like an implementation detail of the action.

Also, automating Revert can get quite complex.

Can the action just posts something to IRC and ask a human being to do it ASAP?

[mmarchini]

Why would this be an issue if the "intend to land" branch is indeed master?

A collaborator (or the author) would still need to change the branch manually. It works, but it's not the most ergonomic experience.

Can the action just posts something to IRC and ask a human being to do it ASAP?

It can also comment on the Pull Request and reopen it I guess. In theory it could just push a Revert commit of itself to commit-queue, I'm just worried we would eventually get too out of sync with master (regardless of the Action doing it or we doing it manually). I also think rebasing and removing commits is not an option, because if there are commits not pushed to master and we rebase/force-push, only the last commit will execute the Action.

I don't think any of the three suggestions are perfect (far from it, as they all are workarounds), so I'm totally fine trying this one out.