TODO list to make this happen

[joyeecheung]

Off the top of my head:

• Something that can land the PR automatically: git-node land
• A protocol for squashing commits
  • Option 1: Use the --autosquash feature of git and ask everybody to use --fixup or --squash when they create commits that should be squashed
  • Option 2: Just squash everything in the same PR. AFAIK the commit queue of V8 does this. I like it because it's simple and we should enforce that commits that can be submitted as separate PRs must be, then it'll be easier to backport PRs since we no longer mix semver-major changes in refactor/clean-ups. And this makes it easier to review PRs, revert commits, etc.
• Master/worker server applications to run the landing tool -> the queue. I think it's pretty obvious that we will implement them in Node.js?
• A bot or a Jenkins CI job for triggering the landing process
• Modification to git-node to store more metadata tracking PRs and queued jobs

For reference:

https://dev.chromium.org/developers/testing/commit-queue
https://dev.chromium.org/developers/testing/commit-queue/design

Added:

• Estimate the workload by looking into the velocity of the project
• Figure out how to isolate the master/worker and orchestrate and where they should be deployed

[maclover7]

Master/worker server applications to run the landing tool -> the queue. I think it's pretty obvious that we will implement them in Node.js?

We might be able to utilize the github-bot's machine for this, I believe it still has a bunch of capacity still, so we won't need new infrastructure donations

[joyeecheung]

@maclover7 I'd be more comfortable if we use some sort of container on the machine and have something to do the orchestration. I am not sure how powerful the machine is so it may just be just a bunch of overhead to use orchestration, we should also try estimating the workload based on the velocity of the project.

[bnoordhuis]

Option 2: Just squash everything in the same PR.

I'd be okay with this.

[benjamingr]

Option 2 works, the only issue is that PRs by several authors would not count towards contribution of other authors.

[tniessen]

I already hinted to @Trott and @joyeecheung that I am working on a prototype.

Design

The current design works like this:

1. The user submits a task description to the CQ server:
   • Which PR to land.
   • Which tests to perform before landing (e.g. lite CI).
   • How to map PR commits to commits on the master branch.
2. The CQ performs pre-defined checks to make sure that the PR can be landed.
3. The CQ adds a new task with the given task description to the task queue.

The task queue is processed as a regular queue, so in FIFO mode. For each task:

1. Produce the commits on a staging branch according to the commit mapping defined in the task description. (These are the final commits and will not be altered after this step!) This is a mixture of cherry-picking, squashing and editing of commit messages.
2. The CQ then runs user-defined tests according to the task description.
3. If all tests pass, update the master branch to the last commit produced by this task and push it to the upstream remote.

If one or more tests failed, the staging branch is reset to the "last known good revision", so the first commit that was not created as part of the task.

On commit mapping

In the current prototype, the commit mapping describes

• an array of output commits (title, description, metadata such as Refs:)
• a mapping from PR commits to output commits

The final commits are generated by iterating over the output commits and for each output commit, cherry-pick the input commits that are mapped to the respective output commit in their original order. Squashing and commit-message editing are performed at the same time, see implementation notes.

There should be reasonable defaults for the commit mapping property, e.g. "squash all" or "keep all".

This is the structure I am using to describe tasks:

  cq.post({
    pr: {
      repo: remoteRepo,
      number: 21075
    },
    tests: ['ci-lite'],
    commits: {
      mapping: [
        {
          from: 'b81729eb011900527b4c6121f4524a7d76dc1378',
          to: 0 // Index of output commit
        },
      ],
      commits: [
        {
          title: 'test: make handling of noWarnCode stricter',
          body: 'This change requires all expected warnings to be specified along with\n' +
            'their respective code and will raise an error if the code does not\n' +
            'match. This also kind of fixes the behavior when the expected warning\n' +
            'code was noWarnCode and there is an actual warning code.',
          metadata: [] // Refs, Fixes, etc.
        }
      ]
    }
  });

Implementation notes

This design allows some cool things:

• Multiple tasks can be processed in parallel: The staging branch allows to push arbitrarily many commits to it from different PRs. As long as all tests for all PRs work, everything is great and the parallelism speeds up the process by a lot. Note that all commits on the staging branch are final, so the CQ won't introduce any problems when pushing them to master. If a test fails, the staging branch is reset to the last commit before that task, but not further, so previously submitted tasks can still finish. The failing task is removed from the queue and subsequent tasks are restarted. This minimizes the overhead of failures:

  Example:
  Tasks A, B, C are pushed to the queue.
  
  Task A is started. Commits 1, 2 are produced.
  Task queue: (A*, B, C)
  Staging branch: (→ master, 1, 2)
  
  Task B is started. Commits 3, 4 are produced:
  Task queue: (A*, B*, C)
  Staging branch: (→ master, 1, 2, 3, 4)
  
  Task C is started. Commit 5 is produced:
  Task queue: (A*, B*, C*)
  Staging branch: (→ master, 1, 2, 3, 4, 5)
  
  Note that the last known good revision differs between tasks.
  For task A, it is → master.
  For task B, it is commit 2.
  For task C, it is commit 4.
  
  Assume a test for task B failed. B is removed from the queue and the staging branch
  is reset to the last known good revision:
  Task queue: (A*, C)
  Staging branch: (→ master, 1, 2)
  
  Task C is restarted and produces commit 6:
  Task queue: (A*, C*)
  Staging branch: (→ master, 1, 2, 6)
  
  Task A finishes successfully and commits 1, 2 are pushed to master:
  Task queue: (C*)
  Staging branch: (→ master, 6)
  
  Task C finishes successfully and commit 6 is pushed to master:
  Task queue: Empty
  Staging branch: (→ master)
  

• All Git operations are performed natively, without having to fork into git processes. The entire configuration can be done from within our software and we don't even need a working copy, a bare Git repository is enough. Using native Git commands also allows more efficient operations, such as changing the commit message and metadata while cherry-picking in a single operation (which - by the way - isn't even possible in bare Git repositories by default). I am in touch with one of the authors of nodegit and we are trying to fix some known problems to make this work as smoothly as possible.

I am trying to keep this design as modular as possible, e.g. checks (before starting the task) and tests (after starting the task) should be customizable. The same applies to processing metadata and formatting the commit message. It will definitely take me some time to even have a presentable prototype, but I will try.

[tniessen]

Also, I think it would be cool to have both a web interface and a CLI.

[joyeecheung]

Option 2 works, the only issue is that PRs by several authors would not count towards contribution of other authors.

@benjamingr If the commits can be submitted in separate PRs, then they should. If they can't, then since we require each commit must be able to pass the test, they have to be squashed anyway. Also we can use the Co-Authored-by field for attribution.

But personally I prefer to force everybody split commits and file separate PRs if possible, even if I sometimes don't want to do that myself when I am feeling lazy. It helps the backporters to minimize the changes between master and release branches as well, especially when it comes to refactoring/cleanup PRs.

[maclover7]

Just noting for posterity that there is prior work from the build WG on this issue: nodejs/build#705

[refack]

Just noting for posterity that there is prior work from the build WG on this issue: nodejs/build#705

And that was not the first attempt to...

I want to point out that information fragmentation is a huge problem with the org. I'm not sure why we need a new repo for this task. IMHO it only exacerbates the problem.

I move to close this repo and return the conversation to the build repo where it will at the least stay traceable if this task concludes or peters out.

[refack]

P.S. There is substantial work already done in the ghprb-plugin.
IMHO building on that will increase the chances that this effort will conclude, and not peter out like it had before.

[tniessen]

We should probably come to an agreement before I or anyone else puts a lot of effort into this.

[joyeecheung]

@refack I think with the current status of CI building the queue on top of successful CI runs would not be very viable for now? That was also what blocked the original issue.

I am fine with moving this back to build or continuing the discussion here, but IMO using a separate repo would reduce the noise when we are still figuring out the protocol e.g. regarding rebasing.

@maclover7 ‘s Jenkins job mentioned in that issue seems to be a good alternative as well, but we may need to be careful with conflicts. Do you still have that around?

[joyeecheung]

@refack @maclover7 @tniessen Or, if you prefer, we can put this back on the build meeting agenda and look at the prior efforts and the current status of things before moving forward?

[joyeecheung]

..or, since the agenda of the build meeting this week seems pretty packed, we can set up a doodle for a separate meeting, just do an overview of existing efforts, summarize the viable solutions so far, and make a concrete plan with assignees to make it happen?