Show progress when waiting

[evmar]

When we've been waiting for commands for more than a second with no output, print some "still waiting for xyz" text.

[sorbits]

Here’s what I have been using locally (for 6 years) sorbits@192fed9

[jhasse]

It would be better to delay re-printing status until we have waited a certain amount of time, which would also allow us to handle the case where more than one edge is stalling the build, but this require some refactoring, as waiting for tasks is presently done without the ability to specify a timeout.

This needs to be done, before we could merge it. See #1405 (comment).

[sorbits]

This needs to be done, before we could merge it. See #1405 (comment).

My patch only prints status when there is a single task running, so for a normal concurrent build, you only get extra status printed when you have everything waiting for a single task, not when the build is running along fine.

I just tested a build which had 365 edges (needing rebuild) and it printed status 367 times, so only 2 additional times, once was linking (which is slow with clang’s lto option) and the other was codesigning (which require network connection for timestamp server).

[jhasse]

Oh I didn't notice, sorry.

That does mean though, that the problem persists if there are two jobs stalling the build, right?

[sorbits]

That does mean though, that the problem persists if there are two jobs stalling the build, right?

Correct. The patch does not address that situation, I have found that builds either stall due to an issue with a single job, or an issue with all the jobs, my patch address the former, and incase of the latter, I think ninja’s behavior is already correct.

Personally I would want as few extra status lines as possible, as I often run ninja outside of a terminal, where lines are not overwritten, but I do want to have the last line reflect what is actually going on when the build is stalled, which I have defined as everything waiting for a single job.

But if you want to generalize it to multiple jobs, I would suggest the following (for all outputs, not just smart terminal):

If all of the below

1. There are less jobs running than capacity (-j)
2. All jobs running have been running for more than n seconds (n = 1 for smart terminal, n = 3 for non-smart terminal)
3. The last status line printed refers to a job that is no longer running

Then

Print status line showing the job most recently started

I pick the most recently started job because if we are dealing with a set of extraordinary slow jobs then that is the one we can assume will be running the longest, and thus the status printed will be valid for the longest amount of time.

[orgads]

I created a prototype that prints all the running jobs, a bit similar to bazel or docker compose output.

It still needs much polish (fix job numbers, improve alignment for long lines, keep last line[s], handle errors and more), but what do you think about the general idea?

If you like it, I'll try to complete it (as opt-in of course) and will push a PR.

ninja-live.mp4

[orenbenkiki]

For me it would be sufficient if ninja printed the list of active tasks every time this list changed, even if it is on a separate line. This seems to require less complex logic than some of the above proposals, but would result with more lines. I opened this proposal as #2249 which isn't quite a duplicate of the issue here, but has the same motivation.

Personally I'm fine with the extract lines #2249 would create, but if not, then the proposals here with a 1s timeout to eliminate most of these lines are also a solution.

BTW - when there's only one task running, it would be "very nice" if it were not buffered, even if it wasn't explicitly console.

This way it would always be clear what is running, and tracing output would be delayed the least amount of time.