Code coverage server is unreliable

[JakeQZ]

CI and PR checks should not fail if Coveralls is offline. This has happened many times recently. It's unworkable as it is.

We need to find a way of disregarding any error from Coveralls, and just treating it as missing information instead.

[JakeQZ]

actions/runner#2347 has some relevant but rather too much discussion. There might a needle in a haystack in there.

[JakeQZ]

allow-failure flag may be possible - actions/runner#2347 (comment)

Edit:
Not it isn't. That was just another rejected suggestion.

[oliverklee]

At the moment, the code coverage is not as required in the branch protection settings, i.e., it should be possible to merge a PR even if the code coverage has that angry red dot (and despite the failure email). Does this work for you, @JakeQZ ?

[JakeQZ]

branch protection settings

Any PR can be merged with failing tests by clicking the "Override branch protection settings" link/button.

IIRC, the Coveralls failure meant the whole CI was 'red', and would need such override to merge. But my memory is not what it used to be, and these were PRs I submitted rather than reviewed. If you were able to merge despite the Coveralls failure, and without overriding branch protection, then I guess this is passable (though not ideal).

If OTOH you had to override branch protection, there may be dragons...

actions/runner#2347 was closed as 'Not Planned', and I did not find a solution in the comments. So there may be no other way for now.

PS. Coveralls seems to be back up and running fine and dandy now, so the situation is no longer (easily) reproducible.

[oliverklee]

Any PR can be merged with failing tests by clicking the "Override branch protection settings" link/button.

Yes, that's true.

If the code coverage job fails, it's possible to use the regular merge button, though (as the "override" button is only necessary if a required job fails).