You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
On CI, `test-fixtures-windows` does `GIX_TEST_IGNORE_ARCHIVES=1`
test runs on Windows, which have a number of known and expected
failures, as described in GitoxideLabs#1358. Since GitoxideLabs#1663, these are listed in
`etc/test-fixtures-windows-expected-failures-see-issue-1358.txt`
and `test-fixtures-windows` compares those expected failures to the
actual failures of a run. If there are any differences, it shows a
unified diff with full context and fails the run.
The default Git foreground color scheme has been used for the diff,
where any tests that are no longer failing are shown as `-` lines
in red, and any tests that are newly failing are shown as `+` lines
in green. These are the usual default `git diff` colors; we did not
pick those colors specifically to be meaningful to this scenario.
While the use of `-` and `+` lines and the specific unified diff
format chosen are intuitive and informative, this is less so for
the color scheme. That is for two reasons:
1. By default and unless another theme is chosen explicitly or
selected based on device settings, the GitHub web interface
uses a light theme. In this theme, as currently coded/styled, on
the pale background in GitHub Actions, foreground normal-styled
green text can be hard to distinguish visually from foreground
normal-style black text. So it is often not immediately clear
which test failures are new. (New test failures are the typical
way a `test-fixtures-windows` job fails, when it does.)
2. A test that is no longer failing would usually mean that a test
or implementation bug has been fixed, or that compatibility has
otherwise been improved. `test-fixtures-windows` fails on these
so that we attend to them, usually by dropping newly passing
tests from the list of expected failures. Likewise, a test that
is failing, especially a newly failing tests, usually if not
always means that something is not right, even if it is merely
the test itself that needs to be improved.
But showing things that have gotten better in red, and things
that have gotten worse in green, is unintuitive. This is partly
because these colors have the opposite meaning in numerous
contexts, including in some parts of the GitHub Actions
interface. It is more specifically because they have the
opposite meaning in the context of previous step output, where
`nextest` shows stderr in red (and specifically shows stderr on
tests that have failed), shows `FAIL` in red, and shows `PASS`
in green.
It may be possible to make red and green show up better. These
colors are clear in the `nextest` output. However, the options for
`diff` colors, at least when configuring them with `color.diff.*`,
are limited. Furthermore, that would only improve (1), not (2).
This bolds both `-` and `+` lines, shows `-` lines in magenta, and
shows `+` lines in blue. These colors seem to be visible on both
light and dark backgrounds, and they seem to be at least as
distinguishable from each other as are red and green, in general.
This change only applies to the specific diff that shows newly
passing and newly failing tests in `test-fixtures-windows`.
0 commit comments