Dumping behaves differently compared to link checking

[thomas-zahner]

According to documentation & help message:

--dump
    Don't perform any link checking. Instead, dump all the links extracted from inputs that would be checked

--dump-inputs
    Don't perform any link extraction and checking. Instead, dump all input sources from which links would be collected

However, in reality both flags behave differently compared to "normal" link checking without one of these flags. Namely, I've observed .gitignored files to be always included with --dump and --dump-inputs whereas with "normal" link checking ignored files are skipped. This might have been introduced with 41c1b97, but maybe dumping always behaved differently than from actual link checking.

@mre do you agree that this is a bug?

[thomas-zahner]

Additionally, I find --dump-inputs not very useful in the current state. If I do lychee . --dump-inputs all I get is .. I would expect the command to traverse all directories and output all files that would be checked by the normal link check process, just as described in the docs and help message.

[mre]

Thanks, I've improved this a bit. Using . as an input works as expected now:

> lychee --dump-inputs .
./lychee-bin/CHANGELOG.md
./lychee-bin/.config.dummy.report.md
./docs/TROUBLESHOOTING.md
./README.md
./CONTRIBUTING.md
./lychee-lib/CHANGELOG.md
./benches/README.md
./fixtures/TEST_VERBATIM.html
./fixtures/exclude-path/exclude/TEST.md
./fixtures/exclude-path/dir/to-exclude.txt
./fixtures/exclude-path/dir/excluded/TEST.md
./fixtures/exclude-path/dir/to-exclude.mdx
./fixtures/exclude-path/dir/TEST.md
./fixtures/exclude-path/TEST.md
...

Same for globs etc. It's probably still not perfect, but closer to what users expect.

[thomas-zahner]

Thanks, this is much better 👍
In theory I think it now does exactly what the user wants, or at least it does exactly what I expect lychee to do.

However, I noticed an inaccuracy. Running cargo run -- --dump-inputs . on the project directory includes the target directory, which is gitignored. This shouldn't happen in my opinion because normal link checking and dumping with --dump skips files which are gitignored.

We should cover this with test cases and update the implementation accordingly. I think if we resolve this inaccuracy the behaviour is "perfect".

[thomas-zahner]

Also I now noticed the following problem:

# no results

lychee --dump https://en.wikipedia.org/wiki/Tony_Hoare

# works as expected

curl https://en.wikipedia.org/wiki/Tony_Hoare | lychee - --dump

[katrinafyi]

I had a look at the Tony Hoare problem and filed this bug/feature suggestion #1883

[thomas-zahner]

This bug was resolved with: #1882

Note that the new issue #1898 is somewhat related to this. At least, it's something I had back in my mind when opening this issue. So thanks @katrinafyi for putting this into words.