Getting 404 for links that certainly work

[ZdenekM]

I checked the log and have no idea why does it happen. Also tried to re-run the check but the result was the same. Please see https://github.com/robofit/arcor2/pull/438/checks?check_run_id=1861222777. Many thanks for any tips.

[gaurav-nelson]

Thanks @ZdenekM Most of the times websites block requests if too many requests are coming from a bot.

Currently, you can handle such URL's that you know are working by using the config-file option and providing a configuration file with the following:

{
  "ignorePatterns": [
    {
      "pattern": "^https://github.com"
    }
  ]
}

Doing this ignores the 404s and let's your checks pass.

[ZdenekM]

Unfortunately, most of the links in my READMEs are within GitHub. Would it be a solution to insert some (configurable) delay between requests? I don't care much how long the check lasts...

[ZdenekM]

Could it be that the problem is in links containing %2 as this one? It seems that other links within the repo are ok. See the report https://github.com/robofit/arcor2/runs/2030470582?check_suite_focus=true.

[philrz]

Per the last comment, there definitely seems to be a problem with links containing %2. For instance, I put up a README.md in one of my GitHub repos containing the following two hyperlinks that both work when clicked on from a browser.

[`count.sql`](https://github.com/brimdata/super/blob/main/scripts/super-cmd-perf/queries/count.sql)
[`search+.sql`](https://github.com/brimdata/super/blob/main/scripts/super-cmd-perf/queries/search%2B.sql)

The Actions run that logged the failure:

=========================> MARKDOWN LINK CHECK <=========================
(node:19) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)

FILE: ./README.md
[✓] https://github.com/brimdata/super/blob/main/scripts/super-cmd-perf/queries/count.sql → Status: 200
[✖] https://github.com/brimdata/super/blob/main/scripts/super-cmd-perf/queries/search%2B.sql → Status: 404

2 links checked.

ERROR: 1 dead links found!
[✖] https://github.com/brimdata/super/blob/main/scripts/super-cmd-perf/queries/search%2B.sql → Status: [40](https://github.com/philrz/linkcheck/actions/runs/12055637032/job/33616408980#step:5:41)4

=========================================================================

That said, right after writing this comment I spotted the note in the README to push users toward the new https://github.com/UmbrellaDocs/action-linkspector and I can confirm that one does not have this problem. So others that find this issue may want to pursue that!