crypto: add tls.setDefaultCACertificates()

[joyeecheung]

This API allows dynamically configuring CA certificates that
will be used by the Node.js TLS clients by default.

Once called, the provided certificates will become the default CA
certificate list returned by tls.getCACertificates('default') and
used by TLS connections that don't specify their own CA certificates.

This function only affects the current Node.js thread.

Background

This API serves two at least use cases:

1. When a party (e.g. corporate infra packages) need to enable CA certificates other than the bundled ones in the process, but they are only to be loaded by application developers (who can start tls connections themselves, or use a third-party library that does it) and have no control over the command line (which might be controlled by other administrators).
2. This allows HTTPS tests - both core and user land - to actually test against a self-signed certificate easily instead of using rejectUnauthorized: false or having to spawn child processes which can affect the validity or debuggability of the test.

The functionality provided by this API already has been possible via monkey patching tls or the global HTTPS agents, and the user land has already been doing it - for example, see syswide-cas, win-ca, ssl-root-cas. I am fairly certain when none of the existing options work there are applications/packages that would just go a nuclear route and use rejectUnauthorized: false in the monkey-patched option bag to avoid whatever woes they have, considering how often it shows up on the Internet and on even public GitHub. Providing a proper API to allow using custom certificates dynamically would overall make the practice less brittle in the ecosystem.

[nodejs-github-bot]

Review requested:

• @nodejs/crypto
• @nodejs/net

[github-actions]

The notable-change PRs with changes that should be highlighted in changelogs. label has been added by @joyeecheung.

Please suggest a text for the release notes if you'd like to include a more detailed summary, then proceed to update the PR description with the text or a link to the notable change suggested text comment. Otherwise, the commit will be placed in the Other Notable Changes section.

[codecov]

Codecov Report

Attention: Patch coverage is 88.48921% with 16 lines in your changes missing coverage. Please review.

Project coverage is 90.07%. Comparing base (049664b) to head (8d040a8).
Report is 21 commits behind head on main.

Files with missing lines	Patch %	Lines
src/crypto/crypto_context.cc	85.04%	5 Missing and 11 partials ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##             main   #58822    +/-   ##
========================================
  Coverage   90.06%   90.07%            
========================================
  Files         645      645            
  Lines      189130   189283   +153     
  Branches    37094    37128    +34     
========================================
+ Hits       170339   170494   +155     
+ Misses      11511    11476    -35     
- Partials     7280     7313    +33     

Files with missing lines	Coverage Δ
lib/tls.js	96.24% <100.00%> (+0.30%)	⬆️
src/crypto/crypto_context.cc	70.79% <85.04%> (+1.63%)	⬆️

... and 27 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

As indicated in comments, I generally don't think this is a good thing to add but don't feel strongly enough about it to block.

[mcollina]

lgtm

[joyeecheung]

Marking it blocked until I investigated whether the tls.setDefaultCACertificates(tls.getCACertificates('system')) suggested by @pimterry is implementable, since that sounds like a nicer API :)

(I gave it some thoughts and I wondered whether allowing more than just system certificates is a bit icky, but then I remembered again user land is already capable of monkey patching the tls methods to always add in random certificates and has been doing so anyways so not really a big deal ¯\(ツ)/¯ so far it looks implementable but I need to double check cleanup is done correctly).

[joyeecheung]

Also noticed another benefit of tls.setDefaultCACertificates() - it can be used to test HTTPS servers more easily without rejectUnauthorized: true (we do this quite a lot in our tests) or installing some certificate into the system, which may introduce differences in how it behaves in the real world v.s. how it behaves in tests with this disabled. This is probably not only limited to our tests but also user HTTPS tests. It might be a good follow up as a stream of good first issues for new contributors to update all the HTTPS tests using rejectUnauthorized: true unnecessarily and make them test with a real certificate instead.

[joyeecheung]

@pimterry @mcollina @jasnell Updated to the new API design tls.setDefaultCACertificates() with a bunch of tests added. I also found a use case of this in one of the tests in https://github.com/nodejs/node/pull/58980/files#diff-f54f03d62c6a7a6045b266ff965c4e09ba7a562a32b5d9f30745850ae7a9541e. Can you take a look again? Thanks!

[pimterry]

Love it! This looks great to me, and super useful 👍 👍 👍

[Ethan-Arrowood]

LGTM - nice work on the comprehensive tests.

I wonder if this could possible fix #54235 & #54251 - seems at least closely related if anything

[joyeecheung]

Fixed test skipping logic for withoutssl builds. @pimterry @Ethan-Arrowood can you take a look again? Thanks!

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/68013/