Native modules ABI compatibility problems on Node 17 with OpenSSL 3 / OpenSSL 1

[mmomtchev]

Version

17.3.0

Platform

ArchLinux

Subsystem

node-addon-api

What steps will reproduce the bug?

The ArchLinux developers have decided to distribute Node.js 17 built against a shared OpenSSL 1.1.1m library.

Is this to be considered a supported configuration?

Because it has lots of implications for native modules which won't be compatible between different distributions.

Should a flag be added to the Node API version?

Welcome to Node.js v17.3.0.
Type ".help" for more information.
> process.versions
{
  node: '17.3.0',
  v8: '9.6.180.15-node.12',
  uv: '1.42.0',
  zlib: '1.2.11',
  brotli: '1.0.9',
  ares: '1.18.1',
  modules: '102',
  nghttp2: '1.46.0',
  napi: '8',
  llhttp: '6.0.4',
  openssl: '1.1.1m',
  cldr: '40.0',
  icu: '70.1',
  tz: '2021a3',
  unicode: '14.0'
}
> 

How often does it reproduce? Is there a required condition?

No response

What is the expected behavior?

Binary compatibility for native modules built using the same ABI level

What do you see instead?

/usr/bin/node: symbol lookup error: /target/lib/binding/node-v102-linux-x64/gdal.node: undefined symbol: SSL_CTX_load_verify_file

Additional information

(https://bugs.archlinux.org/task/73268?string=node&project=0&search_name=&type%5B0%5D=&sev%5B0%5D=&pri%5B0%5D=&due%5B0%5D=&reported%5B0%5D=&cat%5B0%5D=&status%5B0%5D=open&percent%5B0%5D=&opened=&dev=&closed=&duedatefrom=&duedateto=&changedfrom=&changedto=&openedfrom=&openedto=&closedfrom=&closedto=)

[richardlau]

Distros choosing to link against a different version of OpenSSL is not a new issue and has previously come up before. The solution was for the distros to request/reserve a different ABI number for the variant, e.g.

node/doc/abi_version_registry.json

Line 42 in 55ceaec

{ "modules": 65, "runtime": "node", "variant": "debian-openssl_1.1.1", "versions": "10" },

(see nodejs/TSC#621 for the previous discussion on the topic).

[mhdawson]

@richardlau that's a good reference but with Node-API the addon is not necessarily tied to a specific ABI number at least in the checks done automatically. If there are differences required in terms of building/running the add-on would need to handle building with both.

I believe it may be a common case where distros ( I know RH does this as well) continue to use an older OpenSSL on older versions of the OS.

@mmomtchev OpenSSL 3 claims to be compatible with openssl 1.1.1, where there specific issues that you ran into?

[mmomtchev]

Yes, SSL_CTX_load_verify_file is present only in OpenSSL 3.3, it is used by libcurl which detects OpenSSL 3.3 during the build and the module won't run with 1.1

[mmomtchev]

@mhdawson It could be related to #40575 - if you aim to render native addons ABI-compatible between Node.js 17 + OpenSSL 3.3 and Node.js 17 + OpenSSL 1.1, then the modules are to be built with a lower level API? @danbev?

[mmomtchev]

@mhdawson, I confirm OpenSSL is to be built with OPENSSL_API_COMPAT=0x10100001L and no OPENSSL_CONFIGURED_API or OPENSSL_MIN_API, this eliminates the depenency on SSL_CTX_load_verify_file

In this case the ABI is preserved between OpenSSL 3.0 and OpenSSL 1.1

I will update the other issue as well

[mmomtchev]

Look at this:
https://github.com/curl/curl/blob/eeca818b1e8d1e61c2d4d833aed56ce4c510a9d4/lib/vtls/openssl.c#L3100

No matter what we do, this won't work on OpenSSL 1.1 if it is compiled on OpenSSL 3.0
This is the real problem

[mmomtchev]

@danbev, do you think that it would be possible to build the native addons with the OpenSSL 1.1 headers and to link against both OpenSSL 3.0 and OpenSSL 1.1 at runtime?

Or maybe use the 3.0 headers and patch them so they pretend to be 1.1

[mhdawson]

No matter what we do, this won't work on OpenSSL 1.1 if it is compiled on OpenSSL 3.0

For the reverse if you compile on OpenSSL 1.1.1 (ie earlier version of Node.js) will it then run with the later version of Node.js that uses OpenSSL 3.0?

We talked a bit about this in the Node-API team meeting and we don't promise ABI safety if you use APIs outside Node-API (like openSSL) but agreed it would be valuable to have a clear understanding of what will/will not work.

[mmomtchev]

As far as I understand the OpenSSL 3.0 project goals - yes - they are supposed to guarantee this - and it does work for me so far