Executing unoptimized wasm file causes OOM

[Nugine]

Version

v19.1.0

Platform

Linux Nugine-Ub2 5.15.0-52-generic #58-Ubuntu SMP Thu Oct 13 08:03:55 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

Dependencies:

• Rust toolchain (nightly, 2022-11-14)
• just (https://github.com/casey/just)
• wasm-pack (https://github.com/rustwasm/wasm-pack)

git clone https://github.com/Nugine/simd.git -b main
cd simd
git checkout 508bffc65c3b72b889666773fe3e30a2b5a329f4
cd crates/hex-simd
just wasm-test

I uploaded artifacts here for convenience: wbg_tmp.tar.gz

tar xvf wbg_tmp.tar.gz
cd wbg_tmp
ulimit -v 4000000 # 4GB
node run.js

/home/nugine/tests/nodejs-wasm-oom/wbg-tmp/wasm-bindgen-test.js:716
const wasmInstance = new WebAssembly.Instance(wasmModule, imports);
                     ^

RangeError: WebAssembly.Instance(): Out of memory: Cannot allocate Wasm memory for new instance
    at Object.<anonymous> (/home/nugine/tests/nodejs-wasm-oom/wbg-tmp/wasm-bindgen-test.js:716:22)
    at Module._compile (node:internal/modules/cjs/loader:1205:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1259:10)
    at Module.load (node:internal/modules/cjs/loader:1068:32)
    at Module._load (node:internal/modules/cjs/loader:909:12)
    at Module.require (node:internal/modules/cjs/loader:1092:19)
    at require (node:internal/modules/cjs/helpers:103:18)
    at Object.<anonymous> (/home/nugine/tests/nodejs-wasm-oom/wbg-tmp/run.js:3:22)
    at Module._compile (node:internal/modules/cjs/loader:1205:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1259:10)

Node.js v19.1.0

How often does it reproduce? Is there a required condition?

Always

What is the expected behavior?

It should work without OOM.

What do you see instead?

Nodejs allocates too much memory until OOM.

Additional information

See Nugine/simd@48bd619#comments

[bnoordhuis]

Guess no one else is going to pick this up? I can't try myself right now but does it work with the --jitless flag? V8's JIT makes fairly large reservations for WASM code. It's possible you're hitting a resource limit.

[Nugine]

node --jitless run.js

Warning: disabling flag --expose_wasm due to conflicting flags
/home/nugine/tests/nodejs-wasm-oom/wbg-tmp/wasm-bindgen-test.js:715
const wasmModule = new WebAssembly.Module(bytes);
                   ^

ReferenceError: WebAssembly is not defined
    at Object.<anonymous> (/home/nugine/tests/nodejs-wasm-oom/wbg-tmp/wasm-bindgen-test.js:715:20)
    at Module._compile (node:internal/modules/cjs/loader:1205:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1259:10)
    at Module.load (node:internal/modules/cjs/loader:1068:32)
    at Module._load (node:internal/modules/cjs/loader:909:12)
    at Module.require (node:internal/modules/cjs/loader:1092:19)
    at require (node:internal/modules/cjs/helpers:103:18)
    at Object.<anonymous> (/home/nugine/tests/nodejs-wasm-oom/wbg-tmp/run.js:3:22)
    at Module._compile (node:internal/modules/cjs/loader:1205:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1259:10)

Node.js v19.1.0

---

ulimit -v 12000000 # 12GB
node run.js

running 3 tests

test hex_simd::tests::test_safety ... ok
test hex_simd::tests::test_alloc ... ok
test hex_simd::tests::test_str ... ok

test result: ok. 3 passed; 0 failed; 0 ignored



#
# Fatal error in , line 0
# Fatal process out of memory: Zone
#
#
#
#FailureMessage Object: 0x7fb5f0ac6fb0


#
# Fatal error in , line 0
# Fatal process out of memory: Zone
#
#
#
#FailureMessage Object: 0x7fb5eaff3010


#
# Fatal error in , line 0
# Fatal process out of memory: Zone
#
#
#
#FailureMessage Object: 0x7fb5ebff4fc0
 1: 0xc2abf1  [node]
 1: 0xc2abf1  [node]
 1: 0xc2abf1  [node]
 2: 0x1f231d4 V8_Fatal(char const*, ...) [node]
 2: 0x1f231d4 V8_Fatal(char const*, ...) [node]
 3: 0xda71ed  [node]
 3: 0xda71ed  [node]
 4: 0x140d53e  [node]
 5: 0x157353d  [node]
 6: 0x158ed18  [node]
 7: 0x15986b7 v8::internal::wasm::BuildTFGraph(v8::internal::AccountingAllocator*, v8::internal::wasm::WasmFeatures const&, v8::internal::wasm::WasmModule const*, v8::internal::compiler::WasmGraphBuilder*, v8::internal::wasm::WasmFeatures*, v8::internal::wasm::FunctionBody const&, std::vector<v8::internal::compiler::WasmLoopInfo, std::allocator<v8::internal::compiler::WasmLoopInfo> >*, v8::internal::compiler::NodeOriginTable*, int, v8::internal::wasm::InlinedStatus) [node]
 2: 0x1f231d4 V8_Fatal(char const*, ...) [node]
 3: 0xda71ed  [node]
 4: 0x140d53e  [node]
 5: 0x1570b65  [node]
 6: 0x158ceaf  [node]
 7: 0x15986b7 v8::internal::wasm::BuildTFGraph(v8::internal::AccountingAllocator*, v8::internal::wasm::WasmFeatures const&, v8::internal::wasm::WasmModule const*, v8::internal::compiler::WasmGraphBuilder*, v8::internal::wasm::WasmFeatures*, v8::internal::wasm::FunctionBody const&, std::vector<v8::internal::compiler::WasmLoopInfo, std::allocator<v8::internal::compiler::WasmLoopInfo> >*, v8::internal::compiler::NodeOriginTable*, int, v8::internal::wasm::InlinedStatus) [node]
 8: 0x209dc64 v8::internal::compiler::ExecuteTurbofanWasmCompilation(v8::internal::wasm::CompilationEnv*, v8::internal::wasm::WireBytesStorage const*, v8::internal::wasm::FunctionBody const&, int, v8::internal::Counters*, v8::internal::wasm::AssemblerBufferCache*, v8::internal::wasm::WasmFeatures*) [node]
 9: 0x156eac3 v8::internal::wasm::WasmCompilationUnit::ExecuteFunctionCompilation(v8::internal::wasm::CompilationEnv*, v8::internal::wasm::WireBytesStorage const*, v8::internal::Counters*, v8::internal::wasm::AssemblerBufferCache*, v8::internal::wasm::WasmFeatures*) [node]
10: 0x156f5dc v8::internal::wasm::WasmCompilationUnit::ExecuteCompilation(v8::internal::wasm::CompilationEnv*, v8::internal::wasm::WireBytesStorage const*, v8::internal::Counters*, v8::internal::wasm::AssemblerBufferCache*, v8::internal::wasm::WasmFeatures*) [node]
11: 0x15ae590  [node]
12: 0x15af7ab  [node]
13: 0x1beafe8 v8::platform::DefaultJobWorker::Run() [node]
14: 0xc2aea5  [node]
15: 0x7fb5f136cb43  [/lib/x86_64-linux-gnu/libc.so.6]
16: 0x7fb5f13fea00  [/lib/x86_64-linux-gnu/libc.so.6]
Trace/breakpoint trap (core dumped)

[bnoordhuis]

ulimit -v 12000000 # 12GB

Failing with a 12 GB virtual memory limit isn't surprising. V8 allocates really large guard pages for performance reasons. See #45557 for a similar bug report.

[Nugine]

I look into the process status before it gets killed. VmRSS is huge. Is it the expected behavior?

node run.js

cat /proc/14376/status

...
VmPeak:	28405548 kB
VmSize:	28340012 kB
VmLck:	       0 kB
VmPin:	       0 kB
VmHWM:	14549680 kB
VmRSS:	14549680 kB
RssAnon:	14545316 kB
RssFile:	    4364 kB
RssShmem:	       0 kB
VmData:	17059148 kB
VmStk:	     280 kB
VmExe:	   77716 kB
VmLib:	    8700 kB
VmPTE:	   33848 kB
VmSwap:	 1056128 kB
...

It's quite weird that nodejs allocates too much memory after the wasm code is finished. I still think this hehavior may be a bug.

[bnoordhuis]

That's indeed quite high. It could be the JS heap (not WASM), it maxes out at 1.4 GB unless you raise the limit.

As a bit of expectation management: WASM is handled by V8, not node, so if there is a bug, you're going to have to report it upstream.

With that out of the way, it'd be good to test WASM-related flags. See node --v8-options. For example, --liftoff-only lets you check if the problem is in TurboFan, the optimizing compiler (visible in the stack trace above.)

[Nugine]

node --liftoff-only run.js

running 3 tests

test hex_simd::tests::test_safety ... ok
test hex_simd::tests::test_alloc ... ok
test hex_simd::tests::test_str ... ok

test result: ok. 3 passed; 0 failed; 0 ignored

So the problem is in TurboFan.

I have submitted a issue to v8: https://bugs.chromium.org/p/v8/issues/detail?id=13543

[bnoordhuis]

I'd include more details in the bug report if I were you, makes it more likely to get triaged quickly.

I'll close out this bug report as there isn't anything for node to do at this time.