Removing `primordials` from Node.js project

[anonrig]

History

• Starting from November 2019, we migrated core modules to primordials
  • Ref: Migration of core modules to primordials node#30697
• On February 2022, TSC took a vote and came to a conclusion to Use primordials in the error path only if there are no significant perf regression.
  • Ref: https://github.com/nodejs/TSC/blob/main/votes/2022-02-21-0.json

Proposal

Due to the performance and several other concerns I propose removing all primordials from the Node.js project. Remember that this is a living document and I'll try to update the following list upon feedback from the community and the TSC.

Pros

• It ensures that Node.js internals are not tampered or acceptable to prototype pollutions.

Cons

• Any prototype pollution in Node.js core is not accepted as a vulnerability in our security/bug bounty program.
  • We use primordials with the purpose of security, but does not include it in our bounty program, which contradicts the notion of security in Node.js
• It creates a friction for optimizing performance, and requires the contributor to benchmark the code with and without the primordials, causing in reality to regress over the performance.
  • One solution for this is to have a performance regression catcher like the benchmarking.nodejs.org but IMHO that's fighting fire with fire.
• V8 doesn't always optimize the primordials efficiently and causes performance regressions, and what Turbofan optimizes are subject to change. This requires constant benchmarking for each V8 release.
  • We have a section that mentions known performance issues but we never follow them. For example, if you search for ArrayPrototypePush in Node.js core, you'll find a lot of examples and usages of primordials with known peformance issues. Reference to the documentation: https://github.com/nodejs/node/blob/HEAD/doc/contributing/primordials.md#primordials-with-known-performance-issues.
  • There are several issues around primordials in Node.js performance repository. Reference: Performance of primordials performance#70
• It creates a higher bar for contribution to Node.js. The primordials.md document is 775 lines on top of our existing documentation, and we don't follow them even though it has explicit examples of what NOT to use - https://github.com/nodejs/node/blob/main/doc/contributing/primordials.md
  • The issue of disallowing certain primordials can be solved with linter rules, but IMHO that's fighting fire with fire.
• There is an existing proposal in TC39 called proposal-symbol-proto for mitigating prototype pollution.
  • Ref: https://github.com/tc39/proposal-symbol-proto

Suggestion

I recommend the removal of primordials from the Node.js project. I strongly believe that it causes more harm then the intended behavior effecting the future of the project causing performance regressions and friction for new contributors.

---

Appreciate, if you can leave a 👍 or 👎 to this issue.

cc @nodejs/tsc @gireeshpunathil @mcollina @ljharb @BridgeAR

[RafaelGSS]

Can we have a draft to see exactly how much performance improvement are we talking about?

[aduh95]

I don't think this proposal is reasonable, primordials are a useful tool for reliability of the software, and most of the time have no impact on performance. Also as you mentioned, the TSC has already voted for the use of primordials in the error path, so your proposal should be compatible with that vote.

Any prototype pollution in Node.js core is not accepted as a vulnerability in our security/bug bounty program.

That's not a con, it's just a fact.

• We use primordials with the purpose of security, but does not include it in our bounty program, which contradicts the notion of security in Node.js

We do not use primordials with the purpose of security.

• We have a section that mentions known performance issues but we never follow them.

It's really unfair to state "we never follow them", in reality those changes tend to be watched carefully and not land if perf regression is shown.

For example, if you search for ArrayPrototypePush in Node.js core, you'll find a lot of examples and usages of primordials with known peformance issues.

Well what does it tell you? Not every part of lib/ is on a hot path, sometimes reliability is more important than performance.

• The issue of disallowing certain primordials can be solved with linter rules, but IMHO that's fighting fire with fire.

It's already in place, we have the avoid-prototype-pollution rule that forbid the use of some primordials.

• There is an existing proposal in TC39 called proposal-symbol-proto for mitigating prototype pollution.

  • Ref: https://github.com/tc39/proposal-symbol-proto

That's not a con either.

[benjamingr]

I feel strongly that we should keep primordials when it is required for web standards (for example EventTarget should be reasonably tamper proof because not doing so would violate spec semantics).

I also feel strongly we should keep primordials in "critical" error paths that must never throw.

I am +1 on removing primordials everywhere else, multiple people mentioned they are a burden to contribution and they make the code less readable, debuggable and fun to work on, even if performance was not an issue. They create friction in PRs, cause CI reruns and often needless churn. To be clear - it's not that they're not valuable it's that their value is greatly outweighed by their burden IMO.

---

It's important to echo @aduh95 's point - primordials are about reliability and are not a security mechanism, it does not provide tamper proofing nor does it sandbox anything.

---

@aduh95 the fact a TSC vote happened doesn't mean the TSC can't change its mind? Personally I'm a lot less pro-primordials than I was 2 years ago given feedback people several people gave me about how it impacts their contribution experience. Also the TSC itself changes in composition.

[ljharb]

A JS platform that falls apart when someone does a normal JavaScript action would be an embarrassment.

Primordials may not be the best or only way to solve this problem, but simply not solving it is something I would hope is deemed wildly unacceptable.

[aduh95]

@aduh95 the fact a TSC vote happened doesn't mean the TSC can't change its mind? Personally I'm a lot less pro-primordials than I was 2 years ago given feedback people several people gave me about how it impacts their contribution experience. Also the TSC itself changes in composition.

I'm not sure, this has never happened before AFAICT. I still think Yagiz should try to draft a proposal that's compatible with it, or at least that does not overlap with it, so the discussion can be more productive. Depending on the issue of that proposal, you could consider overriding the previous vote, but now's not the time IMO.

[ronag]

A JS platform that falls apart when someone does a normal JavaScript action would be an embarrassment.

This is more of an opinion than a statement of fact. IMHO even though there are clear benefits, the cons outweigh the pros.

@anonrig I don't think the current consensus of only using primordial for the error path is too bad.

[benjamingr]

I'm not sure, this has never happened before AFAICT

I think trying things and learning from them is a strength, the TSC was super opposed to promises at one point for example to the point people were actively mocking me for advocating them (maybe it was the CTC? it was a long time ago when that was still a thing :)). I change my mind around a lot of things personally though admittedly most of those things weren't a TSC vote.

I think it would be productive to focus on arguments for and against usage of primordials (generally and in specific cases) though to be clear if my PoV doesn't reach consensus (primordials in critical error paths and web standards and not elsewhere) I'd be fine with that.

---

A JS platform that falls apart when someone does a normal JavaScript action would be an embarrassment.

I don't think that's a productive statement especially given none of the userland packages use them. So I don't think we're giving a meaningful guarantee with them anyway. Additionally I don't like the language "would be an embarrassment" since "to whom?" and also it assumes other peoples' feelings about stuff.

Primordials may not be the best or only way to solve this problem, but simply not solving it is something I would hope is deemed wildly unacceptable.

The issue isn't "primordials solve nothing so let's remove them" it's "the new problems they create in our code and the friction they add are maybe not worth the thing they're solving for us".

[ronag]

@benjamingr +1 on that

[ljharb]

@benjamingr all of my userland packages do, in fact, use them. I apologize for the choice of language; how better could i indicate how serious I think this is?

[benjamingr]

@ljharb

all of my userland packages do, in fact, use them.

First of all - 😮 😅
Second of all - how? What if someone changes a builtin prototype before your packages' code is loaded?
Third of all - I suspect that for most popular packages that is pretty rare.

how better could i indicate how serious I think this is?

"Hey, I feel strongly that we should keep using primordials. They are beneficial because they prevent issues like A,B or C which are important to users and people have run into (e.g here are some issues people opened complaining about it D, E or F). I think the project would lose from removing them."

Basically just objective unambiguous facts about why you think your opinion is correct would have the best chances of impacting the peoples' (and the TSC's) opinion, ideally with numbers.

[ljharb]

What if someone changes a builtin prototype before your packages' code is loaded?

the nature of javascript is such that you can't defend against that, but, if your code is first-run, you can cache everything you need.

how?

I use https://npmjs.com/call-bind and https://npmjs.com/get-intrinsic to achieve it.

pretty rare

I agree! but that doesn't mean that node shouldn't be robust against that. Lots of use cases for node have zero dependencies - especially with the test runner, where deleting builtins is actually a more common thing to simulate different environments and test an implementation against it, and robustness in that case is important

"…"

Thanks, I appreciate that feedback and will try to incorporate it.

[aduh95]

@benjamingr it's all fine if TSC members change their mind, I do that all the time too, highly recommend it, past Antoine was a complete idiot most of the time. Still the TSC decision needs to be somewhat final otherwise we cannot move forward. I don't know if the TSC charter says anything about it, but anyway, it's going to create off-topic discussions to try to override an existing vote, I think we can discuss primordials while accepting the outcome of the previous vote.

My personal opinion on primordials is that it provides more upside than downsides; I don't know if it's because I lack empathy or something, but I don't think it creates a lot of friction in PRs, and it doesn't seem to me that it's a burden to contribution. If anything, I think it makes the code more fun and debuggable, but I've learned that I might be in the minority here.

[benjamingr]

@ljharb

the nature of javascript is such that you can't defend against that, but, if your code is first-run, you can cache everything you need.

Yes but if you load "Package A" that changes a prototype then "Package B" that comes following it will get a polluted prototype. This can actually be useful (e.g. for instrumentation) and it would make certain use cases impossible (e.g. people writing instrumentation "deeply" hooking a server library).

I do think it would be very helpful to be able to provide this guarantee (though for me it's only meaningful if it's a guarantee for actual user code which is likely to use many dependencies). It would be ideal for the project to promote a proposal to address this at the ECMAScript level (which is where I believe this naturally belongs) rather than attempt to ship "semi correct" approaches that provide very fragile guarantees.

@aduh95

@benjamingr it's all fine if TSC members change their mind, I do that all the time too, highly recommend it, past Antoine was a complete idiot most of the time. Still the TSC decision needs to be somewhat final otherwise we cannot move forward. I don't know if the TSC charter says anything about it, but anyway, it's going to create off-topic discussions to try to override an existing vote, I think we can discuss primordials while accepting the outcome of the previous vote.

The TSC vote about primordials was a year and a half ago, quite a respectable amount of time in software I think? I'm personally in favor of keeping primordials in error paths (and was not part of that vote) but I think it's fine to raise the issue again and see if the opinion of folks changed.

My personal opinion on primordials is that it provides more upside than downsides; I don't know if it's because I lack empathy or something, but I don't think it creates a lot of friction in PRs, and it doesn't seem to me that it's a burden to contribution. If anything, I think it makes the code more fun and debuggable, but I've learned that I might be in the minority here.

I acknowledge that for some people (like yourself and probably others) they don't create a burden to contribution. I don't understand how they make stuff more fun or debuggable though :D

I've had several contributors complain about it to me in the past. I personally find them a bit tedious (a bit more so recently) but I've reviewed, approved and helped land a bunch of primordials changes before that perspective shift. Even before that it was "something I put up with to contribute" like GYP, different/weird debugging/testing flows etc.