subarray returns Uint8Array which can't handle toString('uff-8')

[anion155]

import { Buffer } from 'buffer';

declare const messageRaw: ArrayBuffer;
const message = Buffer.from(messageRaw);
const headCorrect = message.slice(0, size).toString('utf-8'); // 57,57,97,102,101,102,56,57,97....
const headIncorrect = message.subarray(0, size).toString('utf-8'); // 99af...

[personalizedrefrigerator]

I'm also experiencing this issue with version 6.0.3 (running in React Native/Hermes).

Related:

• .subarray method should return Buffer instance #260
• fix: subarray returns Buffer instance, fixes #260 #261

[chjj]

Any runtime which displays this behavior doesn't implement the es2016 "subarray calls child class" behavior and is returning a plain Uint8Array.

Node v4.0.0 to v8.0.0 displayed this behavior, even with the native node.js Buffer object. So if you are running with an older version of v8, it could be argued that feross/buffer is accurately emulating the native node.js Buffer.

This is probably only fixable with an ugly hack. Not sure if people are open to that. Personally, I would just avoid using subarray and stick to slice when using Buffers.

[chjj]

Note: if you are experiencing this in a runtime which does support es2016 (i.e. a modern runtime), please post the runtime/browser/platform here.

If there is actually a popular runtime out there that doesn't implement es2016 properly, we may have to support it.

[hedi-edelbloute]

Hi @chjj 👋

We faced this issue twice, it broke only on this app (React native environment, es2017) :
https://github.com/LedgerHQ/ledger-live/blob/develop/apps/ledger-live-mobile/package.json
https://github.com/LedgerHQ/ledger-live/blob/develop/apps/ledger-live-mobile/tsconfig.json

We polyfill Buffer like this : https://github.com/LedgerHQ/ledger-live/blob/develop/apps/ledger-live-mobile/src/polyfill.ts#L33

And it works perfectly fine usually

Some infos about our issue... it first occured like this :
cardano-foundation/ledgerjs-hw-app-cardano#37

Our partner Strica found that the issue was related to the subarray method, we fixed the issue by gently asking them to use slice instead of subarray (but slice is deprecated on node)

We stumbled on the issue a second time with this PR (that we reverted) : LedgerHQ/ledger-live#6189

Thanks to the analysis of overcat we found some differences between our environments :

The second environment is this one (Electron, React, es2020) :
https://github.com/LedgerHQ/ledger-live/blob/develop/apps/ledger-live-desktop/package.json
https://github.com/LedgerHQ/ledger-live/blob/develop/apps/ledger-live-desktop/tsconfig.json

Issue occurs here with a subarray method call : https://github.com/stellar/js-xdr/blob/master/src/serialization/xdr-writer.js#L79

According to overcat, returned buffer is a Uint8Array 🤔

We are considering doing the ugly hack you are talking about just below our global.Buffer polyfill

Hopefully I gave enough infos to see if there is something to be done on your side or not, and thanks !

[chjj]

@hedi-edelbloute is this affecting iOS, Android, or both? Which versions?

I can't easily check to see if a Safari web view would cause this right now, but I would be shocked if this is occurring inside an Android web view.

edit: The electron one is very strange. Not sure what to make of that.

[chjj]

In the meantime you could try this monkey patch:

if (!(Buffer.alloc(1).subarray(0, 1) instanceof Buffer)) {
  Buffer.prototype.subarray = function subarray() {
    const result = Uint8Array.prototype.subarray.apply(this, arguments);
    Object.setPrototypeOf(result, Buffer.prototype);
    return result;
  };
}

[hedi-edelbloute]

@chjj My phone is a Redmi note 10 Pro on Android 11 and I reproduced the issue, I'm trying to gather more informations. I think it also occured on iOS but i'm going to confirm this.

Thank you for the patch proposal and your help, it's greatly appreciated.

EDIT : We have no issue on Electron environment, sorry if it's confusing, it's only on React native environment

[chjj]

Is your Android web view somehow using an old version of chrome? Is there any way to check? I'm not an expert on mobile web views, but I'm certain chrome version 60.x.x included v8 version 6.0.286 which should have the correct es2016 subarray behavior. (Chrome 60 was released 2017-07-25)

[hedi-edelbloute]

Confirmed that issue also happens on iOS, version 16.6.1 on iPhone 12

Same here, not an expert about this subject 😅 I think I found a solution to get the webview version using chrome://inspect/#devices, let me check !

[vweevers]

stick to slice when using Buffers.

Note that slice() is deprecated in Node.js because the Node.js implementation (which does not copy) is not compatible with TypedArray.prototype.slice() which does copy. Unlikely to ever be removed from the Node.js API though, so you're fine.

[chjj]

@vweevers, agreed.

I think ledger might be in an unfortunate situation though. Their ecosystem is comprised of a number submitted apps from many different developers who are probably expecting subarray to behave the way it does in modern node.js.

I'm still not exactly convinced feross/buffer should account for this, but I am really confused as to why these mobile web views are not exhibiting modern javascript behavior with regards to subarray. It'd be nice to get to the bottom of this.