V8 Changes - HasIndexedPropertiesInExternalArrayData, GetIndexedPropertiesExternalArrayDataLength, GetIndexedPropertiesExternalArrayData no longer available - what to use instead?

[metabench]

Since v2.2.0 V8 has changed in a way so that my addon would no longer compile.

This would work previously:

Local<Object> obj = args[0].As<Object>();
if (obj->HasIndexedPropertiesInExternalArrayData()) {
    arr_data_structure_length = obj->GetIndexedPropertiesExternalArrayDataLength();
    arr = static_cast<double*>(obj->GetIndexedPropertiesExternalArrayData());
}

Now I get errors such as

 error C2039: 'HasIndexedPropertiesInExternalArrayData' : is not a member of 'v8::Object'
 error C2039: 'GetIndexedPropertiesExternalArrayDataLength' : is not a member of 'v8::Object'
 error C2039: 'GetIndexedPropertiesExternalArrayData' : is not a member of 'v8::Object'

How should it be done now?

[rvagg]

Probably one for @trevnorris to explain how it was dealt with for Buffer and what to look at for examples. Basically we've had to use V8 TypedArrays (Uint8Array) for this which may also be possible for your use.

[bnoordhuis]

Haven't you asked this question before in #883?

[metabench]

@bnoordhuis I asked about how to get access to data as arrays, which indeed is the same goal I have here. Since then, V8 has changed, so what I found out then no longer applies. Smalloc is out of the picture too now.

[bnoordhuis]

Okay. The basic premise is still the same: you use an ArrayBuffer for storage and you can optionally put a typed array like Uint8Array on top of that. External array data as a feature on arbitrary JS objects is completely gone now.

Have a look at deps/v8/include/v8.h, particularly the ArrayBuffer, ArrayBufferView and TypedArray classes. If you have specific questions, I'll try to answer them.

[trevnorris]

Key things to watch out for:

• When creating the ArrayBuffer, and you want to pre-allocate the memory, if you want v8 to control the lifetime of the memory will need to create with kInternalized.
• Use ArrayBuffer::GetContents() if you want to retrieve the data without externalizing the contents.

[mikeseven]

@bnoordhuis @trevnorris so this means there is no way to access to a pointer to JS Array data anymore? At least, I came to this conclusion. For node bindings to OpenCL (node-opencl), this implies only supporting ArrayBuffer and TypedArrays, which are more optimal anyway.
Just want to make sure I'm not missing something.
Thanks.

[trevnorris]

so this means there is no way to access to a pointer to JS Array data anymore?

Don't think I fully understand the question, but everything that could be done with the previous implementation can still be done today. I would simply recommend using the node_buffer.h API so you don't have to worry about it. i.e. node::Buffer::Data(Local<Value>) and node::Buffer::Length(Local<Value>).

[mikeseven]

Not really. Doing that would make a copy of the data from JS Array to node::Buffer. This could be OK if the buffer is for read only. But if it is for write, then you'll have to do another copy back to the JS Array.

[metabench]

The current use case I have in mind involves creating Typed Arrays within JavaScript, and then both reading and writing them within C++. Would someone who knows more about the underlying V8 implementation please advise me on if this still possible through casting the typed array to an array of C++ native types?

@trevnorris Buffers may be a good way to approach this if I were to be writing new code (though maybe Typed Arrays would in fact work better), but in this case I have existing code that's using Typed Arrays and would like easy and C++ idiomatic read/write access to them.

@bnoordhuis Thanks. I have not yet had a chance to study those V8 files. It's very useful to know what to refer to in the near future. I do have one question right now though. I'll clarify by saying that I'm only interested in getting read/write access to Typed Arrays. Is that still possible to do using pointers with predictable indexes, or has the underlying V8 engine moved away from that structure?

[targos]

@metabench You may find some interesting things in the code of node-canvas. I think this line answers your question about getting a pointer to the Typed Array data.

[trevnorris]

@mikeal by "JS Array" do you mean the indexed object returned by new Buffer()? That has me confused, because Array.isArray(new Buffer(n)) == false.

There's been an alteration in the native buffer API. Which is node::Buffer::New() takes control of the data when a char* is passed. The memory copy API has been moved to node::Buffer::Copy(). So accessing node::Buffer::Data() will return a char* to the memory located in the buffer object.

@metabench Buffer is now a Uint8Array. Here's part of our implementation that creates a new Uint8Array from existing memory and turns it into a Buffer: https://github.com/nodejs/node/blob/v4.1.1/src/node_buffer.cc#L375-L392

[mikeal]

@trevnorris I think you meant @mikeseven :)

[bnoordhuis]

I'm only interested in getting read/write access to Typed Arrays. Is that still possible to do using pointers with predictable indexes

I'm not really sure what you by 'pointers with predictable indexes'. If you mean if it's possible to create an ArrayBuffer from a pointer to a chunk of memory you own, yes, that's possible.