Lightweight Isolates & Faster isolate communication

[mkustermann]

This issue is tracking our ongoing work on making isolates more lightweight and improving communication between them. This work will mostly focus on isolates which were spawned from the same source.

As part of this work we will investigate a number of approaches, including:

• Faster sending of mutable messages across isolates
• Sharing of program structures (e.g. libraries, classes, functions, fields), JITed code, ...
• Allowing multiple mutator threads to run in parallel the same code (yet using separate global state)

We would like to implement this in several stages, here's our current progress:

• [done] Change embedder API to allow creation and initialization of isolate groups
• [done] Add support for standalone/flutter embedders to initialize isolate-group spawnees
• [done] Enable multiple isolates to share one heap.
• [done] Split up existing field objects into mutable and immutable part.
• [done] Share program structure in AOT mode.
• [done] Implement SendPort.sendAndExit to allow close to 0-cost sending of messages
• [done] Migrate embedders to take advantage of isolate groups.
• Sharing JITed code: Will require making major parts of VM re-entrant from multiple mutators

Subtasks for AOT enable by-default

• Ensure all existing embedders that use Isolate.spawn() API have been augmented to supply Dart_InitializeParams.initialize_isolate callback with correct implementation.

Subtasks for JIT

• Reduce RunWithMutatorsStopped in SwitchableCall miss handler to only around patching the tuple in the pool (and remove hack in symbol table)
• Enable class finalization on BG compiler by adding appropriate locking: Allows us to test this in normal JIT mode and is pre-requisite for many mutator threads
• Make shared BG compiler queue that multiple mutators can use (i.e. move BG compiler functionality from isolate to the group)

Future work tracked at Issue 46752

NOTICE

This work will require substantial restructuring of the Dart VM internals and is therefore relatively risky and giving a concrete timeline is not possible - that being said we expect to make good progress on this in the coming months.

The (old) isolates will continue to be available as they were before and we will try to ensure to not regress any existing code.

What the actual improvements in terms of isolate spawning time, parallelization as well as communications will be can only really be seen once the implementation is done. That being said, we can get numbers for some features in an intermediate stage.

[mkustermann]

Thread local allocation buffers (TLABs) have been restored in df34f65, 5737b45 and the background compiler has been changed to allow allocating into new space in fb88c98.

[mkustermann]

A prototype for faster sending of mutable messages across isolates has been made in cl/92280, which might allow 2-3x faster serialization and close to O(1) deserialization time.

[mkustermann]

A prototype for transferrable byte arrays is available in cl/86640

[mkustermann]

Before we can start working on heap unification we have to change the isolate spawning functionality which currently just calls out to the embedder and assumes the embedder happens to load the exact same code into the isolate. See #36648.

[aam]

transferable typeddata landed in https://dart-review.googlesource.com/c/sdk/+/99623