Single machine tracking issue

[tomwhite]

This is an umbrella issue for tracking the work for making Cubed work better on a single machine.

Processes executor

Improvements to the processes executor

• Document how to run examples on local machine using processes #507
• Check allowed_mem does not exceed total on machine for local executors #510
• Issue warning if a computation actually used more memory than allowed_mem #110
• Remove max_tasks_per_child=1 limitation from processes executor #515

Threads executor

Improvements to the threads executor

• Manually manage the GIL with Cython #497
• Test Cubed using free-threaded Python #511

Accelerators

Work to make Cubed work with GPUs

• Adding Jax tests for the M1 mac. #508

Filesystem

• Automatically delete intermediate files on local filesystems #512
  • We may need Store intermediate data in a directory named after the compute ID #413 for this
• Zarr compression settings for local intermediate files #513