dask: Data.__getitem__, Data.__setitem__

cf/data/QUESTIONS.rst

@@ -0,0 +1,25 @@
+Questions and answers
+=====================
+
+A place to record random thoughts about the daskification of
+`cf.Data`, possibly prior to starting an issue on GitHub.
+
+----
+
+Q. When we run something that executes all of the lazy operations
+   (like `cf.Data.is_masked`), should/could we replace the dask array
+   with a "persisted" version of the computed data? If we did this, we
+   would want to have the ability to cache persisted chunks to disk,
+   as they came into being on each thread (see, for instance,
+   `chest`). To do this or not do this could be controlled by a
+   configuation setting.
+
+A. ?
+
+----
+
+Q.
+
+A. ?
+
+----

cf/data/README.rst

@@ -0,0 +1,24 @@
+`cf.Data` developer notes
+=========================
+
+Hardness of the mask
+--------------------
+
+Any `cf.Data` method that changes the dask array should consider
+whether or not the mask hardness needs resetting before
+returning. This will be necessary if there is the possibility that the
+operation being applied to the dask array could lose the "memory" on
+its chunks of whether or not the mask is hard.
+
+A common situation that causes a chunk to lose its memory of whether
+or not the mask is hard is when a chunk could have contained a
+unmasked `numpy` array prior to the operation, but the operation could
+convert it to a masked `numpy` array. The new masked array will always
+have the `numpy` default hardness (i.e. soft), which may be
+incorrect.
+
+The mask hardness is most easily reset with the
+`cf.Data._reset_mask_hardness` method.
+
+`cf.Data.__setitem__` and `cf.Data.where` are examples of methods that
+need to reset the mask in this manner.