Skip to content

Commit 002a73f

Browse files
authored
Merge branch 'main' into PYL-W0108
2 parents 5eeb6e4 + 23abb5b commit 002a73f

File tree

21 files changed

+615
-443
lines changed

21 files changed

+615
-443
lines changed

changes/2751.bugfix.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Fixed bug with Zarr using device memory, instead of host memory, for storing metadata when using GPUs.

changes/2751.doc.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Added new user guide on :ref:`user-guide-gpu`.

changes/2751.feature.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Added :meth:`zarr.config.enable_gpu` to update Zarr's configuration to use GPUs.

changes/2795.bugfix.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Alters the behavior of ``create_array`` to ensure that any groups implied by the array's name are created if they do not already exist. Also simplifies the type signature for any function that takes an ArrayConfig-like object.

changes/2822.feature.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Add arbitrary `shards` to Hypothesis strategy for generating arrays.

docs/developers/contributing.rst

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -230,6 +230,27 @@ during development at `http://0.0.0.0:8000/ <http://0.0.0.0:8000/>`_. This can b
230230

231231
$ hatch --env docs run serve
232232

233+
.. _changelog:
234+
235+
Changelog
236+
~~~~~~~~~
237+
238+
zarr-python uses `towncrier`_ to manage release notes. Most pull requests should
239+
include at least one news fragment describing the changes. To add a release
240+
note, you'll need the GitHub issue or pull request number and the type of your
241+
change (``feature``, ``bugfix``, ``doc``, ``removal``, ``misc``). With that, run
242+
```towncrier create``` with your development environment, which will prompt you
243+
for the issue number, change type, and the news text::
244+
245+
towncrier create
246+
247+
Alternatively, you can manually create the files in the ``changes`` directory
248+
using the naming convention ``{issue-number}.{change-type}.rst``.
249+
250+
See the `towncrier`_ docs for more.
251+
252+
.. _towncrier: https://towncrier.readthedocs.io/en/stable/tutorial.html
253+
233254
Development best practices, policies and procedures
234255
---------------------------------------------------
235256

docs/user-guide/config.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ Configuration options include the following:
3232
- Whether empty chunks are written to storage ``array.write_empty_chunks``
3333
- Async and threading options, e.g. ``async.concurrency`` and ``threading.max_workers``
3434
- Selections of implementations of codecs, codec pipelines and buffers
35+
- Enabling GPU support with ``zarr.config.enable_gpu()``. See :ref:`user-guide-gpu` for more.
3536

3637
For selecting custom implementations of codecs, pipelines, buffers and ndbuffers,
3738
first register the implementations in the registry and then select them in the config.

docs/user-guide/gpu.rst

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
.. _user-guide-gpu:
2+
3+
Using GPUs with Zarr
4+
====================
5+
6+
Zarr can use GPUs to accelerate your workload by running
7+
:meth:`zarr.config.enable_gpu`.
8+
9+
.. note::
10+
11+
`zarr-python` currently supports reading the ndarray data into device (GPU)
12+
memory as the final stage of the codec pipeline. Data will still be read into
13+
or copied to host (CPU) memory for encoding and decoding.
14+
15+
In the future, codecs will be available compressing and decompressing data on
16+
the GPU, avoiding the need to move data between the host and device for
17+
compression and decompression.
18+
19+
Reading data into device memory
20+
-------------------------------
21+
22+
:meth:`zarr.config.enable_gpu` configures Zarr to use GPU memory for the data
23+
buffers used internally by Zarr.
24+
25+
.. code-block:: python
26+
27+
>>> import zarr
28+
>>> import cupy as cp # doctest: +SKIP
29+
>>> zarr.config.enable_gpu() # doctest: +SKIP
30+
>>> store = zarr.storage.MemoryStore() # doctest: +SKIP
31+
>>> z = zarr.create_array( # doctest: +SKIP
32+
... store=store, shape=(100, 100), chunks=(10, 10), dtype="float32",
33+
... )
34+
>>> type(z[:10, :10]) # doctest: +SKIP
35+
cupy.ndarray
36+
37+
Note that the output type is a ``cupy.ndarray`` rather than a NumPy array.

docs/user-guide/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ Advanced Topics
2323
performance
2424
consolidated_metadata
2525
extending
26+
gpu
2627

2728

2829
.. Coming soon

src/zarr/api/asynchronous.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
from typing_extensions import deprecated
1111

1212
from zarr.core.array import Array, AsyncArray, create_array, get_array_metadata
13-
from zarr.core.array_spec import ArrayConfig, ArrayConfigLike
13+
from zarr.core.array_spec import ArrayConfig, ArrayConfigLike, ArrayConfigParams
1414
from zarr.core.buffer import NDArrayLike
1515
from zarr.core.common import (
1616
JSON,
@@ -856,7 +856,7 @@ async def create(
856856
codecs: Iterable[Codec | dict[str, JSON]] | None = None,
857857
dimension_names: Iterable[str] | None = None,
858858
storage_options: dict[str, Any] | None = None,
859-
config: ArrayConfig | ArrayConfigLike | None = None,
859+
config: ArrayConfigLike | None = None,
860860
**kwargs: Any,
861861
) -> AsyncArray[ArrayV2Metadata] | AsyncArray[ArrayV3Metadata]:
862862
"""Create an array.
@@ -1018,7 +1018,7 @@ async def create(
10181018
mode = "a"
10191019
store_path = await make_store_path(store, path=path, mode=mode, storage_options=storage_options)
10201020

1021-
config_dict: ArrayConfigLike = {}
1021+
config_dict: ArrayConfigParams = {}
10221022

10231023
if write_empty_chunks is not None:
10241024
if config is not None:

0 commit comments

Comments
 (0)