Merge branch 'groupby-aggs-using-numpy-groupies' of github.com:andersy005/xarray into groupby_npg

* 'groupby-aggs-using-numpy-groupies' of github.com:andersy005/xarray:
  Bump actions/github-script from 4.0.2 to 4.1 (pydata#5730)
  Set coord name concat when `concat`ing along a DataArray (pydata#5611)
  Add .git-blame-ignore-revs (pydata#5708)
  Type annotate tests (pydata#5728)
  Consolidate TypeVars in a single place (pydata#5569)
  add storage_options arg to to_zarr (pydata#5615)
  dataset `__repr__` updates (pydata#5580)
  Xfail failing test on main (pydata#5729)
  Add xarray-dataclasses to ecosystem in docs (pydata#5725)
  extend show_versions (pydata#5724)
  Move docstring for xr.set_options to numpy style (pydata#5702)
  Refactor more groupby and resample tests (pydata#5707)
  Remove suggestion to install pytest-xdist in docs (pydata#5713)
  Add typing to the OPTIONS dict (pydata#5678)
  Change annotations to allow str keys (pydata#5690)
  Whatsnew for float-to-top (pydata#5714)
  Use isort's float-to-top (pydata#5695)
  Fix errors in test_latex_name_isnt_split for min environments (pydata#5710)
  Improves rendering of complex LaTeX expressions as `long_name`s when plotting (pydata#5682)
  Use same bool validator as other inputs (pydata#5703)

Author: dcherian

.git-blame-ignore-revs

@@ -0,0 +1,5 @@
+# black PR 3142
+d089df385e737f71067309ff7abae15994d581ec
+
+# isort PR 1924
+0e73e240107caee3ffd1a1149f0150c390d43251

.github/workflows/upstream-dev-ci.yaml

@@ -122,7 +122,7 @@ jobs:
           shopt -s globstar
           python .github/workflows/parse_logs.py logs/**/*-log
       - name: Report failures
-        uses: actions/github-script@v4.0.2
+        uses: actions/github-script@v4.1
         with:
           github-token: ${{ secrets.GITHUB_TOKEN }}
           script: |

doc/contributing.rst

@@ -628,13 +628,7 @@ Or with one of the following constructs::
     pytest xarray/tests/[test-module].py::[TestClass]::[test_method]
 
 Using `pytest-xdist <https://pypi.python.org/pypi/pytest-xdist>`_, one can
-speed up local testing on multicore machines. To use this feature, you will
-need to install `pytest-xdist` via::
-
-    pip install pytest-xdist
-
-
-Then, run pytest with the optional -n argument::
+speed up local testing on multicore machines, by running pytest with the optional -n argument::
 
     pytest xarray -n 4
 

doc/ecosystem.rst

@@ -68,7 +68,7 @@ Extend xarray capabilities
 - `hypothesis-gufunc <https://hypothesis-gufunc.readthedocs.io/en/latest/>`_: Extension to hypothesis. Makes it easy to write unit tests with xarray objects as input.
 - `nxarray <https://github.com/nxarray/nxarray>`_: NeXus input/output capability for xarray.
 - `xarray-compare <https://github.com/astropenguin/xarray-compare>`_: xarray extension for data comparison.
-- `xarray-custom <https://github.com/astropenguin/xarray-custom>`_: Data classes for custom xarray creation.
+- `xarray-dataclasses <https://github.com/astropenguin/xarray-dataclasses>`_: xarray extension for typed DataArray and Dataset creation.
 - `xarray_extras <https://github.com/crusaderky/xarray_extras>`_: Advanced algorithms for xarray objects (e.g. integrations/interpolations).
 - `xpublish <https://xpublish.readthedocs.io/>`_: Publish Xarray Datasets via a Zarr compatible REST API.
 - `xrft <https://github.com/rabernat/xrft>`_: Fourier transforms for xarray data.

doc/whats-new.rst

@@ -22,15 +22,25 @@ v0.19.1 (unreleased)
 
 New Features
 ~~~~~~~~~~~~
+- Xarray now does a better job rendering variable names that are long LaTeX sequences when plotting (:issue:`5681`, :pull:`5682`).
+  By `Tomas Chor <https://github.com/tomchor>`_.
 - Add a option to disable the use of ``bottleneck`` (:pull:`5560`)
   By `Justus Magin <https://github.com/keewis>`_.
 - Added ``**kwargs`` argument to :py:meth:`open_rasterio` to access overviews (:issue:`3269`).
   By `Pushkar Kopparla <https://github.com/pkopparla>`_.
+- Added ``storage_options`` argument to :py:meth:`to_zarr` (:issue:`5601`).
+  By `Ray Bell <https://github.com/raybellwaves>`_, `Zachary Blackwood <https://github.com/blackary>`_ and
+  `Nathan Lis <https://github.com/wxman22>`_.
 
 
 Breaking changes
 ~~~~~~~~~~~~~~~~
 
+- The ``__repr__`` of a :py:class:`xarray.Dataset`'s ``coords`` and ``data_vars``
+  ignore ``xarray.set_option(display_max_rows=...)`` and show the full output
+  when called directly as, e.g., ``ds.data_vars`` or ``print(ds.data_vars)``
+  (:issue:`5545`, :pull:`5580`).
+  By `Stefan Bender <https://github.com/st-bender>`_.
 
 Deprecations
 ~~~~~~~~~~~~
@@ -51,8 +61,15 @@ Internal Changes
   By `Deepak Cherian <https://github.com/dcherian>`_.
 - Explicit indexes refactor: decouple ``xarray.Index``` from ``xarray.Variable`` (:pull:`5636`).
   By `Benoit Bovy <https://github.com/benbovy>`_.
+- Fix ``Mapping`` argument typing to allow mypy to pass on ``str`` keys (:pull:`5690`).
+  By `Maximilian Roos <https://github.com/max-sixty>`_.
+- Annotate many of our tests, and fix some of the resulting typing errors. This will
+  also mean our typing annotations are tested as part of CI. (:pull:`5728`).
+  By `Maximilian Roos <https://github.com/max-sixty>`_.
 - Improve the performance of reprs for large datasets or dataarrays. (:pull:`5661`)
   By `Jimmy Westling <https://github.com/illviljan>`_.
+- Use isort's `float_to_top` config. (:pull:`5695`).
+  By `Maximilian Roos <https://github.com/max-sixty>`_.
 
 .. _whats-new.0.19.0:
 

properties/test_encode_decode.py

@@ -4,9 +4,10 @@
 These ones pass, just as you'd hope!
 
 """
-import pytest  # isort:skip
+import pytest
 
 pytest.importorskip("hypothesis")
+# isort: split
 
 import hypothesis.extra.numpy as npst
 import hypothesis.strategies as st
@@ -24,7 +25,7 @@
 
 @pytest.mark.slow
 @given(st.data(), an_array)
-def test_CFMask_coder_roundtrip(data, arr):
+def test_CFMask_coder_roundtrip(data, arr) -> None:
     names = data.draw(
         st.lists(st.text(), min_size=arr.ndim, max_size=arr.ndim, unique=True).map(
             tuple
@@ -38,7 +39,7 @@ def test_CFMask_coder_roundtrip(data, arr):
 
 @pytest.mark.slow
 @given(st.data(), an_array)
-def test_CFScaleOffset_coder_roundtrip(data, arr):
+def test_CFScaleOffset_coder_roundtrip(data, arr) -> None:
     names = data.draw(
         st.lists(st.text(), min_size=arr.ndim, max_size=arr.ndim, unique=True).map(
             tuple

properties/test_pandas_roundtrip.py

@@ -28,7 +28,7 @@
 
 
 @st.composite
-def datasets_1d_vars(draw):
+def datasets_1d_vars(draw) -> xr.Dataset:
     """Generate datasets with only 1D variables
 
     Suitable for converting to pandas dataframes.
@@ -49,7 +49,7 @@ def datasets_1d_vars(draw):
 
 
 @given(st.data(), an_array)
-def test_roundtrip_dataarray(data, arr):
+def test_roundtrip_dataarray(data, arr) -> None:
     names = data.draw(
         st.lists(st.text(), min_size=arr.ndim, max_size=arr.ndim, unique=True).map(
             tuple
@@ -62,15 +62,15 @@ def test_roundtrip_dataarray(data, arr):
 
 
 @given(datasets_1d_vars())
-def test_roundtrip_dataset(dataset):
+def test_roundtrip_dataset(dataset) -> None:
     df = dataset.to_dataframe()
     assert isinstance(df, pd.DataFrame)
     roundtripped = xr.Dataset(df)
     xr.testing.assert_identical(dataset, roundtripped)
 
 
 @given(numeric_series, st.text())
-def test_roundtrip_pandas_series(ser, ix_name):
+def test_roundtrip_pandas_series(ser, ix_name) -> None:
     # Need to name the index, otherwise Xarray calls it 'dim_0'.
     ser.index.name = ix_name
     arr = xr.DataArray(ser)
@@ -87,7 +87,7 @@ def test_roundtrip_pandas_series(ser, ix_name):
 
 @pytest.mark.xfail
 @given(numeric_homogeneous_dataframe)
-def test_roundtrip_pandas_dataframe(df):
+def test_roundtrip_pandas_dataframe(df) -> None:
     # Need to name the indexes, otherwise Xarray names them 'dim_0', 'dim_1'.
     df.index.name = "rows"
     df.columns.name = "cols"

setup.cfg

@@ -161,7 +161,7 @@ exclude=
 [isort]
 profile = black
 skip_gitignore = true
-force_to_top = true
+float_to_top = true
 default_section = THIRDPARTY
 known_first_party = xarray
 

xarray/backends/api.py

@@ -1319,6 +1319,7 @@ def to_zarr(
     append_dim: Hashable = None,
     region: Mapping[str, slice] = None,
     safe_chunks: bool = True,
+    storage_options: Dict[str, str] = None,
 ):
     """This function creates an appropriate datastore for writing a dataset to
     a zarr ztore
@@ -1330,6 +1331,22 @@ def to_zarr(
     store = _normalize_path(store)
     chunk_store = _normalize_path(chunk_store)
 
+    if storage_options is None:
+        mapper = store
+        chunk_mapper = chunk_store
+    else:
+        from fsspec import get_mapper
+
+        if not isinstance(store, str):
+            raise ValueError(
+                f"store must be a string to use storage_options. Got {type(store)}"
+            )
+        mapper = get_mapper(store, **storage_options)
+        if chunk_store is not None:
+            chunk_mapper = get_mapper(chunk_store, **storage_options)
+        else:
+            chunk_mapper = chunk_store
+
     if encoding is None:
         encoding = {}
 
@@ -1372,13 +1389,13 @@ def to_zarr(
         already_consolidated = False
         consolidate_on_close = consolidated or consolidated is None
     zstore = backends.ZarrStore.open_group(
-        store=store,
+        store=mapper,
         mode=mode,
         synchronizer=synchronizer,
         group=group,
         consolidated=already_consolidated,
         consolidate_on_close=consolidate_on_close,
-        chunk_store=chunk_store,
+        chunk_store=chunk_mapper,
         append_dim=append_dim,
         write_region=region,
         safe_chunks=safe_chunks,

xarray/backends/zarr.py

@@ -713,6 +713,9 @@ def open_zarr(
         falling back to read non-consolidated metadata if that fails.
     chunk_store : MutableMapping, optional
         A separate Zarr store only for chunk data.
+    storage_options : dict, optional
+        Any additional parameters for the storage backend (ignored for local
+        paths).
     decode_timedelta : bool, optional
         If True, decode variables and coordinates with time units in
         {'days', 'hours', 'minutes', 'seconds', 'milliseconds', 'microseconds'}