Description
What happened?
Zarr 2.11.0 made a change where chunks with data equal to the fill value are no longer written to disk by default. I would like to override the default, but this isn't possible using xarray's to_zarr
.
What did you expect to happen?
No response
Minimal Complete Verifiable Example
import numpy as np
import xarray as xr
ds = xr.Dataset({"x": (("d1"), np.full((2,), np.nan))})
ds.to_zarr("/tmp/ds.zarr", encoding={"x": {"write_empty_chunks":True}})
Relevant log output
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/tom/miniconda/envs/sgkit-3.8/lib/python3.8/site-packages/xarray/core/dataset.py", line 2036, in to_zarr
return to_zarr(
File "/Users/tom/miniconda/envs/sgkit-3.8/lib/python3.8/site-packages/xarray/backends/api.py", line 1431, in to_zarr
dump_to_store(dataset, zstore, writer, encoding=encoding)
File "/Users/tom/miniconda/envs/sgkit-3.8/lib/python3.8/site-packages/xarray/backends/api.py", line 1119, in dump_to_store
store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
File "/Users/tom/miniconda/envs/sgkit-3.8/lib/python3.8/site-packages/xarray/backends/zarr.py", line 534, in store
self.set_variables(
File "/Users/tom/miniconda/envs/sgkit-3.8/lib/python3.8/site-packages/xarray/backends/zarr.py", line 581, in set_variables
encoding = extract_zarr_variable_encoding(
File "/Users/tom/miniconda/envs/sgkit-3.8/lib/python3.8/site-packages/xarray/backends/zarr.py", line 220, in extract_zarr_variable_encoding
raise ValueError(
ValueError: unexpected encoding parameters for zarr backend: ['write_empty_chunks']
Anything else we need to know?
No response
Environment
INSTALLED VERSIONS
commit: None
python: 3.8.12 (default, Oct 12 2021, 06:23:56)
[Clang 10.0.0 ]
python-bits: 64
OS: Darwin
OS-release: 19.6.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
LOCALE: ('en_GB', 'UTF-8')
libhdf5: None
libnetcdf: None
xarray: 2022.3.0
pandas: 1.3.5
numpy: 1.21.5
scipy: 1.7.3
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: 2.11.0
cftime: None
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 2022.01.0
distributed: 2022.01.0
matplotlib: 3.5.1
cartopy: None
seaborn: 0.11.2
numbagg: None
fsspec: 2022.01.0
cupy: None
pint: None
sparse: None
setuptools: 58.0.4
pip: 21.2.4
conda: None
pytest: 6.2.5
IPython: 8.0.1
sphinx: 4.2.0