dataset attrs list of strings to_netcdf() error

[nagordon]

What happened?

when I try and save the dataset with an attribute as a list of strings it fails.

See example code

import xarray as xr
import pandas as pd
import numpy as np

ds = xr.Dataset()

temp = 15 + 8 * np.random.randn(2, 2, 3)

precip = 10 * np.random.rand(2, 2, 3)

lon = [[-99.83, -99.32], [-99.79, -99.23]]

lat = [[42.25, 42.21], [42.63, 42.59]]

ds["temperature"] = (("x", "y", "time"), temp)

ds["temperature_double"] = (("x", "y", "time"), temp * 2)

ds["precipitation"] = (("x", "y", "time"), precip)

ds.coords["lat"] = (("x", "y"), lat)

ds.coords["lon"] = (("x", "y"), lon)

ds.coords["time"] = pd.date_range("2014-09-06", periods=3)

ds.coords["reference_time"] = pd.Timestamp("2014-09-05")

ds.attrs['test'] = [1,2,3]
ds.to_netcdf('test.nc')
# no error

ds.attrs['test'] = ['1','2','3']
ds.to_netcdf('test.nc')

  File "C:\Users\xxxx\Anaconda3\envs\base1\lib\site-packages\scipy\io\_netcdf.py", line 555, in _write_att_values
   nc_type = REVERSE[values.dtype.char, values.dtype.itemsize]

 KeyError: ('U', 4)

What did you expect to happen?

No response

Minimal Complete Verifiable Example

No response

MVCE confirmation

• Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• Complete example — the example is self-contained, including all data and the text of any traceback.
• Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

No response

Environment

xr.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.10.9 | packaged by conda-forge | (main, Jan 11 2023, 15:15:40) [MSC v.1916 64 bit (AMD64)]
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 165 Stepping 2, GenuineIntel
byteorder: little
LC_ALL: None
LANG: en
LOCALE: ('English_United States', '1252')
libhdf5: 1.10.6
libnetcdf: None

xarray: 2022.11.0
pandas: 1.5.2
numpy: 1.21.5
scipy: 1.9.3
netCDF4: None
pydap: None
h5netcdf: None
h5py: 3.7.0
Nio: None
zarr: None
cftime: None
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: 1.3.5
dask: 2022.05.0
distributed: 2022.5.0
matplotlib: 3.6.2
cartopy: None
seaborn: 0.12.2
numbagg: None
fsspec: 2022.11.0
cupy: None
pint: 0.20.1
sparse: None
flox: None
numpy_groupies: None
setuptools: 65.6.3
pip: 22.3.1
conda: None
pytest: 7.1.2
IPython: 7.31.1
sphinx: 5.0.2

[DocOtak]

Are you able to install the netcdf4 package in your environment? If my memory serves, the scipy netCDF implementation only supports netCDF3 and array of strings in attributes are a netcdf4 feature.

[kmuehlbauer]

@DocOtak Memory served well 👍, stepping to open issues yesterday: #3374.

[dcherian]

Can we raise a nicer error message here suggesting the user install netcdf4-python

[DocOtak]

@dcherian would the following be a good place to put this check/raise?

xarray/xarray/backends/netcdf3.py

Lines 75 to 84 in b36819b

	def encode_nc3_attr_value(value):
	if isinstance(value, bytes):
	pass
	elif isinstance(value, str):
	value = value.encode(STRING_ENCODING)
	else:
	value = coerce_nc3_dtype(np.atleast_1d(value))
	if value.ndim > 1:
	raise ValueError("netCDF attributes must be 1-dimensional")
	return value

scipy has a short list of allowed attr dtypes, would we want our check to be in the form of an allow list? I guess does scipy implement all that is allowed in netcdf3?

[dcherian]

Seems like we could raise if utils.is_scalar(value)?

[DocOtak]

I think 1d arrays of other data types are allowed... it's just that the 1d dim value is hidden from you in attributes.

[dcherian]

In that case an allow list copied from scipy seems best. This file really just uses scipy so that seems appropriate