Root dimensions missing on large deep nested file

[CptOrange16]

Hello

I am working with EUMETSAT satellite data and i have a large netcdf with many nested groups. The entire dataset is available here at eumetsat website but i uploaded a single file to my google drive here, since the whole dataset is like 40gb and takes a long time to download.

My only issue so far is that the dimensions are missing from the root. Everything else seems ok and consistent with the netcdf file.

If i do
ncdump test_vii.nc >> dump.txt

I can see the overlap dimensions in the output:

 netcdf test_vii{
dimensions:
        overlaps = 2 ;
// global attributes:
                string :Conventions = "CF-1.6" ;

But when i generate the datatree using:

tree = datatree.open_datatree(test_vii.nc, engine='netcdf4', mask_and_scale = False)

I get:

DataTree('None', parent=None)
   Dimensions:  ()
   Data variables:
       *empty*
   Attributes: (12/21)
       Conventions:             CF-1.6
       metadata_conventions:    Unidata Dataset Discovery v1.0
       product_name:            W_XX-EUMETSAT-Darmstadt,SAT,SGA1-3MI-1C-RAD_C__2...
       title:                   3MI L1C Gridded Reflectance facto

Sorry for the complicated netcdf file but it is the only one i have here with groups nested this deeply.
Let me know if you need any other info.

[TomNicholas]

Hi @CptOrange16, and thank you for the reproducible example!

Unfortunately this isn't so much a problem with datatree as a subtle difference between xarray's data model and netCDF's data model.

Your root group has dimensions, but no variables (data or coordinates). In netCDF this is allowed, but in xarray dimensions don't exist independently, instead the dimensions of a Dataset are simply the set of dimensions present across all variables in that dataset.

You can see this without using datatree, just xarray. Using xarray.open_dataset will by default open the root group of the file (use group=... to open a different group) as a dataset, but the overlap dimension won't appear.

In [8]: ds = xr.open_dataset("test_vii.nc", group=None)

In [9]: ds
Out[9]: 
<xarray.Dataset>
Dimensions:  ()
Data variables:
    *empty*
Attributes: (12/21)
    Conventions:             CF-1.6
    metadata_conventions:    Unidata Dataset Discovery v1.0
    product_name:            W_XX-EUMETSAT-Darmstadt,SAT,SGA1-3MI-1C-RAD_C__2...
    title:                   3MI L1C Gridded Reflectance factor
    summary:                 multi-view, spectral and polarized coregistered ...
    keywords:                geoprojected reflectance polarisation multidirec...
    ...                      ...
    sensing_end_time_utc:    2007-09-12 10:47:01.750
    environment:             Development
    references:              www.eumetsat.int
    xml_spec_file:           EPS-SG-PFS-3MI-1C-RAD_v21.xml
    orbit_start:             7
    orbit_end:               7

If you want to save a file which will preserve the dimension the smallest change you could make would be to create a dimension coordinate (i.e. a coordinate variable called overlap). Then xarray will show it, and so will datatree.