Cross-dataset checks

[lewisjared]

Is it possible to also apply additional cross-dataset checks within this framework?

The use-case is that there are some additional checks that would be useful to perform on a collection of datasets. The example would be to validate that the branching attributes correctly map to a valid dataset.

From what I understand the CC checks operate at the dataset level, but I'm not aware if there is something equivalent for cross-dataset checks?

[sol1105]

Hi @lewisjared

The CC checks operate on file level.

esgf-qa conducts additional checks on dataset (i.e. variable) level:

• consistency checks
  • Global attributes (values and data types)
  • Variable attributes (values and data types)
  • Coordinates (values)
  • Dimensions (names and sizes)
• continuity checks (inter-file time and time_bnds continuity)
• compatibility checks
  • currently only xarray.open_mfdataset with different function parameters is run

There are also some cross-dataset checks:

• time coverage
• different datasets may not cover all the same time period of the experiment, which may or may not be on purpose
• inter-dataset consistency
  • datasets are grouped by realm and grid for certain checks
    • Required and non-required global attributes (values and data types)
    • Coordinates (values)
    • Dimensions (names and sizes)
  • false positives are likely for this checks, so any findings are reported at medium (required global attributes) and low priority

Consistency across simulations is currently not checked. Verifying branching attributes against the parent could probably be done on file level in cc-plugin-wcrp if there was a way to retrieve the necessary attributes from the parent simulation (eg. via esgf). It could also be possible in this package, eg. generally registering the necessary attributes of a simulation in a central place, in case that simulation is specified as parent of a simulation that is checked later, in which case the branching attributes could be checked.