CRUNCEP THREDDS issue

[para2x]

Bug Description

I was hoping someone outside BU could run this and let me know if it successfully downloads files.

PEcAn.data.atmosphere::download.CRUNCEP(
  site_id = 1000005129, # This is not used inside the function. Lat/long is used
  lat.in = 46.0827,
  lon.in = -89.9792,
  model = NULL,
  scenario = NULL,
  ensemble_member = NULL,
  method = NULL,
  overwrite = FALSE,
  outfolder = '/fs/data1/pecan.data/dbfiles/CRUNCEP_site_1-5129/',
  start_date = '2005-01-01',
  end_date = '2010-12-31',
  verbose=TRUE
)

Right now, this returns this error here at both test-pecan and pecan2 machine.

Error in ncvar_get_inner(ncid2use, varid2use, nc$var[[li]]$missval, addOffset,  : 
  C function R_nc4_get_vara_double returned error

[ashiklom]

I tried this and got the same error. In my experience, OpenDAP can be pretty finnicky. With method = "ncss", this seems to work OK, so give that a shot.

It may be worth considering defaulting to that instead, since it seems like a more robust download method.

[para2x]

Thank you @ashiklom

[infotroph]

I can confirm the same behavior using ncdf4 functions directly:

> nc <- ncdf4::nc_open("https://thredds.daac.ornl.gov/thredds/dodsC/ornldaac/1220/mstmip_driver_global_hd_climate_tair_2005_v1.nc4", verbose=TRUE)
> ncdf4::ncvar_get(nc, varid="tair", start=c(181, 88, 1), c(1, 1, 1460))
Error in Rsx_nc4_get_vara_double: NetCDF: Access failure
Var: tair  Ndims: 3   Start: 0,87,180 Count: 1460,1,1
Error in ncvar_get_inner(ncid2use, varid2use, nc$var[[li]]$missval, addOffset,  : 
  C function R_nc4_get_vara_double returned error

[serbinsh]

On my mac its slightly different

PEcAn.data.atmosphere::download.CRUNCEP(
  site_id = 1000005129, # This is not used inside the function. Lat/long is used
  lat.in = 46.0827,
  lon.in = -89.9792,
  model = NULL,
  scenario = NULL,
  ensemble_member = NULL,
  method = "ncss",
  overwrite = FALSE,
  outfolder = '~/scratch',
  start_date = '2005-01-01',
  end_date = '2010-12-31',
  verbose=TRUE
)

trying URL 'https://thredds.daac.ornl.gov/thredds/ncss/ornldaac/1220/mstmip_driver_global_hd_landwatermask_v1.nc4?var=land_water_mask&disableLLSubset=on&disableProjSubset=on&horizStride=1&accept=netcdf'
downloaded 262 KB

2019-09-19 13:34:03 INFO   [PEcAn.data.atmosphere::download.CRUNCEP] :
   Downloading ~/scratch/CRUNCEP.2005.nc
2019-09-19 13:34:03 INFO   [PEcAn.data.atmosphere::download.CRUNCEP] :
   Attempting to access file at:
   https://thredds.daac.ornl.gov/thredds/ncss/grid/ornldaac/1220/mstmip_driver_global_hd_climate_tair_2005_v1.nc4/dataset.html
trying URL 'https://thredds.daac.ornl.gov/thredds/ncss/grid/ornldaac/1220/mstmip_driver_global_hd_climate_tair_2005_v1.nc4/dataset.html?var=tair&south=46.0827&west=-89.9792&north=46.082705&east=-89.979195&time_start=2005-01-01T00:00:00Z&time_end=2005-12-31T21:00:00Z&accept=netcdf'
downloaded 18 KB

[1] "ncvar_def: entering"
[1] "ncvar_def: prec= float"
[1] "ncvar_def: making ncvar object for var air_temperature"
2019-09-19 13:34:09 INFO   [PEcAn.data.atmosphere::download.CRUNCEP] :
   Attempting to access file at:
   https://thredds.daac.ornl.gov/thredds/ncss/grid/ornldaac/1220/mstmip_driver_global_hd_climate_lwdown_2005_v1.nc4/dataset.html
trying URL 'https://thredds.daac.ornl.gov/thredds/ncss/grid/ornldaac/1220/mstmip_driver_global_hd_climate_lwdown_2005_v1.nc4/dataset.html?var=lwdown&south=46.0827&west=-89.9792&north=46.082705&east=-89.979195&time_start=2005-01-01T00:00:00Z&time_end=2005-12-31T21:00:00Z&accept=netcdf'
downloaded 18 KB

[1] "ncvar_def: entering"
[1] "ncvar_def: prec= float"
[1] "ncvar_def: making ncvar object for var surface_downwelling_longwave_flux_in_air"
2019-09-19 13:34:14 INFO   [PEcAn.data.atmosphere::download.CRUNCEP] :
   Attempting to access file at:
   https://thredds.daac.ornl.gov/thredds/ncss/grid/ornldaac/1220/mstmip_driver_global_hd_climate_press_2005_v1.nc4/dataset.html
trying URL 'https://thredds.daac.ornl.gov/thredds/ncss/grid/ornldaac/1220/mstmip_driver_global_hd_climate_press_2005_v1.nc4/dataset.html?var=press&south=46.0827&west=-89.9792&north=46.082705&east=-89.979195&time_start=2005-01-01T00:00:00Z&time_end=2005-12-31T21:00:00Z&accept=netcdf'
downloaded 18 KB

[1] "ncvar_def: entering"
[1] "ncvar_def: prec= float"
[1] "ncvar_def: making ncvar object for var air_pressure"
2019-09-19 13:34:19 INFO   [PEcAn.data.atmosphere::download.CRUNCEP] :
   Attempting to access file at:
   https://thredds.daac.ornl.gov/thredds/ncss/grid/ornldaac/1220/mstmip_driver_global_hd_climate_swdown_2005_v1.nc4/dataset.html
trying URL 'https://thredds.daac.ornl.gov/thredds/ncss/grid/ornldaac/1220/mstmip_driver_global_hd_climate_swdown_2005_v1.nc4/dataset.html?var=swdown&south=46.0827&west=-89.9792&north=46.082705&east=-89.979195&time_start=2005-01-01T00:00:00Z&time_end=2005-12-31T21:00:00Z&accept=netcdf'
downloaded 18 KB

[1] "ncvar_def: entering"
[1] "ncvar_def: prec= float"
[1] "ncvar_def: making ncvar object for var surface_downwelling_shortwave_flux_in_air"
2019-09-19 13:34:24 INFO   [PEcAn.data.atmosphere::download.CRUNCEP] :
   Attempting to access file at:
   https://thredds.daac.ornl.gov/thredds/ncss/grid/ornldaac/1220/mstmip_driver_global_hd_climate_uwind_2005_v1.nc4/dataset.html
trying URL 'https://thredds.daac.ornl.gov/thredds/ncss/grid/ornldaac/1220/mstmip_driver_global_hd_climate_uwind_2005_v1.nc4/dataset.html?var=uwind&south=46.0827&west=-89.9792&north=46.082705&east=-89.979195&time_start=2005-01-01T00:00:00Z&time_end=2005-12-31T21:00:00Z&accept=netcdf'
downloaded 18 KB

Seems to be working with ncss

[infotroph]

Hopefully fixed by #2426