Skip to content

ED2 met process failing #1742

@serbinsh

Description

@serbinsh

I'm trying to setup runs at NOBS (1850 burn site, Manitoba Canada) and running into issues using the FLUXNET drivers

> # Write pecan.CHECKED.xml
> PEcAn.settings::write.settings(settings, outputfile = "pecan.CHECKED.xml")
[1] "/data/Model_Output/pecan.output/PEcAn_2000000723/pecan.CHECKED.xml"
> 
> # start from scratch if no continue is passed in
> statusFile <- file.path(settings$outdir, "STATUS")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
> if (length(which(commandArgs() == "--continue")) == 0 && file.exists(statusFile)) {
+   file.remove(statusFile)
+ }
>   
> # Do conversions
> settings <- PEcAn.utils::do_conversions(settings)
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
2017-10-30 15:34:00 DEBUG  [PEcAn.utils::do_conversions] : 
   do.conversion outdir /data/Model_Output/pecan.output/dbfiles 
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
2017-10-30 15:34:00 INFO   [PEcAn.utils::do_conversions] : PROCESSING:  met 
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
2017-10-30 15:34:00 INFO   [PEcAn.utils::do_conversions] : 
   calling met.process: 
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
  scale  format.id           format.name format.mimetype format.inputtype
1  site 5000000001 FLUXNET2015_SUBSET_HH        text/csv              csv
$download.raw
[1] TRUE

$met2cf
[1] TRUE

$standardize
[1] TRUE

$met2model
[1] TRUE

Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
2017-10-30 15:34:00 DEBUG  [PEcAn.utils::convert.input] : 
   Convert.Inputs download.Fluxnet2015 NA localhost 
   /data/Model_Output/pecan.output/dbfiles/Fluxnet2015_site_0-506 
   FLUXNET2015_SUBSET_HH text/csv 506 1995/01/01 2007/12/31 
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
2017-10-30 15:34:00 INFO   [PEcAn.utils::convert.input] : 
   start CHECK Convert.Inputs download.Fluxnet2015 NA localhost 
   /data/Model_Output/pecan.output/dbfiles/Fluxnet2015_site_0-506/ 
   FLUXNET2015_SUBSET_HH text/csv 506 1995/01/01 2007/12/31 
2017-10-30 15:34:00 WARN   [PEcAn.DB::dbfile.input.check] : 
   Found possible matching input. Checking if its associate files are on 
   host machine 
          id site_id notes          created_at          updated_at start_date
1 1000013675     506       2017-10-12 12:38:40 2017-10-30 18:58:58 1990-01-01
    end_date                   name parent_id user_id access_level raw
1 2008-12-31 Fluxnet2015_site_0-506        NA      NA            4  NA
  format_id
1     5e+09
2017-10-30 15:34:00 DEBUG  [PEcAn.utils::convert.input] : 
   File id = 2000018835 File name = 
   FLX_CA-Man_FLUXNET2015_SUBSET_HH_1994-2008_1-3 File path = 
   /data/Model_Output/pecan.output/dbfiles/Fluxnet2015_site_0-506 Input id 
   = 1000013675 10 
2017-10-30 15:34:00 INFO   [PEcAn.utils::convert.input] : 
   end CHECK for existing input record. 
2017-10-30 15:34:00 INFO   [PEcAn.utils::convert.input] : 
   Skipping this input conversion because files are already available. 
2017-10-30 15:34:00 INFO   [.met2cf.module] : 
   Begin change to CF Standards 
2017-10-30 15:34:01 DEBUG  [PEcAn.utils::convert.input] : 
   Convert.Inputs met2CF.csv 1000013675 localhost 
   /data/Model_Output/pecan.output/dbfiles/Fluxnet2015_CF_site_0-506 CF 
   Meteorology application/x-netcdf 506 1995/01/01 2007/12/31 
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
2017-10-30 15:34:01 INFO   [PEcAn.utils::convert.input] : 
   start CHECK Convert.Inputs met2CF.csv 1000013675 localhost 
   /data/Model_Output/pecan.output/dbfiles/Fluxnet2015_CF_site_0-506/ CF 
   Meteorology application/x-netcdf 506 1995/01/01 2007/12/31 
2017-10-30 15:34:01 WARN   [PEcAn.DB::dbfile.input.check] : 
   Found possible matching input. Checking if its associate files are on 
   host machine 
          id site_id notes          created_at          updated_at start_date
1 1000013676     506       2017-10-12 12:39:17 2017-10-12 12:39:17 1995-01-01
    end_date                      name  parent_id user_id access_level raw
1 2008-12-31 Fluxnet2015_CF_site_0-506 1000013675      NA            4  NA
  format_id
1        33
2017-10-30 15:34:01 DEBUG  [PEcAn.utils::convert.input] : 
   File id = 2000018836 File name = 
   FLX_CA-Man_FLUXNET2015_SUBSET_HH_1994-2008_1-3 File path = 
   /data/Model_Output/pecan.output/dbfiles/Fluxnet2015_CF_site_0-506 Input 
   id = 1000013676 10 
2017-10-30 15:34:01 INFO   [PEcAn.utils::convert.input] : 
   end CHECK for existing input record. 
2017-10-30 15:34:01 INFO   [PEcAn.utils::convert.input] : 
   Skipping this input conversion because files are already available. 
2017-10-30 15:34:01 INFO   [.met2cf.module] : 
   Finished change to CF Standards 
2017-10-30 15:34:01 INFO   [.metgapfill.module] : Gapfilling 
No geonamesUsername set. See http://geonames.wordpress.com/2010/03/16/ddos-part-ii/ and set one with options(geonamesUsername="foo") for some services to work
2017-10-30 15:34:01 DEBUG  [PEcAn.utils::convert.input] : 
   Convert.Inputs metgapfill 1000013676 localhost 
   /data/Model_Output/pecan.output/dbfiles/Fluxnet2015_CF_gapfill_site_0-506 
   CF Meteorology application/x-netcdf 506 1995/01/01 2007/12/31 
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
2017-10-30 15:34:01 INFO   [PEcAn.utils::convert.input] : 
   start CHECK Convert.Inputs metgapfill 1000013676 localhost 
   /data/Model_Output/pecan.output/dbfiles/Fluxnet2015_CF_gapfill_site_0-506/ 
   CF Meteorology application/x-netcdf 506 1995/01/01 2007/12/31 
2017-10-30 15:34:01 WARN   [PEcAn.DB::dbfile.input.check] : 
   Found possible matching input. Checking if its associate files are on 
   host machine 
          id site_id notes          created_at          updated_at start_date
1 1000013677     506       2017-10-12 12:40:52 2017-10-12 12:40:52 1995-01-01
    end_date                              name  parent_id user_id access_level
1 2008-12-31 Fluxnet2015_CF_gapfill_site_0-506 1000013676      NA            4
  raw format_id
1  NA        33
2017-10-30 15:34:01 DEBUG  [PEcAn.utils::convert.input] : 
   File id = 2000018837 File name = 
   FLX_CA-Man_FLUXNET2015_SUBSET_HH_1994-2008_1-3 File path = 
   /data/Model_Output/pecan.output/dbfiles/Fluxnet2015_CF_gapfill_site_0-506 
   Input id = 1000013677 10 
2017-10-30 15:34:01 INFO   [PEcAn.utils::convert.input] : 
   end CHECK for existing input record. 
2017-10-30 15:34:01 INFO   [PEcAn.utils::convert.input] : 
   Skipping this input conversion because files are already available. 
$input.id
[1] 1000013677

$dbfile.id
[1] 2000018837

2017-10-30 15:34:01 INFO   [.metgapfill.module] : 
   Finished Gapfilling Met 
2017-10-30 15:34:01 INFO   [.met2model.module] : 
   Begin Model Specific Conversion 
[1] "Convert to model format"
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
2017-10-30 15:34:01 DEBUG  [PEcAn.utils::convert.input] : 
   Convert.Inputs met2model.ED2 1000013677 localhost 
   /data/Model_Output/pecan.output/dbfiles/Fluxnet2015_ED2_site_0-506 
   ed.met_driver_header files format text/plain 506 1995/01/01 2007/12/31 
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
2017-10-30 15:34:01 INFO   [PEcAn.utils::convert.input] : 
   start CHECK Convert.Inputs met2model.ED2 1000013677 localhost 
   /data/Model_Output/pecan.output/dbfiles/Fluxnet2015_ED2_site_0-506/ 
   ed.met_driver_header files format text/plain 506 1995/01/01 2007/12/31 
2017-10-30 15:34:02 DEBUG  [PEcAn.utils::convert.input] : 
   File id = File name = File path = Input id = 10 
2017-10-30 15:34:02 INFO   [PEcAn.utils::convert.input] : 
   end CHECK for existing input record. 
2017-10-30 15:34:02 DEBUG  [PEcAn.utils::convert.input] : 
   convert.input executing the following function: 
   PEcAn.ED2::met2model.ED2(lst=-6, lat=55.8796, lon=-98.4808, 
   spin_nyear=NULL, spin_nsample=NULL, spin_resample=NULL, overwrite=FALSE, 
   in.path='/data/Model_Output/pecan.output/dbfiles/Fluxnet2015_CF_gapfill_site_0-506', 
   in.prefix='FLX_CA-Man_FLUXNET2015_SUBSET_HH_1994-2008_1-3', 
   outfolder='/data/Model_Output/pecan.output/dbfiles/Fluxnet2015_ED2_site_0-506/', 
   start_date='1995/01/01', end_date='2007/12/31') 

R version 3.4.1 (2017-06-30) -- "Single Candle"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-redhat-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> remotefunc <- function() {PEcAn.ED2::met2model.ED2(lst=-6, lat=55.8796, lon=-98.4808, spin_nyear=NULL, spin_nsample=NULL, spin_resample=NULL, overwrite=FALSE, in.path='/data/Model_Output/pecan.output/dbfiles/Fluxnet2015_CF_gapfill_site_0-506', in.prefix='FLX_CA-Man_FLUXNET2015_SUBSET_HH_1994-2008_1-3', outfolder='/data/Model_Output/pecan.output/dbfiles/Fluxnet2015_ED2_site_0-506/', start_date='1995/01/01', end_date='2007/12/31')}
> remoteout <- remotefunc()
2017-10-30 15:34:03 WARN   [PEcAn.ED2::met2model.ED2] : 
   The file already exists! Moving to next month! 
Error in UseMethod("h5write") : 
  no applicable method for 'h5write' applied to an object of class "c('array', 'double', 'numeric')"
Calls: remotefunc -> <Anonymous> -> <Anonymous>
Execution halted
[1] 1
2017-10-30 15:34:03 INFO   [PEcAn.utils::convert.input] : 
   RESULTS: Convert.Input 
2017-10-30 15:34:03 INFO   [PEcAn.utils::convert.input] : 1 
2017-10-30 15:34:03 INFO   [PEcAn.utils::convert.input] :  
2017-10-30 15:34:03 DEBUG  [PEcAn.utils::convert.input] : 
   Processing data failed, please check validity of args:lst=-6, 
   lat=55.8796, lon=-98.4808, spin_nyear=NULL, spin_nsample=NULL, 
   spin_resample=NULL, overwrite=FALSE, 
   in.path='/data/Model_Output/pecan.output/dbfiles/Fluxnet2015_CF_gapfill_site_0-506', 
   in.prefix='FLX_CA-Man_FLUXNET2015_SUBSET_HH_1994-2008_1-3', 
   outfolder='/data/Model_Output/pecan.output/dbfiles/Fluxnet2015_ED2_site_0-506/', 
   start_date='1995/01/01', end_date='2007/12/31' 
2017-10-30 15:34:03 SEVERE [PEcAn.utils::convert.input] : 
   Unable to process data using this function:met2model.ED2 
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
Warning: '$.SafeList' is deprecated.
Use 'PEcAn.settings::$.SafeList' instead.
See help("Deprecated")
> proc.time()
   user  system elapsed 
  7.973   0.680   9.580 
The PEcAn project is supported by the National Science Foundation (ABI #1062547, ABI #1458021, DIBBS #1261582, ARC #1023477, EF #1318164, EF #1241894, EF #1241891), NASA Terrestrial Ecosystems, the Energy Biosciences Institute, and an Amazon AWS in Education Grant.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions