submission: rsat

[unai-perez]

Date accepted: 2021-09-30
Submitting Author: Unai Pérez-Goya (@unai-perez)
Other Authors: Manuel Montesino-SanMartin (@mmontesinosanmartin), Ana F Militino (@militino), María Dolores Ugarte (@lolaugartemartinez)
Repository: https://github.com/spatialstatisticsupna/rsat
Version submitted: 0.1.14
Editor: @jhollist
Reviewers: @khondula, @mhweber

Due date for @khondula: 2021-05-25

Due date for @mhweber: 2021-06-23
Archive: TBD
Version accepted: TBD

---

• Paste the full DESCRIPTION file inside a code block below:

Package: rsat
Type: Package
Title: Tools for Downloading, Customizing, and Processing Time Series of Satellite Images from Landsat, MODIS, and Sentinel 
Version: 0.1.14
Author: U Pérez - Goya [aut, cre] <unai.perez@unavarra.es>,
        M Montesino - SanMartin [aut] <manuel.montesino@unavarra.es>,
        A F Militino [aut] <militino@unavarra.es>,
        M D Ugarte [aut] <lola@unavarra.es>
Maintainer: U Perez - Goya <unai.perez@unavarra.es>
Description: Downloading, customizing, and processing time series of satellite images for a region of interest. 'rsat' functions allow a unified access to multispectral images from Landsat, MODIS and Sentinel repositories. 'rsat' also offers capabilities for customizing satellite images, such as tile mosaicking, image cropping and new variables computation. Finally, 'rsat' covers the processing, including cloud masking, compositing and gap-filling/smoothing time series of images (Militino et al., 2018 <doi:10.3390/rs10030398> and Militino et al., 2019 <doi:10.1109/TGRS.2019.2904193>).
Depends: R (>= 3.5.0), raster, sf, stars
Imports: XML, curl, httr, leafem, leaflet, rjson, rvest, tmap, xml2, zip, methods, sp, Rdpack, fields, calendR
RdMacros: Rdpack
Suggests: 
    rgdal,
    knitr,
    rmarkdown,
    covr,
    testthat
License: GPL-3
Encoding: UTF-8
LazyData: true
RoxygenNote: 7.1.1
Collate: 
    'add2rtoi.R'
    'api.R'
    'extent_crs.R'
    'records.R'
    'rtoi.R'
    'cloud_mask.R'
    'connections.R'
    'data.R'
    'derive.R'
    'download.R'
    'list_data.R'
    'mosaic.R'
    'mosaic_fun_SYN.R'
    'mosaic_fun_ls.R'
    'mosaic_fun_mod.R'
    'mosaic_fun_sen2.R'
    'mosaic_generic.R'
    'package_tools.R'
    'plot.R'
    'preview.R'
    'search_mod.R'
    'search_ls.R'
    'search_sen.R'
    'search.R'
    'smoothing_images.R'
    'variables.R'
VignetteBuilder: knitr

Scope

• Please indicate which category or categories from our package fit policies this package falls under: (Please check an appropriate box below. If you are unsure, we suggest you make a pre-submission inquiry.):

  • data retrieval
  • data extraction
  • data munging
  • data deposition
  • workflow automation
  • version control
  • citation management and bibliometrics
  • scientific software wrappers
  • field and lab reproducibility tools
  • database software bindings
  • geospatial data
  • text analysis

• Explain how and why the package falls under these categories (briefly, 1-2 sentences):
  The package is focuses on searching, downloading, and preprocessing imagery data from Landsat, Modis, and Sentinel. It also include procedures for deriving variables and cloud filling.

• Who is the target audience and what are scientific applications of this package?
  Anyone interested in Remote Sensing or researchers looking for satellite imagery data.

• Are there other R packages that accomplish the same thing? If so, how does yours differ or meet our criteria for best-in-category?
  Nothing that is functionally similar. Many source-specific packages exist, but none that aggregate across sources.

The most similar package could be MODIStsp. But the package only contemplates the use of Modis satellite images, while 'rsat' is focuses on the standardization and homogenization of data between different satellite programs. 'rsat' supports Modis, Landsat and Sentinel data, handling multi platform data in a database and optimizing its processing.

• (If applicable) Does your package comply with our guidance around Ethics, Data Privacy and Human Subjects Research?

• If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.

Technical checks

Confirm each of the following by checking the box.

• I have read the guide for authors and rOpenSci packaging guide.

This package:

• does not violate the Terms of Service of any service it interacts with.
• has a CRAN and OSI accepted license.
• contains a README with instructions for installing the development version.
• includes documentation with examples for all functions, created with roxygen2.
• contains a vignette with examples of its essential functions and uses.
• has a test suite.
• has continuous integration, including reporting of test coverage using services such as Travis CI, Coveralls and/or CodeCov.

Publication options

• Do you intend for this package to go on CRAN?

• Do you intend for this package to go on Bioconductor?

• Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:

MEE Options

• The package is novel and will be of interest to the broad readership of the journal.
• The manuscript describing the package is no longer than 3000 words.
• You intend to archive the code for the package in a long-term repository which meets the requirements of the journal (see MEE's Policy on Publishing Code)
• (Scope: Do consider MEE's Aims and Scope for your manuscript. We make no guarantee that your manuscript will be within MEE scope.)
• (Although not required, we strongly recommend having a full manuscript prepared when you submit here.)
• (Please do not submit your package separately to Methods in Ecology and Evolution)

Code of conduct

• I agree to abide by rOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

[noamross]

@ropensci-review-bot assign @jhollist as editor

[ropensci-review-bot]

Assigned! @jhollist is now the editor

[jhollist]

@noamross Thanks for setting me up!

@unai-perez Just wanted to let you know that I am new to this editor role. So bare with me as I figure things out. Also, my next week is pretty booked. I'll start digging in where I can, but it'll be first of April before I will be able to make real progress on coordinating the review.

Any questions, let me know.

[unai-perez]

Thank for the heads-up @jhollist. Looking forward to receiving your comments and the comments from the reviewers. We will both learn along the way.

[jhollist]

@unai-perez Thanks for your patience. I finally was able to get some time to run through this (and learn what needed to be done!). See below for my comments. Let me know if you have questions.

Editor checks:

• Documentation: The package has sufficient documentation available online (README, pkgdown docs) to allow for an assessment of functionality and scope without installing the package.
• Fit: The package meets criteria for fit and overlap
• Automated tests: Package has a testing suite and is tested via a CI service.
• License: The package has a CRAN or OSI accepted license
• Repository: The repository link resolves correctly

---

Editor comments

This looks to be very useful package and builds nicely on existing functionality. It fits well with the ROpenSci criteria. Installation proceeds as expected from both local and GitHub sources. During my checks I did find a number of items that need to be addressed before we can move along with the review process. Pay close attention to https://devguide.ropensci.org/building.html as you revise your submission but see also my comments below for a detailed list.

If you have any questions, let me know. Items marked with a * will need to be addressed prior to assigning reviewers. Items without a * may still be raised as part of the review process, but the review can proceed prior to addressing those.

Lastly, I do want to reiterate that I think this will be a very useful package. Having a one stop shop for gathering and preparing satellite data is needed and kudos for taking that on!

Automated Tests

• *Expand CI to include Windows, Linux, and Mac and using the latest, previous and development versions of R. You are currently using appveyor which is using the latest version and Windows only. You may want to look at GitHub Actions which is supported via usethis::use_github_action_check_standard(). This will test on all the needed platforms and releases.

Masked function names

• I did notice a few masked function names (e.g. rsat::subset). I agree that masking is sometimes hard to avoid, but it might be worthwhile to rename is possible. It is likely that reviewers might catch this as well.

Checks and Tests

Tests (e.g. devtools::check() and devtools::test()) are currently failing on my machine, plus I am seeing several warnings.

• *Check warnings on tests. Some might be acceptable but others may need
  correction. I see:

-- Warning (test-search.R:17:5): search test -----------------------------------
Error in Earth Explorer api connection. HTTP 404.
Backtrace:
 1. rsat::sat_search(...) test-search.R:17:4
 2. rsat::sat_search(...)
 3. rsat:::.local(region, product, ...)
 5. rsat:::ls_search(region(region), product = s, ...)
 6. rsat:::.local(region, product, ...)
-- Warning (test-search.R:17:5): search test -----------------------------------
Error in Earth Explorer api connection. HTTP 404.
Backtrace:
 1. rsat::sat_search(...) test-search.R:17:4
 2. rsat::sat_search(...)
 3. rsat:::.local(region, product, ...)
 5. rsat:::ls_search(region(region), product = s, ...)
 6. rsat:::.local(region, product, ...)
-- Warning (test-search.R:17:5): search test -----------------------------------
Error in Earth Explorer api connection. HTTP 404.
Backtrace:
 1. rsat::sat_search(...) test-search.R:17:4
 2. rsat::sat_search(...)
 3. rsat:::.local(region, product, ...)
 5. rsat:::ls_search(region(region), product = s, ...)
 6. rsat:::.local(region, product, ...)
-- Warning (test-search.R:17:5): search test -----------------------------------
Error in Earth Explorer api connection. HTTP 404.
Backtrace:
 1. rsat::sat_search(...) test-search.R:17:4
 2. rsat::sat_search(...)
 3. rsat:::.local(region, product, ...)
 5. rsat:::ls_search(region(region), product = s, ...)
 6. rsat:::.local(region, product, ...)
-- Warning (test-search.R:17:5): search test -----------------------------------
Error in Earth Explorer api connection. HTTP 404.
Backtrace:
 1. rsat::sat_search(...) test-search.R:17:4
 2. rsat::sat_search(...)
 3. rsat:::.local(region, product, ...)
 5. rsat:::ls_search(region(region), product = s, ...)
 6. rsat:::.local(region, product, ...)
-- Warning (test-search.R:17:5): search test -----------------------------------
Error in Earth Explorer api connection. HTTP 404.
Backtrace:
 1. rsat::sat_search(...) test-search.R:17:4
 2. rsat::sat_search(...)
 3. rsat:::.local(region, product, ...)
 5. rsat:::ls_search(region(region), product = s, ...)
 6. rsat:::.local(region, product, ...)
-- Warning (test-search.R:17:5): search test -----------------------------------
Cannot perform Landsat search, check your credentials and/or the api status: https://m2m.cr.usgs.gov/api/docs/json/
Backtrace:
 1. rsat::sat_search(...) test-search.R:17:4
 2. rsat::sat_search(...)
 3. rsat:::.local(region, product, ...)
 5. rsat:::ls_search(region(region), product = s, ...)
 6. rsat:::.local(region, product, ...)
-- Warning (test-search.R:35:5): search test -----------------------------------
GDAL Error 1: PROJ: proj_create_from_database: crs not found
Backtrace:
 1. base::plot(c(rcds1, rcds2, rcds3)) test-search.R:35:4
 2. rsat::plot(c(rcds1, rcds2, rcds3))
 3. rsat:::.local(x, y, ...)
 5. sf:::st_crs.numeric(crs(r))
 6. sf:::make_crs(paste0("EPSG:", x))
 7. sf:::CPL_crs_from_input(x)

• *Make sure tests pass. I see:

== Failed tests ================================================================
-- Error (test-search.R:35:5): search test -------------------------------------
Error: attempt to select less than one element in get1index
Backtrace:
    x
 1. +-base::plot(c(rcds1, rcds2, rcds3)) test-search.R:35:4
 2. \-rsat::plot(c(rcds1, rcds2, rcds3))
 3.   \-rsat:::.local(x, y, ...)
 4.     \-connection$getApi(get_api_name(r))

Goodpractice

Many issues are raised with goodpractice::gp(). Pay close attention to the output and fix these issues. In particular:

• Consider simplifying genPlotGIS

  ✖ write short and simple functions. These functions have high
    cyclomatic complexity:genPlotGIS (60).

• *You currently have sf, raster, and stars as Depends. Those can all be moved to Imports. You will need to make sure to import those (e.g. with @importFrom) where you use them in your functions.

  ✖ not use "Depends" in DESCRIPTION, as it can cause name
    clashes, and poor interaction with other packages. Use "Imports"
    instead.

• *Add a URL field. Prior to acceptance as an ROpenSci package this would be https://github.com/spatialstatisticsupna/rsat.

  ✖ add a "URL" field to DESCRIPTION. It helps users find
    information about your package online. If your package does not have a
    homepage, add an URL to GitHub, or the CRAN package package page.

• *Add BugReports field. Prior to acceptance as an ROpenSci package this would be https://github.com/spatialstatisticsupna/rsat/issues.

  ✖ add a "BugReports" field to DESCRIPTION, and point it to a bug
    tracker. Many online code hosting services provide bug trackers for
    free, https://github.com, https://gitlab.com, etc.

• *It appears you are using both '=' and '<-' for assignment. The ROpenSci style guide allows either, but you need to be consistent. Please choose one and use it throughout your code.

  ✖ use '<-' for assignment instead of '='. '<-' is the standard,
    and R users and developers are used it and it is easier to read your
    code for them if you use '<-'.

    R\api.R:21:21
    R\api.R:22:21
    R\api.R:23:20
    R\api.R:24:20
    R\api.R:25:24
    ... and 102 more lines

• Work on getting your long lines shortened. The styler package will help with some of these, but not all (e.g. Enforcing a 80-character line length rule? r-lib/styler#247). You will have to manually edit many of these. You can use lintr::lint(path/to/file.R) to help find these. Also, you can turn on an 80 character guide in RStudio with Tools:Global Options:Code:Display and select Show Margin and set characters to 80. This will help visually identify these lines.

  ✖ avoid long code lines, it is bad for readability. Also, many
    people prefer editor windows that are about 80 characters wide. Try make
    your lines shorter than 80 characters

    R\api.R:6:1
    R\api.R:118:1
    R\api.R:120:1
    R\api.R:133:1
    R\api.R:150:1
    ... and 321 more lines

• *Omit any trailing semicolons.

  ✖ omit trailing semicolons from code lines. They are not needed
    and most R coding standards forbid them

    R\mosaic.R:166:92
    R\preview.R:217:40

• *Replace sapply() with vapply()

  ✖ avoid sapply(), it is not type safe. It might return a vector,
    or a list, depending on the input data. Consider using vapply() instead.

    R\package_tools.R:40:10
    R\search_ls.R:107:17

• *When building a sequence use seq_len() or seq_along()

  ✖ avoid 1:length(...), 1:nrow(...), 1:ncol(...), 1:NROW(...) and
    1:NCOL(...) expressions. They are error prone and result 1:0 if the
    expression on the right hand side is zero. Use seq_len() or seq_along()
    instead.

    R\download.R:74:22
    R\download.R:107:24
    R\download.R:125:24
    R\mosaic_generic.R:61:103
    R\package_tools.R:42:12
    ... and 9 more lines

• *Refine importing of functions. Instead of @import, use @importFrom package function_name

  ✖ do not import packages as a whole, as this can cause name clashes
    between the imported packages. Instead, import only the specific
    functions you need.

• *Same testing error as mentioned above.

  ✖ checking tests ... Running 'testthat.R' ERROR Running the
    tests in 'tests/testthat.R' failed. Last 13 lines of output: Data and
    variable methods provided by rsat Satellite products: ls1, ls2, ls3,
    ls4, ls5, ls7, ls8, mod09ga, myd09ga, mcd43a4, Sentinel-1, Sentinel-2,
    Sentinel-3, SY_2_SYN___. Variable Methods: EVI, MSAVI2, NBR, NBR2, NDMI,
    NDVI, NDWI, RGB, SAVI.== Failed tests
    ================================================================ --
    Error (test-search.R:35:5): search test
    ------------------------------------- Error: attempt to select less than
    one element in get1index Backtrace: x 1. +-base::plot(c(rcds1, rcds2,
    rcds3)) test-search.R:35:4 2. \-rsat::plot(c(rcds1, rcds2, rcds3)) 3.
    \-rsat:::.local(x, y, ...) 4.  \-connection$getApi(get_api_name(r)) [
    FAIL 1 | WARN 8 | SKIP 0 | PASS 0 ] Error: Test failures Execution
    halted

• I only see this one with goodpractice::gp(), not when I run check on its own. You can probably ignore this one.

  ✖ fix this R CMD check WARNING: LaTeX errors when creating PDF
    version. This typically indicates Rd problems. LaTeX errors found:

• *Switch T and F to TRUE and FALSE

  ✖ avoid 'T' and 'F', as they are just variables which are set to
    the logicals 'TRUE' and 'FALSE' by default, but are not reserved words
    and hence can be overwritten by the user.  Hence, one should always use
    'TRUE' and 'FALSE' for the logicals.

    R/api.R:NA:NA
    R/cloud_mask.R:NA:NA
    R/cloud_mask.R:NA:NA
    R/derive.R:NA:NA
    R/derive.R:NA:NA
    ... and 19 more lines

Spelling

• Several items are flagged with devtools::spell_check() but most appear to be names, acronyms, etc. and are probably not really misspellings. Please double check the items returned by devtools::spell_check() to make sure any true misspellings are corrected.

Tests

• *Expand tests: Since I had the one test fail locally, I wasn't able to confirm coverage. On your README, you report 54% and it appears that many of the exported functions in your package are not covered by your current tests. I would like to see the testing expanded to 70% or greater. I do realize that testing with API packages can be a challenge sometimes, so if reaching 70% is a problem you will need to provide a justification. Additionally, if authentication or API keys are required you need to include instructions in your documentation (e.g. in README). In short, if there are any prerequisites for your tests to pass, makes sure users and reviewers can easily set those up on their local machines.

---

[unai-perez]

@jhollist thanks for your comments.

I think we can deal with most of the changes in a short period of time. I think the biggest change is the testing. We will try to get to the 70% that you mention.

I was wondering if I could send the current version of the package to CRAN. It would be great for us to report that for a project. However, we are concerned about the change of the name of the package during the ropensci revision. Therefore, we would like to receive your advice on this regard.

[jhollist]

Sounds good. Let me get some other opinions on the pre-review CRAN submission. Stay tuned.

…

On Fri, Apr 9, 2021 at 1:05 PM Unai Pérez-Goya ***@***.***> wrote: @jhollist <https://github.com/jhollist> thanks for your comments. I think we can deal with most of the changes in a short period of time. I think the biggest change is the testing. We will try to get to the 70% that you mention. I was wondering if I could send the current version of the package to CRAN. It would be great for us to report that for a project. However, we are concerned about the change of the name of the package during the ropensci revision. Therefore, we would like to receive your advice on this regard. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#437 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJPYSYZCZK7SVJO4ILYSRLTH4XWRANCNFSM4ZSADESA> .

-- Jeffrey W. Hollister email: ***@***.*** cell: 401 556 4087 https://jwhollister.com

[jhollist]

@unai-perez, the decision to submit to CRAN is ultimately going to be up to you and your co-authors. That being said the ROpenSci recommendation is to to wait until after review because of the possibility of changes to your package's interface and also because the reviewers will often find problems that could complicate the CRAN submission process.

I would like to encourage you to wait a bit on the submission, if you can. I already have a few reviewers in mind and will do my best to speed that part of the process along.

As I said, the decision is yours to make, but I think waiting until after the reviews come in will be an easier path in the long run.

[unai-perez]

@jhollist thanks for your comments. We decided not to send the package to CRAN until the review process is completed.
We covered most of the requirements. Find below the line by line answers.

Automated Tests

* [x]  `*`Expand CI to include Windows, Linux, and Mac and using the latest, previous and development versions of R.  You are currently using appveyor which is using the latest version and Windows only.  You may want to look at GitHub Actions which is supported via `usethis::use_github_action_check_standard()`.  This will test on all the needed platforms and releases.

Done, we integrated the package in GitHub Actions. Additionally, we change the minimum R version supported by the package to 4.0.0. Packages we depend on have experienced major changes since the 4.0.0. R version, which means that rsat is no longer backward compatible for earlier versions.

Masked function names

* [ ]  I did notice a few masked function names (e.g. `rsat::subset`).  I agree that masking is sometimes hard to avoid, but it might be worthwhile to rename is possible.  It is likely that reviewers might catch this as well.

These masked functions are developed to add functionalities to the new classes created by the package. For example, rsat::subset gets a subset from records, that works as vector.
We created new methods for these functions, so they accept rtois or records as inputs. When used with other classes, their original functionalities remain the same. We think that this strategy benefits R users as they do not need to remember new names specific for rtois/records.

Checks and Tests

Tests (e.g. devtools::check() and devtools::test()) are currently failing on my machine, plus I am seeing several warnings.

* [x]  `*`Check warnings on tests.  Some might be acceptable but others may need
  correction.  I see:

Done. Now only a few warnings related to modis image projections are displayed

-- Warning (test-search.R:17:5): search test -----------------------------------
Error in Earth Explorer api connection. HTTP 404.
Backtrace:
 1. rsat::sat_search(...) test-search.R:17:4
 2. rsat::sat_search(...)
 3. rsat:::.local(region, product, ...)
 5. rsat:::ls_search(region(region), product = s, ...)
 6. rsat:::.local(region, product, ...)
-- Warning (test-search.R:17:5): search test -----------------------------------
Error in Earth Explorer api connection. HTTP 404.
Backtrace:
 1. rsat::sat_search(...) test-search.R:17:4
 2. rsat::sat_search(...)
 3. rsat:::.local(region, product, ...)
 5. rsat:::ls_search(region(region), product = s, ...)
 6. rsat:::.local(region, product, ...)
-- Warning (test-search.R:17:5): search test -----------------------------------
Error in Earth Explorer api connection. HTTP 404.
Backtrace:
 1. rsat::sat_search(...) test-search.R:17:4
 2. rsat::sat_search(...)
 3. rsat:::.local(region, product, ...)
 5. rsat:::ls_search(region(region), product = s, ...)
 6. rsat:::.local(region, product, ...)
-- Warning (test-search.R:17:5): search test -----------------------------------
Error in Earth Explorer api connection. HTTP 404.
Backtrace:
 1. rsat::sat_search(...) test-search.R:17:4
 2. rsat::sat_search(...)
 3. rsat:::.local(region, product, ...)
 5. rsat:::ls_search(region(region), product = s, ...)
 6. rsat:::.local(region, product, ...)
-- Warning (test-search.R:17:5): search test -----------------------------------
Error in Earth Explorer api connection. HTTP 404.
Backtrace:
 1. rsat::sat_search(...) test-search.R:17:4
 2. rsat::sat_search(...)
 3. rsat:::.local(region, product, ...)
 5. rsat:::ls_search(region(region), product = s, ...)
 6. rsat:::.local(region, product, ...)
-- Warning (test-search.R:17:5): search test -----------------------------------
Error in Earth Explorer api connection. HTTP 404.
Backtrace:
 1. rsat::sat_search(...) test-search.R:17:4
 2. rsat::sat_search(...)
 3. rsat:::.local(region, product, ...)
 5. rsat:::ls_search(region(region), product = s, ...)
 6. rsat:::.local(region, product, ...)
-- Warning (test-search.R:17:5): search test -----------------------------------
Cannot perform Landsat search, check your credentials and/or the api status: https://m2m.cr.usgs.gov/api/docs/json/
Backtrace:
 1. rsat::sat_search(...) test-search.R:17:4
 2. rsat::sat_search(...)
 3. rsat:::.local(region, product, ...)
 5. rsat:::ls_search(region(region), product = s, ...)
 6. rsat:::.local(region, product, ...)
-- Warning (test-search.R:35:5): search test -----------------------------------
GDAL Error 1: PROJ: proj_create_from_database: crs not found
Backtrace:
 1. base::plot(c(rcds1, rcds2, rcds3)) test-search.R:35:4
 2. rsat::plot(c(rcds1, rcds2, rcds3))
 3. rsat:::.local(x, y, ...)
 5. sf:::st_crs.numeric(crs(r))
 6. sf:::make_crs(paste0("EPSG:", x))
 7. sf:::CPL_crs_from_input(x)

* [x]  `*`Make sure tests pass.  I see:

Done.

== Failed tests ================================================================
-- Error (test-search.R:35:5): search test -------------------------------------
Error: attempt to select less than one element in get1index
Backtrace:
    x
 1. +-base::plot(c(rcds1, rcds2, rcds3)) test-search.R:35:4
 2. \-rsat::plot(c(rcds1, rcds2, rcds3))
 3.   \-rsat:::.local(x, y, ...)
 4.     \-connection$getApi(get_api_name(r))

Goodpractice

Many issues are raised with goodpractice::gp(). Pay close attention to the output and fix these issues. In particular:

* [x]  Consider simplifying `genPlotGIS`

Done. The function was divided into smaller functions.

  ✖ write short and simple functions. These functions have high
    cyclomatic complexity:genPlotGIS (60).

* [x]  `*`You currently have sf, raster, and stars as Depends.  Those can all be moved to Imports.  You will need to make sure to import those (e.g. with @importFrom) where you use them in your functions.

Done. All the packages as Depends were passed to imports. Now the package only uses importFrom.

  ✖ not use "Depends" in DESCRIPTION, as it can cause name
    clashes, and poor interaction with other packages. Use "Imports"
    instead.

* [x]  `*`Add a URL field.  Prior to acceptance as an ROpenSci package this would be https://github.com/spatialstatisticsupna/rsat.

Done.

  ✖ add a "URL" field to DESCRIPTION. It helps users find
    information about your package online. If your package does not have a
    homepage, add an URL to GitHub, or the CRAN package package page.

* [x]  `*`Add BugReports field.  Prior to acceptance as an ROpenSci package this would be https://github.com/spatialstatisticsupna/rsat/issues.

Done.

  ✖ add a "BugReports" field to DESCRIPTION, and point it to a bug
    tracker. Many online code hosting services provide bug trackers for
    free, https://github.com, https://gitlab.com, etc.

* [x]  `*`It appears you are using both '=' and '<-' for assignment.  The ROpenSci style guide allows either, but you need to be consistent.  Please choose one and use it throughout your code.

Done.

  ✖ use '<-' for assignment instead of '='. '<-' is the standard,
    and R users and developers are used it and it is easier to read your
    code for them if you use '<-'.

    R\api.R:21:21
    R\api.R:22:21
    R\api.R:23:20
    R\api.R:24:20
    R\api.R:25:24
    ... and 102 more lines

* [x]  Work on getting your long lines shortened.  The `styler` package will help with some of these, but not all (e.g. [r-lib/styler#247](https://github.com/r-lib/styler/issues/247)).  You will have to manually edit many of these.  You can use `lintr::lint(path/to/file.R)` to help find these.  Also, you can turn on an 80 character guide in RStudio with Tools:Global Options:Code:Display and select Show Margin and set characters to 80.  This will help visually identify these lines.

Done. There are some lines with more than 80 characters, but these are URLs that it is not possible to cut.

  ✖ avoid long code lines, it is bad for readability. Also, many
    people prefer editor windows that are about 80 characters wide. Try make
    your lines shorter than 80 characters

    R\api.R:6:1
    R\api.R:118:1
    R\api.R:120:1
    R\api.R:133:1
    R\api.R:150:1
    ... and 321 more lines

* [x]  `*`Omit any trailing semicolons.

Done. We fixed this.

  ✖ omit trailing semicolons from code lines. They are not needed
    and most R coding standards forbid them

    R\mosaic.R:166:92
    R\preview.R:217:40

* [x]  `*`Replace `sapply()` with `vapply()`

Done. Now the package uses vapply.

  ✖ avoid sapply(), it is not type safe. It might return a vector,
    or a list, depending on the input data. Consider using vapply() instead.

    R\package_tools.R:40:10
    R\search_ls.R:107:17

* [x]  `*`When building a sequence use `seq_len()` or `seq_along()`

Done. Now the package uses seq_len or seq_along.

  ✖ avoid 1:length(...), 1:nrow(...), 1:ncol(...), 1:NROW(...) and
    1:NCOL(...) expressions. They are error prone and result 1:0 if the
    expression on the right hand side is zero. Use seq_len() or seq_along()
    instead.

    R\download.R:74:22
    R\download.R:107:24
    R\download.R:125:24
    R\mosaic_generic.R:61:103
    R\package_tools.R:42:12
    ... and 9 more lines

* [x]  `*`Refine importing of functions.  Instead of @import, use @importFrom package function_name

Done, now we only use importFrom.

  ✖ do not import packages as a whole, as this can cause name clashes
    between the imported packages. Instead, import only the specific
    functions you need.

* [x]  `*`Same testing error as mentioned above.

Fixed.

  ✖ checking tests ... Running 'testthat.R' ERROR Running the
    tests in 'tests/testthat.R' failed. Last 13 lines of output: Data and
    variable methods provided by rsat Satellite products: ls1, ls2, ls3,
    ls4, ls5, ls7, ls8, mod09ga, myd09ga, mcd43a4, Sentinel-1, Sentinel-2,
    Sentinel-3, SY_2_SYN___. Variable Methods: EVI, MSAVI2, NBR, NBR2, NDMI,
    NDVI, NDWI, RGB, SAVI.== Failed tests
    ================================================================ --
    Error (test-search.R:35:5): search test
    ------------------------------------- Error: attempt to select less than
    one element in get1index Backtrace: x 1. +-base::plot(c(rcds1, rcds2,
    rcds3)) test-search.R:35:4 2. \-rsat::plot(c(rcds1, rcds2, rcds3)) 3.
    \-rsat:::.local(x, y, ...) 4.  \-connection$getApi(get_api_name(r)) [
    FAIL 1 | WARN 8 | SKIP 0 | PASS 0 ] Error: Test failures Execution
    halted

* [x]  I only see this one with `goodpractice::gp()`, not when I run check on its own.  You can probably ignore this one.

Running goodpractice::gc(), there is no warning anymore.

  ✖ fix this R CMD check WARNING: LaTeX errors when creating PDF
    version. This typically indicates Rd problems. LaTeX errors found:

* [x]  `*`Switch T and F to TRUE and FALSE

Fixed! We changed all T and F as TRUE and FALSE.

  ✖ avoid 'T' and 'F', as they are just variables which are set to
    the logicals 'TRUE' and 'FALSE' by default, but are not reserved words
    and hence can be overwritten by the user.  Hence, one should always use
    'TRUE' and 'FALSE' for the logicals.

    R/api.R:NA:NA
    R/cloud_mask.R:NA:NA
    R/cloud_mask.R:NA:NA
    R/derive.R:NA:NA
    R/derive.R:NA:NA
    ... and 19 more lines

Spelling

* [x]  Several items are flagged with `devtools::spell_check()` but most appear to be names, acronyms, etc. and are probably not really misspellings.  Please double check the items returned by `devtools::spell_check()` to make sure any true misspellings are corrected.

We check all the misspellings

Tests

* [x]  `*`Expand tests: Since I had the one test fail locally, I wasn't able to confirm coverage.  On your README, you report 54% and it appears that many of the exported functions in your package are not covered by your current tests.  I would like to see the testing expanded to 70% or greater.  I do realize that testing with API packages can be a challenge sometimes, so if reaching 70% is a problem you will need to provide a justification.  Additionally, if authentication or API keys are required you need to include instructions in your documentation (e.g. in README).  In short, if there are any prerequisites for your tests to pass, makes sure users and reviewers can easily set those up on their local machines.

Testing APIs and large datasets is challenging and time consuming. We propose an alternative solution to meet this requirement while keeping it simple and computationally fast. We added a new argument called test.mode in some functions. This argument allows to run a few more lines of code to speed-up the testing process and avoid very specific errors that very difficult to anticipate (e.g., omitting calls to the API and replacing them with calls to GitHub mimicking the API response). In addition, we added a new function to test internal methods of the package, which are usually omitted by the test, such as functions designed to order images, etc... With these changes we have managed to exceed the 70% of lines executed in codecov.

[maelle]

@ropensci-review-bot assign @jhollist as editor

[ropensci-review-bot]

Assigned! @jhollist is now the editor

[jhollist]

@unai-perez Thanks for addressing these issues. I think we are very close.

I have one last thing that needs to be addressed prior to starting review. The tests, as currently constructed, don't finish on my machine. I dug into it a little and it looks like lines 59-70 in test-download.r (https://github.com/spatialstatisticsupna/rsat/blob/381d12de6d974eee0e99e0b41f7fd2931c0b6c15/tests/testthat/test-download.R#L59-L70) are the source of the problem. If I comment those out and re-run the tests everything completes in about 20 minutes on my machine. These do appear to run on CI (at least GitHub Actions didn't throw any errors) so that is a little weird. Full disclosure, I didn't spend too much time trying find the underlying issue.

I would like you and your co-authors to do the following:

1. See if you can recreate this issue with the most recent version on https://github.com/spatialstatisticsupna/rsat. I cloned this version locally and the ran devtools::check("rsat"). It ran overnight before I stopped it. I think if you let it run an hour or so and it hasn't completed then that indicates an issue with that code.
2. If you recreate it, please try to find the issue and correct it. If the tests run fine for you, then I am happy to move on with the review process.

Any questions or concerns about this, just let me know.

[unai-perez]

Dear @jhollist,
After testing lines 59-70, we find a problem. In these lines we download images from the scihub API, but it seems that since last Wednesday they are migrating the scihub image server.
https://scihub.copernicus.eu/news/News00865
https://scihub.copernicus.eu/news/News00866

There is an error with those images at the moment and the API shows that they are not available. The package interprets that images will be available soon and waits for the images to be ready. However, this is not the case and the process gets stuck.

We have decided to remove those lines from the test and keep only the lines in test.mode=TRUE.