Merge branch 'main' into update-search-user-guide

Author: chuckwondo

.github/workflows/test-mindeps.yml

@@ -24,7 +24,7 @@ jobs:
         uses: actions/[email protected]
 
       - name: Install uv
-        uses: astral-sh/setup-uv@v6.0.1
+        uses: astral-sh/setup-uv@v6.1.0
         with:
           enable-cache: true
 

.pre-commit-config.yaml

@@ -10,6 +10,7 @@ repos:
       # Validate format
       - id: check-yaml
         args: ["--unsafe"]
+        exclude: ".*/vcr_cassettes/.*\\.yaml"
       - id: check-toml
       - id: check-json
       # Check for common mistakes

CHANGELOG.md

@@ -7,58 +7,69 @@ and this project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
 
 ## [Unreleased]
 
+### Changed
+
+- By default, _disable_ progress bars during downloading.
+  ([#612](https://github.com/nsidc/earthaccess/issues/612))
+  ([@Sherwin-14](https://github.com/Sherwin-14))
+- Updated bug and triage label names in bug Issue template.
+  ([#998](https://github.com/nsidc/earthaccess/pull/998))
+  ([@asteiker](https://github.com/asteiker))
+- `download` now raises `DownloadFailure` exception on failure.
+  ([#612](https://github.com/nsidc/earthaccess/issues/612))
+  ([@Sherwin-14](https://github.com/Sherwin-14))
+- `GESDISC` should be `GES_DISC` in docstrings.
+  ([#1037](https://github.com/nsidc/earthaccess/issues/1037))
+  ([@abarciauskas-bgse](https://github.com/abarciauskas-bgse))
 
 ### Added
 
+- Add notebook demonstrating workflow with TEMPO Level 3 data as a virtual dataset
+  ([#924](https://github.com/nsidc/earthaccess/pull/924))
+  ([@danielfromearth](https://github.com/danielfromearth))
+- `get_s3_filesystem` now accepts an `endpoint` argument for specifying a credentials url.
+  ([#602](https://github.com/nsidc/earthaccess/issues/602))
+  ([@rwegener2](https://github.com/rwegener2))
+- s3 `download` now checks for existing files.
+  ([#807](https://github.com/nsidc/earthaccess/issues/807))
+  ([@Sherwin-14](https://github.com/Sherwin-14))
 - Added triaging guide ([#754](https://github.com/nsidc/earthaccess/issues/754))
   ([@Sherwin-14](https://github.com/Sherwin-14))
   ([@mfisher87](https://github.com/mfisher87))
+- `download` now returns Path consistently.
+  ([#595])(https://github.com/nsidc/earthaccess/issues/595)
+  ([@Sherwin-14](https://github.com/Sherwin-14))
+- Users may now authenticate with an existing Earthdata login token with
+  environment variable `EARTHDATA_TOKEN`
+  ([#484](https://github.com/nsidc/earthaccess/issues/484))
+  ([@kgrimes2](https://github.com/kgrimes2))
 
 ### Removed
 
+- **Breaking:** Removed `has_granules=true` and `include_granule_counts=true`
+  as default parameters upon creation of a `DataCollections` instance.
+  ([#884](https://github.com/nsidc/earthaccess/issues/884))
+  ([@Sherwin-14](https://github.com/Sherwin-14))
 - Python 3.10 is no longer supported.
   ([#966](https://github.com/nsidc/earthaccess/pull/966))
   ([@weiji14](https://github.com/weiji14))
 
-### Added
-- Add notebook demonstrating workflow with TEMPO Level 3 data as a virtual dataset ([#924](https://github.com/nsidc/earthaccess/pull/924))
-([@danielfromearth](https://github.com/danielfromearth))
-- `get_s3_filesystem` now accepts an `endpoint` argument for specifying a credentials url.
-([#602](https://github.com/nsidc/earthaccess/issues/602))
-([@rwegener2](https://github.com/rwegener2))
-
-- s3 `download` now checks for existing files.
-([#807](https://github.com/nsidc/earthaccess/issues/807))
-([@Sherwin-14](https://github.com/Sherwin-14))
-
-
-### Changed
-
-- By default, _disable_ progress bars during downloading.
-  ([#612](https://github.com/nsidc/earthaccess/issues/612))
-  ([@Sherwin-14](https://github.com/Sherwin-14))
-
-- Updated bug and triage label names in bug Issue template.
-  ([#998](https://github.com/nsidc/earthaccess/pull/998))
-  ([@asteiker](https://github.com/asteiker))
-
-- `download` now raises `DownloadFailure` exception on failure.
-  ([#612](https://github.com/nsidc/earthaccess/issues/612))
-  ([@Sherwin-14](https://github.com/Sherwin-14))
-
 ### Fixed
+
 - Corrected Harmony typo in notebooks/Demo.ipynb([#995](https://github.com/nsidc/earthaccess/issues/995))([stelios-c](https://github.com/stelios-c))
 
 ## [v0.14.0] - 2025-02-11
 
 ### Added
+
 - `search_datasets` now accepts a `has_granules` keyword argument. Use
   `has_granules=False` to search for metadata about collections with no
   associated granules. The default value set in `DataCollections` remains `True`.
   ([#939](https://github.com/nsidc/earthaccess/issues/939))
   ([**@juliacollins**](https://github.com/juliacollins))
 
 ### Changed
+
 - **Breaking**: earthaccess will now raise an exception when login credentials are
   rejected.  If you need the old behavior, please use a `try` block.
   ([#946](https://github.com/nsidc/earthaccess/pull/946))
@@ -68,6 +79,7 @@ and this project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
 ## [v0.13.0] - 2025-01-28
 
 ### Added
+
 - VirtualiZarr: earthaccess can open archival formats (NetCDF, HDF5) as if they were Zarr by leveraging VirtualiZarr
   In order to use this capability the collection needs to be supported by OPeNDAP and have dmrpp files.
   See [example notebooks](https://github.com/nsidc/earthaccess/blob/main/docs/tutorials/dmrpp-virtualizarr.ipynb)!
@@ -83,7 +95,6 @@ and this project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
   ([#913](https://github.com/nsidc/earthaccess/issues/913))
   ([**@hailiangzhang**](https://github.com/hailiangzhang))
 
-
 ## [v0.12.0] - 2024-11-13
 
 ### Changed

docs/howto/authenticate.md

@@ -31,6 +31,9 @@ use:
 auth = earthaccess.login(strategy="environment")
 ```
 
+Alternatively, you can use an existing Earthdata Login token by setting the environment
+variable `EARTHDATA_TOKEN` to it and using the same "environment" strategy, above.
+
 If you wish to enter your Earthdata Login credentials when prompted, with
 optional persistence to your `.netrc` file (see below), specify the interactive
 strategy:
@@ -54,7 +57,8 @@ credentials in two locations:
    used throughout documentation primarily for convenience.  The only
    requirement is that the *contents* of the file adhere to the
    [`.netrc` file format](https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html).
-2. `EARTHDATA_USERNAME` and `EARTHDATA_PASSWORD` environment variables
+2. `EARTHDATA_USERNAME` and `EARTHDATA_PASSWORD` environment variables (or, optionally, `EARTHDATA_TOKEN`
+   to use an existing Earthdata Login token)
 
 If neither of these options are configured, you can authenticate by calling the
 `earthaccess.login()` method and manually entering your EDL account credentials.

earthaccess/api.py

@@ -184,7 +184,9 @@ def login(
             * **"all"**: (default) try all methods until one works
             * **"interactive"**: enter username and password.
             * **"netrc"**: retrieve username and password from ~/.netrc.
-            * **"environment"**: retrieve username and password from `$EARTHDATA_USERNAME` and `$EARTHDATA_PASSWORD`.
+            * **"environment"**: retrieve either a username and password pair from
+              the `EARTHDATA_USERNAME` and `EARTHDATA_PASSWORD` environment variables,
+              or an Earthdata login token from the `EARTHDATA_TOKEN` environment variable.
         persist: will persist credentials in a .netrc file
         system: the Earthdata system to access, defaults to PROD
 
@@ -233,7 +235,7 @@ def download(
     threads: int = 8,
     *,
     pqdm_kwargs: Optional[Mapping[str, Any]] = None,
-) -> List[str]:
+) -> List[Path]:
     """Retrieves data granules from a remote storage system. Provide the optional `local_path` argument to prevent repeated downloads.
 
        * If we run this in the cloud, we will be using S3 to move data to `local_path`.

earthaccess/auth.py

@@ -110,8 +110,11 @@ def login(
                 * **"interactive"**: Enter a username and password.
                 * **"netrc"**: (default) Retrieve a username and password from ~/.netrc.
                 * **"environment"**:
-                    Retrieve a username and password from $EARTHDATA_USERNAME and $EARTHDATA_PASSWORD.
-            persist: Will persist credentials in a `.netrc` file.
+                    Retrieve either a username and password pair from the
+                    `EARTHDATA_USERNAME` and `EARTHDATA_PASSWORD` environment variables,
+                    or an Earthdata login token from the `EARTHDATA_TOKEN` environment
+                    variable.
+            persist: Will persist username and password credentials in a `.netrc` file.
             system: the EDL endpoint to log in to Earthdata, defaults to PROD
 
         Returns:
@@ -245,7 +248,7 @@ def _interactive(
     ) -> bool:
         username = input("Enter your Earthdata Login username: ")
         password = getpass.getpass(prompt="Enter your Earthdata password: ")
-        authenticated = self._get_credentials(username, password)
+        authenticated = self._get_credentials(username, password, None)
         if authenticated:
             logger.debug("Using user provided credentials for EDL")
             if persist_credentials:
@@ -282,7 +285,7 @@ def _netrc(self) -> bool:
                 f"Password not found in .netrc file {netrc_loc}"
             )
 
-        authenticated = self._get_credentials(username, password)
+        authenticated = self._get_credentials(username, password, None)
 
         if authenticated:
             logger.debug("Using .netrc file for EDL")
@@ -292,21 +295,27 @@ def _netrc(self) -> bool:
     def _environment(self) -> bool:
         username = os.getenv("EARTHDATA_USERNAME")
         password = os.getenv("EARTHDATA_PASSWORD")
+        token = os.getenv("EARTHDATA_TOKEN")
 
-        if not username or not password:
+        if (not username or not password) and not token:
             raise LoginStrategyUnavailable(
-                "EARTHDATA_USERNAME and EARTHDATA_PASSWORD are not set in the current environment, try "
-                "setting them or use a different strategy (netrc, interactive)"
+                "Either the environment variables EARTHDATA_USERNAME and "
+                "EARTHDATA_PASSWORD must both be set, or EARTHDATA_TOKEN must be set for "
+                "the 'environment' login strategy."
             )
 
         logger.debug("Using environment variables for EDL")
-        return self._get_credentials(username, password)
+        return self._get_credentials(username, password, token)
 
     def _get_credentials(
         self,
         username: Optional[str],
         password: Optional[str],
+        user_token: Optional[str],
     ) -> bool:
+        if user_token is not None:
+            self.token = {"access_token": user_token}
+            self.authenticated = True
         if username is not None and password is not None:
             token_resp = self._find_or_create_token(username, password)
 

earthaccess/search.py

@@ -62,9 +62,6 @@ def __init__(self, auth: Optional[Auth] = None, *args: Any, **kwargs: Any) -> No
 
         self._debug = False
 
-        self.params["has_granules"] = True
-        self.params["include_granule_counts"] = True
-
     @override
     def hits(self) -> int:
         """Returns the number of hits the current query will return.
@@ -306,10 +303,10 @@ def has_granules(self, has_granules: bool | None = True) -> Self:
         if has_granules is not None and not isinstance(has_granules, bool):
             raise TypeError("has_granules must be of type bool or None")
 
-        if has_granules is None and "has_granules" in self.params:
-            del self.params["has_granules"]
-        else:
+        if has_granules is not None:
             self.params["has_granules"] = has_granules
+        elif "has_granules" in self.params:
+            del self.params["has_granules"]
 
         return self
 
@@ -360,7 +357,7 @@ def data_center(self, data_center_name: str) -> Self:
         """An alias for the `daac` method.
 
         Parameters:
-            data_center_name: DAAC shortname, e.g. NSIDC, PODAAC, GESDISC
+            data_center_name: DAAC shortname, e.g. NSIDC, PODAAC, GES_DISC
 
         Returns:
             self
@@ -372,7 +369,7 @@ def daac(self, daac_short_name: str) -> Self:
         for the DAAC.
 
         Parameters:
-            daac_short_name: a DAAC shortname, e.g. NSIDC, PODAAC, GESDISC
+            daac_short_name: a DAAC shortname, e.g. NSIDC, PODAAC, GES_DISC
 
         Returns:
             self
@@ -556,7 +553,7 @@ def data_center(self, data_center_name: str) -> Self:
         """An alias for the `daac` method.
 
         Parameters:
-            data_center_name (String): DAAC shortname, e.g. NSIDC, PODAAC, GESDISC
+            data_center_name (String): DAAC shortname, e.g. NSIDC, PODAAC, GES_DISC
 
         Returns:
             self
@@ -568,7 +565,7 @@ def daac(self, daac_short_name: str) -> Self:
         the DAAC.
 
         Parameters:
-            daac_short_name: a DAAC shortname, e.g. NSIDC, PODAAC, GESDISC
+            daac_short_name: a DAAC shortname, e.g. NSIDC, PODAAC, GES_DISC
 
         Returns:
             self

earthaccess/store.py

@@ -494,7 +494,7 @@ def get(
         threads: int = 8,
         *,
         pqdm_kwargs: Optional[Mapping[str, Any]] = None,
-    ) -> List[str]:
+    ) -> List[Path]:
         """Retrieves data granules from a remote storage system.
 
            * If we run this in the cloud,
@@ -545,7 +545,7 @@ def _get(
         provider: Optional[str] = None,
         *,
         pqdm_kwargs: Optional[Mapping[str, Any]] = None,
-    ) -> List[str]:
+    ) -> List[Path]:
         """Retrieves data granules from a remote storage system.
 
            * If we run this in the cloud,
@@ -576,7 +576,7 @@ def _get_urls(
         provider: Optional[str] = None,
         *,
         pqdm_kwargs: Optional[Mapping[str, Any]] = None,
-    ) -> List[str]:
+    ) -> List[Path]:
         data_links = granules
         downloaded_files: List = []
         if provider is None and self.in_region and "cumulus" in data_links[0]:
@@ -616,7 +616,7 @@ def _get_granules(
         provider: Optional[str] = None,
         *,
         pqdm_kwargs: Optional[Mapping[str, Any]] = None,
-    ) -> List[str]:
+    ) -> List[Path]:
         data_links: List = []
         downloaded_files: List = []
         provider = granules[0]["meta"]["provider-id"]
@@ -687,7 +687,7 @@ def _clone_session_in_local_thread(
             local_thread_session.auth = original_session.auth
             self.thread_locals.local_thread_session = local_thread_session
 
-    def _download_file(self, url: str, directory: Path) -> str:
+    def _download_file(self, url: str, directory: Path) -> Path:
         """Download a single file from an on-prem location, a DAAC data center.
 
         Parameters:
@@ -722,7 +722,7 @@ def _download_file(self, url: str, directory: Path) -> str:
                 raise DownloadFailure(msg)
         else:
             logger.info(f"File {local_filename} already downloaded")
-        return str(path)
+        return path
 
     def _download_onprem_granules(
         self,

tests/integration/fixtures/vcr_cassettes/TestServices.test_services.yaml

@@ -297,7 +297,7 @@ interactions:
       User-Agent:
       - earthaccess v0.9.0
     method: GET
-    uri: https://cmr.earthdata.nasa.gov/search/collections.umm_json?has_granules=true&include_granule_counts=true&short_name=MUR-JPL-L4-GLOB-v4.1&cloud_hosted=true&temporal%5B%5D=2024-02-27T00:00:00Z,2024-02-29T00:00:00Z&page_size=0
+    uri: https://cmr.earthdata.nasa.gov/search/collections.umm_json?short_name=MUR-JPL-L4-GLOB-v4.1&cloud_hosted=true&temporal%5B%5D=2024-02-27T00:00:00Z,2024-02-29T00:00:00Z&page_size=0
   response:
     body:
       string: '{"hits":1,"took":730,"items":[]}'
@@ -364,7 +364,7 @@ interactions:
       User-Agent:
       - earthaccess v0.9.0
     method: GET
-    uri: https://cmr.earthdata.nasa.gov/search/collections.umm_json?has_granules=true&include_granule_counts=true&short_name=MUR-JPL-L4-GLOB-v4.1&cloud_hosted=true&temporal%5B%5D=2024-02-27T00:00:00Z,2024-02-29T00:00:00Z&page_size=0
+    uri: https://cmr.earthdata.nasa.gov/search/collections.umm_json?short_name=MUR-JPL-L4-GLOB-v4.1&cloud_hosted=true&temporal%5B%5D=2024-02-27T00:00:00Z,2024-02-29T00:00:00Z&page_size=0
   response:
     body:
       string: '{"hits":1,"took":213,"items":[]}'
@@ -431,7 +431,7 @@ interactions:
       User-Agent:
       - earthaccess v0.9.0
     method: GET
-    uri: https://cmr.earthdata.nasa.gov/search/collections.umm_json?has_granules=true&include_granule_counts=true&short_name=MUR-JPL-L4-GLOB-v4.1&cloud_hosted=true&temporal%5B%5D=2024-02-27T00:00:00Z,2024-02-29T00:00:00Z&page_size=1
+    uri: https://cmr.earthdata.nasa.gov/search/collections.umm_json?short_name=MUR-JPL-L4-GLOB-v4.1&cloud_hosted=true&temporal%5B%5D=2024-02-27T00:00:00Z,2024-02-29T00:00:00Z&page_size=1
   response:
     body:
       string: "{\"hits\":1,\"took\":271,\"items\":[{\"meta\":{\"revision-id\":38,\"deleted\":false,\"format\":\"application/vnd.nasa.cmr.umm+json\",\"provider-id\":\"POCLOUD\",\"has-combine\":true,\"user-id\":\"wenhaoli\",\"has-formats\":true,\"associations\":{\"variables\":[\"V2262655973-POCLOUD\",\"V2112015425-POCLOUD\",\"V2028632047-POCLOUD\",\"V2112015422-POCLOUD\",\"V2028668049-POCLOUD\",\"V2028632042-POCLOUD\",\"V2112015420-POCLOUD\",\"V2028632044-POCLOUD\"],\"services\":[\"S2839491596-XYZ_PROV\",\"S2164732315-XYZ_PROV\",\"S2004184019-POCLOUD\",\"S2874702816-XYZ_PROV\",\"S2606110201-XYZ_PROV\"],\"tools\":[\"TL2108419875-POCLOUD\",\"TL2245207664-POCLOUD\"]},\"s3-links\":[\"podaac-ops-cumulus-public/MUR-JPL-L4-GLOB-v4.1/\",\"podaac-ops-cumulus-protected/MUR-JPL-L4-GLOB-v4.1/\"],\"has-spatial-subsetting\":true,\"native-id\":\"GHRSST+Level+4+MUR+Global+Foundation+Sea+Surface+Temperature+Analysis+(v4.1)\",\"has-transforms\":false,\"association-details\":{\"variables\":[{\"concept-id\":\"V2262655973-POCLOUD\"},{\"concept-id\":\"V2112015425-POCLOUD\"},{\"concept-id\":\"V2028632047-POCLOUD\"},{\"concept-id\":\"V2112015422-POCLOUD\"},{\"concept-id\":\"V2028668049-POCLOUD\"},{\"concept-id\":\"V2028632042-POCLOUD\"},{\"concept-id\":\"V2112015420-POCLOUD\"},{\"concept-id\":\"V2028632044-POCLOUD\"}],\"services\":[{\"concept-id\":\"S2839491596-XYZ_PROV\"},{\"concept-id\":\"S2164732315-XYZ_PROV\"},{\"concept-id\":\"S2004184019-POCLOUD\"},{\"concept-id\":\"S2874702816-XYZ_PROV\"},{\"concept-id\":\"S2606110201-XYZ_PROV\"}],\"tools\":[{\"concept-id\":\"TL2108419875-POCLOUD\"},{\"concept-id\":\"TL2245207664-POCLOUD\"}]},\"has-variables\":true,\"concept-id\":\"C1996881146-POCLOUD\",\"revision-date\":\"2023-05-30T16:42:26.806Z\",\"granule-count\":3,\"has-temporal-subsetting\":true,\"concept-type\":\"collection\"},\"umm\":{\"DataLanguage\":\"eng\",\"AncillaryKeywords\":[\"GHRSST\",\"sea