update plugin download methods to use an asset as input instead of a product

[sbrunato]

Once #2069 and #2070 are done,

update plugin download methods (download, _prepare_download, _finalize) to use an asset as input instead of a product as argument.

Add a plugin._finalize_product(product) method to handle flatten_top_dirs:

• In the case of a whole product download from multiple assets:
  • call _finalize(asset, flatten_top_dirs=False)
  • then _finalize_product(product) only handles flatten_top_dirs (to avoid having all assets at the same level)
• single asset download:
  • simply call _finalize(asset)

Technical note

For now, cosmetics and code structure ll be delayed until functional validation done.
After implement asset mapping upgrade, code using fields "eodag:download_link", "order:status", "order_link" have to be rework first, before rework download:

Processor

Replace Executor by internal class to globalize parallelize

• set design pattern "mediator" to remove wandering references of executer

Main goal is to remove executor reference pass through parameters, makes all parallzlize request queue task in unique and global queue.
Each task can have it's own callback, callback passthrough params, task timeout, task parallelize restricts (that(s not global for all tasks).
Processor should allow to wait for a segments of tasks, and not all tasks, if a download many products, each product can download many asset, and product will wait for it's assets done to callback on it's own task.

Providers tests

• cop_ads

  • http order bahaviour
  • http download
  • file download iterator
  • response download iterator
  • asset.download
  • asset.stream_docnload
  • product.download
  • product.asset_download

• aws_eos

  • rework plugins.auth dependencies, much trouble of dependency cycle with plugin auth, air classes and files, remove non direct functional dependencies (unstubbed / composite)
  • update pyproject.toml to use natural python module exposition, reject to give full path in private file of module, it's make partial dependencies load, only expose public path, et makes local import at root in a module. (strategy)
  • add download fetch_metadata convertion from provider configuration
  • ass download safe_build conertion from provider configuration
  • add download complementary_url_key convertion from provider configuration
    • interpreted as complementary_url_prefix
  • add download requester_pays convertion to was filter "RequestPayer"
  • s3 download iterator
  • steadify processor: force stop thread managing individual timeout when all tasks processed.
  • asset.download
  • asset.stream_download
  • product.download
  • product.asset_download

• cop_dataspace

  • asset.download
  • asset.stream_docnload
  • product.download
  • product.asset_download

• cop_marine

  • fix s3contentiterator, some s3 wrapper object expose iter_chunks instead of iter_content (abstract factory)
  • isolate mime, add extension guess by file headers (identify package hdf5)
  • cache fix: save full href into cache, not s3 key only
  • asset.download
  • asset.stream_docnload
  • product.download
  • product.asset_download

• sara

  • makes all requests (http / s3) pass through same queuer to manage paralellize restrict as global
  • asset.download
  • asset.stream_docnload
  • product.download
  • product.asset_download
  • plugin.download.base.download_all()

• creodias

  • asset.download
  • asset.stream_docnload
  • product.download
  • product.asset_download
  • Download.download_all() ? totp token not enough long to manage long download without error

• creodias_s3

  • rework plugins.search dependencies, much trouble of dependency cycle with plugin auth, air classes and files, remove non direct functional dependencies (unstubbed / composite)
  • externalize event method to isolated object "eventable"
  • Creodiass3Search
    replace patch_dowloader (located on asset) by event handler "register_plugin_manager" located on product to trigger asset update from s3 when plugin_manager is defined on product.
    /!\ It mean that's product asset ll not be update to date until first download launched
    /!\ It mean you can't download an asset before download asset downlink, which is only one defined by default
  • asset.download
  • asset.stream_docnload
  • product.download
  • product.asset_download
  • Download.download_all()

• dedl

  • fix content iterator filesize bug
  • asset.download
  • asset.stream_docnload
  • product.download
  • product.asset_download
  • Download.download_all()

• geodes

  • update log descriptor for http 410
  • asset.download
  • asset.stream_docnload
  • product.download
  • product.asset_download
  • Download.download_all()

• hydroweb_next

  • fix cache search from archive scan for extensions when it's not configured
  • fix: set asset order:status to succeded when no order_link
  • fix: assetdict duplication url post check to cover when technical asset added after basic assets
  • asset.download
  • asset.stream_docnload
  • product.download
  • product.asset_download
  • Download.download_all()

• planetary_computer

  • seems trouble on product.download with progressbar superposition
  • asset.download
  • asset.stream_docnload
  • product.download
  • product.asset_download
  • Download.download_all()