Skip to content

Conversation

lacoak21
Copy link
Contributor

@lacoak21 lacoak21 commented Sep 3, 2025

Change Summary

Overview

Validation and testing updates for l1c and l2. I ran some fake data through l1c using the command below and fixed any bugs.
I also added updates from these power points:
l2_validation.pptx
l1c_validation.pptx

imap_cli --instrument ultra --data-level l1c --descriptor 90sensor-spacecraftpset --start-date 20260926 --version v001 --dependency '[{"type": "science","files": ["imap_ultra_l1b_90sensor-goodtimes_20260926_v002.cdf"]},{"type": "science","files": ["imap_ultra_l1b_90sensor-de_20260926_v001.cdf"]},{"type": "science","files": ["imap_ultra_l1a_90sensor-rates_20260926_v006.cdf"]},{"type": "science","files": ["imap_ultra_l1a_90sensor-params_20260926_v006.cdf"]},{"type": "ancillary","files": ["imap_ultra_l1c-90sensor-sc-pointing-bsf_20250101_v000.csv"]},{"type": "ancillary","files": ["imap_ultra_l1c-90sensor-sc-pointing-theta_20250101_v000.csv"]},{"type": "ancillary","files": ["imap_ultra_l1c-90sensor-sc-pointing-phi_20250101_v000.csv"]},{"type": "ancillary","files": ["imap_ultra_l1c-90sensor-sc-pointing-index_20250101_v000.csv"]},{"type": "ancillary","files": ["imap_ultra_l1b-90sensor-scattering-calibration-data_20250101_v000.csv"]},{"type": "ancillary","files": ["imap_ultra_l1b-scattering-thresholds-per-energy_20250101_v000.csv"]},{"type": "ancillary","files": ["imap_ultra_l1b-45sensor-logistic-interpolation_20250101_v000.csv"]},{"type": "ancillary","files": ["imap_ultra_l1b-90sensor-imgparams-lookup_20250101_v001.csv"]},{"type": "ancillary","files": ["imap_ultra_l1b-sensor-gf-blades_20250101_v000.csv"]},{"type": "repoint","files": ["imap_2026_269_05.repoint.csv"]},{"type": "spin","files": ["imap_2026_268_2026_269_01.spin.csv"]},{"type": "spice", "files": ["naif0012.tls", "imap_sclk_0000.tsc"]}]'

Updated Files

-imap_processing/cli.py

  • Spacecraft pset and helio pset both have SPICE dependencies. Only helio has ephemeris data.
  • imap_processing/ultra/l1c/helio_pset.py, imap_processing/ultra/l1c/helio_pset.py
    • Update epoch to be the beginning of the pointing.
    • Add energy delta minus and plus as requested from the IT team
  • imap_processing/ultra/l2/ultra_l2.py
    • add energy dependent deps. Allow for maps to run with or without new variables. NOTE *** In the future we should probably remove this handling

Testing

Fixed tests to mock pointing data.

@lacoak21 lacoak21 self-assigned this Sep 3, 2025
@lacoak21 lacoak21 added this to IMAP Sep 3, 2025
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements validation and testing updates for ULTRA L1C and L2 processing based on running fake data through the pipeline and fixing identified bugs. The main changes address epoch handling, energy variable additions, error handling improvements, and test data compatibility.

  • Update epoch calculation to use pointing start time instead of event times
  • Add energy delta plus/minus variables to L1C datasets as requested by IT team
  • Improve error handling for dead time calculations and missing L2 variables
  • Fix test mocking and parameter naming for better validation support

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
imap_processing/ultra/utils/ultra_l1_utils.py Add energy_delta_minus/plus to dataset creation
imap_processing/ultra/l2/ultra_l2.py Add backward compatibility for optional L1C variables
imap_processing/ultra/l1c/ultra_l1c_pset_bins.py Improve dead time error handling and duplicate filtering
imap_processing/ultra/l1c/ultra_l1c.py Rename parameter from has_spice to has_ephermis_kernel
imap_processing/ultra/l1c/spacecraft_pset.py Update epoch calculation and add energy delta variables
imap_processing/ultra/l1c/helio_pset.py Update epoch calculation and add energy delta variables
imap_processing/ultra/l1b/lookup_utils.py Fix scattering calibration descriptor name
imap_processing/tests/ultra/unit/test_ultra_l2.py Update tests for new variables and energy-dependent exposure
imap_processing/tests/ultra/unit/test_ultra_l1c_pset_bins.py Fix test expectations for duplicate filtering
imap_processing/tests/ultra/unit/test_ultra_l1c.py Add mocking for pointing times and update parameter names
imap_processing/tests/ultra/unit/test_spacecraft_pset.py Add mocking for pointing time calculations
imap_processing/tests/ultra/unit/conftest.py Update ancillary file descriptor and add MET conversion mock
imap_processing/cli.py Update ephemeris detection logic and parameter naming
imap_processing/cdf/config/imap_ultra_l1c_variable_attrs.yaml Add attributes for new energy delta variables

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link
Contributor

@subagonsouth subagonsouth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The modifications in the L2 code look reasonable to me.

@laspsandoval laspsandoval added Ins: Ultra Related to the IMAP-Ultra instrument Level: L1 Level 1 processing Level: L2 Level 2 processing labels Sep 5, 2025
Copy link
Contributor

@laspsandoval laspsandoval left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Just a question.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ins: Ultra Related to the IMAP-Ultra instrument Level: L1 Level 1 processing Level: L2 Level 2 processing
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

3 participants