Problems processing data from 20240626 #223
Description
We are getting an error in the r0_to_dl1 step for various subruns of 20240626 (e.g. 17851.0075) with which we do not know how to proceed. This is the error traceback that we are getting:
WARNING: non-ASCII characters are present in the FITS file header and have been replaced by "?" characters [astropy.io.fits.util]
non-ASCII characters are present in the FITS file header and have been replaced by "?" characters
WARNING: VerifyWarning: Error validating header for HDU #4 (note: Astropy uses zero-based indexing).
Header size is not multiple of 2880: 880335602
There may be extra bytes after the last HDU or the file is corrupted. [astropy.io.fits.hdu.hdulist]
VerifyWarning: Error validating header for HDU #4 (note: Astropy uses zero-based indexing).
Header size is not multiple of 2880: 880335602
There may be extra bytes after the last HDU or the file is corrupted.
Traceback (most recent call last):
File "/fefs/aswg/software/conda/envs/osa-v0.10/bin/lstchain_data_r0_to_dl1", line 8, in <module>
sys.exit(main())
^^^^^^
File "/fefs/aswg/software/conda/envs/osa-v0.10/lib/python3.11/site-packages/lstchain/scripts/lstchain_data_r0_to_dl1.py", line 230, in main
r0_to_dl1.r0_to_dl1(
File "/fefs/aswg/software/conda/envs/osa-v0.10/lib/python3.11/site-packages/lstchain/reco/r0_to_dl1.py", line 352, in r0_to_dl1
source = EventSource(input_url=input_filename,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/fefs/aswg/software/conda/envs/osa-v0.10/lib/python3.11/site-packages/ctapipe_io_lst/__init__.py", line 304, in __init__
self.multi_file = MultiFiles(
^^^^^^^^^^^
File "/fefs/aswg/software/conda/envs/osa-v0.10/lib/python3.11/site-packages/ctapipe_io_lst/multifiles.py", line 123, in __init__
self._load_next_subrun(stream)
File "/fefs/aswg/software/conda/envs/osa-v0.10/lib/python3.11/site-packages/ctapipe_io_lst/multifiles.py", line 184, in _load_next_subrun
event = next(self._events_tables[stream])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/fefs/aswg/software/conda/envs/osa-v0.10/lib/python3.11/site-packages/protozfits/__init__.py", line 153, in __next__
raise StopIteration
StopIteration
And here is an example of the command that is launched:
lstchain_data_r0_to_dl1 --input-file=/fefs/aswg/data/real/R0G/20240626/LST-1.1.Run17851.0075.fits.fz --output-dir=/fefs/aswg/data/real/running_analysis/20240626/v0.10 --pedestal-file=/fefs/aswg/data/real/monitoring/PixelCalibration/Cat-A/drs4_baseline/20240626/v0.10.10/drs4_pedestal.Run17846.0000.h5 --calibration-file=/fefs/aswg/data/real/monitoring/PixelCalibration/Cat-A/calibration/20240626/v0.10.10/calibration_filters_52.Run17847.0000.h5 --time-calibration-file=/fefs/aswg/data/real/monitoring/PixelCalibration/Cat-A/drs4_time_sampling_from_FF/20231102/v0.10.4/time_calibration.Run15253.0000.h5 --systematic-correction-file=/fefs/aswg/data/real/monitoring/PixelCalibration/Cat-A/ffactor_systematics/20230410/v0.10.3/calibration_scan_fit_20230410.0000.h5 --config=/fefs/aswg/lstosa/config_files_lstchain/lstchain_standard_v0.10_no_heuristic_ff.json --pointing-file=/fefs/onsite/monitoring/driveLST1/DrivePositioning/DrivePosition_log_20240626.txt --run-summary-path=/fefs/aswg/data/real/monitoring/RunSummary/RunSummary_20240626.ecsv
NOTE: input files are already gain selected
Software settings:
(osa-v0.10) [lstanalyzer@tcs06 ~]$ conda list lst
# packages in environment at /fefs/aswg/software/conda/envs/osa-v0.10:
#
# Name Version Build Channel
ctapipe-io-lst 0.23.2 pypi_0 pypi
lstchain 0.10.10 pypi_0 pypi