Argo-like 3D observations and ocean DA problem redesign#1
Open
Argo-like 3D observations and ocean DA problem redesign#1
Conversation
Replace surface-only thinned grid observations with Argo-like profile sampling in the ETKF: - 76 random profiles per cycle (~770 Argo floats / 10-day repeat) - Each profile observes 31 vertical levels in upper 1000m - Total obs: ~4,700 (T+S) per cycle vs 56K surface-only before - Obs-to-ensemble ratio: 470:1 (10 members) or 47:1 (100 members) - MPI-aware: local extraction + single Allreduce for efficiency Add daily 3D tracer output to nature run (T and S at 1-day intervals) for subsurface observation extraction. Add RESEARCH_NOTES.md documenting literature review on ocean DA for hurricane prediction: current perturbations are too easy (surface errors decay under atmospheric forcing in days), the real challenge is subsurface structure (thermocline depth, OHC) which persists for months and controls hurricane intensity. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Documents ensemble Kalman filter variants (ETKF, EnKF, LETKF, DEnKF, SEEK), operational ocean DA systems (GLORYS12, TOPAZ, ORAS5, RTOFS, FOAM, ROMS 4DVAR), and implications for our variational approach. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace surface-focused perturbations (0.3C bias in upper 20 levels) with depth-dependent profile: zero at surface, max 1.5C at thermocline (~150m), decaying below. These persist months-years, making the DA problem meaningful. - Free run uses bias-only (no noise) to represent ensemble mean trajectory. Ensemble members get bias + noise, both scaled by depth profile. - Load free run surface fields in ensemble_da.jl and compute free run RMSE against nature run at each DA cycle. - Three-line RMSE plot: free run (red) vs DA forecast (orange) vs DA analysis (green). Gap between red and green = demonstrated DA value. - Add free run prerequisite check in SLURM DA script. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Weak-scaled from 4-GPU (1/8 deg) to 64 GPUs (16 nodes): - 3200x2240x50 global grid, Partition(8,8), 400x280x50 per GPU - 240 free-surface substeps (scaled with resolution) - Free run uses bias+gradient perturbation (matching single_gpu_da) - Array(znodes(...)) fix to avoid GPU scalar indexing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Distributed ensemble DA via SLURM job dependency chains (da_utils.jl, initialize_ensemble.jl, ensemble_member.jl, etkf_analysis.jl) - LETKF with Gaspari-Cohn localization (1000 km radius) - Fixed critical inflation bug (inflate around analysis mean, not forecast) - Fixed observation errors (representativeness error for 1/4 deg grid) - Vertical subsampling (every 4th level) to reduce overdetermination - 8-GPU and 64-GPU scaling benchmarks - Multi-node Julia ENV iteration bug fix (fix_multinode_env.jl) - Progress report with development history and cost estimates
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Key finding: our current problem is too easy
Surface perturbations (0.3 C bias + 0.5 C noise) decay naturally under atmospheric forcing in ~11-14 days. The 32% RMSE reduction we observed is mostly the atmosphere doing the work, not the DA.
The scientifically meaningful problem is recovering subsurface structure (thermocline depth, ocean heat content at 50-300m), which:
Changes
Argo-like observation network (ensemble_da.jl)
Nature run 3D output (nature_run.jl)
Research notes (RESEARCH_NOTES.md)
Next steps
Test plan
Generated with Claude Code