Skip to content

Prepare for v0.6 release #876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 42 commits into from
Aug 30, 2025
Merged

Prepare for v0.6 release #876

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
ca4f4b6
Rework Readme
fsschneider Jun 24, 2025
2d2fd37
Remove deprecated rules and call for submissions
fsschneider Jun 24, 2025
2544fb9
Clarify automatic versioning
fsschneider Jun 25, 2025
c8da1f9
Mention changelog
fsschneider Jun 25, 2025
d233344
Backlog changes
fsschneider Jun 25, 2025
679e5ec
Increment version + formatting
fsschneider Jun 25, 2025
891fdb7
Formatting
fsschneider Jun 25, 2025
0390e13
Mention our version policy
fsschneider Jun 25, 2025
818710b
More descriptive link text
fsschneider Jun 25, 2025
03114bd
Add versioning protocol
fsschneider Jun 25, 2025
a2ab920
Document dropout PR
fsschneider Jun 25, 2025
64e9961
Compact layout for default dropout values
fsschneider Jun 25, 2025
23004a2
Change from 5 to 3 studies
fsschneider Jun 25, 2025
576c661
Change from 5 to 3 studies
fsschneider Jun 25, 2025
bdfd9d8
Remove held-out workloads
fsschneider Jun 25, 2025
0ac2dbe
First draft at re-ordering algorithms
fsschneider Jul 8, 2025
a83d573
Fix typo in documentation regarding `update_params` function arguments
fsschneider Jul 9, 2025
200ef7c
Add detailed docstrings for functions in submission.py
fsschneider Jul 9, 2025
8cfd172
Move paper baselines into `archived_paper_baselines` subdirectory
fsschneider Jul 9, 2025
1f7915e
Add proper readme for algorithms collection
fsschneider Jul 9, 2025
db6681e
prize_qualification_baselines -> Baselines
fsschneider Jul 9, 2025
dd2cd50
Remove mention of prize_qualification
fsschneider Jul 9, 2025
af89c29
Add Training Algorithm Collection section to README.md
fsschneider Jul 9, 2025
a0d8558
Fix paths to algorithms
fsschneider Jul 9, 2025
a106537
Fix a few more :)
fsschneider Jul 9, 2025
5f0c77f
Update Changelog
fsschneider Jul 9, 2025
4ebefc7
Fixed link to cloud-startup
fsschneider Jul 9, 2025
e31b6d9
Docs rewrite: Part I
fsschneider Jul 9, 2025
10faf4d
Formatting tweaks
fsschneider Jul 9, 2025
37803c6
Typos and clarity
fsschneider Jul 10, 2025
e813bb5
Add important rules from COMPETITON RULES
fsschneider Jul 10, 2025
44dd919
Simplify Intro
fsschneider Jul 10, 2025
3411368
Small readability tweaks
fsschneider Jul 10, 2025
79312f6
Rewrite benchmark process intro
fsschneider Jul 10, 2025
a6d939d
Re-write (parts of) Submission section
fsschneider Jul 10, 2025
29a552a
Remove last mention of model track
fsschneider Jul 10, 2025
e02e401
Merge remote-tracking branch 'upstream/dev' into prepare_v0.6
fsschneider Aug 27, 2025
6f63317
Full pass tweaking documentation
fsschneider Aug 27, 2025
75fdc24
Add `pmap` to `jit` switch to v0.6
fsschneider Aug 27, 2025
a4e6d3a
Fix GitHub math [no ci]
fsschneider Aug 27, 2025
ad26373
Add link to Changelog for details and links to rule change discussion…
fsschneider Aug 29, 2025
f7bd572
Augment with links to rules change discussion/justifications [no CI]
fsschneider Aug 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
Binary file modified .assets/performance_profiles.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
32 changes: 16 additions & 16 deletions .github/workflows/CI.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ jobs:
pip install .[pytorch_cpu]
pip install .[full]
pip install -e .
python tests/reference_algorithm_tests.py --workload=fastmri --framework=pytorch --global_batch_size=8 --submission_path=reference_algorithms/target_setting_algorithms/pytorch_momentum.py --tuning_search_space=reference_algorithms/target_setting_algorithms/fastmri/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=fastmri --framework=jax --global_batch_size=8 --submission_path=reference_algorithms/target_setting_algorithms/jax_momentum.py --tuning_search_space=reference_algorithms/target_setting_algorithms/fastmri/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=fastmri --framework=pytorch --global_batch_size=8 --submission_path=algorithms/target_setting_algorithms/pytorch_momentum.py --tuning_search_space=algorithms/target_setting_algorithms/fastmri/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=fastmri --framework=jax --global_batch_size=8 --submission_path=algorithms/target_setting_algorithms/jax_momentum.py --tuning_search_space=algorithms/target_setting_algorithms/fastmri/tuning_search_space.json
wmt_jax:
runs-on: ubuntu-latest
steps:
Expand All @@ -37,7 +37,7 @@ jobs:
pip install .[pytorch_cpu]
pip install .[full]
pip install -e .
python tests/reference_algorithm_tests.py --workload=wmt --framework=jax --global_batch_size=2 --submission_path=reference_algorithms/target_setting_algorithms/jax_nadamw.py --tuning_search_space=reference_algorithms/target_setting_algorithms/wmt/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=wmt --framework=jax --global_batch_size=2 --submission_path=algorithms/target_setting_algorithms/jax_nadamw.py --tuning_search_space=algorithms/target_setting_algorithms/wmt/tuning_search_space.json
wmt_pytorch:
runs-on: ubuntu-latest
steps:
Expand All @@ -54,7 +54,7 @@ jobs:
pip install .[pytorch_cpu]
pip install .[full]
pip install -e .
python tests/reference_algorithm_tests.py --workload=wmt --framework=pytorch --global_batch_size=2 --submission_path=reference_algorithms/target_setting_algorithms/pytorch_nadamw.py --tuning_search_space=reference_algorithms/target_setting_algorithms/wmt/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=wmt --framework=pytorch --global_batch_size=2 --submission_path=algorithms/target_setting_algorithms/pytorch_nadamw.py --tuning_search_space=algorithms/target_setting_algorithms/wmt/tuning_search_space.json
imagenet_jax:
runs-on: ubuntu-latest
steps:
Expand All @@ -71,8 +71,8 @@ jobs:
pip install .[pytorch_cpu]
pip install .[full]
pip install -e .
python tests/reference_algorithm_tests.py --workload=imagenet_vit --framework=jax --global_batch_size=2 --submission_path=reference_algorithms/target_setting_algorithms/jax_adamw.py --tuning_search_space=reference_algorithms/target_setting_algorithms/imagenet_vit/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=imagenet_resnet --framework=jax --global_batch_size=2 --submission_path=reference_algorithms/target_setting_algorithms/jax_momentum.py --tuning_search_space=reference_algorithms/target_setting_algorithms/imagenet_resnet/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=imagenet_vit --framework=jax --global_batch_size=2 --submission_path=algorithms/target_setting_algorithms/jax_adamw.py --tuning_search_space=algorithms/target_setting_algorithms/imagenet_vit/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=imagenet_resnet --framework=jax --global_batch_size=2 --submission_path=algorithms/target_setting_algorithms/jax_momentum.py --tuning_search_space=algorithms/target_setting_algorithms/imagenet_resnet/tuning_search_space.json
imagenet_pytorch:
runs-on: ubuntu-latest
steps:
Expand All @@ -89,8 +89,8 @@ jobs:
pip install .[pytorch_cpu]
pip install .[full]
pip install -e .
python tests/reference_algorithm_tests.py --workload=imagenet_resnet --framework=pytorch --global_batch_size=2 --submission_path=reference_algorithms/target_setting_algorithms/pytorch_momentum.py --tuning_search_space=reference_algorithms/target_setting_algorithms/imagenet_resnet/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=imagenet_vit --framework=pytorch --global_batch_size=2 --submission_path=reference_algorithms/target_setting_algorithms/pytorch_adamw.py --tuning_search_space=reference_algorithms/target_setting_algorithms/imagenet_vit/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=imagenet_resnet --framework=pytorch --global_batch_size=2 --submission_path=algorithms/target_setting_algorithms/pytorch_momentum.py --tuning_search_space=algorithms/target_setting_algorithms/imagenet_resnet/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=imagenet_vit --framework=pytorch --global_batch_size=2 --submission_path=algorithms/target_setting_algorithms/pytorch_adamw.py --tuning_search_space=algorithms/target_setting_algorithms/imagenet_vit/tuning_search_space.json
# uncomment when https://github.com/mlcommons/algorithmic-efficiency/issues/339 is resolved.
criteo_jax:
runs-on: ubuntu-latest
Expand All @@ -108,7 +108,7 @@ jobs:
pip install .[pytorch_cpu]
pip install .[full]
pip install -e .
python tests/reference_algorithm_tests.py --workload=criteo1tb_test --framework=jax --global_batch_size=1 --submission_path=reference_algorithms/target_setting_algorithms/jax_adamw.py --tuning_search_space=reference_algorithms/target_setting_algorithms/criteo1tb/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=criteo1tb_test --framework=jax --global_batch_size=1 --submission_path=algorithms/target_setting_algorithms/jax_adamw.py --tuning_search_space=algorithms/target_setting_algorithms/criteo1tb/tuning_search_space.json
criteo_pytorch:
runs-on: ubuntu-latest
steps:
Expand All @@ -125,7 +125,7 @@ jobs:
pip install .[pytorch_cpu]
pip install .[full]
pip install -e .
python tests/reference_algorithm_tests.py --workload=criteo1tb_test --framework=pytorch --global_batch_size=1 --submission_path=reference_algorithms/target_setting_algorithms/pytorch_adamw.py --tuning_search_space=reference_algorithms/target_setting_algorithms/criteo1tb/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=criteo1tb_test --framework=pytorch --global_batch_size=1 --submission_path=algorithms/target_setting_algorithms/pytorch_adamw.py --tuning_search_space=algorithms/target_setting_algorithms/criteo1tb/tuning_search_space.json
speech_jax:
runs-on: ubuntu-latest
steps:
Expand All @@ -142,8 +142,8 @@ jobs:
pip install .[pytorch_cpu]
pip install .[full]
pip install -e .
python tests/reference_algorithm_tests.py --workload=librispeech_conformer --framework=jax --global_batch_size=2 --submission_path=reference_algorithms/target_setting_algorithms/jax_adamw.py --tuning_search_space=reference_algorithms/target_setting_algorithms/librispeech_conformer/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=librispeech_deepspeech --framework=jax --global_batch_size=2 --submission_path=reference_algorithms/target_setting_algorithms/jax_adamw.py --tuning_search_space=reference_algorithms/target_setting_algorithms/librispeech_deepspeech/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=librispeech_conformer --framework=jax --global_batch_size=2 --submission_path=algorithms/target_setting_algorithms/jax_adamw.py --tuning_search_space=algorithms/target_setting_algorithms/librispeech_conformer/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=librispeech_deepspeech --framework=jax --global_batch_size=2 --submission_path=algorithms/target_setting_algorithms/jax_adamw.py --tuning_search_space=algorithms/target_setting_algorithms/librispeech_deepspeech/tuning_search_space.json
speech_pytorch:
runs-on: ubuntu-latest
steps:
Expand All @@ -160,8 +160,8 @@ jobs:
pip install .[pytorch_cpu]
pip install .[full]
pip install -e .
python tests/reference_algorithm_tests.py --workload=librispeech_deepspeech --framework=pytorch --global_batch_size=2 --submission_path=reference_algorithms/target_setting_algorithms/pytorch_adamw.py --tuning_search_space=reference_algorithms/target_setting_algorithms/librispeech_deepspeech/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=librispeech_conformer --framework=pytorch --global_batch_size=2 --submission_path=reference_algorithms/target_setting_algorithms/pytorch_adamw.py --tuning_search_space=reference_algorithms/target_setting_algorithms/librispeech_conformer/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=librispeech_deepspeech --framework=pytorch --global_batch_size=2 --submission_path=algorithms/target_setting_algorithms/pytorch_adamw.py --tuning_search_space=algorithms/target_setting_algorithms/librispeech_deepspeech/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=librispeech_conformer --framework=pytorch --global_batch_size=2 --submission_path=algorithms/target_setting_algorithms/pytorch_adamw.py --tuning_search_space=algorithms/target_setting_algorithms/librispeech_conformer/tuning_search_space.json
ogbg:
runs-on: ubuntu-latest
steps:
Expand All @@ -178,8 +178,8 @@ jobs:
pip install .[pytorch_cpu]
pip install .[full]
pip install -e .
python tests/reference_algorithm_tests.py --workload=ogbg --framework=pytorch --global_batch_size=8 --submission_path=reference_algorithms/target_setting_algorithms/pytorch_nesterov.py --tuning_search_space=reference_algorithms/target_setting_algorithms/ogbg/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=ogbg --framework=jax --global_batch_size=8 --submission_path=reference_algorithms/target_setting_algorithms/jax_nesterov.py --tuning_search_space=reference_algorithms/target_setting_algorithms/ogbg/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=ogbg --framework=pytorch --global_batch_size=8 --submission_path=algorithms/target_setting_algorithms/pytorch_nesterov.py --tuning_search_space=algorithms/target_setting_algorithms/ogbg/tuning_search_space.json
python tests/reference_algorithm_tests.py --workload=ogbg --framework=jax --global_batch_size=8 --submission_path=algorithms/target_setting_algorithms/jax_nesterov.py --tuning_search_space=algorithms/target_setting_algorithms/ogbg/tuning_search_space.json
pytest-params:
runs-on: ubuntu-latest
steps:
Expand Down
Loading