v0.5.2

What's Changed

• Fix timeout issues: Update TIME_LIMIT in cloud configuration files to 24 hours and adjust multimodal constraints to 10 minutes accordingly by @tonyhoo in #145
  • Resolves AWS Batch job timeouts that were causing benchmark failures
• Dependency updates:
  • Update aws-cdk-lib requirement from <2.190,>=2.0.0 to >=2.0.0,<2.205 by @dependabot[bot] in #144
  • Update pyarrow requirement from <20,>=19.0.0 to >=19.0.0,<21 by @dependabot[bot] in #135

Full Changelog: v0.5.1...v0.5.2

Full Contributor List (ordered by # of commits):
@tonyhoo, @dependabot

This release supports Python versions 3.9, 3.10, 3.11 and 3.12.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

v0.5.1

We're excited to announce the release of AutoGluon-Bench 0.5.1! This is a patch release for v0.5.0.

What's Changed

• Fix docker build by @suzhoum in #130
• Fix cv import by @suzhoum in #132

This release contains 2 commits from 1 contributor.

For more details, please check out the README.

Full Contributor List (ordered by # of commits):
@suzhoum

This release supports Python versions 3.9, 3.10, 3.11 and 3.12.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

v0.5.0

We're excited to announce the release of AutoGluon-Bench 0.5.0!

Main Features:

1. Added file size loading support (#112),
2. Updated infra to use the latest aws-cdk-lib which fixed a few bugs and improved deployment speed by 50% (#119),
3. Added custom args support for MultiModal module (#120),
4. Added support for Python 3.12 (#119).

This release contains 6 commits from 2 contributor.

For more details, please check out the README.

Full Contributor List (ordered by # of commits):
@suzhoum @Innixma

This release supports Python versions 3.9, 3.10, 3.11 and 3.12.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

v0.4.4

What's Changed

This is a patch release to fix the AutoGluon source install issue due to spacy update.

• Fix AutoMM by adding uv by @prateekdesai04 in #108
• Update release workflow by @suzhoum in #109

Full Changelog: v0.4.3...v0.4.4

v0.4.3

What's Changed

• Add the support of testing on multiple datasets after training on one dataset. by @Harry-zzh in #69
• Bench update by @suzhoum in #70
• Updates for evaluation and aggregation based on AG 1.0 by @Innixma in #71
• Add metric to fill missing logic by @Innixma in #72
• Update for eval refactor by @Innixma in #73
• Update multimodal dep by @suzhoum in #75

New Contributors

• @Harry-zzh made their first contribution in #69

Full Changelog: v0.4.2...v0.4.3

v0.4.2

This is a minor update from the previous release v0.4.1.

Main Features

1. Support few shot learning and custom metrics in config generation script.
2. Bug fix to use the correct file paths when installing autogluon.bench automatically from source or PyPI.
3. Some minor bug fixes.

This release contains 6 commits from 2 contributor.

Full Contributor List (ordered by # of commits):
@prateekdesai04 @suzhoum

This release supports Python versions 3.9. and 3.10.

v0.4.1

We're excited to announce the release of AutoGluon-Bench 0.4.1!

For more details, please check out the README.

Main Features

1. Support custom metrics for multimodal module
2. Improvements on evaluation module
3. Improvements on infrastructure

This release contains 32 commits from 3 contributor.

Full Contributor List (ordered by # of commits):
@Innixma @prateekdesai04 @suzhoum

This release supports Python versions 3.9. and 3.10.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

v0.4.0

We're excited to announce the release of AutoGluon-Bench 0.4.0!

For more details, please check out the README.

Main Features

1. timeseries module is now supported! Similar as tabular module, timeseries benchmarks are running on the AMLB framework but with better support in cloud infrastructure. Custom datasets and frameworks are supported as well.
2. Hardware metrics including CPUUtilizaiton and GPUUtilization can now be queried from CloudWatch with the added option --save-hardeware-metrics in agbench run. They are pulled and saved to S3 bucket immediately after all jobs have run into completion.
3. Updates in evaluation logic

This release contains 4 commits from 3 contributor.

Full Contributor List (ordered by # of commits):
@suzhoum @Innixma @shreyash2106

This release supports Python versions 3.9. and 3.10.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

v0.3.0

We're excited to announce the release of AutoGluon-Bench 0.3.0!

For more details, please check out the README.

Main Features

1. Introduced framework and constraint concepts in multimodal module so that metadata is better logged in the results.
2. Added evaluation support for multimodal module.

This release contains 2 commits from 1 contributor.

Full Contributor List (ordered by # of commits):
@suzhoum

This release supports Python versions 3.9. and 3.10.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

v0.2.2

We're excited to announce the release of AutoGluon-Bench 0.2.2!

For more details, please check out the README.

Main Features

1. Support custom dataset with custom data loader for multimodal module
2. Added model failure support to evaluation output_context.py
3. Added median relative error diff to evaluation evaluate_results.py
4. Added detailed hardware monitoring support with 5s intervals
5. Support CPU base image when using non-GPU machines
6. Updated base images to reduce image size
7. Bug fix for running benchmark on CPU machine.

This release contains 11 commits from 3 contributor.

Full Contributor List (ordered by # of commits):
@Innixma @suzhoum @yinweisu

This release supports Python versions 3.9. and 3.10.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.