Skip to content

Add task affinity support with compute_scope and result_scope in Dagger.jl's @spawn macro #610

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 8, 2025

Conversation

AkhilAkkapelli
Copy link
Contributor

  • Enhanced the Thunk struct to include compute_scope and result_scope for better task execution control.
  • Updated the Thunk constructor to accept new scope parameters.
  • Modified the @spawn macro to handle the new scope parameters appropriately.
  • Introduced a new test suite for task affinity, covering various scenarios with scope interactions.
  • Added comprehensive documentation for task affinity, detailing the usage of scope, compute_scope, and result_scope.
  • Implemented tests to validate behavior when using function, chunk, chunk arguments as inputs in tasks, ensuring correct scope handling.

@AkhilAkkapelli AkhilAkkapelli changed the title Add task affinity support with compute_scope and result_scope in Dagger.jl's @spawn macro Add task affinity support with compute_scope and result_scope in Dagger.jl's @spawn macro Jun 13, 2025
@AkhilAkkapelli AkhilAkkapelli changed the title Add task affinity support with compute_scope and result_scope in Dagger.jl's @spawn macro Add task affinity support with compute_scope and result_scope in Dagger.jl's @spawn macro Jun 13, 2025
Copy link
Member

@jpsamaroo jpsamaroo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! I'm really excited to see this merged 😄 Just a few docs and tests adjustments I'd like to see made; the logic is otherwise solid.

…er.jl's @Spawn macro

- Enhanced the Thunk struct to include compute_scope and result_scope for better task execution control.
- Updated the Thunk constructor to accept new scope parameters.
- Modified the spawn function to handle the new scope parameters appropriately.
- Introduced a new test suite for task affinity, covering various scenarios with scope interactions.
- Added comprehensive documentation for task affinity, detailing the usage of scope, compute_scope, and result_scope.
- Implemented tests to validate behavior when using chunks as inputs in tasks, ensuring correct scope handling.
- Updated documentation for the `@spawn` macro to clarify the usage of `scope`, `compute_scope`, and `result_scope`, including examples with the new syntax.
- Improved error messages in the scheduling logic to provide clearer feedback when scopes are incompatible.
- Refactored test cases for task affinity to ensure they align with the new scope handling and provide better coverage for edge cases.
- Removed deprecated comments and cleaned up the code for better readability.
@jpsamaroo jpsamaroo merged commit 9c7be3f into JuliaParallel:master Jul 8, 2025
13 of 18 checks passed
@jpsamaroo
Copy link
Member

Thanks for the great work @AkhilAkkapelli !

@AkhilAkkapelli AkhilAkkapelli deleted the task-affinity branch July 8, 2025 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants