Skip to content

[SYCL] Don't use zstd context across threads. #19747

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 8, 2025

Conversation

uditagarwal97
Copy link
Contributor

@uditagarwal97 uditagarwal97 commented Aug 8, 2025

ZSTDCompressor holds zstd context as its only data members. The idea behind GetSingletonInstance() method was to re-use these contexts for subsequent compression and decompressions. Re-using context across (de)compression reduces system memory usage.

However, zstd contexts are not meant to be used concurrently, therefore, this PR makes ZSTDCompressor object thread-local, instead of static.
Relevant excerpt from zstd doc (https://facebook.github.io/zstd/zstd_manual.html):

When decompressing many times,
it is recommended to allocate a context only once,
and re-use it for each successive compression operation.
This will make workload friendlier for system's memory.
Use one context per thread for parallel execution.

@uditagarwal97 uditagarwal97 self-assigned this Aug 8, 2025
@uditagarwal97 uditagarwal97 requested a review from Copilot August 8, 2025 00:53
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR addresses thread safety issues with ZSTD compression by making the ZSTDCompressor singleton thread-local instead of global static. The change prevents concurrent access to ZSTD contexts, which are not thread-safe according to the ZSTD documentation.

  • Changed ZSTDCompressor::GetSingletonInstance() to use thread_local storage instead of static
  • Added comprehensive concurrent compression/decompression test with 20 threads
  • Ensures each thread has its own ZSTD context while maintaining the singleton pattern benefits

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
sycl/source/detail/compression.hpp Modified singleton to use thread_local storage for thread safety
sycl/unittests/compression/CompressionTests.cpp Added concurrent test to verify thread-safe compression/decompression

@uditagarwal97 uditagarwal97 marked this pull request as ready for review August 8, 2025 00:55
@uditagarwal97 uditagarwal97 requested a review from a team as a code owner August 8, 2025 00:55
@uditagarwal97 uditagarwal97 requested a review from againull August 8, 2025 00:55
@againull againull merged commit b6a619e into sycl Aug 8, 2025
27 checks passed
@uditagarwal97 uditagarwal97 deleted the private/udit/zstd_ctx_thread branch August 8, 2025 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants