Skip to content

Conversation

@hssyoo
Copy link
Contributor

@hssyoo hssyoo commented Nov 10, 2025

This PR adds support for partial configuration of CRTTransferManager using TransferConfig. It does the following:

  1. Exposes a config parameter in CRTTransferManager initializer that gets passed into S3ClientArgsCreator. S3ClientArgsCreator looks for supported CRT transfer config options and adds them to the request dict.
  2. Exposes a sentinel property UNSET_DEFAULT on TransferConfig. This value can be used to set defaults onto TransferManager subclasses and be able to disambiguate between a value that's there by default or a value that was explicitly provided. Details of its usage in boto3 can be found here: Add new preferred transfer client mode to enable CRT in all environments boto3#4629
  3. Exposes a method get_deep_attr() on TransferConfig that can be used to access to real, underlying value of an attribute. This is useful in cases where __getattribute__ has been overridden. Details of its usage in boto3 can be found here: Add new preferred transfer client mode to enable CRT in all environments boto3#4629
  4. If TransferConfig.multipart_chunksize is UNSET_DEFAULT (ie value was not explicitly provided), then S3ClientArgsCreator will set {'part_size': None} and let CRT determine the part size.

Although UNSET_DEFAULT and get_deep_attr() are primarily used to support boto3's custom TransferConfig class, I decided to implement them in the base class in s3transfer because S3ClientArgsCreator needs to use them to determine what part_size value to set. I wanted to avoid s3transfer classes from requiring knowledge about boto3 APIs since it doesn't depend on the boto3 library.

@hssyoo hssyoo merged commit 3d60212 into boto:develop Nov 28, 2025
76 checks passed
@hssyoo hssyoo deleted the crt-mode branch November 28, 2025 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants