Skip to content

(step-functions): Dynamic passing of bucket and prefix to result writer of distributed map #32687

@Nicolas-SB

Description

@Nicolas-SB

Describe the feature

Analogous to #29409, which was solved in #31619, I would like to pass sfn.JsonPaths to sfn.ResultWriter(bucket=[...], prefix=[...]). Currently I have to know the bucket and can not read it dynamically from the input event.

Use Case

I want to define my state machine using CDK and use all features of the Distributed Map

Proposed Solution

Create a Result Writer that can take a string in place of a bucket.

Example cdk code in python:

my_map = sfn.DistributedMap(
    scope=self,
    id="my_id",
    item_reader=sfn.S3JsonItemReader(
        bucket_name_path=sfn.JsonPath.string_at("$.manifest_bucket"),
        key=sfn.JsonPath.string_at("$.input_manifest_key"),
    ),
    result_writer=sfn.ResultWriter(
        bucket_name_path=sfn.JsonPath.string_at("$.manifest_bucket"),
        prefix=sfn.JsonPath.string_at("$.distributed_map_result_prefix"),
    ),
)

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

2.173.2

Environment details (OS name and version, etc.)

macOS Sequoia 15.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions