(stepfunctions): Add native support for DistributedMap

[ayush987goyal]

Describe the feature

https://docs.aws.amazon.com/step-functions/latest/dg/concepts-orchestrate-large-scale-parallel-workloads.html

Currently there is a workaround this this but we should have inbuilt support to appropriately define ItemReader, ResultWriter etc.

Use Case

Ability to use the new feature.

Proposed Solution

No response

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.53.0

Environment details (OS name and version, etc.)

mac

[peterwoodworth]

Thanks for the feature request!

Currently there is a workaround this this

Could you share an example workaround please in case anyone else is interested?

[ayush987goyal]

Here is an example:

const dummyMap = new Map(this, "DummyMap");
dummyMap.iterator(someIterationState);

const distributedMap = new CustomState(this, "DistributedMap", {
    stateJson: {
        Type: "Map",
        MaxConcurrency: 100,
        ItemReader: {
            Resource: "arn:aws:states:::s3:getObject",
            ReaderConfig: {
                InputType: "CSV",
                CSVHeaderLocation: "FIRST_ROW",
            },
            Parameters: {
                Bucket: "some-bucket-name",
                "Key.$": "$.my_s3_key",
            },
        },
        ItemSelector: {
            "item.$": "$$.Map.Item.Value",
        },
        ItemProcessor: {
            ...(dummyMap.toStateJson() as any).Iterator,
            ProcessorConfig: {
                Mode: "DISTRIBUTED",
                ExecutionType: "STANDARD",
            },
        },
        ResultWriter: {
            Resource: "arn:aws:states:::s3:putObject",
            Parameters: {
                Bucket: "some-bucket-name",
                Prefix: "process_output",
            },
        },
        ResultPath: "$.map_result",
    },
});