@aws-cdk/toolkit-lib: diff command result lacks changed stacks count

[cmaster11]

Describe the feature

When running the diff action, the result lacks information about the number of stacks with changes, despite the toolkit previously providing this information via IoHost.

The CDK CLI uses its formatter to calculate the diff, but other programmatic implementations can benefit from this precalculated info.

At https://github.com/aws/aws-cdk-cli/blob/52457cfc25dce58e8b3d1da8ff20c87cfd336034/packages/%40aws-cdk/toolkit-lib/lib/toolkit/toolkit.ts#L368C1-L371C8 we have the diff span end, which contains the formatted diff, but could also include the number of stacks with diffs. To figure out the number of stacks with diffs, you'd have to parse the log span message or re-do the whole calculation yourself.

I think we can benefit from:

1. Providing access to the currently private https://github.com/aws/aws-cdk-cli/blob/main/packages/%40aws-cdk/toolkit-lib/lib/api/io/private/messages.ts#L527 IO and SPAN enums, which would allow us to extract this info from the IoHost messages without having to hardcode the various IO codes (like CDK_TOOLKIT_I4001) in the host code.
2. Exposing the number of stacks with differences in the log span
3. Exposing the number of changed stacks info also as part of the diff function result, maybe as a boolean inside each stack result object (an overall hasChanges: true/false for each stack)

Diff log:

// Here we have the stack changes count, but it is provided in the message string only
[03:39:00.242] INFO (CDK/44572): ✨ Number of stacks with differences: 0 
    code: "CDK_TOOLKIT_I4001"
    data: {
      "duration": 9383,
      "formattedSecurityDiff": "",
      "formattedStackDiff": "Stack \u001b[1mExample-Stack\u001b[22m\n\u001b[32mThere were no differences\u001b[39m\n\u001b[32m\u001b[39m"
    }
    span: "055e1fcb-9f57-4f7f-b88b-e830883a8c1b"
    action: "diff"

// This is the result of the `.diff` command
// This needs to be re-parsed to figure out if there have been diffs
[03:39:00.243] INFO (44572): Diff result
    diffResult: {
      "Example-Stack": {
        "conditions": {
          "diffs": {}
        },
        "mappings": {
          "diffs": {}
        },
        "metadata": {
          "diffs": {}
        },
        "outputs": {
          "diffs": {}
        },
        "parameters": {
          "diffs": {}
        },
        "resources": {
          "diffs": {}
        },
        "unknown": {
          "diffs": {}
        },
        "iamChanges": {
          "statements": {
            "additions": [],
            "removals": [],
            "oldElements": [],
            "newElements": []
          },
          "managedPolicies": {
            "additions": [],
            "removals": [],
            "oldElements": [],
            "newElements": []
          },
          "ssoPermissionSets": {
            "additions": [],
            "removals": [],
            "oldElements": [],
            "newElements": []
          },
          "ssoAssignments": {
            "additions": [],
            "removals": [],
            "oldElements": [],
            "newElements": []
          },
          "ssoInstanceACAConfigs": {
            "additions": [],
            "removals": [],
            "oldElements": [],
            "newElements": []
          }
        },
        "securityGroupChanges": {
          "ingress": {
            "additions": [],
            "removals": [],
            "oldElements": [],
            "newElements": []
          },
          "egress": {
            "additions": [],
            "removals": [],
            "oldElements": [],
            "newElements": []
          }
        }
      }
    }

Use Case

When using the toolkit-lib programmatically, I need a way to tell the user whether we have differences or not in the stacks. I'd like to do that with already generated info instead of having to do the parsing again.

Proposed Solution

No response

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.195.0

Environment details (OS name and version, etc.)

Windows 11 64bit

[ykethan]

Hey @cmaster11, thank you for reaching out. Currently In the toolkit.ts file, the diff() method performs the following steps:

1. Calculates template differences for each stack
2. Formats the differences using a DiffFormatter
3. Retrieves both the formatted diff text and a count of stacks with changes:

for (const templateInfo of templateInfos) {
  const formatter = new DiffFormatter({
    ioHelper,
    templateInfo,
  });

  if (options.securityOnly) {
    // Security-only logic
    ...
  } else {
    const diff = formatter.formatStackDiff({
      strict,
      context: contextLines,
    });
    formattedStackDiff = diff.formattedDiff;
    diffs = diff.numStacksWithChanges;  // Here the count is calculated
  }
  appendObject(templateDiffs, formatter.diffs);
}

while this count is logged in the console output:

await diffSpan.end(`ⱽ Number of stacks with differences: ${diffs}`, {
  formattedSecurityDiff,
  formattedStackDiff,
});

It's not included in the return value:

return templateDiffs;

of the top, we may need to update the return to expose this:

await diffSpan.end(ⱽ Number of stacks with differences: ${diffs}, {
formattedSecurityDiff,
formattedStackDiff,
});

// Return both the template diffs and the number of stacks with changes
return {
...templateDiffs,
numStacksWithChanges: diffs,
};

we may need to dd this interface(in `packages/@aws-cdk/toolkit-lib/lib/actions/diff/index.ts` ):

```typescript
export interface DiffResult {
  /** Each stack with template differences */
  [stackName: string]: TemplateDiff;
  /** The number of stacks with differences */
  numStacksWithChanges: number;
}

And update the return type of the diff() method to use this interface.

Marking as P2 feature request. We greatly appreciate community contributions! If you or anyone else is interested in implementing this, we'd be happy to review a Pull Request. Please check our contribution guidelines.

[mrgrain]

Split out 3 as #544

[mrgrain]

@cmaster11 The return of diff is a map of stack names to TemplateDiff objects. I appreciate this is currently under documented, but the objet has an isEmpty property (it's a getter, hence it's not logged by default).

I feel this is easy enough:

const theDiff = await cdk.diff(...)
console.log("Number of stacks with differences:", Object.values(theDiff).filter(d => !d.isEmpty).length);

WDYT?

[cmaster11]

@mrgrain Ahhhhhh I hadn't seen that one, and also the differenceCount method can be very useful!

Yeah, works great, thank you!

[mrgrain]

Awesome! I'm just addressing the remaining feedback in a PR because the whole formatted output is kind of broken. This will include the count in the span end event as well then.