[terraform test] Support expect_failures of resouces inside local module in run blocks

[alexis-renard]

Terraform Version

Terraform v1.7.3

Use Cases

Hello,

We would like to expect_failure from a local module call done in a run block with terraform test. You can find some context about our issue down below.

The issue

It seems that resources, when placed inside a local module, are not handled by the expect_failures of the run block. Either we missed something and that is possible, either that would be a nice thing to have to be able to test our module with the integration of its own local modules.

The terraform test output

  run "check_precondition_ssm_parameter_encryption"... fail
╷
│ Error: Invalid `expect_failures` reference
│ 
│   on tests/main.tftest.hcl line 22, in run "check_precondition_ssm_parameter_encryption":
│   22:     module.workshop_participant_users.aws_ssm_parameter.example,
│ 
│ You cannot expect failures from module.workshop_participant_users.aws_ssm_parameter. You can only expect failures from checkable objects such as input variables, output values,
│ check blocks, managed resources and data sources.

Files content

# file : tests/main.tftest.hcl

run "check_precondition_ssm_parameter_encryption" {
  command = plan

  # [...]

  expect_failures = [
    module.workshop_participant_users.aws_ssm_parameter.example,
  ]
}

# file : main.tf

module "workshop_users" {
  source = "./modules/users"

  kms_key_id                    = var.create_kms_key ? try(resource.aws_kms_key.backend[0].arn, "") : ""
}

# file : modules/users/main.tf

resource "aws_ssm_parameter" "example" {

  # [...]

  key_id      = var.kms_key_id

  lifecycle {
    precondition {
      condition     = var.kms_key_id != ""
      error_message = "A key must be created to encrypt the ssm parameter."
    }
  }
}

Any ideas on that ?

Attempted Solutions

We have tried with the expecting failure on the whole module scope as well as for other resources inside the local module.

Proposal

No response

References

We are not sure whether or not it is also the case for the registry modules as well as the local ones tested here.

[liamcervante]

Hi @alexis-renard, thanks for filing this! It's an interesting idea, and I can confirm that this isn't currently supported.

Currently, Terraform Test does respect the encapsulation of modules much like regular Terraform. For example, in regular Terraform you can't refer to attributes of resources within child modules. You have to expose the relevant values through outputs. Terraform Test has the same restrictions, which is to say it's not aware of the inner workings of child modules and so can't validate anything within child modules either in assertions or expect_failures blocks. I will also add that the error message isn't doing a very good job here either. You likely are referencing a checkable object within the module, but it's the scope that is the problem rather than the type of thing being referenced which isn't made clear.

A purists view here would be to say that you should write tests for the child modules directly, and within those you can validate the expected failures are working as expected. Then when it comes to the parent modules you shouldn't need to worry about testing those edge cases - you can be confident the child modules will behave as expected.

However, we do have a longer term project to look at this encapsulation closer. For example, we want to add the ability to assert whether a resource is being created/updated/deleted/replaced rather than simply looking at the final state of a resource (#34500). For this functionality it is entirely reasonable that users would want to see what is happening to resources within child modules (as their infrastructure may, for example, rely on a nested resource being simply updated and not replaced). So, I think in the context of revisiting the scope for that functionality, this request is something that we can likely get working at the same time without too much trouble. But, it will have to wait for that work to be prioritised.

Thanks again for the request!

[alexis-renard]

Hey @liamcervante, thanks for the confirmation and the detailed insights.

For now, I think we will then opt for creating the test files inside the local module and adapt our ci to trigger it specifying the path of the local modules.

Note: purist or not, it would still be nice to be able to test the entire workflow within the root module, including how it interacts with its own submodules. The idea is to validate that "the local submodules are well integrated with the root module" (using tests of at root module level calling the sub modules based on different root input variables) more than validating that "the local submodules work as intended when called them separately" (using tests at the sub module level)

This is not an urgent matter for us, even though it would have been nice to have something specified in the doc, clearly stating that modules resources can not be tested for now. It would have saved us some research and test time I guess :)

Anyhow : thanks for the confirmation that it is not available at the time, we'll stay tuned for this one !

[davidlbyrne]

Hey @alexis-renard, Thank you for your explanation on this issue. I just ran into a similar issue where I am writing a module that wraps another module. We do this often to build in our sane default on top of existing modules. However, currently I'm limited in what I can validate by what the underlining module exposes in the outputs. This is quit frustrating because I can open my state file and see the resources the module is creating and their properties, but I can't write automated test for it. (using terraform native functionality). Please, any work that could be done to expose the resources and attributes for the purpose of writing good test would be extremely useful and greatly appreciated. +1 +1 +1