Remove snapshot tests and provide a better test framework

[jaredallard]

The current template testing framework is very rudimentary; we should replace it with something that supports running validators and basic snapshot tests. We should also design it with the expectation that we'll change it again later if there's more community usage of the tool.

[malept]

I will have a larger set of requirements later from the existing Outreach tests, but for now, one thing that I've wanted for a while is a way to test module hooks, both in the module where it's defined and in consumer modules.

For example, say I have a mise.toml template:

[tools]
{{- range stencil.GetModuleHook "miseTools" }}
{{ . | toToml }}
{{- end }}

In this stencil module, I'd want to test that any defined module hooks get rendered correctly.

But in a consumer stencil module, I might do this:

{{- file.Skip "Virtual file for mise.toml module hooks" }}

{{- define "miseFormatters" }}
{{- if stencil.Arg "formatters.dprint" }}
- dprint: latest
{{- else }}
- "npm:prettier": "3"
{{- end }}
{{- end }}

{{- stencil.AddToModuleHook "example.com/stencil-base" "miseTools" ((stencil.ApplyTemplate "miseFormatters") | fromYaml)

So I'd want to test with both values of formatters.dprint.

---

I have not looked into what the feasibility is of this feature request, but at least the use case is now documented, with potentially a design decision about it in the future.

[malept]

Requirements

• The ability to mock (built-in) Stencil variables, e.g. Git.DefaultBranch, that require complex prerequisites such as a git repository with a remote
• Assert that a file was skipped
• Assert that a file was deleted
• When rendering Stencil templates on the real filesystem, make sure it's done in a temporary directory ¹
• Assert that a rendered file has the given mode (e.g., it's executable)
• Allow the entire service manifest to be customizable for a given test²
• If a snapshot and rendered file differ, the test should fail and:
  • in CI, show a diff
  • in development, update the snapshot
• The ability to set an arbitrary file.Block value, in the case that template logic uses it
• A way to test the files in batches
• Determine how to get template coverage

Other thoughts

I wonder how useful it would be to be able to write rendered files to an in-memory filesystem, and then use those file handles in Go for validation.

Footnotes

1. The comment in this test seems to indicate that Stencil v1 does not do this. ↩

2. Comment indicating that it was not implemented in Stencil v1 ↩