-
Notifications
You must be signed in to change notification settings - Fork 100
feat(relay): snapshot testing for python integration tests #4877
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Just some high level questions:
I think ideally we have both of these capabilities before we start replacing uses. This may still be useful for some very large assertions though. |
It should be relatively easy to add, regular python asserts give a good diff of what the differences between two dicts are.
For inline snapshots it shouldn't be too hard to add it since you could just pass the comparison functions in the local snapshot but for file snapshots it would be more tricky since it would need to have a way to configure them upfront based on key name or something like that. |
After giving it more thought I'm not sure what the benefit would be if this could also do inline snapshots. While I agree that inline snapshots are nicer and easier to review, I think this could be useful for PII related snapshot tests where we can not only test if fields are properly scrubbed but also make sure that |
Adds snapshot testing in python integration tests for relay.
Dynamic values can be redacted to prevent flakyness (or enable snapshot testing in general).
For example,
event_id
get's automatically replaced by<event_id>
in the snapshot.The list is configurable through direct parameters, e.g.
relay_snapshot("json", additional_keys={"foo"})
.Keys need to match the full path, so
timestamp
will only match the root leveltimestamp
key but not the one inspans
(see snapshot in PR).