Recording issues from non-test targets

[mbrandonw]

Motivation

Recording issues from non-test targets

It would be nice if there was a way to record issues from non-test targets. There are already signs from the ecosystem that this is a useful thing since as of Xcode 26 any runtime issues emitted (e.g. thread sanitizer warnings) will be reported as test failures when emitted in a testing context. This is an example of non-test code triggering test failures.

We have also maintained an open source library that provides this tool (reportIssue) and it’s used throughout our other libraries and is currently cloned over a quarter of a million times each week. One can use reportIssue anywhere in their app code and when triggered in a testing context it causes a test failure, and when triggered in a non-testing context it emits a message in a customizable way (print to logs, send to event stream backend, etc…).

Some examples of how we use this tool throughout libraries:

• In our dependencies library if you resolve a dependency in tests that has not been overridden, a test failure is emitted, and if you resolve a dependency in production that has not been overridden, we emit a warning.
• In our back-port of Observation to iOS <17 we report an issue when the tools are used incorrectly, which allows people to catch problems if they are snapshot testing their views, and allows us to write tests that we correctly emit these warnings because they turn into test failures in a test context.
• There is a large class of errors that are programmer errors and cannot be fixed by the user. Preconditions are one way of handling such errors, but they are very disruptive, both when running an app and running tests. reportIssue allows you to capture such errors in a way that is non-obtrusive when running the app and help you catch potential problems when testing.
  • Here are a bunch of examples from one of our libraries. Many of these could be turned into preconditions or throwing methods, but the ergonomics of that are just not very nice,.
  • Here is an example of this in actual app code, not library code. A SQL query can fail, but if it does it would only be because we constructed an invalid query, and so reporting the issue is more appropriate than showing the user an error.

This is just a few examples. There are a lot more.

Proposed solution

Our swift-issue-reporting shows one possible solution to this problem. The implementation details are not important, but having some kind of function that is available from non-test targets that when invoked in a testing context triggers a test failure would be enough.

This function should probably throw when it is invoked in a non-testing context, which would allow us to catch that error and report the issue in a different manner:

do {
  // Name of function TBD
  try reportIssue("This should not happen.")
} catch {
  // When not in testing context print to console
  print("This should not happen")
}

It would also be useful for this function to take file, line, etc. so that it could be used from library code.

Alternatives considered

No response

Additional information

No response

[grynspan]

It is likely that the API for this functionality would live in the stdlib or in a separate package. That API might call an implementation that's in this repo though.