Capture code snippet from diagnostic compiler output #9362
Conversation
* Built tentative test class SwiftBuildSystemOutputParser to handle the compiler output specifically * Added a handleDiagnostic method to possibly substitute the emitEvent local scope implementation of handling a SwiftBuildMessage diagnostic
* the flag `appendToOutputStream` helps us to determine whether a diagnostic is to be emitted or whether we'll be emitting the compiler output via OutputInfo * separate the emitEvent method into the SwiftBuildSystemMessageHandler
bripeticca
force-pushed
the
swb/diagnosticcodesnippet
branch
from
a20f020 to
f3aaabf
Compare
bripeticca
force-pushed
the
swb/diagnosticcodesnippet
branch
from
1dcaaeb to
c48e606
Compare
|
@swift-ci please test |
bripeticca
changed the title
bripeticca
requested review from
bkhouri,
cmcgee1024,
daveyc123,
dschaefer2,
jakepetroules,
plemarquand and
rconnell9
as code owners
owenv
left a comment
owenv
left a comment
There was a problem hiding this comment.
Thie generally lgtm but I have some concerns about the regex-based parsing when we emit the textual compiler output.
- Perf - It's important this is fast so that it doesn't block the end of the build if a command produces huge quantities of output. It's hard to say if this will be a real issue without some testing
- We're re-parsing information which we're already getting from the compiler in structured form. I see the appeal of not reporting a diagnostic twice if multiple compile jobs report it though
I also pointed this out inline, though I'm not sure why the re-parsing relates to deduplication? We should be able to deduplicate diagnostics whether we parse them again or use the existing ones. |
(@jakepetroules @owenv -- tagging for visibility, GitHub notifications can be weird) Re-parsing doesn't affect the de-duplication -- when tracking the data buffer per task, I also track whether we've emitted the associated output for a given task (using its signature) and guard against this before emitting for that task again. We only go down this path (emitting for a given task) once we've received the task completed event. I mentioned this inline as well but for visibility: the re-parsing just addresses the fact that for a given task signature, we have an accumulated data buffer that contains all possible diagnostic messages coming from the compiler. I find that the user ergonomics aren't great when simply emitting the entire string blob through the observability scope, since these diagnostics can have varying severities and we'll have to decide up-front which severity to choose to emit the entire string of all diagnostics. I do also maintain a list of the Perhaps some more discussion is needed here. 😄 |
* Remove taskStarted outputStream emissions * Propagate exception if unable to find path
|
@swift-ci please test |
|
@swift-ci please test windows |
|
Errors seem unrelated to this change, re-triggering tests @swift-ci please test linux |
|
@swift-ci please test macOS |
|
@swift-ci please test linux |
|
@PhantomInTheWire It looks like this change will definitely affect possible solutions for #8877 , but I am skeptical that it will resolve it entirely. If you're interested in taking a look at it still, I would highly recommend doing so once this PR is merged 😄 |
|
sure i'll take a look once this is merged, @bripeticca |
…t stream Some minor fixes to how task output is handled in the SwiftBuildSystemMessageHandler.
|
@swift-ci test |
|
@swift-ci please test |
|
@swift-ci test windows |
|
@swift-ci test |
|
@swift-ci test windows |
| } | ||
|
|
||
| @Test | ||
| func testManyDiagnosticsReported() throws { |
There was a problem hiding this comment.
praise: When it comes to "arrays", I tend to want to validate the empty, 1, 2 and many items. It's great to see the 0, 1 and many diagnostics tests added. Could I ask to cover case where 2 diagnostics are emitted?
|
|
||
| try expectDiagnostics(observability.diagnostics) { result in | ||
| result.check(diagnostic: "Simple diagnostic", severity: .error) | ||
| result.check(diagnostic: "Another diagnostic", severity: .debug) |
There was a problem hiding this comment.
question: maybe I'm missing something, but is this diagnosing a .note kind? if so, why do we expect it to be debug?
| ) | ||
|
|
||
| let events: [SwiftBuildMessage] = [ | ||
| .taskStartedInfo(taskID: 1, taskSignature: "simple-diagnostic"), |
There was a problem hiding this comment.
question: do events need to start and end with the .taskStartedInfo(...) and .taskCompleteInfo(...) for these tests?
There was a problem hiding this comment.
Yes -- the way the message handler works is that it will track when a task is completed to emit the diagnostics/output associated with that task. A task started event is needed before we handle a task completed event, otherwise the handler will throw an error since shouldn't be possible.
| let simpleDiagnosticString: String = "[error]: Simple diagnostic\n" | ||
| let simpleOutputInfo: SwiftBuildMessage = .outputInfo( | ||
| data: data(simpleDiagnosticString), | ||
| locationContext: .task(taskID: 1, targetID: 1), |
There was a problem hiding this comment.
question: do the values of taskID and targetID need to match between all test SwiftBuildMessage that are in the events array? If so, consider storing the values in a let and using said variable name in various places to make the test more readable.
There was a problem hiding this comment.
They would need to match if I'm trying to mock out a stream of events for which I want to track its progress per ID. I'll refactor this in a follow-up PR!
| } | ||
|
|
||
| @Test | ||
| func testDiagnosticOutputWhenOnlyWarnings() throws { |
There was a problem hiding this comment.
suggestion: should we add additional test that verify other diagnostics? e.g.: only .note, .debug and all other possible diagnostic kind?
| } | ||
|
|
||
| /// SwiftBuildMessage.DiagnosticInfo | ||
| package static func diagnosticInfo( |
There was a problem hiding this comment.
suggestion: rename this to diagnostic or diagnosticData as Info may give a false sense that were are creating an info level diagnostics message/event
| "log didn't contain expected linker diagnostics. stderr: '\(stderr)", | ||
| ) | ||
| #expect( | ||
| !stdout.contains(searchPathRegex), |
There was a problem hiding this comment.
issue (possibly-blocking): this should match the string from stderr expectation?
| #expect(!stderr.contains(RelativePath("Sources/MyLibrary/library.foo").pathString)) | ||
| let libraryFooPath = RelativePath("Sources/MyLibrary/library.foo").pathString | ||
| #expect(!stderr.components(separatedBy: "\n").contains { $0.contains("warning: ") && $0.contains(libraryFooPath) }) | ||
| if data.buildSystem == .native { |
There was a problem hiding this comment.
suggestion: It's not related to your change, but since you are modifying this file, can we convert the if data.buildSystem == .native to a switch call, by explicitly adding a case call for each value (ie: do not use default)? this will ensure we handle "new build systems" should we ever add one in the future.
| case .swiftbuild: | ||
| #expect(stdout.contains("MySourceGenBuildTool-product"), "stdout:\n\(stdout)\nstderr:\n\(stderr)") | ||
| #expect(stderr.contains("Creating foo.swift from foo.dat"), "stdout:\n\(stdout)\nstderr:\n\(stderr)") | ||
| #expect(stdout.contains("Creating foo.swift from foo.dat"), "stdout:\n\(stdout)\nstderr:\n\(stderr)") |
There was a problem hiding this comment.
praise: this is great to see!
bkhouri
dismissed
their stale review
The reason for blocking this (missing tests) are included in the PR.
6 participants
This PR refactors diagnostic handling in the Swift build system by introducing a dedicated message handler and per-task output buffering to properly parse and emit compiler diagnostics individually.
Key Changes
SwiftBuildSystemMessageHandler
SwiftBuildMessageevents from the build operationPer-Task Data Buffering
taskDataBufferstruct inBuildStateto capture compiler output per task signatureTaskDataBufferstruct allows for usingLocationContextorLocationContext2as a subscript key to fetch the appropriate data buffer for a task, defaulting to the global buffer if no associated task or target can be identified..outputmessages arrivePer-Task Diagnostic Buffering
diagnosticBufferproperty to theBuildStateto track diagnostics to emit once we receive ataskCompleteeventEmittedTasks
Test Suite
SwiftBuildSystemMessageHandlerTests
SwiftBuildMessageinfo structs that are exposed for testing purposes only