Skip to content

Commit 339bd61

Browse files
committed
feat: Make Vitest runs more token efficient when run through a coding assistant.
1 parent dfb2159 commit 339bd61

18 files changed

Lines changed: 166 additions & 15 deletions

File tree

docs/config/reporters.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@ Note that the [coverage](/guide/coverage) feature uses a different [`coverage.re
4242
- [`tap-flat`](/guide/reporters#tap-flat-reporter)
4343
- [`hanging-process`](/guide/reporters#hanging-process-reporter)
4444
- [`github-actions`](/guide/reporters#github-actions-reporter)
45+
- [`agent`](/guide/reporters#agent-reporter)
4546
- [`blob`](/guide/reporters#blob-reporter)
4647
4748
## Example

docs/guide/browser/visual-regression-testing.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -606,7 +606,6 @@ screenshots should use the service.
606606

607607
The cleanest approach is using [Test Projects](/guide/projects):
608608

609-
610609
```ts [vitest.config.ts]
611610
import { env } from 'node:process'
612611
import { defineConfig } from 'vitest/config'
@@ -663,7 +662,6 @@ export default defineConfig({
663662
})
664663
```
665664

666-
667665
Follow the [official guide to create a Playwright Workspace](https://learn.microsoft.com/en-us/azure/app-testing/playwright-workspaces/quickstart-run-end-to-end-tests?tabs=playwrightcli&pivots=playwright-test-runner#create-a-workspace).
668666

669667
Once your workspace is created, configure Vitest to use it:

docs/guide/cli-generated.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ Hide logs for skipped tests
102102
- **CLI:** `--reporter <name>`
103103
- **Config:** [reporters](/config/reporters)
104104

105-
Specify reporters (default, blob, verbose, dot, json, tap, tap-flat, junit, tree, hanging-process, github-actions)
105+
Specify reporters (default, agent, blob, verbose, dot, json, tap, tap-flat, junit, tree, hanging-process, github-actions)
106106

107107
### outputFile
108108

docs/guide/cli.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -237,4 +237,3 @@ Merges every blob report located in the specified folder (`.vitest-reports` by d
237237
```sh
238238
vitest --merge-reports --reporter=junit
239239
```
240-

docs/guide/reporters.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,10 @@ This example will write separate JSON and XML reports as well as printing a verb
9898

9999
By default (i.e. if no reporter is specified), Vitest will display summary of running tests and their status at the bottom. Once a suite passes, its status will be reported on top of the summary.
100100

101+
::: tip
102+
When Vitest detects it is running inside an AI coding agent, the [`agent`](#agent-reporter) reporter is used instead to reduce output and minimize token usage. You can override this by explicitly configuring the [`reporters`](/config/reporters) option.
103+
:::
104+
101105
You can disable the summary by configuring the reporter:
102106

103107
:::code-group
@@ -637,6 +641,26 @@ export default defineConfig({
637641
})
638642
```
639643

644+
### Agent Reporter
645+
646+
Outputs a minimal report optimized for AI coding assistants and LLM-based workflows. Only failed tests and their error messages are displayed. Console logs from passing tests and the summary section are suppressed to reduce token usage.
647+
648+
This reporter is automatically enabled when no `reporters` option is configured and Vitest detects it is running inside an AI coding agent. If you configure custom reporters, you can explicitly add `agent`:
649+
650+
:::code-group
651+
```bash [CLI]
652+
npx vitest --reporter=agent
653+
```
654+
655+
```ts [vitest.config.ts]
656+
export default defineConfig({
657+
test: {
658+
reporters: ['agent']
659+
},
660+
})
661+
```
662+
:::
663+
640664
### Blob Reporter
641665

642666
Stores test results on the machine so they can be later merged using [`--merge-reports`](/guide/cli#merge-reports) command.

packages/vitest/src/node/config/resolveConfig.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ import {
2323
defaultPort,
2424
} from '../../constants'
2525
import { benchmarkConfigDefaults, configDefaults } from '../../defaults'
26-
import { isCI, stdProvider } from '../../utils/env'
26+
import { isAgent, isCI, stdProvider } from '../../utils/env'
2727
import { getWorkersCountByPercentage } from '../../utils/workers'
2828
import { BaseSequencer } from '../sequencers/BaseSequencer'
2929
import { RandomSequencer } from '../sequencers/RandomSequencer'
@@ -729,7 +729,7 @@ export function resolveConfig(
729729
}
730730

731731
if (!resolved.reporters.length) {
732-
resolved.reporters.push(['default', {}])
732+
resolved.reporters.push([isAgent ? 'agent' : 'default', {}])
733733

734734
// also enable github-actions reporter as a default
735735
if (process.env.GITHUB_ACTIONS === 'true') {
Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
import type { Task } from '@vitest/runner'
2+
import type { UserConsoleLog } from '../../types/general'
3+
import type { TestSpecification } from '../test-specification'
4+
import type { DefaultReporterOptions } from './default'
5+
import type { TestCase, TestModule, TestModuleState, TestResult } from './reported-tasks'
6+
import { DefaultReporter } from './default'
7+
8+
export class AgentReporter extends DefaultReporter {
9+
renderSucceed = false
10+
11+
constructor(options: DefaultReporterOptions = {}) {
12+
super({ ...options, summary: false })
13+
}
14+
15+
onTestRunStart(specifications: ReadonlyArray<TestSpecification>): void {
16+
super.onTestRunStart(specifications)
17+
this.renderSucceed = false
18+
}
19+
20+
protected logFailedTask(task: Task): void {
21+
for (const log of task.logs || []) {
22+
this.onUserConsoleLog(log, 'failed')
23+
}
24+
}
25+
26+
protected printTestModule(testModule: TestModule): void {
27+
if (testModule.state() !== 'failed') {
28+
return
29+
}
30+
super.printTestModule(testModule)
31+
}
32+
33+
protected printTestCase(moduleState: TestModuleState, test: TestCase): void {
34+
const testResult = test.result()
35+
if (testResult.state === 'failed') {
36+
super.printTestCase(moduleState, test)
37+
}
38+
}
39+
40+
shouldLog(log: UserConsoleLog, taskState?: TestResult['state']): boolean {
41+
if (taskState !== 'failed') {
42+
return false
43+
}
44+
return super.shouldLog(log, taskState)
45+
}
46+
}

packages/vitest/src/node/reporters/base.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,7 @@ export abstract class BaseReporter implements Reporter {
117117
this.printTestModule(testModule)
118118
}
119119

120-
private logFailedTask(task: Task) {
120+
protected logFailedTask(task: Task): void {
121121
if (this.ctx.config.silent === 'passed-only') {
122122
for (const log of task.logs || []) {
123123
this.onUserConsoleLog(log, 'failed')

packages/vitest/src/node/reporters/index.ts

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import type { GithubActionsReporterOptions } from './github-actions'
66
import type { HTMLOptions } from './html'
77
import type { JsonOptions } from './json'
88
import type { JUnitOptions } from './junit'
9+
import { AgentReporter } from './agent'
910
import { BlobReporter } from './blob'
1011
import { DefaultReporter } from './default'
1112
import { DotReporter } from './dot'
@@ -19,6 +20,7 @@ import { TreeReporter } from './tree'
1920
import { VerboseReporter } from './verbose'
2021

2122
export {
23+
AgentReporter,
2224
DefaultReporter,
2325
DotReporter,
2426
GithubActionsReporter,
@@ -46,6 +48,7 @@ export type {
4648

4749
export const ReportersMap = {
4850
'default': DefaultReporter as typeof DefaultReporter,
51+
'agent': AgentReporter as typeof AgentReporter,
4952
'blob': BlobReporter as typeof BlobReporter,
5053
'verbose': VerboseReporter as typeof VerboseReporter,
5154
'dot': DotReporter as typeof DotReporter,
@@ -62,6 +65,7 @@ export type BuiltinReporters = keyof typeof ReportersMap
6265

6366
export interface BuiltinReporterOptions {
6467
'default': DefaultReporterOptions
68+
'agent': DefaultReporterOptions
6569
'verbose': DefaultReporterOptions
6670
'dot': BaseOptions
6771
'tree': BaseOptions

packages/vitest/src/public/node.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ export { VmThreadsPoolWorker } from '../node/pools/workers/vmThreadsWorker'
4343
export type { SerializedTestProject, TestProject } from '../node/project'
4444

4545
export {
46+
AgentReporter,
4647
BenchmarkReporter,
4748
BenchmarkReportsMap,
4849
DefaultReporter,

0 commit comments

Comments
 (0)