Skip to content

PROJECT-KNOWLEDGE.md

Latest commit

 

History

History
413 lines (352 loc) · 31 KB

PROJECT-KNOWLEDGE.md

File metadata and controls

413 lines (352 loc) · 31 KB

SPECTRA Project Knowledge

What is SPECTRA?

SPECTRA is an AI-native test case generation and execution framework. It reads product documentation, extracts testable behaviors, generates structured manual test cases (as Markdown files with YAML frontmatter), executes them through an MCP server with AI agent orchestration, and produces coverage reports and dashboards.

Architecture Overview

User → CLI (spectra) → AI Provider (Copilot SDK) → Test Files (Markdown/YAML)
User → VS Code Copilot Chat → SKILL/Agent → CLI commands → Results
User → VS Code Copilot Chat → MCP Server → Test Execution → Reports

Three Main Components

Component Purpose Package
Spectra.CLI CLI tool for generation, coverage, dashboard, validation Spectra.CLI NuGet (global tool: spectra)
Spectra.MCP MCP execution server for AI-driven test execution Spectra.MCP NuGet (global tool: spectra-mcp)
Spectra.Core Shared models, parsers, coverage analyzers Referenced by CLI and MCP

Supporting Components

Component Purpose
dashboard-site/ Static HTML/JS dashboard template with D3.js visualizations
SKILLs (12 files) VS Code Copilot Chat integration — each SKILL wraps CLI commands or guides users
Agents (2 files) Copilot Chat agent prompts for generation and execution workflows

Technology Stack

  • Language: C# 12, .NET 8+
  • AI Runtime: GitHub Copilot SDK (sole runtime — supports github-models, azure-openai, azure-anthropic, openai, anthropic providers)
  • CLI Framework: System.CommandLine
  • Terminal UX: Spectre.Console
  • Serialization: System.Text.Json (JSON), YamlDotNet (YAML)
  • CSV Parsing: CsvHelper
  • MCP Server: ASP.NET Core
  • State Storage: Microsoft.Data.Sqlite (execution state in .execution/spectra.db)
  • Testing: xUnit
  • Dashboard: Vanilla JS + D3.js (treemap/donut charts)

Project Structure

src/
├── Spectra.CLI/
│   ├── Commands/           # CLI command handlers
│   │   ├── Analyze/        # Coverage analysis + criteria extraction/import/list
│   │   ├── Dashboard/      # Dashboard generation
│   │   ├── Docs/           # Documentation index management
│   │   ├── Generate/       # Test generation (direct + interactive modes)
│   │   ├── Init/           # Project initialization
│   │   └── Update/         # Test update with classification
│   ├── Agent/Copilot/      # AI provider integration
│   │   ├── CopilotGenerationAgent  # Test generation via Copilot SDK
│   │   ├── CopilotCritic          # Grounding verification
│   │   ├── CriteriaExtractor      # Per-document criteria extraction
│   │   └── BehaviorAnalyzer       # Documentation behavior analysis
│   ├── Skills/Content/     # Bundled SKILL and agent markdown files
│   ├── Progress/           # Live progress HTML page writer
│   ├── Results/            # Typed JSON result models per command
│   ├── Coverage/           # Coverage report writer
│   ├── Dashboard/          # Dashboard data collection + generation
│   └── IO/                 # File writers for test cases
├── Spectra.Core/
│   ├── Models/             # All data models
│   │   ├── Coverage/       # AcceptanceCriterion, CriteriaIndex, UnifiedCoverageReport
│   │   ├── Dashboard/      # DashboardData, CoverageSummaryData
│   │   ├── Execution/      # Run, TestResult, McpToolResponse
│   │   └── Config/         # SpectraConfig, CoverageConfig
│   ├── Parsing/            # YAML/Markdown parsers, criteria readers/writers, importers
│   ├── Coverage/           # Coverage analyzers (documentation, criteria, automation)
│   └── Update/             # TestClassifier for test updates
├── Spectra.MCP/
│   ├── Tools/              # MCP tool implementations (20+ tools)
│   ├── Execution/          # ExecutionEngine, TestQueue, StateMachine
│   ├── Reports/            # HTML/Markdown/JSON report generation
│   └── Storage/            # SQLite repositories
└── Spectra.GitHub/         # GitHub integration (future)

dashboard-site/             # Static dashboard template
tests/                      # xUnit test projects (1280+ tests)

CLI Commands

Test Generation

spectra ai generate --suite {name}              # Interactive session
spectra ai generate --suite {name} --count 15   # Direct mode with count
spectra ai generate --suite {name} --focus "negative, high priority"  # Filtered
spectra ai generate --suite {name} --analyze-only  # Analysis only
spectra ai generate --suite {name} --skip-critic   # Skip verification
spectra ai generate --suite {name} --no-interaction --output-format json  # CI mode

Acceptance Criteria

spectra ai analyze --extract-criteria            # Incremental extraction from docs
spectra ai analyze --extract-criteria --force    # Full re-extraction
spectra ai analyze --import-criteria ./file.csv  # Import from CSV/YAML/JSON
spectra ai analyze --list-criteria               # List all criteria
spectra ai analyze --list-criteria --component checkout --priority high  # Filtered

Coverage & Dashboard

spectra ai analyze --coverage --auto-link        # Unified coverage report
spectra dashboard --output ./site                # Generate dashboard

Other Commands

spectra ai update --suite {name}                 # Update tests (classify changes)
spectra validate                                 # Validate test files
spectra docs index                               # Rebuild documentation index
spectra list / spectra show {id}                 # Browse tests
spectra init                                     # Initialize project
spectra update-skills                            # Sync bundled SKILL files

Test Case Format

Test cases are Markdown files with YAML frontmatter in test-cases/{suite}/:

---
id: TC-101
title: "Verify login with valid credentials"
priority: high
tags: [authentication, login]
component: auth
criteria: [AC-AUTH-001, AC-AUTH-002]     # Linked acceptance criteria
automated_by: [LoginTests.cs]
requirements: [REQ-001]                  # Legacy field (deprecated)
grounding:
  verdict: grounded
  source: docs/auth.md
---

## Steps
1. Navigate to login page
2. Enter valid email and password
3. Click Sign In

## Expected Result
User is redirected to dashboard with welcome message

Acceptance Criteria System

Architecture

docs/
├── feature1.md                          # Source documentation
├── feature2.md
└── criteria/
    ├── _criteria_index.yaml             # Master index (auto-generated)
    ├── feature1.criteria.yaml           # Per-document criteria
    ├── feature2.criteria.yaml
    └── imported/
        └── jira-sprint-42.criteria.yaml # Imported from external sources

Criterion Format

criteria:
  - id: AC-CHECKOUT-001
    text: "System MUST validate IBAN format before submitting payment"
    rfc2119: MUST
    source_doc: docs/checkout.md
    source_section: "Payment Validation"
    component: checkout
    priority: high
    tags: [payment, validation]

Key Features

  • Per-document extraction: Each doc processed independently (no truncation)
  • SHA-256 incremental: Only re-extracts changed documents
  • Import: CSV (Jira/ADO auto-column-detection), YAML, JSON
  • AI splitting: Compound criteria split into individual entries
  • RFC 2119 normalization: Informal language → MUST/SHOULD/MAY
  • Merge/Replace: --merge (default) matches by ID/source, --replace overwrites target file only
  • Resilient extraction (Spec 047): typed CriteriaExtractionResult distinguishes genuine extraction (cacheable) from parse-class / empty-response failures (not cacheable, retried up to 2 attempts with 1.5 s backoff). The cache hash is recorded only on a genuine result, so a transient AI hiccup no longer poisons the cache. docs index uses a 2-minute per-document deadline (was a single 60-second corpus deadline) — one slow doc no longer aborts the whole run.
  • Coverage guards (Spec 048): CriteriaSource gains an additive outcome field (default "extracted"; legacy entries default by missing-key). docs index emits a non-blocking corpus-zero warning + criteria_warning JSON field when it indexed docs but extracted 0 criteria. ai generate attaches a non-blocking notes entry to its result when zero criteria match the target suite by component / source-doc / file-name. Both guards are output-only — no prompts, no exit-code changes. LoadCriteriaContextAsync was refactored to return a CriteriaContextResult record exposing the suite-match count (the previous string? return collapsed "no match" and "fell back to all criteria").
  • From-description index parity (Spec 049, v1.52.3): introduces Spectra.CLI.IO.TestPersistenceService as the single write-file + regenerate-index entry point for generation flows. ExecuteFromDescriptionAsync (was: write file only, never touched _index.json — making the test invisible to every MCP discovery tool), ExecuteDirectModeAsync (batch), and ExecuteInteractiveModeAsync all route through PersistAsync(...). Eliminates the bug where --from-description tests were silently missing from find_test_cases, start_execution_run, list_available_suites, and saved-selection counts. Downstream: the "high-priority filter from a suite doesn't work for from-description tests" symptom resolves because the filter code was always correct — it was reading from an index the from-description path never populated. spectra index --rebuild is the backfill for pre-fix workspaces (parses .md files of record and regenerates indexes). Out of scope: BatchWriteTestsTool (AI-discretion tool) and UpdateHandler.ApplyChangesAsync (update, not generation).
  • From-description criteria injection (Spec 050): fixes the from-description flow discarding its loaded criteria. UserDescribedGenerator.GenerateAsync was passing criteriaContext: null to agent.GenerateTestsAsync, so the MANDATORY "you MUST map" block (gated on a non-empty criteriaContext in CopilotGenerationAgent.BuildFullPrompt) never reached the model — the criteria frontmatter field was inconsistently populated. The fix forwards the loaded value (normalizing whitespace-only to null) and removes the now-redundant loose ## Related Acceptance Criteria section from BuildPrompt so criteria appear once, with proper framing. A small test seam (optional agentFactory delegate on GenerateAsync, defaulting to AgentFactory.CreateAgentAsync) lets tests inject a fake IAgentRuntime and assert on the captured prompt / criteriaContext. Decision (recorded, not to be re-litigated): grounding.verdict stays manual. Verdict means "an independent critic verified the test is grounded"; from-description runs no critic, so claiming grounded/partial would assert verification that never happened. Populating criteria ≠ verification. No new enum value, no critic invocation, no schema/migration. Documented consequence: from-description tests now count toward acceptance-criteria coverage (populated criteria) but remain excluded from grounded statistics (verdict manual) — intended, not an inconsistency.
  • Filter schema alignment & strict deserialization (Spec 051): closes the binding-side silent-failure class at the MCP request boundary. (1) start_execution_run now accepts the same top-level plural filter shape as find_test_cases (priorities/tags/components), normalized via StartExecutionRunTool.NormalizeFilters into a unified RunFilters; the legacy nested filters: { priority, tags, component } object stays accepted but is [Obsolete] / deprecated: true (top-level wins + warning when both are sent). (2) McpProtocol gained a dedicated strict params options instance (UnmappedMemberHandling.Disallow) and DeserializeParams<T>(args, toolName); an unmapped/misspelled field throws McpInvalidParamsException with a message naming the field and (for known filter confusions) suggesting the correct one, caught at ToolRegistry.InvokeAsync and rendered as INVALID_PARAMS. The JSON-RPC envelope parser stays lenient by design (separate options). (3) Both execution-agent prompt copies show one filter shape. Decision (clarified, recorded): unify on find_test_cases semantics — RunFilters gained plural Priorities/Components (raw strings; unknown values match nothing, so ["high","critical"] just yields the high tests) and run-mode tag matching changed AND→OR to match find_test_cases; the one test that pinned tag-AND (TestQueueFilterTests.Build_FilterByMultipleTags_RequiresAllTags) was rewritten. HasFilters meaning and TestIds dependency-recursion are unchanged. Complements Spec 049 (049 puts from-description tests into the index; 051 ensures filter requests actually bind to them). Out of scope: removing the legacy nested shape or the singular RunFilters fields; a general fuzzy field-suggester (the map is closed). Risk handled: enabling Disallow globally was validated against the full MCP suite (no tool relied on extra-field leniency).

Spec 047 — narrowed defect surface

  • Cache poisoning channel: poisoning occurred only when ParseResponse collapsed a returned response into [] — the per-doc loop's hash-compute failure, per-doc timeout, and thrown-exception branches continued before the hash write and were already correct. The narrow fix replaces the IReadOnlyList<AcceptanceCriterion> return with a typed result and gates the hash write on IsCacheable.
  • Two extractor implementations: the docs index path uses RequirementsExtractor (output: RequirementDefinition); the ai analyze --extract-criteria path uses CriteriaExtractor (output: AcceptanceCriterion). Their downstream writers and YAML schemas differ. Full unification was deferred to a future spec because it expands the regression surface across both commands; Spec 047 only adds a per-doc method on RequirementsExtractor and rewires DocsIndexHandler to call it in a loop.

Spec 052 — the silent-failure pattern (cross-cutting learning)

Specs 047–051 were, at root, five instances of one class of defect: code that swallows a failure and hands back a plausible-looking value instead of surfacing it. Spec 052 is the ship-readiness pass (cross-spec integration tests, named regression guards, a large-corpus scale guard, and this documentation consolidation) that hardens the whole block. The recurring shapes to watch for in future development:

  • Lenient deserialization that drops unknown fields. A misspelled or misshapen request field (priority instead of priorities, or a plural nested under legacy filters) was silently ignored, so start_execution_run ran the whole suite instead of the filtered subset. Fix pattern: opt into JsonUnmappedMemberHandling.Disallow on the params options and return an actionable, field-naming error (Spec 051). Watch for: any JsonSerializerOptions used for request binding that does not reject unmapped members.
  • Returning a value for a "required" outcome instead of failing. UserDescribedGenerator passed criteriaContext: null onward, so the mandatory criteria-mapping block never reached the model and the criteria field came back empty but valid-looking (Spec 050). Watch for: optional parameters that silently default away a behavior the caller assumed was on.
  • catch { return []; } (or => []) that masks a transient failure. A parse-class extraction failure collapsed to an empty list that was then cached, permanently poisoning the document (Spec 047). Fix pattern: a typed result (Extracted | EmptyResponse | ParseFailure with IsCacheable) so callers gate side effects on a real outcome, plus bounded retry. Watch for: any catch { return Enumerable.Empty<…>(); } whose empty return is indistinguishable from a genuine empty result.
  • Write-without-register. From-description tests were written to disk but never added to _index.json, so every discovery tool was blind to them (Spec 049). Fix pattern: a single write-then-index entry point (TestPersistenceService) so a file can never be persisted without its index row.
  • Missing-but-silent corpus deadline. A single corpus-wide 60s budget killed extraction for the whole document set with no per-document signal (Spec 047). Fix pattern: per-item deadlines + explicit timed-out/failed lists surfaced in the result.

Guard discipline going forward: when a function can fail, prefer a typed/explicit outcome over a default value; reject unknown request fields rather than ignoring them; and never cache or register on a path that swallowed an exception.

VS Code Copilot Chat Integration

12 Bundled SKILLs

SKILL Purpose
spectra-generate Test generation (analyze → approve → generate flow)
spectra-update Test updates (classify → review → apply changes)
spectra-coverage Coverage analysis
spectra-dashboard Dashboard generation and opening
spectra-validate Test validation
spectra-list Test listing and browsing
spectra-init-profile Generation profile setup
spectra-help Command reference (terse, flag-oriented)
spectra-criteria Criteria extraction, import, listing
spectra-docs Documentation indexing with progress page
spectra-prompts Prompt template management (list/show/reset/validate)
spectra-quickstart Workflow-oriented onboarding & walkthroughs (12 workflows with example conversations)

Bundled Project-Root Docs

  • CUSTOMIZATION.md — configuration & customization reference (profiles, prompts, branding).
  • USAGE.md — workflow guide for Copilot Chat usage (offline mirror of spectra-quickstart).

Both are written by spectra init and tracked by the SKILL manifest hash system so spectra update-skills refreshes unmodified copies and preserves user edits.

2 Agent Prompts (Delegation Model)

Agent Purpose
spectra-generation (~81 lines) Primary: test generation and update. Delegates all other CLI tasks (update, dashboard, coverage, criteria, validate, list, docs index) to corresponding SKILLs via delegation table.
spectra-execution (~120 lines) Test execution via MCP tools. Delegates all CLI tasks (update, dashboard, coverage, criteria, validate, list, docs index) to corresponding SKILLs via delegation table.

Agents are routers, not executors. Each CLI command has exactly one source of truth — its SKILL file. Agents reference SKILLs by name (e.g., "Follow the spectra-dashboard SKILL") instead of duplicating CLI instructions.

SKILL Format Conventions

  • Name: lowercase-hyphenated (e.g., spectra-generate)
  • Tools: Include browser/openBrowserPage for preview capability
  • CLI flags: Always include --no-interaction --output-format json --verbosity quiet
  • Result reading: Use readFile .spectra-result.json (not terminalLastCommand)
  • Progress: Long-running commands use .spectra-progress.html for live status
  • Steps: Use **Step N** format (not ### Tool call N:)
  • Wait instruction: Include "Between runInTerminal and awaitTerminal, do NOTHING" for long-running commands
  • Preview: Use show preview {file} to open files in VS Code (not start)

Generation Flow (SKILL/Agent)

  1. show preview .spectra-progress.html
  2. runInTerminal: spectra ai generate --suite {suite} --analyze-only --no-interaction --output-format json
  3. awaitTerminal (wait for completion)
  4. readFile .spectra-result.json → show breakdown, ask to proceed
  5. runInTerminal: spectra ai generate --suite {suite} --count {n} --no-interaction --output-format json
  6. awaitTerminal
  7. readFile .spectra-result.json → show results

MCP Execution Server

Key MCP Tools

  • start_execution_run, get_execution_status, pause/resume/cancel/finalize_execution_run
  • get_test_case_details, advance_test_case, skip_test_case, bulk_record_results
  • save_screenshot, save_clipboard_screenshot, add_test_note
  • list_active_runs, cancel_all_active_runs
  • find_test_cases, get_test_execution_history, list_saved_selections

Execution Flow

  1. list_active_runs → check for active runs
  2. start_execution_run → begin with suite/selection/test_ids
  3. get_test_case_details → present test to user (MUST show full details)
  4. advance_test_case → record PASSED/FAILED/BLOCKED
  5. finalize_execution_run → generate reports, instruction to open HTML report

Reports

  • Generated in .execution/reports/ as JSON, Markdown, and HTML
  • HTML report has professional styling (navy theme, pass rate circle, expandable test cards)
  • finalize_execution_run returns instruction: "show preview .execution/reports/{filename}"

Progress Page System

The CLI generates a self-contained .spectra-progress.html during test generation:

  • Auto-refresh: <meta http-equiv="refresh" content="2"> (removed on completion)
  • Embedded JSON: Status data is inline in HTML (no fetch needed)
  • Phase stepper: Analyzing → Analyzed → Generating → Completed
  • Live status: Spinner, current message, timestamp
  • Summary cards: Behaviors found, already covered, recommended, tests written
  • File links: Generated test files as vscode://file/ clickable links
  • Reset: Both .spectra-result.json and .spectra-progress.html are deleted at command start

Coverage System

Three-section unified coverage with distinct semantics:

  1. Documentation Coverage: Which docs have test cases (via source_refs or suite name match). "Covered" = at least 1 test references the document. Automation status is irrelevant.
  2. Acceptance Criteria Coverage: Which criteria are tested (via criteria: [] or legacy requirements: [] in test frontmatter, with per-source-type breakdown). Generation pipeline loads criteria and passes to AI prompt so generated tests include criteria: [AC-XXX].
  3. Automation Coverage: Which tests have automated_by: [] links resolving to existing files in automation_dirs.

Config Schema (spectra.config.json)

{
  "source": { "mode": "local", "local_dir": "docs/" },
  "tests": { "dir": "tests/", "id_prefix": "TC", "id_start": 100 },
  "ai": {
    "providers": [{ "name": "azure-anthropic", "model": "claude-sonnet-4-5", "enabled": true }],
    "critic": { "provider": "azure-anthropic", "model": "claude-sonnet-4-5" },
    "analysis_timeout_minutes": 2,
    "generation_timeout_minutes": 5,
    "generation_batch_size": 30,
    "debug_log_enabled": true
  },
  "coverage": {
    "criteria_file": "docs/criteria/_criteria_index.yaml",
    "criteria_dir": "docs/criteria",
    "automation_dirs": ["tests"]
  },
  "testimize": {
    "enabled": false,
    "mode": "exploratory",
    "strategy": "HybridArtificialBeeColony"
  }
}

Generation & analysis tuning (v1.42.0):

  • ai.analysis_timeout_minutes (default 2) — behavior analysis timeout
  • ai.generation_timeout_minutes (default 5) — per-batch generation timeout
  • ai.generation_batch_size (default 30) — tests per AI call
  • debug.enabled (default false) — writes per-call timing diagnostics to .spectra-debug.log in the project root. Use --verbosity diagnostic for one-shot override without changing config.

Slower / reasoning models (DeepSeek-V3, large Azure deployments) typically need: analysis 5–10 min, generation 15–20 min, batches of 6–10.

When behavior analysis fails (timeout, parse error, empty response), the .spectra-result.json from --analyze-only now sets status: "analysis_failed" and a message field explaining the cause + remediation. The spectra-generate SKILL recognizes this and refuses to present the fallback default (15) as if it were a real recommendation.

Spec 039: critic providers now use the same five names as the generator (github-models, azure-openai, azure-anthropic, openai, anthropic). Legacy github is a soft alias; legacy google is rejected.

Spec 038 / v1.48.3: optional testimize section enables in-process Testimize integration for algorithmic test data optimization (BVA, EP, pairwise, ABC). The library runs in-process as a bundled NuGet dependency — no MCP child process, no separate tool installation. When enabled, TestimizeRunner calls TestimizeEngine.Configure(...).Generate() once per suite and embeds the result in the generation prompt as authoritative facts. Off by default.

Code Conventions

  • C# 12, .NET 8+
  • Naming: PascalCase types/methods, camelCase locals
  • Async: All I/O operations async with Async suffix
  • Nullable: Reference types enabled
  • Tests: xUnit, structured results (never throw on validation errors)
  • Serialization: [JsonPropertyName("snake_case")] for JSON, [YamlMember(Alias = "snake_case")] for YAML
  • Atomic writes: Temp file + rename pattern for all file writes
  • Error handling: Non-critical operations (progress files, browser open) wrapped in try/catch

Completed Feature Specs

# Feature Key Changes
052 Test Hardening & Documentation Audit (047–051) Ship-readiness pass for the 047–051 block: new Spectra.Integration.Tests project with cross-spec EndToEndScenarios (7) + named OriginalBugRegression guards (5, display-named after the original user symptoms); ScaleTests ([Trait Category=Scale]) proving per-document extraction deadlines; full doc/SKILL audit (docs/specs/052-doc-audit-report.md, 052-skill-transcripts.md); consolidated CHANGELOG entry; silent-failure-pattern learning. No production code change.
051 Filter Schema Alignment & Strict Deserialization start_execution_run accepts top-level plural priorities/tags/components (same shape as find_test_cases); legacy nested filters deprecated but honored; UnmappedMemberHandling.Disallow + actionable field-naming error replaces silent property drops.
050 From-Description Criteria Injection From-description generation forwards loaded criteria as the mandatory mapping instruction so the criteria: field is populated; agentFactory test seam on UserDescribedGenerator.GenerateAsync; verdict stays manual by design.
049 From-Description Index Parity TestPersistenceService single write-file + regenerate-index entry point; from-description tests registered in _index.json and discoverable by all MCP tools; spectra index --rebuild backfill.
048 Acceptance Criteria Coverage Guards CriteriaSource.outcome field (default extracted); docs index non-blocking criteria_warning on zero-criteria corpus; ai generate notes entry when no criteria match the suite; CriteriaContextResult exposes suite-match count.
047 Resilient Criteria Extraction Typed CriteriaExtractionResult (Extracted/EmptyResponse/ParseFailure) gates the cache on IsCacheable; bounded retry with IExtractionDelayProvider; per-document 2-min deadline (was a single 60s corpus deadline).
039 Unified Critic Provider List Critic provider validation now matches the generator provider list (github-models, azure-openai, azure-anthropic, openai, anthropic). Legacy github is a soft alias with deprecation warning; legacy google is hard-rejected. New ResolveProvider helper in CriticFactory. Enables Azure-only billing setups.
038 Testimize Integration Optional MCP integration with the external Testimize.MCP.Server global tool for algorithmic test data optimization (BVA, EP, pairwise, ABC). New testimize config section (disabled by default), TestimizeMcpClient (process lifecycle, JSON-RPC, 30s/5s timeouts, idempotent disposal), two new conditionally-registered AI tools (GenerateTestData, AnalyzeFieldSpec), {{#if testimize_enabled}} blocks in behavior-analysis and test-generation templates, new spectra testimize check CLI command, full graceful degradation. No NuGet dependency.
037 ISTQB Test Design Techniques All five built-in prompt templates rewritten to teach the AI six ISTQB techniques (EP, BVA, DT, ST, EG, UC). New IdentifiedBehavior.Technique field, BehaviorAnalysisResult.TechniqueBreakdown map, AcceptanceCriterion.TechniqueHint field. Analysis output includes technique_breakdown alongside breakdown. Terminal and progress page render a Technique Breakdown section in fixed display order. Distribution guideline caps any single category at 40%. Existing user-edited templates preserved; opt in via spectra prompts reset --all.
033 From-Description Chat Flow Dedicated --from-description SKILL section, agent intent routing (focus vs from-description vs from-suggestions), doc-aware manual tests with populated source_refs and criteria (verdict stays manual)
029 spectra-update SKILL (10th) Agent delegation, documentation sync, version 1.35.0
028 Coverage & Criteria Pipeline Fixed criteria propagation in parser, wired criteria into generation pipeline, always write criteria: []
027 SKILL/Agent Deduplication Agents delegate to SKILLs, execution ~120 lines, generation ~81 lines, SKILL consistency fixes
024 Docs Index SKILL & Coverage Fix 9th SKILL (spectra-docs), result/progress files, --skip-criteria, terminology fix
023 Criteria Extraction Overhaul Per-doc extraction, import, rename requirements→criteria, progress page
023 Copilot Chat Integration SKILLs, agents, result file polling, batch generation
022 Bundled Skills 8 SKILLs + 2 agents embedded as resources, hash-tracked updates
021 Generation Session 4-phase flow, suggestions, user-described tests, duplicate detection
020 CLI Non-Interactive --output-format json, --no-interaction, exit codes
017 MCP Tool Resilience Auto-resolve run_id/test_handle, list_active_runs
015 Requirements Extraction AI-powered extraction (now superseded by 023)
014 Open Source Ready CI/CD, README, publish workflows
013 CLI UX Improvements NextStepHints, interactive prompts, config subcommands
012 Dashboard Branding Company logo, colors, dark theme, --preview
011 Coverage Overhaul Unified 3-type coverage, auto-link, requirements parser
010 Document Index docs/_index.md, SHA-256 incremental, sections/entities
010 Smart Test Selection find_test_cases, saved selections, execution history
009 Copilot SDK Consolidation Single AI runtime, removed legacy agents
008 Grounding Verification Dual-model critic, grounded/partial/hallucinated verdicts
006 Conversational Generation Direct/interactive modes, test updates, classification

Test Counts (as of 2026-06-03, Spec 052)

Project Tests
Spectra.Core.Tests 550
Spectra.CLI.Tests 1054
Spectra.MCP.Tests 368
Spectra.Integration.Tests 12
Total 1984

Spectra.CLI.Tests includes the 2 [Trait("Category","Scale")] scale-guard tests; exclude them in fast-feedback runs with dotnet test --filter "Category!=Scale".