fix: tighten optional skill disclosure

[martinfrancois]

Summary

• Moved detailed Optional hard-stop guidance into references/hard-stops.md and kept SKILL.md concise.
• Preserved runtime-critical guidance for lazy fallbacks, nullable Optional entry points, side-effect terminals, primitive Optionals, and narrow checked-boundary branches.
• Made checked-boundary examples Java-baseline-aware in the reference file and marked the inline SKILL.md checked-boundary snippet as Java 11+.
• Clarified the checked-boundary eval rubric so ordinary Optional value flow is still penalized, while the intended checked IO/prompt/parser exception can receive full credit.
• Aligned the eval validator with the intended Optional-quality weighting policy and removed unused style-score bookkeeping.
• Required every active/reference eval task to state the Java version to assume, so future candidate evals cannot omit it.
• Kept Clean up Optional value reads at checked boundaries active, but downweighted it from 100 to 60 points because it still shows useful Optional-quality gaps while being less discriminating than the strongest headline cases.
• Promoted command sanitization into the active headline suite, renamed its active path to evals/50-command-sanitizer-feature, and removed the duplicate reference copy.

Skill quality

• tessl skill review --threshold 90 skills/java-optionals/SKILL.md: 100%
• SKILL.md line count: 66

Active headline suite

• Kept evals/45-workflow-validation-cleanup out of the active headline suite to avoid diluting the focused Optional-quality signal. It remains in evals-reference/45-workflow-validation-cleanup as reference/regression coverage.
• Removed the duplicate promoted command-sanitizer scenario from evals-reference/.
• Renamed the active command-sanitizer scenario to evals/50-command-sanitizer-feature so the active path no longer says reference.

Current active suite:

• evals/04-frontmatter-port-feature: 100 points, natural
• evals/10-first-pass-retry-backoff: 100 points, natural
• evals/11-checked-boundary-selection-cleanup: 60 points, explicit
• evals/50-command-sanitizer-feature: 100 points, natural

Current headline total: 360 points
Natural / explicit split: 300 natural, 60 explicit
Category split: 54 safety, 288 Optional-quality, 18 maintainability

Hosted evals

Latest Sonnet 4.6 focused headline run on d66a3c4:

• https://tessl.io/workspaces/martinfrancois/eval-runs/019e86a3-4f73-7041-b7cc-6c1f63f23c21

Results:

Scenario	Without skill	With skill
Add string support to workflow port lookup	55/100	100/100
Write retry backoff code	19/100	100/100
Clean up Optional value reads at checked boundaries	44/60	60/60
Extend command sanitization	41/100	100/100

Totals:

• Combined headline: 159/360 without skill -> 360/360 with skill
• Raw score lift: 2.26x
• Natural activation subset: 115/300 without skill -> 300/300 with skill
• Explicit invocation subset: 44/60 without skill -> 60/60 with skill
• Optional-quality subtotal: 89/288 without skill -> 288/288 with skill
• Safety subtotal: 54/54 without skill -> 54/54 with skill
• Maintainability subtotal: 16/18 without skill -> 18/18 with skill

Earlier reference sweep used to identify promotion candidates:

• https://tessl.io/workspaces/martinfrancois/eval-runs/019e8545-34ed-7028-a347-6d086ad1e789
• Combined reference result: without skill 2252/2770, with skill 2761/2770

Lift-sensitive changes

• fix: tighten optional skill disclosure

  • Why lift-sensitive: changes runtime skill context and reference navigation.
  • Validation/eval: skill review 100%; Opus 4.8 with-context retests reached 300/300 on the then-active suite.
  • Revert strategy: revert this commit if future hosted evals show lower with-context quality.

• test: clarify checked-boundary eval scoring

  • Why lift-sensitive: changes headline eval scoring wording for the checked-boundary scenario.
  • Validation/eval: Opus 4.8 with-context retests reached 300/300 on the then-active suite.
  • Revert strategy: revert this commit if maintainers decide the rubric should penalize the accepted checked-boundary local-read shape.

• fix: make optional hard stops baseline aware

  • Why lift-sensitive: changes runtime skill wording and reference examples.
  • Validation/eval: skill review 100%; Opus 4.8 with-context retest 300/300 on the then-active suite.
  • Revert strategy: revert this commit if the baseline-aware wording lowers future hosted eval quality.

• test: downweight checked-boundary eval

  • Why lift-sensitive: changes active headline weighting.
  • Validation/eval: latest focused Sonnet 4.6 headline run reached 360/360 with skill and 159/360 without skill.
  • Revert strategy: revert this commit if maintainers want the checked-boundary scenario to carry equal headline weight.

• test: promote optional reference regressions

  • Why lift-sensitive: changed active headline composition and score denominator.
  • Validation/eval: promoted from a completed Sonnet 4.6 all-reference sweep where selected cases showed Optional-related with-skill improvements.
  • Revert strategy: revert this commit if future runs show lower with-context quality or the cases feel less representative than expected.

• test: demote baseline-solved workflow eval

  • Why lift-sensitive: changes active headline composition and score denominator.
  • Validation/eval: focused Sonnet 4.6 headline run reached 360/360 with skill and 159/360 without skill (2.26x raw score lift).
  • Revert strategy: revert if maintainers want workflow validation to carry headline weight despite being baseline-solved in the current-head run.

• test: rename active command sanitizer eval

  • Why lift-sensitive: changes active headline scenario path but not task content or scoring.
  • Validation/eval: focused Sonnet 4.6 headline run reached 360/360 with skill and 159/360 without skill (2.26x raw score lift).
  • Revert strategy: revert if path continuity is more valuable than the clearer active scenario name.

• test: remove duplicate command sanitizer reference

  • Why lift-sensitive: removes duplicate reference-suite coverage of a promoted active scenario.
  • Validation/eval: active headline run above is unchanged in content and passed with-context 360/360.
  • Revert strategy: restore if maintainers want evals-reference/ to keep historical duplicates of active scenarios.

• docs: update headline eval guidance

  • Why lift-sensitive: no. Documentation only.
  • Validation/eval: local docs/validation checks passed.
  • Revert strategy: revert if the wording is less clear.

• test: remove unused style score bookkeeping

  • Why lift-sensitive: no. Validator dead-code cleanup only.
  • Validation/eval: local validation checks passed.
  • Revert strategy: revert if future diagnostics need the removed unused bookkeeping.

• test: promote workpad optional boundary eval / revert: remove forced workpad headline eval

  • Why lift-sensitive: changed active headline composition.
  • Validation/eval: Sonnet candidate showed a useful split, but the scenario was reverted because it was not a natural Optional-native task.
  • Revert strategy: already reverted in the follow-up commit.

Validation

Passed:

• python3 scripts/validate_skill.py skills/java-optionals
• python3 scripts/validate_eval_criteria.py evals evals-reference
• python3 -m py_compile scripts/validate_skill.py scripts/validate_eval_criteria.py
• bash -n scripts/check_publish_dry_run.sh
• JSON parse check
• YAML parse check
• tessl plugin lint .
• tessl skill review --threshold 90 skills/java-optionals/SKILL.md
• tessl eval run . --agent=claude:claude-sonnet-4-6 --variant with-context --variant without-context --label "headline focused sonnet 4.6 d66a3c4" --json

Notes

• Runtime skill behavior is unchanged after a92d568; later commits only change active/reference eval validation, headline composition, docs, or validator cleanup.
• The checked-boundary scenario remains active but has lower headline weight.
• The scorer change is limited to making the checked-boundary criterion match the intended general rule: ordinary Optional value reopening remains disallowed, but a narrow checked IO/prompt/parser branch may use one local read after an empty guard.
• Every eval task now states an explicit Java version to assume.