Skip to content

fix: structure optional skill references - #59

Merged
martinfrancois merged 1 commit into
mainfrom
fix/structure-optional-skill-references
Jun 2, 2026
Merged

fix: structure optional skill references#59
martinfrancois merged 1 commit into
mainfrom
fix/structure-optional-skill-references

Conversation

@martinfrancois

Copy link
Copy Markdown
Owner

Summary

  • Problem: Tessl review still reported progressive disclosure as below 100% because the runtime references were linked inline rather than presented as a structured section.
  • Why it matters: the skill should reach 100% quality without reducing the headline Optional-quality lift.
  • What changed: added the optimizer-suggested Reference Bundle table and clarified the Hard Stops intro as a quick checklist with the detailed marker scan in hard-stops.md.
  • What did not change: no active eval tasks, criteria, weights, runtime reference files, package metadata, or CI/release settings changed.

Change Type

  • Skill behavior
  • Evals or scoring
  • Documentation
  • CI, release, or dependency automation
  • Repository metadata or contribution process
  • Other maintenance

Linked Issue

  • Fixes #
  • Related #

User-Visible Behavior

Users should see the same Java Optional guidance with clearer progressive disclosure. Tessl skill review reports 100%.

Bug Fix Details

  • Root cause: the previous skill text had the right reference files, but Tessl review wanted them integrated into a structured section.
  • Test, eval, or guardrail added: no new eval was added; the existing headline suite was rerun with Sonnet 4.6.
  • If no test or eval was added, why not: this is a runtime wording/structure cleanup, and the existing headline evals already cover the lift risk.

Validation

Checks most contributors can run:

  • python3 scripts/validate_skill.py skills/java-optionals
  • python3 scripts/validate_eval_criteria.py evals evals-reference
  • python3 -m py_compile scripts/validate_skill.py scripts/validate_eval_criteria.py
  • bash -n scripts/check_publish_dry_run.sh
  • tessl plugin lint .
  • markdownlint, if Markdown changed
  • Manual rendered-doc or example review, if docs or examples changed

Tessl-authenticated checks:

  • bash scripts/check_publish_dry_run.sh .
  • tessl plugin publish --dry-run --bump patch .
  • tessl skill review --threshold 90 skills/java-optionals/SKILL.md, if skill text or references changed
  • tessl eval run --variant with-context --variant without-context ., if skill behavior,
    evals, or benchmark claims changed

Details:

python3 scripts/validate_skill.py skills/java-optionals
Skill is valid.

python3 scripts/validate_eval_criteria.py evals evals-reference
Validated 52 scenario(s): 4 natural, 48 explicit.

python3 -m py_compile scripts/validate_skill.py scripts/validate_eval_criteria.py
bash -n scripts/check_publish_dry_run.sh

tessl plugin lint .
Plugin martinfrancois/java-optionals@0.1.27 is valid.

tessl skill review --threshold 100 skills/java-optionals/SKILL.md
Review Score: 100%

tessl skill review --optimize --max-iterations 10 --yes skills/java-optionals
Applied the final reference-table shape used in this PR.

tessl eval run --agent claude:claude-sonnet-4-6 --variant with-context --variant without-context --label "optimized reference table sonnet 4.6 54c0b59" .
Run id: 019e86f2-e65d-776e-a663-a9d1d233c2bd
Baseline: 159/360
With context: 360/360
Lift: 2.26x

Human Verification

Reviewed the SKILL.md diff to confirm it only changes runtime skill wording and does not touch active eval content or scoring. Two hand-written structured-reference variants were rejected because they hurt the checked-boundary scenario. The optimizer table was kept only after the final Sonnet 4.6 run kept every with-context headline scenario at 100%.

Review Checklist

  • Docs updated, or N/A
  • Evals updated, or N/A
  • Scenario directories include task.md, criteria.json, and capability.txt, or N/A
  • Scenario invocation style is classified as natural or explicit, or N/A
  • Natural activation prompts don't explicitly invoke the skill, or N/A
  • Explicit invocation prompts are labeled as explicit, or N/A
  • Headline criteria include compile/artifact checks, or N/A
  • Headline criteria include behavior correctness checks, or N/A
  • Runtime references contain no eval answer keys, scenario inventory, hosted run IDs, or fixed
    score claims
  • Java baseline compatibility has been considered, or N/A
  • OptionalInt, OptionalLong, and OptionalDouble guidance has been considered, or N/A
  • Optional-producing stream terminals and collectors are covered, or N/A
  • Java 26 Javadocs were checked for Optional-family coverage, or N/A
  • Valid README package-runner instructions were preserved, or N/A
  • Tessl package commands match the verified plugin package format
  • Full/reference eval reporting is not hidden or cherry-picked
  • Tessl checks were run, or unavailability is documented
  • PR title or squash title uses Conventional Commits
  • Redaction checked: no Tessl tokens, GitHub tokens, package manager tokens, private repository
    links, private eval artifacts, private registry/workspace links, local host paths, or
    proprietary Java source

AI Assistance (if used)

  • AI-assisted PR
  • I confirm I understand and reviewed the change
AI prompts / session logs (optional)
The maintainer asked to improve Tessl progressive-disclosure quality without hurting headline lift. Codex tested manual variants, rejected those that hurt lift, then kept the Tessl optimize output after local validation, Tessl review, and a hosted Sonnet 4.6 headline eval.

@martinfrancois
martinfrancois merged commit 9d8c602 into main Jun 2, 2026
8 checks passed
@martinfrancois
martinfrancois deleted the fix/structure-optional-skill-references branch June 2, 2026 06:18
martinfrancois pushed a commit that referenced this pull request Jun 2, 2026
🤖 I have created a release *beep* *boop*
---


##
[0.1.28](v0.1.27...v0.1.28)
(2026-06-02)


### Bug Fixes

* structure optional skill references
([#59](#59))
([9d8c602](9d8c602))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant