cookbook: data_labeling on Gemini + quality review as Workflow#8024
Merged
Conversation
Lifts the labeler -> reviewer -> adjudicator pipeline onto the Workflow API so the two labelers run concurrently (Parallel), the adjudicator only runs on disagreement (Condition), and every run is persisted to SQLite for traceability. The reviewer and adjudicator use Step(executor=...) to pull both labelers' outputs out of the Parallel block via step_input.get_step_output(), which recursively searches nested steps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Across all 18 data_labeling folders, switch the default model from OpenAIResponses(gpt-5.5) to Gemini(gemini-3.5-flash) -- multimodal, frontier-grade reasoning at flash prices, single GOOGLE_API_KEY for the whole cookbook. READMEs and TEST_LOGs updated to match. Also picks up a couple of ruff format/import-order fixes in _17_llm_as_judge/. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure formatting / import-order fixes surfaced by ./scripts/format.sh while preparing this PR. No semantic changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ysolanky
approved these changes
May 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three commits to
cookbook/data_labeling/and adjacent files, in service of the upcoming "Agents for data labeling" blog post.1. Migrate all 18 data_labeling cookbooks to Gemini
Across every folder (
_01through_18), switch the default model fromOpenAIResponses(gpt-5.5)toGemini(gemini-3.5-flash). The rationale:GOOGLE_API_KEYalone (the lone exception is_18_quality_review/, which uses Claude for one of its two labelers)._06–_14all run through audio, video, images, and PDFs without rearchitecting the agent. Gemini 3.5 Flash handles every modality in the same call shape.READMEs and TEST_LOGs across all 18 folders updated to match (
OPENAI_API_KEY→GOOGLE_API_KEY).2. Rewrite
_18_quality_review/basic.pyas a WorkflowLifts the labeler → reviewer → adjudicator pipeline off procedural Python and onto the agno Workflow API:
Parallel(label_a, label_b). The old version ran them sequentially, doubling latency.Condition(evaluator=has_disagreement, steps=[adjudicate])— workflow structure instead of anifstatement.SqliteDb(db_file="tmp/labeling.db")— every run captured for auditability, which matters for a labeling pipeline.The reviewer and adjudicator use
Step(executor=fn)and reach into the Parallel block withstep_input.get_step_output("Labeler A"), the documented helper for recursively searching nested steps.3. Ruff format drift in three unrelated files
./scripts/format.shsurfaced pre-existing format / import-order drift in three files outside the data_labeling cookbook (cookbook/90_models/...antigravity.py,cookbook/91_tools/...drive_all_drives_search.py,libs/agno/tests/unit/models/google/test_gemini.py). Pure formatting fixes, no semantic changes. Included here to keepformat.shclean on this branch.Type of change
Checklist
./scripts/format.shand./scripts/validate.sh; cookbook ruff + pattern checks pass; mypy crash inagno_infrais a pre-existing pydantic_settings issue unrelated to this PR)Duplicate and AI-Generated PR Check
Additional Notes
Companion to the upcoming "Agents for data labeling" blog post — the post uses
gemini-3.5-flashthroughout and shows the Workflow version of the quality review pipeline as its wedge example for multi-agent quality control. This PR makes the cookbook match.Stats: 80 files changed, +284/-231 across three commits.