Skip to content

fix(core): extract usage metadata from serialized tracer message outputs#35526

Merged
Mason Daugherty (mdrxy) merged 2 commits into
masterfrom
mdrxy/fix-usage
Mar 2, 2026
Merged

fix(core): extract usage metadata from serialized tracer message outputs#35526
Mason Daugherty (mdrxy) merged 2 commits into
masterfrom
mdrxy/fix-usage

Conversation

@mdrxy
Copy link
Copy Markdown
Member

Fixes missing run.metadata.usage_metadata population in LangChainTracer for real LLM/chat traces following #34414

  • Fix extraction to read usage from serialized tracer message shape: outputs.generations[*][*].message.kwargs.usage_metadata
  • Remove non-serialized direct message shape handling (message.usage_metadata) from extractor to match real tracer output path
  • Clarify tracer docstrings around chat callback naming (on_chat_model_start + shared on_llm_end) to reduce ambiguity

Why

#34414 introduced usage duplication into run.metadata.usage_metadata, but the extractor read message.usage_metadata.

In real tracer flow, messages are serialized with dumpd(...) during run completion, so usage metadata lives under message.kwargs.usage_metadata. Because of this mismatch, duplication did not trigger in real traces.

@github-actions github-actions Bot added internal core `langchain-core` package issues & PRs fix For PRs that implement a fix and removed internal labels Mar 2, 2026
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Mar 2, 2026

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 13 untouched benchmarks
⏩ 23 skipped benchmarks1


Comparing mdrxy/fix-usage (f9ac259) with master (cdf140e)2

Open in CodSpeed

Footnotes

  1. 23 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on master (d2c86df) during the generation of this report, so cdf140e was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@mdrxy Mason Daugherty (mdrxy) merged commit 61fd90a into master Mar 2, 2026
91 checks passed
@mdrxy Mason Daugherty (mdrxy) deleted the mdrxy/fix-usage branch March 2, 2026 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core `langchain-core` package issues & PRs fix For PRs that implement a fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants