Skip to content

fix multi-line buffering issue #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

Bradley-Butcher
Copy link
Contributor

@Bradley-Butcher Bradley-Butcher commented Jun 13, 2025

Fix subprocess transport handling of buffered JSON output

TL;DR

This PR fixes an issue where the SDK fails to parse output from the Claude CLI when multiple JSON objects are concatenated on a single line due to stdout buffering.

Problem

In certain environments, stdout buffering can cause multiple JSON objects to be delivered as a single line with embedded newlines (\n). When this happens, json.loads() successfully parses the first JSON object but then raises a JSONDecodeError: Extra data error because there's additional valid JSON after the first object.

Example of problematic output:
{"type": "message", "id": "msg1", "content": "First message"}\n{"type": "result", "id": "res1", "status": "completed"}

Solution

The fix modifies the receive_messages() method in SubprocessCLITransport to split each received line on newline characters before attempting JSON parsing. This allows multiple concatenated JSON objects to be properly handled.

Changes

  • Modified src/claude_code_sdk/_internal/transport/subprocess_cli.py to split lines on \n before JSON parsing
  • Added comprehensive test in tests/test_subprocess_buffering.py to verify the fix

Testing

  • Added new test that reproduces the buffering scenario

106- added a commit to 106-/claude-code-sdk-python that referenced this pull request Jun 14, 2025
@ltawfik
Copy link
Collaborator

ltawfik commented Jun 18, 2025

thanks @Bradley-Butcher . Do you mind re-creating the pr with commit signing as it's a requirement to merge the PR, thank you

@ltawfik ltawfik self-requested a review June 18, 2025 04:45
@Bradley-Butcher Bradley-Butcher force-pushed the fix/subprocess-buffering-issue branch from 0403bae to 57b3c70 Compare June 19, 2025 10:36
Signed-off-by: Bradley-Butcher <[email protected]>
@Bradley-Butcher Bradley-Butcher force-pushed the fix/subprocess-buffering-issue branch from 57b3c70 to 87e2699 Compare June 19, 2025 10:41
@Bradley-Butcher
Copy link
Contributor Author

Bradley-Butcher commented Jun 19, 2025

Should be good @ltawfik

ltawfik and others added 3 commits June 27, 2025 15:06
- Convert async test to use anyio.run() pattern consistent with other tests
- Remove unused CLIJSONDecodeError import
- Add tests for JSON with embedded newlines
- Add tests for multiple newlines between objects
- All tests passing

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
- Fixed formatting in test_subprocess_buffering.py
- No functional changes, only code style improvements

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
- Fixed quotes to use double quotes consistently
- Adjusted line breaks per ruff formatter rules
- No functional changes

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@ltawfik ltawfik merged commit 9bda4e8 into anthropics:main Jun 27, 2025
5 checks passed
@Mng-dev-ai
Copy link
Contributor

@ltawfik I'm still getting claude_code_sdk._errors.CLIJSONDecodeError: Failed to decode JSON: {"type":"assistant","message":{"id":"msg_01DR48uqZZ3Sru1ttoyG4fD7","type":"message","role":"assistan... after bumping the version to 0.0.13

btw, This PR is working fine for me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants