fixed reasoning failures #634
Conversation
…andling of json types by anthropic, and wrote a bunch of tests
| if is_incomplete_json { | ||
| // Incomplete JSON - buffer for retry with next chunk | ||
| self.incomplete_event_buffer = sse_event.raw_line.as_bytes().to_vec(); | ||
| break; | ||
| } else { | ||
| // Other error (unsupported event type, validation error, etc.) | ||
| // Skip this event and continue processing others | ||
| continue; | ||
| } | ||
| } |
There was a problem hiding this comment.
What if there are more events afterwards? That might be valid?
There was a problem hiding this comment.
Only the corrupted line is discarded. So additional lines with new events on the same payload will work okay, because of the continue. The challenge is if there is an incomplete json on a line, how do we really know if the additional bytes aren't part of the same event or truly part of a new event. I suppose you could do a regex search for proper json start/end. But I think a corrupt line is a corrupt line. Thoughts?
There was a problem hiding this comment.
I think this code here may be hiding the underlying issue where it sees sse events but last event which breaks does not have line break but sse_iter::try_from will return that invalid evnet as event data line
For now it may be okay to take this change but this is where we need to fix where if the chunk doesn't end of line break then carry that buffer forward for the next chunk.
fn try_from(bytes: &[u8]) -> Result<Self, Self::Error> {
// Parse as text-based SSE format
let s = std::str::from_utf8(bytes)?;
let lines: Vec<String> = s.lines().map(|line| line.to_string()).collect();
Ok(SseStreamIter::new(lines.into_iter()))
}
* fixed reasoning failures * adding debugging * made several fixes for transmission isses for SSeEvents, incomplete handling of json types by anthropic, and wrote a bunch of tests * removed debugging from supervisord.conf --------- Co-authored-by: Salman Paracha <[email protected]>
we should build the right reasoning object. Else the conversion can fail in a boundary case where the upstream LLM returns "none" and we can't map it to low/medium/high for v1/responses/api