fix(streaming): assemble chat completions content parts in content index order - #4236
Merged
seratch merged 1 commit intoAug 6, 2026
Merged
Conversation
…dex order When a Chat Completions stream emits a refusal delta before any text delta, the handler announces the refusal at content_index 0 and the text at content_index 1, but the completed assistant message always appended the text part first. The finished message then contradicted the indexes consumers had already received, so replaying the raw events produced a different content layout from response.completed. Sort the two parts by their content index before appending. Text-first streams are unaffected because the text part already holds index 0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ChatCmplStreamHandlerassigns content indexes in the order the parts open: a refusal delta that arrives before any text delta getscontent_index=0and the later text getscontent_index=1. The completed assistant message, however, always appended the text part first and the refusal second, soresponse.completedcontradicted thecontent_part.added,output_text.delta, andrefusal.deltaevents consumers had already received. A consumer that usescontent_indexto index into the finished message'scontentlist reads the wrong part.The fix sorts the two parts by their already-assigned content index before appending them. Text-first streams are unchanged because the text part already holds index 0 there, so this only affects the refusal-before-text ordering that was inconsistent to begin with.
tests/models/test_openai_chatcompletions_stream.py::test_stream_handler_places_text_after_existing_refusal_partalready streams a refusal chunk followed by a text chunk and already asserts that the text part is announced atcontent_index == 1. It then asserted the contradictory completed layout, which is the bug rather than intended behavior, so this updates it to require the completed content to line up with the announced indexes and adds the matching assertion that the refusal was announced at index 0. It fails onmainwithAssertionError: assert False, where False = isinstance(ResponseOutputText(...), ResponseOutputRefusal)and passes with this change.make lintandmake typecheck(mypy and pyright, 0 errors) are green, and the full suite is 6672 passed with 32 skipped. This is a refile of #3822 by @felmonon, which the stale bot closed for inactivity rather than on the merits. It is also the ordering issue @AmirF194 explicitly scoped out of #3757, calling it a separate pre-existing bug on main, so #4177 landing did not cover it.