Skip to content

fix: prevent exception chunk from being passed to _process in astream#547

Open
0xCUB3 wants to merge 6 commits intogenerative-computing:mainfrom
0xCUB3:fix/astream-exception-handling
Open

fix: prevent exception chunk from being passed to _process in astream#547
0xCUB3 wants to merge 6 commits intogenerative-computing:mainfrom
0xCUB3:fix/astream-exception-handling

Conversation

@0xCUB3
Copy link
Contributor

@0xCUB3 0xCUB3 commented Feb 16, 2026 •
edited
Loading

Misc PR

Type of PR

  • Bug Fix
  • New Feature
  • Documentation
  • Other

Description

  • Link to Issue:

Testing

  • Tests added to the respective file if code was changed
  • New code has 100% coverage if code as added
  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

@0xCUB3 0xCUB3 requested review from a team, jakelorocco and nrfulton as code owners February 16, 2026 14:14
@mergify
Copy link

mergify bot commented Feb 16, 2026

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

@0xCUB3
fix: prevent exception chunk from being passed to _process in astream
f2de636 
Pop exception from chunks list (like we do for the None sentinel) so
_process doesn't receive it. Guard chat_response access in ollama
post_processing with .get() for when no valid chunks arrived.

Signed-off-by: 0xCUB3 <skula@mit.edu>
@0xCUB3 0xCUB3 force-pushed the fix/astream-exception-handling branch from 151b892 to f2de636 Compare February 16, 2026 14:15
@0xCUB3 0xCUB3 marked this pull request as draft February 16, 2026 14:15
@github-actions
Copy link
Contributor

github-actions bot commented Feb 16, 2026

The PR description has been updated. Please fill out the template for your PR to be reviewed.

@0xCUB3 0xCUB3 marked this pull request as ready for review February 16, 2026 14:21
@nrfulton
Copy link
Member

nrfulton commented Feb 17, 2026

Assigning @avinash2692 to shepherd.

The code in other backends is similar so we should verify other backends don't have similar bugs. Some defensive programming in _process probably also isn't a bad idea.

@nrfulton nrfulton requested a review from avinash2692 February 17, 2026 16:34
@0xCUB3
Copy link
Contributor Author

0xCUB3 commented Feb 19, 2026

Checked all backends for the same pattern, and yes, there were similar issues -- thanks @nrfulton. Found unsafe bracket access on _meta in post_processing for the other backends... openai, litellm, watsonx, huggingface, and vllm. When an exception occurs during streaming, post_processing still runs (finally block) but similarly, _meta keys were never set, so bracket access crashes with KeyError.

Fixes:

  • openai/litellm/watsonx: switched to .get() for _meta reads, added None
    guards before tool extraction and telemetry
  • huggingface/vllm: replaced assert mot.value is not None with early return
    (value is legitimately None on exception)
  • base.py: added a type guard in astream that skips Exception/None chunks
    before calling _process (belt-and-suspenders for the chunks.pop() fix)

Ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nrfulton nrfulton Awaiting requested review from nrfulton nrfulton is a code owner

@jakelorocco jakelorocco Awaiting requested review from jakelorocco jakelorocco is a code owner

@avinash2692 avinash2692 Awaiting requested review from avinash2692

At least 1 approving review is required to merge this pull request.

Assignees

@avinash2692 avinash2692

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@0xCUB3 @nrfulton @avinash2692

Comments