Python: Add request_handlers parameter for automatic HITL request handling#3842
Python: Add request_handlers parameter for automatic HITL request handling#3842moonbox3 wants to merge 9 commits intomicrosoft:mainfrom
Conversation
Python Test Coverage Report •
Python Unit Test Overview
|
|||||||||||||||||||||||||||||||||||||||||||||
There was a problem hiding this comment.
Pull request overview
Adds first-class, type-dispatched request_handlers support to Python workflows to automatically handle HITL request_info events inline (concurrently) and submit responses back into the same run, including builder-level defaults plus samples/tests.
Changes:
- Add
request_handlerstoWorkflowBuilder(...)andWorkflow.run(...), with inlineasyncio.create_task(...)dispatch onrequest_infoevents. - Update convergence logic so the runner waits for outstanding handler tasks before declaring the workflow idle.
- Add a new getting-started sample and a comprehensive test suite covering dispatch, error paths, streaming, and builder defaults.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| python/packages/core/agent_framework/_workflows/_workflow.py | Implements request_handlers dispatch in run() / core loop and final-state determination based on pending requests. |
| python/packages/core/agent_framework/_workflows/_runner.py | Extends convergence to wait for outstanding handler tasks before stopping. |
| python/packages/core/agent_framework/_workflows/_workflow_builder.py | Adds builder-level request_handlers support and build-time validation. |
| python/packages/core/tests/workflow/test_workflow_response_handlers.py | New tests validating handler dispatch, concurrency, failures, and builder-level defaults. |
| python/samples/getting_started/workflows/human-in-the-loop/fan_out_async_with_request_handlers.py | New sample demonstrating fan-out + looping branch with automatic HITL handling. |
| python/samples/getting_started/workflows/README.md | Links the new sample from the workflows sample index. |
| python/packages/core/AGENTS.md | Adds a brief note on running tests with uv run pytest .... |
python/packages/core/agent_framework/_workflows/_workflow_builder.py
Outdated
Show resolved
Hide resolved
| yield event | ||
| finally: | ||
| # Cancel any outstanding handler tasks on error/cancellation | ||
| if outstanding_tasks: |
There was a problem hiding this comment.
What is going to happen if the workflow goes to idle before a handle returns?
There was a problem hiding this comment.
Similar to what happens in dotnet: the workflow stays alive and waits. In the current code, with request_handlers the runner intentionally waits for the handler tasks before converging. A slow handler will keep the run open.
For "days/months" HITL, the safer path is:
- don't rely on these in-run request handlers
- let the workflow emit request_info events which will end the workflow (if it converges) with IDLE_WITH_PENDING_REQUESTS.
- persist via checkpoint
- later resume with
run(responses=...)
If we want to support both short-running and long-running behaviors we can:
- request_handler_wait_timeout or wait_for_request_handlers=False
- have a brief wait
- then finalize as IDLE_WITH_PENDING_REQUESTS if things are still pending
There was a problem hiding this comment.
My understanding of this if statement is that if there are pending task, we will cancel them. Please correct me if I am wrong.
There was a problem hiding this comment.
This is teardown safety for abnormal exits (error, cancel, stopped consumption, etc.)
| description: Optional description of what the workflow does. | ||
| output_executors: Optional list of executor IDs whose outputs will be considered workflow outputs. | ||
| If None or empty, all executor outputs are treated as workflow outputs. | ||
| request_handlers: Optional default response handlers for automatic HITL request handling. |
There was a problem hiding this comment.
| request_handlers: Optional default response handlers for automatic HITL request handling. | |
| request_handlers: Optional default request handlers for automatic HITL request handling. |
I think the name will cause a lot of confusion but yet I can't come up with a better name. Can bring this up to the team.
There was a problem hiding this comment.
I agree. These are "requests" that need to be fulfilled, as opposed to the @response_handler that we currently have, which is why I went with request_handlers.
Motivation and Context
In fan-out workflows where one branch self-loops (async processing) and another emits request_info (HITL), callers previously had to manually collect requests, call handlers, and resubmit via run(responses=...). The new request_handlers parameter automates this with a simple dict-based API:
Inline dispatch means the handler fires as soon as the request is emitted — it doesn't wait for other branches to complete first. A handler running concurrently with a long-running self-loop completes and injects its response while the loop is still iterating.
Behavior:
Important
Compatibility with checkpointing and manual response submission: request_handlers is additive and does not change the existing HITL contract. Workflows that end as IDLE_WITH_PENDING_REQUESTS — whether because no handlers were registered, a handler failed, or a request type was unmatched — can still be resumed later via run(responses=...)
or run(responses=..., checkpoint_id=...). Checkpointing continues to work: pending requests are captured in checkpoint state and can be restored and responded to in a separate process or session.
Note
Relationship to
@response_handler: The@response_handlerdecorator on executors is where responses get processed -- that is unchanged.request_handlersonWorkflowBuilder()orrun()changes where the response comes from. Without it, the caller manually collects requests and resubmits viarun(responses=...). With it, handler functions source the response automatically and the executor's@response_handlerruns the same as before. The two mechanisms work together, not in place of each other.Note
Type-based dispatch requires unique types: If two executors both do
ctx.request_info(request_data="some string", ...), there's only one slot for str in the dict -- both requests route to the same handler with no way to distinguish them. Unique dataclasses are effectively required for any non-trivial workflow:This is the same pattern as executor input types:
strworks for toy examples but real workflows need typed messages. Therequest_handlersdispatch just makes that constraint explicit.Description
Contribution Checklist