Python: (ag-ui): Add Workflow Support, Harden Streaming Semantics, and add Dynamic Handoff Demo

[moonbox3]

Motivation and Context

• Added first-class AG-UI workflow execution via AgentFrameworkWorkflow, with endpoint support for both wrapped agents and workflows using .run(...).
• Removed duplicated emit logic from _agent_run and consolidated shared emission behavior in _run_common.
• Fixed ToolCallResultEvent serialization bug for non-string tool results.
• Hardened error handling:
  • endpoint now returns proper HTTP error responses (no silent 200 with error dict),
  • streaming/generator/encoding failures now emit RUN_ERROR instead of silently terminating.
• Added usage propagation as AG-UI CUSTOM (name="usage") and surfaced diagnostics in the handoff demo UI.
• Added multimodal capabilities and added agent/workflow multimodal coverage.
• Added handoff demo to show dynamic routing (.add_handoff(...)) and added HITL-gated replacement approval (submit_replacement, submit_refund) as examples on HITL tool approvals with AG-UI.
• Handles duplicate final assistant text replay (chunk + final snapshot tail duplication) with backend and demo UI dedupe guards.

Breaking Changes

• Removed run_agent(...); now uses run(...) for both an agent and a workflow run.
• Narrowed agent_framework.ag_ui facade exports to a more stable public surface.

Demo Screenshots

Description

• Closes Python: [Feature Request] Support custom WorkflowEvent emission for AG-UI/CopilotKit integration #3872
• Closes Python: [Feature]: [Python]: Timeline and Plans for First-Class UI Events Support in AG-UI #3632
• Closes Python: Does AG-UI Support Image Inputs for Agents? #3514
• Addresses Is it possible to run a multi-agent workflow with AG-UI? #3227

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the Contribution Guidelines
• All unit tests pass, and I have added new tests where possible
• Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

[markwallace-microsoft]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/ag-ui/agent_framework_ag_ui
_agent.py	36	1	97%	62
_agent_run.py	405	82	79%	147–154, 193–194, 201, 298, 310, 314, 316–317, 333, 360–361, 416–418, 430–432, 586, 588–589, 642, 653–654, 661, 710–712, 729, 735, 743, 745, 753, 783–789, 792–795, 797–806, 809, 817–820, 827, 830–831, 836, 842–844, 848, 853–857, 883–885
_client.py	151	17	88%	85–86, 90–94, 98–102, 265, 295, 469–471
_endpoint.py	72	12	83%	68, 104, 108–110, 114–118, 136–137
_event_converters.py	80	0	100%
_http_service.py	46	0	100%
_message_adapters.py	590	129	78%	92, 102–103, 112–115, 118–122, 124–129, 132, 141–147, 150, 154–156, 165–167, 187, 193–195, 225, 238–239, 249–250, 290–291, 293–295, 302, 310, 323–324, 326, 338–339, 341–342, 357–359, 361, 366, 400–403, 405–407, 414, 416, 422–424, 426–429, 438–439, 450, 454–456, 463, 466, 510, 513, 515, 518, 521, 537, 554, 576, 607, 612, 623–624, 672, 688–689, 755–758, 760, 766, 774–775, 777, 781–784, 797, 852, 886–889, 891, 956, 994, 997, 999, 1002, 1004, 1010–1011, 1013
_run_common.py	194	31	84%	46, 51–52, 54, 61, 65, 69–70, 72, 96, 173, 191–192, 204–205, 219, 246–248, 273–274, 280–285, 372, 374, 377–378
_types.py	42	0	100%
_workflow.py	44	1	97%	72
_workflow_run.py	502	121	75%	72, 82, 89, 112, 133–134, 154, 166, 173, 177, 181, 185–186, 192, 194, 202–206, 212, 218, 233, 238, 241, 257, 261, 266, 269, 276, 279–287, 291–293, 295, 304–305, 308, 319, 348–349, 353, 358, 370, 372, 375, 386, 397, 404, 408, 419, 421, 431, 437–439, 441–445, 447–450, 454–462, 470, 477–478, 483, 487–493, 497–503, 515, 543, 556–558, 585–586, 589–593, 600, 632, 641, 657, 686–688, 700, 722
packages/ag-ui/agent_framework_ag_ui/_orchestration
_helpers.py	108	0	100%
packages/orchestrations/agent_framework_orchestrations
_handoff.py	387	59	84%	105–106, 108, 161–171, 173, 175, 177, 182, 279, 285, 317, 340, 362, 418, 443, 501, 533, 590–591, 623, 631, 635–636, 674–676, 681–683, 799, 802, 809, 814, 876, 881, 888, 898, 900, 919, 921, 1003–1004, 1036–1037, 1119, 1126, 1198–1199, 1201
_orchestrator_helpers.py	15	2	86%	66–67
TOTAL	21342	3405	84%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
3983	225 💤	0 ❌	0 🔥	1m 12s ⏱️

[Copilot]

Pull request overview

This pull request adds first-class AG-UI workflow execution support, hardens streaming error handling, and introduces a dynamic handoff demo. The changes enable workflows to be served directly through AG-UI endpoints alongside agents, add multimodal input support, and improve error handling to emit RUN_ERROR events instead of silently terminating streams.

Changes:

• Introduces AgentFrameworkWorkflow wrapper for native Workflow objects with thread-scoped factory support
• Breaking change: renames run_agent() to run() across all AG-UI protocol implementations
• Adds streaming error handling that emits RUN_ERROR events for stream/encoding failures
• Implements multimodal input parsing for image/audio/video/document content types
• Adds comprehensive handoff workflow demo with tool approval gates

Reviewed changes

Copilot reviewed 53 out of 56 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
python/packages/ag-ui/agent_framework_ag_ui/_workflow.py	New workflow wrapper with thread-scoped factory support
python/packages/ag-ui/agent_framework_ag_ui/_workflow_run.py	Native workflow-to-AG-UI event stream mapper
python/packages/ag-ui/agent_framework_ag_ui/_run_common.py	Shared emission helpers extracted from agent/workflow runners
python/packages/ag-ui/agent_framework_ag_ui/_agent.py	Breaking change: run_agent() → run()
python/packages/ag-ui/agent_framework_ag_ui/_endpoint.py	Error handling improvements and workflow protocol support
python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py	Multimodal input parsing with base64 decoding
python/packages/orchestrations/agent_framework_orchestrations/_handoff.py	JSON serialization fix for handoff function results
python/samples/demos/ag_ui_workflow_handoff/backend/server.py	New handoff demo backend
python/samples/demos/ag_ui_workflow_handoff/frontend/	React frontend for handoff demo

Files not reviewed (1)

• python/samples/demos/ag_ui_workflow_handoff/frontend/package-lock.json: Language not supported

[TaoChenOSU]

Suggested change

	self.name = name if name is not None else getattr(workflow, "name", "workflow")
	self.name = name if name is not None else workflow.name

[TaoChenOSU]

Suggested change

	self.description = description if description is not None else getattr(workflow, "description", "")
	self.description = description if description is not None else workflow.description or ""