AI Foundations #37: Handoffs vs Agents as Tools
AI Foundations #37 explains when a specialist should take over a conversation, when a manager should call one as a tool, and how to test ownership, privacy boundaries, and final results.
Approximately 6 min read · AI Foundations / Lesson 37
In AI Foundations #36, we examined how agents preserve state, distinguish observations from plans, and verify external effects. That leads to a new decision: should one agent complete the task, or should several specialists divide it?
AI Foundations #37 is about control ownership. In multi-agent systems, the distinction between a handoff and a specialist used as a tool is more important than the number of agents in the diagram.
Start with a single-agent baseline
Suppose a user asks for a comparison of three public API specifications. A single agent can retrieve the documents, extract relevant fields, check for discrepancies and return a sourced table. It may already be accurate and easy to audit.
Adding a router, a researcher, a reviewer and a writer creates new interfaces. Each interface can lose context, duplicate work, or obscure who approves the final answer. Multi-agent orchestration can help when responsibilities truly differ, but it is not automatically an accuracy upgrade.
Anthropic’s Building Effective Agents describes orchestration patterns, including a manager assigning tasks to workers and an evaluator feeding corrections into a loop. Treat those as design choices that need evidence, not as a requirement that every agent workflow become complex.
Pattern A: The manager uses a specialist as a tool
A manager remains responsible for the user-facing task. It calls a specialist for a bounded subtask, receives the specialist’s result, and decides what to do next.
User -> Manager
|
+--> Research specialist -> findings
|
+--> Validation specialist -> checks
|
+--> Manager verifies and answers user
The OpenAI Agents SDK orchestration guide calls this the agents-as-tools pattern. In the Python SDK, an agent can be exposed to a manager through Agent.as_tool(). The caller remains active after the delegated tool finishes.
This fits a workflow where one owner must combine evidence, enforce a shared completion criterion or produce a single coherent answer. A specialist’s confident prose still is not independent evidence: its cited source and verification result need checking.
Pattern B: A handoff changes the active agent
A triage agent chooses a specialist, then transfers control. The specialist becomes the active agent for the rest of that turn.
User -> Triage
|
+-- handoff --> Support specialist -> user
In the Agents SDK handoffs guide, handoffs are represented as tools visible to the model. The framework provides a handoff target, optional descriptions, callbacks, and filters for the input history. A handoff is not merely asking another agent for a paragraph and then returning to the original manager; the active agent actually changes.
This design is useful when routing to the specialist is itself the correct decision: for example, when a domain-specific assistant should own the rest of an interaction. It requires deliberate routing rules and input boundaries.
Comparison: who owns what?
| Dimension | Specialist as tool | Handoff |
|---|---|---|
| Active owner after delegation | Original manager | Receiving specialist |
| Specialist’s job | Bounded contribution | Continue the interaction |
| Final synthesis | Usually manager | Usually receiving specialist |
| Input sharing | Explicit tool input | Handoff history/filter configuration |
| Primary risk | Manager trusts unchecked sub-results | Wrong route or excess context passed |
The exact behavior depends on the SDK and application. The Agents SDK Agent reference distinguishes Agent.as_tool() from handoffs by control ownership and what context the receiving agent gets.
Context is not permission
A handoff may give the receiving agent access to more conversation history than it needs. The SDK documents input filters and history-mapping controls to narrow what it sees. Those are data-flow controls, not a guarantee that a specialist is authorized to take an external action.
A safer design separates at least three things:
- Routing: Does this task belong to the specialist?
- Disclosure: Which user data or source material does it genuinely need?
- Authority: Which side effects is the application allowed to execute?
For instance, a document-review specialist may be allowed to read a supplied public document but not to send an email or delete a record. The receiving model cannot grant itself that permission simply because a manager delegated to it.
The handoff documentation also distinguishes handoff metadata from local application context. A model-generated reason for routing is not a trusted authorization token.
What about error recovery and checkpoints?
A single manager can own a common checkpoint ledger, or individual specialists can publish structured, scoped results. Both require a rule for what counts as verified completion.
Imagine a research workflow that asks a specialist to retrieve a specification. An appropriate returned record might include source URL, retrieval time, extract, and any unresolved ambiguity. The manager may then compare it to the original source before marking the evidence as usable.
When a handoff changes active ownership, responsibility for final-state verification must be explicit. The receiving specialist should not infer that the original router already completed actions that it only proposed.
Read the previous lesson’s checkpoint model before building a multi-step delegation chain.
A fair way to evaluate multi-agent designs
A good test compares the same tasks and success criteria under three designs: one capable agent, one manager with a bounded specialist tool, and a routed handoff. Use synthetic inputs where private data or external side effects would otherwise be involved.
Track outcomes separately:
| Metric | Why it matters |
|---|---|
| Correct, source-supported completion | The actual user goal |
| Unnecessary handoffs | Added complexity without value |
| Missing or excess context transfer | Reliability and privacy |
| Duplicate or unauthorized external effects | Safety of actions |
| Tool calls and latency | Operating cost |
| Accurate incomplete-status reporting | Avoids false completion |
The test should not count the number of agents as a positive metric. An orchestration design wins only if it improves a meaningful outcome enough to justify the additional interfaces.
Decision rule
Start with the smallest architecture that can do the task correctly. Use an agent as a tool when a manager needs a bounded specialist contribution but retains accountability. Use a handoff when the specialist should genuinely become the active owner. In both cases, keep authorization in the application, minimize shared context, and check final claims against external evidence.
The next Foundations lesson can examine how evaluation and guardrails measure such systems without confusing fluent text with successful action.
Primary sources
- OpenAI Agents SDK: Agent orchestration
- OpenAI Agents SDK: Handoffs
- OpenAI Agents SDK: Agent reference
- Anthropic: Building Effective Agents