Agent State and Checkpoints: Planning Beyond One Tool Call

In AI Foundations #35, we separated the model’s proposal to use a tool from the application’s execution of that tool. But real tasks often need several steps: search for information, compare sources, create an artifact, inspect it, and correct problems. How can an agent know which steps actually succeeded?

This is AI Foundations #36: state, planning, checkpoints, and verification. All examples are conceptual, not records from a production agent.

One tool call versus a multi-step loop

A one-off tool call has a straightforward path: validate arguments, execute an allowed action, read its result, and answer. A multi-step agent repeats this process and must decide when to change its plan or stop.

User goal
   |
Choose a permitted next step
   |
Execute tool -> capture result or error
   |
Update working state
   |
Is the goal verified?
   | yes -> finish
   | no  -> revise plan or stop at a limit

The application, not the model’s confident prose, should enforce permission boundaries and termination criteria. Anthropic’s Building Effective Agents emphasizes environmental feedback and stopping conditions in agent loops.

The loop is useful only if its intermediate observations are treated as evidence, not as magical proof of everything the agent intended.

Four things people call “memory”

State comes in several forms with different evidentiary value.

The OpenAI Agents SDK sessions guide describes maintaining conversation history across agent runs. That helps continuity, but a session is not an authoritative record of changes made in an external system.

A model statement such as “I created the document” is a claim. The provider returning a resource ID is stronger. Fetching that ID and comparing the contents with the intended document is stronger still.

Plans are hypotheses, not receipts

Imagine a task to compare three public specification pages. A plausible plan is:

1. Retrieve the original sources.
2. Extract comparable fields.
3. Resolve disagreements or mark uncertainties.
4. Produce a summary with citations.
5. Verify source coverage and final output.

The plan should change if a source is unavailable or uses incompatible units. The agent must not silently fill in missing numbers merely to complete its checklist.

A small working-state record could look like this:

{
  "goal": "Compare three public specifications",
  "step": "check_source_coverage",
  "status": "IN_PROGRESS",
  "observed": ["source_a_found", "source_b_found"],
  "unknown": ["source_c_unavailable"],
  "next": "report_gap_or_retry_within_budget"
}

This JSON is an illustrative application schema, not an official SDK interface. The crucial distinctions are unknown versus false, planned versus completed, and tool-reported versus independently verified.

Capture provenance with each observation

An agent can receive structured JSON, text, webpages, or files from tools. A useful execution record captures the tool identity, request identifier, timestamp, success or error status, and any validated output identifier. These details help establish what was actually observed.

Ask three questions of every result:

• Origin: Which real tool produced it, and for which request?
• Freshness: Does its timestamp still fit the user’s task?
• Scope: Did the result cover the full resource or just a limited page of it?

A search returning two results does not prove that only two results exist. A task marked “queued” has not necessarily executed. An HTTP success response can confirm acceptance without proving final visibility.

Tool-returned content is also not an instruction channel. If a retrieved page says to ignore user permissions or reveal private data, that text is part of the source being read, not new authorization for the agent.

Checkpoints prevent duplicate side effects

For an informational question, a durable checkpoint may be unnecessary. For a workflow that uploads files, sends messages, changes records, or publishes articles, it can be essential.

An illustrative checkpoint:

{
  "task_id": "demo-042",
  "step": "save_report",
  "status": "VERIFIED",
  "provider_result_id": "example-id",
  "verification": "readback_matched"
}

Record only what the system has actually checked. Do not write “published” when the best evidence is “upload request accepted.”

After a timeout, the result may be unknown, not necessarily failed. The server could have processed the request before the connection dropped. Blindly repeating a non-idempotent action may create duplicate comments, emails, or financial transactions.

A safe recovery pattern is to ask the provider whether the intended action exists, verify any returned identifier, and retry only if the outcome is confirmed absent and the user’s authorization still applies. Use provider-supported idempotency keys when available.

Verification is not a single yes/no flag

Different claims require different evidence:

These boundaries matter because a locally generated artifact, a committed file, and a deployed public page are not the same state. The agent should report the highest phase actually verified and preserve the rest as unfinished.

The principle extends beyond publishing: an API saying it accepted a job is not evidence that the downstream work completed successfully.

Use an explicit state machine

A small application can enforce transitions such as:

PLANNED -> EXECUTING -> OBSERVED -> VERIFIED -> DONE
                |
                +---> FAILED
                |
                +---> UNKNOWN -> RECONCILE

The UNKNOWN branch is important. A lost response after submission can leave the side effect uncertain. Reconciliation is preferable to treating uncertainty as failure and automatically replaying the action.

Similarly, a tool may succeed while the user’s goal fails. A generated comparison can be structurally valid yet omit one of the requested sources. That requires an honest partial status or correction, not a fabricated completion report.

Budgets, approval, and traceability

Every agent loop needs limits on tool calls, elapsed execution, retries, and side effects. When steps stop producing new information, more tool calls can make a system slower without improving correctness.

Actions with serious consequences should require permission scoped to the specific resource and operation. General permission to inspect documents is not permission to delete them. A model choosing an action does not grant that permission.

The OpenAI Agents SDK overview documents tools, handoffs, sessions, guardrails, and human-in-the-loop capabilities. Its tracing guide explains capturing events such as model generations, tool calls, and handoffs.

Traces help debug what an agent attempted. They do not replace external readback verification. Traces can also contain prompts or sensitive tool results, so logging requires deliberate access and retention controls.

A focused agent reliability test

To test the loop without putting real systems at risk, use synthetic sources and a mock action endpoint. Evaluate separate injected scenarios:

Grade verified task completion, unwanted duplicate actions, unauthorized side effects, and accurate uncertainty reporting separately. A fluent final answer does not compensate for a failed external action.

What to remember

A reliable agent distinguishes what the user asked, what the model planned, what a tool reported, and what the application verified. More steps do not create more truth by themselves. Checkpoints, authorization, provenance, and termination rules make those steps controllable.

Next, the series can compare single-agent workflows with handoffs and specialist agents, including who owns the final result.

Primary sources

• Anthropic: Building Effective Agents
• OpenAI Agents SDK: Overview
• OpenAI Agents SDK: Sessions
• OpenAI Agents SDK: Tracing