47 · Agent graph verification¶

Agent Graph Verification checks every route through an agent, or a team of agents, before it goes live. You send the agent's graph and the catalogue its tools are classified by; every tool the graph can reach is held to its check, its confirmation, its order rules and what an undoable write needs, and each route that escapes one comes back as a finding with the path that reaches it. This chapter is the export it reads, what it decides, and how to read what it finds.

An agent built with the Bot Builder is verified this way before its first turn, and in CI by stalwart verify (chapters 45 and 46). This chapter is the same check for any graph: one you built with LangGraph, or one you export yourself.

What it reads¶

Three inputs, to the graph mode:

the export, the graph as a qls-graph/1 document: its nodes, its edges, and where it pauses for a person;
the catalogue, which says what each tool is: a read or a write, how a write is undone, its order rules, whether it reads the outside world, and the rest of the words of chapter 30;
a rulebook, optionally: a Colang policy whose rules the tool check enforces, hunted with the graph.

The export's fields:

Field	Meaning
`nodes`	Each node's `id` -- unique in its graph and holding no `/`, which qualifies a subgraph's nodes by their path -- its `kind`, and the tools a tool node binds.
`edges`	Each edge's `from`, its `kind` (`fixed`, `conditional`, `command` or `stalwart_edge`) and the `destinations` its author declared. A conditional or Command edge exported with `destinations: null` can reach any node, and is reported as `QG015`.
`sends`	`Send` fan-outs: a node that starts several branches at once.
`interrupts`	Where the graph pauses for a person, and the tool whose call the person confirms there.
`subgraphs`	The graphs that `subgraph` nodes run, by name, exported by the same rules and inlined at each use.

A Bot Builder agent writes its export with stalwart verify agent.yaml --export-only graph.json, or Agent.export() in Python. A LangGraph graph is exported with export_graph (chapter 27), which reads the compiled graph and names the nodes where a model chooses.

Asking¶

result = client.verify_graph("desk.graph.json", catalogue="desk.catalogue.json", session_steps=2)
for finding in result.findings:
    print(finding.rule, finding.severity, finding.location)   # the line that declared the node
    print("  fix:", finding.fix)

The request is a job like any other (chapter 3): mode: "graph", the export and the catalogue as inputs, and session_steps as an operand. It needs the graph_check grant, each job is one graph_checks, and a graph is held to your plan's graph_nodes, counted once its subgraphs are inlined where they are used (chapter 16).

What it decides¶

For each tool, the graph is searched for a route that breaks one of these:

Obligation	Raised by	Broken when	Reported as
The tool check	Every tool a node binds	A path runs the tool on a `tools_plain` node, where its call skips the check.	`QG001`
A confirmation	A write the graph interrupts for	A path runs the write on a plain node before the person has confirmed it.	`QG001`
The order rule	`order.after`	The tool runs before every one of the tools it must come after.	`QG014`
The order rule	`order.not_after`	The tool runs after one it must never follow, or after any tool that reads the outside, where the rule names `untrusted`.	`QG014`
The order rule	`order.at_most`	The tool runs more times than its cap.	`QG014`
Taint	A write, and every tool marked `reads_outside`	The write runs, unconfirmed, after a tool that read the outside: in the same agent, or after a hand-off from another.	`QG011`, `QG012`
The saga	A compensable write and an `irreversible` one	The compensable write can run after the irreversible one.	`QG013`
Recovery	A write with no `compensation` that is not `irreversible`	Another write shares a path with it.	`QG105`

A route that breaks one is a finding, and each finding names the rule it is reported under:

Rule	Title	Raised when	How to fix it
`QG001`	Missing check	the action is reachable and nothing in the rules relates it to the control	Add the control as a condition on every route that registers the action. The patch names the condition and the flows to put it in; a new flow cannot tighten the action, because guards on one action merge as an OR.
`QG003`	Multi-turn escape	the obligation holds at one turn and fails at the declared conversation depth	Bind the control to the step that uses it -- read it in the same turn as the action, or carry it with the call it approved -- and re-scan at the same depth.
`QG011`	Confused deputy	a privileged tool is reachable under taint with no confirmation or refusal	Bind the tool to the guarded tool node, so a call carrying the label is confirmed or refused; or mark the source tool reads_outside where it is not yet, and re-scan.
`QG012`	Cascading injection	as a confused deputy, where the taint entered in another agent or subgraph than the one acting	Bind the tool to the guarded tool node in the acting agent, so a call carrying a label from the handing agent is confirmed or refused; or route the hand-off through a confirmation.
`QG013`	Irreversible before undoable	a compensable write can run after an irreversible one on a path, without the irreversible step's confirmation having said so	Move the irreversible step last, or bind it to the guarded tool node so its confirmation says that the later steps cannot undo it.
`QG014`	Order rule escapable	a tool is reachable with an order rule's latch in the forbidden state	Bind the tool to the guarded tool node, where its order rules are checked over the session's history, and re-scan.
`QG015`	Undeclared route	a conditional or Command edge with no declared destinations	Declare the edge's destinations: a path_map, a Command[Literal[...]] annotation, or a stalwart_edge with the allowed set.
`QG016`	Unmodelled path	an open attacker campaign realised a privileged call on a graph where the engine found no way through	Re-export the graph from the code that runs and re-scan; then compare the attacker's trace with the export to find what the export is missing.
`QG105`	Unrecoverable write	a write with no compensation, not marked irreversible, on a path with another write	Declare the tool's compensation in the catalogue, or mark it irreversible so a person confirms it knowing so.
`QG107`	Unauthenticated agent channel	a hand-off to or from a remote agent travels over a channel the catalogue declares without authentication	Declare and use mtls, oauth2 or signed_messages on the channel; or accept the hotspot with the layer that authenticates it.

The full page of each rule, with its category, its AI RMF functions and its OWASP entries, is under the rule pages.

Reading a finding¶

Each finding carries:

the rule and its severity, by what the tool can do: a destructive or irreversible write is Critical, a write High, a read Medium;
the witness: the path through the graph that reaches the tool, node by node, with the choices the models made along it;
the location: the line of the file that declared the node, so a finding on an agent.yaml opens at the agent's line;
the fix, in the catalogue's or the graph's terms;
a fingerprint that survives unrelated edits, so the same finding is recognised scan after scan.

The findings come back as SARIF as well, for code scanning. A witness can be replayed on your own graph, with every tool a stub and a scripted model making the witness's choices: replay_witness, or stalwart verify --replay for a Bot Builder agent. A finding that does not reproduce is a disagreement between the engine and the graph that runs, and is reported first.

How deep it looks¶

A route is searched over a number of conversation turns: session_steps, one where not given. A check read once and relied on later, such as an approval given at one turn and spent at another, can hold at one turn and fail at two; it is reported as QG003, a multi-turn escape. Ask at the depth your conversations reach.

The limits¶

A model node is read as choosing anything. The verification proves no route reaches a tool without its check, its confirmation or its order rule; it does not prove the model chooses well.
A node that is not Stalwart's own is opaque. Its code is not read: it is taken to be able to do anything its edges allow, which is why an undeclared conditional edge, one whose destinations nobody declared, can reach every node and is reported as QG015.
A clean result holds to the depth checked, and no further.
The catalogue's declarations are taken as stated. A tool declared a read that writes is verified as a read. The tools' own code is checked by the code checks, not here (chapter 46).
The graph verified is the export you sent. If the graph that runs differs from it, the result is about the export; replaying the witnesses, and the red team of chapter 33, are how a difference shows.