Skip to content

28 · Hunting bugs in your guardrails

This chapter holds, for now, the parts of the bug hunter that read a finding in OWASP's terms and that match the rails you run against published advisories. The rest of the chapter -- the scan, the rule catalogue, reproductions, fixes, baselines and the gate -- is written from the hunter's own reference and lands with the full guide.

Reading a finding in OWASP's terms

Every finding carries the entries of the OWASP Top 10 for LLM Applications 2026 and the OWASP Top 10 for Agentic Applications 2026 that it is an instance of, each with its edition (LLM01:2026, ASI04:2026), so a security team can file it where it belongs. A rule mapped to LLM03:2026 Excessive Agency also carries the root cause in brackets -- functionality, permissions or autonomy -- as owasp_root_cause. A rule with no OWASP entry says why, in the same place. SARIF carries the same mapping as taxonomies and relationships, and each result is tagged OWASP <id>, so code scanning can be filtered on it.

The fix line of each finding names the hunter's own instruction first, then the mitigation OWASP lists for the entry, in this guide's words.

The coverage statement, at the policy level

Every scan's summary carries a coverage statement: which OWASP entries the hunter has rules for, which had no finding in this scan, and which it has no rule for. The hunter's rules decide the part of each risk that the policy and its catalogue control -- at the policy level -- and nothing beyond it.

With the nine rules described in chapter 31 and below, the hunter has rules for nine of the ten LLM entries and ten of the ten agentic entries. The one entry no rule covers is LLM05:2026 Data and Model Poisoning: a property of training data and model weights, which no guardrail policy decides. The statement names it with that reason in every scan.

A clean scan under an OWASP entry

A clean scan under an OWASP entry means no finding at the policy level, not that the risk is absent. An entry with no finding in this scan is listed as such, which says nothing more than that.

The summary also lists every rule that had nothing to read -- a rule whose declarations the catalogue does not make raises nothing, and says so there -- and every claim the scan used as stated, such as a tool the team declares sandboxed, under assumptions.

Rails with a published advisory

A defect found in a library rail travels with every copy of the file and every configuration that names the rail. The hunter carries a registry of published advisories, and QG022 raises a finding where a scanned flow matches one:

  • by its text: a flow whose normalised text is an advisory's flow -- reindented, with comments added or removed, with line continuations joined -- matches, and a flow with one condition changed does not;
  • by reference: a config.yml that names a library rail an advisory describes matches at the line that names it.

The registry ships with two advisories, QGA-2026-0001 (NeMo's fiddler bot faithfulness output rail) and QGA-2026-0002 (a cart flow published with the ACE examples). Every scan records the registry's advisories_version and advisories_digest in its metadata and in SARIF's run properties, so a finding can always be read against the registry it was raised under.

The version you install: --dependencies

A reference names a rail, not a version. Without a dependency file the finding says version: unknown and names the advisory's affected range. Give the hunter the file that pins what you install -- requirements.txt, pyproject.toml, uv.lock or poetry.lock -- and the reference is checked against the installed version: inside the range it is raised with that version, outside it nothing is raised. Supplying the file narrows the finding; it does not make a new one, so its fingerprint is the same.

from stalwart_agents.client import Client

report = Client().hunt(inputs={"rails_config": "config.yml"}, dependencies="uv.lock")

The hunt mode takes the same file as its dependencies input, and stalwart verify as its --dependencies option.

The limits

  • The hunter works on the rules around actions, not on prompts.
  • A clean scan covers the rules it read and no others. A rule whose declaration is missing raises nothing, and the summary lists it as having had nothing to read.
  • The advisory registry holds the advisories published at the version the scan names; a rail with a defect nobody has published is not found by QG022.