Skip to content

9 · Watching a long solve

Most verifications finish in under a second. Some, such as an enumeration of every way a policy breaks, a large random forest, or a hunt over a big rulebook, take minutes. While a job runs, the service streams what the engine is doing, frame by frame, and this chapter says how to read it, how to resume it, and what to do where a network will not let a stream through.

The stream

GET /v1/solves/{id}/events
Accept: text/event-stream
Last-Event-ID: 41                 (optional; ?after=41 is the same)
Authorization: Bearer qls_live_…

The stream first replays everything the job has already recorded, then follows it live, and closes when the job reaches a terminal state. From the client, stream=True on any verification method gives you the frames as an iterator that ends at the terminal one:

frames = client.verify_guardrail(
    "gemini.co", "respond with permitted investment answer", "has_advising_permission",
    pins={"personal_recommendation": 1}, profile="enumerate", stream=True,
)
for event in frames:                      # resumed by itself if the connection drops
    if event.type in ("phase", "verdict", "succeeded", "failed", "stream_degraded"):
        print(event.type, event.data.get("phase") or event.data.get("status") or "")

With curl, pass -N. Without it, curl buffers the response and prints the whole solve when it finishes, which looks exactly like the endpoint not working.

The frames

Every frame has the same envelope: id, type, job_id, seq, at and data. Only data varies, and type is one of a fixed set:

Event Meaning
accepted the request validated and a job id exists
queued the job is waiting for a worker; re-emitted after a lease lapses, with the raised attempt count
leased a worker claimed it
lease_lost the worker holding it stopped renewing its lease; the attempt is abandoned and the job goes back to the queue
started the binary is executing, and the argv it was executed with
compiled the policy parsed and the circuit exists; the compile-time refusals have been evaluated by this point
generation a step of an assisted authoring request: the model asked, its answer received, the draft compiled, a round finished
progress a solver progress sample
phase a solver phase began or ended
verdict the verdict projection, as soon as the payload is parsed and before the artefacts are uploaded
artifact one artefact finished uploading and can be fetched
succeeded terminal: the run worked and a payload was parsed, whatever the verdict in it
failed terminal: no payload was produced; carries the classified reason and the stderr tail
cancelled terminal: the caller withdrew the job
drain_requeue a deploy's drain timeout was exceeded with the job still running; the worker is deliberately letting its lease lapse so the job is re-queued rather than lost
expired the job was never claimed and its time to live ran out; terminal, and nothing ran
refused a rule the edge could not evaluate was evaluated with the policy in hand and the request refused; terminal, and no solve budget was spent
heartbeat the stream is alive; no state changed

The terminal frame carries the whole job, exactly as GET /v1/solves/{id} would return it. A client that watched a job to its end needs no further request to learn the verdict.

What the progress numbers say

A progress frame is a sample from the phase that is running, and a phase frame marks a phase beginning or ending, with the reason the engine gives. Four phases report progress:

  • phase0, the SAT solver: how many facts it has settled (unary), how many two- and many-variable relations it has learned (binary_xor, multi_xor), and how many rounds have gone by without one.
  • cascade, the algebraic expansion: the rank reached and how much it grew since the last sample (rank_delta). Rank climbing is the expansion working; rank still is the expansion finished, or stuck.
  • phase3, the closure that counts every solution: its round, the basis it has built, and the obstructions left. While it runs the rank is frozen and says nothing; basis climbing while obstructions falls towards zero is the honest signal of progress.
  • seatbelt, the repair search: how many candidates it has scanned of the total, the best score found and the target it stops at. best is null until something has been scored.

A short run emits almost no samples, and its phase boundaries are the whole story. A property table marks each row with a solve_begin boundary carrying the row's id, so one row's work is never attributed to another.

There is no percentage

Deliberately. The engine cannot know how long a closure will take until it has finished, so a progress bar would be a guess presented as a measurement. The numbers above are what is known, and the frames carry them as they are.

Resuming

id is the frame's sequence number, so Last-Event-ID is a cursor: send back the last id you saw, and you receive everything after it, once each. ?after=N is the same cursor for an HTTP stack that will not set the header. Across any number of reconnects, the frames you receive are the job's sequence exactly, once each. The client resumes for you.

A worker that fails mid-job is shown, not smoothed over: the stream carries lease_lost, then leased and started again, with attempt increased. The engine's own clock restarts from zero while seq continues, and every frame carries attempt, so a chart can keep the two runs apart.

A comment, : heartbeat <time>, arrives every 15 seconds, shorter than any proxy's idle timeout, so a quiet stretch of a long solve does not get the connection cut. A stream on which nothing at all arrives for twenty minutes is closed; reconnect with your cursor and it continues where it stopped.

When a stream will not get through

Some networks buffer or cut long-lived responses. The same frames are available as a poll: GET /v1/solves/{id}/events?mode=poll&after=N returns a bounded JSON array, read from the same cursor in the same order, and GET /v1/solves/{id} carries a progress block, the newest frame per phase, with the seq to poll from. It is the same information at a coarser rate.

The client falls back to polling by itself after two failed streams, and says so with one stream_degraded frame before it carries on. If a stream arrives all at once, at the end, there is a buffering proxy between you and us; chapter 15 says how to find it. stalwart doctor checks whether streaming gets through your network.

The limits on streams

Limit Value When it is reached
Streams per key your plan's streams 429 stream_limit
Streams per job 8 429 stream_limit_job
New streams per key per minute 60 429 stream_limit_rate

Each carries Retry-After. A key revoked while a stream is open is disconnected within 15 seconds.

What a stream is not

  • Not a control. The stream is read-only. To cancel a job, DELETE /v1/solves/{id}.
  • Not a webhook. The service makes no outbound calls to you; a team that wants a push builds it on this stream.
  • Not the verdict. Only the terminal frame, or the job itself, carries the verdict. A progress frame that looks promising promises nothing.

The limits

  • Progress numbers describe the engine's work, not how much of it is left.
  • A dropped stream loses nothing if you resume with your cursor; a client that starts again from the beginning receives the frames again.
  • Only the terminal frame, or GET /v1/solves/{id}, is the answer.