Kill switch

A canary trip can stop the model where it runs. Set provenance.on_trip on a traced inference, or make it your account default:

| Mode | What happens on a trip | | --- | --- | | flag (default) | The completion is returned. The readings carry tripped: true. | | block | The completion is withheld: choices[0].message.content is null and finish_reason is "canary_tripped". The readings are kept. | | block_and_lock | Block, and lock the agent session or agent named in provenance.session until a person clears it. |

The response is still an OpenAI-compatible 200, so existing clients keep working and can branch on finish_reason.

curl $API/v1/trace/chat/completions \
  -H "Authorization: Bearer $PRV_ACCOUNT_KEY" -H "Content-Type: application/json" \
  -d '{
    "model": "qwen2.5-0.5b-instruct", "max_tokens": 40, "temperature": 0,
    "messages": [ … ],
    "provenance": {
      "on_trip": "block_and_lock",
      "trip_thresholds": { "prompt_injection": 0.9 },
      "session": { "agent_id": "agt_…", "agent_session_id": "sess-42" }
    }
  }'

{
  "choices": [{ "index": 0, "message": { "role": "assistant", "content": null }, "finish_reason": "canary_tripped" }],
  "provenance": {
    "trace_id": "trc_…",
    "canary_readings": { "prompt_injection": { "score": 1, "tripped": true, "threshold": 0.9 } },
    "kill_switch": {
      "mode": "block_and_lock",
      "decision": "block",
      "preflight": false,
      "preflight_blocked": false,
      "tripped_families": ["prompt_injection"],
      "thresholds": { "prompt_injection": 0.9 },
      "lock": { "lock_id": "lck_…", "status": "active", "agent_session_id": "sess-42", "actions": { "agent_suspended": false, "session_revoked": false } }
    }
  }
}

provenance.kill_switch appears whenever the request or your account sets the kill switch. Without either, the response is exactly what it was before. With block_and_lock and no provenance.session, the request blocks and lock is null. An agent_id must be one of your registered agents (404 resource_missing otherwise).

Thresholds

Each family's score is in [0, 1]. A family is tripped when score ≥ threshold. With no threshold set, the threshold is 1 (at least one member reached its calibrated activation), as before. Set thresholds per family:

• per request: provenance.trip_thresholds: { "prompt_injection": 0.6 }
• per account: thresholds in the account setting below

Request thresholds merge over the account's. The readings in the response, and the canary_hash bound into tokens, show the threshold that was applied. On the sandbox fixture model the clean vendor email reads prompt_injection at 0.678: it passes at the default threshold and trips at 0.6.

Preflight

provenance.preflight: true reads the canaries over the prompt before generating. If a family trips under block or block_and_lock, the model never runs: finish_reason: "canary_tripped", completion_tokens: 0, and only the prompt tokens are billed. In flag mode preflight is skipped.

Stop during generation

When the kill switch is set, the worker reads the canary neurons on every token it generates and stops the model on the step where a family reaches its threshold. The check runs inside the model process, between tokens, so a block stops the model before it writes the rest of the completion.

• Prompt readings stay the same. canary_readings is the prompt reading, byte-identical to the reading without the kill switch, so canary_hash doesn't change.
• Generated tokens get their own readings. generation_readings holds each family's highest score over the tokens the model wrote, judged with the same thresholds.
• The stop point is reported. kill_switch.trip names the family, the phase (prefill when the prompt itself tripped, generation otherwise), the token_index of the completion token whose reading crossed the threshold, and its score. kill_switch.stopped_at_token is the number of completion tokens written before the stop.
• Billing stops at the trip. usage.completion_tokens counts only the tokens before the stop.
• flag never stops. It records the trip point and returns the whole completion.

{
  "choices": [{ "index": 0, "message": { "role": "assistant", "content": null }, "finish_reason": "canary_tripped" }],
  "usage": { "prompt_tokens": 70, "completion_tokens": 31, "total_tokens": 101 },
  "provenance": {
    "canary_readings": { "credential_exfiltration": { "score": 0.3071, "tripped": false, "threshold": 0.35 } },
    "generation_readings": { "credential_exfiltration": { "score": 0.3919, "tripped": true, "threshold": 0.35 } },
    "kill_switch": {
      "mode": "block",
      "decision": "block",
      "source": "in_model",
      "stopped_at_token": 31,
      "trip": { "family": "credential_exfiltration", "phase": "generation", "token_index": 30, "score": 0.3919 },
      "tripped_families": ["credential_exfiltration"],
      "thresholds": { "credential_exfiltration": 0.35 },
      "lock": null
    }
  }
}

kill_switch.source says what decided:

| source | Meaning | | --- | --- | | in_model | The worker checked every token and could stop between them. | | post_hoc | Only the prompt reading decided, after generation (a worker image without the per-token check, or a sandbox completion without a per-token recording). | | preflight | A preflight read stopped the request before generation. |

Under block_and_lock, an in-model stop writes the lock, its ledger row and the webhook right after the response. The response carries lock: { "lock_id": "lck_…", "status": "pending" }; fetch it with GET /v1/trace/session-locks/:lock_id.

The per-token check reads only the canary members' columns. Its budget is under 1 ms per token at p99 on a GPU worker; the worker reports its measured overhead, and the ops alert trace_in_model_overhead_high fires when it goes over.

Sandbox: the fixture worker replays per-token activations recorded on Qwen2.5-0.5B-Instruct for the injected vendor email. With canary_sets: ["credential_exfiltration"] and a threshold of 0.35, the model stops at token 30, partway through the account number. The fixture canary sets are example sets on a small model and also fire on ordinary text while it generates; block on validated sets in production.

Account defaults

curl $API/v1/account/trace-kill-switch -H "Authorization: Bearer $PRV_ADMIN_KEY"
# → { "configured": false, "default_mode": "flag", "thresholds": {}, "auto_suspend_agent": false, "auto_revoke_session": false }

curl -X PUT $API/v1/account/trace-kill-switch -H "Authorization: Bearer $PRV_ADMIN_KEY" -H "Content-Type: application/json" \
  -d '{ "default_mode": "block_and_lock", "thresholds": { "prompt_injection": 0.8 }, "auto_suspend_agent": true, "auto_revoke_session": true }'

# Back to the defaults (flag, no thresholds, no auto-actions)
curl -X DELETE $API/v1/account/trace-kill-switch -H "Authorization: Bearer $PRV_ADMIN_KEY"

Anyone on the account can read the setting. Changing it needs an owner or admin dashboard session, or an API key with the admin scope. A request's on_trip always wins over default_mode. You can also set it in the dashboard under Account → Trace kill switch.

Session locks

While a lock is active, /v1/intent/verify denies every token for that agent session or agent with session_canary_locked (the response carries lock_id). The check runs right after agent_suspended, before policy.

curl "$API/v1/trace/session-locks?status=active" -H "Authorization: Bearer $PRV_ACCOUNT_KEY"
curl $API/v1/trace/session-locks/lck_… -H "Authorization: Bearer $PRV_ACCOUNT_KEY"
curl -X POST $API/v1/trace/session-locks/lck_…/clear -H "Authorization: Bearer $PRV_ADMIN_KEY" \
  -H "Content-Type: application/json" -d '{ "note": "Reviewed the vendor email; safe to resume." }'

Clearing needs an owner or admin session or an admin-scope key, and is recorded in the decision ledger (kind lock_clear). The lock itself is ledgered too (kind lock). You can also clear locks in the dashboard under Trace → Locks.

Two optional auto-actions run when a lock is created, from your account setting:

• auto_suspend_agent suspends the registered agent, so it can't authorize new actions (agent_suspended).
• auto_revoke_session revokes the agent session (revocation scope session, token_revoked), which also fires intent.revoked.

The lock's actions field records which ran. Clearing a lock also reactivates the agent when that lock suspended it (actions.agent_suspended_by_lock is true), and the unsuspend is recorded in the decision ledger. An agent a person suspended stays suspended; resume it with POST /v1/agents/:id/unsuspend.

Check a reading in one call

Sidecar users (and anyone holding a trace id) can turn a reading into a decision with POST /v1/trace/check:

curl $API/v1/trace/check -H "Authorization: Bearer $PRV_ACCOUNT_KEY" -H "Content-Type: application/json" \
  -d '{ "trace_id": "trc_…", "mode": "block_and_lock", "thresholds": { "prompt_injection": 0.8 }, "session": { "agent_session_id": "sess-42" } }'

{
  "decision": "block",
  "mode": "block_and_lock",
  "trace_id": "trc_…",
  "source": "sidecar",
  "tripped_families": ["prompt_injection"],
  "readings": { "prompt_injection": { "score": 0.91, "tripped": true, "threshold": 0.8 } },
  "thresholds": { "prompt_injection": 0.8 },
  "canary_hash": "…",
  "lock": { "lock_id": "lck_…", "status": "active", "…": "…" },
  "decision_id": "01J…"
}

decision is allow when nothing tripped, flag on a trip in flag mode, and block on a trip in either block mode. The reading comes from a hosted traced inference, or from a sidecar reading for that id. An unknown id, or one from another account, is 404 resource_missing.

A reading older than 15 minutes (TRACE_READING_MAX_AGE_S) is never allowed. The response is decision: "stale" with reason: "reading_too_old" and the reading's age in reading_age_s. It is not metered and creates no lock. For after-the-fact review of an older reading, pass max_age_s (1 second to 30 days). Each check is recorded in the decision ledger (kind check) and counts as one trace_canary_read.

Authorize also binds sidecar-only trace ids: pass the sidecar's trace_id to /v1/intent/authorize and the token carries its canary_hash.

Webhook

A block fires trace.canary_tripped with:

| Field | Meaning | | --- | --- | | trace_id, canary_hash, canary_readings | the reading that tripped | | tripped_families | sorted family slugs | | mode | block, block_and_lock, or policy (a verify-time canary policy deny) | | source | inference, preflight, check — or the verify reading's source | | agent_id, agent_session_id | from provenance.session (or the token, on verify) | | lock_id | the lock created, else null | | jti, matched_rule_id | set on verify-time policy denies |

Pricing

Blocking, locking, clearing and the account setting are free on every plan. POST /v1/trace/check is one trace_canary_read (within your monthly allowance, then metered). A preflight block bills the prompt tokens only. A stop during generation bills the completion tokens written before the stop.

SDKs

guard wraps your function: it runs the traced completion with your kill-switch options, throws CanaryTrippedError on a block, and otherwise calls your function with the completion.

import { CanaryTrippedError } from "@provenance/sdk";

const summarize = prov.trace.guard(async (completion) => completion.choices[0].message.content, {
  onTrip: "block_and_lock",
  thresholds: { prompt_injection: 0.8 },
  session: { agentId, agentSessionId },
});

try {
  await summarize({ model: "qwen2.5-0.5b-instruct", messages });
} catch (err) {
  if (err instanceof CanaryTrippedError) console.log(err.trippedFamilies, err.stoppedAtToken, err.trip);
}

const c = await prov.trace.check({ traceId, mode: "block" });
const locks = await prov.trace.sessionLocks.list({ status: "active" });
await prov.trace.sessionLocks.clear(locks.data[0].lockId, { note: "reviewed" });

from provenance import CanaryTrippedError

summarize = prov.trace.guard(lambda c: c["choices"][0]["message"]["content"],
                             on_trip="block", thresholds={"prompt_injection": 0.8})
try:
    summarize(model="qwen2.5-0.5b-instruct", messages=messages)
except CanaryTrippedError as e:
    print(e.tripped_families)

prov.trace.check(trace_id=trace_id, mode="block")