Provenance
Read the docs

Neuron-CI registry diffs

When you fine-tune a model, the neurons your canary policies rely on can move, stop firing, or change meaning. A registry diff compares two neuron registries of the same architecture neuron by neuron, so you can catch that before the new weights ship. A typical pair is the public base model and your private registry build of the fine-tune.

Diffs are Enterprise and metered at $25 per diff (trace_diff), charged when the diff completes. They are computed over registries that are already stored, so they don't need the GPU and they never call the trace backend.

Run a diff

curl $API/v1/trace/diffs -H "Authorization: Bearer $PRV_ENTERPRISE_KEY" -H "Content-Type: application/json" \
  -d '{"base":"qwen2.5-0.5b-instruct@7ae55760","target":"qwen2.5-0.5b-instruct@7ae55760-…","canary_sets":["prompt_injection"],"tau":0.25}'
# → 202 {"diff_id":"tjob_…","status":"pending","poll_url":"/v1/trace/diffs/tjob_…", …}

curl $API/v1/trace/diffs/tjob_… -H "Authorization: Bearer $PRV_ENTERPRISE_KEY"
# → {"status":"completed","pass":true,"report":{…},"manifest":"eyJ…","report_hash":"…"}

| Field | | | --- | --- | | base, target | A model_version_id or slug you can see: public models plus your own private builds. An unknown or foreign model is a 404 resource_missing. | | canary_sets | Families to check (default: every family with an implementation on either model). A family on neither model is a 404 resource_missing. | | tau | Drift tolerance, 0–1 (default 0.25). A family whose drift exceeds τ fails the gate. |

The two models must share an architecture (n_layers × d_mlp). Neuron L13.4209.pos only means the same position across one architecture, so a mismatch is a 400 on target. GET /v1/trace/diffs lists your diffs, and the dashboard's Trace → Diffs page runs and reads them.

The report

| Section | What it measures | | --- | --- | | coverage | Neurons in each registry and how many are shared. | | label_changes | Shared neurons labeled differently. Labels are estimates from the labeler, so treat a change as a prompt to look, not a verdict. | | new_unexplained | Neurons that are unexplained in the target but were explained in the base, or are new to the target. | | ablation_delta_drift | For shared neurons measured on both sides: mean, p95 and max of the absolute change in the measured ablation effect, plus the top 50. This is the measurement. | | canary_member_shift | One row per canary family; see below. | | summary | families_drifted, families_not_comparable and pass. pass is true when no family drifted beyond τ. |

Lists are capped at 200 entries; totals are always complete. Numbers are rounded to 4 decimal places so the report is deterministic.

Canary families

Each family is compared like with like:

Masked neurons (curated canary members, neurons you can't see) appear as opaque n_… ids with no label. Their ablation numbers are kept.

Gate CI on it

Every completed diff carries an ES256 manifest (typ: provenance-trace-diff+jwt) signed with the keys at /.well-known/jwks.json. It binds report_hash = sha256(canonical JSON of the report), plus pass, tau and both model ids. Verify it offline, then gate on it:

import { Provenance, verifyTraceDiffManifest } from "@provenance/sdk";

const prov = new Provenance({ apiKey: process.env.PRV_ENTERPRISE_KEY! });
const d = await prov.trace.diffs.wait((await prov.trace.diffs.create({ base: BASE, target: TUNED, tau: 0.25 })).diffId);
const jwks = await (await fetch(`${process.env.PROVENANCE_API}/.well-known/jwks.json`)).json();
const v = await verifyTraceDiffManifest(d.manifest!, jwks, d.report);
if (d.status !== "completed" || !v.valid || !v.reportMatches || !v.claims!.pass) {
  console.error("Neuron-CI failed:", d.report?.summary.familiesDrifted, v.reason ?? "");
  process.exit(1);
}

In Python, provenance_verify.verify_trace_diff_manifest(token, jwks, report) does the same check over the snake_case report.

Webhook

trace.diff_completed fires when a diff completes, with diff_id, both models, pass, families_drifted and report_hash. See webhooks.

What a diff does not prove