Webhooks and events

TypeScript

const endpoint = await pensieve.webhooks.create({
  url: "https://example.com/hooks/pensieve",
  events: ["release.ready_for_signoff", "frontier.retested"],
});
endpoint.secret; // whsec_..., shown once

Python

endpoint = client.webhooks.create(
    url="https://example.com/hooks/pensieve",
    events=["release.ready_for_signoff", "frontier.retested"],
)
endpoint.secret  # whsec_..., shown once

HTTP

curl https://api.pensievelabs.co/v1/webhooks \
  -H "Authorization: Bearer $PENSIEVE_API_KEY" -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/hooks/pensieve", "events": ["release.ready_for_signoff", "frontier.retested"]}'

In the sandbox

The sandbox records events but doesn’t send webhooks yet. Read them with events.list(), or GET /v1/events.

Events

Type	When
release.ready_for_signoff	A release finished its shadow run and waits for an expert
release.promoted	A version went live
release.rolled_back	A model went back to its previous version
frontier.retested	A frontier or general model release was scored on every task’s test
model.lead_narrowed	A re-test narrowed a model’s lead; agents have started experiments
experiment.ended	An experiment was kept, dropped or stopped
review.case_added	A case or a release joined the review queue
batch.completed	A batch finished

Each delivery is an event: id, type, created_at, and data, the object it is about. The same events are kept for 30 days at /v1/events, for catching up after downtime.

Checking a delivery

Each delivery carries a Pensieve-Signature header:

Pensieve-Signature: t=1794209235,v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd

v1 is the HMAC-SHA256 of <t>.<raw body>, keyed with the endpoint’s secret. Reject a delivery whose signature doesn’t match, or whose t is more than five minutes old. The SDKs do both. Pass the raw body exactly as it arrived:

TypeScript

app.post("/hooks/pensieve", express.raw({ type: "application/json" }), async (req, res) => {
  const event = await pensieve.webhooks.unwrap(req.body, req.headers); // uses webhookSecret
  if (event.type === "release.ready_for_signoff") notifyReviewers(event.data);
  res.sendStatus(200);
});

Python

@app.post("/hooks/pensieve")
async def hook(request: Request):
    event = client.webhooks.unwrap(await request.body(), request.headers)  # uses webhook_secret
    if event.type == "release.ready_for_signoff":
        notify_reviewers(event.data)
    return {"ok": True}

HTTP

# T and V1 are the two parts of the Pensieve-Signature header; body.json is the raw body as it arrived.
EXPECTED=$({ printf '%s.' "$T"; cat body.json; } | openssl dgst -sha256 -hmac "$PENSIEVE_WEBHOOK_SECRET" | awk '{print $NF}')
[ "$EXPECTED" = "$V1" ] && [ $(( $(date +%s) - T )) -le 300 ] && echo "valid"

Answer with any 2xx to acknowledge a delivery.