Pensieve
Overview
Find the post-training that gets the best outcome from your data, and run the model it makes.
Pensieve API 2026-11-08
Section titled “Pensieve API 2026-11-08”Pensieve runs experiments on your data. For each decision your software makes, its agents look for the post-training that gets the best outcome. Each experiment trains the current recipe (A) and one change (B) on the same three seeds, and scores both on your held-out test, which no model trains on. A change is kept only if B wins on every seed and on a last check against cases neither has seen. Kept changes add up to a recipe; a trained recipe is a version. A version ships once it beats the live version and the best frontier model on that test, runs in shadow, and is signed off by your expert. Each post-training method is a plain markdown file the agents follow: Pensieve’s own, or one you write.
Your software calls a model with a case and gets back one of its fixed answers, a confidence and the version that answered. What really happened next goes back in, and feeds the next experiments.
Keys. Send Authorization: Bearer <key>. A key that starts pk_test_ reaches a sandbox
workspace, where models answer from sample data; pk_live_ reaches your live workspace.
pk_test_sandbox is a public key everyone shares. Any other pk_test_ key you make up gets a sandbox
workspace of its own, with the same sample data.
The sandbox. It keeps your changes in memory for about an hour, and may reset sooner. Waits are
skipped: shadow runs and batches finish as soon as they start, so you can try a whole release in a
minute. Nothing trains there: experiments you start stay queued, and weight downloads answer 501. It
records events at /v1/events but does not send webhooks yet.
Versions. Pin the API version with Pensieve-Version: 2026-11-08. Without the header you get
the version your workspace was created on.
Retries. Every POST takes an Idempotency-Key header. Sending the same key again within 24 hours
returns the first response instead of doing the work twice.
Errors. Every error has the same body: {"error": {"type", "message", "param", "request_id"}}.
A not_available error (501) means the operation is part of the API but not served by the sandbox
yet.
Enums. Enum values in responses may grow: a new status, answer kind or event type can appear without a new API version. Treat a value you don’t know as “other” rather than failing.
Lists. Lists return {"object": "list", "data": [...], "has_more", "next_cursor"}; pass
cursor to get the next page.
Webhooks. Each delivery carries Pensieve-Signature: t=<unix seconds>,v1=<hex>, where v1 is
the HMAC-SHA256 of <t>.<raw body> with the endpoint’s secret. Reject deliveries more than five
minutes old. The SDKs check both for you.
Authentication
Section titled “Authentication”bearer
Section titled “bearer”An API key from the app’s Settings. pk_test_ keys reach the sandbox.
Security scheme type: http