Skip to content
PensievePensievePensieve

Start a shadow run

POST
/v1/models/{model}/releases
curl --request POST \
  --url https://api.pensievelabs.co/v1/models/grade-audit/releases \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{ "version": "v8", "shadow_days": 7 }'

A version that beats the live version and the best frontier model on your test runs beside live traffic for shadow_days; nothing it decides reaches your users. Refused with conflict while a gate before the shadow run is not passed.

Authorizations

Section titled “Authorizations”

Parameters

Section titled “Parameters”

Path Parameters

Section titled “Path Parameters”
model
required
string

The model’s slug, such as grade-audit, or its mdl_ id.

Example 

grade-audit

Header Parameters

Section titled “Header Parameters”
Idempotency-Key
string
<= 255 characters

Any unique string. Sending it again within 24 hours returns the first response.

Request Bodyrequired

Section titled “Request Bodyrequired”
Media typeapplication/json
object
version
required
string
shadow_days

The workspace’s rule if left out.

integer
>= 1 <= 30

Example 

{
  "version": "v8",
  "shadow_days": 7
}

Responses

Section titled “Responses”

200

Section titled “200”

The release in its shadow run.

Media typeapplication/json
object
id
required
string
object
required
Allowed value: release
model
required
string
version
required
string
status
required

superseded: it went live and a later release replaced it.

string
Allowed values: pending shadow ready_for_signoff signed_off split live superseded rolled_back stopped
gates
required
Array<object>
object
kind
required
string
Allowed values: beats_live beats_frontier shadow signoff
label
required
string
passed
required
boolean
passed_at
required
string | null format: date-time
shadow
required
One of:
object
day
required
integer
days
required
integer
agreement
required

Share of shadow calls where it matched your experts or the outcome.

number | null
split_percent
required
integer | null
signed_off_by
required
string | null
created_at
required
string format: date-time

Example 

{
  "id": "rel_grade_audit_v8",
  "object": "release",
  "status": "pending",
  "gates": [
    {
      "kind": "beats_live"
    }
  ]
}

default

Section titled “default”

Something went wrong; error.type says what.

Media typeapplication/json
object
error
required
object
type
required
string
Allowed values: invalid_request authentication permission not_found conflict rate_limit not_available server
message
required
string
param

The request field the error is about.

string | null
request_id
required
string

Example 

{
  "error": {
    "type": "invalid_request"
  }
}