Skip to content
PensievePensievePensieve

Rename a model, pause its experiments, or set its budget

PATCH
/v1/models/{model}
curl --request PATCH \
  --url https://api.pensievelabs.co/v1/models/grade-audit \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{ "experimenting": true, "monthly_budget_usd": 2000 }'

Authorizations

Section titled “Authorizations”

Parameters

Section titled “Parameters”

Path Parameters

Section titled “Path Parameters”
model
required
string

The model’s slug, such as grade-audit, or its mdl_ id.

Example 

grade-audit

Request Bodyrequired

Section titled “Request Bodyrequired”
Media typeapplication/json
object
name
string
>= 2 characters <= 80 characters
experimenting

False pauses the agents: running experiments stop after their current runs, and no new ones start.

boolean
monthly_budget_usd
number | null

Example 

{
  "experimenting": true,
  "monthly_budget_usd": 2000
}

Responses

Section titled “Responses”

200

Section titled “200”

The model.

Media typeapplication/json
object
id
required
string
object
required
Allowed value: model
slug
required
string
name
required
string
decision
required

The decision the model makes, in one sentence.

string
status
required

building: no version beats the frontier yet. shadow: a version runs beside live traffic. live: a version takes traffic. paused: taken off traffic in the app, so your code answers every call until it is resumed. archived: retired in the app; it answers no calls, and lists leave it out unless you ask for status=archived.

string
Allowed values: building shadow live paused archived
live_version
required
string | null
answers
required
Array<object>
object
value
required

What your code receives.

string
label
required
string
description
string | null
outcomes
required

The outcomes you send back for its calls.

Array<string>
input_schema

JSON Schema for input, when the model takes structured cases.

object | null
metric
required
string
lead
required
One of:

The live version (or best candidate) against the best frontier model, on your test.

object
model_score
required
number
frontier_score
required
number
points
required

The lead in points out of 100; negative when behind.

number
frontier_model
required

The best frontier model at the last re-test.

string
tested_at
required
string format: date-time
routing
required
object
object
required
Allowed value: routing
model
required
string
mode
required
string
Allowed values: observe model split
threshold
required

Under this confidence a call falls back.

number
<= 1
split_percent
required
integer | null
<= 100
updated_at
required
string format: date-time
experimenting
required

Whether Pensieve’s agents are running experiments on it. Pause and resume them with models.update.

boolean
budget
required

Compute for experiments this month. Serving is not counted here.

object
monthly_usd
required

The cap; null for none.

number | null
spent_usd
required
number
owner
required
string
calls_per_day
required
integer
created_at
required
string format: date-time

Example 

{
  "id": "mdl_7f3a91c2",
  "object": "model",
  "slug": "grade-audit",
  "name": "Grade audit",
  "status": "building",
  "live_version": "v7",
  "answers": [
    {
      "value": "false_pass",
      "label": "False pass"
    }
  ],
  "metric": "Right verdict on the grade",
  "routing": {
    "object": "routing",
    "mode": "observe"
  },
  "owner": "Environments team"
}

default

Section titled “default”

Something went wrong; error.type says what.

Media typeapplication/json
object
error
required
object
type
required
string
Allowed values: invalid_request authentication permission not_found conflict rate_limit not_available server
message
required
string
param

The request field the error is about.

string | null
request_id
required
string

Example 

{
  "error": {
    "type": "invalid_request"
  }
}