Skip to content
PensievePensievePensieve

Retrieve an experiment

GET
/v1/experiments/{experiment_id}
curl --request GET \
  --url https://api.pensievelabs.co/v1/experiments/exp_0420 \
  --header 'Authorization: Bearer <token>'

Authorizations

Section titled “Authorizations”

Parameters

Section titled “Parameters”

Path Parameters

Section titled “Path Parameters”
experiment_id
required
string

Example 

exp_0420

Responses

Section titled “Responses”

200

Section titled “200”

The experiment.

Media typeapplication/json

One controlled test of a change to the model’s post-training. A is the current recipe; B is the current recipe with one change. Both are trained on the same data and seeds and scored on your held-out test. A change is kept only if B beats A on every seed and on a last check against cases neither has seen.

object
id
required
string
object
required
Allowed value: experiment
model
required
string
status
required
string
Allowed values: queued running kept dropped stopped
idea
required

The idea, in a line.

string
hypothesis
required

What the agents expect, and why it would beat the current recipe. Null until they design it.

string | null
method
required
One of:

A post-training method’s slug, such as fine-tuning. See List methods for what your workspace has.

string
<= 64 characters /^[a-z0-9]+(-[a-z0-9]+)*$/
method_version
required

The version of the method’s file the experiment ran. Retrieve that version to read exactly what the agents followed.

integer | null
>= 1
lever
required

The part of the recipe the change pulls on, such as Data mix, Reward, Synthetic data or Model size.

string | null
control
required

A: the current recipe.

string | null
variant
required

B: the current recipe with the change.

string | null
prep
required

The step that builds B’s data or model before training, such as writing practice cases or building a reward.

string | null
seeds_done
required
integer
seeds_total
required
integer
gains
required

B minus A on each finished seed, in points on your test.

Array<number>
lead_points
required

Kept: B’s lead on the last check. Dropped: the mean of gains. Null while queued or running, and when stopped.

number | null
throughput
required

B’s cases per second as a multiple of A’s. Under 1 is slower to serve.

number | null
checks
required

What was checked before the result counted, such as no test case leaking into training.

Array<string>
cost_usd
required

Compute spent so far.

number
max_cost_usd
required

Its cost cap, if one was set. It stops before its compute would cost more.

number | null
gpu_hours
required
number
why
required

The evidence that led the agents to try it.

string | null
result
required

What happened: kept and part of a version, or dropped and why.

string | null
started_at
required
string | null format: date-time
ended_at
required
string | null format: date-time

Example 

{
  "id": "exp_0420",
  "object": "experiment",
  "status": "queued",
  "method": "fine-tuning",
  "lever": "Reward"
}

default

Section titled “default”

Something went wrong; error.type says what.

Media typeapplication/json
object
error
required
object
type
required
string
Allowed values: invalid_request authentication permission not_found conflict rate_limit not_available server
message
required
string
param

The request field the error is about.

string | null
request_id
required
string

Example 

{
  "error": {
    "type": "invalid_request"
  }
}