Skip to content
PensievePensievePensieve

Post-training methods

for await (const m of pensieve.methods.list()) {
  console.log(m.id, m.version, m.built_in, m.name);
}

A method is how an experiment trains B. Each one is a plain markdown file: when to try it, what it needs, its steps, the settings an experiment can change, and what to check before a result counts. The agents read the file to design an experiment, and follow it as written.

Because a method is only text, it can be anything your team can describe: a training order, a new way to build pairs, a reward made from a signal only you have. Write your own, or copy one of Pensieve’s and change it.

Built-in methods

Section titled “Built-in methods”
Method How the model learns Try it when
fine-tuning From examples of the right answer: your experts’ decisions, corrected calls, checked practice cases There is any history; most recipes build on it
rl From a reward for matching what really happened next Outcomes judge better than your experts’ first call
preference-tuning From pairs of a better and a worse answer for the same case A right and a wrong answer look alike
distillation From a larger open model’s step-by-step reasoning The decision takes several steps to reason through
meta-training From a checkpoint trained to adapt fast across many decision tasks New kinds of case arrive with little history

Methods chain. On one sample model, the recipe distils a teacher first, then fine-tunes with overturned grades weighted 4×, then preference-tunes on pairs of right and wrong grades. Each step was its own kept experiment.

What a method file looks like

Section titled “What a method file looks like”

Only the title and the first paragraph are required: they become the method’s name and description. Pensieve’s own files then have four sections. This is the built-in fine-tuning, as the agents read it:

fine-tuning.md
# Fine-tuning


Train the model on cases and the answers your experts gave them. Try it first on any history: most recipes build on it, and many changes worth testing are changes to what it learns from.


## Needs


- Cases with a known right answer: your experts' decisions, corrected calls, or outcomes that settle which answer was right.
- At least a few hundred per answer value. Fewer, and the experiment says so instead of training.


## Steps


1. Build the training mix from the model's data sources, leaving out every case held for the test.
2. Weight the mix as the settings say.
3. Train from the starting checkpoint, with the same steps for A and B.
4. Score every checkpoint on the held-out test.


## Settings


- `weighting`: how often each kind of case is shown. Default: every case once. Experiments often weight the cases the current version gets wrong, or ones whose outcome overturned the expert.
- `inputs`: which fields of a case the model sees. Default: every field in the model's input schema.
- `context`: how many tokens of each case it reads. Default 8k.
- `synthetic`: practice cases written by agents and checked by your experts, as a share of the mix. Default none.
- `epochs`: passes over the mix. Default 2.


## Checks


- No test case, or a near copy of one, is in the training mix or in any practice case.
- Practice cases are scored by your experts on a sample before training on them.
- A gain must not come from one kind of case getting worse: the test is also scored by kind.
  • Needs is what must be true of your data. If it isn’t, the experiment says so instead of training.
  • Steps is what the agents do, in order.
  • Settings are what an experiment can change, with their defaults. An experiment’s lever names its change in words, such as Data mix, not by a setting’s name such as weighting.
  • Checks are what must hold before a result counts, on top of the checks every experiment gets.

A method decides how B is trained, never how it is judged. Whatever a file says, A and B are trained on the same seeds and scored on your held-out test, and the test never enters training. A method can add checks; it can’t remove them.

Versions

Section titled “Versions”

Every change to a method raises its version. Each experiment records the method and version it ran, so a result always leads back to the exact text the agents followed:

const e = await pensieve.experiments.retrieve("exp_0418");
const m = await pensieve.methods.retrieve(e.method!, { version: e.method_version! });

Experiments already designed keep the version they started with when the file changes. Versions outlive the method: a removed method leaves the list, but each of its versions stays readable with version, and its slug is never used again.

What experiments change

Section titled “What experiments change”

The same method can pull on different parts of the recipe:

lever What changes A sample experiment
Data mix Which cases are shown, and how often Weight grades the labs later overturned 4×: kept, +1.3
Synthetic data Practice cases written by agents and checked by your experts 3,400 practice trials for 340 new Rust tasks: running, +0.4 on two seeds
Reward What RL rewards The labs’ accept-or-return verdict as the reward: running
Inputs What the model sees for each case Add the coding agent’s stated goal to each command: kept, +1.1
Context How much of each case it reads The whole patch, 32k tokens, not just the failing tests: dropped, −0.1, and 3× slower
Starting point Which checkpoint training starts from A meta-trained checkpoint: dropped, −0.4, and −2.1 on rare languages
Model size How large the model is A 1B model instead of 3B: dropped after two seeds, −0.9

How agents choose what to try

Section titled “How agents choose what to try”
  • From the errors. Agents read where the current version is wrong, on your test and in live outcomes, and look for what those cases share. Then they look for a method whose file says to try it there.
  • From what has paid before. Every experiment’s result is kept with its method and lever. Ideas like the ones that paid on your model go first; ideas like the ones that keep failing go last.
  • Within the budget. Cheap, likely ideas run first. A costly one, such as a larger context, runs when the evidence for it is strong.
  • From you. Ideas you suggest, and methods you write, are designed and judged the same way. See Follow and steer experiments.

Only open models are trained. Frontier models are not fine-tuned; they are the baseline every version must beat on your test.