# Pensieve

> Pensieve runs controlled experiments on a client's data to find the post-training that gets the best outcome for a decision their software makes. It ships five post-training methods: fine-tuning, RL on outcomes, preference tuning, distillation and meta-training. Each is a plain markdown file the agents follow, and clients can add their own. An experiment can also pull a lever within a method, such as the data mix, synthetic data, the inputs or the starting checkpoint. Each experiment trains the current recipe (A) and one change (B) on the same three seeds and scores both on a held-out test no model trains on. The change is kept only if B wins on every seed and on a last check against cases neither has seen. Kept changes add up to a recipe; a trained recipe is a version. A version ships once it beats the live version and the best frontier model on that test, runs in shadow, and is signed off by the client's expert. Software calls the model and gets back one of its fixed answers.

- The API is at https://api.pensievelabs.co. `pk_test_sandbox` is a public sandbox key; any other `pk_test_` key you make up (letters, digits and underscores after the prefix, up to 64) gets a private sandbox workspace.
- The OpenAPI spec is at https://docs.pensievelabs.co/openapi.json.
- SDKs: `@pensievelab/sdk` (TypeScript) and `pensievelab` (Python).
- Never ask a person to paste a live key into a chat. They set PENSIEVE_API_KEY themselves.

## Documentation Sets

- [Abridged documentation](https://docs.pensievelabs.co/llms-small.txt): a compact version of the documentation for Pensieve, with non-essential content removed
- [Complete documentation](https://docs.pensievelabs.co/llms-full.txt): the full documentation for Pensieve

## Notes

- The complete documentation includes all content from the official documentation
- The content is automatically generated from the same source as the official documentation