Skip to content
PensievePensievePensieve

Decide many cases

POST
/v1/models/{model}/batches
curl --request POST \
  --url https://api.pensievelabs.co/v1/models/grade-audit/batches \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{ "cases": [ { "input": "Grader: FAIL · TestRetryTimeout timed out after 30s on runner ci-7", "key": "go-118" }, { "input": "Grader: PASS · 22 of 22 tests passed, existing tests unchanged", "key": "ts-402" } ] }'

Up to 10,000 cases. Poll the batch, or listen for batch.completed.

Authorizations

Section titled “Authorizations”

Parameters

Section titled “Parameters”

Path Parameters

Section titled “Path Parameters”
model
required
string

The model’s slug, such as grade-audit, or its mdl_ id.

Example 

grade-audit

Header Parameters

Section titled “Header Parameters”
Idempotency-Key
string
<= 255 characters

Any unique string. Sending it again within 24 hours returns the first response.

Request Bodyrequired

Section titled “Request Bodyrequired”
Media typeapplication/json
object
cases
required
Array<object>
>= 1 items <= 10000 items
object
input
required
One of:
object
key
additional properties
any
key
string
version
string

Example 

{
  "cases": [
    {
      "input": "Grader: FAIL · TestRetryTimeout timed out after 30s on runner ci-7",
      "key": "go-118"
    },
    {
      "input": "Grader: PASS · 22 of 22 tests passed, existing tests unchanged",
      "key": "ts-402"
    }
  ]
}

Responses

Section titled “Responses”

200

Section titled “200”

The batch.

Media typeapplication/json
object
id
required
string
object
required
Allowed value: batch
model
required
string
status
required
string
Allowed values: queued running done failed
total
required
integer
done
required
integer
decisions
required

The decisions, in the order sent, once the batch is done.

Array<object> | null
object
id
required

The call id; send outcomes and corrections with it.

string
object
required
Allowed value: decision
model
required
string
version
required
string
answer
required

One of the model’s answer values.

string
label
required
string
confidence
required
number
<= 1
scores
required

The probability of each answer value.

object
key
additional properties
number
reason
required

A short reason, when the model gives one.

string | null
routed
required

fallback when your code should answer instead: routing is observe, the confidence is under the threshold, or the split sent the call to your code.

string
Allowed values: model fallback
key
required
string | null
latency_ms
required
integer
created_at
required
string format: date-time
created_at
required
string format: date-time
completed_at
required
string | null format: date-time

Example 

{
  "id": "bat_5d2e81f0",
  "object": "batch",
  "status": "queued",
  "decisions": [
    {
      "id": "cl_8f21c0d4",
      "object": "decision",
      "model": "grade-audit",
      "version": "v7",
      "answer": "false_pass",
      "label": "False pass",
      "routed": "model"
    }
  ]
}

default

Section titled “default”

Something went wrong; error.type says what.

Media typeapplication/json
object
error
required
object
type
required
string
Allowed values: invalid_request authentication permission not_found conflict rate_limit not_available server
message
required
string
param

The request field the error is about.

string | null
request_id
required
string

Example 

{
  "error": {
    "type": "invalid_request"
  }
}