# Plugin (MCP) tools - Obss

Plugin (MCP)
Plugin (MCP) tools
Every tool on the Obss MCP server, grouped by area, with its parameters, what it changes, and the matching CLI command.
All 80 tools on the Obss MCP server (
https://mcp.obss.app/mcp
), grouped by area. Each entry starts with the title your host shows in bold, then its badge, matching CLI command, and parameters. Sign-in, approvals, paging, retries, and errors are on the
overview
.
Areas:
Identity and projects
·
Catalog, resources, and uploads
·
Experiments
·
Schedules
·
Runs and results
·
Frictions
·
Evals and Insights
·
Tags
·
Data
·
Agents, models, and setup
·
Quota and notifications
·
Ask Obss
Badges
are explained under
Approvals
:
Read-only
,
Changes data
,
Destructive
(your host asks first),
Irreversible
, and
You approve each call
(spends runs, creates a standing rule, or makes results public).
project_id
is optional on most tools. Without it, the server picks a project for that call only (
Projects
).
expected_organization_id
is an optional string on every tool that changes data, left out of the tables. Send
organization.id
from
get_identity
(
Organization check
).
limit
above its maximum is capped, not refused.
Identity and projects
get_identity
Get the signed-in user and organization.
Returns your Obss user and signed-in organization. Send
organization.id
as
expected_organization_id
on every write.
Read-only · CLI:
obss whoami
This tool takes no parameters.
get_project_context
Get the organization and its projects.
Returns the organization and its projects, with the values
manage_project
needs to rename one.
Read-only · CLI:
obss projects list
,
obss projects current
,
obss projects use
,
obss projects inspect
Parameter
Type
Required
Description
project_id
string
No
Use this project for this call only. Obss does not remember it.
manage_project
Create or rename a project.
It follows the same rules as the dashboard (
Projects
).
Changes data · CLI:
obss projects create
,
obss projects update
Parameter
Type
Required
Description
action
create
,
update
Yes
Create a project or rename one.
name
string
Yes
The project name, 1 to 120 characters.
mutation_attempt_id
string
With
create
An ID you choose, so that a retry doesn’t repeat the change. It has 1 to 100 characters.
project_id
string
With
update
The
prj_
project to rename.
expected_updated_at
string (date-time) or null
No
With
update
: the project’s
updatedAt
from
get_project_context
. A stale value returns
project_update_stale
.
preview_delete_project
Preview project deletion.
Counts what deleting a project removes by kind, the running experiments it stops, and the blockers. Show the result to the person before
delete_project
.
Read-only · CLI:
obss projects delete --dry-run
Parameter
Type
Required
Description
project_id
string
Yes
The
prj_
project to preview.
delete_project
Permanently delete a project.
Deletes a project and everything in it, from tasks, tools, and treatments to experiments, schedules, runs, Evals, and Insights. Running experiments stop first; the result says how many.
Destructive · Irreversible · CLI:
obss projects delete
Parameter
Type
Required
Description
project_id
string
Yes
The
prj_
project to delete.
With blockers, it returns
project_deletion_blocked
and deletes nothing. An already deleted project returns
project_not_found
.
Catalog, resources, and uploads
Find, create, change, upload, and delete the project’s
tasks
,
files and machines
,
tools
, and
treatments
.
inspect_catalog
Search the project catalog.
Lists the project’s tasks, files, machines, tools, treatments, and agents, in a fixed order, each with its tags.
total
and
kindCounts
cover every match, not just the page.
Read-only · CLI:
obss catalog list
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
kinds
array of
task
,
file
,
machine
,
skill
,
mcp
,
cli
,
sdk
,
treatment
,
agent
No
Only these kinds.
query
string
No
Text to match.
projection
compact
,
full
No
compact
(default) is for browsing.
full
adds what a change needs, such as each record’s key and
updatedAt
, an agent’s configuration revisions, and
turnsOffMcpTools
: whether the agent can run a treatment that turns MCP tools off.
tags
array of strings
No
Only records with any of these tags, by name or
tag_
ID. 1 to 20 tags.
untagged
boolean
No
true
: only untagged records. With
tags
: those tags or none.
limit
integer
No
Results per page, up to 500 (default 100).
cursor
string
No
The
nextCursor
from the previous page.
Result fields
items
,
total
,
kindCounts
, and
page
(
requestedLimit
,
returnedCount
,
hasMore
, and
nextCursor
). Each item lists its tags. An agent item’s
id
is its
agm_
ID, which a launch plan’s
agentSelections
names. With
projection: "full"
, an MCP server’s
facts.toolCount
is the number of tools its latest version listed at its last check, or
null
until a check lists them.
browse_github_source
Browse public GitHub repositories.
Finds a public github.com repository, ref, and path to import as a file or skill, and saves nothing. Pass the result to
author_resource
as
source.repository
,
source.ref
, and
source.path
.
Read-only · CLI:
obss github browse
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
view
repositories
,
refs
,
paths
Yes
repositories
: matches for
query
.
refs
: branches and tags of
repository
.
paths
: folders and files, or skill folders, at one commit.
repository
string
With
refs
and
paths
A public
owner/repository
, or a repository, tree, blob, or commit URL. Up to 500 characters.
ref
string
No
With
paths
: a branch, tag, or commit. Omit it to use the ref in the URL or the default branch.
kind
skill
,
file
With
paths
What the import creates.
skill
lists skill folders with their skill counts;
file
lists folders and files.
query
string
With
repositories
A name,
owner/name
,
owner/
, or GitHub URL, up to 200 characters. With
refs
: a name prefix. With
paths
: text the path contains.
limit
integer
No
Default and maximum:
repositories
10 and 30,
refs
100 and 200,
paths
100 and 1,000.
When
truncated
is true, narrow
query
. A URL result also returns its ref and path.
inspect_resource
Get a resource and its versions.
Returns one record’s details, direct dependencies, and a page of version history.
Read-only · CLI:
obss resources inspect
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
request.kind
task
,
file
,
machine
,
skill
,
mcp
,
cli
,
sdk
,
treatment
Yes
The kind of record.
request.id
string
Yes
The record’s ID or key.
request.view
summary
,
versions
,
dependencies
,
definition
No
What to return. Default
summary
.
request.version
integer
No
One version.
request.beforeVersion
integer
No
Continue version history from
nextBeforeVersion
while
hasMore
is true.
request.versionLimit
integer
No
Up to 100.
request.cursor
string
No
The
nextCursor
from the previous page of a list view.
request.limit
integer
No
Results per page in a list view, up to 500 (default 100).
request.offset
integer
No
With
definition
: continue from
content.nextOffsetBytes
until
content.eof
.
request.limitBytes
integer
No
Bytes per page. A page holds at most 12,288 bytes.
A machine’s
definition
view adds
variableOverrides
: machine variables that a CLI or SDK in the project also sets, with those
tools
and a
note
. In that tool’s treatments, the tool’s value wins (
When a tool sets the same name
).
inspect_resource_impact
Check what depends on a resource.
Shows the tasks and treatments that use a record directly, and whether it is managed or pinned. Call it before a change.
Read-only · CLI:
obss resources impact
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
request.kind
task
,
file
,
machine
,
skill
,
mcp
,
cli
,
sdk
,
treatment
Yes
The kind of record.
request.id
string
Yes
The record’s ID or key.
author_resource
Create or update a resource.
Creates or updates a task, file, machine, tool, or treatment, with the dashboard’s rules. The tool’s schema has the exact definition fields and limits.
Changes data · CLI:
obss resources apply
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
operation
object
Yes
One create or update. Its fields depend on
action
and
kind
, as below.
Fields of
operation
:
Field
Applies to
Description
action
All
create
or
update
.
kind
All
task
,
file
,
machine
,
skill
,
mcp
,
cli
,
sdk
, or
treatment
.
mutationAttemptId
All
Required. An ID you choose, so that a retry doesn’t repeat the change. Use a new one for each change.
targetKey
,
expectedUpdatedAt
Updates
Required, from
inspect_catalog
with
projection: "full"
.
targetKey
is the record’s key; a CLI or SDK also accepts its public ID.
versionId
,
version
Task updates
Required on a task update: the version you read.
name
,
instructionsMarkdown
,
environmentId
Tasks
Required on a task create.
environmentId
is the machine’s
env_
ID, from
inspect_catalog
with
kinds: ["machine"]
.
rubric
Tasks
The task’s criteria and how they decide a pass. See the rubric paragraph below the table.
assetAttachments
Tasks
Files to attach, each with
projectAssetId
and
mountPath
(relative to the agent’s working directory). Refer to each
mountPath
in the instructions.
name
,
description
,
definition
Machines, MCP servers, CLIs, SDKs
name
and
definition
are required on a create.
name
,
description
,
skillVersionIds
,
mcpServerVersionIds
,
cliVersionIds
,
sdkVersionIds
Treatments
name
is required on a create. The version ID lists name the treatment’s tools.
disabledMcpTools
Treatments
MCP tools to turn off
: a list of
mcpServerVersionId
and
tools
, using names whose
canTurnOff
is true in
list_mcp_server_tools
. On an update, omit it to keep the current choices; sending it replaces them, and
[]
turns every tool back on.
source
Files, skills
A public GitHub source:
kind: "github"
,
repository
, and optional
ref
and
path
.
Rubric:
a rubric has
schemaVersion: 1
and up to 100
criteria
, each with
index
and
text
. Criteria use
Yes / no
by default, and Advanced mode adds fields such as
type
,
negate
, and
aggregation
(
Tasks and rubrics
).
Shape:
follow the branch for your
action
and
kind
exactly. Don’t copy catalog-only fields such as
id
,
key
,
version
, or
versionId
into a branch that doesn’t take them. An exact retry returns the first result, and a stale
expectedUpdatedAt
returns
resource_revision_conflict
.
GitHub source:
a GitHub source saves a snapshot at the resolved commit. Later GitHub changes apply only when you call
manage_resource
with
sync_github
. Omit
ref
for the default branch and
path
for the whole repository. A whole-repository skill import finds every
SKILL.md
at any depth and names each skill by the
name
in its
SKILL.md
(
Skills
). For local archives and task imports, use
create_upload
.
Secrets:
a definition names the environment variables and headers it needs, never their values. A value in an argument returns
secure_input_requires_web
, and the saved record returns the dashboard link where a person enters each value.
Result fields
Field
Meaning
action
,
status
,
message
What the call did, as below
resources
Each resource’s
kind
,
id
,
key
,
name
,
versionId
,
version
,
contentHash
,
dashboardUrl
, and
setupUrl
when a value is missing. A task’s
versionId
is the
tskver_
ID that a launch plan needs. A treatment’s
versionId
is
null
, and its
trt_
ID is in
id
.
status
Meaning
created
,
updated
,
renamed
,
unchanged
Obss created, updated, or renamed the resource, or found nothing to change.
matched_existing
A treatment with the same tools exists. It is returned and nothing is saved.
name_conflict
Another treatment has that name. It is returned and nothing is saved.
Every project already has the
Raw Agent
treatment for a baseline.
bulk_author_resources
Create or update many resources.
Applies up to 20
author_resource
operations in order, each with its own
mutationAttemptId
and revision.
Changes data · CLI:
obss resources bulk-apply
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
request.operations
array of objects
Yes
1 to 20 operations, each shaped like the
operation
of
author_resource
.
On a failure, the result keeps the completed entries and gives
failedIndex
,
failedMutationAttemptId
, the error, and
notStartedCount
. Fix the failed operation and resend the whole batch: completed attempt IDs return their first result. For archives, use
create_upload
.
manage_resource
Set official versions and defaults, or sync from GitHub.
Sets or clears a tool’s official version or the default machine, refreshes a file or skill from GitHub, or moves a treatment to its tools’ current versions.
Changes data · CLI:
obss resources lifecycle
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
operation
object
Yes
One change, as below. Every operation takes
action
,
kind
,
id
,
mutationAttemptId
, and
expectedUpdatedAt
from
inspect_catalog
with
projection: "full"
.
action
kind
Also takes
set_official
skill
,
mcp
,
cli
,
sdk
versionId
: the version to make official
clear_official
skill
,
mcp
,
cli
,
sdk
set_default
,
clear_default
machine
sync_github
file
,
skill
update_members_to_current
treatment
sync_github
saves a new snapshot only when the selected content changed. Past runs keep their snapshots.
download_resource_archive
Download a file or skill archive.
Returns the ZIP as base64, with its size and SHA-256. Archives over 8 MB download from the record’s dashboard page.
Read-only · CLI:
obss resources archive
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
request.kind
file
,
skill
Yes
The kind of record.
request.id
string
Yes
The record’s ID or key.
request.version
integer
No
A skill version.
request.offset
integer
No
Continue from
nextOffset
until
eof
.
request.limitBytes
integer
No
Bytes per call. A call returns at most 12,288 bytes.
create_upload
Start an archive upload.
Creates or replaces a file, creates or versions skills, or imports tasks from an archive. Never put archive bytes or base64 in arguments.
Changes data · CLI:
obss uploads create
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
operation
object
Yes
What the completed upload does, as below.
file
object
No
The local archive:
fileName
(
.zip
, or
.skill
for skills; at most 255 bytes), exact
byteSize
, and lowercase hex
sha256
. Omit it when a person picks the archive in a browser.
operation.kind
operation.action
Also takes
file
create
Optional
name
(derived from the archive name when omitted) and
description
file
update
targetKey
,
expectedUpdatedAt
, and
name
; optional
description
skill
create
Optional
markOfficial
: make every imported skill version official
skill
update
targetKey
and
expectedUpdatedAt
; optional
markOfficial
task
import
Nothing. The archive is an
obss-task-import
ZIP.
Choose one path:
You can read the archive.
Send
file
, then PUT the raw bytes to
transfer.url
with
curl -X PUT --upload-file <archive>
and one
-H
per entry in
transfer.headers
. Never send Obss credentials there. Then call
complete_upload
.
A person picks the archive.
Omit
file
. The person opens
upload.uploadUrl
where they’re signed in (hosts that support URL requests show it), and that page completes the upload. Poll
get_upload
.
Pass
upload.projectId
as
project_id
to the other upload tools. Each upload expires at
upload.expiresAt
, and an abandoned upload creates nothing. A stale
expectedUpdatedAt
fails at once.
Archive
Size
Expanded content
Entries
Files and task imports
Up to 512 MiB
Up to 512 MiB
Up to 65,535 files and folders
Skills
Up to 256 MiB
Up to 256 MiB
Up to 65,535 files and folders
A task import creates at most 50 tasks.
get_upload
Get an archive upload.
Returns an upload’s status, declared file,
uploadUrl
,
lastError
, and, once completed, its records in
result
. Only its creator can read it.
Read-only · CLI:
obss uploads get
Parameter
Type
Required
Description
project_id
string
No
The upload’s project. Pass
upload.projectId
.
upload_id
string
Yes
The
upl_
ID from
create_upload
.
Poll while
status
is
awaiting_upload
. If it has
lastError
, upload a corrected archive.
completed
,
failed
,
cancelled
, and
expired
are final.
complete_upload
Complete an archive upload.
Checks the archive against the declared size and SHA-256, validates it, and applies the operation once, including any file replacement or official skill version change.
Destructive · CLI:
obss uploads complete
Parameter
Type
Required
Description
project_id
string
No
The upload’s project. Pass
upload.projectId
.
upload_id
string
Yes
The
upl_
ID, also the retry key.
The call returns
upload_not_ready
until the bytes arrive. An archive error keeps the upload
awaiting_upload
with
lastError
. An operation error, such as a stale revision or a deleted target, ends the upload as
failed
.
cancel_upload
Cancel an archive upload.
Cancels an unfinished upload and deletes its archive.
Destructive · CLI:
obss uploads cancel
Parameter
Type
Required
Description
project_id
string
No
The upload’s project. Pass
upload.projectId
.
upload_id
string
Yes
The
upl_
ID.
Cancelling again returns the cancelled upload. A completed upload can’t be cancelled, so change or delete its records instead.
preview_delete_resources
Preview resource deletion.
Shows what deleting the records removes or cancels (related experiments, runs, Evals, and stored files) and what blocks it.
Read-only · CLI:
obss resources delete --dry-run
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
targets
array of objects
Yes
1 to 100 records. Each has
resourceType
and the record’s
id
or
key
, plus
version
for one version.
resourceType
is
task
,
task_version
,
skill
,
skill_version
,
file
,
machine
,
mcp_server
,
mcp_server_version
,
cli
,
cli_version
,
sdk
,
sdk_version
,
treatment
, or
experiment
.
Result fields
Field
Meaning
plan.summary
Counts of targets, nodes, stored files, active runs, and blocked items (
blockedCount
)
plan.nodes
,
plan.nodesTruncated
What the deletion removes, cancels, or is blocked by, up to 500 items. An experiment’s own runs read
Experiment runs
.
plan.schedules
The
schedules
that the deletion deletes, with their experiment
plan.pausedSchedules
The active schedules that it pauses because their experiment uses what it deletes. Each needs an edit before it can launch again.
planHash
The value to pass as
plan_hash
to
delete_resources
Show the plan to the person, including each schedule, before you call
delete_resources
.
delete_resources
Permanently delete resources.
Deletes experiments, tasks, files, machines, tools, or treatments (or one version of a task or tool), with the experiments, runs, Evals, and Insights that depend on them, after cancelling active runs. Schedules of a deleted experiment are deleted; schedules whose experiment uses what it deletes are paused.
Destructive · Irreversible · CLI:
obss resources delete
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
targets
array of objects
Yes
1 to 100 records, as in
preview_delete_resources
.
attempt_id
string
Yes
An ID you choose, so that a retry doesn’t repeat the change. It has 16 to 100 characters.
plan_hash
string
No
planHash
from
preview_delete_resources
. If the plan changed, the call returns
resource_deletion_plan_changed
and deletes nothing.
Blocked items return
resource_deletion_blocked
. Otherwise the deletion starts in this call, and you poll
get_resource_deletion
until its status is final.
get_resource_deletion
Get resource deletion status.
Returns the status of a deletion from
delete_resources
.
Read-only · CLI:
obss deletions status
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
deletion_id
string
Yes
The
del_
ID.
Poll while the status is
preview_ready
,
queued
, or
deleting
. These statuses are final:
completed
,
failed
,
failed_to_start
,
needs_attention
, and
cancelled_before_start
. Any other status reads
unknown
.
Experiments
An
experiment
runs tasks × treatments × agents × trials. These tools check,
launch
, watch, cancel, and
share
experiments. A typical launch calls
inspect_catalog
,
preflight_experiment
,
launch_experiment
,
get_launch_status
,
get_experiment_status
, and then
list_runs
.
preflight_experiment
Check an experiment plan before launch.
Checks the exact plan, its tools’ and machines’ setup, your provider access, the revisions, and your run balance, without reserving runs.
Read-only · CLI:
obss experiments preflight
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
plan
object
Yes
The launch plan, as in
launch_experiment
.
The call succeeds even when the launch is blocked, so check
ready
. A plan with
evalsMode
or
insightsMode
set to
agent
gets the blocker
agent_mode_unavailable
when your organization can’t use that
Agent mode
, or
agent_mode_model_unsupported
, with a
setupUrl
to
Obss AI features
, when a model can’t run it.
Result fields
These are the same fields as in
obss experiments preflight
.
Field
Meaning
ready
,
runCount
Whether the plan can launch, and the runs it uses
issues
Required variables, headers, and OAuth connections without a value, each with the
setupUrl
where a person enters it. Variables without
required: true
aren’t checked.
blockers
Everything else that stops the launch, such as
variable_name_clash
for a
name that two places set to different values
notices
Facts that don’t block it, such as a custom agent that isn’t tested or needs attention, with its page
quota
requiredRuns
,
availableRuns
, and
covered
agentModeEstimates
Only for a feature that the plan runs in Agent mode while your organization can use it: one entry per feature, with
feature
(
evals
or
insights
) and
estimate
(the
runs
its agent would review, the
model
it uses, and for Insights
experimentModel
when the Experiment insights model is a different model)
plan
Your plan with the current configuration revisions filled in
launch_experiment
Launch an experiment.
Launches an exact plan that names catalog versions by ID, and returns a
launchId
(
adm_…
) for
get_launch_status
.
Changes data · You approve each call · CLI:
obss experiments launch
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
plan
object
Yes
The launch plan, as below.
get_experiment_plan
builds one from an earlier experiment.
Fields of
plan
:
Field
Required
Description
selectedTaskDefinitionIds
Yes
Up to 100
tskver_
IDs: each task’s
versionId
from
inspect_catalog
, not its
tsk_
ID.
treatments
Yes
Up to 100 treatments. Each has
clientId
,
name
, and
treatmentDefinitionId
, plus optional
preset
,
skillVersionIds
,
mcpServerVersionIds
,
cliVersionIds
,
sdkVersionIds
,
treatmentKey
,
sourceTreatment
, and
contentHash
.
referenceTreatmentClientId
Yes
The
clientId
of the Baseline treatment, usually
Raw Agent
, that
compare_treatments
reads the others against.
agentSelections
Yes
Up to 100 agents, each with
projectAgentModelBindingId
(an
agm_
ID from
inspect_catalog
with
kinds: ["agent"]
and
projection: "full"
) and optional
effortLevel
. A custom agent’s model is named the same way.
projectConfigurationRevision
,
organizationConfigurationRevision
Yes
From an agent item in
inspect_catalog
with
projection: "full"
, or
preflight_experiment
. A stale value returns
configuration_changed
.
trialCount
Yes
Trials of each combination, 1 to 20. Each trial is one run from your balance.
launchAttemptId
Yes
An ID you choose, so that a retry doesn’t repeat the change. An exact retry returns the first launch.
evals
,
insights
No
Turn Evals and Insights on or off. Both default to
true
.
evalsMode
No
standard
(default) or
agent
, which needs
evals: true
.
Agent mode
is in private beta; an organization without it fails before runs are reserved.
insightsMode
No
standard
(default) or
agent
, which needs
insights: true
.
Agent mode
is in private beta; an organization without it fails before runs are reserved.
projectAiSelections
No
Up to 3 model overrides for this experiment:
role
(
judge
Evals,
sweep
Insights,
consolidation
Experiment insights),
providerConnectionId
,
providerModelId
, and optional
effortLevel
.
judge
needs
evals: true
; the others need
insights: true
.
projectAiInstructionOverrides
No
Up to 3 custom instructions for this experiment, each with
role
and
customInstructions
.
maxParallelAgentSessions
No
Max Concurrency
: agent sessions active at once (1 runs them one at a time). Obss can apply a lower limit. Doesn’t affect Evals or Insights.
label
,
description
No
The experiment name (1 to 120 characters) and description.
templateKey
,
rerunSourceExperimentPublicId
,
entrySource
,
mode
,
reasoningHook
No
Filled in by
get_experiment_plan
; keep them as they are.
entrySource
defaults to
agent
.
Result fields
Field
Meaning
launchId
The
adm_
ID to poll with
get_launch_status
experimentId
The
exp_
ID, or
null
until the launch is admitted
status
accepted
runCount
The runs the launch uses
dashboardUrl
The launch’s page in the dashboard
A refused launch returns one of these codes:
Code
Fix
quota_exceeded
,
no_entitlement
,
grant_expired
The run balance doesn’t cover it. A person fixes that on
Organization settings → Usage & billing
.
configuration_changed
,
version_selection_changed
Something changed since you read the plan. Run
preflight_experiment
again and launch its
plan
with a new
launchAttemptId
.
launch_attempt_spent
The ID belongs to an earlier launch that failed or expired. Use a new one.
launch_attempt_conflict
The same ID came with a different plan.
get_launch_status
Get launch status.
Returns a launch’s progress by the
launchId
from
launch_experiment
,
rerun_experiment
, or
test_custom_agent
, before it has an
exp_
ID, with the same phase,
terminal
, and counts as
get_experiment_status
.
Read-only · CLI:
obss experiments status
,
obss experiments watch
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
launch_id
string
Yes
The
adm_
launch ID.
wait_ms
integer
No
Wait up to this many ms, 0 to 25,000, for a status change, a finished run, or
terminal
. A run changing phase doesn’t end the wait.
Once
experiment.experimentId
is set, use it with the other experiment tools (
Polling
).
list_experiments
List experiments.
Lists the project’s experiments with their tags and lineage. Experiments launched again or by a schedule share their source’s name, so tell them apart by lineage.
Read-only · CLI:
obss experiments list
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
query
string
No
Text to match, up to 200 characters.
statuses
array of
running
,
completed
,
completed_with_errors
,
cancelled
,
errored
No
Only these statuses.
schedule_id
,
task_id
,
task_version_id
,
treatment_id
,
tool_id
,
tool_version_id
,
file_id
,
machine_id
,
agent_id
,
agent_model_id
,
friction_id
string
No
Only experiments with a run that matches every one. Each takes the ID that
list_runs
describes.
based_on_experiment_id
string
No
Only the child experiments of this
exp_
parent experiment: launched again, edited, or scheduled from it.
created_after
,
created_before
string (date-time)
No
Only experiments created in this range.
tags
array of strings
No
Only experiments with any of these tags, by name or
tag_
ID. 1 to 20 tags.
untagged
boolean
No
true
: only untagged experiments. With
tags
: experiments with those tags or with none.
limit
integer
No
Results per page, up to 200 (default 50).
cursor
string
No
The
nextCursor
from the previous page.
An experiment is listed when one of its runs matches every scope field.
schedule_id
also lists a launched experiment that has no runs yet. An unknown ID returns
scope_not_found
. The result has no
scope
.
Result fields
Each experiment has its tags,
infraBlockedCount
(
Progress counts
), and
lineage
as in
get_experiment
.
hasMore
and
nextCursor
page the list.
get_experiment
Get an experiment.
Returns an experiment’s details, progress counts, launcher, runs used, and lineage.
Read-only · CLI:
obss experiments get
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment.
Result fields
lineage
has these parts:
launchedVia
:
new
,
from_experiment
,
schedule
, or
schedule_launch_now
.
basedOn
: the parent experiment, the one it was launched from, with
deleted: true
once it’s gone.
schedule
: the
sch_
schedule that launched it, and its slot.
children
: its child experiments (list them with
list_experiments
and
based_on_experiment_id
),
scheduleCount
, and the 10 newest of those schedules (all of them with
list_schedules
and
experiment_id
).
get_experiment_status
Get experiment status.
Returns the phase,
terminal
, and
progress counts
.
Read-only · CLI:
obss experiments status
,
obss experiments watch
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment.
wait_ms
integer
No
Wait up to this many ms, 0 to 25,000. Omit or 0 for the current snapshot.
While
terminal
is false, call again with
wait_ms
or after
pollAfterMs
. When
terminal
is true, read the final runs with
list_runs
.
get_experiment_plan
Get an experiment’s launch plan.
Builds a launchable plan from an earlier experiment, at its task versions or the latest ones. Edit it, then pass it to
launch_experiment
.
Read-only · CLI:
obss experiments clone-plan
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment.
launch_attempt_id
string
Yes
An ID you choose for the plan’s
launchAttemptId
, so that a retry doesn’t repeat the launch. It has 1 to 100 characters.
version_mode
experiment
,
latest
No
The experiment’s task versions, or the latest. Default
latest
.
comparison_id
string
No
The
cmp_
ID of a likely difference. The plan then runs that likely difference again.
after_fix
boolean
No
With the
comparison_id
of a clear difference that lists a friction to fix in the product (its
afterFix
in
get_experiment_brief
): the plan runs it again after your fix, on the tasks where its two sides differed, with both sides and the experiment’s runs each. Default
false
.
With
comparison_id
from
get_evals_comparisons
, the plan
runs that likely difference again
: the tasks that can still show a difference, the two sides (both treatments and agents for an
interaction
), and enough runs each to likely settle it. Launching the plan spends the person’s runs, so say the run count and ask first.
version_mode
then defaults to
experiment
, treatments use their current tools as with
Launch again
, and Evals uses the latest Evals attempt’s settings.
Code
When
comparison_not_found
The ID names no comparison in the experiment’s latest Evals.
comparison_not_confirmable
The comparison isn’t a likely difference in the latest Evals, the experiment or its Evals is still running, or no confirmation can settle it.
details.reason
is
clear
,
unclear
,
in_progress
, or
cannot_settle
.
after_fix_unavailable
With
after_fix
: the difference isn’t clear in the latest Evals, the experiment, its Evals, or its Insights is still running, or no friction listed with it is fixed in the product.
details.reason
is
not_clear
,
in_progress
, or
no_product_friction
.
rerun_experiment
Launch an experiment again.
Relaunches a finished experiment with the same tasks, treatments, agents, trials, Evals, and Insights, or launches a schedule now with
schedule_id
. Evals and Insights run in the modes the experiment launched with, including
Agent mode
. If your organization can no longer use that Agent mode, the launch is refused with
agent_mode_unavailable
; use
get_experiment_plan
and set
evalsMode
or
insightsMode
to
standard
.
Changes data · You approve each call · CLI:
obss experiments rerun
,
obss schedules launch
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
One of these two
The finished
exp_
experiment to launch again.
schedule_id
string
One of these two
The
sch_
schedule to launch now.
launch_attempt_id
string
Yes
An ID you choose, so that a retry doesn’t repeat the change. It has 1 to 100 characters, and a retry returns the first launch.
version_mode
experiment
,
latest
No
The experiment’s task versions, or the latest. Default
latest
.
With
schedule_id
, it launches the schedule’s experiment at the latest task versions, under the schedule’s name, without moving the next launch. It refuses with
experiment_schedule_run_active
while the last one still runs. Poll the
launchId
with
get_launch_status
. To change the plan first, use
get_experiment_plan
, then
launch_experiment
.
cancel_experiment
Cancel an experiment or run.
Cancels a whole experiment, or every still-active trial of one task, treatment, and agent combination. Completed work stays available.
Destructive · CLI:
obss experiments cancel
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment, or the
adm_
ID of a launch that has no experiment yet.
run_id
string
No
A
run_
ID. Cancels every still-active trial of the same task, treatment, and agent.
A finished or stopping target returns its status. Poll
get_experiment_status
until the cancellation finishes.
publish_experiment
Publish an experiment.
Creates a public link that anyone can view without an account, with runs, traces, outputs, Evals, Insights, and text files up to 256 KiB. Larger or binary files and private machine settings aren’t shared.
Changes data · You approve each call · CLI:
obss experiments publish
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment.
title
string
Yes
The public title, 1 to 120 characters.
description
string or null
No
The public description, up to 2,000 characters.
show_organization
boolean
No
Show your organization’s name. Defaults to
false
.
attempt_id
string
Yes
An ID you choose, so that a retry doesn’t repeat the change. It has 16 to 200 characters.
Later results go live only through
update_experiment_publication
. To share privately, invite the person to your organization (
Share results
).
list_experiment_publications
List published experiments.
Lists your organization’s public links.
Read-only · CLI:
obss experiments publications list
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
No
Only the links of this
exp_
experiment.
statuses
array of
preparing
,
active
,
failed
,
revoked
No
Only these statuses.
limit
integer
No
Results per page, up to 200 (default 50).
cursor
string
No
The
nextCursor
from the previous page.
get_experiment_publication
Get an experiment’s public link.
Returns a link’s status, current version, whether an update is available, and the
lockVersion
a change needs.
Read-only · CLI:
obss experiments publications get
Parameter
Type
Required
Description
publication_id
string
Yes
The
pub_
public link ID.
update_experiment_publication
Update a published experiment.
Replaces the results at the same link with the latest ones. The earlier results stay until the update is ready.
Destructive · You approve each call · CLI:
obss experiments publications update
Parameter
Type
Required
Description
publication_id
string
Yes
The
pub_
public link ID.
lock_version
integer
Yes
From
get_experiment_publication
. A stale value returns
publication_changed
.
attempt_id
string
Yes
An ID you choose, so that a retry doesn’t repeat the change. It has 16 to 200 characters.
retry_pending
boolean
No
true
takes over an update stuck preparing, instead of refusing with
publication_preparation_in_progress
. Default
false
.
unpublish_experiment
Unpublish an experiment.
Stops sharing an experiment, and the public link stops working at once.
Destructive · CLI:
obss experiments publications unpublish
Parameter
Type
Required
Description
publication_id
string
Yes
The
pub_
public link ID.
lock_version
integer
Yes
From
get_experiment_publication
. A stale value returns
publication_changed
.
Unpublishing again returns the unpublished experiment.
get_public_experiment
Get results from a public link.
Reads the results at a public experiment link, one section at a time.
Read-only · CLI:
obss gallery get
Parameter
Type
Required
Description
publication_id
string
Yes
The
pub_
public link ID.
section
overview
,
matrix
,
runs
,
frictions
,
evals
,
insights
,
trajectory
,
output
,
run-evals
,
files
,
task-details
,
treatment-details
No
The part of the results to read. Default
overview
.
run_key
string
No
A run on the public page, such as
r1
.
trial_key
string
No
A trial, such as
trial1
.
task_key
string
No
A task, such as
task1
.
treatment_key
string
No
A treatment, such as
treatment1
.
evaluation_key
string
No
An Evals attempt, such as
e1
.
insights_key
string
No
An Insights attempt, such as
i1
.
limit
integer
No
Results per page, up to 200 (default 50).
cursor
string
No
The
nextCursor
from the previous page.
A large Frictions result arrives in parts (
part.index
of
part.count
). Read every page, then append each part’s clusters, family clusters, and outcome badge audits in order. A repeated
clusterKey
continues the previous cluster’s occurrences.
Schedules
A
schedule
launches one finished experiment again on a cron cadence in a time zone.
list_schedules
List schedules.
Lists a project’s schedules, newest first, with
nextRunAt
, the person each runs as, the last result, and
needsAttention
with
pauseReason
when Obss paused it.
Read-only · CLI:
obss schedules list
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
query
string
No
Text to match, up to 200 characters.
statuses
array of
active
,
paused
No
Only these statuses.
experiment_id
string
No
Only the schedules that launch this
exp_
experiment.
launched_experiment_id
string
No
Only the schedule that launched this
exp_
experiment.
limit
integer
No
Results per page, up to 200 (default 50).
cursor
string
No
The
nextCursor
from the previous page. Send it with the same filters.
For a schedule’s experiments, call
list_experiments
with
schedule_id
.
get_schedule
Get a schedule.
Returns its cadence in plain words,
nextRunAt
, experiment, run use, last result, and the
lock_version
a change needs.
Read-only · CLI:
obss schedules get
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
schedule_id
string
Yes
The
sch_
schedule.
Result fields
runsPerLaunch
is the runs each launch uses.
runsPer30Days
is the runs in the next 30 days, not a monthly average. It’s 0 while the schedule is paused, or when no launch falls in the next 30 days.
preview_schedule
Preview a schedule.
Shows what a new schedule or a change would do. Call it before
manage_schedule
and tell the person the runs it uses.
Read-only · CLI:
obss schedules preview
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
For a new schedule
The finished
exp_
experiment to run again. For a change, omit it to keep the current one.
cron
string
For a new schedule
Five cron fields in
time_zone
. The minute is 0, 15, 30, or 45, so the fastest cadence is hourly.
0 9 * * 1-5
is 9:00 AM on weekdays.
time_zone
string
For a new schedule
An IANA name such as
America/New_York
, or
UTC
; abbreviations such as
EST
are refused. Ask the person, because agents often run in UTC.
schedule_id
string
For a change
The
sch_
schedule. Send only what changes.
Result fields
The preview has the cadence in plain words, the time zone it stores, the next 3 launches,
runsPerLaunch
, and
runsPer30Days
. A preview of a paused schedule counts it as resumed, and
schedule.status
says whether a changed schedule stays paused.
shortfall
is a warning, not a refusal. It’s null while your runs cover the launches, or while the schedule is paused. Otherwise it gives
at
(the first uncovered launch),
runsShort
,
availableRuns
,
launchesCovered
,
runsToAdd
(to cover every launch until
until
),
until
,
untilReset
(true when
until
is your next run reset), a
message
, and an
action
linking Usage & billing.
blockers
lists everything that would stop a launch now, each with a
code
,
message
, and
action
with its fix link, so the person can fix them all at once. While it isn’t empty, a create, a resume, or a change to the experiment, cron, or time zone refuses with
schedule_launch_blocked
.
blocker
(the first one) is for older CLI releases.
manage_schedule
Create or change a schedule.
Creates, changes, pauses, or resumes a schedule. Each launch uses the project’s tasks, treatments, and agents as they are then.
Changes data · You approve each call · CLI:
obss schedules create
,
obss schedules update
,
obss schedules pause
,
obss schedules resume
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
action
create
,
update
,
pause
,
resume
Yes
What to do.
experiment_id
string
With
create
The finished
exp_
experiment to launch again. Optional with
update
.
cron
string
With
create
Five cron fields, as in
preview_schedule
. Optional with
update
.
time_zone
string
With
create
An IANA name, or
UTC
. Optional with
update
.
name
string
No
With
create
or
update
: the schedule name, 1 to 120 characters.
schedule_id
string
With
update
,
pause
,
resume
The
sch_
schedule.
lock_version
integer
With
update
,
pause
,
resume
From
get_schedule
. A stale value returns
schedule_changed
.
attempt_id
string
With
create
and
update
An ID you choose, so that a retry doesn’t repeat the change. It has 16 to 200 letters, digits,
_
, or
-
.
A create, a resume, or a change to the experiment, cron, or time zone checks the launch as you, and makes you the person the schedule runs as. If blocked, it refuses with
schedule_launch_blocked
, with
details.blockers
shaped as in
preview_schedule
; the message is the blocker’s own, or a count when there are several.
update
changes only the fields you send. A schedule still named after its experiment takes the new experiment’s name.
Obss pauses a schedule when a launch needs a fix, or after 3 scheduled launches in a row fail. A checked change clears the reason, but the schedule stays paused until you resume it.
After the person fixes the cause,
resume
checks the launch and resumes. If something still blocks it, give the person every fix link in
details.blockers
, then resume again. Pause and resume are safe to repeat, and resume never launches at once.
A create or resume past 50 active schedules in a project returns
schedule_limit_reached
.
To launch a schedule now, call
rerun_experiment
with
schedule_id
.
delete_schedule
Delete a schedule.
Deletes a schedule. The experiments it launched stay.
Destructive · Irreversible · CLI:
obss schedules delete
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
schedule_id
string
Yes
The
sch_
schedule.
lock_version
integer
Yes
From
get_schedule
. A stale value returns
schedule_changed
.
An already deleted schedule returns
schedule_not_found
.
Runs and results
A run is one execution of a task with a treatment and an agent. These tools read
results
and
run details
.
list_runs
List run results.
Returns a page of runs with stable IDs, typed errors, and a plain explanation of each result in
criteriaSummary
.
Read-only · CLI:
obss runs list
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
No
Only the runs of this
exp_
experiment.
schedule_id
string
No
Only the runs of the experiments this
sch_
schedule launched.
task_id
string
No
Only the runs of this
tsk_
task, every version.
task_version_id
string
No
Only the runs of this
tskver_
task version.
treatment_id
string
No
Only the runs of this
trt_
treatment.
tool_id
string
No
Only the runs whose treatment gave the agent this tool (
skl_
,
mcp_
,
cli_
, or
sdk_
), in any version.
tool_version_id
string
No
Only the runs whose treatment gave the agent this tool version (
sklver_
,
mcpver_
,
cliver_
, or
sdkver_
).
file_id
string
No
Only the runs whose task had this
fld_
file.
machine_id
string
No
Only the runs whose task ran on this
env_
machine.
agent_id
string
No
Only the runs of this agent, with any model: its key, such as
claude-code
.
agent_model_id
string
No
Only the runs of this
agm_
agent and model.
friction_id
string
No
Only the runs where this
frc_
friction happened.
query
string
No
Text to match, up to 200 characters.
statuses
array of strings
No
Only these run statuses. Up to 20.
outcomes
array of strings
No
Only these outcomes. Up to 20.
task_key
string
No
Only the runs of this task key.
treatment
string
No
Only the runs of this treatment.
agent
string
No
Only the runs of this agent.
model_id
string
No
Only the runs of this model.
effort_level
string
No
Only the runs at this effort level.
provider
string
No
The exact provider connection name, as in
providerDisplayName
. Runs with the same agent, model, and effort can differ only by provider.
completed_after
,
completed_before
string (date-time)
No
Only runs that completed in this range.
limit
integer
No
Results per page, up to 200 (default 50).
cursor
string
No
The
nextCursor
from the previous page.
Result fields
Each run has its stable IDs, typed errors, and
criteriaSummary
. It can also have:
treatmentUse
: each tool’s state and calls, as in
get_run_profile
. It’s null when the run has no usable profile.
blockedReason
: on a blocked run, its
cause
and a one-line
message
.
setupFailure
: present when a setup command stopped the run before its agent started. It has the
owner
step,
commandNumber
when the owner has several commands of that kind (null for a package install, where
manager
and
packageOwners
name the package manager and who declared the packages), the
command
,
exitCode
(null on a timeout, with
timeoutSeconds
), the last
output
lines, and
outputTruncated
. See
troubleshooting
.
The scope fields,
experiment_id
to
friction_id
, name the records whose runs to read; a run must match them all.
list_frictions
and
get_friction
take the same ones, and
list_experiments
all but
experiment_id
. An unknown ID returns
scope_not_found
, not an empty list.
scope
in the result names each record with its dashboard page.
get_run_profile
Get a run profile.
Returns the same profile as the dashboard’s
Run profile
rail: treatment use, tool calls, what stood out, tokens, cost, and time.
Read-only · CLI:
obss runs profile
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
run_id
string
Yes
The
run_
run.
Result fields
Field
Meaning
use
For each tool the run had:
used
,
not_used
,
not_measured
(never visible, as for an SDK),
inconclusive
(the log can’t tell), or
not_applicable
, with calls by operation, measured time, and the first step and turn that used it.
calls
is null until use is decided.
treatment
answers for the whole treatment.
signals
What stood out, each list empty when nothing qualifies: a call repeated with the same input, input that grew by at least 10k tokens and 10% in one turn, a call of at least 5 s and 5% of working time, and a gap between turns of at least 10 s and 3× the median gap
costSource
agent_reported
,
estimated
, or
unknown
. Counts and times are exact or null, never estimated.
stopReason
,
reportedModelId
,
reportedModelName
,
agentError
Custom agents only.
agentError
says why the run failed before the agent answered.
setupFailure
As in
list_runs
Step ordinals refer to
get_run_trajectory
, and the per-turn series is the
runTurns
dataset of
query_data
.
get_run_trajectory
Get a run trajectory.
Returns a page of a run’s steps in order, each with its model turn and tool, plus the run’s input, cached-input, and output token totals (or null).
Read-only · CLI:
obss runs trajectory
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
run_id
string
Yes
The
run_
run.
after_ordinal
integer
No
Start after this step ordinal. Send
nextAfterOrdinal
from the previous page. Omit it to start at the first step.
limit
integer
No
Results per page, up to 500 (default 100).
projection
compact
,
full
No
compact
(default): each step’s kind, tool, and a short text preview of up to 512 bytes.
full
: every recorded field of each step.
Continue with
nextAfterOrdinal
while
truncated
is true.
Result fields
steps
(each with its ordinal, model turn, and tool; tool steps include their measured duration when the agent reported it), the run’s token totals,
truncated
,
nextAfterOrdinal
, and
coverage
, which gives the returned range, the total steps, and whether this page holds the whole trajectory. For a custom agent, the run also has the same
stopReason
, reported model, and
agentError
as
get_run_profile
.
get_run_transcript
Get a run’s raw transcript.
Returns a page of the original transcript: UTF-8 text in
contentText
, other bytes as base64 in
contentBase64
.
sha256
covers the whole transcript.
Read-only · CLI:
obss runs transcript
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
run_id
string
Yes
The
run_
run.
trial_id
string
No
One trial, when the combination has more than one.
offset
integer
No
Continue from the returned next offset until
eof
is true.
limit_bytes
integer
No
Bytes per page. A page holds at most 12,288 bytes.
get_run_output
Get a run’s output and files.
Returns a page of the final message, the unified diff, the file manifest, one kept file, or the output archive.
Read-only · CLI:
obss runs output
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
run_id
string
Yes
The
run_
run.
trial_id
string
No
One trial, when the combination has more than one.
part
final_message
,
diff
,
manifest
,
archive
,
file
No
What to read. Default
manifest
; read it first.
path
string
No
With
diff
: one file’s section of the patch. With
file
: that file’s bytes.
offset
integer
No
Continue from the returned next offset until
eof
is true.
limit_bytes
integer
No
Bytes per page. A page holds at most 12,288 bytes.
Text pages are UTF-8 in
contentText
, and other pages are base64 in
contentBase64
.
sourceTruncated
marks a preview that was capped.
analyze_results
Analyze experiment results.
Runs one ranked analysis over experiment and run results.
Read-only · CLI:
obss results analyze
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
mode
see below
Yes
The analysis to run.
query
string
No
Text to search for, in the search modes.
scope
object
No
The runs to analyze. See the fields of
scope
below.
limit
integer
No
Most ranked items, up to 100.
scope
takes these fields. A run counts only when it matches every field you send.
Field
Takes
CLI flag
experimentId
exp_
ID
--experiment-id
runId
run_
ID
--run-id
trialId
A trial’s
run_
ID
--trial-id
runs
Up to 3
run_
IDs to analyze together, or
run~trial
items
--runs
taskId
tsk_
ID
--task-id
taskVersionId
tskver_
ID
--task-version-id
taskKey
Task key
--task-key
skillId
skl_
ID
--skill-id
skillKey
Skill key
--skill-key
treatmentId
trt_
ID
--treatment-id
treatment
Treatment key or label
--treatment
toolId
skl_
,
mcp_
,
cli_
, or
sdk_
ID
--tool-id
toolVersionId
sklver_
,
mcpver_
,
cliver_
, or
sdkver_
ID
--tool-version-id
fileId
fld_
ID
--file-id
machineId
env_
ID
--machine-id
agentId
Agent key, such as
claude-code
--agent-id
agentModelId
agm_
ID
--agent-model-id
agent
Agent name or key
--agent
modelId
Model ID
--model-id
effortLevel
Effort level
--effort-level
provider
Exact provider connection name
--provider
frictionId
frc_
ID
--friction-id
scheduleId
sch_
ID
--schedule-id
outcome
pass
,
fail
,
evals_failed
,
infra_blocked
, or
inconclusive
--outcome
completedAfter
ISO 8601 date-time
--completed-after
completedBefore
ISO 8601 date-time
--completed-before
mode
What it returns
summarize_scope
Runs, passes, and failures for the scope, per treatment and per agent:
totals
and
treatments[]
find_failure_patterns
Failure patterns across runs
search_experiment_evidence
Runs whose recorded work matches
query
search_frictions
Frictions that match
query
get_run_evidence
One run’s evidence, with its
status
,
evalsPending
, and
insightsPending
compare_treatments
Each treatment’s use rate, mean tokens, working time, tool calls,
passedRuns
,
failedRuns
,
failureRate
, and
deltaFromReference
against the Baseline.
means.cost
gives mean cost per cost source, and a cost delta compares only the same source.
rank_tasks_by_friction
Tasks ranked by friction
analyze_reasoning_gaps
Where agents reasoned wrongly
find_similar_failures
Failures similar to one run’s
analyze_skill_usage
How agents used skills
Ranked analysis is never paged:
limit
is an upper bound, and the response size limit can return fewer items.
Result fields
When
page.truncated
is true, these fields say why and where to read the rest:
Field
Meaning
page.reasons
row_limit
or
byte_limit
page.rankedCount
How many items were ranked
page.completeData
The complete read: a
query_data
dataset, or
get_run_trajectory
. Its rows can be finer-grained than the ranked items, so read its hint before you compare counts.
The analysis is in
result
:
summarize_scope
:
result.totals
(
runs
,
passedRuns
,
failedRuns
,
evalsFailedRuns
,
infraBlockedRuns
,
inconclusiveRuns
,
pendingRuns
) and
result.treatments[]
(
treatmentKey
,
runCount
,
passedRuns
,
failedRuns
).
compare_treatments
:
result.treatments[]
with the same counts plus
failureRate
,
isReferenceTreatment
, and
deltaFromReference.failureRate
, and
result.referenceTreatmentKey
.
The pass rate is
passedRuns / (passedRuns + failedRuns)
, or
1 - failureRate
.
pendingRuns
counts runs with no verdict that Evals can still judge. They stay out of the pass and fail counts and out of
inconclusiveRuns
, so wait until
pendingRuns
is 0.
summarize_scope
and
search_frictions
return
insightsCoverage
:
analyzedRuns
,
pendingRuns
, and
unanalyzedRuns
(Insights off, skipped, failed, or never run). An empty friction list is final when
pendingRuns
is 0.
Frictions
A
friction
is a problem that agents hit, tracked across runs under a stable
frc_
ID.
list_frictions
List a project’s frictions.
Finds what keeps going wrong for agents, in the project or any scope, and what to fix first. By default, the most runs the agent never got past come first.
Read-only · CLI:
obss frictions list
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
,
schedule_id
,
task_id
,
task_version_id
,
treatment_id
,
tool_id
,
tool_version_id
,
file_id
,
machine_id
,
agent_id
,
agent_model_id
,
friction_id
string
No
Count only the runs that match every one, as in
list_runs
.
fix_in
array of
product
,
agent
,
setup
,
agent_behavior
No
Where to fix
: a tool or the product’s API, errors, or docs; your custom agent; the treatment, task, or machine; or no change (agent behavior).
families
array of
product_gap
,
docs_gap
,
environment_gap
,
agent_harness_gap
,
agent_reasoning_gap
,
task_quality_gap
No
Only these
families
.
recoveries
array of
recovered
,
unrecovered
,
unknown
No
Count only runs where the agent recovered, didn’t, or Insights couldn’t tell.
query
string
No
Text in the title or summary, up to 200 characters.
seen_after
string (date-time)
No
Count only runs created at or after this time.
sort
impact
,
runs
,
recent
,
new
No
impact
(default): most unrecovered runs, then most runs.
runs
: most runs.
recent
: last seen.
new
: first seen.
limit
integer
No
Results per page, up to 100 (default 25).
cursor
string
No
The
nextCursor
from the previous page. Send it with the same filters.
A friction is listed when one of its runs matches every filter.
Result fields
Field
Meaning
scope
Each record that the scope fields named, with its dashboard page
totalFrictions
,
hasMore
,
nextCursor
Matches across all pages, and the paging fields
frictions
Each friction’s
frictionId
,
title
,
summary
,
family
,
fixIn
,
fixSurface
(the place its fix names, or null),
concentration
(where it happens most, by agent and treatment, as counts and a
sentence
, never a cause),
whoActs
(retiring; read
fixIn
),
origin
,
runs
,
unrecoveredRuns
,
tasks
,
experiments
,
firstSeenAt
,
lastSeenAt
, and
dashboardUrl
coverage
Runs in scope with Insights off (
insightsOffRuns
), not reviewed (
unreviewedRuns
), not grouped yet (
ungroupedRuns
), or stopped by an Obss error (
obssErrorRuns
, which aren’t frictions). A short list isn’t proof that the runs went well.
models
The Insights and Experiment insights models that found and grouped the frictions, each with the runs it covered
Each field is explained under
obss frictions list
.
get_friction
Read one friction with its evidence.
Returns its summary, impact, suggested fix, what the agent did, where it happens most, and up to ten recent occurrences with run, recovery, cited steps, and quotes.
Read-only · CLI:
obss frictions get
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
friction_id
string
Yes
The
frc_
friction from
list_frictions
.
experiment_id
,
schedule_id
,
task_id
,
task_version_id
,
treatment_id
,
tool_id
,
tool_version_id
,
file_id
,
machine_id
,
agent_id
,
agent_model_id
string
No
Count only the runs that match every one, as in
list_runs
.
Result fields
The friction has the fields of
list_frictions
, plus these:
Field
Meaning
impact
Why it matters
suggestedFix
What to change at
fixSurface
.
null
for
agent_behavior
, or when no wording passed its check.
agentNote
What the agent did, beside a product or setup fix. It’s never an instruction.
byTreatment
,
byAgent
The rate of runs with the friction for each treatment and each agent
recentOccurrences
Up to ten recent runs, each with its recovery, cited steps, and quotes
models
The models that found and grouped the friction, each with the runs it covered
To list all the friction’s
runs
, call
list_runs
with
friction_id
and the same scope.
Evals and Insights
Evals
score each run against its task’s rubric, and
Insights
read each run’s trajectory to find frictions. Preview, trigger, then poll the results with
wait_ms
.
preview_evals
Preview Evals for an experiment.
Shows which runs Evals can score, rubrics that need attention or changed, model readiness, and the modes available.
Read-only · CLI:
obss evals preview
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment.
Result fields
ready
,
modelsReady
,
userProviderReady
,
setupUrl
,
eligibleRunCount
,
runsWithoutRubricsCount
,
runsWithInvalidRubricsCount
,
runsNotJudgeableCount
,
rubricChanges
,
failedRunIds
, and
mode
:
experimentMode
(the mode the experiment launched with),
latestMode
,
agentAvailable
, and, only while your organization can use Agent mode,
estimate
(
runs
and
model
). Their meanings are under
obss evals preview
.
trigger_evals
Start Evals for an experiment.
Starts an Evals attempt, or reruns Evals on the same recorded work after a rubric change.
Changes data · You approve each call · CLI:
obss evals trigger
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment.
trigger_attempt_id
string
Yes
An ID you choose, so that a retry doesn’t repeat the change. It has 1 to 100 characters. A retry returns the same
eval_
attempt with
replayed: true
.
mode
standard
,
agent
No
How this attempt scores runs. Default: the experiment’s launch mode.
project_ai_selections
array of objects
No
One Evals model override:
role: "judge"
,
provider_connection_id
(
aic_
),
provider_model_id
(
aim_
), and optional
effort_level
. Default: the organization’s
Evals model
.
project_ai_instruction_overrides
array of objects
No
One override:
role: "judge"
and
custom_instructions
. Default: the organization’s setting.
run_ids
array of strings
No
Only these
run_
runs, such as failed cells. Default: every eligible run.
Standard
scores each run in one pass with your Evals model and condenses very long runs to fit its context window. It’s the fastest and lowest-cost mode.
Agent
has Obss’s evals agent read the full run and the changed files and check the agent’s claims before it scores. It makes many calls to your Evals model, so it costs more and takes a few minutes per run. Both modes score against the same rubric (
how the modes differ
).
A request for Agent mode, stated or inherited, is refused with
agent_mode_unavailable
when your organization can’t use it: “Agent mode isn’t available for your organization. Use Standard mode instead.” Without
mode
, the refusal has
details.inheritedMode
set to
agent
and says “This experiment launched Evals in Agent mode, which your organization can’t use now. Rerun with mode: “standard”.” When the Evals model can’t run Agent mode, the refusal is
agent_mode_model_unsupported
, with a
setupUrl
to
Obss AI features
.
Agent mode
is in private beta.
Result fields
evalsId
(the new
eval_
attempt),
status
,
mode
(the mode the attempt uses),
pollAfterMs
(60 seconds in Agent mode, 10 seconds in Standard mode, or
null
when the attempt has already finished), and
replayed
(
true
when a repeated
trigger_attempt_id
returned the earlier attempt).
list_evals
List Evals history.
Returns the experiment’s Evals attempts, newest first, without reports, each with its
models
and
mode
. A failed attempt has
error
(
code
and
message
). An Agent mode attempt also has
agentUsage
once its agent has reviewed runs: the
inputTokens
,
cacheReadTokens
, and
outputTokens
that the evals agent has used so far.
Read-only · CLI:
obss evals list
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment.
limit
integer
No
Results per page, up to 100 (default 25).
cursor
string
No
The
nextCursor
from the previous page.
get_evals_results
Get Evals results.
Returns a summary for each run of an attempt, or the exact rubric and verdict JSON for one trial.
Read-only · CLI:
obss evals get
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment.
evals_id
string
No
The
eval_
attempt. Omit it to read the latest attempt.
view
summary
,
trial_evidence
No
Every run’s summary, or one trial’s exact JSON. Default
summary
.
trial_id
string
No
With
trial_evidence
: the
run_
trial.
cursor
string
No
With
summary
: the
nextCursor
from the previous page.
limit
integer
No
With
summary
: results per page, up to 200 (default 50).
offset
integer
No
With
trial_evidence
: continue from the returned next offset until
eof
is true.
limit_bytes
integer
No
With
trial_evidence
: bytes per page, at most 12,288.
wait_ms
integer
No
Wait in the call for a change, up to 25,000 ms.
Result fields
Field
Meaning
attempt
The attempt this read covers, with its
evalsId
,
models
,
mode
,
error
, and, in Agent mode,
agentUsage
.
models
is empty for an attempt from before Obss recorded models.
status
Each row’s
status
:
queued
or
running
until its result is written, then
completed
,
skipped
, or
failed
, which are final
error
For a failed row,
code
and
message
. The message starts with “Evals could not evaluate this run, so it has no Pass or Fail.” and gives the cause and the next step.
outcomeExplanation
Whether the run passed, the rule used, and how each criterion affected the result. Percentage rules also explain the overall percentage.
evidenceCoverage
What was read of the run.
null
for older runs.
hasMore
,
nextCursor
Paging for the
summary
view. A cursor stays on its attempt, even after a newer attempt starts.
get_evals_comparisons
Compare treatments and agents.
Compares an experiment’s treatments and agents task by task, from the Matrix’s Evals verdicts. It returns the sentence that answers the experiment’s question and a block for each difference first, as at the top of
Comparisons
, then every comparison with its result.
Read-only · CLI:
obss evals compare
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment.
evals_id
string
No
The
eval_
attempt. Omit it to read the latest attempt.
kind
treatment
,
agent
,
interaction
No
Only comparisons of this kind.
label
clear
,
likely
,
unclear
No
Only comparisons with this label.
limit
integer
No
Results per page, up to 200 (default 200).
cursor
string
No
The
nextCursor
from the previous page. Send it with the same
kind
and
label
.
kind
is
treatment
(a treatment against the baseline, for all agents together and for each agent, or each pair of treatments without a baseline),
agent
(two agents with the same treatment), or
interaction
(
Does a treatment help one agent more?
).
label
is
clear
(
Clear difference
),
likely
(
Likely difference
), or
unclear
(
No clear difference
).
unclear
means that the runs can’t tell the two sides apart, not that the sides are the same.
The most decisive comparisons come first, the main comparisons before the breakdowns within a label.
kind
and
label
only filter the list: labels don’t change, and
summary
,
differences
,
restLine
, and
comparisonCount
stay the same. A read returns up to 200 comparisons that fit in the response. A cursor fails with
invalid_cursor
once the comparisons change. More than 10,000 Matrix cells fails with
experiment_too_large
.
Result fields
Field
Meaning
state
,
message
ready
, or the first reason that applies (
nothing_to_compare
,
too_many_agents
,
evals_off
, or
no_verdicts
), and its words
summary
The sentence that answers the experiment’s question over its main comparisons:
text
, and
parts
, where each treatment or agent it names carries its
treatmentKey
or
agentKey
.
null
when the experiment has no main comparison.
differences
One block for each clear or likely difference, in the order to say them, up to 5: the main comparisons first, then any other comparison with a clear difference. A comparison with no clear difference is never in it. Each has
comparisonId
,
label
(
clear
or
likely
),
name
(the two sides, and what the comparison holds fixed when it isn’t a main comparison),
sides
(each with
key
,
label
,
passes
,
runs
, and
percent
),
taskLine
(“Better on 14 tasks, worse on 2, same on 26.”),
verdict
,
nextStep
,
byAgent
, and
headline
.
restLine
One sentence on what
summary
and
differences
leave out, or
null
notes
One sentence each on what limits the comparisons
evals
,
running
The Evals attempt and its
status
, and whether the experiment still has runs to finish
comparisonCount
,
comparisons
Every comparison’s count, and the listed comparisons
treatments
,
agents
The experiment’s treatments, and the agents the listed comparisons name
taskCount
,
runsEach
,
scoredRuns
,
unscoredRuns
Tasks, trials of each combination, and runs with and without a verdict
noiseFloor
,
ceilingTaskCount
,
floorTaskCount
Repeated runs that disagreed (
cases
,
disagreeing
), and tasks that passed or failed in every run
agentPairsOmitted
true
with more than 6 agents. Then no
agent
or
interaction
comparisons are made.
hasMore
,
nextCursor
Paging, with the same
kind
and
label
Each comparison has
comparisonId
(
cmp_
),
kind
,
label
,
family
(
primary
for a main comparison, or
breakdown
),
title
,
scope
,
headline
,
sides
,
verdict
,
nextStep
,
sentence
,
groupLabel
,
taskSplit
,
treatmentKey
,
referenceTreatmentKey
,
agentKey
,
referenceAgentKey
,
arm
and
reference
(or
first
and
second
for an
interaction
),
tasks
(
higher
,
lower
,
same
),
pValue
(an exact two-sided sign test),
adjustedPValue
(the Holm correction within its
family
),
runsLeftOut
,
confirmation
, and
confirmationUnavailable
(
earlier_evals
,
in_progress
, or
cannot_settle
).
confirmation
holds the runs that check a likely difference again.
cannot_settle
means that running these tasks again can’t settle it, and the experiment needs more tasks. To plan them, pass its
comparisonId
to
get_experiment_plan
as
comparison_id
, with a new
launch_attempt_id
.
Say the result in the words of
summary
and each
verdict
, which every surface shows, with
verdict
and
nextStep
on one line.
nextStep
is
run_again
(with
text
and
runs
) or
more_tasks
(with
text
), or
null
. Each field is explained under
obss evals compare
.
get_experiment_brief
Get an experiment’s Brief.
Reads an experiment’s
Brief
: its clear and likely differences, worded as in
get_evals_comparisons
, with one written sentence and the frictions seen with the side that did worse. Obss counts every number from the experiment’s Evals verdicts and frictions. A model writes only each
sentence
and, in Agent mode, each pair’s
line
.
Read-only · CLI:
obss experiments brief
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment.
state
says how complete the Brief is.
ready
,
failed
,
not_written
, and
no_insights
are final for the Brief. While
state
is
writing
, the differences and counts are already in the read: call again after
waitMs
milliseconds until
state
changes. In Agent mode, also call again after
waitMs
while any finding’s
pendingTaskCount
is above 0. The tool takes no
wait_ms
.
state
Meaning
ready
The differences, each with its
sentence
writing
The differences, while the Brief is being written
failed
The differences, and the Brief could not be written.
error.code
is
brief_model_unavailable
,
brief_model_refused
, or
brief_write_failed
, and
error.message
says why.
not_written
The differences, with no Brief for them. One is written the next time the experiment’s Evals or Insights finish.
no_insights
Insights hasn’t reviewed the experiment’s runs. Run it with
trigger_insights
.
not_available
The experiment, its Evals, or its Insights is still running.
Result fields
Field
Meaning
state
,
message
,
error
How complete the Brief is, and its words
waitMs
Only while
state
is
writing
, or while the insights agent is still reading some pairs of runs: call again after this many milliseconds.
hasMore
true
when the read left out some pairs of runs in
explanations
to fit its size limit. There is no cursor for them.
briefId
The
brf_
ID, set only when
state
is
ready
or
failed
summary
The sentence that opens
Comparisons
, as
get_evals_comparisons
returns it. “Nothing to compare in this experiment.” or “This experiment is too large to compare.” when there’s nothing to compare.
null
when
state
is
no_insights
or
not_available
.
findings
Up to 5 differences, as below
comparisonsLeftOut
,
restLine
Clear or likely comparisons that are not findings, and the sentence on what the differences leave out.
list_frictions
lists every friction of the experiment.
notes
What limits the comparisons, as in
get_evals_comparisons
evalsId
,
insightsId
,
dashboardUrl
The attempts the findings were read from, and the experiment’s
Comparisons
tab
Each finding has the fields of its block in
differences
(
comparisonId
,
label
,
name
,
sides
,
taskLine
,
verdict
,
nextStep
,
byAgent
,
headline
), and
kind
,
title
,
scope
,
arm
and
reference
,
tasks
,
pValue
,
adjustedPValue
,
worseSide
,
worseTaskCount
,
notReviewed
,
sentence
,
confirmation
,
confirmationUnavailable
,
frictions
,
afterFix
, and
issueMarkdown
. Each friction has
frictionId
(
frc_
),
title
,
summary
(what goes wrong, in one line),
fixIn
,
tasks
,
dashboardUrl
,
failedRuns
(how many of the worse side’s failed runs had it, out of
runs
),
passedRuns
(the same for the other side’s passed runs), and, for
arm
and
reference
,
runsWithFriction
out of
runsReviewed
, which is how the friction was picked. The counts use only the
worseTaskCount
tasks and the runs that Insights reviewed.
afterFix
is set for a clear difference with a friction to fix in the product: its
text
,
taskCount
, and
runs
.
issueMarkdown
is the difference as an issue in Markdown, the text that
Copy as issue
copies.
When the experiment’s latest Insights ran in
Agent mode
, each finding also has these fields:
Field
Meaning
explainedTaskCount
How many tasks have a pair of runs to read side by side: a failed run of
worseSide
and a passed run of the other side
explanations
Every pair. Once
exampleTaskVersionId
is set, the example comes first, then the tasks in
differentTaskVersionIds
, then the rest. Before that, the largest pass-rate gap comes first.
pendingTaskCount
How many pairs the insights agent is still reading. The Brief is ready before they are, so call again after
waitMs
.
exampleTaskVersionId
Once every pair is read and
sentence
says what the runs did across the tasks: the
tskver_
task whose pair shows it
differentTaskVersionIds
With
exampleTaskVersionId
: the tasks that went differently, which
sentence
names, in order
similarTaskCount
With
exampleTaskVersionId
: how many other tasks with lines went like the example
Each pair has
taskVersionId
(
tskver_
),
taskName
,
dashboardUrl
, and
state
:
pending
while the agent reads it,
completed
when its lines are written, or
failed
when the agent gave no answer that passed Obss’s checks, which shows only the two runs. Its
worse
and
better
runs each have these fields:
Field
Meaning
runId
,
trialId
The run, and the trial of it that was read
sideRuns
,
sideMatched
The side’s compared runs on the task, and how many of them failed or passed
sideCountLine
”Raw Agent failed 2 of 3 runs” when the side’s runs disagree, or
null
explained
stepIndex
,
line
, and
quote
(up to 5 lines of that step, as the run’s trace shows them), or
null
unless the pair is
completed
dashboardUrl
The run’s trace, at that step when there is one
Show the pairs in that order, as the dashboard and
issueMarkdown
do: the example with “Like N other tasks” (from
similarTaskCount
), each task that went differently with “Went differently”, then the rest. A pair says where two runs went different ways, not why the difference happened. What the agent used to read the pairs counts in the Insights attempt’s
agentUsage
(
list_insights
).
When you report a friction, give both sides’ counts (“Raw Agent 24 of 28 failed runs · Stripe CLI 0 of 26 passed runs”) and say it was seen with the side that did worse. Don’t call it the cause. To run a likely difference again, pass its
comparisonId
to
get_experiment_plan
as
comparison_id
. To run a clear difference again after a fix, add
after_fix
set to
true
. Each field is explained under
obss experiments brief
.
preview_insights
Preview Insights for an experiment.
Shows whether the Insights models and your provider access are ready, with a dashboard link for any secure setup, and the modes this experiment can use.
Read-only · CLI:
obss insights preview
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment.
Result fields
ready
,
modelsReady
,
userProviderReady
,
setupUrl
,
failedRunIds
, and
mode
:
experimentMode
(the mode the experiment launched with),
latestMode
,
agentAvailable
, and, only while your organization can use Agent mode,
estimate
(
runs
,
model
, and
experimentModel
when the Experiment insights model is a different model). Their meanings are under
obss insights preview
.
trigger_insights
Start Insights for an experiment.
Starts an Insights attempt, or reruns Insights on the same recorded work.
Changes data · You approve each call · CLI:
obss insights trigger
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment.
trigger_attempt_id
string
Yes
An ID you choose, so that a retry doesn’t repeat the change. It has 1 to 100 characters. A retry returns the same
insights_
attempt with
replayed: true
.
mode
standard
,
agent
No
How this attempt reviews runs. Default: the experiment’s launch mode.
project_ai_selections
array of objects
No
Up to two model overrides:
role
(
sweep
Insights,
consolidation
Experiment insights),
provider_connection_id
(
aic_
),
provider_model_id
(
aim_
), and optional
effort_level
. A role you omit uses the organization’s model.
project_ai_instruction_overrides
array of objects
No
Up to two overrides, each with
role
and
custom_instructions
. A role you omit uses the organization’s setting.
run_ids
array of strings
No
Only these
run_
runs. Default: every eligible run.
Standard
reviews each run in one pass with your Insights model. It’s the fastest and lowest-cost mode.
Agent
has Obss’s insights agent read the whole run and check each friction against what the tools returned. It costs more and takes a few minutes per run. It also does the experiment’s grouping of frictions, which Standard mode does in one pass. For each task of each difference in the Brief, it says what a failed run and a passed run did differently (
get_experiment_brief
). Both modes report frictions in the same way. Without
mode
, the attempt uses the experiment’s launch mode, not the latest attempt’s mode as the dashboard does.
A request for Agent mode, stated or inherited, is refused with
agent_mode_unavailable
when your organization can’t use it: “Agent mode isn’t available for your organization. Use Standard mode instead.” Without
mode
, the refusal has
details.inheritedMode
set to
agent
and says “This experiment launched Insights in Agent mode, which your organization can’t use now. Rerun with mode: “standard”.” When the Insights model or the Experiment insights model can’t run Agent mode, the refusal is
agent_mode_model_unsupported
, with a
setupUrl
to
Obss AI features
.
Agent mode
is in private beta.
Result fields
insightsId
(the new
insights_
attempt),
status
,
mode
(the mode the attempt uses),
pollAfterMs
(60 seconds in Agent mode, 10 seconds in Standard mode, or
null
when the attempt has already finished), and
replayed
(
true
when a repeated
trigger_attempt_id
returned the earlier attempt).
list_insights
List Insights history.
Returns the experiment’s Insights attempts, newest first, without reports, each with its
models
and
mode
. A failed attempt has
error
(
code
and
message
); when the insights agent couldn’t group the frictions,
error.message
starts with “Insights could not group this experiment’s frictions.” An Agent mode attempt also has
agentUsage
once its agent has reviewed runs: the
inputTokens
,
cacheReadTokens
, and
outputTokens
that the insights agent has used so far, including grouping the frictions and the pairs of runs in the Brief.
Read-only · CLI:
obss insights list
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment.
limit
integer
No
Results per page, up to 100 (default 25).
cursor
string
No
The
nextCursor
from the previous page.
get_insights_results
Get Insights results.
Returns a summary for each run of an attempt, the attempt’s public report, or the exact Insights JSON for one trial.
Read-only · CLI:
obss insights get
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
experiment_id
string
Yes
The
exp_
experiment.
insights_id
string
No
The
insights_
attempt. Omit it to read the latest attempt.
view
summary
,
attempt_report
,
trial_evidence
No
Every run’s summary, the attempt’s report, or one trial’s exact JSON. Default
summary
.
trial_id
string
No
With
trial_evidence
: the
run_
trial.
cursor
string
No
With
summary
: the
nextCursor
from the previous page.
limit
integer
No
With
summary
: results per page, up to 200 (default 50).
offset
integer
No
With an exact view: continue from the returned next offset until
eof
is true.
limit_bytes
integer
No
With an exact view: bytes per page, at most 12,288.
wait_ms
integer
No
Wait in the call for a change, up to 25,000 ms.
Result fields
attempt
(with its
insightsId
,
models
,
mode
,
error
, and, in Agent mode,
agentUsage
), and for each row its
status
, its
error
(a failed row’s message starts with “Insights could not analyze this run.” and gives the cause), which follows the same rules as in
get_evals_results
, and
evidenceCoverage
(
null
for older runs).
hasMore
and
nextCursor
page the
summary
view.
Tags
Tags
are your own labels on tasks, files, machines, tools, treatments, and experiments. Limits are under
obss tags
.
list_tags
List a project’s tags.
Lists every tag with its
tag_
ID, usage by kind, and the
updatedAt
that
rename_tag
and
delete_tag
need. One call returns all of them (at most 500).
Read-only · CLI:
obss tags list
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
query
string
No
Only tags whose name contains this text, in any case. Up to 40 characters.
Result fields
Each tag has
id
,
name
,
usage
(
total
and
byKind
),
createdAt
, and
updatedAt
.
inspect_catalog
and
list_experiments
filter by tag with
tags
and
untagged
.
update_tags
Add or remove tags on records.
Adds and removes tags on up to 100 records at once. Adding a name that the project doesn’t have yet creates the tag.
Changes data · CLI:
obss tags add
,
obss tags remove
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
resource_ids
array of strings
Yes
1 to 100 record IDs:
tsk_
,
fld_
,
env_
,
skl_
,
mcp_
,
cli_
,
sdk_
,
trt_
, or
exp_
. Tags go on the record, not a version.
add
array of strings
No
Up to 20 tags to add, by name in any case or by
tag_
ID.
remove
array of strings
No
Up to 20 tags to remove, by name or
tag_
ID.
Result fields
records
(each record’s
id
,
kind
, and all its
tags
after the change) and
createdTags
(the tags this call created).
You can send the same call again safely. A change that would give a record more than 20 tags changes nothing and returns
tag_limit_exceeded
. Tags never change a record’s definition, versions, or
updatedAt
.
rename_tag
Rename a tag.
Renames a tag. Every record keeps it, because its
tag_
ID doesn’t change.
Changes data · CLI:
obss tags rename
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
tag
string
Yes
The tag to rename: its
tag_
ID or its current name.
name
string
Yes
The new name: at most 40 characters, with no commas.
expected_updated_at
string (date-time)
Yes
The tag’s
updatedAt
from
list_tags
.
Result fields
The renamed
tag
, with its
id
,
name
,
usage
,
createdAt
, and
updatedAt
.
A tag that changed since you read it returns
tag_changed
with the current
updatedAt
. A name that another tag in the project has, in any case, returns
tag_name_taken
with that tag’s ID.
delete_tag
Delete a tag from every record.
Deletes a tag and removes it from every record that has it. Tell the person how many records have it, from
list_tags
, before you call it.
Destructive · Irreversible · CLI:
obss tags delete
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
tag
string
Yes
The tag to delete: its
tag_
ID or its name.
expected_updated_at
string (date-time)
Yes
The tag’s
updatedAt
from
list_tags
. A stale value returns
tag_changed
.
Result fields
tag
(the deleted tag’s
id
and
name
) and
removedFrom
(how many records had it). A tag the project doesn’t have returns
tag_not_found
.
Data
Query
datasets
with fields, filters, and aggregates, for questions the other reads don’t answer, such as results by model or mode.
inspect_data_model
List the data you can query.
Lists every dataset, its fields and types, each field’s filters and aggregates, the default order, and the limits.
Read-only · CLI:
obss data describe
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
Result fields
datasets
, each with
name
,
title
,
description
,
grain
,
fields
,
identityFields
,
defaultSelect
, and
defaultOrder
, and
limits
. Each field has
name
,
type
,
nullable
,
description
,
filterOperators
, and
aggregates
.
query_data
Query Obss data.
Runs one bounded query over a dataset. It never accepts SQL or returns internal IDs, secrets, or credential-like values.
Read-only · CLI:
obss data query
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
dataset
string
Yes
A dataset from
inspect_data_model
, such as
runs
,
evals
, or
insights
.
select
array of strings
No
1 to 12 fields to return.
filters
array of objects
No
Up to 20, each with
field
,
operator
(
eq
,
neq
,
lt
,
lte
,
gt
,
gte
,
in
,
not_in
,
contains
,
starts_with
,
is_null
,
is_not_null
), and
value
.
groupBy
array of strings
No
Up to 4 fields to group by.
aggregates
array of objects
No
Up to 8, each with
function
(
count
,
count_distinct
,
sum
,
avg
,
min
,
max
), optional
field
, and
as
(the column name).
orderBy
array of objects
No
Up to 4, each with
field
and
direction
(
asc
default, or
desc
).
limit
integer
No
Results per page, up to 200 (default 100).
cursor
string
No
The
nextCursor
from the previous page. A cursor whose row count or order changed returns
stale_cursor
.
Result fields
columns
,
rows
,
query
(the query as Obss ran it),
page
(
returned
,
hasMore
,
nextCursor
,
truncated
, and
reasons
), and
limits
. The datasets and their fields are listed under
obss data
, including
evals.mode
, the model fields of
evals
and
insights
, and the per-turn
runTurns
series.
Agents, models, and setup
Manage a project’s
agents
,
custom agents
, and
models
, your
Obss AI features
, and the setup steps that need the dashboard.
Custom agents are in private beta. Email
support@obss.app
to enable them for your organization.
inspect_project_ai
Get built-in and custom agents, models, and Obss AI settings.
Returns agents, provider and model readiness, your personal connections, model refresh state, and each Obss AI feature’s model and whether it can run. It never returns secrets or starts a refresh.
Read-only · CLI:
obss project ai inspect
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
view
roles
,
providers
,
agents
,
models
,
agent_registry
No
What to list. Defaults to
roles
.
provider_connection_id
string
With
models
, unless
agent_key
is set
An
aic_
connection from the
providers
view.
agent_key
string
No
With
models
: a custom agent’s
acp-
key, instead of
provider_connection_id
.
query
string
No
With
agent_registry
: match the registry ID, name, or description. Up to 200 characters.
access
shared
,
private
No
With
providers
: the organization’s API-key connections, or your own accounts.
connection_state
connected
,
needs_attention
,
not_connected
No
With
providers
: only this state.
needs_attention
is saved but must be fixed, such as by signing in again.
limit
integer
No
Results per page, up to 500 (default 100).
cursor
string
No
The exact
nextCursor
from the previous page. Omit it on the first page.
view
Use it for
roles
The model for each Obss AI feature, with
availabilityReason
and
availabilityMessage
when it cannot run, and the custom instructions
agents
Whether each agent is on, and its kind (
built_in
or
custom
)
models
A provider’s models, or a custom agent’s models (
Agent default
first, then each model with the agent’s own name;
agentState
is
current
,
stale
, or
unavailable
)
providers
Each provider’s access (shared or private) and
connectionState
agent_registry
ACP Registry agents to create a custom agent from, with the environment variables each one usually reads
Result fields
The
roles
,
providers
,
agents
, and
models
views return agent, provider, model, and role counts, provider readiness, model refresh counts, and
blockers
.
blockers
is empty when you can run models; otherwise it says why not, with the dashboard
setupUrl
where a person fixes it.
A custom agent’s
custom
block has
status
(
needs_values
,
testing
,
needs_attention
,
not_tested
, or
ready
),
description
,
revision
,
connectionId
,
url
,
definition
,
env
,
envNote
,
testBlockers
,
handshake
(start-up details such as
failure.message
,
failure.detail
, and
authMethods
),
fix
(the failed step and what to change),
mcpServersRefused
, and
latestRun
. See
Custom agent fields
.
In the
providers
view, each provider’s
modelRefresh
has
status
,
listedAt
, and the counts of the last refresh.
listedAt
is set when the refresh has read the provider’s list, so the
models
view has those models while
status
is still
running
. See
obss project ai inspect
.
A cursor returns
stale_cursor
when the configuration changed between pages, usually from a model refresh.
manage_project_ai
Create custom agents and change agents, models, and Obss AI settings.
Creates or changes custom agents, turns agents and models on or off, refreshes model lists, states context windows, and sets each Obss AI feature’s model and instructions. No operation carries a secret.
Changes data · CLI:
obss project ai apply
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
operation
object
Yes
One change, chosen by
action
, as below.
action
Fields
What it does
set_agents
expectedRevision
, optional
agents
(each
key
and
enabled
), optional
enabled
Turns agents on or off. Send
enabled
without
agents
for every agent.
set_roles
expectedOrganizationRevision
,
roleBindings
Sets each Obss AI feature’s model. A binding has
role
(
ask
,
judge
,
sweep
,
consolidation
),
providerConnectionId
,
providerModelId
,
effortLevel
,
preserveUnavailable
, and
customInstructions
. To keep a saved model that can’t run, send
preserveUnavailable
with null IDs.
set_model_enabled
connectionId
,
modelId
,
enabled
Turns one model on or off. For a custom agent, use its
custom.connectionId
.
set_models_enabled
connectionId
,
enabled
, optional
modelIds
Turns a connection’s models on or off, all of them without
modelIds
.
set_model_context_windows
connectionId
,
models
(each
modelId
and
contextWindowTokens
; null clears)
States context windows on an organization connection. The provider’s current window wins.
refresh_models
optional
connectionIds
Queues refreshes for these connections, or all you can refresh. Poll
inspect_project_ai
.
create_custom_agent
name
,
source
, optional
description
,
env
,
skillsFolder
Creates a custom agent.
update_custom_agent
agentKey
,
expectedRevision
, and any of
name
,
source
,
description
,
env
,
skillsFolder
Changes a custom agent. Send a new
source
to change how it installs.
A custom agent’s
source
is one of:
kind: "registry"
with
registryId
and an optional
version
: an agent from the ACP Registry.
kind: "manual"
with
install
and optional
args
.
install
is an npm package (
kind: "npm"
,
package
), a Python package (
kind: "python"
,
package
), or a download (
kind: "download"
,
url
,
sha256
,
command
).
kind: "agent_json"
with
agentJson
: an
agent.json
definition.
env
lists up to 64 environment variable names. Values are entered only in the dashboard, and the result returns the agent’s page and the dashboard link for each value.
skillsFolder
is the folder where the agent reads skills, up to 256 characters.
A stale revision changes nothing and returns
configuration_changed
, with
details.currentRevision
when the target has one. An invalid operation returns
validation_error
. Read
inspect_project_ai
again before you retry. When a public catalog lists a context window for a model,
inspect_project_ai
reports it as
suggestedContextWindow
, which applies only after you state it. Connecting, signing in again, switching accounts, and entering credentials stay on the provider’s dashboard link from
inspect_project_ai
.
delete_project_ai
Remove custom agents or provider connections.
Removes a custom agent, deletes a custom provider connection, resets a built-in organization connection, or disconnects your personal provider account.
Destructive · CLI:
obss project ai delete
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
operation
object
Yes
One removal, chosen by
action
, as below.
action
Fields
remove_custom_agent
agentKey
,
expectedRevision
delete_connection
connectionId
,
expectedOrganizationRevision
,
expectedSecretRevision
,
expectedEditRevision
reset_connection
connectionId
,
expectedOrganizationRevision
,
expectedSecretRevision
,
expectedEditRevision
disconnect_personal_connection
connectionId
A removed custom agent no longer shows in new experiments or the Agents list, and Obss clears its environment variable values. Past results keep its name. Obss refuses the removal while a run of the agent is in progress, either a Test agent run or an experiment. Removing it again returns the same result.
Pass the revisions from
inspect_project_ai
. A refused removal changes nothing and returns one of these codes:
Code
When
configuration_changed
A stale revision on
remove_custom_agent
.
details.currentRevision
has the current one.
project_ai_configuration_changed
A stale revision on the other actions
project_ai_target_not_found
The target is already removed.
test_custom_agent
Test a custom agent.
Checks that a custom agent installs, starts, and answers, with one run of the Agent check task:
Agent default
, the
Raw Agent
treatment (no tools), no project files, and Evals and Insights off. It uses 1 run.
Changes data · You approve each call · CLI:
obss project ai test-agent
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
agent_key
string
Yes
The custom agent’s
acp-
key from
inspect_project_ai
with
view: "agents"
.
launch_attempt_id
string
Yes
An ID you choose, so that a retry doesn’t repeat the change. It has 1 to 100 characters, and a retry returns the first launch instead of a second run.
Poll the returned
launchId
with
get_launch_status
. Each finished run with these settings updates the agent’s status in
inspect_project_ai
. A Test agent run is a check, not an experiment. It sends no notice and doesn’t show on Home or in the dashboard’s experiments list, but the CLI and the plugin still list it.
It refuses while
custom.testBlockers
names a blocker, with the first one’s code:
Code
Fix
custom_agent_test_running
A Test agent run with these settings is in progress. Open its
runUrl
.
custom_agent_disabled
Turn the agent on with
set_agents
in
manage_project_ai
.
custom_agent_value_missing
Set each named value in the dashboard with its
setupUrl
.
custom_agent_default_model_disabled
Send
set_model_enabled
with the blocker’s
connectionId
and
modelId
.
get_setup_links
Get secure setup links.
Returns the dashboard links for OAuth connections, API keys, secret headers, and other secret values.
Read-only · CLI:
obss setup links
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
Result fields
issues
lists each machine, MCP server, CLI, and SDK in the project that still needs a required value or your account connection, with the
setupUrl
where a person adds it. An MCP server whose saved values aren’t bound to its current address has a field of scope
address
, and its
setupUrl
is the server’s
Tools
tab, where a person selects
Refresh
. The list is empty when nothing is missing.
inspect_mcp_account
Check your MCP server connection.
Checks whether your account can use an MCP server in a treatment, and returns the latest problem, a next step, and the dashboard setup link. A failed reconnect never replaces a working connection.
Read-only · CLI:
obss mcp accounts inspect
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
mcp_server_id
string
Yes
The
mcp_
MCP server.
Result fields
ready
,
recovery
(with
message
,
action
,
setupUrl
,
retryAt
, and
apiKeyAlternative
),
setupUrl
,
redirectUri
, and
oauthAppConfigured
. Their meanings are under
obss mcp accounts inspect
. When the server’s saved values aren’t bound to its current address,
ready
is
false
and
setupUrl
is the version’s
Tools
tab, where a person selects
Refresh
.
detect_mcp_auth_mode
Detect how an MCP server authenticates.
Probes the server’s current definition and reports how it authenticates. It never returns the server’s endpoint or credentials, and the OAuth app’s Client ID and Client secret are entered only in the dashboard. See
MCP server sign-in
.
Read-only · CLI:
obss mcp accounts detect-auth
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
mcp_server_id
string
Yes
The
mcp_
MCP server.
Result fields
Field
Meaning
detectedAuthMode
oauth
,
headers
,
unreachable
, or
not_applicable
configurationMatches
false
when the detected mode differs from the saved one
registration
For
oauth
:
automatic
,
oauth_app_required
, or
unknown
. For
oauth_app_required
with no saved app,
setupUrl
opens the
OAuth app
section.
redirectUri
For
oauth
, the address to add to your OAuth app
oauthAppConfigured
Whether a Client ID is saved
setupUrl
The server’s dashboard setup page
list_mcp_server_tools
List an MCP server’s tools.
Reads the tools the server listed at Obss’s last check, with the check’s
status
,
message
, time, tool count, and
Tools
page (
setupUrl
). Titles, descriptions, and hints are the server’s own text.
Read-only · CLI:
obss mcp tools list
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
mcp_server_id
string
Yes
The
mcp_
MCP server.
version_id
string
No
An
mcpver_
version. Default: the latest.
query
string
No
Only tools whose name, title, or description contains this text.
after_position
integer
No
The
nextAfterPosition
of the previous page.
limit
integer
No
Results per page, up to 200 (default 50).
detail
summary
,
full
No
full
adds each tool’s
inputSchema
. Default
summary
.
Continue with
after_position
while
truncated
is true.
Result fields
The check’s
status
,
message
, and time, the tool count,
setupUrl
,
truncated
,
nextAfterPosition
,
usageRunCount
,
lastChange
,
headerValuesBound
, and each tool with
canTurnOff
,
onRunCount
,
runCount
,
callCount
,
offInTreatmentCount
, and
offInTreatmentIds
. Their meanings are under
obss mcp tools list
.
When
headerValuesBound
is
false
, only a person’s
Refresh
at
setupUrl
binds the saved values:
refresh_mcp_server_tools
fails with
mcp_tool_check_needs_web
and launches stop with
mcp_header_values_unbound
. Give the person
setupUrl
instead of calling it (
Saved header values and the server address
).
refresh_mcp_server_tools
Refresh an MCP server’s tools.
Checks the server’s tools now and returns the list as
list_mcp_server_tools
does. A running check is joined, not restarted. A server that needs your account connection or a header value returns that status and its
setupUrl
.
Changes data · CLI:
obss mcp tools refresh
Parameter
Type
Required
Description
project_id
string
No
The
prj_
project.
mcp_server_id
string
Yes
The
mcp_
MCP server.
version_id
string
No
An
mcpver_
version. Default: the latest.
wait_ms
integer
No
Wait up to 25,000 ms for the check to finish.
query
,
after_position
,
limit
,
detail
No
As for
list_mcp_server_tools
.
Result fields
The fields of
list_mcp_server_tools
, plus
refresh
, which is
running
until the check finishes. If the wait ends first, read the list again with
list_mcp_server_tools
.
It never sends saved header values to an address they aren’t bound to, and it never binds them. When
headerValuesBound
is
false
, it fails with
mcp_tool_check_needs_web
. Give the person
details.setupUrl
(the version’s
Tools
tab) to check the address and select
Refresh
, and don’t call the tool again until they have.
Quota and notifications
Read your
run balance
and manage your
notifications
. Buying runs stays in the dashboard.
inspect_quota
Get the run balance.
Returns the run balance, and whether a launch of
requested_runs
runs can start now. It offers no purchases or billing actions.
Read-only · CLI:
obss quota inspect
Parameter
Type
Required
Description
requested_runs
integer
No
Runs to check, 1 to 1,000,000.
Result fields
remainingRuns
,
reservedRuns
,
usedRuns
,
totalRuns
,
remainingBreakdown
,
state
, and
readiness
, as in
obss quota inspect
. The plugin returns no
billingUrl
.
list_quota_ledger
List run balance history.
Returns a page of run balance transactions. It offers no purchases or billing actions.
Read-only · CLI:
obss quota ledger
Parameter
Type
Required
Description
types
array of
grant
,
reserve
,
capture
,
release
,
expire
,
adjust
No
Only these transaction types.
limit
integer
No
Results per page, up to 100 (default 50).
cursor
string
No
The
nextCursor
from the previous page.
Result fields
Each entry has a
qtx_
transactionId
,
type
,
amount
,
effectiveAt
,
expiresAt
,
source
, and the
experimentId
and link of the experiment that caused it, if any.
hasMore
and
nextCursor
page the list.
list_notifications
List notifications.
Returns a page of your notifications, optionally for one project.
Read-only · CLI:
obss notifications list
Parameter
Type
Required
Description
project_id
string
No
Only this
prj_
project’s notifications.
status
all
,
unread
No
All notifications, or unread ones only.
limit
integer
No
Results per page, up to 50 (default 25).
cursor
string
No
The
nextCursor
from the previous page.
Result fields
Each notification has
notificationId
,
insertedAt
,
status
,
title
, and a safe link. A schedule’s paused, skipping, and short-of-runs notices, and the problem notices of the experiments it launched, also carry its
scheduleId
, which you can read with
get_schedule
.
acknowledge_notifications
Mark notifications as read.
Marks one notification, or up to 500 unread ones, optionally in one project. Repeating a call gives the same result.
Changes data · CLI:
obss notifications acknowledge
Parameter
Type
Required
Description
action
one
,
all
Yes
Mark one notification, or a page of unread ones.
notification_id
string
With
one
The notification to mark.
limit
integer
No
With
all
: notifications in one call, up to 500 (default 500).
project_id
string
No
Only this
prj_
project’s notifications.
Result fields
acknowledgedCount
and, with
all
,
hasMore
. Repeat the call while
hasMore
is true.
inspect_notification_preferences
Get notification preferences.
Returns your account-wide setting for each category and channel, and whether each can be changed, is required, or doesn’t apply.
Read-only · CLI:
obss notifications preferences
This tool takes no parameters.
manage_notification_preferences
Change notification preferences.
Changes only the categories and channels you send. The rest stay as they are.
Changes data · CLI:
obss notifications preferences update
Parameter
Type
Required
Description
patch
object
Yes
Category keys, each set to
true
or
false
for every channel, or to an object of channels.
Category key
Channels
experiment-updates
email
,
in-app
billing-updates
email
,
in-app
connection-alerts
email
,
in-app
product-marketing
email
For example,
{"experiment-updates": {"email": false, "in-app": true}}
keeps experiment updates in-app only.
Ask Obss
Ask Obss
, the in-app assistant, uses the tools on this page.
disconnect_ask_oqo
Disconnect Ask Obss.
Disconnects Ask Obss from your account in this organization until you connect it again on the Ask Obss page. Your chats stay.
Destructive · CLI:
obss ask disconnect
This tool takes no parameters other than
expected_organization_id
.
Result fields
removed
is
false
when Ask Obss wasn’t connected, and then nothing changes.
Next steps
Plugin (MCP) reference overview
Sign-in, approvals, paging, retries, and errors.
Install the Obss Plugin
Install the plugin and start working from your AI assistant.
CLI reference
The same operations as terminal commands.
Launch an experiment
Build a launch plan and run it.
Overview
IDs and limits
Powered by
This documentation is built and hosted on Mintlify, a developer documentation platform