https://mcp.obss.app/mcp), grouped by area. Each entry starts with the title your host shows in bold, then its badge, matching CLI command, and parameters. Sign-in, approvals, paging, retries, and errors are on the overview.
Areas: Identity and projects · Catalog, resources, and uploads · Experiments · Schedules · Runs and results · Frictions · Evals and Insights · Tags · Data · Agents, models, and setup · Quota and notifications · Ask Obss
- Badges are explained under Approvals: Read-only, Changes data, Destructive (your host asks first), Irreversible, and You approve each call (spends runs, creates a standing rule, or makes results public).
project_idis optional on most tools. Without it, the server picks a project for that call only (Projects).expected_organization_idis an optional string on every tool that changes data, left out of the tables. Sendorganization.idfromget_identity(Organization check).limitabove its maximum is capped, not refused.
Identity and projects
get_identity
Get the signed-in user and organization. Returns your Obss user and signed-in organization. Sendorganization.id as expected_organization_id on every write.
Read-only · CLI: obss whoami
This tool takes no parameters.
get_project_context
Get the organization and its projects. Returns the organization and its projects, with the valuesmanage_project needs to rename one.
Read-only · CLI: obss projects list, obss projects current, obss projects use, obss projects inspect
manage_project
Create or rename a project. It follows the same rules as the dashboard (Projects). Changes data · CLI:obss projects create, obss projects update
preview_delete_project
Preview project deletion. Counts what deleting a project removes by kind, the running experiments it stops, and the blockers. Show the result to the person beforedelete_project.
Read-only · CLI: obss projects delete --dry-run
delete_project
Permanently delete a project. Deletes a project and everything in it, from tasks, tools, and treatments to experiments, schedules, runs, Evals, and Insights. Running experiments stop first; the result says how many. Destructive · Irreversible · CLI:obss projects delete
With blockers, it returns
project_deletion_blocked and deletes nothing. An already deleted project returns project_not_found.
Catalog, resources, and uploads
Find, create, change, upload, and delete the project’s tasks, files and machines, tools, and treatments.inspect_catalog
Search the project catalog. Lists the project’s tasks, files, machines, tools, treatments, and agents, in a fixed order, each with its tags.total and kindCounts cover every match, not just the page.
Read-only · CLI: obss catalog list
Result fields
items, total, kindCounts, and page (requestedLimit, returnedCount, hasMore, and nextCursor). Each item lists its tags. An agent item’s id is its agm_ ID, which a launch plan’s agentSelections names. With projection: "full", an MCP server’s facts.toolCount is the number of tools its latest version listed at its last check, or null until a check lists them.
browse_github_source
Browse public GitHub repositories. Finds a public github.com repository, ref, and path to import as a file or skill, and saves nothing. Pass the result toauthor_resource as source.repository, source.ref, and source.path.
Read-only · CLI: obss github browse
When
truncated is true, narrow query. A URL result also returns its ref and path.
inspect_resource
Get a resource and its versions. Returns one record’s details, direct dependencies, and a page of version history. Read-only · CLI:obss resources inspect
A machine’s
definition view adds variableOverrides: machine variables that a CLI or SDK in the project also sets, with those tools and a note. In that tool’s treatments, the tool’s value wins (When a tool sets the same name).
inspect_resource_impact
Check what depends on a resource. Shows the tasks and treatments that use a record directly, and whether it is managed or pinned. Call it before a change. Read-only · CLI:obss resources impact
author_resource
Create or update a resource. Creates or updates a task, file, machine, tool, or treatment, with the dashboard’s rules. The tool’s schema has the exact definition fields and limits. Changes data · CLI:obss resources apply
Fields of
operation:
- Rubric: a rubric has
schemaVersion: 1and up to 100criteria, each withindexandtext. Criteria use Yes / no by default, and Advanced mode adds fields such astype,negate, andaggregation(Tasks and rubrics). - Shape: follow the branch for your
actionandkindexactly. Don’t copy catalog-only fields such asid,key,version, orversionIdinto a branch that doesn’t take them. An exact retry returns the first result, and a staleexpectedUpdatedAtreturnsresource_revision_conflict. - GitHub source: a GitHub source saves a snapshot at the resolved commit. Later GitHub changes apply only when you call
manage_resourcewithsync_github. Omitreffor the default branch andpathfor the whole repository. A whole-repository skill import finds everySKILL.mdat any depth and names each skill by thenamein itsSKILL.md(Skills). For local archives and task imports, usecreate_upload. - Secrets: a definition names the environment variables and headers it needs, never their values. A value in an argument returns
secure_input_requires_web, and the saved record returns the dashboard link where a person enters each value.
Every project already has the Raw Agent treatment for a baseline.
bulk_author_resources
Create or update many resources. Applies up to 20author_resource operations in order, each with its own mutationAttemptId and revision.
Changes data · CLI: obss resources bulk-apply
On a failure, the result keeps the completed entries and gives
failedIndex, failedMutationAttemptId, the error, and notStartedCount. Fix the failed operation and resend the whole batch: completed attempt IDs return their first result. For archives, use create_upload.
manage_resource
Set official versions and defaults, or sync from GitHub. Sets or clears a tool’s official version or the default machine, refreshes a file or skill from GitHub, or moves a treatment to its tools’ current versions. Changes data · CLI:obss resources lifecycle
sync_github saves a new snapshot only when the selected content changed. Past runs keep their snapshots.
download_resource_archive
Download a file or skill archive. Returns the ZIP as base64, with its size and SHA-256. Archives over 8 MB download from the record’s dashboard page. Read-only · CLI:obss resources archive
create_upload
Start an archive upload. Creates or replaces a file, creates or versions skills, or imports tasks from an archive. Never put archive bytes or base64 in arguments. Changes data · CLI:obss uploads create
Choose one path:
- You can read the archive. Send
file, then PUT the raw bytes totransfer.urlwithcurl -X PUT --upload-file <archive>and one-Hper entry intransfer.headers. Never send Obss credentials there. Then callcomplete_upload. - A person picks the archive. Omit
file. The person opensupload.uploadUrlwhere they’re signed in (hosts that support URL requests show it), and that page completes the upload. Pollget_upload.
upload.projectId as project_id to the other upload tools. Each upload expires at upload.expiresAt, and an abandoned upload creates nothing. A stale expectedUpdatedAt fails at once.
A task import creates at most 50 tasks.
get_upload
Get an archive upload. Returns an upload’s status, declared file,uploadUrl, lastError, and, once completed, its records in result. Only its creator can read it.
Read-only · CLI: obss uploads get
Poll while
status is awaiting_upload. If it has lastError, upload a corrected archive. completed, failed, cancelled, and expired are final.
complete_upload
Complete an archive upload. Checks the archive against the declared size and SHA-256, validates it, and applies the operation once, including any file replacement or official skill version change. Destructive · CLI:obss uploads complete
The call returns
upload_not_ready until the bytes arrive. An archive error keeps the upload awaiting_upload with lastError. An operation error, such as a stale revision or a deleted target, ends the upload as failed.
cancel_upload
Cancel an archive upload. Cancels an unfinished upload and deletes its archive. Destructive · CLI:obss uploads cancel
Cancelling again returns the cancelled upload. A completed upload can’t be cancelled, so change or delete its records instead.
preview_delete_resources
Preview resource deletion. Shows what deleting the records removes or cancels (related experiments, runs, Evals, and stored files) and what blocks it. Read-only · CLI:obss resources delete --dry-run
resourceType is task, task_version, skill, skill_version, file, machine, mcp_server, mcp_server_version, cli, cli_version, sdk, sdk_version, treatment, or experiment.
Result fields
Show the plan to the person, including each schedule, before you call
delete_resources.
delete_resources
Permanently delete resources. Deletes experiments, tasks, files, machines, tools, or treatments (or one version of a task or tool), with the experiments, runs, Evals, and Insights that depend on them, after cancelling active runs. Schedules of a deleted experiment are deleted; schedules whose experiment uses what it deletes are paused. Destructive · Irreversible · CLI:obss resources delete
Blocked items return
resource_deletion_blocked. Otherwise the deletion starts in this call, and you poll get_resource_deletion until its status is final.
get_resource_deletion
Get resource deletion status. Returns the status of a deletion fromdelete_resources.
Read-only · CLI: obss deletions status
Poll while the status is
preview_ready, queued, or deleting. These statuses are final: completed, failed, failed_to_start, needs_attention, and cancelled_before_start. Any other status reads unknown.
Experiments
An experiment runs tasks × treatments × agents × trials. These tools check, launch, watch, cancel, and share experiments. A typical launch callsinspect_catalog, preflight_experiment, launch_experiment, get_launch_status, get_experiment_status, and then list_runs.
preflight_experiment
Check an experiment plan before launch. Checks the exact plan, its tools’ and machines’ setup, your provider access, the revisions, and your run balance, without reserving runs. Read-only · CLI:obss experiments preflight
The call succeeds even when the launch is blocked, so check
ready. A plan with evalsMode or insightsMode set to agent gets the blocker agent_mode_unavailable when your organization can’t use that Agent mode, or agent_mode_model_unsupported, with a setupUrl to Obss AI features, when a model can’t run it.
Result fields
These are the same fields as in obss experiments preflight.
launch_experiment
Launch an experiment. Launches an exact plan that names catalog versions by ID, and returns alaunchId (adm_…) for get_launch_status.
Changes data · You approve each call · CLI: obss experiments launch
Fields of
plan:
Result fields
A refused launch returns one of these codes:
get_launch_status
Get launch status. Returns a launch’s progress by thelaunchId from launch_experiment, rerun_experiment, or test_custom_agent, before it has an exp_ ID, with the same phase, terminal, and counts as get_experiment_status.
Read-only · CLI: obss experiments status, obss experiments watch
Once
experiment.experimentId is set, use it with the other experiment tools (Polling).
list_experiments
List experiments. Lists the project’s experiments with their tags and lineage. Experiments launched again or by a schedule share their source’s name, so tell them apart by lineage. Read-only · CLI:obss experiments list
An experiment is listed when one of its runs matches every scope field.
schedule_id also lists a launched experiment that has no runs yet. An unknown ID returns scope_not_found. The result has no scope.
Result fields
Each experiment has its tags, infraBlockedCount (Progress counts), and lineage as in get_experiment. hasMore and nextCursor page the list.
get_experiment
Get an experiment. Returns an experiment’s details, progress counts, launcher, runs used, and lineage. Read-only · CLI:obss experiments get
Result fields
lineage has these parts:
launchedVia:new,from_experiment,schedule, orschedule_launch_now.basedOn: the parent experiment, the one it was launched from, withdeleted: trueonce it’s gone.schedule: thesch_schedule that launched it, and its slot.children: its child experiments (list them withlist_experimentsandbased_on_experiment_id),scheduleCount, and the 10 newest of those schedules (all of them withlist_schedulesandexperiment_id).
get_experiment_status
Get experiment status. Returns the phase,terminal, and progress counts.
Read-only · CLI: obss experiments status, obss experiments watch
While
terminal is false, call again with wait_ms or after pollAfterMs. When terminal is true, read the final runs with list_runs.
get_experiment_plan
Get an experiment’s launch plan. Builds a launchable plan from an earlier experiment, at its task versions or the latest ones. Edit it, then pass it tolaunch_experiment.
Read-only · CLI: obss experiments clone-plan
With
comparison_id from get_evals_comparisons, the plan runs that likely difference again: the tasks that can still show a difference, the two sides (both treatments and agents for an interaction), and enough runs each to likely settle it. Launching the plan spends the person’s runs, so say the run count and ask first. version_mode then defaults to experiment, treatments use their current tools as with Launch again, and Evals uses the latest Evals attempt’s settings.
rerun_experiment
Launch an experiment again. Relaunches a finished experiment with the same tasks, treatments, agents, trials, Evals, and Insights, or launches a schedule now withschedule_id. Evals and Insights run in the modes the experiment launched with, including Agent mode. If your organization can no longer use that Agent mode, the launch is refused with agent_mode_unavailable; use get_experiment_plan and set evalsMode or insightsMode to standard.
Changes data · You approve each call · CLI: obss experiments rerun, obss schedules launch
With
schedule_id, it launches the schedule’s experiment at the latest task versions, under the schedule’s name, without moving the next launch. It refuses with experiment_schedule_run_active while the last one still runs. Poll the launchId with get_launch_status. To change the plan first, use get_experiment_plan, then launch_experiment.
cancel_experiment
Cancel an experiment or run. Cancels a whole experiment, or every still-active trial of one task, treatment, and agent combination. Completed work stays available. Destructive · CLI:obss experiments cancel
A finished or stopping target returns its status. Poll
get_experiment_status until the cancellation finishes.
publish_experiment
Publish an experiment. Creates a public link that anyone can view without an account, with runs, traces, outputs, Evals, Insights, and text files up to 256 KiB. Larger or binary files and private machine settings aren’t shared. Changes data · You approve each call · CLI:obss experiments publish
Later results go live only through
update_experiment_publication. To share privately, invite the person to your organization (Share results).
list_experiment_publications
List published experiments. Lists your organization’s public links. Read-only · CLI:obss experiments publications list
get_experiment_publication
Get an experiment’s public link. Returns a link’s status, current version, whether an update is available, and thelockVersion a change needs.
Read-only · CLI: obss experiments publications get
update_experiment_publication
Update a published experiment. Replaces the results at the same link with the latest ones. The earlier results stay until the update is ready. Destructive · You approve each call · CLI:obss experiments publications update
unpublish_experiment
Unpublish an experiment. Stops sharing an experiment, and the public link stops working at once. Destructive · CLI:obss experiments publications unpublish
Unpublishing again returns the unpublished experiment.
get_public_experiment
Get results from a public link. Reads the results at a public experiment link, one section at a time. Read-only · CLI:obss gallery get
A large Frictions result arrives in parts (
part.index of part.count). Read every page, then append each part’s clusters, family clusters, and outcome badge audits in order. A repeated clusterKey continues the previous cluster’s occurrences.
Schedules
A schedule launches one finished experiment again on a cron cadence in a time zone.list_schedules
List schedules. Lists a project’s schedules, newest first, withnextRunAt, the person each runs as, the last result, and needsAttention with pauseReason when Obss paused it.
Read-only · CLI: obss schedules list
For a schedule’s experiments, call
list_experiments with schedule_id.
get_schedule
Get a schedule. Returns its cadence in plain words,nextRunAt, experiment, run use, last result, and the lock_version a change needs.
Read-only · CLI: obss schedules get
Result fields
runsPerLaunch is the runs each launch uses. runsPer30Days is the runs in the next 30 days, not a monthly average. It’s 0 while the schedule is paused, or when no launch falls in the next 30 days.
preview_schedule
Preview a schedule. Shows what a new schedule or a change would do. Call it beforemanage_schedule and tell the person the runs it uses.
Read-only · CLI: obss schedules preview
Result fields
The preview has the cadence in plain words, the time zone it stores, the next 3 launches,
runsPerLaunch, and runsPer30Days. A preview of a paused schedule counts it as resumed, and schedule.status says whether a changed schedule stays paused.
shortfallis a warning, not a refusal. It’s null while your runs cover the launches, or while the schedule is paused. Otherwise it givesat(the first uncovered launch),runsShort,availableRuns,launchesCovered,runsToAdd(to cover every launch untiluntil),until,untilReset(true whenuntilis your next run reset), amessage, and anactionlinking Usage & billing.blockerslists everything that would stop a launch now, each with acode,message, andactionwith its fix link, so the person can fix them all at once. While it isn’t empty, a create, a resume, or a change to the experiment, cron, or time zone refuses withschedule_launch_blocked.blocker(the first one) is for older CLI releases.
manage_schedule
Create or change a schedule. Creates, changes, pauses, or resumes a schedule. Each launch uses the project’s tasks, treatments, and agents as they are then. Changes data · You approve each call · CLI:obss schedules create, obss schedules update, obss schedules pause, obss schedules resume
- A create, a resume, or a change to the experiment, cron, or time zone checks the launch as you, and makes you the person the schedule runs as. If blocked, it refuses with
schedule_launch_blocked, withdetails.blockersshaped as inpreview_schedule; the message is the blocker’s own, or a count when there are several. updatechanges only the fields you send. A schedule still named after its experiment takes the new experiment’s name.- Obss pauses a schedule when a launch needs a fix, or after 3 scheduled launches in a row fail. A checked change clears the reason, but the schedule stays paused until you resume it.
- After the person fixes the cause,
resumechecks the launch and resumes. If something still blocks it, give the person every fix link indetails.blockers, then resume again. Pause and resume are safe to repeat, and resume never launches at once. - A create or resume past 50 active schedules in a project returns
schedule_limit_reached. - To launch a schedule now, call
rerun_experimentwithschedule_id.
delete_schedule
Delete a schedule. Deletes a schedule. The experiments it launched stay. Destructive · Irreversible · CLI:obss schedules delete
An already deleted schedule returns
schedule_not_found.
Runs and results
A run is one execution of a task with a treatment and an agent. These tools read results and run details.list_runs
List run results. Returns a page of runs with stable IDs, typed errors, and a plain explanation of each result incriteriaSummary.
Read-only · CLI: obss runs list
Result fields
Each run has its stable IDs, typed errors, and
criteriaSummary. It can also have:
treatmentUse: each tool’s state and calls, as inget_run_profile. It’s null when the run has no usable profile.blockedReason: on a blocked run, itscauseand a one-linemessage.setupFailure: present when a setup command stopped the run before its agent started. It has theownerstep,commandNumberwhen the owner has several commands of that kind (null for a package install, wheremanagerandpackageOwnersname the package manager and who declared the packages), thecommand,exitCode(null on a timeout, withtimeoutSeconds), the lastoutputlines, andoutputTruncated. See troubleshooting.
experiment_id to friction_id, name the records whose runs to read; a run must match them all. list_frictions and get_friction take the same ones, and list_experiments all but experiment_id. An unknown ID returns scope_not_found, not an empty list. scope in the result names each record with its dashboard page.
get_run_profile
Get a run profile. Returns the same profile as the dashboard’s Run profile rail: treatment use, tool calls, what stood out, tokens, cost, and time. Read-only · CLI:obss runs profile
Result fields
Step ordinals refer to
get_run_trajectory, and the per-turn series is the runTurns dataset of query_data.
get_run_trajectory
Get a run trajectory. Returns a page of a run’s steps in order, each with its model turn and tool, plus the run’s input, cached-input, and output token totals (or null). Read-only · CLI:obss runs trajectory
Continue with
nextAfterOrdinal while truncated is true.
Result fields
steps (each with its ordinal, model turn, and tool; tool steps include their measured duration when the agent reported it), the run’s token totals, truncated, nextAfterOrdinal, and coverage, which gives the returned range, the total steps, and whether this page holds the whole trajectory. For a custom agent, the run also has the same stopReason, reported model, and agentError as get_run_profile.
get_run_transcript
Get a run’s raw transcript. Returns a page of the original transcript: UTF-8 text incontentText, other bytes as base64 in contentBase64. sha256 covers the whole transcript.
Read-only · CLI: obss runs transcript
get_run_output
Get a run’s output and files. Returns a page of the final message, the unified diff, the file manifest, one kept file, or the output archive. Read-only · CLI:obss runs output
Text pages are UTF-8 in
contentText, and other pages are base64 in contentBase64. sourceTruncated marks a preview that was capped.
analyze_results
Analyze experiment results. Runs one ranked analysis over experiment and run results. Read-only · CLI:obss results analyze
scope takes these fields. A run counts only when it matches every field you send.
Ranked analysis is never paged:
limit is an upper bound, and the response size limit can return fewer items.
Result fields
When page.truncated is true, these fields say why and where to read the rest:
The analysis is in
result:
summarize_scope:result.totals(runs,passedRuns,failedRuns,evalsFailedRuns,infraBlockedRuns,inconclusiveRuns,pendingRuns) andresult.treatments[](treatmentKey,runCount,passedRuns,failedRuns).compare_treatments:result.treatments[]with the same counts plusfailureRate,isReferenceTreatment, anddeltaFromReference.failureRate, andresult.referenceTreatmentKey.
passedRuns / (passedRuns + failedRuns), or 1 - failureRate. pendingRuns counts runs with no verdict that Evals can still judge. They stay out of the pass and fail counts and out of inconclusiveRuns, so wait until pendingRuns is 0. summarize_scope and search_frictions return insightsCoverage: analyzedRuns, pendingRuns, and unanalyzedRuns (Insights off, skipped, failed, or never run). An empty friction list is final when pendingRuns is 0.
Frictions
A friction is a problem that agents hit, tracked across runs under a stablefrc_ ID.
list_frictions
List a project’s frictions. Finds what keeps going wrong for agents, in the project or any scope, and what to fix first. By default, the most runs the agent never got past come first. Read-only · CLI:obss frictions list
A friction is listed when one of its runs matches every filter.
Result fields
Each field is explained under
obss frictions list.
get_friction
Read one friction with its evidence. Returns its summary, impact, suggested fix, what the agent did, where it happens most, and up to ten recent occurrences with run, recovery, cited steps, and quotes. Read-only · CLI:obss frictions get
Result fields
The friction has the fields of
list_frictions, plus these:
To list all the friction’s
runs, call list_runs with friction_id and the same scope.
Evals and Insights
Evals score each run against its task’s rubric, and Insights read each run’s trajectory to find frictions. Preview, trigger, then poll the results withwait_ms.
preview_evals
Preview Evals for an experiment. Shows which runs Evals can score, rubrics that need attention or changed, model readiness, and the modes available. Read-only · CLI:obss evals preview
Result fields
ready, modelsReady, userProviderReady, setupUrl, eligibleRunCount, runsWithoutRubricsCount, runsWithInvalidRubricsCount, runsNotJudgeableCount, rubricChanges, failedRunIds, and mode: experimentMode (the mode the experiment launched with), latestMode, agentAvailable, and, only while your organization can use Agent mode, estimate (runs and model). Their meanings are under obss evals preview.
trigger_evals
Start Evals for an experiment. Starts an Evals attempt, or reruns Evals on the same recorded work after a rubric change. Changes data · You approve each call · CLI:obss evals trigger
Standard scores each run in one pass with your Evals model and condenses very long runs to fit its context window. It’s the fastest and lowest-cost mode. Agent has Obss’s evals agent read the full run and the changed files and check the agent’s claims before it scores. It makes many calls to your Evals model, so it costs more and takes a few minutes per run. Both modes score against the same rubric (how the modes differ).
A request for Agent mode, stated or inherited, is refused with
agent_mode_unavailable when your organization can’t use it: “Agent mode isn’t available for your organization. Use Standard mode instead.” Without mode, the refusal has details.inheritedMode set to agent and says “This experiment launched Evals in Agent mode, which your organization can’t use now. Rerun with mode: “standard”.” When the Evals model can’t run Agent mode, the refusal is agent_mode_model_unsupported, with a setupUrl to Obss AI features. Agent mode is in private beta.
Result fields
evalsId (the new eval_ attempt), status, mode (the mode the attempt uses), pollAfterMs (60 seconds in Agent mode, 10 seconds in Standard mode, or null when the attempt has already finished), and replayed (true when a repeated trigger_attempt_id returned the earlier attempt).
list_evals
List Evals history. Returns the experiment’s Evals attempts, newest first, without reports, each with itsmodels and mode. A failed attempt has error (code and message). An Agent mode attempt also has agentUsage once its agent has reviewed runs: the inputTokens, cacheReadTokens, and outputTokens that the evals agent has used so far.
Read-only · CLI: obss evals list
get_evals_results
Get Evals results. Returns a summary for each run of an attempt, or the exact rubric and verdict JSON for one trial. Read-only · CLI:obss evals get
Result fields
get_evals_comparisons
Compare treatments and agents. Compares an experiment’s treatments and agents task by task, from the Matrix’s Evals verdicts. It returns the sentence that answers the experiment’s question and a block for each difference first, as at the top of Comparisons, then every comparison with its result. Read-only · CLI:obss evals compare
kind is treatment (a treatment against the baseline, for all agents together and for each agent, or each pair of treatments without a baseline), agent (two agents with the same treatment), or interaction (Does a treatment help one agent more?). label is clear (Clear difference), likely (Likely difference), or unclear (No clear difference). unclear means that the runs can’t tell the two sides apart, not that the sides are the same.
The most decisive comparisons come first, the main comparisons before the breakdowns within a label. kind and label only filter the list: labels don’t change, and summary, differences, restLine, and comparisonCount stay the same. A read returns up to 200 comparisons that fit in the response. A cursor fails with invalid_cursor once the comparisons change. More than 10,000 Matrix cells fails with experiment_too_large.
Result fields
Each comparison has
comparisonId (cmp_), kind, label, family (primary for a main comparison, or breakdown), title, scope, headline, sides, verdict, nextStep, sentence, groupLabel, taskSplit, treatmentKey, referenceTreatmentKey, agentKey, referenceAgentKey, arm and reference (or first and second for an interaction), tasks (higher, lower, same), pValue (an exact two-sided sign test), adjustedPValue (the Holm correction within its family), runsLeftOut, confirmation, and confirmationUnavailable (earlier_evals, in_progress, or cannot_settle). confirmation holds the runs that check a likely difference again. cannot_settle means that running these tasks again can’t settle it, and the experiment needs more tasks. To plan them, pass its comparisonId to get_experiment_plan as comparison_id, with a new launch_attempt_id.
Say the result in the words of summary and each verdict, which every surface shows, with verdict and nextStep on one line. nextStep is run_again (with text and runs) or more_tasks (with text), or null. Each field is explained under obss evals compare.
get_experiment_brief
Get an experiment’s Brief. Reads an experiment’s Brief: its clear and likely differences, worded as inget_evals_comparisons, with one written sentence and the frictions seen with the side that did worse. Obss counts every number from the experiment’s Evals verdicts and frictions. A model writes only each sentence and, in Agent mode, each pair’s line.
Read-only · CLI: obss experiments brief
state says how complete the Brief is. ready, failed, not_written, and no_insights are final for the Brief. While state is writing, the differences and counts are already in the read: call again after waitMs milliseconds until state changes. In Agent mode, also call again after waitMs while any finding’s pendingTaskCount is above 0. The tool takes no wait_ms.
Result fields
Each finding has the fields of its block in
differences (comparisonId, label, name, sides, taskLine, verdict, nextStep, byAgent, headline), and kind, title, scope, arm and reference, tasks, pValue, adjustedPValue, worseSide, worseTaskCount, notReviewed, sentence, confirmation, confirmationUnavailable, frictions, afterFix, and issueMarkdown. Each friction has frictionId (frc_), title, summary (what goes wrong, in one line), fixIn, tasks, dashboardUrl, failedRuns (how many of the worse side’s failed runs had it, out of runs), passedRuns (the same for the other side’s passed runs), and, for arm and reference, runsWithFriction out of runsReviewed, which is how the friction was picked. The counts use only the worseTaskCount tasks and the runs that Insights reviewed. afterFix is set for a clear difference with a friction to fix in the product: its text, taskCount, and runs. issueMarkdown is the difference as an issue in Markdown, the text that Copy as issue copies.
When the experiment’s latest Insights ran in Agent mode, each finding also has these fields:
Each pair has
taskVersionId (tskver_), taskName, dashboardUrl, and state: pending while the agent reads it, completed when its lines are written, or failed when the agent gave no answer that passed Obss’s checks, which shows only the two runs. Its worse and better runs each have these fields:
Show the pairs in that order, as the dashboard and
issueMarkdown do: the example with “Like N other tasks” (from similarTaskCount), each task that went differently with “Went differently”, then the rest. A pair says where two runs went different ways, not why the difference happened. What the agent used to read the pairs counts in the Insights attempt’s agentUsage (list_insights).
When you report a friction, give both sides’ counts (“Raw Agent 24 of 28 failed runs · Stripe CLI 0 of 26 passed runs”) and say it was seen with the side that did worse. Don’t call it the cause. To run a likely difference again, pass its comparisonId to get_experiment_plan as comparison_id. To run a clear difference again after a fix, add after_fix set to true. Each field is explained under obss experiments brief.
preview_insights
Preview Insights for an experiment. Shows whether the Insights models and your provider access are ready, with a dashboard link for any secure setup, and the modes this experiment can use. Read-only · CLI:obss insights preview
Result fields
ready, modelsReady, userProviderReady, setupUrl, failedRunIds, and mode: experimentMode (the mode the experiment launched with), latestMode, agentAvailable, and, only while your organization can use Agent mode, estimate (runs, model, and experimentModel when the Experiment insights model is a different model). Their meanings are under obss insights preview.
trigger_insights
Start Insights for an experiment. Starts an Insights attempt, or reruns Insights on the same recorded work. Changes data · You approve each call · CLI:obss insights trigger
Standard reviews each run in one pass with your Insights model. It’s the fastest and lowest-cost mode. Agent has Obss’s insights agent read the whole run and check each friction against what the tools returned. It costs more and takes a few minutes per run. It also does the experiment’s grouping of frictions, which Standard mode does in one pass. For each task of each difference in the Brief, it says what a failed run and a passed run did differently (
get_experiment_brief). Both modes report frictions in the same way. Without mode, the attempt uses the experiment’s launch mode, not the latest attempt’s mode as the dashboard does.
A request for Agent mode, stated or inherited, is refused with agent_mode_unavailable when your organization can’t use it: “Agent mode isn’t available for your organization. Use Standard mode instead.” Without mode, the refusal has details.inheritedMode set to agent and says “This experiment launched Insights in Agent mode, which your organization can’t use now. Rerun with mode: “standard”.” When the Insights model or the Experiment insights model can’t run Agent mode, the refusal is agent_mode_model_unsupported, with a setupUrl to Obss AI features. Agent mode is in private beta.
Result fields
insightsId (the new insights_ attempt), status, mode (the mode the attempt uses), pollAfterMs (60 seconds in Agent mode, 10 seconds in Standard mode, or null when the attempt has already finished), and replayed (true when a repeated trigger_attempt_id returned the earlier attempt).
list_insights
List Insights history. Returns the experiment’s Insights attempts, newest first, without reports, each with itsmodels and mode. A failed attempt has error (code and message); when the insights agent couldn’t group the frictions, error.message starts with “Insights could not group this experiment’s frictions.” An Agent mode attempt also has agentUsage once its agent has reviewed runs: the inputTokens, cacheReadTokens, and outputTokens that the insights agent has used so far, including grouping the frictions and the pairs of runs in the Brief.
Read-only · CLI: obss insights list
get_insights_results
Get Insights results. Returns a summary for each run of an attempt, the attempt’s public report, or the exact Insights JSON for one trial. Read-only · CLI:obss insights get
Result fields
attempt (with its insightsId, models, mode, error, and, in Agent mode, agentUsage), and for each row its status, its error (a failed row’s message starts with “Insights could not analyze this run.” and gives the cause), which follows the same rules as in get_evals_results, and evidenceCoverage (null for older runs). hasMore and nextCursor page the summary view.
Tags
Tags are your own labels on tasks, files, machines, tools, treatments, and experiments. Limits are underobss tags.
list_tags
List a project’s tags. Lists every tag with itstag_ ID, usage by kind, and the updatedAt that rename_tag and delete_tag need. One call returns all of them (at most 500).
Read-only · CLI: obss tags list
Result fields
Each tag has
id, name, usage (total and byKind), createdAt, and updatedAt.
inspect_catalog and list_experiments filter by tag with tags and untagged.
update_tags
Add or remove tags on records. Adds and removes tags on up to 100 records at once. Adding a name that the project doesn’t have yet creates the tag. Changes data · CLI:obss tags add, obss tags remove
Result fields
records (each record’s id, kind, and all its tags after the change) and createdTags (the tags this call created).
You can send the same call again safely. A change that would give a record more than 20 tags changes nothing and returns tag_limit_exceeded. Tags never change a record’s definition, versions, or updatedAt.
rename_tag
Rename a tag. Renames a tag. Every record keeps it, because itstag_ ID doesn’t change.
Changes data · CLI: obss tags rename
Result fields
The renamed
tag, with its id, name, usage, createdAt, and updatedAt.
A tag that changed since you read it returns tag_changed with the current updatedAt. A name that another tag in the project has, in any case, returns tag_name_taken with that tag’s ID.
delete_tag
Delete a tag from every record. Deletes a tag and removes it from every record that has it. Tell the person how many records have it, fromlist_tags, before you call it.
Destructive · Irreversible · CLI: obss tags delete
Result fields
tag (the deleted tag’s id and name) and removedFrom (how many records had it). A tag the project doesn’t have returns tag_not_found.
Data
Query datasets with fields, filters, and aggregates, for questions the other reads don’t answer, such as results by model or mode.inspect_data_model
List the data you can query. Lists every dataset, its fields and types, each field’s filters and aggregates, the default order, and the limits. Read-only · CLI:obss data describe
Result fields
datasets, each with name, title, description, grain, fields, identityFields, defaultSelect, and defaultOrder, and limits. Each field has name, type, nullable, description, filterOperators, and aggregates.
query_data
Query Obss data. Runs one bounded query over a dataset. It never accepts SQL or returns internal IDs, secrets, or credential-like values. Read-only · CLI:obss data query
Result fields
columns, rows, query (the query as Obss ran it), page (returned, hasMore, nextCursor, truncated, and reasons), and limits. The datasets and their fields are listed under obss data, including evals.mode, the model fields of evals and insights, and the per-turn runTurns series.
Agents, models, and setup
Manage a project’s agents, custom agents, and models, your Obss AI features, and the setup steps that need the dashboard.Custom agents are in private beta. Email support@obss.app to enable them for your organization.
inspect_project_ai
Get built-in and custom agents, models, and Obss AI settings. Returns agents, provider and model readiness, your personal connections, model refresh state, and each Obss AI feature’s model and whether it can run. It never returns secrets or starts a refresh. Read-only · CLI:obss project ai inspect
Result fields
The
roles, providers, agents, and models views return agent, provider, model, and role counts, provider readiness, model refresh counts, and blockers. blockers is empty when you can run models; otherwise it says why not, with the dashboard setupUrl where a person fixes it.
A custom agent’s custom block has status (needs_values, testing, needs_attention, not_tested, or ready), description, revision, connectionId, url, definition, env, envNote, testBlockers, handshake (start-up details such as failure.message, failure.detail, and authMethods), fix (the failed step and what to change), mcpServersRefused, and latestRun. See Custom agent fields.
In the providers view, each provider’s modelRefresh has status, listedAt, and the counts of the last refresh. listedAt is set when the refresh has read the provider’s list, so the models view has those models while status is still running. See obss project ai inspect.
A cursor returns stale_cursor when the configuration changed between pages, usually from a model refresh.
manage_project_ai
Create custom agents and change agents, models, and Obss AI settings. Creates or changes custom agents, turns agents and models on or off, refreshes model lists, states context windows, and sets each Obss AI feature’s model and instructions. No operation carries a secret. Changes data · CLI:obss project ai apply
A custom agent’s
source is one of:
kind: "registry"withregistryIdand an optionalversion: an agent from the ACP Registry.kind: "manual"withinstalland optionalargs.installis an npm package (kind: "npm",package), a Python package (kind: "python",package), or a download (kind: "download",url,sha256,command).kind: "agent_json"withagentJson: anagent.jsondefinition.
env lists up to 64 environment variable names. Values are entered only in the dashboard, and the result returns the agent’s page and the dashboard link for each value. skillsFolder is the folder where the agent reads skills, up to 256 characters.
A stale revision changes nothing and returns configuration_changed, with details.currentRevision when the target has one. An invalid operation returns validation_error. Read inspect_project_ai again before you retry. When a public catalog lists a context window for a model, inspect_project_ai reports it as suggestedContextWindow, which applies only after you state it. Connecting, signing in again, switching accounts, and entering credentials stay on the provider’s dashboard link from inspect_project_ai.
delete_project_ai
Remove custom agents or provider connections. Removes a custom agent, deletes a custom provider connection, resets a built-in organization connection, or disconnects your personal provider account. Destructive · CLI:obss project ai delete
A removed custom agent no longer shows in new experiments or the Agents list, and Obss clears its environment variable values. Past results keep its name. Obss refuses the removal while a run of the agent is in progress, either a Test agent run or an experiment. Removing it again returns the same result.
Pass the revisions from
inspect_project_ai. A refused removal changes nothing and returns one of these codes:
test_custom_agent
Test a custom agent. Checks that a custom agent installs, starts, and answers, with one run of the Agent check task: Agent default, the Raw Agent treatment (no tools), no project files, and Evals and Insights off. It uses 1 run. Changes data · You approve each call · CLI:obss project ai test-agent
Poll the returned
launchId with get_launch_status. Each finished run with these settings updates the agent’s status in inspect_project_ai. A Test agent run is a check, not an experiment. It sends no notice and doesn’t show on Home or in the dashboard’s experiments list, but the CLI and the plugin still list it.
It refuses while custom.testBlockers names a blocker, with the first one’s code:
get_setup_links
Get secure setup links. Returns the dashboard links for OAuth connections, API keys, secret headers, and other secret values. Read-only · CLI:obss setup links
Result fields
issues lists each machine, MCP server, CLI, and SDK in the project that still needs a required value or your account connection, with the setupUrl where a person adds it. An MCP server whose saved values aren’t bound to its current address has a field of scope address, and its setupUrl is the server’s Tools tab, where a person selects Refresh. The list is empty when nothing is missing.
inspect_mcp_account
Check your MCP server connection. Checks whether your account can use an MCP server in a treatment, and returns the latest problem, a next step, and the dashboard setup link. A failed reconnect never replaces a working connection. Read-only · CLI:obss mcp accounts inspect
Result fields
ready, recovery (with message, action, setupUrl, retryAt, and apiKeyAlternative), setupUrl, redirectUri, and oauthAppConfigured. Their meanings are under obss mcp accounts inspect. When the server’s saved values aren’t bound to its current address, ready is false and setupUrl is the version’s Tools tab, where a person selects Refresh.
detect_mcp_auth_mode
Detect how an MCP server authenticates. Probes the server’s current definition and reports how it authenticates. It never returns the server’s endpoint or credentials, and the OAuth app’s Client ID and Client secret are entered only in the dashboard. See MCP server sign-in. Read-only · CLI:obss mcp accounts detect-auth
Result fields
list_mcp_server_tools
List an MCP server’s tools. Reads the tools the server listed at Obss’s last check, with the check’sstatus, message, time, tool count, and Tools page (setupUrl). Titles, descriptions, and hints are the server’s own text.
Read-only · CLI: obss mcp tools list
Continue with
after_position while truncated is true.
Result fields
The check’s status, message, and time, the tool count, setupUrl, truncated, nextAfterPosition, usageRunCount, lastChange, headerValuesBound, and each tool with canTurnOff, onRunCount, runCount, callCount, offInTreatmentCount, and offInTreatmentIds. Their meanings are under obss mcp tools list.
When headerValuesBound is false, only a person’s Refresh at setupUrl binds the saved values: refresh_mcp_server_tools fails with mcp_tool_check_needs_web and launches stop with mcp_header_values_unbound. Give the person setupUrl instead of calling it (Saved header values and the server address).
refresh_mcp_server_tools
Refresh an MCP server’s tools. Checks the server’s tools now and returns the list aslist_mcp_server_tools does. A running check is joined, not restarted. A server that needs your account connection or a header value returns that status and its setupUrl.
Changes data · CLI: obss mcp tools refresh
Result fields
The fields of
list_mcp_server_tools, plus refresh, which is running until the check finishes. If the wait ends first, read the list again with list_mcp_server_tools.
It never sends saved header values to an address they aren’t bound to, and it never binds them. When headerValuesBound is false, it fails with mcp_tool_check_needs_web. Give the person details.setupUrl (the version’s Tools tab) to check the address and select Refresh, and don’t call the tool again until they have.
Quota and notifications
Read your run balance and manage your notifications. Buying runs stays in the dashboard.inspect_quota
Get the run balance. Returns the run balance, and whether a launch ofrequested_runs runs can start now. It offers no purchases or billing actions.
Read-only · CLI: obss quota inspect
Result fields
remainingRuns, reservedRuns, usedRuns, totalRuns, remainingBreakdown, state, and readiness, as in obss quota inspect. The plugin returns no billingUrl.
list_quota_ledger
List run balance history. Returns a page of run balance transactions. It offers no purchases or billing actions. Read-only · CLI:obss quota ledger
Result fields
Each entry has a
qtx_ transactionId, type, amount, effectiveAt, expiresAt, source, and the experimentId and link of the experiment that caused it, if any. hasMore and nextCursor page the list.
list_notifications
List notifications. Returns a page of your notifications, optionally for one project. Read-only · CLI:obss notifications list
Result fields
Each notification has
notificationId, insertedAt, status, title, and a safe link. A schedule’s paused, skipping, and short-of-runs notices, and the problem notices of the experiments it launched, also carry its scheduleId, which you can read with get_schedule.
acknowledge_notifications
Mark notifications as read. Marks one notification, or up to 500 unread ones, optionally in one project. Repeating a call gives the same result. Changes data · CLI:obss notifications acknowledge
Result fields
acknowledgedCount and, with all, hasMore. Repeat the call while hasMore is true.
inspect_notification_preferences
Get notification preferences. Returns your account-wide setting for each category and channel, and whether each can be changed, is required, or doesn’t apply. Read-only · CLI:obss notifications preferences
This tool takes no parameters.
manage_notification_preferences
Change notification preferences. Changes only the categories and channels you send. The rest stay as they are. Changes data · CLI:obss notifications preferences update
For example,
{"experiment-updates": {"email": false, "in-app": true}} keeps experiment updates in-app only.
Ask Obss
Ask Obss, the in-app assistant, uses the tools on this page.disconnect_ask_oqo
Disconnect Ask Obss. Disconnects Ask Obss from your account in this organization until you connect it again on the Ask Obss page. Your chats stay. Destructive · CLI:obss ask disconnect
This tool takes no parameters other than expected_organization_id.
Result fields
removed is false when Ask Obss wasn’t connected, and then nothing changes.
