Skip to main content
Setup takes a few minutes, and the runs take several more after you launch. You can leave the page while they finish.

Prerequisites

  • An Obss account. Every organization starts on the free plan, which resets its runs each month.
  • Access to models through an API key (such as OpenAI or Anthropic), a gateway (such as OpenRouter), or a personal subscription (such as ChatGPT or Claude). Model providers lists them all.
  • Optional: the CLI or the Obss Plugin, to follow the CLI or Plugin (MCP) tabs.

Run your first experiment

1

Sign in

Go to obss.app, select Get started, and create an account or sign in. You land on Home in My first project, a project that Obss creates for you.
2

Connect a model provider

On Home, select Connect models to get started (or open Organization settings → Model providers) and connect an API key or a gateway. Connect a personal subscription, such as OpenAI (ChatGPT Subscription) or Anthropic (Claude Subscription), under Profile → Integrations instead.When the provider lists its models, Home shows New experiment.
You enter secrets, such as API keys and OAuth sign-ins, only in the dashboard. The CLI, the plugin, and Ask Obss can’t accept them.
3

Check agents and the default model

Open Manage → Agents. Built-in agents are on by default, so check that at least one shows models, such as Claude Code with an Anthropic key or Codex with an OpenAI key. Agents lists which providers each agent supports.Then open Organization settings → Obss AI features and check that Default model has a model and no warning. Obss’s AI features (Evals, Insights, and Ask Obss) use this model from your providers unless you choose another under Advanced, and Obss can choose it for you when your first provider connects. If you plan to use Ask Obss, also check that the Ask Obss model meets its prerequisites.
4

Start from a template

On Home, under Templates, select one of these:Getting started is fastest, but its only treatment is Raw Agent, so it can’t show lift (the change in pass rate that a tool causes). Skill lift shows lift for a skill. Both have three tasks, so Comparisons shows No clear difference for them. Use them to learn the screens, and add tasks before you judge a skill.Obss adds the template’s tasks, tools, and treatments to your project and opens New experiment with them selected. Learn more in Templates.
The new tasks appear under Manage → Tasks.
5

Launch the experiment

In the Configuration panel, review Tasks, Agents, Treatments (Toolsets), and Trials, and keep Evals and Insights on. The footer shows the runs the launch uses and how many you have left, for example Uses 3 runs. Select Launch.
Each run counts against your organization’s run balance. Learn more in Launch an experiment.
6

Read the Matrix

When the runs finish, open the experiment on Matrix, a heat map of pass rates for each task, agent, and treatment. On the Evals metric, hover over a cell to see the trial count, such as “2 of 3 trials passed”. The metric control also shows cost, tokens, duration, and more.With Skill lift, compare each skill version’s cells with the Raw Agent cells for the same task. Matrix shows no lift number: the difference between the cells is the lift.This Matrix is from the larger Stripe interface evals template, turned with Transpose so each row is an agent and treatment and each column is a task.
Matrix with the Evals metric: rows are agent and treatment combinations, columns are five tasks, and each cell shows a pass rate.Matrix with the Evals metric: rows are agent and treatment combinations, columns are five tasks, and each cell shows a pass rate.
From the CLI, run obss results analyze summarize_scope --experiment-id exp_8f2c41d0. From the plugin, ask for a summary: it calls Analyze experiment results (analyze_results).
7

Check Frictions and open a run

Select the Frictions tab. Each friction is one problem that agents hit, and it shows where to fix it. Then select Runs and open a run. Its Traces, Output, and Evals tabs show what the agent did, what it produced, and the verdict on each rubric criterion.With Skill lift, a notice can flag runs that hit machine or task frictions. A problem in the task or its machine caused those, so don’t count their results against the agent or the treatment.From the CLI, run obss frictions list --experiment-id exp_8f2c41d0. From the plugin, ask for the frictions: it calls List a project’s frictions (list_frictions).
You launched an experiment and read its Matrix, Frictions, and runs. To ask about the results in plain language, open Ask Obss from the dashboard header.

Use the CLI or the plugin instead

Next steps