Skip to main content
Where Evals check a run’s outcome, Insights check how the agent got there: failed calls, missing context, long detours, and wasted retries. Obss groups those problems into your project’s frictions so you know what to fix first. Two models do this work. The Insights model reviews each run and reports the problems the agent hit. The Experiment insights model matches each problem to an existing friction or names a new one. It also writes the sentence in each Brief, the note under a difference on Comparisons. An experiment without Insights has no Brief. Insights don’t read the changed files or change Evals verdicts. A run can have Evals, Insights, both, or neither.

Turn on Insights

1

Assign the models

Assign the Insights model and the Experiment insights model under Organization settings → Obss AI features.
2

Choose Insights for an experiment

Turn on Insights when you launch an experiment, or choose Run insights on an experiment that already ran.
3

Read the frictions

Frictions appear on the experiment’s Frictions tab, on All frictions, and on each run’s Traces at the steps they cite.

Choose a mode

If your organization has Agent mode for Insights, pick a mode when you launch, or in Run insights or Rerun Insights. Both modes use your Insights model and Experiment insights model, and report frictions in the same way. In Agent mode, each difference on Comparisons also shows what the runs did differently.

What Insights read

Your Insights model reads the run’s recorded steps. If the run doesn’t fit the model’s context window, a note above the Timeline on Traces (and on each Traces column in Compare) says what was left out:
  • A numbered range of steps didn’t fit. The model gets a count of those steps by kind and tool instead.
  • A long step was shortened.
  • Some friction signals were shortened.
  • The trace is larger than the 32 MB limit for one run file, so the steps couldn’t be read.
No note means the model read everything. A friction covers only what the model read. In Agent mode, nothing is shortened, so a note shows only when the saved run is incomplete, for example when the trace was too large to save. If the work still doesn’t fit after Obss shortens it, Insights fail for that run and the message names the model and the feature: Insights or Experiment insights. Choose a model with a larger context window under Obss AI features and run Insights again. A failed Insights read isn’t a friction.
Runs processed before Obss added these notes show none, so on an older run a missing note doesn’t prove a full read.

Which models ran

Each Insights attempt keeps the models it started with, even if you change a model later.

Pending frictions

When a run shows no frictions, Obss tells you whether it has none or Insights haven’t finished:
  • With no filters set, the experiment’s Frictions tab says Warming up while Insights can still add frictions.
  • Until grouping is done, a run’s frictions show only on its trace, and the friction list counts the runs with ungrouped frictions and the runs Insights haven’t reviewed.
  • Rerun Insights replaces each run’s frictions with the new review. Frictions keep their IDs.
From the CLI and the plugin, obss results analyze counts the runs that Insights reviewed, the runs still pending, and the runs they didn’t review. An empty friction list is final when no run is pending. obss frictions list also reports runs that had Insights off.

Run Insights again

Rerun Insights to review the runs again, for example after you assign a different model. Evals verdicts don’t change. When Insights finish, Obss writes the experiment’s Brief again.
Open the experiment and choose Rerun Insights (or Run insights if Insights haven’t run yet). You can change the Insights model, the Experiment insights model, and the mode.

From the CLI and the plugin

Next steps

Frictions

Rank the problems agents hit and decide what to fix.

Run details

Read the trace at the steps a friction cites.

obss evals and insights

Every command and flag.

Agent mode

Have Obss’s insights agent review each run in depth.