Skip to main content

The evals platform
your coding agent can operate.

Give your coding agent a framework and managed infrastructure
to build evals and custom benchmarks for real-world tasks.

Connect your coding agent

Setup guide

Plugin Recommended

codex plugin marketplace add Obss-Inc/agent-plugin \
  --sparse .agents/plugins \
  --sparse distributions/openai
codex plugin add obss@obss
Or add MCP directly
codex mcp add obss --url https://mcp.obss.app/mcp
codex mcp login obss
Sign in when prompted. Use one connection method per agent.

Four modules.
One evals platform.

Evaluate products and agents, inspect how the work gets done,
and build benchmarks around your workflows.

Product evaluations01

Can agents actually use your product?

Evaluate the agent-facing surfaces your customers give to their agents. Follow real workflows from setup to completion.

Agent evaluations02

Does your agent get the job done?

Compare your agent’s baseline with its next version on the same tasks. Inspect outcomes, time, and token usage.

Agent experience03

See your product through an agent’s workflow.

See which docs and tools agents use, where they get stuck, and how they recover.

Custom benchmarks04

Build a benchmark around your work.

Benchmark an internal workflow or compare products. Use a shared taskset and rubrics across repeated trials.

From one question
to a repeatable eval.

Stay in your coding agent.
Obss gives it a workflow to follow and the tools to do the work.

01 / 04

Start with the context you already have.

Give your coding agent your product docs, SDK, and available workflow evidence. It follows the Obss framework to draft tasks, realistic synthetic data, and rubrics for you to review.

Explore the framework
obssEvaluation / Design
ASK YOUR CODING AGENT

Review my docs, SDK, and connected production traces. Create a taskset I can start with.

Representative tasksIntegration · Retrieval · Updates
Explicit rubricsExpected outcome + evidence

Real environments.
Repeatable experiments.

Your tasks, your tools, and the starting state every trial needs.

Your evaluation workspace
tasks.jsonThe work to complete
rubrics.jsonWhat success means
fixtures/Realistic data and files
workspace/Repos and dependencies
Real appsCloudRepos + data

A framework your agent can follow.

Design tasksets, realistic synthetic data, and outcome-based rubrics with a structured evaluation workflow.

One workflow.
Across agents and models.

Create tasks, launch repeated trials, and inspect results across supported agent and model configurations.

Fresh workspaces

Start each run from the task’s pinned files and machine configuration.

Rubric-level results

Review each requirement’s verdict alongside its recorded evidence.

Versioned experiments

Keep the task, rubric, file, and toolset versions behind each result for later review.

Before your first eval.

Am I evaluating my product or my coding agent?

Your coding agent operates Obss. You choose what to evaluate: how agents use your product, how your custom agent performs, or how agent and model configurations compare on your tasks.

What does Obss add to my coding agent?

A framework for authoring tasks and rubrics, plus managed infrastructure for run environments, repeated trials, comparisons, and saved evidence. Your coding agent can create resources, launch experiments, and investigate results through the plugin, MCP, or CLI.

How are outcomes graded?

Obss uses a model-based judge to assess your rubric against the recorded trajectory and changed files. For work in external apps, design tasks to read back the resulting state and retain that evidence. Review the criterion-level verdicts alongside the underlying run.

Can I evaluate during development?

Yes. Launch experiments from your coding agent before deploying a change. Attach starting repositories, sample data, and the tools you want to evaluate; Obss runs the tasks in managed cloud sandboxes.

How do I keep trials independent?

Each run starts from the task’s pinned files and machine configuration. For external apps, prepare test accounts or separate resources so concurrent trials do not change each other’s data.

What do the results tell me?

Results describe the tasks, toolsets, agent and model configurations, and starting conditions you evaluated. Repeat trials to observe variation, then inspect criterion-level evidence and ungraded runs alongside aggregate pass rates.

Your next eval starts
with your coding agent.

Give it the framework and infrastructure
to design, run, and analyze your evaluations.

Read the quickstart