A framework your agent can follow.
Design tasksets, realistic synthetic data, and outcome-based rubrics with a structured evaluation workflow.
Give your coding agent a framework and managed infrastructure
to build evals and custom benchmarks for real-world tasks.
Agent-facing surfaces
Agents
Models & providers
Connect through API keys, subscriptions, or gateways. Available models depend on your connection and agent.
Connect your coding agent
Setup guidePlugin Recommended
codex plugin marketplace add Obss-Inc/agent-plugin \
--sparse .agents/plugins \
--sparse distributions/openaicodex plugin add obss@obsscodex mcp add obss --url https://mcp.obss.app/mcpcodex mcp login obssPlugin Recommended
claude plugin marketplace add Obss-Inc/agent-plugin \
--scope user \
--sparse .claude-plugin distributions/claude-codeclaude plugin install obss@obss --scope userThen start Claude Code and run /mcp.
claude mcp add --transport http --scope user obss https://mcp.obss.app/mcpPlugin Recommended
cursor-agent plugin marketplace add https://github.com/Obss-Inc/agent-plugin.git~/.cursor/mcp.json
{
"mcpServers": {
"obss": {
"type": "http",
"url": "https://mcp.obss.app/mcp"
}
}
}cursor-agent mcp login obsscursor-agent mcp enable obssPlugin Recommended
copilot plugin marketplace add Obss-Inc/agent-plugincopilot plugin install obss@obssThen start copilot and run /mcp auth obss.
copilot mcp add --transport http obss https://mcp.obss.app/mcpCommand line access
curl -fsSL https://obss.app/install | shThen run obss login.
Evaluate products and agents, inspect how the work gets done,
and build benchmarks around your workflows.
Evaluate the agent-facing surfaces your customers give to their agents. Follow real workflows from setup to completion.
Compare your agent’s baseline with its next version on the same tasks. Inspect outcomes, time, and token usage.
See which docs and tools agents use, where they get stuck, and how they recover.
Benchmark an internal workflow or compare products. Use a shared taskset and rubrics across repeated trials.
Stay in your coding agent.
Obss gives it a workflow to follow and the tools to do the work.
Give your coding agent your product docs, SDK, and available workflow evidence. It follows the Obss framework to draft tasks, realistic synthetic data, and rubrics for you to review.
Explore the frameworkReview my docs, SDK, and connected production traces. Create a taskset I can start with.
Ask your agent to prepare a starter repository and synthetic tenant data. Define the expected results, and give every trial its own workspace.
Explore the frameworkSeparate workspaces. Consistent starting conditions.
Keep the SDK, tasks, and rubrics fixed. Compare reference docs with workflow notes across Codex and Claude Code. Pin each agent’s model configuration and repeat both conditions.
Explore the frameworkSame SDK, tasks, and rubrics. Only the guidance changes.
Ask your coding agent to review rubric verdicts, investigate failures, and sample the passes. Connect findings to recorded trajectories and captured files before changing the docs, tools, or agent.
Explore the frameworkImprove the interface. Repeat the same evaluation.
Your tasks, your tools, and the starting state every trial needs.
tasks.jsonThe work to completerubrics.jsonWhat success meansfixtures/Realistic data and filesworkspace/Repos and dependenciesDesign tasksets, realistic synthetic data, and outcome-based rubrics with a structured evaluation workflow.
Create tasks, launch repeated trials, and inspect results across supported agent and model configurations.
Start each run from the task’s pinned files and machine configuration.
Review each requirement’s verdict alongside its recorded evidence.
Keep the task, rubric, file, and toolset versions behind each result for later review.
Your coding agent operates Obss. You choose what to evaluate: how agents use your product, how your custom agent performs, or how agent and model configurations compare on your tasks.
A framework for authoring tasks and rubrics, plus managed infrastructure for run environments, repeated trials, comparisons, and saved evidence. Your coding agent can create resources, launch experiments, and investigate results through the plugin, MCP, or CLI.
Obss uses a model-based judge to assess your rubric against the recorded trajectory and changed files. For work in external apps, design tasks to read back the resulting state and retain that evidence. Review the criterion-level verdicts alongside the underlying run.
Yes. Launch experiments from your coding agent before deploying a change. Attach starting repositories, sample data, and the tools you want to evaluate; Obss runs the tasks in managed cloud sandboxes.
Each run starts from the task’s pinned files and machine configuration. For external apps, prepare test accounts or separate resources so concurrent trials do not change each other’s data.
Results describe the tasks, toolsets, agent and model configurations, and starting conditions you evaluated. Repeat trials to observe variation, then inspect criterion-level evidence and ungraded runs alongside aggregate pass rates.
Give it the framework and infrastructure
to design, run, and analyze your evaluations.