Fixture-verified evaluation kit — reviewed September 9, 2026

Test whether a multi-agent workflow beats one agent

Read the Agency Agents source review: →

For teams considering extra agent roles for a repeatable software or knowledge-work job.

Stack

Agency Agents: Multi-Agent Systems Architect → Agency Agents: Experiment Tracker → One-agent baseline → Fixed fixture and verifier → Blind human review

Run this recipe

Work through it here

Progress stays in this browser. The downloaded Markdown kit works in any notes app or repository.

0/6 complete

Freeze one resettable task and predeclare correctness, cost, latency, intervention, safety, and privacy thresholds before running anything.
Run a one-agent baseline with the model, budget, tools, permissions, instructions, and fixture recorded.
Use the Multi-Agent Systems Architect to select no more than three roles and define each role's inputs, outputs, permissions, fallback, one human gate, and a two-retry cap.
Run the multi-agent workflow on an identical fresh fixture, passing only agreed artifacts and summaries between roles instead of accumulated chat transcripts.
Apply the same deterministic verifier and blind human rubric to both arms; compare correctness, regressions, elapsed time, tokens or cost, and human intervention.
Keep orchestration only when it clears the predeclared threshold; otherwise retain the one-agent workflow and publish the failed or stopped result.

Is this recipe useful?

Procedure

Freeze one resettable task and predeclare correctness, cost, latency, intervention, safety, and privacy thresholds before running anything.
Run a one-agent baseline with the model, budget, tools, permissions, instructions, and fixture recorded.
Use the Multi-Agent Systems Architect to select no more than three roles and define each role's inputs, outputs, permissions, fallback, one human gate, and a two-retry cap.
Run the multi-agent workflow on an identical fresh fixture, passing only agreed artifacts and summaries between roles instead of accumulated chat transcripts.
Apply the same deterministic verifier and blind human rubric to both arms; compare correctness, regressions, elapsed time, tokens or cost, and human intervention.
Keep orchestration only when it clears the predeclared threshold; otherwise retain the one-agent workflow and publish the failed or stopped result.

Acceptance artifact

A frozen fixture, two pinned run manifests, raw artifacts, comparable verifier results, blind scores, and a keep-or-stop decision

Do not use it blindly

One comparison cannot establish a universal winner; do not expose production secrets or irreversible operations, and do not add roles whose outputs have no acceptance check or consumer.

Work through it here

Procedure

Acceptance artifact

Do not use it blindly

Evidence and setup