Two measured comparisons · September 3, 2026
The same agent add-on made Claude faster and Codex smaller.
Twelve blinded runs passed the same tiny repository task. Ponytail helped different dimensions in Claude Code and Codex—which is why “best add-on” claims need a harness, model, fixture, and verifier attached.
The measured medians
| Harness | Result | Time | Output tokens | Code lines |
|---|---|---|---|---|
| Claude Code 2.1.259 · Opus 5 | 3/3 each arm | 38.425s → 27.231s | 2,698 → 1,844 | 7 → 7 |
| Codex CLI 0.152.1 · requested GPT-5.6-sol | 3/3 each arm | 57.055s → 68.281s | 1,420 → 1,632 | 7 → 3 |
Arrows run from baseline to Ponytail 4.8.4. Every artifact scored 10/10 and changed only the target file. Claude reported list-cost estimates; Codex did not, so no cross-harness dollar comparison is made.
What stayed fixed
Why did the tradeoff flip?
The measurements establish the difference, not its cause. Plausible hypotheses are that each harness has a different control loop and built-in instruction stack, that loading a skill changes context and tool behavior differently, and that this task is already close to its natural minimum. A larger suite would need to vary task size and failure mode before testing those explanations.
The useful decision is therefore conditional: on this fixture, Ponytail was an efficiency win in Claude Code and a code-size win with an execution cost in Codex. Neither result transfers automatically to another repository, model, effort level, or add-on version.
A better add-on test
- Pick a repeated job and freeze the input plus a checkable acceptance artifact.
- Pin the harness, requested model, effort, permissions, tools, and add-on version.
- Run fresh workspaces in randomized order and repeat each arm.
- Score correctness and regressions before revealing labels.
- Keep time, tokens, cost, intervention, and output size separate; do not collapse them into one “best” score.
Failures stay in the record
The Claude setup had two pre-model failures before the measured batch. The first Codex batch failed closed in all six attempts because its separate code-mode helper was not mounted. Those are excluded from model comparisons but published because setup reliability is part of choosing a workflow.
Get the next measured recipe
One maintained workflow at a time. Explicit opt-in, no vendor-paid ranking, unsubscribe any time.