For the last year, the industry has largely treated the model as the product.

Which model should we buy? Which benchmark did it win? How many agent skills should we load into its context? How much can it do with one enormous prompt?

Those are reasonable questions, but they miss a more useful one:

What runtime have we built around the model?

Tokens are not free. Neither is model attention. Every skill, tool description, policy document, repository summary, and repeated instruction consumes context that the model must interpret before it can do useful work. General-purpose coding agents need that flexibility, because they have to operate in almost any repository and almost any workflow. That flexibility is overhead once the workflow is known.

My Spec-Driven with Harness project puts the workflow, the guardrails, and the verification in the harness. The model still drafts, refines, and attempts the implementation. It does not have to remember the process, or police its own compliance.

The model should not be the process

The workflow is the one in Spec-First Was Always Right : a versioned requirement, executable scenarios, and a test bar.

The harness exposes 25 MCP tools when a general agent connects to it. A model-backed command gets a smaller profile: three tools to reword a requirement or generate a scenario, four to draft or to generate steps and unit tests, five for refactor and for implement advice, seven to implement or to report status. ask is the exception, twelve read-only tools. The defaults are in tool_profile.rs . Running tests, reading state, validating a spec, and enforcing the phase transition do not call a model. The command surface is documented with the project.

A skill can tell a model, “do not refactor on red.” A runtime can make the invalid transition impossible.

A small experiment, with real clocks

I drove the same workshop exercises on the same laptop, from the same commit, with the same local Ollama model, qwen3.8-flash-next:125b-mlx. One path was pi 0.85.1, connected to spec mcp serve. The other was the spec binary, 0.5.5, sequencing the workflow itself. Five phases: draft REQ-007, generate the scenario and unit test, commit and show RED, implement, then refactor and mark implemented.

Both finished with the workshop verifier at 7 of 7. The runtime took 176.7 seconds. The agent took 642.6 seconds, 3.6× on this single run. The wall-clock record keeps two costs that belong to the agent path: 145.7 seconds of scenarios rejected at review, and 38.4 seconds repairing an invalid import. Without them the agent path is 458.5 seconds, still 2.6×.

The last phase went the other way. pi finished it in 32.6 seconds, spec refactor in 59.7, because the runtime spends a model call and a test run on every round and then judges the result. On the pi path, start_refactor only flips the phase.

One laptop, one kata, and no token measurement. The graded outcome is a separate pair of full walks, re-walked on 0.5.4 and written up in the workshop validation notes : all seven requirements implemented, and 29 of 29 kata tests passing, on both paths.

You do not need a frontier model

The spec half of that argument is the August post . This run is the other half. The harness path made no cloud call.

A smaller model can still finish at the same bar. It spends the difference on iteration and validation. S* revises code against execution feedback, and with that loop a 3B model outperforms GPT-4o mini. Qwen2.5-Coder 7B Instruct with the loop outperforms the 32B Instruct model without it by 10.1% on LiveCodeBench. On math problems, where a smaller model already succeeds some of the time, Snell and colleagues found that more test-time compute can beat a model fourteen times larger (ICLR 2025 ). I have not timed a smaller model on this kata. The 125B run is the one on the clock. The harness is what makes the trade possible. A scenario that does not match the requirement is rejected, and refactor waits until the tests are green. A weaker model takes more trips through those gates. The outcome it is allowed to declare stays the same.

A skill does not hold that gate. The Agent Skills specification loads the instructions only after the agent has decided to activate the skill, and following them remains the model’s choice. On the τ-bench airline tasks, a 2025 EMNLP industry paper reports that agents struggle to follow complex company policy written as instructions. The approach they test is to compile the policy into guard code and check it before each action. S* finds the same limit in code: a model asked to judge which program is correct is a weak judge of what the program will do. Execution is what selects.

One use case, outside the delivery loop

The kata is software delivery. A test suite makes that an easy place to prove the loop. The same harness can be built wherever the outcome can be checked without asking the model: a spec, a bounded action, a check, an approval, and a record.

That harness is what you can have confidence in. The model may draft the step. The result on record is the one that passed the check. Once the harness is built, a failed match does not post, a failed limit does not pay, and a failed reconciliation does not close, for the same reason a red bar does not refactor. The confidence is that refusal.

It stops where nothing independent can fail the work. Strategy, negotiation, and diagnosis are that kind of work. I have not built or timed a harness outside this workshop. The workshop is the case with a clock on it.

The model will keep improving. It cannot release the payment. With the harness, the process can complete.