What GitHub Means by 'Harness' in Its New Agent Glossary

GitHub published a glossary of coding-agent jargon on Sept. 2, written by Cassidy Williams, its senior director for developer advocacy. It defines seven terms in circulation among people who spend their day directing AI agents rather than typing code by hand. The post is a good map of the vocabulary. It is thinner on what the vocabulary actually buys you, which is where two outside practitioners' writing fills the gap.

What GitHub means by a coding agent 'harness'

Williams's post describes a harness as "everything surrounding [a model] that makes it useful in your workflows: the tools it can call, the permissions it operates under, the memory it keeps, the context you feed it, and the way all of that is orchestrated." GitHub Copilot, in this framing, is a harness wrapped around whichever model is doing the work underneath.

Simon Willison, whose guide to how coding agents work predates GitHub's post, reduces the same idea further: a coding agent is "a piece of software that acts as a harness for an LLM," and the loop underneath it is just "LLM + system prompt + tools in a loop." Claude Code, Cursor, Codex and Aider are all harnesses in that sense, often wrapped around the same handful of underlying models, which is Willison's point: swap the harness and the behavior changes even when the model doesn't.

The number GitHub's glossary leaves out

That claim is easy to make and hard to back up without a before-and-after. Google engineer Addy Osmani supplied one in an April 19 post, describing a team that took its agent "from Top 30 to Top 5" on the Terminal Bench 2.0 leaderboard "by changing only the harness," while running the identical Claude Opus 4.6 model throughout. Osmani credits the framing to engineer Viv Trivedy: "Agent = Model + Harness. If you're not the model, you're the harness." His summary of the lesson: "A decent model with a great harness beats a great model with a bad harness."

GitHub's Sept. 2 post never cites a comparable figure. It defines the term correctly but leaves the reader to take the payoff on faith, which is a strange choice for a post from the company that ships one of the most widely used harnesses.

Term	GitHub's definition	Where it's demonstrated
Harness	Everything around the model: tools, permissions, memory, orchestration	Terminal Bench 2.0 rank jumped from Top 30 to Top 5 on the same model (Osmani, Apr. 19)
Loop engineering	Designing repeatable systems around agents instead of manual prompting	"Tools in a loop to achieve a goal" (Willison's guide)
Ralph loop	Giving an agent a spec and letting it run until the job is done	Named for its token cost, not a specific benchmark, in GitHub's post
Hill climbing	Iteratively improving a harness against an evaluation metric	Terminal Bench 2.0 score itself is the hill being climbed

Loops, squads and fleets are about who does the waiting

Williams's post separates "loop engineering" from a "Ralph loop" by scope: loop engineering is the general practice of building scheduling, validation and escalation around an agent, while a Ralph loop is one specific pattern, where you hand the agent a detailed spec, often a product requirements document, and let it keep working unsupervised until it decides the job is finished. The post flags Ralph loops as the pattern most likely to run up a token bill, without a number attached.

"Squads" and "fleets" describe multiplying that same loop. A squad assigns agents specialized roles that mirror a human team's division of labor; a fleet runs multiple agents on the same problem in parallel. Both exist to let one person supervise more concurrent work than they could review line by line, which is the same motivation behind the scheduled Copilot runs VS Code 1.137 shipped in September: the human stops being the bottleneck the loop waits on. Model makers are building for the same pattern directly; Nex-AGI's Sept. 8 release shipped three sizes of the same architecture specifically pitched at agentic use, rather than one general-purpose model.

Hill climbing needs a hill

"Hill climbing," in Williams's definition, is the iterative process of improving an agent and its harness using evaluation metrics and feedback, adjusting one variable at a time and checking whether the score moves. It is a borrowed optimization term, and it only means something once a team has a metric to climb, whether that is Terminal Bench 2.0 or a private eval suite measuring cost per session instead of accuracy. Williams's post names the practice; it does not name a hill.

The last term, "forward deployed engineer," is the least new: a customer-facing engineer, sales engineer or solutions engineer, Williams writes, "often with an AI focus," whose job is helping a customer's team wire an AI harness into systems that already exist. It describes a role that predates agents by years, relabeled for a market where the integration work is now mostly about tool permissions and context, not connectors and APIs.