GitHub Puts Numbers Behind Its New AI Coding Vocabulary

What GitHub means by "harness," "loop" and "squad"

GitHub published a glossary on Sept. 2 for the vocabulary that has built up around AI coding agents, written by Cassidy Williams, the company's senior director for developer advocacy. Her argument: engineers now say "harness" and "hill climbing" constantly and define them rarely.

A harness, Williams wrote, is "everything surrounding [a model] that makes it useful in your workflows... the tools, permissions, memory, context, orchestration." GitHub Copilot, in her framing, is "a good example of a software harness" because it "connects models to codebases, editors, pull requests, terminals, and so on."

The rest of the glossary follows the same pattern. "Loop engineering" is "the practice of designing repeatable systems around agents, instead of manually prompting them for one task at a time." A "Ralph loop" is the specific case where "you give an agent a detailed task, often from a product requirements document or spec, and have it keep working until the job is done." A "squad" is "a group of agents with different roles" modeled on "a real-world team," while a "fleet" is "parallel agents working on tasks at the same time." "Hill climbing" is Williams's word for the ongoing work of improving an agent and its harness once both are already running.

What hill climbing cost, in GitHub's own numbers

Williams did not have to reach for a hypothetical to illustrate that last term. GitHub published a second post the same day, by staff software engineer Erik Kristensen and software engineer Napalys Klicius, that reads like hill climbing done in a spreadsheet. The pair had found that shortening individual tool outputs to save tokens was backfiring: an agent that could not see enough of a file or a log made extra recovery calls, which cost more than the compression saved. Their fix was to optimize, in their words, "the completed task, not the tool call."

1. Selective output compaction5.5 %
2. Remove view prefixes3.1 %
3. Compact task-tool prompt2.9 %
4. Reduce notification roundtrips2.3 %

Source: GitHub Engineering blog, accessed 2026-09-11

Two of those four changes get more detail in the post. Removing line-number prefixes from file reads, which Kristensen and Klicius said current coding agents no longer parse, cut model-inference cost by about 5 percent in offline benchmarks and by 3 percent in average daily cost per active user once it shipped. Compressing the task-tool prompt by roughly half, while holding agent behavior steady through targeted tests, removed about 1,300 tokens per turn. None of the four changes touched the underlying model.

What a harness has to guarantee, from the person who leads one

GitHub's post treats the harness as plumbing. A post from a different company, the same week, treats it as the thing standing between an agent and a production incident. Boris Cherny, Anthropic's head of Claude Code, wrote on X on Sept. 11 that "production code written by Claude should have a higher bar than if it was written by a human," and that Anthropic enforces this with "lots of lint rules, lots of tests, Claude-driven end to end tests, Claude-powered fuzzers running daily, automated code reviews and security reviews, automated code refactoring, and so on." Without those guardrails, he wrote, "you can end up with a mess that is hard to maintain down the line."

That list is, term for term, a harness in Williams's sense: tools, permissions and orchestration wrapped around a model so its output is usable. Cherny's post did not use GitHub's vocabulary. It described the same object anyway, which is closer to what "hill climbing" looks like when the hill is a codebase other people depend on, not a benchmark score.

Where the vocabulary still splits

GitHub is not the only company that spent 2026 trying to pin these words down, and the attempts do not agree. A Hugging Face glossary published May 25 by researchers Sergio Paniego and Aritra Roy Gosthipaty splits GitHub's single "harness" into two layers: a "harness," which is strictly "the execution layer inside the agent" that "calls the model, handles its tool calls, decides when to stop," and a separate "scaffolding," the "behavior-defining layer around the model: system prompt, tool descriptions, how the model's responses get parsed." Databricks, in its own glossary entry, defines a harness as "the software infrastructure that wraps around a large language model... and enables it to act on tasks, not just respond to prompts" — closer to GitHub's version, but framed around the model's shift from answering to acting rather than around the tools it uses to do so.

GitHub's post does not mention that Hugging Face already split its umbrella term in two, or that Databricks anchors the same word to a different distinction. Three technology companies published three glossaries for the same handful of terms inside four months, and a reader who took only one of them home would use "harness" to mean a different piece of software depending on which post they'd read.

Software coverage of AI coding tools keeps running into this: the GitHub Copilot cost-cutting work covered here in August and GitHub's own multi-model orchestration project, HydraFusion, both describe harnesses without ever using the word. The vocabulary GitHub published on Sept. 2 is an attempt to stop that, inside one company at least. Whether Hugging Face's split between harness and scaffolding or Databricks' action-versus-response framing wins out, or whether the industry ends up with three incompatible dictionaries, is not something any of the three posts addresses.