Loop engineering for GitHub Copilotfrom the basics to the advanced
A didactic, hands-on briefing on the loops that turn a model into a system, the harness that equips each run, and how to apply it in VS Code and the GitHub Copilot CLI, with model routing and cost control.
AuthorPaula Silva
RoleAI-Native Software Engineer
Duration60 to 90 minutes
Date2026-07-01
Agenda
Four parts, one idea: stop being the bottleneck.
PART IFoundations, why the loop is the differentiator and where this view departs from the usual advice
PART IILoop engineering, the four stacked loops and the economics of verification
PART IIIThe harness as a control plane, context, capabilities, model routing, and cost control
PART IVGitHub Copilot in practice, the surfaces, when to use a loop, and real use cases
Part I
I
Foundations.
The model is a commodity. The system you build around it is the differentiator. That system is a loop.
The shift
The model is the engine. The loop is the car.
Raw horsepower does nothing on its own. A drivetrain, wheels, controls, and a feedback gauge turn power into motion. The model is the engine you and your competitor both rent. What converts it into shipped work is the loop you wrap around it: how work arrives, how output is checked, what it may touch, and how it improves.
Loop engineering is replacing yourself as the person who prompts the agent.
PromptThe instruction you send right now.
HarnessEverything that equips one run. It persists across prompts.
LoopThe system that chains runs: run, verify, trigger, improve.
Where this is different
Most advice stops at the model. The edge is everything around it.
Five things the typical post gets wrong, and what running this at scale actually teaches.
Agents are about better prompts→They are about better verification and cost control
Tokens are cheap→Cost per outcome swings 100x, and the failed-run tax is the real bill
AGENTS.md is tips for the AI→The harness is a governed, versioned control plane
More context and more tools, smarter agent→Subtraction wins: less context, fewer tools, lower cost, fewer errors
Ship the agent→Ship the loop that checks it and improves it
A short history that explains the present
Four eras. Each one wraps the last.
1PromptYou iterate, each turn
→
2ContextYou iterate the window
→
3HarnessYou equip one run
→
4LoopThe system iterates
The defining question of each era is who does the iterating. Loop engineering moves it from you to the system.
The first idea, the loop
You already run this loop every day. It is called driving.
Picture this
The engine roars, but nothing useful happens until someone drives: read the road, steer, glance at the mirror, and again. Driving is not one action. It is a cycle.
Part II adds what a car alone does not have: a gate that stops anything unchecked from leaving the garage.
In practice, one agent turn is the same three beats:
Reason, read the roadThe model looks at the task and the context and decides the next move.
Act, steerIt executes: an edit, a command, a tool call. Something changes in the world.
Observe, check the mirrorIt reads the result, a test output, an error, a diff, and feeds it into the next beat.
The atom · interactive
An agent is a model inside a reason, act, observe cycle.
The observe step is what makes it an agent, not a chatbot. It takes an action, reads the real result, and adjusts. Loop quality is bounded by feedback quality: give it tests and errors, and it self-corrects. The exit condition, done, is a design decision, best handed to an external check.
01Reason
02Act
03Observe
turn0idle
Yao et al. · ReAct · ICLR 2023
End of Part
Engine, car, and one loop. There, the base.
01The model is the engine both sides rent. The loop is the car that turns it into shipped work.
02The edge is not better prompts. It is better verification and better cost control.
03One loop is three beats, reason, act, observe, the same cycle as driving.
Next we stack loops on top of this one, and that is where the leverage lives.
Part II
II
Loop engineering.
The art of stacking loops. Wrap the bare agent loop in a loop that checks it, a loop that triggers it, and a loop that improves it.
Loopcraft · interactive
Build the stack. Each loop adds a capability.
System capabilityNothing enabled yet. Turn on Loop 1 to start.
Hill climbing
Events
Verification
Agent
Loop 2 · the highest leverage
Letting the model grade its own homework is the bug.
The hybrid norm: two checks that answer different questions. Tests gate, rubrics grade.
Check 01 · verifiable rewards
Did it work?
Deterministic and not negotiable: unit tests, type checks, a passing build. This is the what. Cheap, fast, and a hard gate before anything else runs.
Check 02 · LLM as a judge
Is it good?
Readability, security, efficiency. A judgment a test cannot make. At scale, an LLM judge agrees with humans about 85 percent of the time, more than two humans agree with each other.
Check 03 · the living rubric
Does it still hold?
A rubric is not written once. It is a versioned artifact, tested against real trajectories and meta-evaluated against humans. This is the bridge to the improvement loop.
Zheng et al. · MT-Bench · NeurIPS 2023Shinn et al. · Reflexion · NeurIPS 2023
The economics of verification
Verification is the cheapest insurance you will ever buy.
The three checks differ in cost by orders of magnitude, and so does the thing they prevent.
~0
A deterministic gateTests, types, build. Pennies of compute, and it runs first.
cents
An LLM judgeOne model call against a rubric, only if the gate already passed.
hours
A shipped defectA bad merge in production: incident, rollback, lost trust.
A model that costs twice as much per token but needs half the retries is cheaper per delivered result.
This is why the unit that matters is cost per successful outcome, not cost per token. Cutting token cost without a gate makes every call cheaper and still raises the cost per result.
Zheng et al. · 2023 · judge 85% vs 81% human-human
Loops 3 and 4 · the outer circuit
Run without you. Improve without you.
Both outer loops sit on one foundation most teams skip: the ability to see what the agent actually did.
Loop 3 · events
Triggered, not typed
A schedule, a webhook, an assigned issue fires a run in the background. The agent stops being a tool you hold and becomes infrastructure. Autonomy and guardrails move together.
Loop 4 · hill climbing
Reads traces, rewrites the harness
Every run emits a trace. An analysis step studies them and sharpens the instructions, the tools, or the rubric. This is the loop that compounds. It rewrites the harness, so the harness must be clean files.
Foundation · observability
You cannot improve the unseen
Capture every model and tool call as a structured span, ideally on the OpenTelemetry standard. Agents fail in ways that look like success. Only a trace tells you why. Cost and quality live in the same trace.
End of Part
Four loops, a gate before a judge, and one unit of cost.
01The stack answers four questions in order: does it run, is it right, does it run alone, does it improve.
02Never self-graded: a deterministic gate first, for pennies, then an LLM judge, which agrees with humans 85 percent of the time, above the 81 percent two humans reach.
03The unit that matters is cost per successful outcome, never cost per token.
Now we open the harness: everything that equips one run, written down in files.
Part III
III
The harness as a control plane.
Everything that equips one run, in versioned files. That is what makes the system reviewable, improvable, and governable, including its cost.
The climbing-harness analogy
The model is the climber. The harness is the gear you put on before the route.
01
Context and rulesAGENTS.md · *.instructions.md
What the agent knows by default. Enters every run.
02
On-demand knowledge.github/skills/<name>/SKILL.md
Procedures loaded only on a match. Zero idle cost.
03
Behavior*.agent.md · *.prompt.md
Roles and reusable tasks. Handoffs compose loops.
04
CapabilitiesMCP servers · toolsets · model
What it can touch, and which model runs. A cost lever.
05
Environmentcopilot-setup-steps.yml
The setup for an autonomous cloud run.
06
Guardrails and budgetsleast-privilege · CI · caps
What keeps it safe and affordable. Automate the reversible, gate the rest.
Context engineering · layer 01
95→60%
Accuracy of leading models as the input grows past a length threshold. More context can make a model dumber.
A model has a finite attention budget. The skill is subtraction.
Keepthe decision-relevant recent steps active.
Compressolder context into dense summaries.
Externalizedurable knowledge into files and memory.
Retrievethe right slice back only when needed.
Selective retention beats full history. It improves answers and cuts tokens at the same time.
Liu et al. · Lost in the Middle · TACL 2024Hong et al. · Context Rot · Chroma 2025
Capabilities · layer 04
Tools are how an agent touches the world. MCP is the standard plug.
The Model Context Protocol is the USB-C of AI: write a server once, any compliant agent can use it. GitHub Copilot is a first-class MCP client across its surfaces.
The plug
MCP, agent to tool
Open standard, host, client, server over JSON-RPC. About 97M monthly downloads by early 2026, governance under the Linux Foundation, native across GitHub Copilot, Claude, ChatGPT, and Cursor.
The coordination
A2A, agent to agent
MCP connects agents to tools. The Agent2Agent protocol connects agents to each other. Together they wire multi-agent systems: capabilities plus coordination.
The governance
Registry only, by policy
An enterprise MCP registry is the catalog. The Registry only policy blocks any server outside it at runtime, local or remote, across IDEs and GitHub Copilot CLI. Keep the set minimal: every schema costs tokens and adds a failure mode.
GitHub Docs · MCP registry and allowlist policyAnthropic · Model Context Protocol · 2024
The harness · official primitives
The harness is not a convention. It is a product surface.
Every layer of the harness is a documented GitHub Copilot primitive, not a folk pattern:
AGENTS.md is primaryThe root file is read first by GitHub Copilot CLI and the coding agent, with a documented hierarchy: personal, then repository, then organization.
Custom agents ship as filesAn .agent.md profile declares name, tools, and MCP servers, and the pull request records which agent did the work.
Skills load on demandA SKILL.md is injected only when relevant, which is context engineering as a platform feature, not a trick.
Model choice is a budgeting decision. Route the task to the cheapest model that clears it.
Pick a task
Routed toSelect a task on the left
Relative cost per run·
Routing matches task complexity to model power. Most daily work belongs on the cheapest tier. Reserve frontier models for genuine reasoning.
The official rate card spans US$ 0.60 to US$ 75 per million output tokens across tiers, a 125x chasm. A ten-person team routing eighty percent of its work to the utility tier saves on the order of fifty thousand dollars a year.
GitHub Docs · models and pricing
The bridge · loops and token efficiency
Efficiency is per iteration. The loop multiplies every lever.
68k
Tokens of pure overhead in a ten-step agent run: fifteen thousand of tool definitions, fifty thousand of tool results, three thousand of reasoning, on top of the task itself.
A chat turn costs a few thousand tokens. The same request as an agent run costs eighty to two hundred thousand, because the machine, not you, is in the loop.
Each iteration reloads the window, so every lever compounds by the number of turns:
Fewer tools, paid at every stepTool schemas are re-sent on each turn. Removing the tools the agent does not use saves on every iteration, not once.
Trimmed results and cached contextA tool that answers with two thousand tokens instead of twenty thousand, and a cached long prefix, compound across the whole run.
The root fix is Loop 2The three burn signals in a log, the same tool five times, the same file three times, reasoning in circles, share one fix: an explicit success criterion. That is the verification loop.
GitHub Copilot Token Optimization Workshop · 2026
Cost control · the real bill
The token invoice is the most visible cost and the least complete.
100–1000x
A single API call can cost a tenth of a cent. One agentic decision cycle runs ten cents to a dollar. That multiplier is invisible to a monthly invoice review.
The token bill is one of nine cost buckets. One is metered, eight are not. A forecast anchored on the invoice alone is wrong.
What the invoice hides shows up as taxes you pay on every run:
Retry taxEvery failed attempt is paid in full before the one that works.
False-success taxA run that looks done but ships a defect, paid twice: once to run, once to fix.
Context-bloat taxTokens re-sending history that lowers accuracy while it raises the bill.
No-cache taxRe-reading the same long context uncached, run after run.
Wrong-tier taxA frontier model doing utility work, paying reasoning rates for autocomplete.
FinOps Foundation · State of FinOps 2026EY · token economics · 2026
Cost control · the official shift
June 1, 2026: the billing model caught up with the loop.
06/2026
GitHub Copilot retired premium request units and moved to usage-based billing: every interaction is now priced by model and by tokens, input, output, and cached, at published per-model rates.
Everything this deck argued about token-level economics stopped being a framing and became the official pricing physics of the platform.
What the official model gives you, out of the box:
One meter: AI CreditsEvery plan includes monthly credits, one credit equals one cent, and Business and Enterprise credits pool across the billing entity.
Budgets down to the userEnterprise, organization, cost center, and user budgets, with alerts and hard caps: the enforcement plane is a product setting.
Cached input at a tenthThe official rate card lists cached input at a tenth of fresh input on flagship models, so prompt caching becomes a verifiable line item.
Completions stay off the meterCode completions and next edit suggestions remain unlimited on paid plans, and the silent cheap-model fallback is retired: budgets govern now.
GitHub Blog · usage-based billing · 2026GitHub Docs · models and pricing
Make cost a property of the harness, not a per-developer decision. Three planes, all declared in files and enforced in CI.
Policy · what is allowed
Model allowlist and a default cheap tier
Token and turn budgets per scope
Tool scope and a caching mandate
Org policy · AGENTS.md · model configObservability · what happened
Cost per successful outcome, not per token
Attribution by team, repo, and task type
Every model and tool call as a trace
OpenTelemetry · usage dashboardsEnforcement · the hard stops
Required CI gates before merge
Retry and iteration caps, budget ceilings
A kill switch for runaway loops
Required checks · budgets · max-turns
Gartner · agentic AI ROI · 2025FinOps X · Tokenomics · 2026
Cost control · interactive
Optimize cost per successful outcome, not cost per token.
Model tier
First-pass success rate70%
Cost per delivered task
·relative units
Pick a tier and a success rate. Watch how retries and escaped defects, not the sticker price, drive the real cost.
End of Part
Gear in files, behavior governed, the gate decides.
01The harness is the climber's gear, and every layer is an official file: AGENTS.md, .agent.md profiles, SKILL.md.
02Cloud FinOps governs capacity. Agentic FinOps governs behavior, with budgets down to the user.
03Overhead compounds per iteration, so every lever multiplies, and the gate, not the tier, sets the cost per outcome.
Time to land all of it where you work: the four surfaces of GitHub Copilot.
Part IV
IV
GitHub Copilot in practice.
The same markdown harness works on four surfaces, from VS Code to the GitHub Copilot CLI to the cloud. Here is where the loops live.
The four agent surfaces
One harness, four places to run it. All managed from the Chat view.
Local agentInside VS Code
Interactive partner: Agent, Plan, Ask. Full tools and MCP. Stops when VS Code stops.
Background agentGitHub Copilot CLI
Autonomous on your machine, parallel sessions. Survives restarts and can delegate to the cloud.
Cloud agentOn GitHub infrastructure
Assign it an issue, it works on a branch and opens a pull request while you are away.
Sub-agentsInside a parent session
Isolated subtasks with their own context window, keeping the main window clean.
Surface one, the IDE
VS Code ships the three brakes as settings.
The brakes from the token workshop are not advice anymore. In VS Code they are settings, and an organization can manage them:
Step limit: chat.agent.maxRequestsA ceiling of requests per agent session. The default is generous; lower it for tasks you already know the shape of.
Fewer tools: tool setsCurated groups of tools in a .jsonc file, picked per request, so unused schemas stop taxing every iteration.
Guarded autonomy: permission levelsDefault approvals, bypass, or Autopilot, plus terminal allow and deny lists and OS-level sandboxing of the agent.
Loop 2, as a product settingAdvanced autopilot puts a small, fast model after each turn to judge whether the task is complete: a verification loop in the box.
Autopilot consumes AI credits exactly like interactive chat, and chat.agent.enabled is managed at the organization level: autonomy is a governed dial, not a default.
VS Code Docs · agent approvals and autopilot
Surface two, the terminal
The CLI makes the loop scriptable, and the spend visible.
github copilot cli · the loop as a pipeline step
copilot -p "/review the changes on this branch" \
-s --allow-tool 'shell(git:*)' --deny-tool 'shell(git push)'
# headless: scripts, cron, CI steps
/usage # session tokens and AI Credits
/fleet # parallel subagents, one converged result
/init # scaffold copilot-instructions.md
A budget that asks, per sessionSet a credit limit on the session and the CLI stops to ask before spending past it.
Context that trims itselfHistory is compressed automatically as the window nears its limit, and supported models offer an extended one million token window when you choose it.
The model, decided in orderCustom agent model first, then the --model flag, then the config file, then the default: routing becomes configuration, not habit.
Everything above runs headless with -p, which is what turns a loop into a pipeline step. OpenTelemetry export adds tokens per call to your own dashboards.
GitHub Docs · GitHub Copilot CLI
Surface three, the pipeline
In CI, the loop runs with guardrails you can audit.
The coding agent, on Actions
An ephemeral Actions environment. It pushes only to copilot branches, and its pull requests wait for human approval before any CI workflow runs. It bills Actions minutes plus AI Credits.
Agentic Workflows, in Markdown
Automation goals written in plain Markdown, compiled by gh aw into standard Actions workflows: continuous triage, documentation, test improvement, CI-failure investigation.
Read-only by default, writes gated
Agents run behind a firewall with safe outputs: only approved action types reach GitHub, and a threat-detection job scans what they propose.
Cost control starts with visibility: gh aw logs and gh aw audit show which runs consume the most time, tokens, and AI Credits.
Loop 3, EventsBackground CLI, handoff to cloud*.agent.md · cloud triggersPartial
Loop 4, Hill climbingEdit harness files, Agentic MemoryAGENTS.md · SKILL.mdManual
Judgment · when to reach for a loop
Not everything should be an agent. Match the tool to the stakes.
Low stakes, low frequencyJust chat. A single prompt is enough.
Repeatable, fully verifiableA deterministic workflow. Do not pay a model to do what a script does.
Open-ended, low cost of errorA single agent loop. Useful even if it sometimes misses.
Open-ended, high cost of errorA verified, observable loop. The whole stack earns its keep here.
The question is never can an agent do this. It is whether the cost of error, the frequency, and the verifiability justify the loop.
Agentic ROI fails far more often on governance and unclear value than on model capability. Choosing not to agentize is a senior move.
Anthropic · Building Effective Agents · 2024
Scenarios · real use cases
Four loops you can propose to a team on Monday.
Loops 1, 2, 3FinOps cost agent
Watches cost per successful output, checks each reading against budget and policy, runs on a schedule or when spend crosses a threshold.
Outcome: spend attributed by team, drift caught in minutesOfficial: usage metrics API · budgets · GitHub Copilot CLILoops 1, 2Pull request reviewer
A custom agent grades each pull request against a versioned rubric and the tests, posts findings, and blocks merge until CI is green.
Outcome: faster review, the hybrid norm made routineOfficial: GitHub Copilot code review · reviewer .agent.md · required checksLoops 1, 3Docs and onboarding agent
Triggered when code changes, it updates the docs and the AGENTS.md, opens a pull request, and keeps the onboarding path current.
Outcome: knowledge that maintains itselfOfficial: coding agent on issues · AGENTS.md hierarchyLoops 1, 2, 4Legacy modernization agent
Migrates a module behind tests, verifies against the suite, and its rubric is refined from traces across each migration.
Outcome: the system improves as the codebase movesOfficial: custom agent · agent skills · CI gates
Applicability · the official meter
The meter is official. The FinOps agent reads an API, not a spreadsheet.
copilot-usage-meter · REST
GET /enterprises/{e}/copilot/metrics/
reports/enterprise-28-day/latest
GET .../reports/users-1-day?day=DAY
GET .../reports/user-teams-1-day?day=DAY
# NDJSON reports, daily and 28-day windows# covers IDE and GitHub Copilot CLI activity# gated by enterprise policy + fine-grained permission
This is the read side of the cost agent from the next slide, and it is all documented surface:
Enterprise, org, and user levelExportable usage reports at every scope, refreshed daily, one year of history.
Team-level attributionJoin the user-teams report with per-user usage and you get cost and adoption by team, the attribution the control plane asked for.
Drift becomes a queryUsage reports surface the heavy consumers, so the budget-drift alert is one scheduled comparison, not a hunt.
# Loop 1, the agentDefault model: a cheap tier for daily workFrontier models only for hard reasoning# Loop 2, verificationRun the test suite, fix failures before finishingA reviewer agent grades against the rubric# Cost controlKeep this file short, load skills on demandCap turns and retries, cache long context