Token Management & Agent DesignGestão de Tokens e Design de AgentesGestión de Tokens y Diseño de Agentes

The inflection point

On June 1, 2026, GitHub Copilot retired PRUs and switched every paid SKU to AI Credits — a per-million-token meter where the model you pick decides the bill. If your team still treats Copilot as one undifferentiated commodity, your spend is about to triple silently. This playbook is what you do to keep that from happening.

What a token actually is

The model never sees your text. It operates on integers indexed into a vocabulary of 32K to 200K entries. Without this foundation every cost estimate is a guess.

A token is not a word — it is a sub-word unit discovered by BPE on a training corpus. The density varies significantly by content type:

Content type	Words per token	Why
English prose	~0.75	Common words map to single tokens
Brazilian Portuguese	~0.60	Accents and cedillas split into sub-words
Spanish	~0.65	Slightly denser than Portuguese
Python source	~0.40	Symbols, indentation, identifiers are token-heavy
TypeScript / JSX	~0.35	Generics and JSX symbols inflate further
Structured JSON	~0.30	Quotes, braces, separators dominate

Rule of thumb: a 1,000-line Python file is approximately 12,000 tokens. A 2,000-word PT-BR markdown is about 3,300 tokens. A 500-word English response is around 670 tokens.

Where the tokens go

In real agentic SDLC workloads, input dominates and output is a rounding error. A typical session distributes as: input 60%, cache read 21%, output 14%, cache write 5%.

Each phase carries a different cost and optimization lever:

• Input (parallel attention) is cheap per token but expensive in volume — Sonnet 4.6 runs at $3/M input. It grows quadratically in agent loops. The levers are caching, compaction, and lean tool definitions.
• Output (autoregressive) is 5x the input price but represents a small share — Sonnet 4.6 at $15/M output. It is strictly sequential with no parallelism. Cap it with concise prompts and structured JSON.
• Cache read (KV reuse) costs 10% of input — $0.30/M on Sonnet 4.6. It triggers when the prefix hash matches and is an architecture decision, not a flag.
• Cache write costs $3.75/M at a 5-minute TTL or $6/M at 1 hour. It is worth paying when you expect more than roughly 3 reads per write.

The agentic multiplier

A 10-turn agent loop does not cost 10x a single call. Without compaction or caching, input grows quadratically: Turn 1 is 1x baseline. Turn 5 is 15x. Turn 10 is 55x. The loop re-sends the full conversation history and tool results every turn. Caching and compaction collapse this curve back toward linear. This is why “agent for everything” architectures always blow their budget.

Model selection: the largest lever

Model is the first-order cost lever. Sonnet vs. Opus on the same task can be 5x without 5x more quality.

The Anthropic family at post-June 2026 list prices:

• Opus 4.7 — $15/M input, $75/M output. For architecture, distributed debugging, and complex tradeoff analysis. Use only when reasoning depth justifies 5x the bill of Sonnet.
• Sonnet 4.6 — $3/M input, $15/M output. The default for 80% of agentic tasks: deep code review, agent mode, multi-file work. Same quality envelope as Opus, one-fifth the cost.
• Haiku 4.5 — $1/M input, $5/M output. For classification, extraction, and low-complexity batch completions. 15x cheaper than Opus, good enough for fixed-shape tasks.

The five-level heuristic: Level 1 trivial tasks (completions, format fixes, doc lookups) go to Bundled or Haiku. Level 2 routine tasks (edit mode, single-file refactors) stay on Bundled. Level 3 default work goes to Sonnet 4.6 — this is the 80% case. Level 4 hard reasoning (architecture, distributed debugging, security review) goes to Opus 4.7 scoped to plan-only, then hands off. Level 5 long-context analysis (whole-codebase forensics, >200K context) goes to Gemini 3.1 Pro.

The most expensive anti-pattern: Opus as default. Five times the bill, less than 1.1x the quality on routine tasks.

Context engineering

More context is not better. After a threshold, every extra token adds cost without adding signal — and starts subtracting quality. The golden rule: minimum signal-rich context, never maximum context.

Context rot measured on real workloads:

• At 0x noise: quality 0.95, cost $0.10 — optimal.
• At 1x noise: quality 0.94, cost $0.20 — cost doubled, quality flat, already losing money.
• At 5x noise: quality 0.88, cost $0.60 — quality starts dropping.
• At 10x noise: quality 0.79, cost $1.10 — visible degradation.
• At 50x noise: quality 0.62, cost $5.10 — severe rot, the agent forgets the goal.

The three levers against context rot: compaction (structured compression of conversation history), tool-result clearing (drop verbose tool outputs after extraction), and retrieval-on-demand (load large context only when the agent explicitly needs it).

Prompt caching: the 90% lever

In conversational workloads, prefix caching cuts input cost by up to 90% without changing one line of prompt. The architecture requirement is strict: stable prefix, dynamic suffix, cache_control on the right blocks.

A single byte change in the prefix invalidates the whole cache. The pattern: Session 1 (cold) pays full input on the prefix plus a 25% write surcharge. From Session 2 onward, the prefix is read at 0.1x of input cost. Saving on session 2 is approximately 89% of input cost; from session 3 onward, it reaches ~90%.

Cache pays off above roughly 3 reads per write. Choose the 5-minute TTL for per-session use, the 1-hour TTL for multi-session workloads. The canonical anti-pattern: timestamps inside the cached prefix. Each request hashes differently, the cache never warms.

Agent design: workflow vs. agent

There is a hype bias — agents for everything. Workflows solve 70% of enterprise cases at a fraction of the cost.

Workflows (deterministic): LLMs and tools orchestrated by code paths. The sequence is known and stable. Cost is predictable — the same N calls per execution regardless of input. Easy to instrument, debug, and audit. Low and stable latency.

Agents (dynamic): the LLM decides which tools, in which order, based on intermediate results. The sequence varies per input — cost is a distribution, not a number. It pays for open exploration and is worth it when the problem is genuinely open-ended and tools are first-class. It demands stop criteria, budgets, and telemetry. Without them it loops forever.

The decision heuristic: if you can sketch the steps on paper, use a workflow. If you genuinely cannot, an agent earns its keep.

The five Anthropic patterns

These patterns compose. Do not invent alternatives when these cover the case.

1. Prompt chaining — decompose into ordered steps. Cheapest and most auditable. Default for SDLC subroutines.
2. Routing — a cheap classifier sends each request to the right specialist model. Massive savings on mixed traffic.
3. Parallelization — fan out independent subtasks to reduce latency. Watch the multiplier on cache writes.
4. Orchestrator-workers — plan with Opus, execute with Sonnet or Haiku. The single best premium-to-bundled bridge.
5. Evaluator-optimizer — loop with a critic, bounded iterations. Pair with a budget or it never stops.

Anti-pattern: multi-agent without coordination — N agents, no shared memory, no contracts. Pays N× and contradicts itself.

Less is more

A 700-token review prompt and a 50-token review prompt produce equivalent reviews. The verbose version — “Review this code carefully, considering security implications, performance characteristics, architecture decisions…” — pays $700/M tokens just on instructions, repeated every turn. The compact version — “Senior reviewer. Flag security, perf, correctness. JSON: {issues:[{severity, file, line, fix}]}.” — achieves a 93% reduction. In tests, review quality is equivalent or better because the model has more attention budget for the actual code.

Pair lean prompts with progressive disclosure: a small SKILL.md on every call, large checklists loaded only on demand via a load_section tool.

The June 2026 change: seven things shipping together

Four of the seven changes are silent. Two will double your bill if you do nothing:

1. PRUs retired in monthly plans — old budgets stop applying, per-call meter starts.
2. GitHub AI Credits replace PRUs — $1 of credits equals approximately $1 USD of metered usage at list price.
3. Plan base price unchanged — $10 / $19 / $39 / $99. The illusion that nothing changed.
4. Completions and NES included — code completions and Next Edit Suggestions do not consume credits.
5. Fallback experiences removed — no more silent downgrade when credits run out.
6. Code review doubles in billing — each review now bills as two interactions.
7. Pro/Pro+ annual stays in PRUs until renewal — decide path A, B, or C before that date.

Three paths for Pro/Pro+ annual holders: Path A — stay on annual PRUs until expiration (best for steady, predictable usage). Path B — convert to monthly with AI Credits (best for power users who want premium models on demand). Path C — cancel for a prorated refund and move to direct API (best for teams already running mostly on Anthropic API directly).

FinOps for AI: Inform, Optimize, Operate

Apply the FinOps Foundation phases to token spend in sequence:

Phase 1 — Inform (visibility before action): cost-per-token, per-feature, per-user dashboards; tag every call with cost-center and feature; surface the top-10 outliers daily.

Phase 2 — Optimize (pull each lever in ROI order): model routing first (largest delta), prompt caching second (silent 60–90%), compaction and lean tools third.

Phase 3 — Operate (FinOps as a habit): weekly operational report, monthly executive summary, anomaly alerts at 3σ over rolling baseline, quarterly tariff review tied to roadmap.

The savings waterfall

In sequence, the four levers compound from $1.00 to $0.18 per equivalent session, indexed against a baseline of Opus-by-default:

• Baseline (Opus-by-default): $1.00
• • Model routing (Sonnet default, Opus scoped): $0.50 — minus 50%
• • Prompt caching (stable prefix, 5-min TTL): $0.28 — minus 44%
• • Compaction (tool-result clearing): $0.22 — minus 21%
• • Lean prompts (progressive disclosure): $0.18 — minus 18%

Order matters. Each lever stacks on the previous one. Skipping model routing first leaves the largest delta unclaimed.

Applied: 50-developer org, before and after 90 days

Day 0 — $48K/month: ~70% of agent calls hit Opus even on routine tasks. Cache hit rate: 4% (prefix not stable). Top 10% of users absorb 52% of total spend. No cost-per-feature tracking, no anomaly alerts.

Day 90 — $22K/month: same throughput, more agentic features shipped. Sonnet handles 80% of agent traffic, Opus scoped to plan-only. Cache hit rate: 71% (stable prefix, cache_control, 5-min TTL). Top 10% rebalanced via routing, outliers actioned weekly. Cost-per-feature dashboard live, 3σ anomaly alerts on.

The 90-day operator checklist

• Day 0–7: inventory PRU spend per persona; tag every agent call with cost-center and feature.
• Day 8–14: set Sonnet 4.6 as the workspace default; allowlist Opus per repo with justification.
• Day 15–30: roll out cache_control on the three biggest stable blocks — system prompt, tool definitions, AGENTS.md.
• Day 31–60: compaction plus tool-result clearing in the agent loop; lean prompt rewrite of top-3 templates.
• Day 61–90: model routing in the pipeline; Plan/Execute split for Opus; anomaly alerts at 3σ over rolling baseline.

The model is one lever. The architecture is the other four.