AI coding agents: every setting explained.
A coding-agent company runs multi-turn agent sessions over large code repositories at 2026 lengths: about 25,000 tokens (word pieces) of code and context per turn, most of it repeated from the turn before. The example serves the very large DeepSeek-V3 model on NVIDIA Vera Rubin to agents, chat and live chat, with a daily peak 1.9 times the average and about 2.6 sudden bursts a day.
01Tool by tool
Factory settings vs AlphaTango.
The same table as the diagram, with each setting explained. AlphaTango keeps every tool in place and controls only their settings, all tuned together for one goal for the whole service: the lowest cost per answer delivered within its speed promise.
| Tool | Factory settings (every tool as shipped) | AlphaTango (same tools, settings controlled by AlphaTango) |
|---|---|---|
| Front doorthe router that decides which copy of the model handles each request | Half of an agent's turns reach the copy that remembers itAbout half of turns go to the copy that holds the session | Every turn reaches the copy that remembers itEvery turn goes to the copy holding that session's memory |
| Autoscaleradds and removes copies of the model as traffic changes | Plans as if every prompt is read from scratchPlans as if nothing is re-used; waits 5 minutes before releasing GPUs | Half the spare room; frees GPUs at onceHalf the usual spare room, because each copy now does more work; releases GPUs at once; batch work smoothed over an hour |
| Enginethe serving engine that runs the model on the GPUs | Forgets each session between turns; rereads it allSession memory dropped between turns | Remembers each session for 5 minutes, stored compactlySession memory kept for 5 minutes between turns and stored as 8-bit numbers, so more sessions fit; smaller work batches |
| Scheduler & GPU managerdecides where copies run, how many GPUs each gets and when batch work runs | The standard number of GPUs per copyThe standard number of GPUs per copy of the model | Twice the GPUs per copy: room for more sessionsTwice the GPUs per copy, so each copy has room to hold many sessions' memory |
| GPUs & powerGPU power limits and clock settings | As shipped | No change |
02Where factory settings fall short
Why the factory set-up costs more.
Each tool runs with its vendor’s default settings and knows nothing about the others. Where that falls short in this case:
Between turns the engine throws away a session's working memory (the computed context of the conversation). Every new turn makes the model reread all 25,000 tokens from scratch.
The front door sends a returning turn to the copy of the model that remembers it only about half the time. The other half land on a copy with no memory of the session.
The autoscaler sizes the fleet as if every prompt has to be processed in full. At 2026 agent lengths it holds about 3.5 times more capacity than needed, and it waits 5 minutes before releasing GPUs.
Even fairly sized, the fleet averaged 44 GPUs ($8,608 a day) and still missed the speed promise on 3 of the 10 unseen days.
03Whole-Service Gate
Kept only if the whole service improved.
✓ The gate for this case
Alone, each change saved little. On 2024 traffic, keeping memory longer saved 1.4% and trimming the autoscaler 14.1%; chosen together for the whole service, they saved 56.6%.
In a smaller Vera Rubin variant, putting routing and memory back to factory settings broke the promise on all 10 days, so those settings stay, even where their own saving looks small.
Every candidate change is tested on the AlphaTango Digital Twin, on traffic days the settings had never seen.
04Result
4.3× fewer GPUs: 44 down to 10
44 down to 10 on a daily average
about $6,590 a day saved at this size
against 7 of 10 at factory settings
the diagram's “cost per answer 4.3× lower”: 4.3 times the on-time answers for the same spend
Other tests of the same idea
70% cheaper; 7 GPUs down to 2; promise 5 of 10 → 10 of 10 days
69% cheaper; 33 GPUs down to 10; promise 1 of 10 → 9 of 10 days
72% cheaper; 60 GPUs down to 17; promise 8 of 10 for both
69% to 74% cheaper; promise kept on 4 of 4 days
'3.4× fewer GPUs' compares GPUs switched on, averaged over the day. 'Lower cost per on-time answer' divides what the whole service costs (GPU rental plus electricity) by the work it delivered within its speed promise. Both compare AlphaTango with every tool left at the settings its vendor ships (factory settings). Results come from AlphaTango's whole-system simulator, built on its Digital Twin and run on public traffic recordings, scored on days the settings had never seen. GPU prices are rental list prices.