Twice the AI from the GPUs you already own: every setting explained.
A government operator owns its GPUs and cannot simply buy more. The diagram shows one site: 80 NVIDIA H200 GPUs, exactly what fairly sized factory settings need at their busiest moment, serving chat and document search on three models. Today's traffic is 59 requests a second, with a daily peak 1.5 times the average and about one burst a day. To measure capacity, traffic was raised step by step until each set-up broke its speed promise.
01Tool by tool
Factory settings vs AlphaTango.
The same table as the diagram, with each setting explained. AlphaTango keeps every tool in place and controls only their settings, all tuned together for one goal for the whole service: the lowest cost per answer delivered within its speed promise.
| Tool | Factory settings (every tool as shipped) | AlphaTango (same tools, settings controlled by AlphaTango) |
|---|---|---|
| Front doorthe router that decides which copy of the model handles each request | Half of returning chats reach the copy that remembers themAbout half of returning turns go to the copy with the conversation's memory | Every returning chat reaches that copyEvery returning turn goes to that copy |
| Autoscaleradds and removes copies of the model as traffic changes | No minimum number of copies; waits 5 minutes to free GPUsNo minimum number of copies; waits 5 minutes before releasing GPUs | At least 8 copies; 50% more room; frees GPUs at onceNever fewer than 8 copies of the medium model; 50% more spare room; releases GPUs at once |
| Enginethe serving engine that runs the model on the GPUs | Full-size 16-bit numbers; no helper model16-bit numbers; standard work batches | Compact 8-bit numbers; a helper model drafts words8-bit numbers; smaller work batches; drafting helper on |
| Scheduler & GPU managerdecides where copies run, how many GPUs each gets and when batch work runs | As shipped | No change |
| GPUs & powerGPU power limits and clock settings | As shipped | No change |
02Where factory settings fall short
Why the factory set-up costs more.
Each tool runs with its vendor’s default settings and knows nothing about the others. Where that falls short in this case:
The models run with the 16-bit numbers they were downloaded with, which take twice the memory and work of 8-bit numbers.
The autoscaler has no minimum number of copies for the medium model, and it waits 5 minutes before releasing GPUs.
With every vendor default, the fleet kept the promise up to 1.25 to 2 times today's traffic, depending on the day. Beyond that, answers ran late.
The fleet is fixed: 80 GPUs owned, no more. At today's traffic factory settings keep the promise (averaging 38 GPUs busy through the day); the question is how much more traffic the same 80 GPUs can carry.
03Whole-Service Gate
Kept only if the whole service improved.
✓ The gate for this case
Keeping a floor of 8 copies costs 15% on its own. Without it the promise slips on 1 of 10 days, so the Whole-Service Gate kept it.
Every candidate change is tested on the AlphaTango Digital Twin, on traffic days the settings had never seen.
04Result
2× the useful work from the same 400 GPUs
averaged over the ten days; 2× even on the worst day
against 1.25 to 2 times at factory settings
for both, at the traffic each could carry
16 GPUs busy instead of 38: $2,332 → $976 a day, 58% cheaper per on-time answer
Other tests of the same idea
104% more useful work from the same 400 GPUs, about the work of 816; matching it at factory settings would take about 416 more GPUs
2.6× the users, and still holding at 10 times today's traffic on 6 days; promise at today's load 1 of 10 → 10 of 10 days
'3.4× fewer GPUs' compares GPUs switched on, averaged over the day. 'Lower cost per on-time answer' divides what the whole service costs (GPU rental plus electricity) by the work it delivered within its speed promise. Both compare AlphaTango with every tool left at the settings its vendor ships (factory settings). Results come from AlphaTango's whole-system simulator, built on its Digital Twin and run on public traffic recordings, scored on days the settings had never seen. GPU prices are rental list prices.