New Models Arrive. Old Models Keep Consuming Capacity.

More than 200 internal applications were sharing a finite GPU pool at T-Tech, and the model fleet kept growing.

New generations arrived roughly quarterly, but applications running on older models did not automatically move with them. Changes in instruction following, function calling, latency, and application-specific behavior made migration risky enough that predecessors stayed online.

The eventual response had an apparent contradiction at its center: to reduce model sprawl, T-Tech built another model.

Instead of asking whether a newer model was generally better, T-Tech's researchers asked what prevented existing workloads from moving onto one common model.

They analyzed production failures, turned those weaknesses into evaluations, and post-trained a roughly 32-billion-parameter model around the behaviors applications required. Six months after rollout, it was handling 116 million requests per month and half of platform traffic.

The result was partial consolidation, but it exposed a broader production problem: model behavior had become part of the infrastructure contract.

An application can depend on far more than a model endpoint.

T-Tech reviewed 2,500 production responses and found that formatting and other instruction-following violations together represented 37.9 percent of classified failures in that sample. Problems included JSON schemas, structural requirements, length constraints, tone, and other instructions downstream workflows expected the model to obey. About 12 percent of production traffic also used tools, where researchers found problems involving tool descriptions and argument construction, particularly in Russian-language workflows.

These were operational dependencies. A model that becomes less reliable at producing required structures or tool arguments can force teams to retest prompts, parsers, and workflows before migrating.

That friction helps explain why older models stayed.

How the Mechanism Works

T-Tech turned those migration blockers into a training agenda.

Starting from Qwen3-32B with an adapted tokenizer, the researchers performed supervised fine-tuning and then tried reinforcement learning across three areas: general behavior, instruction following, and function calling.

The first attempt reproduced the conflict they were trying to solve.

Optimizing the objectives together meant improving one capability could damage another. In one 8-billion-parameter experiment, instruction-following performance improved while English function-calling performance fell from 61.2 to 54.5. Adding the general objective then pushed instruction following backward.

The researchers instead trained three separate GRPO experts and merged their weights through sequential SLERP into one checkpoint. Offline testing showed that the resulting 32B model could compete with a much larger Qwen3 candidate on T-Tech's internal evaluations, although it still trailed on some long-context and memory-heavy tasks.

In production, the model ran on 16 to 48 single-GPU FP8 vLLM pods, with reported p95 latency of 3.2 seconds.

Analysis

The important part of the experiment is what T-Tech chose to optimize for.

Production traffic became a map of migration debt. Formatting failures showed where applications depended on instruction behavior. Tool-call failures exposed structured interactions that had become compatibility requirements. Later rollbacks identified workloads the consolidated model still could not absorb.

Evaluation therefore became a way to identify why infrastructure could not be consolidated.

That changes model lifecycle management. A platform team cannot determine that a checkpoint is obsolete because another model scores better overall. It needs evidence that the replacement preserves the behaviors keeping applications attached to the older deployment.

T-Tech reports a few rollbacks for teams needing frontier-scale agentic capabilities beyond the 32B model. The remaining traffic still required other models.

Implications for Enterprises

For platform teams, model inventory is also dependency inventory.

Before retiring a checkpoint, teams need to understand which applications depend on its structured outputs, tool behavior, language handling, latency, or other characteristics. Otherwise, deprecation leaves application teams to discover incompatibilities during migration.

Production-derived test sets can turn those dependencies into retirement criteria. They can show which workloads are ready to move, where a replacement needs additional training, and where an older model still serves a necessary role.

T-Tech also reports 2.8 to 3.9 times lower per-token serving cost than the roughly seven-times-larger alternative, but does not quantify the corresponding change in fleet GPU capacity.

Risks and Open Questions

Consolidation concentrates operational risk. If many applications depend on one checkpoint, a regression can affect more of the platform, increasing the importance of staged deployment, monitoring, and rollback.

The study also leaves the degree of infrastructure consolidation unresolved. It does not report the number of models before and after the project or confirm that displaced legacy deployments were shut down.

The evidence comes from one self-hosted environment with primarily Russian-language traffic and experiments centered on the Qwen3 family, limiting how directly its quantitative results can be generalized.

Still, the production story has a clear ending. T-Tech's fleet grew because successive models could not reliably inherit all the behaviors applications had come to depend on. To reduce the sprawl created by years of adding models, it built one more, this time specifically around the reasons the others could not leave.

Further Reading

• Tsymboi et al., From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix
• T-Tech, T-Pro 2.1
• Qwen3 Technical Documentation
• vLLM Documentation