An intelligent AI routing framework combines multiple open-source language models with task-aware model selection to reduce inference costs, improve performance, and optimise enterprise AI deployments.
AT&T has introduced an AI strategy that uses intelligent model routing to direct requests to the most appropriate open-source language models instead of relying on a single large model for every task. The approach dynamically balances cost, latency, and response quality, enabling organisations to reduce AI inference costs while maintaining performance. According to the company, the routing framework has lowered AI operating costs by up to 90% in production environments.
The heart of the platform is the AI gateway that analyses the complexity of the task, processing speed, expected result quality, availability of cache availability and cost of operation before sending any request. The gateway can switch between models in case of multi-turn conversations; each part of the workflow will be managed by the most appropriate model. That way companies can use small models for regular tasks while larger models handle more complex tasks.
The company’s framework supports multiple open-source language models, including Google’s Gemma 4, OHI-4, OSS-120B, and other specialised models according to the requirements of the workload. Alongside these general purpose models, the company developed OTel 2.0 – a telecom-oriented model based on Gemma 4 and post-trained with more than 400 billion telecom-related tokens. The model is designed to improve the interpretation of telecom standards, network documentation and operational workflows. The training process was supported by Microsoft’s Managed Compute platform, AMD GPUs and telecom datasets by GSMA.
The initiative forms part of the overall ModaaS program implemented by the company, which integrates intelligent AI routing, specialised models, governance, and security within a unified management system. This solution is expected to facilitate efficient data management, computing resource optimisation, and AI deployment for organisations, both in cloud and on-premise environments.
The integration of intelligent model routing and numerous open-source AI models showcases how enterprises can enhance AI efficiency without relying exclusively on a single frontier model. This solution reflects the increasing popularity of AI solutions tailored for certain tasks, providing a balance of performance, scalability, governance, and cost.
