AT&T cut its AI coding costs by 56% by routing routine queries to cheaper open-source models through LiteLLM technology, with only a 2% decline in performance quality. The telecom giant processes roughly 45 billion tokens daily through its internal « Ask AT&T » platform across a workforce of approximately 150,000 employees. Open-source models currently handle 40% of queries, with a target of 60-70%. Custom telecom-tuned models delivered up to 90% savings in inference costs.
Source: Read the original article
