Rising AI costs and the limits of cheap models

Many businesses have offset rising AI usage costs by using inexpensive open-weight models, many of which are developed in China. However, as discussed in the latest AI Daily Brief, reports that China may restrict overseas access to its models expose this as a fragile strategy. This potential shift makes it urgent for organisations to adopt more sophisticated and resilient approaches to managing AI expenditure.

As organisations scale their use of AI, from simple chatbots to complex agentic workflows, the associated operational costs are climbing steeply. A common tactic to manage this expense has been to substitute premium, proprietary models with cheaper, high-performing open-weight alternatives.

However, a recent episode of the AI Daily Brief highlights a significant emerging risk to this approach. Many of the most popular and cost-effective open-weight models are developed by Chinese organisations. The podcast notes that China is reportedly considering placing these models under tighter export controls, potentially restricting or cutting off access for businesses in Western countries.

This development challenges the assumption that a steady stream of cheap, powerful models will always be available to keep AI operational costs down. The 'easy fix' for managing token expenditure may be less reliable than many have assumed, forcing a strategic re-evaluation for businesses that depend on it.

For any business integrating AI, this news is a critical signal about cost, risk, and strategy. Relying on a single source of low-cost models, especially one subject to geopolitical tensions, introduces significant operational vulnerability.

The strategic implications

• Budget Volatility: If access to cheap models is suddenly restricted, the cost of running existing AI workflows could multiply overnight. This creates enormous budget uncertainty and risk, particularly for businesses that have built agentic systems which can consume a high volume of tokens.

• Operational and Continuity Risk: Building core business processes on a technology stack that can be withdrawn with little notice is a major strategic misstep. It's the digital equivalent of building a factory on land you don't own and can be evicted from at any time.

• Urgency for a Mature AI Strategy: The potential disappearance of the 'cheap model' shortcut means cost optimisation is no longer a 'nice-to-have'. It must become a core competency. Businesses can no longer just chase the cheapest model; they need a multi-faceted strategy to manage performance and cost sustainably.

Pathways to cost optimisation

Instead of relying on a single cheap model, a more resilient strategy involves a portfolio of techniques. This is essential for scaling agentic AI, where different tasks require different levels of model intelligence and expense.

This isn't a reason to panic, but it is a call to act deliberately. Business leaders should treat AI cost management with the same seriousness as any other major operational expense.

1. Audit Your AI Stack and Spend: Before you can optimise, you need clarity. Identify every AI model your organisation uses, whether directly or embedded in third-party software. Map the workflows they power and, most importantly, quantify the associated token costs. Identify the top 3-5 cost drivers.

2. Diversify Your Model Portfolio: A single-provider strategy is a high-risk strategy. Begin exploring and testing a mix of models. This should include leading proprietary models like Anthropic's Claude 3.5 Sonnet or OpenAI's GPT-4o for high-value tasks, alongside high-quality Western open-weight models like Meta's Llama 3 or models from Mistral for less demanding jobs.

3. Implement a Model Router: For any organisation serious about scaling AI, a model router is becoming essential infrastructure. It acts as a smart traffic controller, sending each API call to the most cost-effective model capable of handling the task. This system alone can dramatically reduce costs without sacrificing quality.

4. Invest in Efficiency Skills: Encourage your technical teams to develop skills in prompt engineering and, where appropriate, fine-tuning. For high-volume, repeatable tasks, a fine-tuned smaller model can often outperform a large, general-purpose model at a fraction of the cost.

5. Revisit Your AI Governance: Update your organisation's AI strategy to include policies on cost management, model selection, and risk diversification. AI is no longer a fringe experiment; its financial and operational impact must be managed from the centre.