Do you really need a frontier model for everything? Over the past two years, UK public sector organisations have been working to find value and create efficiencies using generative AI. From summarising policy consultation responses to automating administrative sorting in local councils, pilot projects across central departments and local authorities have demonstrated proof of concepts.
Yet, as civil service leaders attempt to transition these pilots into permanent, department-wide operational tools, there is a challenge that can arise from scale: unpredictable unit economics.
When an AI assistant moves from a trial team of 10 policy analysts to 10,000 caseworkers processing daily citizen queries, API pricing models can transform from minor operational expenses into major budget liabilities. For accounting officers, the prospect of uncapped, usage-based cloud tokens makes long-term fiscal planning difficult, perhaps impossible in the case of citizen-facing services.
The solution to scaling AI across government isn't bigger budgets or larger models, it’s getting the architecture right. By shifting requests from defaulting to massive frontier models every time, to targeted self-hosted models running on open, enterprise-grade platforms, government departments can deliver production-scale AI with much more control over financial certainty.
To understand why AI costs escalate, it helps to look at the tasks departments will actually need AI to perform. The vast majority of daily public sector workloads involve domain-specific text analysis. This could be redacting sensitive details from Freedom of Information (FOI) responses, triaging housing benefit applications, or parsing planning documents against local policy frameworks.
These tasks require precise semantic inference, that is the ability to understand structure, context and intent within specific prompts, and then direct the API calls appropriately. They do not typically require a multi-trillion parameter frontier model capable of writing poetry or reasoning through quantum physics.
Using a general purpose frontier LLM for routine civil service workflows is the computational equivalent of taking a high-speed train to cross the street. It consumes unnecessary energy, introduces latency, and inflates token costs with every single request.
Achieving cost predictability requires a fundamental shift in how government departments handle model serving.
Rather than relying strictly on public cloud APIs with their per-token cost model, departments can adopt freely available, open-weight models, such as Llama, Gemma and Qwen. With fine tuning or RAG integration, these compact, enterprise-focused models can deliver the capabilities required for specialised tasks, but with a drastically reduced computational footprint.
By opting for local models over frontier models, departments can achieve the following benefits:
From OpEx Volatility to CapEx Stability: Instead of paying variable usage fees that spike during heavy workload cycles, departments deploy models on predictable, flat-rate infrastructure, whether on-premise, in sovereign cloud environments, or across hybrid setups.
Low Latency, High Throughput: Smaller parameter sizes mean faster inference speeds, allowing civil service workflows to process thousands of citizen records in minutes without performance degradation.
Granular Optimisation: Self-hosted models can be integrated to specific government datasets, ensuring high accuracy on niche departmental taxonomy, without the exorbitant training costs associated with giant architectures.
Financial predictability is only half the battle, risk predictability is the other. Departmental heads cannot scale a solution if it compromises data boundaries or creates compliance blind spots.
This is where open-weight models solve the AI governance bottleneck. Public sector deployment demands full transparency over data lineage, model weights, and decision auditing. Sending citizen data to opaque, third-party public cloud endpoints introduces sovereignty and regulatory risks that slow down approval pipelines.
By serving open-weight models on containerised, enterprise platforms, government IT leaders can retain control:
Data Sovereignty: Citizen data can stay within a department controlled security perimeter or sovereign cloud boundary.
Auditability and Lineage: Departments can track how a model was trained, tuned, and served, fulfilling central government transparency guidelines and ICO compliance standards.
Model Portability and Co-location: Standardised, containerised model serving unlocks portability, so departments can co-locate models where their operational data is, rather than routing sensitive workloads through external ecosystems.
As government departments prepare their operational strategies, the focus must shift from theoretical AI capabilities to pragmatic deployment.
Frontier models will always have a place in specialised research and high-level strategy. But for the core, day-to-day work of government transformation, smaller, purpose-built models should be the real workhorses. By taking control of model serving and aligning model size to task complexity, public sector leaders can scale from pilot to production in a controlled manner, delivering smarter public services without token sticker shocks and with appropriate governance.