How to Deploy AI in Public Sector Budgets
▼ Summary
– Government AI deployment is complicated by strict requirements for data control, operational continuity, and system verifiability, which conflict with standard private-sector assumptions.
– Public sector agencies often operate in environments with limited or no internet connectivity, preventing reliable cloud access that many private AI systems depend on.
– Infrastructure challenges, like a lack of experience in procuring and managing GPU hardware, create a significant bottleneck for running complex AI models in government.
– Large language models (LLMs) are often untenable for the public sector due to their computational demands and the security risks of offsite, centralized hosting.
– Smaller, specialized language models (SLMs) offer a practical alternative, as they can be housed locally for greater security and control while performing effectively.
The public sector faces a distinct set of hurdles when integrating artificial intelligence into its operations and budgets. Unlike private companies, government agencies must prioritize data sovereignty, operational continuity, and security above all else, often within environments that lack the robust, always-on infrastructure common in the commercial world. These unique operational challenges fundamentally alter the calculus for AI deployment.
Private sector expansion typically assumes constant cloud connectivity, centralized infrastructure, and flexible data movement. For state institutions, these conditions are often nonstarters. Agencies must maintain absolute control over sensitive information, ensure systems are verifiable and auditable, and guarantee minimal service disruption. They frequently operate in areas with limited, unreliable, or entirely absent internet access. This reality stalls many promising projects in the pilot phase. As noted by Xiao, the operating challenge of AI is frequently undervalued. Public sector AI must perform reliably on diverse data and scale without failure, where continuity is paramount. Supporting this, an Elastic survey found 65 percent of public sector leaders struggle to use data continuously in real time and at scale.
Infrastructure constraints further complicate adoption. Government bodies often lack ready access to the graphics processing units (GPUs) essential for training and running complex models. Xiao highlights this bottleneck, noting that unlike the private sector, governments are not accustomed to purchasing or managing GPU infrastructure, making access a significant barrier.
Given these nonnegotiable requirements, massive, cloud-dependent large language models (LLMs) are often impractical. A more viable path forward lies with smaller, more practical models. Specialized small language models (SLMs) use billions of parameters instead of hundreds of billions, drastically reducing computational demands. Crucially, they can be housed locally on an agency’s own servers, offering superior security, control, and the ability to function offline.
The public sector does not need ever-larger models in centralized, offsite locations. Empirical research indicates SLMs can perform as well as or better than LLMs for many targeted tasks. They enable the effective use of sensitive data while sidestepping the immense operational complexity of maintaining giant models. Xiao summarizes the contrast succinctly: using a service like ChatGPT for proofreading is simple, but running your own large language model just as smoothly in a disconnected environment is exceedingly difficult. For budget-conscious public institutions, the SLM approach provides a secure, controllable, and operationally feasible route to harnessing AI’s potential.
(Source: MIT Technology Review)
