Compact AI Models Drive Life-Saving Advances

One morning in 2019, Adebayo Alonge stood in a Cape Town hotel room, ready to demonstrate his startup’s AI-powered solution to a deadly crisis in African health care: counterfeit medication, which claims thousands of lives across the continent each year.

The RxScanner, a handheld spectrometer, uses infrared light to scan a pill and transmits its molecular profile to an AI model backed by a pharmaceutical database. Within seconds, the AI identifies the medication or flags it as fake. Pharmacies in over a dozen countries, including Ghana, Kenya, Myanmar, and Alonge’s native Nigeria, were already using the system. But that morning in South Africa, it failed. “I was shocked,” Alonge recalls.

The spectrometer connected to the AI model, but the data center sat 14,000 kilometers away, and bandwidth was severely limited. “Our server was in the United States, and just to get the result of a single scan was taking me over 5 minutes.” So Alonge immediately instructed his engineers to shrink the AI model into a smaller, low-power, offline version that could run entirely on his Android phone. They delivered it in just two hours, saving the demo.

More importantly, that work gave rise to a new version of the device capable of authenticating a pill in areas without broadband, computers, or even reliable electricity. It also turned Alonge into a passionate advocate for what he calls “small AI.”

Small AI for Global Health Care Access

Small AI stands in stark contrast to the colossal large language models (LLMs), hyperscale data centers, and multibillion-dollar investments dominating wealthy nations. Yet for millions worldwide, small AI is the only kind that matters,and often the only kind available. According to a World Bank report issued in November, just 0.7 percent of internet users in the world’s poorest countries have used ChatGPT, compared to a quarter of all internet users in the most developed nations.

“Most people are discussing AI from the LLM/generative side. But that needs a lot of computing power, electricity, massive data, and skilled people to manage it,” said Ajay Banga, president of the World Bank, at the World Economic Forum in Davos last January. “Outside the developed world, other than maybe India and China, very few countries have that combination.”

By contrast, Banga noted, small AI can deliver useful, even life-saving services to people in areas lacking those resources. In India, where the government’s AI strategy emphasizes small AI development, systems are already helping farmers. For instance, a drone-based system developed by Bala Murugan and colleagues at the Vellore Institute of Technology captures images of cashew plants and quickly identifies those with disease-indicating splotches. All processing occurs on the drone itself, eliminating the need for an on-site computer or connection to a central server.

Other small-AI implementations, using small language models trained for specific tasks and often running on cheap, low-power devices, have been deployed to identify ant infestations in a Uruguayan vineyard, detect malaria-carrying mosquitoes in multiple countries, and run electrocardiograms from an Arduino device in parts of Brazil lacking access to complex equipment.

“This is the most important area in AI nowadays,” says Marcelo José Rovai, a professor at the Institute of Engineering and Information Systems at the Federal University of Itajubá in Brazil, who contributed to all three projects. “It’s growing very fast.”

Low-Power, Small-AI Models on Devices

Small AI models can run on various low-power devices, including an Arduino Nano 33 BLE Sense, a Seeed Wio Terminal, and an Arduino Portenta. For Alonge, Rovai, and other advocates, small AI is not merely “a promising trend,” as the World Bank report describes it. It may, in the long run, become the form of AI that touches the most lives and remains sustainable after some giant models grow too costly for most users.

“I think the future of AI is not like one giant model, at a center. I think it’s millions of small, precise models deployed at the edge, each one solving a specific problem, a specific context,” Alonge says. This is partly because much of humanity,including people in parts of rich countries and the developing world,lives without access to cutting-edge frontier models. But, he adds, it’s also because those models are not sustainable.

“If someone is not subsidizing it, most people will not be able to afford those models. So those of us who are said to be small-AI developers are the ones who will have to build for the majority of the world,” Alonge says.

There is no strict definition of “small AI,” but the term often refers to language models with at most a few billion parameters. Compare that to cutting-edge models, which can exceed a trillion. That’s small enough to run directly on a phone or a Raspberry Pi, enabling applications to operate without a connection to a data center and using only a few watts of power, often supplied by a battery or solar panel.

Despite their small footprint, these models are not fundamentally different from gigantic AI models, Rovai explains. Many small language models were created the same way the phone-based version of Alonge’s scanner was,by “pruning” large models, removing parameters not involved in the task. The result is a system that is less capable generally but still highly effective for its specific purpose, Rovai says.

Other small models are created through “distillation,” where they are trained to mimic a large model until their performance approaches that of their “teacher,” Rovai notes. In some cases, a larger model’s precision is reduced, for example, so that a model run on 32-bit architecture can work on 8-bit designs. For machine learning applications used to classify data or predict patterns (like an ant infestation), the model is trained from the start on a small device, not derived from a larger model.

Running these small, specialized systems is becoming easier, Rovai says, for two reasons. First, hardware is improving,becoming more capable while using less power. This means more phones, especially those equipped with neural processing units for AI tasks like facial recognition and photo adjustments, can run small AI. In 2025, slightly more than a third of all smartphones shipped worldwide were capable of running generative AI, and that figure is expected to reach 45 percent by the end of this year, according to Counterpoint, a technology research firm. By the end of next year, slightly more than half of all smartphones will be able to run a small AI model.

Second, the footprint of language models is shrinking. Both Google DeepMind’s Gemma 4 (released in April) and Alibaba’s Qwen 3.5 are “fantastic” for small AI, Rovai says. Both are “open weight,” meaning users can adjust connections between parameters to suit their needs. This makes it easy, for example, “to take a lot of data from, say, the milk industry and retrain the model specifically on that,” Rovai says.

Rovai demonstrated these reasons during a Zoom call, holding up one of his latest experiments. “This is the new Arduino UNO Q,a $50 device with a Qualcomm chipset. I’m running a language model here, which collects data from sensors and analyzes that data to detect tiny pools of water where mosquitoes might be breeding. It takes 3 watts to run it.”

Support for Small-AI Development

Convinced that millions of people are already benefiting from such applications, the World Bank now actively promotes small AI through grants, mentorship programs, financing, technical advice, and models of government policies that support small-AI development. In Rwanda, for example, the World Bank is backing a government program to help low-income households acquire devices capable of running AI.

That said, no one claims large language models will disappear. Creating generative AI that can run on a phone or small device requires the architectural insights, data processing, and results of a larger model, Rovai says. “We need the big models to create these smaller models.”

And while small AI can benefit people without access to big AI, it cannot solve the larger problems of development and digital inequality, Alonge says. Implementing small AI will not allow nations to escape the challenge of building an ecosystem to support AI: reliable power, a functioning supply chain, and an educational system that develops the talents needed to create AI tools.

Though his drug-scanning system can run for days on an unconnected phone, “you still want to be able to enable periodic syncing for updates with new signatures for the medications and analytics,” Alonge says. “And even when you are using batteries, reliable power is important. That phone battery is not going to last forever.”

In many parts of the world, the future of small AI is not assured, he says. “It works, and many places will eventually need to use it. The question is whether or not the political actors are wise enough to invest in infrastructure to support it long term.”