Raspberry Pi’s New AI Board Packs 8GB RAM for Gen AI Models
▼ Summary
– Raspberry Pi has launched a new $130 AI HAT+ 2 add-on board to run generative AI models locally on the Raspberry Pi 5.
– The upgraded board features 8GB of RAM and a Hailo chip, allowing it to run models like Llama 3.2 while freeing the main CPU for other tasks.
– Unlike its predecessor focused on image processing, this version can run small gen AI models and be used for training and fine-tuning.
– Testing by a tech reviewer found a standalone Raspberry Pi 5 with 8GB of RAM generally outperformed the AI HAT+ 2, partly due to power limitations.
– While the new board suits specific cases, the reviewer suggests a 16GB Pi 5 may offer more flexibility and speed for a similar cost.
The new Raspberry Pi AI HAT+ 2 is a significant upgrade for developers and hobbyists looking to run generative AI models directly on their Raspberry Pi 5 systems. Priced at $130, this add-on board represents a more powerful, albeit more expensive, successor to last year’s model. It integrates a Hailo 10H chip capable of delivering 40 TOPS of AI processing power and, crucially, includes 8GB of onboard RAM. This dedicated hardware is designed to handle AI workloads independently, freeing up the main board’s Arm CPU for other tasks. Unlike its predecessor, which was primarily focused on image processing, the AI HAT+ 2 can run small-scale generative AI models such as Llama 3.2, DeepSeek-R1-Distill, and various Qwen models. It also supports the training and fine-tuning of AI models locally.
Demonstrations from Raspberry Pi illustrate practical applications. One example shows the board powering a model that generates descriptive text from a live camera feed and can answer questions about the scene’s content, like identifying the presence of people. Another demo highlights its ability to perform text translation, such as converting French to English using the Qwen2 model.
However, independent testing has revealed some performance considerations. Tech reviewer Jeff Geerling found that a standard Raspberry Pi 5 with 8GB of RAM often outperformed the AI HAT+ 2 across the same supported models. He attributes this difference largely to power constraints; the Pi 5 can draw up to 10 watts, while the AI HAT+ 2 is limited to a 3-watt power budget. This limitation can impact the speed and efficiency of model execution.
For specific, optimized use cases, the AI HAT+ 2 holds value. Yet, as Geerling points out, the additional 8GB of RAM may not provide a decisive advantage. For many users, investing in a Raspberry Pi 5 with 16GB of RAM could be a more flexible and faster solution for running AI models, albeit without the dedicated AI accelerator. The previous AI HAT, starting at $70, remains a cost-effective option for those whose needs are centered on AI image processing.
Raspberry Pi has indicated that its software support will expand, with more and larger AI models prepared for updates expected to become available shortly after the board’s launch. This suggests the platform’s capabilities will continue to grow, offering users a broader range of local AI processing options over time.
(Source: The Verge)
