Microsoft’s Mu: A New On-Device Small Language Model for Windows 11
▼ Summary
– Microsoft introduced Copilot+ PCs last year, featuring a Neural Processing Unit (NPU) with over 40 TOPS and Phi-Silica, an on-device small language model (SLM).
– Microsoft recently unveiled Mu, a new on-device SLM for Windows 11, designed to power an AI agent in the Settings app for natural language queries.
– Mu is a 330M encoder-decoder model optimized for NPUs, achieving 47% lower latency and 4.7× faster decoding than similar-sized decoder-only models.
– Trained on NVIDIA A100 GPUs via Azure Machine Learning, Mu performs nearly as well as Phi-3.5-mini despite being one-tenth the size.
– The Mu-powered AI agent in Settings is now available for Windows Insiders in the Dev Channel on Copilot+ PCs via Build 26120.3964 or higher.
Microsoft has introduced Mu, a compact on-device language model designed specifically for Windows 11, marking another step forward in bringing AI capabilities directly to users’ devices. This new model powers an intelligent agent within the Settings app, allowing users to interact with their system using natural language queries. Unlike cloud-dependent solutions, Mu runs locally on Copilot+ PCs equipped with Neural Processing Units (NPUs), ensuring both speed and privacy.
The tech giant initially launched Phi-Silica as part of its Copilot+ PC initiative, but Mu represents a significant upgrade in efficiency and performance. Built as a 330M parameter encoder-decoder model, it delivers over 100 tokens per second while operating entirely on-device. Microsoft’s engineering team optimized Mu to reduce latency, achieving 47% faster first-token response times compared to similarly sized decoder-only models.
One of Mu’s standout features is its weight-sharing technique, which minimizes parameter count without sacrificing capability. Trained on NVIDIA A100 GPUs via Azure Machine Learning, the model performs nearly as well as the larger Phi-3.5-mini, despite being only a tenth of its size. This efficiency makes it ideal for handling multi-word queries within the Settings app, though traditional search methods still handle shorter inputs.
Currently available to Windows Insiders in the Dev Channel, Mu enhances the search experience by interpreting complex requests and surfacing relevant settings. Users testing Build 26120.3964 or later can explore this feature firsthand, experiencing how AI streamlines system navigation. With Mu, Microsoft continues to push the boundaries of on-device AI, ensuring faster, more intuitive interactions without relying on cloud processing.
(Source: Manual URL Processing)
