Title: Microsoft Expands AI Horizons: Nvidia GPUs Now Support Local Features on Windows 11
A New Chapter in On-Device AI
The Bigger Picture: Microsoft is reshaping its approach to on-device AI, loosening the strict barriers it had previously set around its Copilot+ PCs. An important update indicates that any Windows 11 machine with an Nvidia RTX 30-series GPU or newer, equipped with a minimum of 6GB of VRAM, can leverage local AI capabilities. While this may seem like a minor adjustment aimed at developers, it hints at a significant shift in Microsoft’s strategy regarding the integration of AI into its ecosystem.
When the groundbreaking Copilot+ PCs debuted on June 18, 2024, Microsoft made it abundantly clear that specialized AI hardware was necessary for harnessing local AI capabilities effectively. These machines were characterized by their neural processing units (NPUs), paired with essential specifications such as 16GB of RAM and solid-state drives. The heavy emphasis on NPUs was presented as critical for unlocking unique AI functionalities within Windows.
However, the narrative around NPUs overlooked the power of GPUs. Modern graphics processing units, especially the latest models, excel at parallel processing and have become essential tools for executing machine learning models. In many scenarios, they can provide higher raw performance for AI tasks than NPUs, albeit typically with greater energy consumption.
Until recently, Microsoft had tightly coupled its AI features with devices featuring NPUs, rendering many robust GPU-based systems unable to access local functionalities like text and image generation or tools like Windows Recall. This imbalance has now begun to rectify itself.
In updated documentation and a recent post on GitHub, Microsoft confirmed that developers can now run language model APIs on non-Copilot+ PCs when paired with suitable GPUs.
The company outlined the new feature succinctly: “Language Model APIs on GPU [Experimental]. These APIs can now operate on non-Copilot+ PCs equipped with supported GPUs, broadening access to local language capabilities on Windows 11 devices.” The specified hardware support includes NVIDIA GeForce RTX 30 series and newer GPUs with at least 6GB of VRAM.
At present, this capability exists within a developer-focused realm, requiring specific applications to tap into Windows’ AI framework. Though it’s still largely hidden from everyday users, this update paves the way for a more extensive range of Windows machines to harness local AI.
At the core of this enhancement is a compact on-device model named Phi Silica. Unlike being pre-installed, this model can be downloaded via Windows Update whenever an application demands it. Once activated, it operates locally on the device, utilizing the GPU where available.
Currently, the functionality largely revolves around text-based operations. Through the Windows.AI.Text APIs, applications can summarize content, rewrite text, convert text into structured formats, and generate prompts. This mirrors the capabilities you’d typically find in cloud-based AI tools but with the essential distinction of running directly on the device.
The ability to run tasks locally offers multiple benefits. It diminishes the reliance on cloud computing, ultimately promoting faster response times and better data security by keeping sensitive information on local machines instead of transmitting it to external servers. For both developers and corporate users, this can significantly influence the adoption rate of AI functionalities.
Despite these advancements, the rollout is still in its infancy. Many of the more recognizable Copilot+ features, including Windows Recall and Click to Do, remain exclusive to systems featuring NPUs. Current GPU support is, for the time being, limited to the language model API, rather than extending to the full suite of AI integrations.
Nevertheless, the overarching trend is unmistakable: Microsoft is evolving its stance on local AI. By enabling GPUs to handle these advanced workloads, the range of compatible hardware has expanded, and the previously exclusive nature of Copilot+ PCs is becoming more accessible to a broader audience.