News for hackers. Stuff that matters.
Re:Windows ML is generally available by MrCoffee7 on Thursday September 25, 2025 @06:32PM (#45380098)
How does Windows ML compare to just using something like Ollama plus an LLM that you download to your device (which seems like it would be much simpler)? What are the privacy implications of using Windows ML with respect to how much of your data it is sending back to Microsoft?
Re:Windows ML is generally available by wmf on Thursday September 25, 2025 @09:13PM (#45381426)
Ollama doesn't support NPUs.
Re:Windows ML is generally available by pjmlp on Friday September 26, 2025 @02:40PM (#45389688)
Windows ML is an abstraction to use local LLM models across CPU, GPU or NPU, making the code independent of the actual hardware.

It is the evolution of DirectX for ML, previously known as DirectML.

Re:Windows ML is generally available by gloyoyo on Saturday September 27, 2025 @04:47PM (#45399195)
I think this is very neat. So many possibilities.
Re:Windows ML is generally available by catgary on Saturday September 27, 2025 @04:49PM (#45399208)
It’s kind of a bummer because this is the exact same playbook as DirectX, which ended up being a giant headache for the games industry, and now everyone is falling for it again.
Re:Windows ML is generally available by thanksgiving on Saturday September 27, 2025 @05:08PM (#45399321)
Exactly, any work you do on top of this makes your work hostage to Windows.
Re:Windows ML is generally available by sorenjan on Saturday September 27, 2025 @05:38PM (#45399544)
What's stopping you from using ONNX models on other platforms? A hardware agnostic abstraction to make it easier for consumers to actually use their inference capable hardware seems like a good idea, and exactly the kind of stuff I think an operating system should provide.

> Call the Windows ML APIs to initialize EPs [Execution Providers], and then load any ONNX model and start inferencing in just a few lines of code.

Re:Windows ML is generally available by electroglyph on Sunday September 28, 2025 @12:30AM (#45401772)
i exclusively use ONNX models across platforms for CPU inference. it's usually the fastest option on CPU. hacking on ONNX graphs is super easy, too...i make my own uint8 output ONNX embedding models
Re:Windows ML is generally available by adzm on Saturday September 27, 2025 @05:11PM (#45399334)
Yeah DirectX ended up being a giant headache but there were times in its history where it was the easiest api to use and very high performance. DirectX came about because the alternatives at the time were, frankly, awful.
Re:Windows ML is generally available by Flere-Imsaho on Saturday September 27, 2025 @06:14PM (#45399802)
OpenGL (the main competition to DirectX) really wasn't that bad in the fixed-function days. Everything fell apart when nVidia / AMD came up with their own standards for GPU programming.

DirectX was nice in that the documentation, and example/sample code was excellent.

Re:Windows ML is generally available by sorenjan on Saturday September 27, 2025 @08:11PM (#45400528)
The fixed function version of OpenGL was non thread safe with global state, it made for some super fun bugs when different libraries set different flags and then assumed they knew which state the OpenGL runtime was in the next time they tried to render something.
Re:Windows ML is generally available by kg on Saturday September 27, 2025 @06:14PM (#45399796)
I would be curious to see whether it's a common opinion that DirectX was a bad thing for the games industry. It was preceded by a patchwork of messy graphics/audio/input APIs, many of them proprietary, and when it started to gain prominence, Linux gaming was mostly a mirage.

A lot of people still choose to build games on Direct3D 11 or even 9 for convenience, and now thanks to Proton games built that way run fine on Linux and Steam Deck. Plus technologies like shadercross and mojoshader mean that those HLSL shaders are fairly portable, though that comes at the cost of a pile of weird hacks.

One good thing is that one of the console vendors now supports Vulkan, so building your game around Vulkan gives you a head start on console and means your game will run on Windows, Linux and Mac (though the last one requires some effort via something like MoltenVK) - but this is a relatively new thing. It's great to see either way, since in the past the consoles all used bespoke graphics APIs (except XBox, which used customized DirectX).

An OpenGL-based renderer would have historically been even more of an albatross when porting to consoles than DX, since (aside from some short-lived, semi-broken support on PS3) native high-performance OpenGL has never been a feature on anything other than Linux and Mac. In comparison DirectX has been native on XBox since the beginning, and that was a boon in the XBox 360 era when it was the dominant console.

IMO historically picking a graphics API has always been about tradeoffs, and realities favored DirectX until at least the end of the XBox 360 era, if not longer than that.

Re:Windows ML is generally available by pjmlp on Sunday September 28, 2025 @01:34AM (#45402008)
While Switch supports Vulkan, if you really want to take advantage of Switch hardware, NVN is the way to go, or make use of the Nintendo Vulkan extensions that are only available on the Switch.

So it isn't that portable as people think.

Re:Windows ML is generally available by Rohansi on Sunday September 28, 2025 @02:43AM (#45402235)
Anyone who thinks DirectX was bad for the games industry needs to go back and review the history of graphics APIs.
Re:Windows ML is generally available by pjmlp on Sunday September 28, 2025 @02:53AM (#45402281)
Usually it is an opinion held by folks without background in the industry.

Back in my "want to do games phase", and also during Demoscene days, going to Gamedev.net, Flipcode, IGDA forums, or attending GDCE, this was never something fellow coders complained about.

Rather how to do some cool stuff with specific hardware, or gameplay ideas, and mastering various systems was also seen as a skill.

Re:Windows ML is generally available by handbanana_ on Saturday September 27, 2025 @09:45PM (#45401038)
In the last 25 years I have never gotten this vibe from devs, DirectX likely enabled a ton of games that would have never seen the light without it.
Re:Windows ML is generally available by pjmlp on Sunday September 28, 2025 @01:31AM (#45401995)
It is a FOSS, complaining about proprietary APIs, because there is this dissonance between communities.

Game developers care about IP, how to make it go beyond games, getting a publisher deal, gameplay, the proprietary APIs is a set of plugins on a middleware engines in-house or external, and done it is.

Also there is a whole set of companies whose main business is porting games, where is where several studios got their foot into the door before coming up with their own ideas, as a means to get experience and recognition in the industry, they are thankful each platform is something else.

Finally anyone claiming Khronos APIs are portable never had the pleasure to use extensions or deal with drivers and shader compiler bugs.

Re:Windows ML is generally available by ynx on Saturday September 27, 2025 @10:32PM (#45401275)
DirectX carried the games industry forward because there weren't alternatives. OpenGL was lagging, and Vulkan didn't exist yet. I hope everyone moves to Vulkan, but DX was ultimately a net positive.
Re:Windows ML is generally available by pjmlp on Sunday September 28, 2025 @02:48AM (#45402255)
There were others, there is this urban myth that games consoles used OpenGL and only Windows was the outlier.

Even Mac OS only adopted OpenGL after the OS X reboot, before it was QuickDraw 3D, and Amiga used Warp 3D during its last days.

Re:Windows ML is generally available by pjmlp on Sunday September 28, 2025 @01:24AM (#45401976)
It is only an headache for FOSS folks, games industry embraces proprietary APIs, it isn't the elephant problem FOSS culture makes it to be, as anyone that has ever attended game development conferences, or Demoscene parties can tell.
Re:Windows ML is generally available by ta12653421 on Sunday September 28, 2025 @07:14AM (#45403485)
People mainly think of Direct3D when referring to DirectX.

The other components were very well: DirectInput etc.

Re:Windows ML is generally available by SteveJS on Saturday September 27, 2025 @09:57PM (#45401093)
It is not llm specific. A large swathe of it isn’t that much Microsoft specific either.

And it is a developer feature hidden from end users. e.g. - In your ollama example, does the developer ask end users to install ollama? Does the dev redistribute ollama and keep it updated?

The ONNX format is pretty much a boring de-facto standard for ML model exchange. It is under the linux foundation.

The ONNX Runtime is a microsoft thing, but it is an MIT licensed runtime for cross language use and cross OS/HW platform deployment of ML models in the ONNX format.

That bit needs to support everything because Microsoft itself ships software on everything.(Mac/linux/iOS/Android/Windows.

ORT — https://onnxruntime.ai

Here is the Windows ML part of this —https://learn.microsoft.com/en-us/windows/ai/new-windows-ml/...

The primary value claims for Windows ML (for a developer using it)— This eliminates the need to: Bundle execution providers for specific hardware vendors

Create separate app builds for different execution providers

Handle execution provider updates manually.

Since ‘EP’ is ultra-super-techno-jargon:

Here is what GPT-5 provides:

Intensional (what an EP is)

In ONNX Runtime, an Execution Provider (EP) is a pluggable backend that advertises which ops/kernels it can run and supplies the optimized implementations, memory allocators, and (optionally) graph rewrites for a specific target (CPU, CUDA/TensorRT, Core ML, OpenVINO, etc.). ONNX Runtime then partitions your model graph and assigns each partition to the highest-priority EP that claims it; anything unsupported falls back (by default) to the CPU EP.

Extensional (how you use them) • You pick/priority-order EPs per session; ORT maps graph pieces accordingly and falls back as needed. • Each EP has its own options (e.g., TensorRT workspace size, OpenVINO device string, QNN context cache). • Common EPs: CPU, CUDA, TensorRT (NVIDIA), DirectML (Windows), Core ML (Apple), NNAPI (Android), OpenVINO (Intel), ROCm (AMD), QNN (Qualcomm).

Re:Windows ML is generally available by solarkraft on Sunday September 28, 2025 @10:08AM (#45404436)
It's an optimized backend for running LLMs, much like CoreML on macOS, which has been received very positively due to the acceleration it enables (ollama/llama.cpp use it).

Since this uses ONNX you probably won't be able to use ollama directly with it, but conceptually you could use an app like it to run your models in a more optimized way.