News for hackers. Stuff that matters.
Re:Microsoft BitNet: inference framework for 1-bit LLMs by not_a_bot_4sho on Friday October 18, 2024 @12:10PM (#41880730)
WSL is where it's at today. It's not quite what you're asking for, as it is a separate virtual OS, but the integration is so tight that it feels like you're using your favorite shell natively in Windows.
Re:Microsoft BitNet: inference framework for 1-bit LLMs by diggan on Friday October 18, 2024 @12:18PM (#41880840)
> integration is so tight that it feels like you're using your favorite shell natively in Windows

WSL1 certainly felt that way, WSL2 just feels like any other virtualization manager and basically works the same. Not sure why people sings the praise of WSL2, I gave it a serious try for months but there is a seemingly endless list of compatibility issues which I never had with VMWare or VirtualBox, so I just went back to those instead and the experience is the same more or less.

Re:Microsoft BitNet: inference framework for 1-bit LLMs by throwaway314155 on Friday October 18, 2024 @12:33PM (#41880994)
Probably because it has relatively painless GPU sharing with pass through. As far as I know that sort of feature requires a hypervisor-level VM, which is not something you get with VirtualBox.
Re:Microsoft BitNet: inference framework for 1-bit LLMs by diggan on Friday October 18, 2024 @12:43PM (#41881111)
Someone correct me if I'm wrong, but I think you can use a KVM or QEMU backend for VirtualBox and that way get GPU pass-through. Probably not out of the box though.
Re:Microsoft BitNet: inference framework for 1-bit LLMs by Datagenerator on Friday October 18, 2024 @05:31PM (#41883686)
Close Windows, many doors open
Re:Microsoft BitNet: inference framework for 1-bit LLMs by EgoIncarnate on Friday October 18, 2024 @09:23PM (#41884904)
The WSL2 GPU passthrough is more like a virtual GPU than KVM style device passthrough. I believe it's effectively a device specific linux userland driver to device specific windows kernel driver with a linux kernel shim bridging the too. If I recall correctly, the linux userland drivers are actually provided by the windows driver.