News for hackers. Stuff that matters.
Re:GPU Video acceleration in the Windows Subsystem for Linux... by jameshilliard on Monday February 13, 2023 @07:26PM (#34783211)
This is Microsoft's new EEE strategy, except that the new "Extinguish" target isn't all Linux systems but rather just any non-WSL based Linux systems. Microsoft offers WSL based Linux on Azure and they want developers to write software that's locked into WSL based Linux.

Microsoft isn't even hiding this at all as they even list it as a specific goal of the directx related changes being pushed to mesa: https://devblogs.microsoft.com/directx/in-the-works-opencl-a...

> Make it easier for developers to port their apps to D3D12. For developers looking to move from older OpenCL and OpenGL API versions to D3D12

Re:GPU Video acceleration in the Windows Subsystem for Linux... by naikrovek on Monday February 13, 2023 @07:44PM (#34783479)
> Microsoft offers WSL based Linux on Azure

No they don't, not if you're not using a Windows desktop in Azure. They have their own Linux distro for embedded stuff, extremely low power stuff, but that's very far from anything WSL2 related.

How would they even offer a WSL-based Linux on Azure without Windows?

Re:GPU Video acceleration in the Windows Subsystem for Linux... by jameshilliard on Monday February 13, 2023 @11:09PM (#34785170)
Via a Windows WSL server on Azure, direct WSL based Linux VM's on Azure should be possible as well although it doesn't look to have been implemented yet:

https://learn.microsoft.com/en-us/windows/wsl/install-on-ser...

Re:GPU Video acceleration in the Windows Subsystem for Linux... by naikrovek on Tuesday February 14, 2023 @12:02AM (#34785518)
that's installing WSL on Windows server; that's a Windows Server feature, not an Azure thing. and, that's a long way from offering standalone WSL VMs.

standalone WSL 2 doesn't even make sense. it isn't WSL without the "W": Windows.

Re:GPU Video acceleration in the Windows Subsystem for Linux... by jameshilliard on Tuesday February 14, 2023 @12:22AM (#34785695)
WSL2 and Azure are both Hyper-V based, offering standalone WSL VMs should be fairly straightforward from my understanding due to this.

> The Azure hypervisor system is based on Windows Hyper-V.

https://learn.microsoft.com/en-us/azure/security/fundamental...

> The newest version of WSL uses Hyper-V architecture to enable its virtualization.

https://learn.microsoft.com/en-us/windows/wsl/faq

Re:GPU Video acceleration in the Windows Subsystem for Linux... by Dylan16807 on Tuesday February 14, 2023 @01:30AM (#34786155)
But again, if you run the WSL2 kernel in a standalone VM then it's just Linux. All the "Extend" goes away.
Re:GPU Video acceleration in the Windows Subsystem for Linux... by naikrovek on Tuesday February 14, 2023 @09:10AM (#34789467)
yeah exactly, it doesn't make sense.
Re:GPU Video acceleration in the Windows Subsystem for Linux... by jameshilliard on Tuesday February 14, 2023 @03:03PM (#34795146)
To clarify it's the functionality which is provided using the dxgkrnl(which has both Linux kernel and WSL2/Hyper-V hypervisor components that are required to function) passthrough that makes the "Extend" functionality not work in a normal Linux VM with say a KVM based hypervisor.

The issue is that the WSL2 Linux userspace(which is basically a port of a number of windows userspace components to WSL2 Linux) has closed source libraries like libd3d12.so and libdirectml.so which have a hard requirement on dxgkrnl and a Hyper-V hypervisor/Windows based VM host. If Linux applications are developed targeting these closed source libraries they will not function on normal non-WSL2/Hyper-V VM's.

See the diagrams/details here: https://devblogs.microsoft.com/directx/directx-heart-linux/

> Over the last few Windows releases, we have been busy developing client GPU virtualization technology. This technology is integrated into WDDM (Windows Display Driver Model) and all WDDMv2.5 or later drivers have native support for GPU virtualization. This technology is referred to as WDDM GPU Paravirtualization, or GPU-PV for short. GPU-PV is now a foundational part of Windows and is used in scenarios like Windows Defender Application Guard, the Windows Sandbox or the Hololens 2 emulator. Today this technology is limited to Windows guests, i.e. Windows running inside of a VM or container.

> To bring support for GPU acceleration to WSL 2, WDDMv2.9 will expand the reach of GPU-PV to Linux guests. This is achieved through a new Linux kernel driver that leverages the GPU-PV protocol to expose a GPU to user mode Linux. The projected abstraction of the GPU follows closely the WDDM GPU abstraction model, allowing API and drivers built against that abstraction to be easily ported for use in a Linux environment.

> Projecting a WDDM compatible abstraction for the GPU inside of Linux allowed us to recompile and bring our premiere graphics API to Linux when running in WSL.

> This is the real and full D3D12 API, no imitations, pretender or reimplementation here… this is the real deal. libd3d12.so is compiled from the same source code as d3d12.dll on Windows but for a Linux target.

Re:GPU Video acceleration in the Windows Subsystem for Linux... by naikrovek on Tuesday February 14, 2023 @09:13AM (#34789517)
yes, I know about Hyper-V and I know that WSL is virtualized, but a standalone Linux VM is just Linux. It isn't WSL. WSL isn't just a name, it's the "Windows Subsystem for Linux" which is worded weird, but it's an NT Kernel subsystem which provides Linux compatibility. WSL v1 used syscall translation from Linux to Windows, meaning it was not a VM, and WSL 2 uses a Hyper-V virtual machine (two, actually) along with some glue to pull it together into a nice feature for Windows.

If you remove Windows, it's just a Linux VM. Just Linux. Azure already offer this as do all other cloud providers.

Re:GPU Video acceleration in the Windows Subsystem for Linux... by jameshilliard on Tuesday February 14, 2023 @03:15PM (#34795329)
> yes, I know about Hyper-V and I know that WSL is virtualized, but a standalone Linux VM is just Linux. It isn't WSL. WSL isn't just a name, it's the "Windows Subsystem for Linux" which is worded weird, but it's an NT Kernel subsystem which provides Linux compatibility. WSL v1 used syscall translation from Linux to Windows, meaning it was not a VM, and WSL 2 uses a Hyper-V virtual machine (two, actually) along with some glue to pull it together into a nice feature for Windows.

I'm probably not being that clear with the terminology, I'm referring to WSL2(WSL v1 is the legacy translation layer that isn't important here, I was always referring to WSL2).

> If you remove Windows, it's just a Linux VM. Just Linux. Azure already offer this as do all other cloud providers.

Different VM hypervisors have the ability to expose additional features to guest operating systems, in this case a Hyper-V/WSL2 host like Azure can provide critical functionality to the guest Linux VM that other hypervisors are not able to provide.

If you remove the Hyper-V/WSL2 host you lose the ability for the Linux guest userspace applications to make use of libraries like libd3d12.so and libdirectml.so.

The fundamental issue is that the WSL2 Linux userspace(which is basically a port of a number of Windows userspace components to Linux) has closed source libraries like libd3d12.so and libdirectml.so which have a hard requirement on dxgkrnl and a compatible Hyper-V hypervisor/Windows based VM host. If Linux applications are developed targeting these closed source libraries they will not function on normal non-WSL2/Hyper-V VM's.

See the diagrams/details here: https://devblogs.microsoft.com/directx/directx-heart-linux/

> Over the last few Windows releases, we have been busy developing client GPU virtualization technology. This technology is integrated into WDDM (Windows Display Driver Model) and all WDDMv2.5 or later drivers have native support for GPU virtualization. This technology is referred to as WDDM GPU Paravirtualization, or GPU-PV for short. GPU-PV is now a foundational part of Windows and is used in scenarios like Windows Defender Application Guard, the Windows Sandbox or the Hololens 2 emulator. Today this technology is limited to Windows guests, i.e. Windows running inside of a VM or container.

> To bring support for GPU acceleration to WSL 2, WDDMv2.9 will expand the reach of GPU-PV to Linux guests. This is achieved through a new Linux kernel driver that leverages the GPU-PV protocol to expose a GPU to user mode Linux. The projected abstraction of the GPU follows closely the WDDM GPU abstraction model, allowing API and drivers built against that abstraction to be easily ported for use in a Linux environment.

> Projecting a WDDM compatible abstraction for the GPU inside of Linux allowed us to recompile and bring our premiere graphics API to Linux when running in WSL.

> This is the real and full D3D12 API, no imitations, pretender or reimplementation here… this is the real deal. libd3d12.so is compiled from the same source code as d3d12.dll on Windows but for a Linux target.

Re:GPU Video acceleration in the Windows Subsystem for Linux... by DiabloD3 on Monday February 13, 2023 @07:51PM (#34783545)
Thats still a mistake in your thinking, however.

Non-WSL Linux systems are the majority. They are in your pocket, in your car, your TV, in space, on Mars. Microsoft will never be a player here but is still invited to the bazaar if they wish to participate.

And yes, I agree with Microsoft on this: all apps that people care about their 3D performance should move to DX12 or Vulkan. Microsoft in-house works on supporting DX9/10/11 and OpenGL purely as a legacy API, and eventually, Windows won't allow drivers to implement them anymore, and it will be some shim. I'm not going to shame Microsoft for promoting DX12 over Vulkan; they're both largely the same API, driven entirely by AMD and Nvidia development teams to reflect the current state of hardware, not the whims of some bald CEO that threw chairs at people.

Could the shim be just DXVK-based? Sure. Could the shim be DX9/10/11 state trackers being added to Mesa, and then the Mesa-on-DX12-on-Windows method used? Also sure. Intel is already using DXVK as a legacy shim in their new Arc drivers, and its performing quite well. DXVK also outperforms Nvidia's DX9/10/11 emulation in some games. I could easily see a DXVK-based solution just ship with a future Windows.

Re:GPU Video acceleration in the Windows Subsystem for Linux... by jameshilliard on Monday February 13, 2023 @11:49PM (#34785430)
> Non-WSL Linux systems are the majority. They are in your pocket, in your car, your TV, in space, on Mars. Microsoft will never be a player here but is still invited to the bazaar if they wish to participate.

I agree this isn't what they are targeting, at the moment based on the design this doesn't appear to be a play for the embedded Linux market, it's a play for controlling the server/desktop markets(at least the segments of those markets that would potentially use the WSL only Linux userspace API).

> And yes, I agree with Microsoft on this: all apps that people care about their 3D performance should move to DX12 or Vulkan. Microsoft in-house works on supporting DX9/10/11 and OpenGL purely as a legacy API, and eventually, Windows won't allow drivers to implement them anymore, and it will be some shim. I'm not going to shame Microsoft for promoting DX12 over Vulkan; they're both largely the same API, driven entirely by AMD and Nvidia development teams to reflect the current state of hardware, not the whims of some bald CEO that threw chairs at people.

The issue is that they are effectively bringing a WSL only userspace to Linux(https://devblogs.microsoft.com/directx/directx-heart-linux/) and are pushing for developers to target that WSL only userspace. Specifically they provide Linux native libraries like libd3d12.so and libdirectml.so which have a hard requirement on WSL.

> Could the shim be just DXVK-based? Sure. Could the shim be DX9/10/11 state trackers being added to Mesa, and then the Mesa-on-DX12-on-Windows method used? Also sure. Intel is already using DXVK as a legacy shim in their new Arc drivers, and its performing quite well. DXVK also outperforms Nvidia's DX9/10/11 emulation in some games. I could easily see a DXVK-based solution just ship with a future Windows.

The Mesa work seems to be effectively used for the "Embrace" part of the EEE strategy, it ensures that any existing Linux only software runs well under WSL, it's not really the main worry IMO.

The WSL only userspace(ie Linux applications targeting libd3d12.so and libdirectml.so) is the "Extend" part(extends the Linux userspace with WSL only userspace functionality).

Encouraging developers to drop support for non-WSL only API's/libraries on Linux is the "Extinguish" part.

So the main issue is that critical libraries like libd3d12.so and libdirectml.so have been designed to be WSL only. If this wasn't an EEE strategy I would expect that there would be some path to eventually use libd3d12.so and libdirectml.so without WSL, but that isn't possible and there appears to not be any plans to make it possible.

> This is the real and full D3D12 API, no imitations, pretender or reimplementation here… this is the real deal. libd3d12.so is compiled from the same source code as d3d12.dll on Windows but for a Linux target.

> libd3d12.so and libdxcore.so are closed source, pre-compiled user mode binaries that ship as part of Windows.

> D3D12 wouldn’t be able to operate without a GPU specific user mode driver (UMD) provided by our GPU manufacturer partners. The UMD is responsible for things like compiling shaders to hardware specific byte code and translating API rendering requests into actual GPU instructions in command buffers to be executed by the GPU. Working closely with our partners, they have recompiled their D3D12 UMD to a Linux target, enabling execution of these drivers in a WSL environment. This support is being integrated in upcoming WDDMv2.9 drivers such that GPU support in WSL is seamless to the end user. WDDMv2.9 drivers will carry a version of the DX12 UMD compiled for Linux.