On Windows Vulkan video would work. But not on macOS (that mpv thread mentions it).

On Linux it should be a better option than VAAPI once drivers will support it. Since Firefox is using ffmpeg anyway, it probably shouldn't be too hard to make it an option.

Beware on Windows, out-of-the-box Microsoft will ship e.g. nvidia drivers that have DirectX 12, but not Vulkan. This can lead to confusing user situations ('DirectX 12 games work fine.. but not Vulkan') and probably means DXVA would be a better choice if targeting Windows

> nvidia drivers that have DirectX 12, but not Vulkan

not a thing. at worst you get a situation where the ICD loader isn't installed

In Windows 10 and above, windows update provides a default graphics driver for each IHV, which comes from the manufacturer, has DirectX support, but doesn't have Vulkan.

Often nvidia users will try to debug it, and discover 'the vulkan checkbox is off but I have the latest driver version!' in gpuZ[0][1] - but it happens across all IHVs, AMD, Intel, etc. I was told by someone in the industry the default Microsoft drivers across all IHVs indeed have DirectX, but not Vulkan.

To make matters more annoying, Windows Update often _overwrites_ the user's driver which they installed from the manufacturer directly, leading to Vulkan becoming broken overnight[2][3]

Easy to find multiple instances of this happening[4][5][6], I became aware of it because of someone asking on /r/vulkan a few weeks ago.

[0] https://i.imgur.com/qj3Rn9p.png https://rog-forum.asus.com/t5/rog-strix-series/nvidia-driver...

[1] https://forums.x-plane.org/index.php?/forums/topic/228597-vu...

[2] https://www.reddit.com/r/Amd/comments/1145ffp/windows_update...

[3] https://answers.microsoft.com/en-us/windows/forum/all/window...

[4] https://i.imgur.com/PXZRxYh.png

[5] https://www.reddit.com/r/pcmasterrace/comments/qhfesc/vulkan...

[6] https://www.reddit.com/r/techsupport/comments/110bytq/vulkan...

> has DirectX support, but doesn't have Vulkan

The Vulkan implementation is still there. You just have to install the redistributable. (see https://vulkan.lunarg.com/sdk/home - Runtime)

There's no separate Vulkan-less version of the drivers AFAIK - it's just that the installation program of the manufacturer doesn't run so that there's no opportunity to install the Vulkan runtime.

Thanks for the info, cool to know you can install that separately.

My original point remains: out of the box, many Windows users find they have the 'latest drivers' for their graphics card, find that DirectX 12 is working fine on their system, while simultaneously finding Vulkan does not.

For games, it's fine to require users to fix their Vulkan drivers. But if you're making a video player and want to leverage hardware decoding.. I think this is an aspect you should be concerned about and aware of.

Why would it be a better option than VAAPI?

Decoding h264/h265 in fixed (CPU) hardware is much more efficient than on the GPU.

Don’t get me wrong, its still miles and miles better than software decode.

VAAPI is also using the GPU. The fixed-function video decoders are on the GPUs, not the CPU. That just happens to be using the iGPU on Intel so you don't need a discreet GPU necessarily, but this is also missing if you have a CPU-only sku like the -f suffix parts

The only difference is the vulkan spec is _in theory_ cross-platform. However, since MacOS doesn't support Vulkan and MoltenVK doesn't implement this extension (and probably won't), and this extension also isn't implemented on Android, then your scope of actual availability is quite pathetic. So it probably doesn't make sense to move off of VAAPI regardless unless you really want Linux & Windows to share a codepath, even though you have something OS-specific anyways for MacOS/iOS & Android.

> MoltenVK doesn't implement this extension (and probably won't)

Since Khronos appears to have decided to require manual DPB management, this cannot be implemented in HW on macOS by MoltenVK.

It's unhelpful to bundle this into the "GPU" at large and is only true for desktop systems anyway. On mobile, video decoding, GPU and display pipeline are typically different hardware blocks often made by different companies.

Huh, I stand corrected. I always figured they stuffed it in its own little bit of silicon. Thanks for clearing it up!

Intel does do this, they call it Quick Sync Video. There's support for it in vaapi. AV1 is only if you have a Gen12 iGPU or newer, or Arc.

I might just be bad at Google-fu, but my understanding is that -F processors don’t have Quick Sync, which seems to imply that the iGPU is required in some capacity.

Vulkan Video Extensions use the same physical hardware as VAAPI, but with a little bit less of forcing you to do codec-specific nonsense. It should be better in that it's much easier to get a zero-copy Vulkan texture from the decoded frame.

The GPU has also the fixed function video encode like integrated CPUs. Both VAAPI and Vulkan Video address the same hardware. Differences are in the API itself (with Vulkan being more explicit) and potentially in which vendors (or OSS driver developers) support which API in what degree.

Do CPUs have fixed video decoders? I thought those were only included in GPUs, including iGPUs.

Yes, every Intel consumer CPU has included a hardware video block since Haswell, called Quick Sync Video.

That block is a part of the included iGPU though.

Intel CPUs without iGPU, like the F-series, do not support QSV.

Could anyone elaborate on the difference between software decoding and CPU decoding? Software decoding uses the CPU right?

That's right. "Software decoding" means the decoding algorithms run on the CPU. (As opposed to "hardware decoding" which typically means the work is done by some fixed-function video units on e.g. a GPU)

I think not being tied to VAAPI reduces the chain of development (like you don't need to wait for Intel to add something first since they are the primary developers of VAAPI in general).

But why would Firefox go out of its way to use Vulkan on Windows or macOS when it doesn’t have to because it can just use the native OS functionality and let the OS worry about implementation details?

I don't really care much about Windows and macOS. But I'd like to see it using Vulkan video on Linux in addition or instead of VAAPI.

As pointed out elsewhere, it's not possible on macOS either way. Apple are too stuck up not supporting Vulkan and it won't work with MoltenVK.