Glad to see more support for OpenGL, but I really hope we'll soon move to a compute-only way of handling graphics. The overhead of vulkan is absolutely insane (and not warranted, in my opinion), and OpenGL is on its last legs.

Things like Nanite spark a little hope, since they've shown that software-rasterization via compute can be faster than the standard graphics pipeline with the hardware rasterizer for small, dense triangles. Seems like a matter of time until everything goes compute, even large triangles. Maybe the recent addition of work-graphs in DirectX is one step in that direction?

> I really hope we'll soon move to a compute-only way of handling graphics

Not happening anytime soon. The industry has lined up pretty solidly behind Khronos/Vulkan.

I know it won't happen overnight, but since it's already possible to do software rasterization for small triangles faster than hardware, having a graphics API framework starts losing its purpose. After all, we want to have the detail of small triangles anyway. Just let us draw to the screen in CUDA without the need for OpenGL/Vulkan interop, and I believe we'll soon see a shift to serious compute-based real-time rendering.

Basically, instead of graphics being a framework, I want graphics to be a straightforward library you include and use in your CUDA/HIP/SYCL/OpenCL code.

> let us draw to the screen in CUDA without the need for OpenGL/Vulkan interop

how would that work? like GPU frameworks would just be compute (like cuda) and some small component of it would just allow to write the end result to a buffer which would be displayed or smth?

Exactly. Things like that already work with a workaround: You can use Cuda-OpenGL interop to expose an OpenGL framebuffer in CUDA, then you can simply write into that framebuffer from your CUDA kernel, and afterwards you get back to OpenGL to display it on screen. Just directly integrate that functionality in CUDA by providing a CUDA native framebuffer and a present(buffer) or buffer swap functionality.

The industry targeting Android and GNU/Linux devices, that is.

>. The industry has lined up pretty solidly behind Khronos/Vulkan.

What industry? Gaming industry is mostly using DirectX or Gnm/PSSL.

Only if you take a very narrow view of the gaming industry that only includes high end consoles and PCs.

The Nintendo Switch and PCs support OpenGL and Vulkan. But those are both dwarfed by mobile gaming which I think makes up more than half of the entire gaming market, and both iPhone and Android support OpenGL. Although I don't know if the majority of mobile games use it or not.

While switch supports OpenGL and Vulkan, in practice very little games use it. The native API on Switch is a proprietary API called NVN.

OpenGL on Apple is on life support to support existing software, Apple platforms are all in on Metal now.

The only platforms where OpenGL and Vulkan are first class citizens is Linux, Android and Windows (barely). And Vulkan is still second fiddle to DirectX on Windows.

All fair points. However, I was taking issue specifically with the claim that the "Gaming industry is mostly using DirectX or Gnm/PSSL".

It would be hard to quantify exactly how much of it is using OpenGL, as a percentage.

However, for a very rough upper bound I quickly looked up these:

* Mobile gaming apparently makes 52% of the entire gaming market in 2023

* Android is 78% of mobile gaming

That could mean OpenGL makes up to 40% of the entire gaming market just from Android.

Although I have no idea what percentage of Android games use OpenGL and wouldn't know how to look that up. I also don't know how accurate those percentages are. I got them from brief Google queries and got both the 52% and 78% figures from the featured snippet.

But even if that 40% is much too high, we still need to add switch and PC games and I would find a figure of 30% of the gaming industry using openGL to be believable. That would put OpenGL as the largest API which I didn't expect, even as a possibility.

Just to reiterate on this point: Pretty much every Vulkan game on Windows forgoes using Vulkan Swapchains in favor of presenting from compute and using native DXGI Swapchains. If you are shipping a production quality game or application, you really want to use the platforms native APIs to get the full capabilities.

Interesting, like which games? I know RDR2 uses normal Vulkan swapchains.

What percentage of applications *natively* target Vulkan?

Imho Vulkan is the least used of all the APIs if you disregard it as a compatibility layer

Isn't nanite hardware rasterization with software shading? Like using the GPU to draw triangle/cluster ID #s into the FB and then shading those?

It uses the rasterization HW for large triangles and uses a software rasterizer in compute for small triangles.

The HW rasterizer is not that efficient if the triangles are tiny, which is the case for nanite.

Not really trivial if you have to support any input set of triangles and don't know much about them. Software rasterizion exploits things like localized chunks of triangles, but the hardware rasterizer does not know about that in advance. Also, these kinds of software rasterization algorithms add limitations, which aren't much of an issue with your own rendering pipeline that specifically knows how to deal with these limitations and works with them.

Part of Nanite is software rasterization by rendering triangles with 64 bit atomics. You can simply draw the closest fragment of a triangle to screen via atomicMin(framebuffer[pixelID], (depth << 32) | triangleData). Not sure right now if they are using a visibility buffer, but if they do than "triangleData" would be the triangle index, otherwise it could be the actual shaded color.

For small triangles, they use a software rasterizer into the V-buffer. Obviously for 1px triangles since they don't want to waste quad overdraw, but I think they found it's faster up to 12px/tri or so on AMD.

> The overhead of vulkan is absolutely insane (and not warranted, in my opinion)

Would you mind expanding on this?

I meant the development/learning overhead. With Vulkan you can do incredible low-level optimizations to squeeze every last bit of performance out of your 3D application, but because you are basically mandated to do it that way, you have a very harsh learning curve and need lots of code for the simplest tasks. I'd rather prefer approaches that make the common things that everyone wants to do easy (draw your first simple scenes), and then optionally gives you all the features to sqeeze out performance where you really need to. Because at least for me, I don't work with massive scenes with millions of instances of thousands of different objects. I do real-time graphics research, mostly with compute, and I'd just like to present the triangles I've created or the framebuffers I created via compute (like shadertoy).

I'll readily admit, I'm neither smart nor patient enough for Vulkan so I quickly gave up and learned CUDA instead, because it was way easier to write a naive software-rasterizer for triangles in CUDA, than it was to combine a compute shader and a vertex+fragment shader in Vulkan. I'm just rendering a single buffer with ~100k compute-generated triangles, and learning Vulkan for that just wasn't worth it.

Well, then there's OpenGL and DirectX.

Yeah, but OpenGL doesn't get updates anymore. My timeline goes like: I needed pointers(and pointer casting) for my compute shaders so I checked the corresponding GLSL extension, which was only available in Vulkan so I tried switching from OpenGL to Vulkan. After a week I gave up - the pointer/Buffer reference extension did not look promising anyway - and I tried out CUDA instead. That's when I found out that CUDA is the greatest shit ever. That's what I want graphics programming to be like. Since then I just render all the triangles and lines in CUDA, because it easily handles hundreds of thousands of them in real-time with a naive, unoptimized software-rasterizer, and that's all I need. In addition to the billion points you can also render in real-time in CUDA with atomics.

> That's when I found out that CUDA is the greatest shit ever. That's what I want graphics programming to be like.

Something which requires you to buy new hardware from a specific brand, and load an out-of-tree binary-only module on your kernel? That's not what I want graphics programming to be like.

The Vulkan API might be clunkier (I don't know, I haven't looked at the CUDA API, since I don't have the required hardware), but at least it can work everywhere.

It's the exact reason why I've avoided CUDA for years, but I hit a dead end with OpenGL and Vulkan, and CUDA happened to be a fantastic, easy and fast solution. Of course I don't want graphics programming to be NVIDIA-only, but I want it to be like CUDA, just for all platforms.

Except it doesn't work everywhere, far from it.

I'm confused by what you mean here by "software-rasterizer" when you are compiling it to run on the GPU. What do you think the graphics driver is doing when you program it with OpenGL? It's doing more or less the same thing under the hood.

I guess you can technically call that software rasterization now that GPUs are very programmable. But it's not how the word is usually used.

GPUs have dedicated hardware for rasterisation. Using compute shaders to do it would be wasteful.

Not necessarily. I'm working on an application that does exactly that, as we are rendering spheres. It ends up being much more efficient to rasterize the sphere in a combination of vertex and fragment shaders than to instantiate triangles.

Nanite renders small triangles faster by avoiding the dedicated hardware and using compute, instead.

Micro-triangles are a special case and Nanite still uses the hardware for more reasonably sized triangles.

"Turns out we can beat the hardware with triangles much bigger than expected, far past micropoly. We software rasterize any clusters whos triangles are less than 32 pixels long." - https://advances.realtimerendering.com/s2021/Karis_Nanite_SI...

It's not just micro-triangles, it works for triangles that span multiple pixels. And that's not a special case, that's the standard nowadays, except for games targeting very low-end devices.

The dedicated hardware is nice for general-purpose support for arbitrary triangle-soups. But if you structure triangles a certain way and have a certain amount of density (which you want for modern games), you can specifically optimize for that and beat the general-purpose hardware rasterizer.

And webgpu.

Sure, the implementations today are thin layers over D3D12, Vulkan and Mantel. But it is a proper API, cross platform and without the implementation overhead.

Next time I need to draw some graphics I will definitely use WebGPU instead of OpenGL or Vulkan.

PS. Don't let the name fool you. It is a proper render API, Mozilla has written their implementation in rust and Google in C++.

Would not be surprised if WebGPU do get native support by the drivers in the future.

> The overhead of vulkan is absolutely insane

Overstatement of the year award candidate.

Vulkan and OpenGL both already support mesh shaders, which is a compute-oriented alternative to the traditional rasterization pipeline.

> Vulkan and OpenGL both already support mesh shaders, which is a compute-oriented alternative to the traditional rasterization pipeline.

Mesh shaders are a step in the right direction, but they are still embedded in all that unnecessary Vulkan fluff. I would want these things in CUDA because it does the opposite approach of Vulkan - it makes the common things easy, and the hard/powerful things optional. Just let me draw things directly in CUDA, and maybe give access to the hardware rasterizer via a CUDA call.

> CUDA [...] makes the common things easy

Only if the common thing does not include running on hardware of more than one vendor. Which is a pretty common requirement for any consumer software.

You're missing the point. CUDA is easy but also powerful, and it shows there is no reason that other APIs need to be hard and cumbersome.

> Overstatement of the year award candidate.

I think he meant development overhead, not performance overhead.

Yes, sorry for the confusion. I'd just rather have an API where the common things are easy, and the super powerful low-level optimizations are optional.

There were several pushes to produce a "sane defaults" library that made vulkan a lot less verbose.

OpenGL is exactly what you said, "an API where the common things are easy, and the super powerful low-level optimizations are optional."

except the easy common things are very slow, and the optimizations which are basically required become a careful dance of understanding the underlying hardware interface anyway (without any real documentation) and figuring out how to map that onto the high-level interface.

You could always just use a library that simplifies Vulkan or keep some skeleton apps around. When you're past all of the verbose set up, the actual meat of it isn't too bad.

What happened is that Khronos Group banked on third parties writing OpenGL-esque "middleware libraries" that bridged the gap between the hopelessly complex Vulkan API and people who want to write simple 3D graphics. Unfortunately, Khronos learnt the hard way that you can't just expect people to do work for you without a sufficient incentive, since after all, OpenGL still exists for anyone who wishes to have an easy API for graphics programming, and anyone who doesn't want to do that just uses an engine or high-level graphics API.

Of course I'm not saying that these libraries don't exist, but they haven't taken off in the way they expected

> who wishes to have an easy API for graphics programming, and anyone who doesn't want to do that just uses an engine or high-level graphics API.

I'm guessing this niche is just pretty small in commercial sense; most people probable end up just using UE/Unity/... as their graphics API.

I agree, so I think Khronos expected the open source community to fill in the gaps

* ... a compute-oriented continuation of the practice where they take the average of Nvidia and AMD's implementations, and then force the developer to deal with the overhead of harmonizing the details without going insane.

GPU compute is awful, almost as bad as GPU rendering.

>but I really hope we'll soon move to a compute-only way of handling graphics.

Would you have the time to expand on this thought a bit? I am curious. Thanks!!

Software rendering. In parallel on the GPU.

I mean, about 5 years ago there were many folks that were trying to do raytracing using GPU compute rather than the current methods, it was essentially treating the CPU as a giant parallel software rendered. The results were pretty good even then.

* * *

I don't get what that means. You mean configuring a shader to render pixels?

How do you think OpenGL/Vulkan works under the hood? It's been a very long time since fixed function pipelines.

The standard rendering pipeline is still fairly fixed and mandates shaders for vertices and fragments. With software rasterization, there are no more vertex or fragment shader, only compute. And you don't use the hardware rasterization units - you rasterize the triangles yourself and write the results to the framebuffers with atomic-min/max operations. You decide for yourself how and when you compute the shading in your compute shader. This can be multiple times faster for small triangles, and 10-100 times faster for points. And once you do things that way, there isn't much point for graphics APIs anymore - everything is just buffers and functions that process them.

Why is there a pipeline at all? Give me a framebuffer, a place to upload some CUDA or equivalent, and some dumb data pipe between the host program and GPU program.

Good to see Zink now provides OpenGL over Vulkan for smaller GPU makers. I wonder if big ones will also eventually use it for it.

I suspect that, by now, the only reason the other mesa3d opengl drivers exist is historical.

If these vendors came up with different enough hardware to need a new driver from scratch, they'd just focus on Vulkan and use Zink for opengl.

After all, opengl is a relatively high level API. It is sensible to implement it in a hardware-independent manner on top of Vulkan.

No, not even close. On Intel in some chipset(s) Vulkan is not as performant as OpenGL 4.6. Gallium3D with Iris it's the future against the old pipeline, tho. For instance, I get OpenGL 2.1 in my netbook with the new drivers, 1.4 with the old ones instead.

The big players will probably switch once we get to the point that nothing still running on OpenGL needs optimally efficient usage of the GPU (either because it's old or because it was never performance-critical to start with, such as a student project).

I had to double check that OpenGL 4.6 (released in 2017) really still is the latest version.

It is, and there is no red book edition covering it.

Vulkan was initially called OpenGL-next.

What is this?

This company makes GPUs?

>This company makes GPUs?

Yes, and they've been around for a while. If you're old enough, you might remember the name "PowerVR".

Today, they mostly license GPU designs to SoC vendors. You'll find their designs in e.g. Android phones.

Notably, JH7110 (RISC-V SoC used in VisionFive2 and Star64) uses one of their recent GPUs.

does this mean you can run, say, modern openCL code on smth like a VisionFive2?

Oomph-wise, they claim a good 4x over what the Raspberry Pi 4/400 have. Note that rPi's is known to be anemic due to serious memory bandwidth bottlenecks, barely able to keep up with filling a 1080p screen.

Currently they do not support the specific variant used in JH7110, but they seem to both be variants of the same architecture.

JH7110 has the BXE-4-32MC1[0], whereas the driver currently supports the BXS-4-64-MC1[1].

No idea about compute, and AIUI openCL suport in mesa3d is still a disaster for all drivers.

0. https://www.imaginationtech.com/product/img-bxe-4-32-mc1/

1. https://www.imaginationtech.com/product/img-bxs-4-64-mc1/

I'm curious how you can tell that it was BXS-4-64-MC1 that was supported. I looked and couldn't find that.

Besides the JH7110 there's TH1520 and it's using the BXM-4-64-MC1, also not supported, but 2X the speed of BXE-4-64-MC1 if I understand it correctly.

Yes; and if you used an iPhone before 2017, you used GPU made by them.

Technically, their new GPUs are also directly descended from them. Though, deviating more and more each generation.

I don't mean to detract from the technical merits of this at all, but for those not aware you might be interested to know that ImaginationTech is a Chinese-owned firm.

It's nice to see Chinese tech embrace existing standards.

There are a lot of chips that are very poorly documented and supported by abysmal proprietary SDK. The software part of hardware is so often an afterthought yet it is also what makes or breaks the product.

They're a British tech company, simply owned by a Chinese private equity fund.

And they're sort of known for historically being very anti open source and their SDK difficult to continuously integrate into a larger product.

really no better than a us tech firm, unless you for some reason have a preference for nationality characteristics of your hardware backdoors lol

I am hopeful this will help mesa3d support with RISC-V SoCs such as JH7110 (and thus VisionFive2 and Star64) be excellent.

There are links from the blog post to the Mesa sources and to a Linux tree containing their DRM kernel driver.

I am aware.

Currently they do not support the specific variant used in JH7110, but they seem to both be variants of the same architecture.

JH7110 has the BXE-4-32MC1[0], whereas the driver currently supports the BXS-4-64-MC1[1].

0. https://www.imaginationtech.com/product/img-bxe-4-32-mc1/

1. https://www.imaginationtech.com/product/img-bxs-4-64-mc1/

I didn't know any other companies made GPUs.

You'd think some hardware player would buy them to get a foot in the GPU/AI game.

Are these GPUs fully open source?

No, they're not open source at all.

Their business model is to license the GPU IP block to other companies to integrate into their own chips.

It's worth noting that Imagination designed the GPUs used in iPhones up until the iPhone 8 when Apple switched to their own design.

They're closed source. Just parts of the driver are opensource.

Note that these GPU's are widely used in (mostly low end) phones, and I think power the UI in some other household items with screens. They provided iPhone GPU's till 2017 (and there is debate if Apples new GPU might contain some imagination tech)

Closed source.

That’s just crazy. If there’s anything a minor GPU player needs its community support.

Crazy is that they were aggressively anti-foss even with their drivers not too long ago. Intel graphics usually has good Linux support, except for GMA500 which was based on Imagination tech. N900 was the Linux darling phone, and the graphics drivers where god-awful buggy blob from Imagination. And so on..

For a company who historically has mostly made GPUs for phones and set-top boxes, I don't think that's true. People don't usually program these devices at this level. Especially in the case of all the hardware they did for Apple, no requirements to be open there.

Maybe these days that's changing but it's a big change for a company like this to embrace.

Is ImgTec back as a real going concern? Last I heard (back in 2016 or so) they'd basically lost the Apple GPU work (Apple opened up an office one town across, poached a load of engineers and bought the design work in house), and since then have been desperately trying to pivot to vision or VR of AI, but I've not heard of them having much traction.