News for hackers. Stuff that matters.

mosra

A new Magnum feature provides efficient compile-time and runtime CPU detection and dispatch on x86, ARM and WebAssembly. The core idea behind allows adding new variants without having to write any dispatching code.
> on mobile platforms, I found that UBOs were a win from call overhead alone

That was what I got from it also. Be it a phone or WebGL translated using ANGLE to something else, every call mattered. But on desktop drivers that were relatively low-overhead to begin with, going to UBOs was not always a win, especially on Intel GPUs with dynamic indexing into a UBO array or if I didn't use the GL 4.4 immutable buffer storage. And it didn't matter if I was on Linux with Mesa or on an Intel Mac. Of course no such problem on NV or AMD, but it made me approach UBOs with caution. I still need to dive deeper, because it'd be silly if such implicit overhead carried over to Vulkan.

So what I found ideal is submitting a multi-draw call with as many draws as possible, trading the weird extra UBO overhead with less time spent in the driver. That's limited by max bound UBO size, so depending on how well I pack the data I might only get as low as 256 draws in a single submit, and then I'd have to rebind another UBO range and submit another multi-draw call.

> you need to track your current state and the draw call's state

Yup, and that's the thing -- once I ship it, it has to be completely bug-free to not break existing apps in boundary conditions. And since a large portion of users is doing crazy advanced stuff like sharing the GL context with Qt or Gtk, or using it together with 3rd party GL libraries for drawing vector graphics and whatnot, the state tracker needs to be able to reset itself (or, worse, reset the GL state to not make a buggy 3rd party lib misbehave). Altogether it's a lot of tiny things to get right, and somehow there was always something more important that users needed so this fell on the bottom of the priority list. If I would be creating a new GL wrapper from scratch, I'd definitely go this way from the start, it's harder to retrofit a codebase with over a decade of history and a ton of depending projects that learned to expect smooth upgrades :)

Project author here. Hello :)

Let me just clarify a few bits. The GL wrapper is deliberately this low-level, because the library provides a baseline for a much broader range of uses than just games themselves. For an application that performs real-time image / video processing on a GPU, draw call or transparency sorting would be extra overhead. An _actual game engine_ that implements its own custom draw batching would have to work around the implicit batching, if it was there. The wrapper is just lower-level than you expected, that's all, and same is for the currently unadvertised in-progress Vulkan wrapper.

Regarding uniforms, other people already pointed out there is a UBO interface [1]. But there are also users who _rely_ on WebGL 1.0 or GL 2.1 compatibility (imagine that!), so I can't just drop the classic uniform path (and other related suffering). On the other hand, not all hardware and drivers have the UBO path faster in all cases, so providing an UBO iterface and failing back to classic uniforms only if UBOs are not available doesn't make sense either. In my measurements, the only case where UBOs actually improved performance across all platforms was in a multi-draw scenario, with UBOs being split by frequency of update, containing data for 100+ draws and being uploaded at most once per frame. The engine supports such use case [2], but doing that efficiently means having your data and logic ready upfront. If the library would try to attempt that automagically implicitly, it wouldn't end well and the gains would be minimal. That being said, I don't rule out a possibility for an opt-in high-level "render graph" or "pass management" API that does fancy things on its own, but it needs a lower-level layer built upon. Not all or nothing, layers. Layers so projects with specific needs can ignore that and use the layer below directly.

There's no global state anymore except for the Renderer, and I'm painfully aware of that last piece, yes. It's getting fixed, but because on certain targets (WebGL) every call matters, it's not as simple as packing all that state into a single struct and then resetting it in a bulk before a draw call.

"SceneGraph"? Yes, that part is in a maintenance mode, and I recommend new users to use entt, flecs or other ECS frameworks instead, until I build a replacement for this ancient technology that's there since 2010 (yes). But a lot of projects still want to use it because it's _damn convenient_, so I can't really remove it yet either to save the project from immediately raising a "red flag" to certain people ;)

Finally, the project has been under frantic development since its last version annoucement in 2020. Not everything is done and documented yet, the website shows slightly outdated info and the examples don't really use the fresh APIs yet. The Vulkan parts you saw as "unrelated" or "unused", those are still getting finished. Plus there's of course a roadmap to satisfy the needs of long-running Magnum-based projects, so the focus is often in different areas than the "yikes"-triggering ones. Such as asset management and conversion [3].

Have a nice day.

[1] https://doc.magnum.graphics/magnum/shaders.html#shaders-usag... [2] https://doc.magnum.graphics/magnum/shaders.html#shaders-usag... [3] https://blog.magnum.graphics/announcements/new-geometry-pipe...

While I'm here, a shameless self-promo -- Magnum uses a Python doc generator that includes type annotations by default, moving it closer to what you're used to from C++: https://blog.magnum.graphics/announcements/introducing-pytho...

(Sure, it still means the library has to make those annotations in the first place, and have them correct, but the newer and more complex libraries fortunately do, if even just to make IDE autocompletion work.)

Hah, good point, seems like this class suffers from documentation erosion. It got expanded and expanded and expanded and the essential piece of information got somehow lost down there in the gory details. The actual usage info and code snippet is in the base class documentation [1], it's just that it's no longer linked as prominently as it should be. I'll fix that.

It's not this bad everywhere tho, docs of fresher APIs didn't have a chance to erode yet: https://doc.magnum.graphics/magnum/classMagnum_1_1Trade_1_1S...

[1] https://doc.magnum.graphics/magnum/classMagnum_1_1Trade_1_1A...

Re:Improved Doxygen documentation and search for C++ projects on Thursday January 03, 2019 @03:01PM (#18818690)
For the record, nobody complained when I removed all the class diagrams and other things mentioned above, so my takeaway was that ... yes, those features were useless :) On the other hand, people complained about lack of essential features that Doxygen didn't have, like proper search or #include information for non-class members.
Re:Improved Doxygen documentation and search for C++ projects on Thursday January 03, 2019 @02:27PM (#18818338)
Definitely. Maintaining the Magnum documentation is the most time-consuming portion of the development (tests are second, the actual code takes the least amount of time), but so far it was very worth it.
Re:Improved Doxygen documentation and search for C++ projects on Thursday January 03, 2019 @02:16PM (#18818220)
That's a well-written and nicely organized documentation, congrats :)

Btw., I'm now working on a C# support for this documentation theme, but by using XMLDoc directly instead of Doxygen (and thus also with none of its bugs). See here: https://github.com/mosra/m.css/issues/76

For the IDE docs integration, I know about QHP (by Qt), which is supported by QtCreator, KDevelop and has a VS support using the Qt VS Tools plugin. But that's C++. Weird that VS doesn't have something similar builtin for C#...

Re:Improved Doxygen documentation and search for C++ projects on Thursday January 03, 2019 @01:41PM (#18817802)
Not really an alternative, since C++ projects often use Breathe to "transpile" Doxygen to Sphinx. I even contributed a few patches to Doxygen from which the Breathe/Sphinx project directly benefited, but after trying it out a bit and hitting a few walls, I realized I would achieve better flexibility by having my own frontend.

Especially because (if I can dare to say) I ended up with a much better search than what Sphinx (or Doxygen) has -- try it out: https://doc.magnum.graphics/magnum/?q=max#search ;)

Re:Improved Doxygen documentation and search for C++ projects on Thursday January 03, 2019 @01:28PM (#18817647)
Exactly. Doxygen implicitly generates useless info about what file is included by what file, list of places from where a function gets called, what line was this and that function declaration in (and where is the definition), huge class inheritance diagrams, entangled monstrous file dependency diagrams, alphabetical file index, alphabetical symbol index, including every possible undocumented symbol and file it can find and tons and tons of other stuff that has a total value of 0.

The user should visit docs to get to know a high-level overview of a library or human-readable explanation of an algorithm. Not to see stuff that's already explained by the code itself.

So here I threw away all this noise and the theme is actively forcing the library authors to focus on important stuff in the docs, explicitly excluding useless things that could be auto-generated. "Installing Doxygen on a project" achieves nothing, one has to write the actual docs first.

The result? See for yourself: https://doc.magnum.graphics/magnum/namespaceMagnum_1_1Animat...

Older »