It felt almost the old days where your UI is derived strictly from business logic with fixed set of widgets is better for accessibility than our new crop of apps where you have more gestures and shortcuts. It is no wonder that imgui shines if you need more gestures and eye-candy animations.
IMO, the bigger problem is the one that Raph pointed out: widgets (including simple ones like static text) need to have stable identities throughout their lifetimes, even as their state, contents, and locations change. It seems to me that IMGUIs, or at least some of them, make this particularly difficult.
As for the stable identities, yes, it is important. But I also don't see much of a problem with explicit Key: the UI components ultimately will have map back into a data object, which should have enough contextual information to provide the said Key.
I am not sure I understand this question.
If the ability to add metadata to any widget is not sufficient then I don't know.
What about the “zoom under my cursor” type things? That seems much harder (though not impossible) if your interface to clients is “just draw directly”. Though thinking about it, it’s actually probably what macOS does: as a post process, take your frame buffer and then magnify the stuff in a region of interest and re-render that on top.
You’re winning me over :).
Edit: I assume your metadata solution is “if someone asks, the text under this region of interest says < string >”.
A structure is built every frame with all the data to render later, as a batch.
And of course if you need a magnify lense the obvious way would be to render it as a post process.
I rest my case.
You have to set a compilation flag to enable the keyboard/gamepad navigation, if you missed it from the source code.
Even with integration into system accessibility APIs, Narrator (and other screen readers) performs a lot of heuristics to improve the user experience. These heuristics depend on a separate tree data structure [2] and how it changes over time, so in order to implement even the most basic accessibility, IMGUI would have to retain that information instead of throwing it away every frame. Component identity is crucial to how most (all?) screen readers work today and it's a happy accident that it also allows an efficient hybrid retained-immediate mode architecture.
I don't know if it has been fixed or not, but a year or two ago you could completely nuke the Windows screen reader just by assigning React components randomly generated keys so that the DOM nodes would be destroyed and recreated every redraw.
[1] https://dequeuniversity.com/screenreaders/narrator-keyboard-...
[2] https://docs.microsoft.com/en-us/windows/win32/winauto/inspe...
I’m a big fan of immediate mode myself, because it’s kind of amazing how much you can actually push through via brute force.
However, wouldn’t you agree that Unity, Open Scene Graph, etc. are successful counter arguments?
I would say it depends on your audience. The bottom layer of the stack might be a very simple retained mode, or fully brute force immediate, but it’s not clear to me that a “UI framework” for application developers should be immediate mode.
An UI, from an abstract point of view, is nothing but a big structure.
When you're writing an application or a game, you very often already have an existing structure containing everything you need.
Take a simple example : a leaderboard.
Using anything but an immediate-mode API will force you to create a second, complex structure to hold this leaderboard, and a lot of glue code to copy data to and from this structure.
I think that argument is “people can’t usefully agree on common abstractions, why bother forcing a translation”. And so for users of scene graphs, it’s basically “well, we can all agree there’s a basic graph and bounding boxes are a thing” so you’re not giving up much.
But for UI, is the argument that it’s often more like drawing/painting, so you spend more cycles (in both mental and processing senses) on trying to express the thing you already knew you wanted to achieve?
I've seen that made many times.
Its design is fundamentally much simpler than that of the immediate mode UI libraries. There is no need for implicit state. There is no need for layout to be a core feature of the library; layout can be 'just another widget'. This opens up the possibility for different types of layouts: constraint-based, grid-based, etc.
It's even possible to implement an imgui on top of this fairly easily; but not the other way around, because imgui has implicit global state to manage.
Imgui is not slow, especially with GPUs. But it definitely loses on the simplicity front.
If you have a static UI, you can skip rendering frames until "something" happens. That's going to burn no more power than a retained-mode UI.
Sure, the immediate mode UI may redraw 100,000 triangles vs the retained-mode 100 triangles. However, that's just not a bottleneck on practically every device nowadays. And I doubt that the difference is that stark as rendering scaled text properly is still just egregiously expensive in terms of computation.
However, everybody nowadays has to "ANIMATE! <jazzhands>" so you're rendering 60Hz anyway irrespective of type of UI.
And if animation is required by application (film editor, digital audio workstation, etc.) it's not clear that you really can do anything other than immediate mode. Do note how many of those kinds of programs wrote their own GUI over the years--that says something.
One thing that people don't mention is that immediate-mode UI's tend not to suffer from as many synchronization issues and retained glitches as retained-mode UI's do. Even if you don't get the update correct in an immediate-mode UI, the next frame the error goes away. In a "retained-mode" UI your widget needs to "refresh", which may not happen except in very rare circumstances. I have had to grab the corner and adjust a window size a couple of pixels innumerable times in order to clear UI glitches over the years.
I should probably comment about threading and the "UI Master Thread", but I don't think I've seen a good example of an immediate-mode UI running multi-threaded, yet. You probably need Vulkan to even have a hope of pulling that off (OpenGL is notoriously multi-thread hostile).
I am running a prototype immediate mode gui that I built for my yet unreleased engine and the frame time is staying below 500ns for a fullscreen app with many widgets and layers.
The whole thing is built on C + Vulkan but is not entirely optimized yet.
And this is running on a decent but not extremely fast computer. (An Intel NUC from 2018)
There are many ways to improve performances of idle/almost idle ui.
Caching can be useful for large blocks of text as it tend to be the slowest thing to render.
But it is preferable to incrementally optimize/add complexity and check assumptions and not make large architectural decisions ahead of time.
One way to reason about it is to think in terms of pure data transformations.
We've all seen "smarter" approaches failing more than a few times, obviously on the simplicity side but also, sadly, on the performance side, even with all the caching and retained structures.
This is the strange beauty of brute force algorithms, they may look dumb and ugly for our refined human minds, but they tend to map very well to the capacities of computers.
It is often faster to not bother and blindly render something instead of running complex logic to decide if you really need to render or not.
And if you're not satisfied with the performances of a "dumb" imgui, you can always add some smarter caching without touching the API.
Altough I bet this approach would only get you marginal benefits.