I sincerely hope the web as a whole doesn't move in this direction. The inspectibility of the DOM is one of the web's greatest strengths. Lets not throw that away.
And input element behaviour is non-native, which will never be properly fixable since different UAs have different behaviours, some of which cannot be emulated even if you sniff which platform you’re on (e.g. scroll bar grab and drag behaviour when you move far outside the window). (Then there are also simple bugs: holding Left or Right while the range element is focused only bumps the value once, not according to key repeat. And Up and Down don’t work.)
I have written about these things a number of times, fundamental limitations of the pure-canvas approach that make it unavoidably and permanently unsuitable. https://news.ycombinator.com/item?id=33861831 is probably the best thread.
(And just for a bit of fun: the demo can have slightly negative utilisation, drawing outside the box!)
I wonder if your experience is different?
As best I've been able to tell, right now you have to make a choice between (a) fast and (b) native, and you only get to choose one. And in my application domain, fast is nonnegotiable.
Smoothly rendering 3k rectangles isn't a common use case for UI libraries, so I'm not surprised they perform poorly. They generally aren't optimized for that. The normal answer for weird, high performance rendering like that is to just use opengl or something and program against the GPU directly. 3d rendering contexts can render millions of rectangles on modern hardware without breaking a sweat.
This advice is the same on desktop, mobile or the web. I'm not surprised rust UI frameworks have the same limitation. Most UIs don't need thousands of rectangles. But they do need lots of custom widgets, and implementing all of those well is a higher priority job for anyone building a custom UI library.
Going to 3d rendering via OpenGL / WebGL was one of the options I considered. But honestly, this is a boatload of complexity. Why do I need to pick up a full 3D rendering library just to draw rectangles on the screen? Plus, consider: the slot viewer (where the rectangles are shown) still needs to draw some UI components (e.g., on hoverover). I still need that UI integration there, even if it's not as performance-sensitive. So I end up needing something along the lines of egui anyway.
I definitely understand that not all UI libraries are going to be optimized for this. But I was surprised that it wasn't easy to e.g. get a high-performance canvas dropped into libraries that specifically advertised themselves as high-performance.
By limiting yourself to a 2D engine you're ensuring there are more layers of abstraction between you and the hardware. While # layers of abstraction doesn't always correlate with overhead, it does place limits on performance.
It’s still a bit of a mess though. I understand wanting to keep everything in a single UI library if you can.
Example: https://share.firefox.dev/40QUthv . (Though this trace is probably smaller than the profiling traces you're demoing.)
Maybe you have graphs/charts that need that kind of rendering effect. Great, use Canvas for them, they're not accessible anyway. But if something like your main user controls are running into that problem, I feel like something has probably gone wrong with your user interface.
In the demo you linked, it makes a lot of sense to render to Canvas for the individual charts, flame charts, breakdowns. It does not make sense to me to render to Canvas for the sidebar. It doesn't make sense to me that the list element itself holding the charts is going through Canvas. That's something in your UI that honestly shouldn't be updating at the speed you're talking about.
To be honest, I'm not sure Canvas can handle either part of this: either drawing 80k rectangles at 60 fps or culling so that the other 10 billion don't need to waste CPU cycles. It requires a very tight integration with the UI library to make sure we do this fast.
Yes, it's a very narrow use case. That's why I went with a specialized UI library.
To be clear, WebGL/WebGPU is still using Canvas. You're not using the 2D api stuff, but that's just a high-level API on top of Canvas. You're still rendering to Canvas.
> I don't know if you noticed, but my test has on the order of 10 billion rectangles
Why? I've built apps like this before. Your choice is not "abandon the DOM" or "have zero performance." You can have most of your interface in the DOM and use Canvas (again WebGL/GPU is still Canvas) for specifically the parts of your app that wouldn't be accessible anyway (mainly charts).
It does not require tight integration with the GPU to render the sliders on the left hand side of the screen in this demo. No one is saying that you need to render your charts in DOM. We're wondering why you got rid of `ul` elements.
I didn't test Canvas specifically in my explorations, but I did test a few JavaScript based frameworks, and the ones I found that were efficient sat on top of WebGL, not Canvas. Ultimately I didn't pick those because they had a bunch of other stuff baked in that I didn't need or want, and were already equally non-native (sitting on top of WebGL). And I frankly didn't want to be writing JavaScript.
You are using Canvas right now. WebGL uses Canvas to render to the screen. If you open up your demo's DOM inspector right now, you will see a canvas element. It's:
<canvas id="the_canvas_id" width="1914" height="580" style="width: 1914px; height: 580px;"></canvas>
It's a little bit hard for me to trust this kind of dismissive talk about the browser adding too many abstraction layers when it's not clear to me that you're fully aware of what browser features your demo is using right now. Targetting the DOM and using embedded canvases really doesn't have anything to do with whether or not you're writing Javascript, and it doesn't mean you can't use WebGL to render charts.Ehhh sort of. Canvas is much older than webgl. When we talked about canvas a few years ago, that always referred to the canvas DOM element and its associated 2D shape based API. (fillRect / moveTo / lineTo / etc). When Webgl appeared, the nomenclature was "using canvas" vs "using webgl", since webgl's API is completely different from the 2d canvas API.
But browser manufacturers didn't give webgl its own DOM element. Webgl also uses a canvas DOM element, then configures it differently. (getContext("webgl") instead of getContext("2d")). So if you're using webgl (which I think this app is doing), then you're "using canvas" in the sense that your HTML contains a <canvas> element. But you aren't using the canvas 2d API - which, as I said, is totally different from webgl and is also called "canvas" because it was here first. And it doesn't have another name.
In defence of the GP comment, I think its pretty clear from context that "Canvas layers on top of WebGL/GPU draw calls" refers to the canvas API, not the canvas DOM element.
If it's just a context misunderstanding of what specifically "Canvas" is referring to, fine, I don't mind clarifying what I mean by that. And you're right, it's not unreasonable for someone to see the word Canvas and think "2D APIs." But I just want to be very clear that targeting the DOM for UI controls doesn't mean you can't use WebGL or WebGPU and direct graphics programming for charts, or that you can't render a chart in a web worker and then use the pixel buffer with a canvas. And none of that stuff requires you to program in Javascript (except for glue code, which you'd have to write for all of those methods no matter what). In modern web terms when I talk about targeting the Canvas, I just mean rendering pixels onto a canvas element as opposed to manipulating the DOM, regardless of how or where those pixels are calculated.
There is no additional set of abstractions you need to go through to do that, and there's no graphical system for the web that gets rid of the singular abstraction of needing to eventually put the pixels onto a canvas element.
Yes, if you were calling browser 2D APIs, that would have an additional performance cost, definitely. In a context where that's not performant enough, then don't do that? But there seems to be a suggestion in this comment thread that embedding canvases would require working with a higher level more abstracted graphical API when rendering the charts, and that's just not true.
Can you recommend an alternative to egui that let's you write entirely in Rust and get the performance of egui but still maintain some level of accessibility?
Ideally yes. This is a failing of the egui toolkit, not necessarily a failing of the people using it. The whole point of a high-level toolkit like this is that you shouldn't have to worry about how things are being rendered.
However, it is still important to counter claims that egui is making if the people using the toolkit are parroting those claims.
> Can you recommend an alternative to egui that let's you write entirely in Rust and get the performance of egui but still maintain some level of accessibility?
I don't think I've ever built a web GUI in Rust so I have no idea what the ecosystem looks like. To my detriment, I tend to build most of my UIs from scratch. I can promise that it is possible to have performant UIs with a lot of embedded charts (like this demo has) using the DOM, but I have no idea what Rust framework devs have been getting up to or publishing.
There's also a little bit of a catch-22 here because yes, any time you introduce the DOM, you are going to lose some performance. That doesn't mean that you're going to lose so much performance that the demo above would be stuttery or wouldn't work. But if you're just measuring "how many quads can I refresh on the screen", that's kind of missing the point that you really shouldn't be refreshing thousands and thousands of quads of information in a UI at 60fps. That's not a thing that most interfaces should be doing in the first place.
And if you are genuinely changing thousands of boxes of information on a screen at the same time, kind of by definition that isn't accessible period. There is no way to present that volume of informational change to a screen reader in a way that will be understandable. But a lot of people when building UIs tend to think they're in the rare cases where they need that level of performance and so they need to sacrifice that accessibility, when in reality what they actually have is a giant list of charts and aside from the actual chart rendering, there's nothing particularly expensive going on.
eg. I used to be able to programmatically grab hWnd's (like Spy++ does), and manipulate the forms of an app. Apps used to show up as one (or two) simple processes with rich instrumentation metrics in tools like Resource Monitor.
Chrome abstracted all that away in an effort to 'displace' the OS.
I think canvas-only technologies could eventually do the same (the endless cycle continues) and look forward to some of the benefits (eg. more deterministic and straightforward positioning and layout). But I strongly agree they need to accommodate most of the features you mentioned and until they do it's two steps backward.
Do appreciate your thoughtful comment and the concise summary of some fundamental limitations that are presently hard barriers.
eg. I used to be able to programmatically grab hWnd's (like Spy++ does), and manipulate the forms of an app. Apps used to show up as one (or two) simple processes with rich instrumentation metrics in tools like Resource Monitor.
---
I disagree pretty hard with this sentiment. Yes, it's harder to programmatically interact with some of the browser features from outside of the browser. Programmatic interaction with basically any application that isn't yours is getting harder across every operating system as security becomes more important.
That doesn't really change that introspection of the source that makes up a displayed site was easy and approachable. I can look at the css/html/js. Further, all of those resources are contained in files intended to be opened and looked at by humans (they're human readable text).
This is my biggest issue with WASM. You're literally back at reading assembly again. And while that's possible, and there is tooling that can help - it's just not the same as being able to right-click on something, alter a couple of attributes in human readable words, hit enter and see that change.
WASM on its own isn't the problem. WASM just lets you use another language to replace javascript. When you right click on something in a browser and alter human-readable attributes, you're usually messing with CSS and HTML. Any half decent web UI framework should pass HTML and CSS to the browser in order to render the UI. Thats just as true with wasm frameworks as it is with javascript frameworks.
The problem in this example is that the wasm code is bundling its own UI library that is trying (badly) to replace the browser's layout engine. All the browser can see is a bunch of drawing calls to an html canvas. Thats not the fault of wasm. Its just this UI library, which isn't really designed for the web.
For what it's worth, egui does have an accessibility layer. I haven't gone to any particular effort to wire it up yet (besides what you get by default): https://github.com/emilk/egui/issues/167
So what are you doing that couldn't be performant in the DOM? Because any kind of fast updates that you're doing that's too fast to render in an XML tree is also going to be too happening too fast to describe with a screen reader?
The DOM is asking you to be able to describe your interface -- performantly -- in entirely pure text. If you can't do that, then there's no way you can be fully accessible, because the screen reader needs to be able to describe your interface -- performantly -- in entirely pure text.
If you are updating thousands of rectangles in your UI at 30/60fps, it is impossible for you to fully convey the same amount of information that is changing visually to a screen reader.
Either your interface can be described using pure text, or it can't. Either I could use your interface by closing my eyes and having someone describe it to me, or I can't.
Accessibility could be fixed easily if there would be a web accessibility API. There's basically an entire 'middle layer' of web APIs missing, current APIs they are either low level (like WebGL/WebGPU) or too high level (like the DOM).
i mean, it sucked, but it was kind of that middle layer.
It does have one. The DOM is the web accessibility API. A really big point of the DOM is that you don't get to make multiple interfaces, you have to make one interface that works for everyone, which forces you to provide some degree of feature parity between those interfaces.
The web accessibility API is "you have to describe your interface to us in pure text, because that format is more likely to be accessible."
I'm not happy about it, but I think it will.
> The inspectibility of the DOM is one of the web's greatest strengths.
Not if you're pushing ads.
Egui slaps and is clearly going places.
I've run it before. I know what it looks like.
I use egui. It's OK. But you have to code up all your dialog boxes in Rust. This is misery if you have fifty dialog boxes and widgets you need. The widget library is weak and the themes are poor. I don't care, because I'm doing a metaverse client, and all the visuals are in the 3D image. The 2D GUI on top is minimal, as is normal for games. So minimal that it disappears completely, like YouTube controls, when you're not using it, to give you a clean 3D world.
(Egui hint: don't use bottom-up layout. Always work top-down, left to right. Egui layout is one pass, and while it tries to look ahead, it's not good at it.)
It's certainly nice to work with from a developer's perspective, what could be easier than
if ui.button("Say hello").clicked() {
println!("Hello!");
]
?But it's quite a pain to style if you're particular about typography and pixel-perfect layouts. Between styling an egui app and styling something with CSS (or some flexbox/grid system), I'll take CSS every time.
Still looking forward to how Xilem turns out, and what people build on top of that.
https://github.com/leptos-rs/leptos
These are already orders of magnitude faster than React, and as the WASM<->DOM bridge improves, they will only get faster.
Sycamore:
https://github.com/sycamore-rs/sycamore
https://sycamore-rs.netlify.app/examples/todomvc/#
Dioxus:
In about 3-5 years these frameworks will start to consolidate and reach maturity.
It's already possible to build reactive isomorphic apps in Rust.
If you haven't checked out Actix/Axum, it feels a lot like Python/Flask. The ecosystem is coming along quickly.
Electron builds native software by embedding a web browser and using the browser's layout engine and DOM. Canvas based rendering like this is the inverse. Here the application discards the browser's layout engine and DOM, and instead embeds its own layout engine. - Despite the browser being right there and available.
Hilariously, in a demo like this there's 3 layers of UI controls nested inside each other. First, there's the OS's native UI toolkit. Then there's the browser. The browser discards most of those UI elements, and reimplements its own controls. And then Egui discards all of that and embeds its own, third layout engine all written in wasm. No inertial scrolling. No web inspector. No native controls (especially a problem on iOS or android). No CSS - so I hope Egui's layout engine supports the layout you're after.
Downside: Its a 5mb wasm bundle. But on the plus side, 5mb is positively tiny compared to shipping electron!
Egui only use wasm in situations where you must use the user browser because you have no choice. Eg: showing a demo of the app on the web without having to install anything.
> Its a 5mb wasm bundle. But on the plus side, 5mb is positively tiny compared to shipping electron
This comparison makes no sense. Situation where electron could be used are exactly the kind of situations where a wasm bundle would not be shipped (a native binary would be shipped instead)
Sure; but given the insane size of shipping chrome, it’s fun seeing a full app complete with widget library, layout engine and text input elements in 5mb. It’s terrible to make websites that big. But if chrome could do all that stuff in only a 5mb binary then I’d be much happier with electron.
If you didn’t catch it, the wasm bundle is running in a 32mb memory slab. Positively slim by the standards of modern gui apps!
Next I clicked on the combo box, it opens the dropdown, and I try to use arrow keys to navigate it. That doesn't work. Is that because I opened it using the mouse? Nope, still doesn't work if you tab through the widgets and use Enter to open. Looking up items by typing the first few characters doesn't work, either.
This is all really basic stuff that native widgets offer to any app for free on any platform.
I don't like how those textboxes get bigger when you click into them.
Would it be faster than iced or Slint at rendering scrollable list with thousands of rows with images and text? For the sort of use case you’d use a “virtualized list” implementation for React for example.
I guess that egui works similar to Dear ImGui: only (font) texture changes and clip regions (e.g. scroll areas) require a new draw call, and that's typically just a handful to a few dozen draw calls even for complex UIs.
For long lists, Dear ImGui has a special 'ListClipper' class which allows to iterate over the visible 'slots' of a list, so that the code which describes the list UI can skip clipped items early. Not sure if egui has something similar.
An egui ScrollArea can request only visible rows from a callback using show_rows:
https://docs.rs/egui/latest/egui/containers/scroll_area/stru...
Btw: when clicking any 'task' to get to see the details, I see a repeating pattern of almost vertical stripes in some colors. I guess that's not what it should look like.
That may not have been clear, but I am talking about drawing less nodes in any case, also when using egui.
In any case, it looks like a good base for further optimisation to ignore offscreen UI elements early (e.g. not sure if egui has something like Dear ImGui's ListClipper).
I bet the canvas approach would not slow-down the notebook.
Alternatively, try pygfx for ThreeJS graphics in Python leveraging wgpu. It works great in Notebooks through notebook-rfb. https://github.com/pygfx/pygfx
If you're adventurous, figure out how to make pygfx work with webgpu via wasm
https://github.com/bokeh/bokeh
I did do a basic test, and the raw rects-on-screen performance is roughly comparable to my final solution.
WASM demo here if you want to see it (with dummy, and not particularly realistic, profiling data): https://elliottslaughter.github.io/legion-prof-viewer/
For comparison, the UI I'm replacing falls down with about 16 nodes worth of data. (The demo has 8192 "nodes" worth of fake data.)