News for hackers. Stuff that matters.
Re:GTK: Introducing Graphics Offload by diath on Saturday November 18, 2023 @05:46AM (#38317933)
I wonder if there are plans to make it work with X11 in the future, I've yet to see the benefit of trying to switch to Wayland on my desktop, it just doesn't work as-is the way my 8 year old setup works.
Re:GTK: Introducing Graphics Offload by mnd999 on Saturday November 18, 2023 @06:39AM (#38318343)
If it worked exactly the same there would indeed be no benefit.
Re:GTK: Introducing Graphics Offload by maccard on Saturday November 18, 2023 @06:28AM (#38318256)
> I've yet to see the benefit of trying to switch to Wayland on my desktop

how about Graphics Offload?

Re:GTK: Introducing Graphics Offload by diath on Saturday November 18, 2023 @06:49AM (#38318426)
This feature would be nice-to-have but is not impactful enough (at least to me) to outweigh the cons of having to switch to Wayland, which would include migrating my DE and getting accustomed to it as well as looking for replacement applications for these that do not work properly with Wayland (most notably ones that deal with global keyboard hooks). Admittedly I have never tried XWayland which I think could potentially solve some of these issues.
Re:GTK: Introducing Graphics Offload by maccard on Saturday November 18, 2023 @08:22AM (#38319250)
I think if your waiting for a magic bullet of a feature to upgrade you might be waiting a long time, and even Wayland will be replaced at that point. Instead, look at the combination of features (like this) and think about it and future upgrades. I think you're right that xwayland is probably a compromise for now if you need it for things like global shortcuts.
Re:GTK: Introducing Graphics Offload by knocte on Saturday November 18, 2023 @05:49AM (#38317955)
I doubt they have the energy to backport bleeding edge tech.
Re:GTK: Introducing Graphics Offload by PlutoIsAPlanet on Saturday November 18, 2023 @06:01AM (#38318031)
This is one of the benefits of the Wayland protocol over X, being able to do this kind of thing relatively straightforwardly.

Once support for hardware planes becomes more common in Wayland compositors, this can be tied to ultimately allow no-copy rendering to the display for non-fullscreen applications, which for video playback (incl. likes of Youtube) equals to reduced CPU & GPU usage and less power draw, as well as reduced latency.

Re:GTK: Introducing Graphics Offload by AshamedCaptain on Saturday November 18, 2023 @08:34AM (#38319409)
> This is one of the benefits of the Wayland protocol over X

What.

The original design of X actually encouraged a separate surface / Window for each single widget on your UI. This was actually removed in Gtk+3 ("windowless widgets"). And now they are bringing it back just for wayland ("subsurfaces"). It is practically the same concept.

Re:GTK: Introducing Graphics Offload by tadfisher on Saturday November 18, 2023 @09:19AM (#38319878)
The original design of X had clients send drawing commands to the X server, basically treating the server as a remote Cairo/Skia-like 2D rasterizer, and subwindows were a cheap way to avoid pixel-level damage calculations. This was obviated in the common case by the Xdamage extension. Later use of windows as a rendering surface for shared client/server buffers was added with Xshm, then for device video buffers with Xv.

GTK3 got rid of windowed widgets because Keith Packard introduced the Xrender extension, which basically added 2D compositing to X, which was the last remaining use for subwindows for every widget.

Re:GTK: Introducing Graphics Offload by AshamedCaptain on Saturday November 18, 2023 @09:22AM (#38319903)
This is completely wrong. Xrender is completely orthogonal to having windows or not. Heck, Xrender takes a _window_ as target -- Xrender is just an extension to allow more complicated drawing commands to be sent to the server (like alpha composition). You make your toolkit's programmer's life more complicated, not less, by having windowless widgets (at the very minimum you now have to complicate your code with offsets and clip regions and the like).

The excuse that was used when introducing windowless widgets is to reduce tearing/noise during resizing, as Gtk+ had trouble synchronizing the resizing of all the windows at the same time.

[dead]
Re:GTK: Introducing Graphics Offload by AshamedCaptain on Saturday November 18, 2023 @03:01PM (#38323737)
> When you can tell Xrender to efficiently composite some pixmaps then there's really no reason to use sub-windows ever.

There is, or we would not be having subsurfaces on Wayland or this entire discussion in the first place.

Are you seriously arguing that the only reason to using windows in Xorg is to have composition? People were using Xshape/Xmisc and the like to handle the lack of alpha channels in the core protocol? This is not what I remember. I would be surprised if Xshape even worked on non-top level windows. heck, even MOTIF had windowless widgets (called gadgets iirc), and the purpose most definitely was not composition-related.

Re:GTK: Introducing Graphics Offload by play_ac on Saturday November 18, 2023 @06:08PM (#38325782)
>There is, or we would not be having subsurfaces on Wayland or this entire discussion in the first place.

No. The subsurfaces in Wayland are only designed for two things:

1. Direct scan-out, as in TFA. (Because a subsurface can be directly translated to a dmabuf)

2. Embedding content from one toolkit/library into another. (Because without it, lots of glue code would be needed)

It's discouraged to use them otherwise, they would complicate things for no benefit.

>Are you seriously arguing that the only reason to using windows in Xorg is to have composition?

If you mean sub-windows, yes, that and consequently because of the way that XRDB worked. I don't see why you would ever use them just for input events within the same toolkit, they don't do anything special there.

Re:GTK: Introducing Graphics Offload by DonHopkins on Sunday November 19, 2023 @10:27AM (#38333500)
Even XCalc has always had trouble synchronizing the resizing of all the windows at the same time, without shitting its pants.

https://www.donhopkins.com/home/catalog/unix-haters/x-window...

That's what happened when I simply resized stock xcalc several times in a row to different sizes and aspect ratios. I wonder if they've finally figured out how to fix that bug in 35 years?

That was why I was compelled to hack an X11 window manager to take a command line argument telling it which window id to treat as the root window, then ran xcalc, discovered its window id with "xwininfo" or some such utility, then ran the window manager on the calculator, putting window frames around each of the calculator's buttons, so you could resize them, move them around, open and close them to icons, etc! That was a truly customizable calculator.

Re:GTK: Introducing Graphics Offload by audidude on Saturday November 18, 2023 @10:50AM (#38320829)
Drawables on Drawables doesn't help here at all.

Sure it lets you do fast 2d acceleration but we don't use 2d accel infrastructure anywhere anymore.

Subsurfaces have been in Wayland since the beginning of the protocol.

This is simply getting them to work on demand so we can do something X (or Xv) could never do since Drawables would get moved to new memory (that may not even be mappable on the CPU-side) on every frame.

And that's to actually use the scanout plane correctly to avoid powering up the 3d part of the GPU when doing video playback on composited systems.

Re:GTK: Introducing Graphics Offload by hurryer on Saturday November 18, 2023 @06:25AM (#38318222)
No screen tearing is a major benefit of using a compositor.
Re:GTK: Introducing Graphics Offload by mrob on Saturday November 18, 2023 @06:36AM (#38318322)
And screen tearing is a major benefit of not using a compositor. There's an unavoidable tradeoff between image quality and latency. Neither is objectively better than the other. Xorg has the unique advantage that you can easily switch between them by changing the TearFree setting with xrandr.
Re:GTK: Introducing Graphics Offload by RVuRnvbM2e on Saturday November 18, 2023 @06:46AM (#38318408)
It's not unique. Wayland has a tearing protocol.

https://gitlab.freedesktop.org/wayland/wayland-protocols/-/t...

Re:GTK: Introducing Graphics Offload by mrob on Saturday November 18, 2023 @07:06AM (#38318579)
This is something that every application has to opt in to individually. It's not a global setting like TearFree.
Re:GTK: Introducing Graphics Offload by RVuRnvbM2e on Saturday November 18, 2023 @07:40AM (#38318895)
This is just untrue.
Re:GTK: Introducing Graphics Offload by mrob on Saturday November 18, 2023 @08:06AM (#38319095)
From the XML file that describes the protocol:

"This global is a factory interface, allowing clients to inform which type of presentation the content of their surfaces is suitable for."

Note that "global" refers to the interface, not the setting. Which Wayland compositor has the equivalent feature of "xrandr --output [name] --set TearFree off"?

Re:GTK: Introducing Graphics Offload by kaba0 on Saturday November 18, 2023 @12:27PM (#38321958)
Which is the correct default? No application should unknowingly render half-ready frames, that’s stupid. The few niches where it makes sense (games, 3D applications) can opt into it, and do their own thing.
Re:GTK: Introducing Graphics Offload by badsectoracula on Saturday November 18, 2023 @12:46PM (#38322189)
That is subjective, i do not want the input latency induced by synchronizing to the monitor's refresh rate in my desktop as it makes it feel sluggish. The only time i want this is when i watch some video (and that is of course only when i actively watch the video, sometimes i put a video at the background when i do other stuff) - so for my case the correct default is to have this disabled with the only exception being when watching videos.
Re:GTK: Introducing Graphics Offload by kaba0 on Saturday November 18, 2023 @02:05PM (#38323138)
Okay, and then the program will get an event, that propagates down to the correct component, which reacts some way, everything that changes due to that are damaged, and every damaged component is re-rendered with synchronization from the framework itself. It has to be specifically coded (e.g. text editor directly writing to the same buffer the rendered character) to actually make efficient use of tearing, it will literally just tear otherwise with zero benefits.
Re:GTK: Introducing Graphics Offload by mrob on Saturday November 18, 2023 @04:39PM (#38324800)
You don't need to do anything special. Just render to the front buffer immediately and don't worry about the current scanout position. If it's above the part of the screen you're updating, great, you saved some latency. If it's below, latency is the same as if you waited for vsync. And if it's in the middle, you at least get a partial update early.
[dead]
Re:GTK: Introducing Graphics Offload by sph on Sunday November 19, 2023 @11:17AM (#38334033)
The meme was "the eye can't see more than 25fps anyway", which is less than 50ms. It's a meme because it's of course incorrect. There is a huge difference between how long it takes us to acquire and react to what our retina is sensing (in the order of hundreds of ms), and keeping track of a moving/evolving shape (<5ms).

Here you are saying that latency of 250ms is unnoticeable, which is utter nonsense.

Re:GTK: Introducing Graphics Offload by play_ac on Monday November 20, 2023 @01:47PM (#38352493)
Your comment has nothing to do with the conversation. The reason to have low latency when typing text is so you can correct mistakes. That requires the full response time. There's no moving or evolving shapes. Maybe proofread your own comments before throwing around accusations of "utter nonsense".
Re:GTK: Introducing Graphics Offload by sph on Monday November 20, 2023 @06:27PM (#38356490)
Are you saying that latency in the order of 250ms when editing text is unnoticeable?

Yeah, that is utter nonsense. Just try it for yourself, instead of pulling statements like that out of thin air. You would notice a difference in tens of ms even when editing text. Why do you think people cannot stand writing with VSCode for example?

Re: moving/evolving shapes, I did not think I had to clarify that the brain is a massively parallel system with multiple modes of operation. Editing text does not require you to reprocess all visual signals from scratch, because that is not how the visual cortex works. The perceived latency when editing text is between pressing a key and your brain telling you "my eyes have detected a change on the screen. I will assume that it is the result of me pressing a key". It does NOT take 250ms to make this type of assumption, and is basically how our vision operates. It's a prediction engine, not a CCD sensor.

Re:GTK: Introducing Graphics Offload by play_ac on Tuesday November 21, 2023 @12:32PM (#38366848)
>Why do you think people cannot stand writing with VSCode for example?

Which people? Every recent study I've seen shows VSCode as the most popular code editor by a large margin. Maybe latency isn't as important as you think?

>Are you saying that latency in the order of 250ms when editing text is unnoticeable?

No. Sorry for the info dump here but I'm going to make it absolutely clear so there's no confusion. The latency of the entire system is the latency of the human operator plus the latency of the computer. My statement is that, assuming you have a magical computer that computes frames and displays pixels faster than the speed of light, the absolute minimum bound of this system for the average person is 250ms. You only see lower response time averages in extreme situations like with pro athletes: so basically, not computer programmers who actually spend much more time thinking about problems, and going to meetings, than they actually spend typing.

Now let's go back to reality: with a standard 60Hz monitor, the theoretical latency added by display synchronization is a maximum of about 16.67ms. That's the theoretical MAXIMUM assuming the software is fully optimized and performs rendering as fast as possible, and your OS has realtime guarantees so it doesn't preempt the rendering thread, and the display hardware doesn't add any latency. So at most, you could reduce the total system latency by about 6% just by optimizing the software. You can't go any higher than that.

However, none of those things are true in practice. Making the renderer use damage tracking everywhere significantly complicates the code and may not even be usable in some situations like syntax highlighting where the entire document state may need to be recomputed after typing a single character. All PC operating systems may have significant unpredictable lag caused by the driver. All display hardware using a scanline-based protocol also still has significant vblank periods. Adding these up you may be able to sometimes get a measurement of around 1ms of savings by doing things this way, in exchange for massively complicating your renderer, and with a high standard deviation. Meaning that you likely will perceive the total latency as being HIGHER because of all the stuttering. This is less than 1% of the total latency in the system and it's not even going to be consistent or perceptible.

Now instead consider you've got a 360Hz monitor. The theoretical maximum you can save here is about 2.78ms. This can give you a CONSISTENT 5% latency reduction against the old monitor as long as the software can keep up with it. Optimizing your software for this improves it in every other situation too, versus the other solution which could make it worse. If it doesn't make it worse, it could only save another theoretical 1% and ONLY in a badly perceptible way. It just doesn't make sense to optimize for this less than 1% when it's mostly just caused by the hardware limitations and nobody actually cares about it and they're happy to use VSCode anyway without all this.

So again, you can avoid these accusations of "utter nonsense" when it's clear you're arguing against something that I never said.

>The perceived latency when editing text is between pressing a key and your brain telling you "my eyes have detected a change on the screen.

Your brain needs to actually process what was typed. Prediction isn't helping you type at all, if it did then the latency wouldn't matter anyway. If you're not just writing boilerplate code then you may have to stop to think many many times while you're coding too.

Re:GTK: Introducing Graphics Offload by phkahler on Saturday November 18, 2023 @06:30PM (#38326052)
>> That is subjective, i do not want the input latency induced by synchronizing to the monitor's refresh rate in my desktop as it makes it feel sluggish.

Only if the DE or application can't keep up. Oh, it overran the frame time, so let's just dump half of it to screen...

I'm still baffled that this is a problem. I tried ray tracing a hierarchy of rectangles decades ago and it was pretty fast. Should be better than real time today, and that's doing it the expensive way. I suppose the challenge is composting stuff from different processes every frame.

Re:GTK: Introducing Graphics Offload by badsectoracula on Sunday November 19, 2023 @01:54AM (#38329908)
It is not about performance/throughput. There are various factors at play: first of all, the synchronization is basically waiting for the monitor to finish displaying the last image before you send the new one. Second, unless the application takes over the screen (e.g. fullscreen games), the applications do not run in sync with the window system (and you do not want that because you do not want a single application to cause problems for other applications like hogging the screen updates). When you combine these two there is nothing really the application can do. As you wrote:

> I suppose the challenge is composting stuff from different processes every frame.

This is a significant part of the issue, though the fact that synchronization essentially means waiting is also another one (without synchronizing you get partial frames but those partial frames are still feedback for the user that makes the desktop feel more responsive).

Re:GTK: Introducing Graphics Offload by JelteF on Saturday November 18, 2023 @07:27AM (#38318771)
Could you explain in what scenario you think it is better to have a display show two half images slightly faster (milliseconds) than one full one?
Re:GTK: Introducing Graphics Offload by mrob on Saturday November 18, 2023 @08:02AM (#38319060)
Text editing. I mostly work on a single line at a time. The chance of the tear ending up in that line is low. And even if it does, it lasts only for a single screen refresh cycle, so it's not a big deal.

And you're not limited to two images. As the frame rate increases, the number of images increases and the tear becomes less noticeable. Blur Busters explains:

https://blurbusters.com/faq/benefits-of-frame-rate-above-ref...

Re:GTK: Introducing Graphics Offload by kaba0 on Saturday November 18, 2023 @12:29PM (#38321980)
As the frame rate increases, the latency decreases, making it a non-issue. I rather chose this option, over blinking screens.
Re:GTK: Introducing Graphics Offload by mrob on Saturday November 18, 2023 @04:40PM (#38324811)
The minimum latency is bottlenecked by the monitor unless you allow tearing.
Re:GTK: Introducing Graphics Offload by kaba0 on Saturday November 18, 2023 @06:20PM (#38325939)
No, the monitor’s refresh rate is a hard limit on the minimal maximal-latency. It doesn’t mean that you are actually close to that minimum, or that it is the hard limit that is active, aka the bottleneck.
Re:GTK: Introducing Graphics Offload by Too on Sunday November 19, 2023 @01:54AM (#38329912)
Text editing is mostly about reading, not writing. Scrolling up and down code is actually one of the worst scenarios i can think of, where you absolutely don't want to have tear. Because even as the document is moving you are still reading it.

When touch typing, fingers work decoupled from the eyes anyway, unless you are waiting for the intellisense or copilot prompt, that is usually constrained by language servers anyway, not the framerate.

Re:GTK: Introducing Graphics Offload by AshamedCaptain on Saturday November 18, 2023 @08:08AM (#38319110)
Many technologies have been invented to allow to display "two half images slightly faster", such as interlaced scanning...

Most humans will actually prefer the "slightly faster" option. (Obviously if you can do both, then they'd prefer that; but given the trade-off...)

Re:GTK: Introducing Graphics Offload by andreyv on Saturday November 18, 2023 @11:47AM (#38321444)
First person shooters. Vertical synchronization causes a noticeable output delay.

For example, with a 60 Hz display and vsync, game actions might be shown up to 16 ms later than without vsync, which is ages in FPS.

Re:GTK: Introducing Graphics Offload by badsectoracula on Saturday November 18, 2023 @12:44PM (#38322156)
Input latency. I find the forced vsync by compositors annoying even when doing simple stuff like moving or resizing windows - it gives a sluggish feel to my desktop. This is something i notice even on a high refresh rate monitor.
Re:GTK: Introducing Graphics Offload by beebeepka on Saturday November 18, 2023 @02:36PM (#38323481)
Personally - gaming. Never liked vsync
Re:GTK: Introducing Graphics Offload by aktuel on Saturday November 18, 2023 @07:12AM (#38318613)
I am sorry to tell you that X11 is completely unmaintained by now. So the chances of that happening are zero.
Re:GTK: Introducing Graphics Offload by NGRhodes on Saturday November 18, 2023 @07:30AM (#38318806)
FYI 21.1.9 was released less than a month ago (https://lists.x.org/archives/xorg/2023-October/061515.html), they are still fixing bugs.
Re:GTK: Introducing Graphics Offload by dralley on Saturday November 18, 2023 @08:53AM (#38319608)
You mean, they're still fixing critical CVEs.
Re:GTK: Introducing Graphics Offload by badsectoracula on Saturday November 18, 2023 @12:27PM (#38321965)
Which are still bugs.

Also only two of the four changes mentioned in the mail are about CVEs.

Re:GTK: Introducing Graphics Offload by AshamedCaptain on Saturday November 18, 2023 @08:06AM (#38319090)
Frankly, it was X11 which introduced "Graphics Offload" in the first place, with stuff like XV, chroma keying, and hardware overlays. Then compositors came and we moved to texture_from_surface extensions and uploading things into GPUs. This is just the eternal wheel of reinventing things in computing (TM) doing yet another iteration and unlikely to give any tangible benefits over the situation from decades ago.
[dead]
Re:GTK: Introducing Graphics Offload by AshamedCaptain on Saturday November 18, 2023 @02:51PM (#38323624)
> Xorg still doesn't really have support for non-RGB surfaces

You really need to add context to these statements, because _right now_ I am using through Xorg a program which uses a frigging colormap, which is as non-RGB as it gets. The entire reason Xlib has this "WhitePixel" and XGetPixel and XYPixmap and other useless functions which normally fetch a lot of ire is because it tries to go out of its way to support practically other-wordly color visuals and image formats. If anything, I'd say it is precisely RGB which has the most problems with X11, specially when you go more than 24bpp.

> there for attaching GPU buffers

None of this is about the GPU, but about about directly presenting images for _hardware_ composition using direct scan-out, hardware layers or not. Exactly what Xv is about, and the reason Xv supports formats like YUV.

> There isn't any incentive to implement this in X11 either because X11 is supposed to work over the network and none of this stuff would

As if that prevented any of the extensions done to X11 in the last three decades, including Xv.

Re:GTK: Introducing Graphics Offload by kaba0 on Saturday November 18, 2023 @12:25PM (#38321930)
There are plenty of wheel reinventions in IT, but let’s not pretend that modern graphics are anything like it used to be. We have 8k@120Hz screens now, the amount of pixels that have to be displayed in a short amount of time is staggering.
Re:GTK: Introducing Graphics Offload by AshamedCaptain on Saturday November 18, 2023 @02:55PM (#38323668)
At the same time, you also have hardware that can push those pixels without problem. When X and these technologies were introduced, the hardware was not able to store the entire framebuffer for one screenful in memory, let alone two. Nowadays you are able to store a handful at the very minimum. Certainly there's a different level of performance here in all parts, but the concepts have changed very little, and this entire article kind of shows it.
Re:GTK: Introducing Graphics Offload by mananaysiempre on Sunday November 19, 2023 @04:24PM (#38338242)
Storing is one thing, pushing another. 32bpp×4K@60Hz works out to 1.8GB/s, which even on modern CPUs requires you to not do stupid things—e.g. a C loop with a (modest) iteration per byte will likely end up an order of magnitude slower. And that’s if you’re doing 32bpp SDR like it’s 2005; a floating-point pixel format will be four times more data than that.
Re:GTK: Introducing Graphics Offload by audidude on Saturday November 18, 2023 @10:47AM (#38320785)
This would require protocol changes for X11 at best, and nobody is adding new protocols. Especially when nobody does Drawable of Drawables anymore and all use client-side drawing with Xshm.

You need to dynamically change stacking of subsurfaces on a per-frame basis when doing the CRTC.

Re:GTK: Introducing Graphics Offload by AshamedCaptain on Saturday November 18, 2023 @10:58AM (#38320916)
I really don't see why it would need a new protocol. You can change stacking of "subsurfaces" in the traditional X11 fashion and you can most definitely do "drawables of drawables". At the very least I'd bet most clients still create a separate window for video content.

I agree though it would require a lot of changes to the server and no one is in the mood (like, dynamically decide whether I composite this window or push it to a Xv port or hardware plane? practically inconceivable in the current graphics stack, albeit it is not a technical X limitation per-se). This entire feature is also going to be pretty pointless in Wayland desktop space either way because no one is in the mood either -- your dmabufs are going to end up in the GPU anyway for the foreseeable future, just because of the complexity of liftoff, variability of GPUs, and the like.

Re:GTK: Introducing Graphics Offload by audidude on Saturday November 18, 2023 @01:27PM (#38322734)
> I really don't see why it would need a new protocol.

You'll need API to remap the Drawable to the scanout plane from a compositor on a per-frame basis (so when submitting the CRTC) and the compositor isn't in control of the CRTC. So...

Re:GTK: Introducing Graphics Offload by AshamedCaptain on Saturday November 18, 2023 @02:56PM (#38323692)
This assumes it would be the role of the window manager or compositor rather than the server who decides that, which I didn't think that way. But I guess it'd make sense (policy vs mechanism). "per-frame basis" I don't see why and it just mirrors Wayland concepts. Still, as protocols go, it's quite a minor change, and one applications don't necessarily have to support.
Re:GTK: Introducing Graphics Offload by kelnos on Saturday November 18, 2023 @02:24PM (#38323354)
I would very much doubt it. This would likely require work on Xorg itself (a new protocol extension, maybe; I don't believe X11 supports anything but RGB, [+A, with XRender] for windows, and you'd probably need YUV support for this to be useful), which no one seems to care to do. And the GTK developers seem to see their X11 windowing backend as legacy code that they want to remove as soon as they can do so without getting too many complaints.