macOS uses a PDF-like canvas as the core display engine^, upon which everything writes object layers that have refresh loops independent from the main canvas 'flatten layers into the display' loop. This is all inside WindowServer, and is why <Shift-Command-4> <Spacebar> lets you select an entire window including the shadowed border, without picking up anything from above or beneath it, because Spacebar activates object selection mode.

And we know that Apple has figured out secure pairing of T2 chips for external accessories (the touchID keyboard), so they can run secure-trusted components outside of the main hardware.

So they could in theory run a WindowServer process on the display's internal Cortex chip, paired via T2 to provide verifiable security, hosting the WindowServer canvas for that display and lets the host device offload the 'flatten layers to display' problem, handle the EDR mapping, and so on. That would be a boon to laptops and also to thermally-constrained desktops, and here's why:

On my Mac, WindowServer is the highest user of CPU time in total, exceeding kernel_task, coreaudiod, everything except Zoom (which does non-accelerated H.264 encoding/decoding on x64). So if I were to plug in a second 5K display alongside my built-in, my WindowServer CPU usage would double. Fortunately, I can afford that — but maybe a laptop cannot. Offloading that WindowServer burden for managing traffic to/from the LCD, and for dealing with layer flattening and security property management (you can't screenshot certain OS dialogs with a specific security flag set!), would be a tremendous win not only for battery life of the host, but also a theoretical performance win for latency, as now macOS would only need to shuttle specific layer changes over the wire.

Of course, it could still act as a dumb display for any host that isn't ready to negotiate a T2-protected WindowServer connection, but my idea above would definitely go a long way to explaining why Apple dropped support for x64-style eGPU while still investigating hardware-accelerated monitors.

^ it used to be implemented as PDF internally, not sure if it still is or not

^^ you used to be able to kernel panic macOS by making a window so tall that, when screenshot using the Spacebar layer selection method, an out-of-bounds write would occur from within WindowServer

I have rarely ever learned this much from a single comment here.

I do my best, I just hope I got everything right enough. I should add: EDR is https://developer.apple.com/wwdc21/10161

This comment here is what I come to HN for.

I think that OS X was the first OS that did hardware compositing. How come that WindowServer consumes so much CPU instead of GPU?

I didn't closely check for "is it CPU+GPU or CPU only" when I looked at Activity Monitor yesterday. I could, but I suspect that isn't a stable answer year-over-year yet.

The CPU isn't compositing, it's rendering 2D objects, I think—text, vector graphics, etc.

> as now macOS would only need to shuttle specific layer changes over the wire.

All this makes me think they're about to introduce wireless connection to display, akin to what is now possible with Sidekick and an iPad, especially given the recent improvements in Continuity.

Oh, that's a dreamy idea. I sure hope so.

.. or iPad. Many professional creative types use iPads in their professional work. Being able to hook up to a high end display wirelessly would be a great help to this segment of professional customer.

Perhaps ironically the original Apple Watch wasn’t much more than an external display for the iPhone.