With Stage Manager on macOS now, it feels like they have all the primitives in place to "transpose" macOS stage manager windows textures to Vision OS/ the iPad OS foundation.

Though this will be tricky to get right for all apps. Will be interesting to see if it's a macOS App store only feature/ API, opt-in, or some other option

They just haven’t done it.

The first iphone didn’t have copy/paste.

Apple will always prioritize critical scenarios over nice to have. None of these things are technically difficult, it’s just time. I’m willing to believe they released too early, but at some point you have to start learning from real users.

What fools Apple engineering management must be, then!

Compositor work isn't tremendously hard when the graphics primitives are already done.

UI state handling and input is a majority of the work from there. I've implemented this work before. A large portion of my background is in compositors and UI primitives.

What's really cool about having window backing layer handles, is you can do all sorts of crazy fun stuff in the office and show off to your coworkers or tech demo that basically will never make it to production because it's totally impractical.

The best example of that which actually did end up in a final product in my opinion was Windows Flip 3D. Totally fun implementation, I'm sure.

Try it with a full suite of Mac Apps and you’ll find it falls apart because they aren’t all well behaved.

You are making this sound much more complicated than it really is.

The trivial case is simple. You are imagining it's all the trivial case.

Remember the original comment was the claim that Xorg could easily do remote windows years ago. This is only true using the low performance path that modern operating systems have universally rejected.

In fact, there's already a project for it on GitHub. https://github.com/saagarjha/Ensemble

480 lines total, including comments, headers, whole shebang.

It's all of the edge cases and UX refinements that takes time.

- [Profit on basic innovation] Did they want to wait and see how their customers would adopt VisionOS's native free-floating windows, so as to avoid cognitive overload by commingling with MacOS windows?

- [Benevolence to fellow competitors] Did they not want to takeover the existing market of virtualized VR desktops?

The Vision Pro is effectively using AirPlay to mirror the whole screen. If you used AirPlay to mirror each window as a whole screen, you'd run out of bandwidth pretty quickly.

The windowing system in MacOS, Quartz Compositor, also isn't built to stream window information. Right now it has a big built in assumption that any windows its displaying are on a screen it also controls. It was probably too big a lift across teams to also re-write the graphics stack for MacOS for the launch of the Vision Pro. Hopefully they get it working in the future, but neither of these problems are easy to solve.

Literally nobody has done it. It's beyond ridiculous that you can't already show or duplicate an application window on any display you want and allow it to be controlled from anywhere it is visible.

Searching for ways to do this lead one into extremely niche software ecosystems. Please is there any collaboration app out there that makes it seamless to toss windows around like everyone actually wants?

This would mean the goggles would be basically just a dumb display for the Mac. It would also be weird to try to move an AR app onto your Mac.

Even classic Mac OS, which wasn’t designed to be “rootless” could be run that way could have its windows mixed with OS X windows in the classic days.

As others have pointed out, there is already a PoC app that is doing it, so it seems if Apple wants to, it’s completely in their power. However, does this match their vision? (pun intended) Time will tell.

EDIT: -5* doesn't make sense, this is the most polite way you can point out that getting macOS apps windowed on visionOS has ~0 to do with double-buffered windows on iPad OS. n.b. I didn't use half-baked, OP did.

I use iPad Pro as a kind of sidecar daily driver, in the magnetic dock magic keyboard w/ trackpad.

As I type this, the screen shows a traditional MacOS style dock across bottom, four Stage Manager window clusters I can tap with a thumb on the left, and Safari plus Messages taking 2/3 and 1/2 of screen respectively.

There's more app and pixel real estate than most Windows laptops, and bringing screen sets to the foreground or swapping them back to the side is so natural I almost feel like giving up that space on my Mac as well.

The big thing I saw happen from apps over the past two version of iOS is app devs realizing their windows will not always be full screen or 1/2 screen size, but arbitrary size.

By now, most iPad apps of any serious nature are effectively window size independent, making them play well with others in stage manager. It's easy to see how that would make them play well with the headset one day.

It feels like the Vision Pro would definitely be a great replacement for people who (want to) buy multiple expensive monitors, but it doesn't fully reach that potential today, and mostly because of software? Although rendering 3 or 4 virtual workspaces through ad-hoc Wifi at 4K 60fps+ low-latency would certainly be a huge challenge.

But this is sufficient for many use cases (or at least, mine). I pre-ordered one with the idea that my main work will be on the 4K monitor, with most of my superfluous apps floating around as native visionOS apps. That's mail, a web browser, and zoom, which all have apps now, and Slack, which I could just use Safari for but may have a native app in the future.

It's more like 1080p monitor. The virtual monitor only covers a small part of the VisionPro's display. You can compensate a bit for a lack of resolution by making the virtual screen bigger or by leaning in, but none of that gives you a 4k display.

To really take proper advantage of the VR environment you really need the ability to pull out apps into their own windows, as than you can move lesser used apps into your peripheral vision and leave only the important stuff right in front of you. You also miss out on the verticality that VR offers when you are stuck with a virtual 16:9 screen.

Which is the resolution that the majority of PC users are likely using.

“There is a lot of very complicated display scaling going on behind the scenes here, but the easiest way to think about it is that you’re basically getting a 27-inch Retina display, like you’d find on an iMac or Studio Display. Your Mac thinks it’s connected to a 5K display with a resolution of 5120 x 2880, and it runs macOS at a 2:1 logical resolution of 2560 x 1440, just like a 5K display. (You can pick other resolutions, but the device warns you that they’ll be lower quality.) That virtual display is then streamed as a 4K 3560 x 2880 video to the Vision Pro, where you can just make it as big as you want. The upshot of all of this is that 4K content runs at a native 4K resolution — it has all the pixels to do it, just like an iMac — but you have a grand total of 2560 x 1440 to place windows in, regardless of how big you make the Mac display in space, and you’re not seeing a pixel-perfect 5K image.”

We're pretty close to "Write once, (rewrite a bit,) Run everywhere"

even the big UI update recently as completely overwritten how the old app used to look in my memory

the only feature i have tried and not really cared for is the "Canvas" feature

edit: Yes I know you can build apps before they're in the store

Yes I know you can technically do this without a paid dev account, but it's practically useless because it has to be re-done every 7 days.

So even though you have a sequestered Mac output alongside Vision apps, you can use the same controls for all them simultaneously. This should help in the interim.

If Apple approves it, of course. This is one of my major concerns; there's a lot of potentially useful functionality that could be implemented, but you have to jump through the app store hoops and hope that Apple doesn't decide that it conflicts with their idea of what you should be allowed to do.

I think mostly people don't (for high end IDEs) because they expect a keyboard and pointing device that isn't a touchscreen and those are faily rare per ipads sold.

The ipad pro certainly is seeing some more beefy productivity apps it seems.

Windows, BeOS and Commodore 64 apps also don't run natively on the iPad or Vision Pro.

This allows you to restrict full bitrate to a single window with a maximum resolution and keep the user experience high and astonishment low.

OSX had the opportunity to follow that path before settling on the “render windows, capture the screen, compress the image, send it over the network to be decompressed” VNC-style remote access that’s bog-standard today, and if they had Vision Pro would be set up to be an absolute mind-blowing macOS experience.

It falls apart as UIs got richer, browsers in particular: they're entirely composited in-app and not via GDI, because GDI isn't an expressive enough interface. So you end up shipping a lot of bitmaps, and to optimize you need to compress them. You might as well compress the whole screen then.

https://www.anandtech.com/show/3972/nvidia-gtc-2010-wrapup/3

1: e.g. https://learn.microsoft.com/en-us/openspecs/windows_protocol...

This has been done many times before (see e.g. X Windows) and has known downsides. Off the top of my head:

- You need the same fonts installed on both sides for native font rendering to work

- Applications that don't use native drawing functions will tend to be very chatty, making the total amount of data larger than VNC/rdesktop/&c. style "send compressed pictures"

- Detaching and re-attaching to an application is hard to get right, so it's either disallowed or buggy.

I guess one downfall is that that your pipe has to be lossless, and there's no way to recover from a broken pipe (unless you keep a shadow copy of the window state, and have a protocol for resynchronizing from that, and a way to ensure you don't get out of sync.)

Applications which bypass the native APIs to render their window contents, in particular video players or games, get a compressed streamed video which has very decent performance. The video quality seems to be dynamic as well, so if there's a scene with very few changes you can see the quality progressively improve.

All of this is done per window, so a small VLC window playing a video in a corner gets the video treatment, while everything else still works like native UI.

How did you know it sends windows hooks? Was it some sort of binary serialization?

1. The receiving side has to have at least as much rendering power as the original side, since it will be the one actually having to render things on screen. This is always going to be the opposite case with any kind of glasses, where you'll always want to put as little compute as possible for weight and warmth reasons.

2. Each application actually has to send draw instructions instead of displaying photos or directly taking on control of the graphics hardware themselves. No or very few modern applications work like this for any significant part of their UI.