When I open two .doc files, I can only have two separate windows or tabs. But I don't need those windows or tabs. What I actually need is just two open files on the screen. And some Word functions. From UI perspective, it can be a "function bar" on top of those two files.

If I want to open an image file and Photoshop, currently I can only have a separate Photoshop window. What I actually need is just an open image file on the screen. And some Photoshop functions - a "function bar" on top of that file.

So the visual difference is that you now have 3 different open files and some "function bars" - all in one workspace. And you can arrange or scale your files in any way you want. And you can arrange "function bars" on the screen or hide them.

But, whatever you do, you need some kind of borders to understand where one document ends and the other begins, and you need some controls to change the view of each individual document (such as minimizing or maximizing it) and you need a button of some kind to close the document. Then, if we have two documents and floating bars that you can put wherever you want on the screen, it will be completely impossible to tell which bar controls which document (or displays the state of which document), so we should probably put the function bar somewhere in the document container. And, hey presto, we've reinvented the Window, because it turns out to be a really really useful and natural abstraction.

Many websites break down when you scale them beyond. For example, news.ycombinator.com goes out of screen at >200% on my screen.

My terminal default font size is 20, but I often scale it above that when lines output is short enough for a period of time.

I love big output without fluff like task bars and clocks and other clutter.

What time is it? I don't know, I'll check my WRIST WATCH! :-)

Why would you want this? Apparently, everyone these days likes watching movies and such on vertical screens, so that's the way the industry needs to move.

Large laptop screens (>15") can be sufficient here, but two external monitors are priceless.

or at least the Acme editor, which is also available for Mac and various Unix systems like Linux, FreeBSD etc.

https://github.com/9fans/plan9port

It became unbearably obtuse to use on a laptop, and I didn’t like any of the workarounds, so I eventually upgraded my “terminal” to Linux and a custom LXDE setup.

e.g. You have a TSV file which you might want to align visually and edit multiple fields simultaneously, so you enable `tsv-mode', and then you enable `multiple-cursors` and you have the desired editing functionality on that file.

But to answer your first question: I genuinely envision my ideal future desktop as being some smart-glasses kind of thing, where I can type by thinking. Right now, what my mind wants and what my hands want are often aligned, but my mind wanders. If I can type at the speed that I think, I have the impression that this will greatly increase my overall focus and mind-discipline.

(I can also envision employers forcing their workers to install intrusive mind-type monitoring apps, to ensure that they're actually working and not just drooling into space, but that's perhaps a separate discussion.)

You should be able to tell the AI engine something like "as soon as the weather forecast predicts snow, order a snow blower from Amazon". The AI can only do this if it has access to weather data and an API from Amazon it can control to get you the snow blower. It should also be able to check your calendar, because you can't accept the delivery if you are on vacation.

Apps are silos that prevents the AI from doing these things. At best the AI could use the equivalent of screen scraping to use existing apps, but eventually all apps should be replaced by APIs that just extend the capabilities of the AI engine.

And what about GUIs? I think they will be pretty irrelevant. The AI engine should be able to render your data any way you want. There will certainly be designs to chose from, but you should be able to customize them any way you want ("remove that italic button, I never need it").

>At best the AI could use the equivalent of screen scraping to use existing apps, but eventually all apps should be replaced by APIs that just extend the capabilities of the AI engine.

Why would companies want to expose APIs in this manner? Instead, they can force you to access them only through their app, and that way they can force you to sit through advertising, and otherwise control how you see the information and how you're allowed to interact with it. Open APIs cut them out of a lot of profits.

>You should be able to tell the AI engine something like "as soon as the weather forecast predicts snow, order a snow blower from Amazon".

This is crazy: this way, someone could just have their AI do these tasks in an automated way, without paying some company for the privilege.

Instead, what we need is an app that's the only way to access these APIs, and the app will give you ways to set up automation, but only in the ways the company wants you to, and poorly. Then they can charge you a monthly subscription fee for access to this automation app, and also embed annoying advertising in it.

They don't. But why would record companies want to license their music to streaming services instead of selling CDs? If the AI is there, and it offers a superior experience to users, the companies that don't support it will disappear.

> this way, someone could just have their AI do these tasks in an automated way, without paying some company for the privilege.

You're assuming that APIs need to be free, but that doesn't need to be the case. Companies that provide data will need to charge for that, just like record companies charge Spotify when you stream music.

Companies that provide a service (Amazon, Uber etc) won't be happy that they lose marketing opportunities and brand recognition, but again, if the user experience is superior to the old app ways, those companies that don't adapt will die.

For that to happen, the AI needs to already be working. If the companies proactively prevent this from happening, then users won't have any basis for comparison. Closed APIs will prevent them from having the ability to program their own AI to do these things.

>You're assuming that APIs need to be free, but that doesn't need to be the case. Companies that provide data will need to charge for that, just like record companies charge Spotify when you stream music.

Sure, companies could require you to have a subscription to access the API, but they can make more money by controlling the experience instead of just giving you (paid) access to the API.

>Companies that provide a service (Amazon, Uber etc) won't be happy that they lose marketing opportunities and brand recognition,

Yes, this is exactly my point.

>but again, if the user experience is superior to the old app ways, those companies that don't adapt will die.

Users won't easily go backwards to proprietary access when they're used to (more) open access to something, but this isn't the case in this scenario: the users haven't had easy access to APIs before in a way that a regular person can just, for instance, give a voice command to their system saying "when the weather is like this, do these things with these various services", so they don't know what they're missing. The services already control the experience, and will want to keep it that way.

I don't think that the AI will be primarily developed to replace apps. I would rather expect Alexa or Google Assistant or Siri to start supporting arbitrary APIs.

> Closed APIs will prevent them from having the ability to program their own AI to do these things.

I would expect that a company like Google is able to find partners, but if not, there's always screen scraping.

> but they can make more money by controlling the experience instead of just giving you (paid) access to the API.

Not if people stop using the controlled experience. It just takes one competitor to offer the API.

They can shut that down with laws banning the practice. It might not stop someone working on a personal project, but it would prevent any significant company from offering any kind of paid product or service using screen-scraping. In fact, it might already be forbidden just by ToS.

If you're a company, that's not a bug, that's a feature. Data is your asset and releasing it for free is almost always a net loss.

Actually one more way towards few companies controlling the internet. Because having an agreement with one supplier is more easy then contracts with many.

There is absolutely nothing in the above declaration that requires an AI.

The Affinity Suite has a nifty feature where, if you own the other packages, you seamlessly can edit a photo from Photo in Publisher for instance. Or a design in Designer from Photo etc. nothing loads, it just works.

For example in Windows you can drag and drop file(s) between windows to move those file(s) between directories. In my application you can achieve all the same interaction which may move those file(s) to a different directory, different personal device, or different user. The experience is the same whether the application is running on Linux, Windows, or OSX.

In this application when you double click a file, in the browser to execute it, the OS default application associated with that file type opens just like in the native OS. If that file is on another personal device or another user it is copied to the local device for execution.

In order to minimize clutter I have a button to minimize all windows to a tray, but that is hardly original. Soon I will start working on a terminal emulator (also displaying in the browser) for remote terminal access to personal devices.

I know many people don’t like interfaces written in JavaScript but it is portable to things like Electron and Tauri. The test automation executes in less than 10 seconds on just the local device or just under 2 minutes for full peer to peer testing. Because this is all JavaScript there is a bunch of low level concerns I don’t have to reinvent, like accessibility, but the socket based decentralization does require new transmission handling and some original security work.

My WFH environment I use every day has three screens stitched together for 7680x1440. On the left third is my primary remote desktop app that connects me to work with a window for my old desktop PC. The middle third has 320pixels reserved on the right for context-sensitve notes, then splits the rest in half with the left half being my timesheeting/ticketing app and the right half being a decent plain text editor as a scratch pad / diary combo. On top of that I have a set of tabbed windows (using Groupy) from various apps like a file manager, browser, Slack, email, or command prompt. The right third has the left 320 pixels reserved for a music player (Winamp), VOIP softphone, and clipboard manager. The rest of that portion is another set of tabbed windows usually at least including a second remote desktop session to my VDI and another browser window. My browser and command prompt are tabbed. My file manager is dual-pane and tabbed with a preview pane.

What you're describing sounds like what was tried and failed with Windows 8.

Longer term, I think voice + AI assistants can help with this. In programming speak, our UX will become more declarative and less imperative. Combined with AR/VR interfaces, this will see a big improvement in productivity.

This idea of a "universal container" as strong similarities to Apple/IBMs "project pink" and OpenDoc [1] in the 90s that went on to become Taligent [2] that went on to become... not very much. There was also Microsoft's roughly concurrent efforts with OLE [3] and Windows NT Cairo - OLE was a stab at what might be described as componentising application functionality at the UI level, and Cairo had (iirc) the ambitious goal of replacing (augmenting?) the windows file system with a semantic data store with OLE/ActiveX as the UI layer.

OLE did kind of work, and actually made it into products like Word and Excel - though whether that was a good thing is debatable. The rest of it basically got canned, probably because it was too difficult to implement and/or sell. But I'm sure the influence lives on to some extent. I'm not aware of any serious attempts to go down this route again.

I still have some OpenDoc SDK beta CDs somewhere. It might be interesting to get them out and have a look.

Personally I think that "application" as a discrete concept provides a useful conceptual boundary [4] - although I will admit that this may just be that my brain is by now hard-wired for it.

[1] https://en.wikipedia.org/wiki/OpenDoc

[2] https://en.wikipedia.org/wiki/Taligent

[3] https://en.wikipedia.org/wiki/Object_Linking_and_Embedding

[4] Both for people using the application and for the grubby activity of selling it to them. Got to have a "thing" to sell.

Fascinating responses thus far.

We're in a terrible state, some odd and ugly place between native and online, where even native apps are picking the worst from online and online is denied access from the best of native.

I want an operating system GUI that dictates the available widgets to the program, and hands me control of their presentation, I don't want applications, online nor native to be able to dictate their own look, only for them to present their available functionality to the OS, and have the OS compose the screens for me, probably guided by my predefined preferences, sort of how HTML could have been if CSS hadn't ruined the day, sort of how native winforms programs used to be done.

A concrete example is things that can save a file, I don't want the program to dictate the gui for this, I want it to keep the OS updated "I have these documents open which have been changed since last save" or "There are these things which can be printed", and have the OS dictate GUI for activating that functionality.. It used to LOOK that way, for a lot of things, you'd go to the file menubar, and there you would open/save/print your file, this was easy to learn and very consistent..

Currently, the most consistent interface experience on my computer is bash in xterm, in elinks, websites look pretty consistent, if they're well designed.

But if I open firefox, chrome, gimp and kate, nothing looks or feels the same.. Then I open blender and a fifth kind of interface appears. No thanks.

So I sort of agree with you that it'd be nice if we did away with the window metaphor to some degree.. But I was also no fan of the way Mac had the "god bar" at top (which I believe Ubuntu did too) terrible idea..

For now, the least crappy desktop experience for me is i3, but it does not change anything fundamental, it's still program windows, free to do whatever they want.

But, for some things, the most futuristic version of the environment is the one that does not need a screen.. The tasks that are silently done for you without needing you to transfer any significant amount of information between you and the machine, or being able to do so on another level than a screen (brain machine interfaces and whatnot)

I think the "dynamic-ism" of many interfaces has gone too far, therefore losing the predicability. Some interfaces try to guess what I am supposed to do next and it is correct maybe 60% of the cases.

But even when it is correct it actually slows me down because I am constantly second-guessing the interface and double-checking that what I am about to press is the one I actually wanted.

I wish I could rely much more on muscle memory and that things re-open where I left them. Like a window from a folder _always_ opening exactly where it was when I closed it previously. Or the homekit buttons on my home iOS pull-down screen not rearranging themselves randomly.

I want there to be places for things, a fairly rigid system so I can depend on things being in their place, not having to constantly "try to understand what's happening now"

My browser can register that it can also display files of those encodings, and so I could effortlessly open a "viewer" in context of my text-editor, have it be my browser, and then when I edit the text-file, it is automatically shared (maybe even accessing same memory by pointer if I'm living dangerously) with the browser so I get live preview of my html.

The preview functionality in this case is not something that's explicitely supported by the text-editor, nor the browser.. It's just that since my OS already knows that the editor provides a source of text, and that the browser can view a source of text, then I can chose to open a browser perspective into the editor-source..

It might be that the operating system provides a "save file to disk" source, that I can then use to save the file from any program that provides a source of data.. It'd work the same way, instead of opening a browser as a perspective into my editor-text, I'd open a file destination into it, or I could open a LZ77 compression destination into it, which then opens a file destination, and my file is automatically compressed and saved, no program needs anything but tell the os what it can read and write..

When I was developing Palapeli (a puzzle game that ships as part of KDE Games), I did a KParts version of it for fun in a few lines of code. That made it so that if you had a puzzle file stored in a folder, you could navigate to it in Konqueror and the game view would come up inside of Konqueror.

Back in the day, I dreamed of a fully document-oriented computing experience. Sadly the world has settled on the service-oriented experience of today, largely controlled by a small number of gigantic fiefdoms.

The GUI would need a similarly smart approach and given an OO persistance layer I'd say that applications would just be objects that composed other objects - the document object would be your wordprocessor and it would be good at arranging and presenting other types of objects into 2d representations at A4 size.

The web browser would presumably just be a superset of the document object or might not even be needed at all as a web page is just another kind of document.

You'd still need a "fresh sheet of paper" concept but also a history so you could go back and search for what you were looking at/working on previously. Also different contexts - like all the stuff you need for one bit of work and another context for some other kind or play/work. I don't think windows are the worst way of doing this and I'm certain that you'll end up wanting to look at different things side by side or whatever and that will become windows by another name but there might be subtle improvements.

https://www.apple.com/newsroom/2022/10/macos-ventura-is-now-...

Still, it felt uncanny the same domain of discussion was invoked just hours apart.

Suppose you work on a research and you have 10 different .doc files. Currently, you need to open 10 separate windows/tabs. Can we have 10 different files all stacked together in one window/tab? I think it will dramatically improve productivity, since you don't need to arrange windows or move back and forth between them. You just have 10 files in 1 window and you can arrange them any way you want.

There should be some functions to work with those files. Currently, in Word you can only have function on the top. But I more prefer the way its made in Adobe Illustrator, where you can arrange functions in any way you want - on the left, right, top, bottom.

Just imagine - all 10 files stacked around the screen and some function bars on top of them. That's how I see the future OS.

What I would like to get rid of is the constraint of the limited screen area. There were some attempts, but I would really love to see a set of AR goggles that would work as an "unlimited screen", allowing me to open windows and position them anywhere in the "space" around me. The windows would remain in the spot where I dragged them to, and I could get them "closer" or "further away" as needed.

Combine this with a rotating char with a built-in keyboard and a pointing device, and this is my ideal workstation. Note that the goggles wouldn't be VR - I want to be able to see my environment behind (or "through", as per configuration) the windows, so I can still interact with people around me.

Most everyone here on HN should be considered a power user and are in the minority of personal computing users statistically. A good idea is to give people options to configure things how they like so power users can feel more powerful. I think the reality is most people fool themselves with any perceived productivity benefit on window/screen management - its just a comfort feeling more than productivity. If people truly wanted to be more productive they'd spend less time on internet forums :D

This sounds a bit like a concept that was popular in the 90s, I think it was called document-centric computing. Microsoft's implementation was called OLE and Apple's/IBMs was called OpenDoc.

The idea in these systems was that you don't really work with apps anymore, just generic document canvases into which you can drop arbitrary components. There is also a generic "compound" document format which the components can share in a hierarchical manner, and controls/menus/toolbars appear and disappear on demand.

This vision was somewhat impure on Windows, in that there was no generic document app - instead you were expected to use one of the Office apps as the canvas and then embed objects into those documents. OpenDoc in contrast had a sort of blank canvas system iirc, and if you wanted to work on a word processor document then you started a generic document and dropped a rich text area into it. A bit closer to visual programming than the app-centric model.

Suffice it to say, this model didn't win out. Embedding complex objects into compound documents turned out to have a lot of practical problems, OpenDoc got de-staffed when Jobs came back to Apple and OLE was steadily de-emphasized in Windows development. It was never that popular anyway, and was seen mostly as an exotic feature you had to do if you wanted your app to be embeddable in Word or Excel. The technology lived on somewhat in the form of OCX controls, which were for a time Microsoft's solution for shared libraries that you could use in a visual form designer and across languages.

You could try again but you'd probably want to study those earlier systems to learn why they didn't take off, and bring a new spin to the idea.

One problem with current systems, and even more so with mobile, is that it is app-centric, and your data is locked into apps, rather than being something you can manipulate directly and independently.

I had a similar idea in the 90s/2000s which I called 'data-centric' (which now means something different, in gamedev).

The idea was that instead of a world of apps, which contained the data items you cared about (emails, documents, etc), or even files, that were tied to applications, you'd have more granular data items and collections of items, and ways to organise and present them. And then functions that operate on different types of data item, or collections thereof.

An 'application' would be just a collection of functions, and completely customizable by just adding new functions or removing ones you didn't like, at least at the UI level.

There have been a number of projects roughly related to this idea I think, but things like OLE don't count IMHO. That was less like 'object' embedding , and more 'entire app embedding', and in any case 'embedding' is not really the point. None of these projects have succeeded, even though I think the idea still has merit.

I can/could envisage many problems, eg data type classification/standards, incompatibility, standards bloat, etc, but the latest AI techniques might be a way to work through some of those issues.

This sounds a bit like what you are after.

“One example that I use often — the Photos app on Mac has a built-in editing interface, but also supports extensions that allow third-party apps (e.g. Affinity Photo and Pixelmator) to take over that whole UI and present their editing UI inside the Photos app.”

Honestly, I still wish for alot of the aspects of the 1990s tried to sell like being able to develop as easily as on a NeXT workstation (without objective-c) or the entire concepts MSFT was trying to do with Active Desktop.

We see it all the time where people get excited about whatever screen is on XYZ show until you learn the reality of the screens being "Hollywood OS". Like most of the screens seen on mac computers during the 1990s, it was clear and visually functional for whatever they portrayed - like ever seen a usenet browser like shown in Mission Impossible or The Saint? Neither have I. But it would have made it a hell of a lot more user friendly. And usenet still exists!

Just saying, there is a time when less is more, but other times you need it to be just a smidge more than it is today.