If something is a repetitive habit that you can do almost without thinking, there is a good chance an AI could infer that entire chain.

I think what's more likely is that an AI based interface will end up being superior after it has had a chance to observe your personal preferences and approach on a conventional UI.

So both will still be needed, with an AI helping at the low end and high end of experience and the middle being a training zone as it learns you.

I think I wasn't clear enough -- these habits I'm talking about are things like "press cold water button, press start" or "press warm water button, press start" or "tap 'News' app grouping, tap 'NY Times' icon".

There's nothing to infer. The sequence is already short. There are no benefits from AI here.

But you raise a good point, which is that there are occasionally things like 15-step processes that people repeat a bunch of times, that the AI can observe and then take over. So basically useful in programming macros/shortcuts as well. But that still requires the original UI -- it doesn't replace it.

I don't know - the timer app on my oven is trivial too. But I always, always use Alexa to start timers. My hands are busy, so I can just ask "How many minutes left on the tea timer?"

Voice is not really clumsy, compared to finding a device, browsing to an app, remembering the interface etc.

Already when we meet a new app, we (I) often ask someone to show me around or tell me where the feature is that I want. Not any easier than asking my house AI. Harder really.

Hard to underestimate the laziness of humans. I'll get very accustomed to asking my AI to do ordinary things. Already I never poke at the search menu in my TV; I ask Alexa to search for me. So, so much easier. Always available. Never have to spell anything.

I think we're basically in agreement though.

Everyone agrees setting timers in the kitchen via voice is great precisely because your hands are occupied. It's a special case. (And often used as the example of the only thing people end up consistently using their voice assistant for.)

And asking an AI where a feature is in an app -- that's exactly what I was describing. The app still has its UX though. But this is exactly the learning assistance I was describing.

And as for searching with Alexa, of course -- but that's just voice dictation instead of typing. Nothing to do with LLM's or interfaces.

Alexa's search is a little different - it's context-independent. I can ask for a search from any point in the TV app - in some other menu, while watching another show, heck even when the TV is turned off.

And when describing apps - I imagine the AI is an app-free environment, where I just ask those questions of my AI assistant, in lieu of poking at an app at all.

I totally get your point, but I think that AI will allow much "smarter" behavior. Where every appliance is an expert in doing what it is intended to do.

So sure, it will still have buttons, but those buttons are really just preset AI prompts on the backend. You can also just talk to your appliance and nuance your request however you want to.

A TV with a remote whose channel button just prompts "Next channel" but if you want you would just talk to your TV and say "Skip 10 channels" or "make the channel button do (arbitrary behavior)"

The shortcuts will definitely stay, but they will behave closer to "ring bell for service" than "press selection to vend".

Most user interfaces already have a much finer granularity and number of options than your examples.

When taking a shower, I would like fine control over the water temperature, preferably with a feedback loop regulating the temperature. (Preferably also the regulation changes over the duration of the showering.)

Choosing to read the NY times indeed is only a few taps away, but navigating through and within its list of articles is nowadays done quite fast and intuitively thanks to quite a lot of UI advancements.

My point being, short sequences are a very limited set within a vast UI space.

People go for convenience and speed, oftentimes even if there's some accuracy cost. AI fulfills this preference, especially because it can learn on the go.

> When taking a shower, I would like fine control over the water temperature, preferably with a feedback loop regulating the temperature. (Preferably also the regulation changes over the duration of the showering.)

That exists, but it’s expensive because of the electronics and mechanics involved. There are so many interfaces with this exact problem.

You also almost certainly don’t want non-deterministic hallucination prone AI controlling physical systems.

Maybe I misunderstood the op, but isn’t the mechanical faucet and our body’s temperature receptors this “fine control with feedback loop”?

Indeed, and to take the UI a step further, humans often prefer automation, if it works reliable. A complicated UI would become simple, just step into the shower.

There’s no complicated UI. You just turn a knob that sets a digital temperature readout.

If you want the shower to save your temperature preferences and start automatically, there’s no reason to build in a computer capable of running an AI.

But in reality you almost certainly don’t want a system like this because you don’t want an AI accidentally turning on your shower when you’re not home, when you do ok to clean it, or grab a razor, or when your toddler wanders in.

Granted an AI could try to determine intent, but it’s never going to get it 100% right. Which is why for physical systems like this you almost always want a physical button to signal intent.

Prompt engineering and using multiple AI models in parallel might find ways to cancel out most hallucinations similar to how consensus-based replication works.

It might. If hallucinations are truly random and not correlated to anything shared between models. For example, something inherent to the data they are trained on. Given how locked down I think potential training data is going to become, and the amount of data required, I think that sharing data between models is almost guaranteed.

Also that sounds like an awful lot of computing power for everyday UIs. It also doesn’t solve the non determinism problem.

It would become less expensive, using less sensors and actuators, when using the predictive and learning abilities of an ai. You can, for safety reasons, keep a mechanical temperature limiter in the loop.

My background is in embedded systems. There is absolutely no way that an AI could make a temperature controlled shower cheaper.

There’s no way to use fewer actuators, and the control system is already dead simple, and uses one temperature sensor per outlet.

Temperature can be measured in different ways. IR radiation and sound can be measured from a distance. The relationship between temperature at the source, of the water exiting the showerhead and time can be learned. Water can be heated in different ways. The valve could also be a pump. Our reaction to the temperature of the water can be sensed.

Who knows, AI can come up with simpler or cheaper solutions that did not cross our mind. I would say, time will tell.

AI might also invent sonic showers, or teleportation to beam the dirt away.

The lack of AI isn’t what’s holding back your dream of a temperature controlled shower in every house.

Like on Apple Watch Ultra, when I press the start exercise button, the exercises that it suggests are in an order unique to my baseline activities.

Not… “write my works cited page… use hanging indents”?

Don't think it of it as the machine performing repetitive tasks you specify. Think of it instead as the machine accomplishing goals you specify, by figuring out the tasks necessary for accomplishing them.

For example, instead of telling the machine something like, say, "increase the left margin by a quarter inch," you'd say something like "I want to create a brochure for this new product idea I just had, and I want the brochure to evoke the difficult-to-describe feeling of a beautiful sunshine. Create 10 brochures like that so I can review them."

This presumes, of course, that the technology continues to improve at the same pace. No one knows if that will happen.

There’s no reason chatbots have to be the interface to an LLM. Imagine dynamically generated interfaces redesigning themselves to your needs as you work through a task.

Exactly. Chat is just the first, most general purpose UI. Like a terminal was for computers.

Agreed but lets keep in mind that graphical UIs involve significantly more effort than CLIs. In the case where you shoerhorn an ai to be your primary UI I would bet this is as hard if not harder since you typically have to code around significant limitations of current llms

http://gitwit.dev shows us that LLMs can generate various GUIs just fine, when prompted.

This is not what I said. Sure you can use such tools or even just classical boilerplate scripts (like we used for a decade now) to get started with react fast. But building out a system that fails well when the underlying llm starts behaving erratically or not at all is a completely different league of engineering as executing a boilerplate script.

Sorry for misinterpreting you. So the underlying LLM starts misbehaving, and the difficulty you see, is that the system as a whole should fail gracefully. What would that look like, in your eyes? A proctor LLM/whatever that observes the output and decides that it has gone awry and decides to take over?

So to your point, I do think it would be nice to have an AI assistant suggest to make changes to the view mode of an application. For instance, if I navigate to a certain area the AI might suggest that I make that option visible in my menu. I might say no, because I prefer to use the hotkey. Aside from that, I'm going to have to absolutely side with the parent commenter on this one - When I'm doing serious work I want a familiar interface that I know how to move around efficiently in. I don't want any surprises.

Suggestions to improve workflow sound great. But nullifying hard earned knowledge of an interface... I can't see that helping me.

The shining example in my mind is audio/video/graphics applications, where there are good reasons to routinely switch between different views. Knowing your way around those views (which might be custom, but still static), and being able to navigate through them quickly is very valuable.

I think there’s room for an AI-native interface that knows the full context of the project you’re working on and can make extremely intelligent suggestions or tailor the interface for a specific task. Or you could just literally describe the interface you want for something and it redesigns it for you immediately.

There is room, absolutely. Perhaps I took too hard of a line. I do recognize that a compromise can be made between a user retaining their knowledge of a system, and that system evolving to accommodate user requirements. It's easy to be fearful of the rug being pulled right from under you, but I suppose the reality might be closer to earnestly asking to place things upon the rug. In other words, the things that are familiar will probably remain familiar and accessible, but AI will help us arrange them in a way that optimizes our workflow.

I would hate that. An interface is useful when you can anticipate and remember what things are available and where they are. It doesn't matter if it's a visual UI or keyboard shortcuts. If you start moving stuff around on us, we revolt. (See every UI update on an MS Office product ever.)

Well you have to make it useful obviously. Keep an open mind!

So, an interface that keeps changing randomly? That sounds like hell, no thanks.

As we all know, the one human skill computers can never replace is doing repetitive tasks quickly.

This is the Dropbox comment of our time.