>there is absolutely nothing that replaces physical buttons, encoders, sliders and toggles

Even space, versatility, & cost aside, there are definitely tasks that a touch screen does better. Using a map with a touch screen is incredibly intuitive compared to a mouse. (I cannot imagine using sliders or other "physical" interfaces)

>A physical button guarantees that the action was performed by the means of feedback

So you've never experienced pressing on a TV remote and nothing happening? On touch screens I can see if the app responded to my interaction. On many button interfaces I cannot.

>taptic stuff still isn't as good as the click of a button

Not sure why you're so confident with this when most people I tell are surprised that their new Macbook touchpad is entirely haptic and not actually moving.

>Don't trust a company that sells ads at the same time as building hardware

So by this definition Apple is worse than Google for privacy?

> Using a map with a touch screen is incredibly intuitive compared to a mouse. (I cannot imagine using sliders or other "physical" interfaces)

What sort of map use you have in mind? I find panning (drag+move) and zooming (mousewheel) maps with the mouse very intuitive.

Swiping and zooming a map on a phone or tablet is quicker and far more intuitive than clicking and dragging with a mouse. You can't scroll to a precise depth without moving your mouse to a specific UI slider. Whereas with touch you control it with the spread of your fingers.

One big piece of functionality missing from mouse-controlled maps is rotation. I can't imagine doing it with a mouse, but "grabbing" a map with two fingers and intuitively rotating it to the angle you want is huge.

Similarly, two-finger swipe up/down to adjust viewing angle is something that could technically be done with a slider you drag with a mouse, but I definitely wouldn't call that better.

I edited and added Macbook trackpad example before your response. I agree when its done right, it works but those are rare examples. Vast majority of products do not go to the lengths that Apple does.

> Even space, versatility, & cost aside, there are definitely tasks that a touch screen does better. Using a map with a touch screen is incredibly intuitive compared to a mouse. (I cannot imagine using sliders or other "physical" interfaces)

Have you used a space mouse? I think it is vastly superior to moving hands, pinching and getting tired after about 3 mins. CAD work requires moving things around for hours on . Pinching, zooming and moving things around with hands is completely insane to a CAD engineer - take a look at this: https://www.3dconnexion.com/products/spacemouse.html

I've been using a space mouse since 2005 for mechanical CAD. I wish I can explain what it is like to use to others because you can't. It is as if its an extension of my senses, and it feels so incredibly natural.

The space mouse is like a joystick or a thinkpad “trackpoint”. It doesn’t move around or measure movements of the body, but just uses force on it to control velocity. This is much less intuitive/effective than a direct movement. A mouse beats a trackpoint hands down for speed and precision in any context where mouse-user performance is important per se (the trackpoint’s advantage is that it is already directly under the fingers while typing, so does not require moving the hands to a separate device and reduces switching time between mousing and typing).

For just inputting rotation, the best ever tried is a trackball which senses a full 3 dimensions of rotation (most trackballs only sense 2 dimensions, with rotations in the third dimension ignored). These have only ever been made as research prototypes, but they were dramatically faster and more precise for people to use than a space mouse.

3D movement is hard to build physical devices to measure; for that maybe the space mouse is the best we can do.

I totally disagree based on experience with using a CAD software for many years. A space mouse isn’t a trackpoint. It is a 6 degrees of freedom device that feels insanely natural once you get used to it.

There is a reason why professionals working on 3D software use a Space Mouse - animation studios to Boeing. Everyone got one when you start a new job.

You should try one if an opportunity comes up. Honestly, it will change your mind forever about what it does. And it’s nothing like a trackpoint, trackball or a joystick. I wish I could show it in person to you and blow all previous expectations.

Edit: a couple of things:

* Trackpoint: 2 axis device

* Trackball: 2 axis device

* Joystick: Usually 2 rotational axes, but you can get 3 axes (twist about Z-axis)

* Space Mouse: 6 axis device

You don't need to move things physically... try flipping through pages on iPad for 1 hour straight vs. using PageUp and PageDown keys on a keyboard. See it for yourself which one is more tiring.

> I totally disagree

Nothing you wrote is in disagreement with anything I wrote, except for the inaccurate assumptions about my past experience and your misunderstanding the main thrust of my comment.

I have used several different space mice, and also taken a couple of them completely apart. I understand reasonably well how they work. They have a fixed internal part which is attached to the (slightly movable) outside shell by several springs, with little position sensors used to measure a proxy for the force and torque exerted on their outside shell. I believe the early versions used strain gauges instead of position sensors, but the result was more or less the same.

> A space mouse isn’t a trackpoint.

You completely missed my point. The part that is similar between a space space mouse and a trackpoint or joystick is that all of these essentially measure force on them, with that input converted by software to velocity (or angular velocity). If you let go they move back to a neutral position instead of staying where you put them. They do not measure the direct displacement or rotation of some physical component or body part. This is different than the input of a mouse or a touchscreen or a knob or a slider or a trackball or a scroll wheel or a dial or a stylus, but similar to the input of an analog trigger or pedal or steering wheel.

> Trackball: 2 axis device

A trackball with the appropriate sensor(s) can be a 3-axis input device (i.e. if the sensor fully characterizes the rotation). There have been several such devices made as research prototypes in various universities and corporate labs, and empirically they are much more efficient to use than space mouse for inputting rotations. However, they have never (as far as I know) been commercially produced.

This is a shame, because I would love to have one, and building hardware for myself is a lot more effort than buying something off the shelf.

> try flipping through pages on iPad for 1 hour straight vs. using PageUp and PageDown keys on a keyboard. See it for yourself which one is more tiring.

This is a non sequitur. But to answer you, I am perfectly happy to flip physical pages on a book for 5 hours straight, without my fingers ever getting tired.

> You completely missed my point. The part that is similar between a space space mouse and a trackpoint or joystick is that all of these essentially measure force on them, with that input converted by software to velocity (or angular velocity). If you let go they move back to a neutral position instead of staying where you put them. They do not measure the direct displacement or rotation of some physical component or body part. This is different than the input of a mouse or a touchscreen or a knob or a slider or a trackball or a scroll wheel or a dial or a stylus, but similar to the input of an analog trigger or pedal or steering wheel.

Yep, I did. I am sorry. I personally cannot stand using a trackpoint for precisely the same reasons as you're describing. Conversely, I tremendously enjoy using a space mouse and you would have to snatch it from my dead cold hands! It is so good, I don't know how to explain.

> This is a non sequitur. But to answer you, I am perfectly happy to flip physical pages on a book for 5 hours straight, without my fingers ever getting tired.

As compared to PageUp/PageDown? Which one do you think would make you less tired?

The physical book is a lot easier to carry to different kinds of body positions on varying furniture, walk around with, etc.

I suppose a tap on the iPad screen is probably technically the least effort, but I don’t find flipping book pages to be a hardship, and being able to write in the book margins, organize books on the shelf, etc. is nice. YMMV.

The most stressful aspect of using a typical workstation computer to read a long document is probably sitting in a typical office chair. You could compensate for that with a standing desk, but the book is still nicer IMO.

> It is so good, I don't know how to explain.

This is mostly because a 2D mouse + keyboard is quite a poor tool for navigating in 3 dimensions. A multitouch screen could conceivably be decent with a well-designed interface, but the common ones aren’t great.

> Not sure why you're so confident with this when most people I tell are surprised that their new Macbook touchpad is entirely haptic and not actually moving.

I'm not really sure I understand this, I remember a fair amount of discussion when it came out. But, just trivially, I have the latest MacBook Pro, and if I look at it from the side, it is indeed clearly moving as I press down, and not as some sort of weird millisecond-delayed sort of thing, if I press hard (for force touch), the trackpad clearly angles downward. If I continue holding down, the trackpad continues to be tipped down.

From an experience perspective, whatever the MacBook Pro trackpad does is quite different than the haptic thing my phone does, which does not feel like a press at all (and does indeed feel different in the new iPhone 11 Pro's from the previous XS's that had actually "pressable" screens).

Yeah, but what you are feeling with the "click" is not directly physical. Turn your computer completely off (not just power off, but like, "hold the power button" hard shutdown, and that "click" will stop happening.

Sure, I’m just saying that it’s really weird that this is referred as no moving parts when things clearly move. I guess my point is that haptic without movement still feels “uncanny valley” (like on the phone), but with movement (or whatever else they do different on the MacBook) feels better.

I suppose Tinder-style rapid decision apps might excel with a touch screen. Infinite scrollable timelines are certainly built for touchscreens, though I can’t think the last time I asked for one....

99% of the time the reason I put up with touch screens is so I don’t have to sit at my desk.

OTOH, a physical keyboard is vastly preferable over a touch screen one.

This isn't true for smartphones anymore. Touchscreen keyboards provide much more data that can be used for autocorrect. Also, you can't swipe type on a keyboard.

> Touchscreen keyboards provide much more data that can be used for autocorrect.

My spelling mistakes are either from muscle memory hitting an incorrect sequence of characters, transposing letters, or from me not knowing how to spell a word. How is a touchscreen going to improve any of those problems?

I find myself, when I have the choice, always gravitating to Google maps on my laptop. Enough that it frustrates me how far behind the web version is in terms of features compared to mobile.

My mouse is far more accurate than my finger and I can zoom more accurately and more quickly with my scroll wheel, it is milliliters of finger travel as opposed to a much larger, and slower mulit finger operation.

> So by this definition Apple is worse than Google for privacy?

Wait what? I think you have that backwards. Unless you’re suggesting Apple sells ads?

I have an apple trackpad device (i.e. separate from the laptop) and I believe it uses a fake haptic feedback to indicate a "click" as opposed to a real mechanism. It's indistinguishable from a real click. HOWEVER, it makes my skin crawl when the driver doesn't register my click and there is zero feedback when I press down on it. It's like pressing down on the surface of my desk and it feels absolutely gross. This happens more frequently than I'd like.

Tangental: Reminds me how buttons in various places (I've noticed this in elevators and POS terminals) beeps when you press them. However, the beep is not really tied to the CPU registering the event but rather the button itself being pressed. Ie, sometimes when you press the button, it clearly beeps, but nothing happens. Not sure if it's a result of underspecification or cheaping out on the electronics or both.

With the "Magic trackpad 2" if you have it turned off, there is no haptic feedback. At the time I didn't know it was haptic feedback and I was so confused as to how the whole click was broken.

The trackpad 1 does indeed have a real click - the way it works is that the little feet that sit on the desk are tactile pushbuttons; clicking moves the entire trackpad while the feet click in. It's disappointing this was removed in the trackpad 2; it's a very satisfying mechanism.

How is Soli any different? I know that if I put my hand in this spot and make the motion of turning a dial then it will turn down the volume, and I can hear the volume go down as I do it, whats the difference?

Or if I tap the air in a specific way, get an audible tap noise, and know I've completed a preset action (index finger tap = set timer for 5 minutes) its the exact same thing.

How is sound feedback not as good? 99% of phone interactions don't use buttons - are you saying that UX is bad?

Also - every major company sells ads and other products. Every. single. one. Google? Apple? Microsoft? Valve? Netflix? Amazon? Bueller?

Hypothetical scenario:

User pays the initial cost of engagement - Atleast 2 seconds to unlock the phone. After that, all actions user performs are in the "Engaged State".

Imagine if you have to press a button every 3 mins but you have to pay the 2 second cost of unlocking the phone, searching for the button(which can be memorized) and then getting visual/sound feedback.

Compare this with a Remote Control that is sitting on the table. Pressing a button on it every 3 mins has lower impedance.

The examples you're citing have a fixed 2 second "initialization" cost and then phones are fine as long as you're engaged in a session. All feedback is visual (and sometimes audible). Soli is far inferior to a phone in UX because it literally has no feedback besides "hoping" your gesture would get recognized. Moving hands in air has substantially higher ambiguity than touching your phone in "engaged session" mode.

On my current phone I don't have to unlock it to do media controls. I'm not sure why you'd expect anything different with Soli. Also from what I'm aware of, the phone unlocks extremely quickly. Reporters saying (partially because of Soli) that they barely see the lockscreen if picking up the phone.

Good thought experiment, though I do think Soli still makes sense. There is no reason Soli doesn't work exactly the same as a remote - there is no need to unlock the phone if the chip is always on, and can even detect when a user is engaged prior to an action.

In my mind the only difference between a button on a remote and a "button" that sits in the air above my coffee table is reliability, which will be solved in the future. I know pressing button x does y, it makes equal sense that moving my hand like x does y. Feedback could be anything from a puff of air, tiny light flash, small 'ding' sound, etc.

Sound is OK for a feedback of success. What we also need is visible affordances -- signals of what can be done, when, where, and how. Also, what I hate the most with these gestures is fearing that I'll get it close but not quite, and something else (or nothing) will happen. With a button, you can feel it and immediately make micro-adjustments while pressing to ensure success. This can possibly be done with sound, but I haven't seen it done well, yet...

Agreed - but those are software problems that can be solved with training or good ux. iPhone has no indication of what can be down (How do you know you can swipe your homescreen up, down, left, and right?). Discover-ability is a challenge that has been overcome previously and can be overcome again.

Part of that is the two games on the announcement page which show you examples of gestures. Doing gesture x accomplished y in the game, so maybe in the next app you open doing gesture x will accomplish something.

Your comment reminds of “Design for everyday things” book. He talks about Affordances and Signals.

I agree with you, sound can be ok as a signal that something has happened. It is fine with some users, but annoys others (beeps when a camera auto focuses, always a split opinion. Some people like it and some don’t).

That tv remote in the picture you linked is awful. Very little tactile differentiation to let you feel your way around, and way too many buttons.

I remember when the TiVo came out, that remote was a game-changer:

https://www.ebay.com/itm/Original-Tivo-Remote-Control-/26439...

Found the vim user ;)

That remote control is how I explain modal interfaces to people when they can't understand how my editor works.

What was modal about the remote? Switching between TV/display/audio while the main set stays the same?

Yeah. The same buttons do different things, but what they'll do depends which mode you've got selected. If you remember the remotes where it blinks an LED under the selected mode (tv/aux/DVD/whatever) as you're giving input with other buttons, that's a helpful reference.

Ahha good call.

I know it’s one data point but I never developed muscle memory for remotes the same way as, let’s say, Emacs. I always stumble and press the wrong keys. More importantly the interface on the TV is convoluted. I don’t want next channel - I usually want to go to a specific channel. With, say, Apple TV it means scrolling through a bunch of apps - much faster. Or YouTube tv via chrome cast - scrolling through the guide on the touch screen.

Same here. Except for one or two functions, I would always end up squinting at a remote in the dark to try and read the labels.

I like remotes like the Apple TV remote and the Roku remote (except for the fact that the Roku remote is a piece of utter garbage and I want to have strong words with the designer) because they focus on the essentials and let you do everything else through menus.

(The Roku remote is an utter piece of trash for a couple reasons. First, it chews through AA batteries like it’s going out of style. Second, it is not responsive—there is a agonizing delay between when you press a button for the first time and when the action takes effect. My guess is that these defects are because it uses WiFi to communicate with the device, and I can’t understand why anybody thought that was a reasonable technology to choose.)

Right on the mark here. Feedback for devices like this are going to have to involve other senses, not touch. The most obvious ones being a visual + audio feedback of some kind.

I built a project at school using a Kinect in 2011. Similar UI/UX model where you can essentially 'sense' the skeleton - we routed that data through a Node app that allowed you to swipe in the air to move through a photo album.

One of the hardest parts of this is getting what people call the 'clutch' right. Basically, how is the interface supposed to know that my arm movement is meant to target the device, and not say a friendly wave to my neighbor? In voice interfaces this is the equivalent of 'hey Siri' or 'Ok google'. With skeleton sensing interfaces we could still do an audio clutch if needed or you'll need to use another body part to engage the action on the device. Fascinating problem and I'm curious as to what clever solutions will surface.

While I personally agree with your preference, I think it's dangerous to assume that a new generation of users will have the same preferences we do despite being raised with so many new user interface paradigms.

Users tend to gravitate towards what feels "native" to them.

While agree with most of what you said, I have seen a lot of happy Face ID (over Touch ID). I think your 3 important points set a good framework for anything buttonless.

I added one more - Expose state of the system to user. Examples include:

- Vintage radios with sliding marker that shows what the current station freq is - 3 state sliders that show whether something is state A/B/C by simply looking at it. When performing action on such a slider, user immediately knows by the means of performing the action, what the state of the system is! - Volume knobs, shows the current state. - Big bright ON light when a Keithley power supply is supplying power - Lots of Braun products, too many to list.

May be these are obvious but it just reminds me that things we take granted are really important in UX/UI.

The difference between primary interfaces and secondary interactions...

Good points. Reminds me of an HN discussion about Star Trek: Voyager, where they build tactile, non-touchscreen controls for a shuttle, and there are backup tactile controls for when a crewman loses vision, for example--can still fly the ship etc.

https://news.ycombinator.com/item?id=20670810

What if one used audio feedback while using soli, eg swiping made a literally swiping noise, turning your hand clicked every notch you turned on a volume slider.

> From cockpits to nuclear power plants to home tv remote control, there is absolutely nothing that replaces physical buttons, encoders, sliders and toggles.

My anecdote/complaint related to this. My previous car had knobs, sliders, and just a few well-positioned buttons for the radio and HVAC controls. A very classic arrangement that I got along with just fine.

However, about three years ago, I bought a 3rd-gen Prius. Everything about the car is lovely except for the godawful interior controls. Evidence: https://i.pinimg.com/originals/82/44/a6/8244a6c6662a354dc7ba...

I mean, I get that they were going for a Star Trek: TNG vibe with the whole thing but they totally let form ravish function in this case. _Everything_ is a button, none of the buttons line up, none have tactile feedback to tell you what you're about to press. There are buttons that pretty much never get used, there are _fake_ blank buttons, there are things which never should have been buttons in the first place.

The HVAC controls are particularly bad:

- Except for some little bumps, there is no tactile feedback anywhere around them, so there is no way to use any of the buttons without looking at them first. Which means taking your eyes off the road to change anything. - If you want to make drastic changes to the air temperature, you have to push the button several dozen times _all while watching the display_. On traditional HVAC controls, this job is just quick twist of a knob or push of a lever, neither of which is a great risk to your life. - If you want to change which vents the air is coming out of, you have to cycle through several different options with a single button. Once you've located it, that is. And I _always_ shoot past the one I actually want, which leads to more button pushing, swearing and taking my eyes off the road for _even longer_. - Depending on the configuration, outside temperature, and phase of the moon, the car will automatically decide whether to use interior air or exterior air for the intake. I _always_ have to _constantly_ check visually to make sure the HVAC is configured the way I want it because I can't trust that the car didn't change it for me. - Also the AC will be on whenever the defrost vents are, even though the AC light never comes on. - There is a button on the HVAC panel whose _only purpose_ is the switch the air intake to interior and crank up the fan to 100% for 30 seconds or so. I guess this is the "fart remover" function. I have never seen this on any other car, maybe it's a Japanese thing. - There is a button to turn the ventilation fan off, but not on. You turn on the fan by increasing the fan speed. So if you have the fan running on, say, fan speed 3, and want to turn it off momentarily and then go back to what it was, well, you just can't.

The _only_ knobs on the whole car are at the top of the radio. The are impossible to reach without leaning forward, they are small, and they are stiff. If not for these flaws, the volume knob could be useful but the other knob is the 'tuning' knob which has essentially no purpose in 2019. The few among us who actually still listen to FM radio find their stations via the Seek buttons and program their stations to memory if desired.

And finally, the button to engage the hazard lights is at the _bottom_ of the console within easy reach of passengers, kids, your resting hand, etc. Right where something more useful (and less obnoxious) could be.

/rant

> Feedback for an action

ULtrasonic haptic feedback maybe?