News for hackers. Stuff that matters.
Re:A Developer's View of Vision Pro by Temporary_31337 on Monday June 12, 2023 @01:54PM (#36297954)
I was kind of waiting for the day where a touchscreen based app would be more productive than a traditional laptop with keyboard. Yet, after many years all we got is fruit ninja and a concession from apple in the form of attachable keyboard for the iPad. So I’m equally sceptical that the investment in headset based interface is actually worth it.
Re:A Developer's View of Vision Pro by waboremo on Monday June 12, 2023 @02:18PM (#36298389)
As usual, the devil is in the details.

We can see this with voice to text, in which despite in theory being so much faster than typing things out, tends to not be due to these details (processing lag, clunkiness of handling different forms such as whether a word is part of a command or should be added to the text).

Same will happen here. People will get over the hype period and then realize hey this isn't actually faster or more efficient than a traditional tool. Apple knows this as well, it's why they're marketing it as a media consumption device first and foremost, where a lot of these problems can be safely ignored.

Re:A Developer's View of Vision Pro by ladberg on Monday June 12, 2023 @02:08PM (#36298214)
I agree the keyboard will remain as the best text input device but I'm pretty confident that eye tracking will be categorically better than a mouse/trackpad. I'll expect that eye tracking solutions for normal monitors will also become more mainstream once people get used to it in the Vision Pro and want to use it elsewhere.
Re:A Developer's View of Vision Pro by chrisrogers on Monday June 12, 2023 @02:17PM (#36298358)
Eye tracking will probably lose the productivity metric, though. When you remove the ability to look at one thing and interact with another, you get farther from the way that humans tend to interact with their immediate physical environment.

I can cut vegetables without looking at them. I can use that dynamic to offset the planning and acting phases of my thought process. Falling short of that efficiency will feel limiting.

Re:A Developer's View of Vision Pro by htag on Monday June 12, 2023 @02:26PM (#36298543)
I don't click without looking at my mouse pointer. I think it's possible eye tracking is just better than mouse/trackpad.
Re:A Developer's View of Vision Pro by yurishimo on Monday June 12, 2023 @03:15PM (#36299406)
Interesting, I just tried to use my pointer with my peripheral vision, and while possible, it was indeed more difficult and harder to focus on. Something I hadn't tried before.
Re:A Developer's View of Vision Pro by BakeInBeens on Monday June 12, 2023 @03:38PM (#36299737)
I often am looking at another element before interacting with it with my mouse. With eye tracking enabled menus hovering over an element immediately because I'm looking at it becomes an annoyance.
Re:A Developer's View of Vision Pro by htag on Monday June 12, 2023 @04:00PM (#36300075)
Yea, I don't think a 1-to-1 translation of mouse movements with eye tracking is the correct answer here. I do think it's probable that eye tracking + {X} can be a 1-to-1 translation for clicking for 80%+ clicks.

Maybe X is a button on the keyboard. Maybe X is a gesture.

Re:A Developer's View of Vision Pro by CrazyStat on Monday June 12, 2023 @04:56PM (#36300983)
I do, when gaming.

I can think of some Portal puzzles in particular where timing is important, and you need to hold your aim but wait to click until something happens somewhere else on the screen (so the place you're clicking is not the same as the place you're looking).

I think the same thing applies to e.g. recording network activity in Chrome dev tools. My eyes are on the page to see when the thing I'm interested in finishes loading; my mouse cursor is on the button to stop recording.

It's not a super common pattern, but probably common enough that it would be annoying not to be able to do it.

Re:A Developer's View of Vision Pro by htag on Monday June 12, 2023 @10:39PM (#36304459)
Yea, I agree with this for gaming. Eye tracking as a gaming interface probably requires rethinking a lot about games. In VR for example the movespeed is much slower. In normal video games the character movement is constantly super-human speed, and this is jarring while in VR.

I am speaking mostly about the desktop interactions. In your Chrome Dev situation, I would look at the cursor before clicking on the stop recording button. I think I might be able to trust the MBP trackpad to do a primed click without looking at the cursor, but I wouldn't trust a traditional desktop mouse to have stayed steady enough.

Re:A Developer's View of Vision Pro by jasonpbecker on Monday June 12, 2023 @02:22PM (#36298464)
I definitely want to see a monitor with Vision Pro-style eye tracking and hand tracking. I'd love for eyesight and subtle gestures to fully replace pointer devices.
Re:A Developer's View of Vision Pro by samwillis on Monday June 12, 2023 @02:14PM (#36298310)
As we use an iPhone so close to our face, I would love to see an eye tracking system work for them. But maybe it really need that incredible close proximity, two cameras per eye, and IR illumination in the Vision Pro to work.
Re:A Developer's View of Vision Pro by henryfjordan on Monday June 12, 2023 @02:13PM (#36298302)
How do I click though? Blink? What about a right-click?

Asking me to "blink twice" or anything like that is going to make my eyes lose focus on the screen/content

Re:A Developer's View of Vision Pro by ladberg on Monday June 12, 2023 @02:24PM (#36298515)
The pinch gesture on Vision Pro is pretty smooth and easy, about as much physical exertion as a mouse click. Right click is harder, I'd expect most app developers to do something similar to iOS and treat long clicks as right clicks but that's pure speculation.

I'd love to have an eye tracking setup though on a normal desktop computer where I could devote a keyboard key to clicking and I'd never have my hands leave the keyboard.

Re:A Developer's View of Vision Pro by samwillis on Monday June 12, 2023 @02:16PM (#36298339)
On the Vision Pro, tap your index finger and thumb together.

Maybe they could add other features from just your face though for other platforms, wiggling nose, raise eyebrows, stick out tong, blow a raspberry?

That or they add voice commands, "open", "select" and such.

I suppose ultimately in spatial computing with voice interface and eye tracking the concept of a "click" may die.

Re:A Developer's View of Vision Pro by crooked-v on Monday June 12, 2023 @02:27PM (#36298561)
There's also apparently other gestures and voice commands that can be selected in the settings, though I haven't seen any real detail on it other than "they're there".
Re:A Developer's View of Vision Pro by UI_at_80x24 on Monday June 12, 2023 @02:53PM (#36299058)
I primarily use i3wm with 4-40 terminal windows + firefox open. I'd LOVE it if the terminal that I'm looking at was the `active` one. I can't count the number of times I'd lookup some error code, type in more commands, hit enter; only to discover that I had been typing away in the wrong window.
Re:A Developer's View of Vision Pro by sleepybrett on Monday June 12, 2023 @03:25PM (#36299546)
This will be no problem at all in the sdk. Hell I saw someone do this as some point on the mac using the macbook camera popping windows to the front based on eye-dwell... I think they started with cursor control.. essentially eye-mouse.

Oh wait some version of that is built into accessibility on mac already (eye tracking mouse): https://support.apple.com/lv-lv/guide/mac-help/mchl437b47b0/...

Re:A Developer's View of Vision Pro by mrguyorama on Monday June 12, 2023 @04:05PM (#36300158)
Eye tracking has been $100 to own for many years now through a certain company. Why hasn't it taken off?

The mouse is the superior input device. When people who actually need a better input device get one, they get more advanced mice:

https://3dconnexion.com/us/spacemouse/

Re:A Developer's View of Vision Pro by ladberg on Monday June 12, 2023 @04:40PM (#36300725)
I'm not sure what the SOTA is currently but the Vision Pro has much better eye tracking that other systems I've used in the past. Idk if the sensors are actually better or if it's just the native integration into the OS and apps, but it feels more seamless than a mouse and I've never "misclicked" on the wrong thing with my eyes. It kinda feels like if the mouse cursor instantly jumped to the right place every time with no travel time or physical movements required.

I've used a 3D mouse for CAD but am not sure where else it would be helpful?

Re:A Developer's View of Vision Pro by mrguyorama on Tuesday June 13, 2023 @04:44PM (#36316941)
Tobii https://gaming.tobii.com/product/eye-tracker-5/ is standard in consumer eye tracking, they even have integration with some popular video games! There's just no value in it. I tried an older version in a store, and it was basically perfect, tracking my eye's movements exactly, with zero delay.

Nobody wants it for day to day computer interaction. Most people using eye tracking for computer interaction are disabled, because it's a terrible experience.

Re:A Developer's View of Vision Pro by throwuwu on Monday June 12, 2023 @04:19PM (#36300381)
Voice input is going to replace the keyboard shortly.
Re:A Developer's View of Vision Pro by frant-hartm on Monday June 12, 2023 @05:12PM (#36301215)
Sure, just right after Linux takes over desktop.
Re:A Developer's View of Vision Pro by crazygringo on Monday June 12, 2023 @02:28PM (#36298582)
Well, touchscreens are generally smaller than laptop screens, and a lot of productivity requires a keyboard.

But the screens on Vision Pro are larger, and it works great with a keyboard.

So anything that requires larger/more screens seems like it might be more productive than a laptop.

Re:A Developer's View of Vision Pro by sebzim4500 on Monday June 12, 2023 @02:05PM (#36298153)
I think some artists use an ipad with a stylus rather than tablet input to a laptop/desktop, but other than this I can't see how anything else could compete with a keyboard in terms of productivity. Maybe if someone makes a really, really good voice to text interface.
Re:A Developer's View of Vision Pro by noelsusman on Monday June 12, 2023 @02:29PM (#36298600)
Here's a short demo of a pretty good voice to text interface that's available for free: https://twitter.com/lunixbochs/status/1378159234861264896

I had to use it for a while when I was unable to touch a keyboard or mouse while recovering from RSI and I was surprised by how quickly I was able to get to about 80% of my previous productivity using just my voice. I still use it sometimes even though my RSI is fully healed.

Re:A Developer's View of Vision Pro by elzbardico on Monday June 12, 2023 @02:38PM (#36298764)
> Maybe if someone makes a really, really good voice to text interface. Maybe it could work, unless you're programming in APL :-)
Re:A Developer's View of Vision Pro by bamfly on Monday June 12, 2023 @02:48PM (#36298955)
I could see walking around the room thinking and doing clicky-stuff and looking at things, then sitting in front of a keyboard to type. Basically, what I do when I'm on a call and have wireless headphones on.
Re:A Developer's View of Vision Pro by nektro on Monday June 12, 2023 @02:24PM (#36298517)
i fully plan on using a keyboard with the vision pro, that doesnt make me any less excited for it
Re:A Developer's View of Vision Pro by mbesto on Monday June 12, 2023 @03:53PM (#36299959)
It's all very well researched that touchscreens above our arms can't be used with longevity: https://www.wired.com/2010/10/gorilla-arm-multitouch/