Re:Handsfree.js – integrate face, hand, and/or pose tracking...
by blensor on Saturday June 05, 2021 @05:17PM
(#27408521)
I can't speak about this specific project but pose tracking in the browser can actually be really useful to get it on mobile without the need for dedicated apps.
We are currently experimenting with using phones (or tablets or basically anything with a browser and a camera) as a tracking source for full body tracking on the Oculus Quest ([1]) and being able to just use a Tensorflow model in the browser has cut down the time of development on the mobile device side to almost nothing.
There are of course a whole lot of other issues one buys into with that approach (browser security policies, the whole thing not working in certain countries due to problems with loading the model) but that's manageable
We are currently experimenting with using phones (or tablets or basically anything with a browser and a camera) as a tracking source for full body tracking on the Oculus Quest ([1]) and being able to just use a Tensorflow model in the browser has cut down the time of development on the mobile device side to almost nothing.
There are of course a whole lot of other issues one buys into with that approach (browser security policies, the whole thing not working in certain countries due to problems with loading the model) but that's manageable
[1] https://www.youtube.com/watch?v=zz9S5hgrWpM (relevant part starts at 0:29)