News for hackers. Stuff that matters.
Re:Learning to Estimate 3D Hand Pose from Single RGB Images by kefka on Thursday May 04, 2017 @04:51PM (#14268687)
I'm glad they're making inroads here, cause gesture and pose calculation and tracking is awesome.

But.. it also seems like more usage of black-box magic math of Neural Networking. I'm glad it gets results and all, but it just seems.. inelegant. What's the algo really doing underneath? Did the algo figure out the joints and their ranges of freedom?

The results are spectacular, indeed. But it also seems a bit not like science. It seems more like an oracle system was able to deduce the results - and we're nowhere near closer to understanding how to do this. Just, we have a trained system that can.

Re:Learning to Estimate 3D Hand Pose from Single RGB Images by return0 on Friday May 05, 2017 @02:35AM (#14271522)
Maybe the problem itself is not elegant (hence why it's not solved in closed form)? Neural networks learn thousands of different algorithms in parallel. It's probably not a single idea behind it, it may be thousands. It's more like experimental science: the system is there, it works, someone else needs to analyze it to figure out how it does what it does.
Re:Learning to Estimate 3D Hand Pose from Single RGB Images by kefka on Friday May 05, 2017 @01:13PM (#14275037)
That's my point. Anybody can shove in gigs of data, clean it up some, provide easily digestible samples, make a whole lot of parameters (or let the system decide), and scrape the data out.

How is it working? What criteria does it work and fail? Does it work for black people? Does it work for women's hands? (Or does it work for anyone out of university?) does it handle people with hand defects or missing digits?

That's right... We don't know unless we test this. And only by adding more data can we even determine those questions. And we haves no clue how its working, what features its using, or anything. Just, that it does work. And it doesn't for conditions were unsure of.

Now this is a great starting plank for determining the underlying math. But even the cost of compute seems high for what it could ideally be, if we understood what was going on.

Re:Learning to Estimate 3D Hand Pose from Single RGB Images by devrandomguy on Thursday May 04, 2017 @07:50PM (#14269937)
Crazy brute force idea for analyzing a neural net, from someone who has little experience with neural nets:

Graph the response of the neural network, over the range of the stimuli that you care about. This is going to be a ridiculously huge dataset, but bear with me. Then, use a genetic algorithm to evolve equations that have reasonably similar behavior, perhaps over a much smaller domain.

This collection of equations and their valid input ranges, are the raw material for your program. You would simplify them using algebraic solvers, when possible, and attempt to hand-optimize them for readability. Then, when you are done, the whole thing gets compiled down to big switch-case statement in, say, C. From here on, the process looks sort of like yacc.

So, by adding a whole new layer of magic, we get the system to explain itself in a way that a programmer could understand. Come to think of it, this feels sort of like how I do personal introspection. In fact, I'm doing it right now.

Re:Learning to Estimate 3D Hand Pose from Single RGB Images by nojvek on Thursday May 04, 2017 @11:32PM (#14270937)
This is kind of close to a relatively new openai paper that uses annealing/genetic algorithms for making a great reinforcement learning algorithm.

I do love the idea of machines figuring out the simplest model with maximum accuracy. When we can do that across different domains using same algo then we can say we have figured out secrets of intelligence.

Re:Learning to Estimate 3D Hand Pose from Single RGB Images by AndrewKemendo on Thursday May 04, 2017 @11:00PM (#14270833)
Deep Neural Nets are inelegant now? Have you tried building effective DL models?

In fact it's more like science than you let on. You have a hypothesis for the data you need, test it, evaluate results, test etc... and result in optimized weighting.

The elegance is inside of the architecture.

Re:Learning to Estimate 3D Hand Pose from Single RGB Images by iaw on Thursday May 04, 2017 @02:30PM (#14267421)
I think this post would be a lot more informative with the original title "Learning to Estimate 3D Hand Pose from Single RGB Images"

The results are astounding for anyone that's tried to do similar work in the past. I'm so pleased that they made the code available.

Re:Learning to Estimate 3D Hand Pose from Single RGB Images by sctb on Thursday May 04, 2017 @03:44PM (#14268075)
Agreed, thanks! We've updated the title from “A 2D Image of your hand and TensorFlow tell you everything”.
Re:Learning to Estimate 3D Hand Pose from Single RGB Images by throwanem on Thursday May 04, 2017 @03:03PM (#14267712)
dang, sctb, et al: Please update the title to match that of the original article, per iaw's comment above. Thanks!
Re:Learning to Estimate 3D Hand Pose from Single RGB Images by hypertexthero on Thursday May 04, 2017 @07:53PM (#14269964)
Interesting.

I just finished reading a book called The Hand, by Frank R. Wilson. The author tells how the human hand has shaped our evolution and suggests, among other things, that the development of language came from our hands.

A bit heavy at times, but nevertheless warmly recommended.

http://www.penguinrandomhouse.com/books/191866/the-hand-by-f...

Re:Learning to Estimate 3D Hand Pose from Single RGB Images by mrfusion on Thursday May 04, 2017 @04:25PM (#14268460)
The vive already has a camera... could this be applied there?

What are some other applications?

Re:Learning to Estimate 3D Hand Pose from Single RGB Images by revelation on Thursday May 04, 2017 @05:07PM (#14268841)
Okay, but depth cameras exist, full stop, and they obviate the need for an entire GPU worth of processing. It's why Uber and Waymo are sinking billions into cheap, solid-state LIDAR. Actual measured sensor data is just so much better.
Re:Learning to Estimate 3D Hand Pose from Single RGB Images by genericpseudo on Thursday May 04, 2017 @07:42PM (#14269884)
The challenge with LIDAR (for the foreseeable future) is ubiquity. If you're building a car, then sure, LIDAR, no questions asked. If you're trying to pair this with say a structure-from-motion pipeline for photogrammetry targeting mobile phones as capture devices then this becomes very very important.

The best data for many applications is often the data you can get at scale.

Re:Learning to Estimate 3D Hand Pose from Single RGB Images by highd on Thursday May 04, 2017 @07:01PM (#14269639)
You'd prefer a $10,000 lidar over a $8 mobile GPU?
Re:Learning to Estimate 3D Hand Pose from Single RGB Images by blt on Friday May 05, 2017 @03:48AM (#14271792)
it's hard to make a small active sensor that produces enough light to work outdoors on a sunny day.
Re:Learning to Estimate 3D Hand Pose from Single RGB Images by jvanderbot on Thursday May 04, 2017 @11:13PM (#14270878)
Here's something from six years ago ish with similar results (faster) https://www.youtube.com/watch?v=qok636pe_qw
Re:Learning to Estimate 3D Hand Pose from Single RGB Images by wyldfire on Thursday May 04, 2017 @02:49PM (#14267585)
The current title "A 2D Image of your hand and TensorFlow tell you everything" is a little head-scratching. "2D image" as opposed to "stereo" 2D images (or depth-mapped image?). And "tell you everything" makes it sound like some kind of fortune teller.
Re:Learning to Estimate 3D Hand Pose from Single RGB Images by jordache on Thursday May 04, 2017 @02:51PM (#14267604)
I don't think this is all that amazing.

Human fingers only have a very narrow range of motion in a single direction. When the 2D profile of the hand changes, the set of finger movements to arrive at that profile is fairly predictable.

Re:Learning to Estimate 3D Hand Pose from Single RGB Images by ska on Thursday May 04, 2017 @03:24PM (#14267884)
You think so in the first 5 minutes of thinking about it, then you try it, and then you realize why lots of smart people have struggled with improving this in the last decades.

Compute vision is full of problems that turn out to be a lot harder than they look. Part of the reason is that our intuition about this is skewed by having access to a very good vision processing unit that we can't reproduce :)

Re:Learning to Estimate 3D Hand Pose from Single RGB Images by KineticLensman on Thursday May 04, 2017 @04:23PM (#14268440)
> "Human fingers only have a very narrow range of motion in a single direction"

Each finger has three joints that can bend in and a little bit back. Each finger can move from side to side. There is a small amount of rotation around the base of the finger. The thumb has three joints, the bottom of which is opposable to create a gripping hand. The entire hand can bend, yaw and rotate with respect to the arm, and there is further flexibility in the bones of the palm.

If you doubt how complicated the hand is, download a free 3D figure posing system like Daz Studio [1], instantiate a figure, and then try to make a hand grip a small object, or gesture (e.g. v-sign, 'ok', hitchhiking, vulcan greeting).

[1] https://www.daz3d.com/

Re:Learning to Estimate 3D Hand Pose from Single RGB Images by jordache on Thursday May 04, 2017 @07:02PM (#14269644)
A lot of the variables you listed is not relevant. The solution linked here is only deriving the 3d positions of the fingers, none of the other nuances that can be expressed by the hand's muscular and skeletal system.
Re:Learning to Estimate 3D Hand Pose from Single RGB Images by regularfry on Thursday May 04, 2017 @03:11PM (#14267769)
As someone who's been keeping tabs on the field for the past 20 years since briefly bouncing through it at university, you're dead wrong. It's one of those things that sounds easy, and... just isn't. The shots of predicting the partially occluded pose are a bit special, too.
Re:Learning to Estimate 3D Hand Pose from Single RGB Images by iaw on Thursday May 04, 2017 @04:52PM (#14268695)
Have you tried to implement visual algorithms? Especially those prior to the prevalence of GPU-based neural networks?

Even just ten years ago it was non-trivial to predict the shape of a room from a single 2d RGB image.

Re:Learning to Estimate 3D Hand Pose from Single RGB Images by anjc on Thursday May 04, 2017 @04:52PM (#14268697)
Try it and see. You're right that skeletal constraints help.