BodyPix: Real-Time Person Segmentation in the Browser with Tensorflow.js

Re:BodyPix: Real-Time Person Segmentation in the Browser wit... by netwanderer3 on Monday November 18, 2019 @06:20PM (#21569187)

Can this potentially be used in 2D animation replacing the manual skeletal animating processes?

[ Parent | on HN ]

Re:BodyPix: Real-Time Person Segmentation in the Browser wit... by soylentgraham on Monday November 18, 2019 @07:11PM (#21569525)

That's how we did this; https://m.youtube.com/watch?v=7jfZ6zLkWF0 (was running live on-set)

[ Parent | on HN ]

Re:BodyPix: Real-Time Person Segmentation in the Browser wit... by echelon on Monday November 18, 2019 @10:28PM (#21570415)

That's so impressive!

What company do you work for? This is super cool.

Can you talk about the rest of your stack? Or other projects of a similar nature?

[ Parent | on HN ]

Re:BodyPix: Real-Time Person Segmentation in the Browser wit... by soylentgraham on Tuesday November 19, 2019 @06:55AM (#21572187)

http://newchromantics.com/ is my own little company, (website is criminally out of date) I did the tech, http://analogstudio.co.uk/ did the art/comp/post work.

The setup was essentially an app (c++ engine on osx running models), ingesting camera feeds, outputting skeletons, running sound->mouth poses, to unity which puppeteered 2D sprites.

On-set we output to a monitor which mixed the real shots with our graphics.

Nothing crazy! But it was a solid setup (except for overheating DSLRs)

[ Parent | on HN ]

Re:BodyPix: Real-Time Person Segmentation in the Browser wit... by alexcnwy on Tuesday November 19, 2019 @02:38AM (#21571274)

That’s so epic! Can you please give a little bit more detail on your process?

[ Parent | on HN ]

Re:BodyPix: Real-Time Person Segmentation in the Browser wit... by Hitton on Tuesday November 19, 2019 @05:23AM (#21571822)

That doesn't look like it needs person segmentation.

[ Parent | on HN ]

Re:BodyPix: Real-Time Person Segmentation in the Browser wit... by soylentgraham on Tuesday November 19, 2019 @06:43AM (#21572144)

No, but the post above was about skeletal tracking, not the OP content.

[ Parent | on HN ]

Re:BodyPix: Real-Time Person Segmentation in the Browser wit... by cryptozeus on Tuesday November 19, 2019 @12:41AM (#21570887)

Really cool ! Tensor js ?

[ Parent | on HN ]

Re:BodyPix: Real-Time Person Segmentation in the Browser wit... by xsmasher on Monday November 18, 2019 @06:55PM (#21569416)

Do you mean 3D animation?

It looks like the interface returns 2D coordinates for the joints, not 3D coordinates; so it can't directly create 3D animation data.

It was also very jumpy in my browser, so it would require smoothing to be usable; or maybe if it wasn't running real-time it would be more accurate?

[ Parent | on HN ]