News for hackers. Stuff that matters.
Re:Figure 01 robot demos its OpenAI integration by daveguy on Wednesday March 13, 2024 @12:05PM (#39693228)
The speech to servo movement is impressive as others pointed out. What strikes me as amazing is the speed with which it is performing tasks that require dexterity. This is the first object manipulation robot demo I have seen that didn't require speeding up the video for it to look "natural".
Re:Figure 01 robot demos its OpenAI integration by huytersd on Wednesday March 13, 2024 @12:51PM (#39693968)
How though. It’s probably just predefined actions that are triggered by the LLM output. At the same time it would be impressive if the LLM determined the right function to call in real time, was able to deal with the ambiguity in placement of the garbage and bonus points if it could do that in a scenario that wasn’t hardcoded to exactly standing behind that table in that spot.
Re:Figure 01 robot demos its OpenAI integration by og_kalu on Wednesday March 13, 2024 @12:56PM (#39694066)
The robot movements are separate end-to-end neural networks. They're triggered by the LLM but aren't hardcoded.
Re:Figure 01 robot demos its OpenAI integration by margorczynski on Wednesday March 13, 2024 @01:31PM (#39694558)
Why does it then after performing an action or whole plan return to the "default" position with the hands in that strange stance? Looks kinda like there is some "hard coded" flow that simply uses the LLM(s) to perform actions.
Re:Figure 01 robot demos its OpenAI integration by og_kalu on Wednesday March 13, 2024 @03:44PM (#39696437)
That's a closed loop operation. It's pretty common to train these robots on closed loop demonstrations.
Re:Figure 01 robot demos its OpenAI integration by huytersd on Wednesday March 13, 2024 @01:56PM (#39694925)
Do you know this for a fact or are you speculating?
Re:Figure 01 robot demos its OpenAI integration by StarCyan on Wednesday March 13, 2024 @03:30PM (#39696268)
One of the Figure engineers described the system at a high level: https://twitter.com/coreylynch/status/1767927506332721154
Re:Figure 01 robot demos its OpenAI integration by m3kw9 on Wednesday March 13, 2024 @01:12PM (#39694278)
Is it just me or are people easily impressed by these robot demo that moves nothing like humans and doing the simplistic tasks like passing a large object. The freaking Apple is within his arms reach and he asked to pass the Apple. I almost laughed out loud.
Re:Figure 01 robot demos its OpenAI integration by bigyikes on Wednesday March 13, 2024 @01:58PM (#39694954)
Easily impressed? These are difficult tasks! Even being able to grip the apple without crushing it is impressive.

Not sure anything here is state of the art, but that doesn’t make it easy.

Re:Figure 01 robot demos its OpenAI integration by darth_avocado on Wednesday March 13, 2024 @02:45PM (#39695672)
I am 100% convinced that the demo is partly fabricated, or at least very far from the robot’s capabilities. The inference part was probably true, but the dexterity would put them waaaaay ahead of what robots are capable of, if not programmed finely for a specific task. Industrial robots obviously have that kind of dexterity and precision because they’re specifically programmed for a task. General purpose robots however are nowhere as close to this level of accuracy or fluidity in their movements.
Re:Figure 01 robot demos its OpenAI integration by fragmede on Wednesday March 13, 2024 @04:26PM (#39697015)
It is. Have you seen ok robot?

https://ok-robot.github.io/

Re:Figure 01 robot demos its OpenAI integration by binoct on Wednesday March 13, 2024 @04:55PM (#39697395)
OK-Robot is super impressive, but there is a huge difference in the manipulation ability OK-Robot shows with a simple 1-DOF gripper and parallel-axis arm to pick up objects and place them largely on flat surfaces, and the human-analog arms and fingers of Figure 01 picking up a plate in both hands and placing it in the right slot in a drying rack, or dropping the apple into a person's moving hand.

It would be absolutely amazing if they really are at that level of manipulation in general, and it would put them vastly beyond what anyone has been able to do date. However robotics demos have a great history of being a mix of slight of hand (partial/full tele-op), heavily cherry picked, or tuned to a extremely specific example.

Because it's such a leap being implied by this video, it's reasonable to want significantly more evidence before believing they can do this type of interaction and manipulation in a general way. But even if it is heavily leaning say on imitation learning for this exact scenario, there are tons of potential applications for this level of capability.

Re:Figure 01 robot demos its OpenAI integration by fragmede on Wednesday March 13, 2024 @09:32PM (#39699717)
Yes, there are two half’s to this. This is being able to understand the command which OK robot shows is possible, and then there’s the control of motors and actuators as you pointed out. Not needing a team of engineers to laboriously write gcode to manipulate motors is part two and is the other astounding thing here. I haven’t been following this sort of thing, so I have no idea what previous state of the art was, so I can’t judge how much of an advancement it really is.
Re:Figure 01 robot demos its OpenAI integration by whatever1 on Wednesday March 13, 2024 @03:24PM (#39696184)
Probably some sort of imitation learning
Re:Figure 01 robot demos its OpenAI integration by StarCyan on Wednesday March 13, 2024 @03:32PM (#39696303)
Yeah the motion looks way more fluid then similar systems I've seen before.
Re:Figure 01 robot demos its OpenAI integration by btbuildem on Wednesday March 13, 2024 @04:01PM (#39696674)
The slightly dismissive nature of the toss seemed like a hack or maybe a deliberate way to speed up the [pick up, carry, drop] pathway.