> How does this square with the video where they showed it running continuously for an hour doing an actual Amazon package sorting job? https://www.youtube.com/watch?v=lkc2y0yb89U

The video shows several of glitches. From the comments:

  14:18 the Fall
  28:40 the Fall 2
  41:23 the Fall 3 

Also many of the packages on the left are there throughout the video.

But then I think lots of this can be solved in software and having seen how LLMs have advanced in the last few years, I'd not be surprised to see these robots useful in 5 years.

Three mistakes in an hour isn’t terrible, especially if that’s the last generation. As another commenter put it, this is the worst it’s ever going to be.

> this is the worst it’s ever going to be.

People keep parroting this line, but it's not a given, especially for such an ill-defined metric as "better". If I ask an LLM how its day was, there's no one right answer. (Users anthropomorphizing the LLM is a given these days, no matter how you may feel about that.)

Yeah but for things like packets dropped per hour there will probably be improvement.

I only watched for 5 minutes, at double speed, but I saw a lot more than 3 mistakes fwiw

> But then I think lots of this can be solved in software and having seen how LLMs have advanced in the last few years, I'd not be surprised to see these robots useful in 5 years.

Would asking the robot for a seahorse emoji leave you in a puddle of blood?

Thinking about it, I am sure it is only a matter of time until a self driving car or a robot will be used to kill a human. Or on a lower level for a DDoS attack - all cars/robots going to the white house.

> self driving car or a robot will be used to kill a human

They already have. We just don't hold the perpetrators accountable.

I think "has been used to kill" is a more serious accusation than "accidentally killed" (even if due to recklessness).

What case(s) are you thinking of?

Now there's an idea for Terminator 7.

I was confused by this because I assumed the robot fell over a few times - these are times that one of the piled up packages falls off or is a bit knocked off behind the robot (the second one seems to be knocked off by the elbow?).

Is it really sorting? All I see is the humanoid robot moving similarly shaped / sized packages from one conveyor belt to a platform to another conveyor belt. A little industrial automation design would be much more effective, cheaper, and faster compared to the task it is performing.

Plus it's missing the large stack of packages already in the corner that...seems like they will forever be stuck there.

The actual sorting is typically automated with scanners reading the labels and shunting packages from one conveyor belt onto another, basically a physical sorting network.

Tasks left for human "sorters" to do are:

- put packages on conveyor belt so the scanner can read the label (as done by the robot in the video)

- deal with damaged or unreadable packages that can't be processed automatically

- when a package gets jammed and forces the conveyor belt to stop, remove the offending package before restarting

- receive packages at the other end and load them into vehicles

Generally the difficulty with all of these is dealing with variability and humans act as variability absorbers so the machines can operate smoothly.

Which is why robots that can also absorb variability in the same way humans do would be so valuable.

I'm really confused by this video. What is it even supposed to be doing?

Is it supposed to be taking packages and placing them label face down?

I cannot understand how a robot doing this is cheaper than a second scanner so you can read the label face down or face up. I mean you could do that with a mirror.

But I'm not convinced it is even doing that. Several packages are already "label side down" and it just moves them along. Do those packages even have labels? Clearly the behavior learned is "label not on top", not "label side down". No way is that the intended behavior.

If the bar code is the issue, then why not switch to a QR code or some other format? There's not much information you need in shipping so the QR code can have lots of redundancy, making it readable from many different angles and even if significantly damaged.

The video description also says "approaching human-level dexterity and speed". No way. I'd wager I could do this task at least 10x its speed, if not 20x. And that I'd do it better! I mean I watched a few minutes at 2x speed and man is it slow. Sure, this thing might be able to run 24/7 without breaks, but if I'm running 10-20x faster then what's that matter? I could just come in a few hours a day and blow through its quota. I'd really like to see an actual human worker for comparison.

But if we did want something to do this very narrow task for 24/7, I'm pretty sure there are a hundred different cheaper ways to do it. If there aren't, then it is because there is some edge cases that are pretty important. And without knowing that then we can't actually properly evaluate this video. Besides, this video seems like a pretty simple ideal case. I'm not sure what an actual amazon sorting process looks like, but I suspect not like this.

Regardless, the results look pretty cool and I'm pretty impressed with Figure even if it is an over-simplified case.