The training does require external GPUs (but we provide that infra for free, straight from the app!), but the onboard jetson can run models trained though, as you can see in the examples. Everything you see in the vids is running onboard when it comes to manipulation, because we use a special version of ACT made specifically by us for this robot, that also includes a reward model (like DYNA does).
We have developed this system to also be able to run the other components smoothly so it also does SLAM, and has room for more processing even when running our ACT.
Now indeed this cannot run Pi-0 but from our experience - and the whole community in general - VLAs are not particularly better than ACT in the low data regime, and need a lot more compute.
As for community-level datasets, yes this is the plan. Anything you train can already be shared with others - just share the files. We didn't develop a centralized place for sharing datasets and behaviors but it is on the plan.
You could simply host the raw grunt in a base station somewhere else in the premises, keeping the device lighter and lower power.
This one is really, really convenient and intuitive. Turn it on anywhere, even outside, it just works. Even when I want to dev on it, it's super convenient.
On some level I truly believe robotics has to become more "complete", we can't always just piece things together, it makes it very hard to have a beautiful product.
I realize this is more of a philosophical answer, but I also think it is the right one to take this field to the next level
How does that fit into your ‘complete’ ethos?
For this one, it's just the only feasible way we found to bring the kind of experience we created to folks.
Also I am curious about a couple of the parts, if you don't mind sharing - are those wheels the direct drive wheels from waveshare? And what is the RGBD camera? (Fwiw, even if it's hefty the MARS price tag seems fair to me).
There is also a possibility for it to tip the base if the arm is fully extended. And the SO-101 has quite poor repeatability.
The base is also slow to move, and depending on which surface you are the omniwheels can get dirt in quickly.
Finally, external compute means you need in particular to teleoperate from your computer, so you have to be far from the robot and not necessarily in the same orientation than it which is very, very uncomfortable. This app system we made is one of the things people love the most about MARS.
Ah, and RGBD really does matter for navigation AND for learning (augmenting ACT with depth yields better results).
The wheels are indeed these ones, and the camera on the video is a luxonis oak-d wide, pretty expensive but comfortable to work with. However, the version we're shipping includes a much cheaper stereo-depth camera that we calibrate ourselves - I can't get you the reference right right now cause it's late at night but feel free to reach out on discord
Does the MARS hardware really remove the hidden extras (computer with a gpu) mentioned as the downside of HF SO-101 or LeKiwi? While a jetson is good for inference, I feel like to train VLAs you would need access to a powerful GPU regardless. For Lerobot based hardware training ACT was relatively low profile if you use low resolution for the camera feeds, but with increased resolution or with more than one camera I already saw needing more than 8GB of VRAM. If VLA is on the table, finetuning something like the open sourced version of pi0 should already necessitate access to more than one 4090 or above I think.
Also, do you have plans for community-level datasets? I think Lerobot sort of does this with their data recording pipeline and HF integration.