It’d be nice also to think of some common format that includes depth, calibration, trajectory and doesn’t require nonstandard software on an iPhone. Standard camera app/cinematic format could be one example, I think.
1. I'm ordering a microscope mount right now! We went with 3D printing because we could get a very good coupling between the stick and the robot mount. See this figure for a comparison: https://i.imgur.com/vWopcFB.jpeg
2. Our dataset (and dataset export code) gives the depth as numpy files and trajectory information in a JSON file with calibration and trajectory information! The r3d export format from the Record3D app we used is simply a .zip file with renamed extension so we were ok with using it. Do you have suggestion as to which cinematic format you are thinking?
In a longer term the stick is likely not needed too. Ideally you’d want a human demonstrating with regular human tool, like pliers to generalize.
This is a particular mount that I’ve used. https://www.amazon.com/Starboosa-Smartphone-Microscope-Unive...
It’s pretty solid, but it doesn’t open completely, so you need to disassemble and reassemble the stick to put it on (which is not that difficult).
I agree with you there. My impression was that the cinematic format is closed, unlike .r3d which is a bunch of open format files in a .zip. Do you know whether Apple has published the specs for the cinematic video format anywhere, or if there are there good libraries for handling it?
Do you and/or anyone else who contributed have specific plans or desires for continuing to contribute to the home robot space?
As for continued contribution, right now the plan is to help get the community on-boarded, and just keep building on the platform. As you can tell, the policy class is pretty bare-bones, the in-home navigation system needs work, and we aren’t taking ANY advantage of the recent generative AI boom. All of those need to change if we want a real robot butler :)
But that’s why we open source things, right? What’s a mountain for one team is a feather for the community.
Our goal is to get more robotics and AI researchers, engineers, and enthusiasts to get out of labs and into homes, so we open sourced everything!
I also submitted a Show HN with more details that unfortunately didn’t get much traction: https://news.ycombinator.com/item?id=38453047