Ive done my bachelor's thesis in 2013 on this. We were working on a robot built for exploration of disaster sites (earthquake, fire, ...).

I built:

1) Virtual camera: The robot was equipped with 6 fixed cameras. You can position virtual camera anywhere in the coordinate space and it constructs it's image with the information from the real cameras.

2) System for semi-automatic external calibration of cameras from LIDAR sensor. External calibration is the process of estimating the position and orientation of the camera. This system was not as precise as calibration using calibration patterns in lab, but it was useful when the robot was in the wild and you wanted to adjust the camera or strap a new camera on it quickly. However, to be frank I am not sure ft the system was ever actually used that way. I left the lab after finishing my thesis.

The ROS ecosystem is reasonably simple to understand and definitely makes things much easier. Apart from having some issues with compatibility between versions and incomplete functionality I remember it as quite a nice platform. On the other hand I have not tried any of it's competitors if there are any.

that's pretty cool! I'm particularly interested in how you made the virtual camera work?

can I read your thesis, or (dare i ask) source code?

The virtual camera vas done during summer project but the idea IIRC is quite simple.

I will try some hasty writeup ;)

1) Create a virtual image plane (or cylinder/sphere for panoramic camera) in the 3D space. For this you need to specify the virtual camera's position, orientation, focal length and FOV.

2) Define pixels on the image plane. You need to specify the image resolution and just create grid on the the image plane. Each pixel corresponds to a 3D point in the world coordinate system.

3) To get a color of each pixel you project it's corresponding 3D point to all real cameras. Usually only some of the cameras see the point. Each camera that sees the point gives you a color according to the color of the pixel in it's image. You have to perform blending of these colors. What worked well in our case was to give more weight to cameras where the point projected close to the center of their image.

As you can see, there will definitely be some artifacts, usually more the farther away the virtual camera is from the real cameras. However, it worked to my surprise quite well. It was even possible to look at the robot from 3rd person even though all of the real cameras were on it.

Also it is crucial to have well calibrated cameras. Apart from their projection matrices you also need to consider their radial and tangential distortion which was very significant with our camera models.

https://www.wikiwand.com/en/Pinhole_camera_model

https://www.wikiwand.com/en/Camera_matrix

https://www.wikiwand.com/en/Distortion_(optics)

It's a huge ecosystem of components, but from what I understand it's pretty ubiquitous in the research robotics and commercial space.

My startup is using it to power an autonomous customer service robot. ROS modules handle things like pose calculations, trajectory planning, collision avoidance, and basically all other interaction between software and hardware. From my perspective as a developer on the user-facing software, it's basically a simple event bus and RPC framework that has a load of prebuilt datatypes and modules for handling common robotics tasks.

Caveats – a lot of modules seem to follow pretty sketchy coding practices compared to what I'm used to in other industries. It seems to be a bit of a paragon of bad design. There also seems to be a lot of version confusion – lots of packages seem to support different subsets of versions, with most of the community possibly on older releases. There's an ongoing ROS 2.0 effort that I think uses MQTT as the transport. But I'm not a ROS expert, so take the above with a grain of salt!

Yeah, ROS is a very good example of worse being the enemy of good (to mangle some sayings). It's a common framework which does basically everything you'd want to do in a robotic system, with a bunch of useful functionality on top of it, but basically every part of it is incomplete, poorly documented, awkwardly designed, and buggy. However, since there's basically nothing else which addresses the same space, people feel compelled to use it. I know of two robotics projects, one with a pretty substantial budget, which have been basically hamstrung by developing initially on ROS and running headlong into its limitations while being unable or unwilling to pay the cost of moving off of it.

So basically, ROS is the UNIX of robotics (i.e. "worse is better", + judging by how many serious projects using it are mentioned elsewhere in this comment thread)?

Kind of. Fundamentally, it has never had much competition due to the narrowness of the scope. It is ubiquitous in university robotics due to its accessibility, but in my experience most (successful) industrial robotics applications will simply provide an interface to ROS: internally they will not use it at all.

One thing with ROS is that reimplementing it is not particularly difficult: I have, for various reasons and projects (in some cases before ROS was available) implemented basically all the core components of it (to similar levels of completeness and reliability). So the cost of doing it yourself is actually not all that high (and I think people overestimate it).

NASA's Valkyrie, and Robonaut 2, in addition Fetch, and Baxter,a couple of industrial arms via ROS-i. I know that it's pretty horrid, but it's also the devil I know. This allows for rapid prototyping. In addition, the API hasn't changed much so I can walk away for months and come back and still use the robot. Several organizations support OSRF, but they're still just a couple of people. It's all open source so I've been able to document bugs and submit fixes. That being said, they're still robotics people doing a systems job, but I've failed so far in convincing systems people that robotics is the next mobile.

> but they're still just a couple of people.

27 people are listed on their staff page.

https://www.openrobotics.org/team/

Woah, guess I lost track..

Home service robots, controlled via 'natural' language: https://www.youtube.com/watch?v=b0P2MqmQ-DI

Wall cleaning robot (still testing): https://www.youtube.com/watch?v=4dMjt_Pj4og

There are some nice projects on https://discourse.ros.org/c/ros-projects, I really like Kyler Laird's Tractobots: https://www.youtube.com/watch?v=oPI7f5kQRKo

The whole ROS ecosystem is huge, most of research platform robots are ROS enabled. Check the PR2 (http://wiki.ros.org/Robots/PR2), Turtlebot (https://www.turtlebot.com/), the Nao series (http://wiki.ros.org/nao), even Boston Dynamic's Atlas robot (http://gazebosim.org/tutorials?tut=drcsim_atlas_siminterface...) or Nasa's Robonaut (http://wiki.ros.org/Robots/Robonaut2).

It facilitates work by not having to rewrite a bunch of code every time. Also, most robotic related libraries like OMP, or TF are supported.

Well, apart from adding some fish[1] support to ROS core I worked on these guys[2] at Vecna and they mostly use ROS. A pretty common pattern in commercial use of ROS is to start out with a vanilla setup and replace important nodes with ones you've written to work the way you need them to.

At my current job we don't use ROS but being able to look at the ROS drivers of commercial arms is a big help in making our own drivers.

[1]https://fishshell.com/

[2]https://robotics.vecna.com/

Academics use ROS a lot. The canonical ROS paper has been cited 4,714 times. This is an extreme outlier.

https://scholar.google.com/citations?user=fMDLYCUAAAAJ&hl=en...

Use the 'cited by' link to see thousands of robot projects that use ROS:

https://scholar.google.com/scholar?oi=bibs&hl=en&cites=14376...

(I'm not affiliated with Open Robotics)

I published a research paper as an undergrad on improving an existing algorithm for a team of mobile robots to navigate space and create a map of their environment.

I also used it as an intern at a robotics company to use a microsoft kinect as a depth sensor for helping the robot perceive space in three dimensions.