Just to be clear, neural networks typically aren't used to work with lidar in self driving cars. NN for point clouds are still at the research stage.

Does this mean this is just a research project that Uber doesn't actually use in their cars yet?

From https://eng.uber.com/sbnet/

"By applying convolutional neural networks (CNNs) and other deep learning techniques, researchers at Uber ATG Toronto are committed to developing technologies that power safer and more reliable transportation solutions."

"CNNs are widely used for analyzing visual imagery and data from LiDAR sensors. In autonomous driving, CNNs allow self-driving vehicles to see other cars and pedestrians"

There are many projects using deep learning with lidar. Google PointNet, PointNet++, also https://www.youtube.com/watch?v=UXHX9kFGXfg . These are all much newer than 2D CNNs and I don't know if it works well enough to actually be used in SDCs. Also, using CNNs on point clouds comes with all sorts of problems.

I was really hoping I wouldn't need to be the first person to point this out! Thanks Jeff

My understanding is that you don’t do analysis or training inside the control loop at all. Basically, build a model and then it becomes a go/no-go check when you really want to use it.

I don't know how the system is integrated into the self driving logic as a whole. But a few years ago when we were working on a self driving train, LIDAR was used as a system that reports back a list of obstacles (position, size, maybe a simple shape descriptor) or as an obstacle map that is laid out in a grid and shows which cells are occupied and how high the thi g occupying the cell is.

If your system that processes the pointcloud and creates this data does not detect faulty (or missing) data from the sensor the higher level logic will happily hum along believing that nothing is amiss.

And some LIDARs even have the option to do the processing on the device itself (i.e. IBEO). In which case you can theoretically work with only a list of obstacles reported back by the sensor.