News for hackers. Stuff that matters.
Re:Waymo One is now open to everyone in San Francisco by bangaladore on Tuesday June 25, 2024 @03:31PM (#40792609)
> That's simply untrue. All the hard stuff continues to be reliability and sensor gated.

IR and thermal sensing are unnecessary if the bar is human level and neither is the lidar. The point is overused, but humans rely on two eyes in the driver seat. I don't see any evidence to suggest the modern model that Tesla has developed for their vision system is their limiting factor in the slightest to reach L4/L5.

Dogs jump into the road in front of cars all the time and get killed, and kids get endangered at school bus crossings. That's a reality of life that robotaxis do not need to solve.

Re:Waymo One is now open to everyone in San Francisco by rurp on Tuesday June 25, 2024 @03:52PM (#40792815)
That vision-only argument is marketing spin from Tesla. The biggest thing it leaves out is that humans process their vision input with a human brain, which Tesla vehicles very much do not have. If and when we create true AGI they will have a good argument, but a world where that exists will be wildly different from our current one and who knows if Tesla's tech will even be relevent anymore.
Re:Waymo One is now open to everyone in San Francisco by bangaladore on Tuesday June 25, 2024 @04:51PM (#40793437)
Why are you so confident that AGI, or a human brain, is necessary to be able to drive a car with only cameras?

I get annoyed with statements like this because technology changes and advances so quickly, and Tesla has made substantial technical leaps in this field of machine learning. They have the state-of-the-art vision -> voxels/depth models and are only improving.

Re:Waymo One is now open to everyone in San Francisco by michaelt on Tuesday June 25, 2024 @05:26PM (#40793817)
Tesla, who use cameras only, have not demonstrated full self driving, despite trying for a decade. Elon Musk has stated "It is increasingly clear that all roads lead to AGI. Tesla is building an extremely compute-efficient mini AGI for FSD" [1]

Waymo, who use additional sensors like lidar, have a driverless taxi service which needs no safety drivers.

The evidence kinda speaks for itself, IMHO.

[1] https://twitter.com/elonmusk/status/1740641473849352450?s=20

Re:Waymo One is now open to everyone in San Francisco by diebeforei485 on Tuesday June 25, 2024 @07:16PM (#40794794)
Waymo does have safety drivers, they're just driving the vehicle remotely when it's in certain areas instead of being in the vehicle. So it isn't "full" self driving either.

> Tesla, who use cameras only, have not demonstrated full self driving

There are entire youtube channels with hours of continuous video showing Teslas driving around SF, but also other parts of California, with no human intervention.

Re:Waymo One is now open to everyone in San Francisco by kajecounterhack on Tuesday June 25, 2024 @07:53PM (#40795056)
No, Waymo is not driving remotely. Remote operators can only answer simple questions. They're at the point of commercialization so it's all about unit economics. There's no point in driving remotely especially since it does not scale cost-wise.

Waymo is geofenced, but within its geofence it requires zero human intervention. Tesla on the other hand is famous for mistaking the moon for a traffic light. Saying "Tesla has so many miles on YouTube" is hilarious because first of all there are channels with lots of Cruise & Waymo footage too, and more importantly it's not the # of miles that matters, but the # of non-trivial scenarios you can handle.

Re:Waymo One is now open to everyone in San Francisco by diebeforei485 on Tuesday June 25, 2024 @08:51PM (#40795353)
I don't see why Tesla can't handle those scenarios if they also use remote operators. I wouldn't be surprised if they do.

Btw Waymo is nowhere near achieving unit economics. Their cars cost like 5 times what Teslas cost, and the sensors require a lot of upkeep and maintenance.

Re:Waymo One is now open to everyone in San Francisco by kajecounterhack on Wednesday June 26, 2024 @12:16AM (#40796326)
Who’s to say what unit economics they’ve achieved but I’d hazard to guess that their investors wouldn’t support expanding their fleet and service unless the unit economics are at least close to break even. Cost for sensors and overall BOM keeps going down as more suppliers enter the market.
Re:Waymo One is now open to everyone in San Francisco by the-rc on Tuesday June 25, 2024 @07:54PM (#40795059)
Are you saying that there are times when a Waymo car's ability to respond to events is at the mercy of a random Internet connection? What happens if the safety driver is steering remotely, from another town, and there's packet loss for a couple of seconds in the middle of a curve?
Re:Waymo One is now open to everyone in San Francisco by diebeforei485 on Tuesday June 25, 2024 @08:47PM (#40795329)
In that case if the car isn't able to decide what to do, it stalls in the middle of the street. Happens all the time in SF.

It's probably not accurate to say that the remote operator is "steering" though. It's more like pathfinding.

Re:Waymo One is now open to everyone in San Francisco by ladon86 on Tuesday June 25, 2024 @08:58PM (#40795391)
Again, they don’t drive or steer remotely. What sometimes happens is a multiple choice question is presented to the operator in an ambiguous situation:

<photo of construction zone> Can I drive through here? [Yes] [No]

When this is happening, the car is stopped and lets the passenger know that it’s reaching out for remote help to figure out what to do. For me this has happened two times across my 125 Waymo rides (571 miles) so far, and was resolved in under 20 seconds. Though I must say, 20 seconds feels like ages when you’re in the car and blocking traffic!

Re:Waymo One is now open to everyone in San Francisco by the-rc on Tuesday June 25, 2024 @10:23PM (#40795818)
I agree with you, I was just highlighting for the other poster how difficult/crazy/riskier remote driving would be.
Re:Waymo One is now open to everyone in San Francisco by rootusrootus on Tuesday June 25, 2024 @03:45PM (#40792745)
> humans rely on two eyes in the driver seat

Eyes which are orders of magnitude capable than the best cameras, and Teslas come with mediocre cameras, not the best. Eyes which are connected to a brain, and ML is a looooong ways from rivaling that.

> That's a reality of life that robotaxis do not need to solve.

Robotaxis do not need to account for things jumping out unexpectedly in front of them?

Re:Waymo One is now open to everyone in San Francisco by mike_hearn on Tuesday June 25, 2024 @04:44PM (#40793374)
Orders of magnitude more capable than the best cameras? I wish. I need corrective lenses for my eyes to even work at all. With that fixed they feed my brain an image that's upside down, black and white except in the centre, which is covered in blood vessels and which has a blind spot. They also take a long time to adjust to sudden changes in lighting conditions, don't do any true depth sensing, suffer frequent frame drops and can't run for more than about 20 hours at a time before they basically stop working.

My brain tries to hides all this from me, and makes me think that I see the world in glorious 3D technicolor all the time, but that's a lie as revealed by the many amusing optical illusions that have been discovered over the years.

Meanwhile, today I used ML that knows more than me, can think and type faster than me, which is a much better artist than me and which can read and react far faster than me to visual stimuli. Oh, it can also easily look in every direction simultaneously without pausing or ever getting distracted or bored.

Somehow it doesn't feel like I have a big advantage over computers when it comes to driving.

Re:Waymo One is now open to everyone in San Francisco by mlyle on Tuesday June 25, 2024 @05:20PM (#40793750)
The cameras we are talking about have poor angular resolution, worse dynamic range than the human eye, and don’t do any direct depth sensing either.
Re:Waymo One is now open to everyone in San Francisco by mike_hearn on Tuesday June 25, 2024 @05:30PM (#40793863)
Are we talking about Tesla's cameras or the "best" cameras? There are smartphone cameras that do depth sensing and HDR, and cameras are cheaper than eyeballs so composing them to get more angular resolution seems OK.
Re:Waymo One is now open to everyone in San Francisco by mlyle on Tuesday June 25, 2024 @06:35PM (#40794481)
ToF/structured illumination cameras are honestly not that capable.

The maximum dynamic range of the eye is ~130dB. It's very difficult to push an imaging system to work well at the dark end of what the eye will do with any decent frame rate.

It's not as different as it used to be, but even so: the Mk. I eyeball does pretty damn well compared to quite fancy cameras.

Re:Waymo One is now open to everyone in San Francisco by kajecounterhack on Tuesday June 25, 2024 @07:11PM (#40794764)
> There are smartphone cameras that do depth sensing and HDR

Depth sensing is again, estimated or using time of flight sensors which is pretty much short-range lidar. HDR is used already in AV perception, but still loses to your eyeballs in dynamic range and processing time.

Re:Waymo One is now open to everyone in San Francisco by mike_hearn on Wednesday June 26, 2024 @04:00AM (#40797453)
Yes, but human depth sensing is also estimated.

Eyeballs have high dynamic range but with high mode switching times. Walk from a bright area to a dark area and it'll take seconds for your eyes to adjust. Cameras are so cheap you can just have a regular day camera and a dedicated night vision camera together, switching between feeds can be done in milliseconds.

Re:Waymo One is now open to everyone in San Francisco by kajecounterhack on Wednesday June 26, 2024 @04:20AM (#40797584)
> Yes, but human depth sensing is also estimated.

Robots aren't humans. You need accurate depth perception to maneuver a robot precisely, and you need ground truth depth measurements to train learned depth perceivers as well as to understand their overall performance. Humans learn it by combining their other senses and integrating over very long time using very powerful compute hardware (brain). To date, robots learn it best when you just get the raw supervision signal directly using LiDAR.

> Walk from a bright area to a dark area and it'll take seconds for your eyes to adjust

You do realize cameras have the same issue, and that HDR isn't free / is very computationally intensive?

Re:Waymo One is now open to everyone in San Francisco by kajecounterhack on Tuesday June 25, 2024 @05:24PM (#40793806)
> My brain tries to hides all this from me

Your brain is _really really_ good at surmounting challenges including many that you did not mention. We don't know how to get close to this in terms of reliability when using cameras and ML alone. Cameras and ML alone can go very far, but every roboticist understands the problem of compounding errors and catastrophic failure. Every ML person understands how slow our learning loops are.

Consider that ML models used in the field have to get by with a fixed amount of power and ram. If you want to process time context of say 5 seconds, and with temporal context 10Hz and with resolution 1080p, how much data bandwidth are you looking at? Comparing what you see with your eyes with a series of 1080p photos, which is better? Up it to 4k: how long does it take to even run object detection and tracking with a limited temporal context?

Your brain is working with more temporal context, more world context, and has a much more robust active learning loop than the artificial systems we're composing today. It's really impressive what we can achieve, but to those who've worked on the problem it feels laughable to say you can solve it with just cameras and compute.

There are plenty of well respected researchers who think only data and active learning loops are the bottlenecks. In my experience they're focused on treating the self driving task as a canned research problem and not a robotics problem. There are as many if not more respected researchers who've worked on the self-driving problem and see deeper seated issues -- ones that cannot be surmounted without technologies like high fidelity sensors grounded in physics and HD maps.

Even if breadth of data is the problem and Tesla's approach is supposedly yielding more data -- there is also the question of the fidelity of said data (e.g. the distances and velocities from camera-only systems are estimated and have noiser gaussians than ones generated with LiDAR). If you make what you measure, and your measurements are noisy, how can you convince yourself or your loss function for that matter that it's doing a good job of learning?

It's relatively straightforward to build toy systems where subsystems have something on the order of 95% reliability. But robotics requires you to cut the tail much further. https://wheretheroadmapends.com/game-of-9s.html

Re:Waymo One is now open to everyone in San Francisco by bangaladore on Tuesday June 25, 2024 @04:52PM (#40793454)
Again, another claim that a brain or AGI is required to drive a car. Does anyone have any research to cite that establishes this seemingly known fact?
Re:Waymo One is now open to everyone in San Francisco by mlyle on Tuesday June 25, 2024 @05:19PM (#40793740)
I am not sure that the vision in Teslas is adequate with -any- amount of processing to drive a car. Spatial resolution is limited, as is seeing distant vehicles during merges, etc.

Secondarily, there is no guarantee that the amount of processing is enough, because the extant human systems use much more.

“Cheating” by using more sensors to simplify out complexities and to cover for the shortcomings of other sensors in the suite seems wise.

Re:Waymo One is now open to everyone in San Francisco by kajecounterhack on Tuesday June 25, 2024 @05:50PM (#40794079)
> “Cheating” by using more sensors to simplify out complexities and to cover for the shortcomings of other sensors in the suite seems wise.

Also, "cheating" is just a necessary step to build baseline metrics. You need ground truth.

It may very well be the case that cheating is needed to generate the training data necessary to stop cheating...someday.

Re:Waymo One is now open to everyone in San Francisco by RankingMember on Tuesday June 25, 2024 @03:45PM (#40792743)
> if the bar is human level

IMO the bar is well above human-level if you actually want to attain the level of trust necessary to remove the steering wheel from a car.

Re:Waymo One is now open to everyone in San Francisco by rootusrootus on Tuesday June 25, 2024 @03:51PM (#40792808)
Agree 100%. And IMO it is worth remembering that a really significant share of collisions are caused by well known risk factors. For those of us who avoid being in those situations to begin with, the robotaxi would need to be a good bit safer than our average.
Re:Waymo One is now open to everyone in San Francisco by kajecounterhack on Tuesday June 25, 2024 @03:41PM (#40792707)
> I don't see any evidence to suggest the modern model that Tesla has developed for their vision system is their limiting factor in the slightest to reach L4/L5

For one, frame rate and processing rate on human eyes is way higher than cameras. Dynamic range is another. Also, Cruise and Waymo are some of the only companies that have hard internal data / ability to simulate how well their safety drivers do, and in the very same scenario what their software driver will do. Without LiDAR you can't build that simulation, and once you have that data if you continue to use HD Maps and LiDAR there's probably a good reason.

> Dogs jump into the road in front of cars all the time and get killed, and kids get endangered at school bus crossings. That's a reality of life that robotaxis do not need to solve.

Robotaxis need to avoid any accident that a human would be able to avoid.

> IR and thermal sensing are unnecessary if the bar is human level

See, you could say this if you had some data that showed that incidents per X miles (when the vehicle is driving at night) is sufficiently low, + if the software passes some contrived scenarios to gut-check its ability to see in the dark with the necessary reliability. But you don't have that data, do you? Someone has it though :) and I'd argue regulators should have it too.

Re:Waymo One is now open to everyone in San Francisco by bangaladore on Tuesday June 25, 2024 @04:48PM (#40793402)
> For one, frame rate and processing rate on human eyes is way higher than cameras.

I don't think it's exciting to say that you must have theoretical parity with something to use it for this use case. Tesla's solution monitors ~6? cameras at once with accurate depth in each. That's 6x more views than a human can see. I wish people would stop comparing apples to oranges.

> Robotaxis need to avoid any accident that a human would be able to avoid.

I never said anything to the contrary. Animals get hit all the time, not just because a human wasn't paying attention.

Re:Waymo One is now open to everyone in San Francisco by kajecounterhack on Tuesday June 25, 2024 @05:33PM (#40793905)
> Tesla's solution monitors ~6? cameras at once with accurate depth in each

No, the depth is estimated. It's not accurate, at least not in the way you need for L4.

> I never said anything to the contrary. Animals get hit all the time, not just because a human wasn't paying attention.

I was just clarifying what the bar is. The bar is that avoidable accidents need to be avoided. Nobody will get mad if a plane crashes due to unavoidable circumstances (freak accident where two engines go out due to bird strikes or something). People will stop flying in the plane when it becomes clear that the airline is not doing everything it can to avoid fatalities.