One piece of evidence is disengagement rates. In 2017, Waymo led with ~6,000 miles per disengagement. Cruise was in second with ~1,000 miles per disengagement. Also, if you look only at the last few months of 2017, Cruise had improved to around ~3,000 or so as I recall. The other players testing in CA are far behind Waymo and Cruise.

Of course, there are many caveats to comparing disengagement rates. These are self-reported by companies, and each company may have different policies about which disengagements are serious enough to report. Also, not all miles are equal. Cruise likes to emphasize that its SF miles are more difficult than Waymo's more suburban Mountain View miles.

Here's a nice summary of the 2017 data I found for you: https://thelastdriverlicenseholder.com/2018/02/01/disengagem...

To be fair, I feel like I, as a human, get confused about what I’m supposed to be doing driving in SF at a rate worse than once every thousand miles.

To "get confused" is not the same thing as giving up completely and needing another individual to rescue you, in traffic. I think people are overly minimizing what disengagement signifies.

It might be interesting to know in what percent of disengagements the car was actually doing the right thing, and only the human driver thought it was wrong and intervened anyway.

Those are actually already removed from the disengagement count if the operator can show (with simulation) that the car would have done the right thing without intervention.

If it didn't work this way you would be incentivizing operators to train their drivers to take risks in order to hit disengagement metrics which would be bad.

I've seen a few where the car was right and a few where the driver was right, but my assumption is that the car just got lucky because, in my experience, the Cruise cars are very hesitant to make left turns.

>in my experience, the Cruise cars are very hesitant to make left turns

I'm sure unprotected (i.e. no arrow) left hand turns remain one of the more difficult routine scenarios to fine tune. What works in (I assume) Chandler Arizona and San Francisco are probably two very different things. A large busy city like San Francisco simply requires a degree of aggression--Pittsburgh lefts, cutting through fairly small gaps, etc.--that would be inappropriate in a calmer and less busy environment.

Well one failure was at 19th and Mission where left turns are illegal, another was at 13th and Folsom which I think is a protected turn, another was on 11th street where the drivers decided to make an illegal u-turn.

The only times I've come across Cruise vehicles they've been under manual control — and their drivers certainly aren't making much of an attempt to follow the traffic laws. Illegal left turn off of Mission? Why not? Illegal u-turn on 11th St? Sure. Run a red light because the car wouldn't make a legal turn on its own? Of course!

Note: I've already got a negative opinion of Cruise regardless of their purported abilities. I followed up on some headhunter spam from Cruise a year or so ago. I bailed before even getting to a phone screen because as persistent as they were, they bailed or were late every time I agreed to one of their proposed times.

Most publicly available statistics https://qz.com/1220576/the-race-to-build-a-self-driving-car-...