An algorithm cannot BE "careless", in the legal sense of the word. You are comparing apples and oranges. Carelessness (negligence) is a different problem than a design flaw.
An algorithm may be flawed, in which case blame lies with the companies involved in creating it and putting it in peoples' hands. Similarly, any mechanical component of a vehicle can be flawed -- and many can cause an accident if they fail. The companies responsible are legally liable already.
> In the end, we usually determine that the rights of individuals outweigh the utilitarian benefit
Cars. Airplanes. Vaccines. All of these benefit the majority at the expense of a few.
The idea that there is no liability here is absolute and utter NONSENSE. Minimally, Tesla can be sued for the flaw in their product and the harm it caused.
No, but you can sue the company that built the faulty product, and in certain extreme cases imprison the responsible individuals.
rights of individuals outweigh the utilitarian benefit
What rights are being violated here? People are choosing to use unproven technology, which actually appears to be approximately as safe as humans driving.
As far as I see, the point of legal recourse is to deter it from happening in the first place, I.e. it is purely consequentialist. If you reject such consequentialist thinking, what's the point of having someone to blame?
To me, the key factor is blame. Who do we fault in an accident? We can't imprison an algorithm or put it in jail. But we can implicate a careless driver. A loved one should not have to die because a truck happened to be the right shade of white — not without much discussion and the possibility of legal recourse.
There are many horrible, systemic things we could do to improve the lives of the majority at the expense of a few. In the end, we usually determine that the rights of individuals outweigh the utilitarian benefit. Does the same reasoning apply to self-driving cars? I don't know. If the technology proves as safe as people make it out to be, maybe things will end up OK. But it's not a closed case by any means.