I don't know about "pursuing their own agenda" but making their own decisions - yes, probably. There isn't much point to autonomous cars which are essentially drones.

Decisions don't create agency. My router makes decisions about what packets to route (and where) and what packets to drop. It does so based on a policy I set and those decisions are an extension of my agency. If I configure my car to prefer saving me to saving 100 schoolchildren (or pick a car with that configuration over the alternative), I'm responsible for that.

Currently there's no requirement in law to be a hero. Some famous court case about a taxi driver co-opted by a bank robber or some such.

Autonomous car algorithms must face this issue immediately. Choosing to serve the driver/owner's interest will be a compelling sales point.

I wasn't speaking about what is (or should be) required by law. I was speaking about our social reaction, which certainly has a relationship with law but is not the same thing.

If it's acceptable for a person, in the heat of the moment, to decide to save themselves at a cost to others (once they've considered what other options are available to the best of their ability), then it's acceptable to decide to put in place a system that does the same, assuming it is (at least) comparably capable of considering other options.

If it's heroic for a person to, in the heat of the moment, decide that it's important to save another person even at risk to themselves, then it's also heroic for them to put in place a system that does the same.

Of course, opting out of the whole thing and sticking with a human-driven car is also a choice, and once autonomous cars are sufficiently better than human drivers it may itself become an unacceptable choice.

In the long run, the law should probably catch up to wherever we've arrived at socially and forbid more egregiously unacceptable choices, but I don't see any reason we should start there.

But to sell an autonomous car, you have to survive lawsuits. That's the paramount issue in front of manufacturers. The engineering issues are simple in comparison.

What's acceptable or not will be decided in a court, unfortunately. And like most other traffic-related issues, some default rule (driver in rear at fault etc) will be built over many cases over time.

Unfortunately its unlikely a reasonable rule will prevail. Instead some enforceable rule will win out. Regardless, early manufacturers of autonomous cars are likely to be sued to oblivion in the process.

What's decided in a court depends in large part on social judgements. We should be deciding what's reasonable and convincing others (and being convinced - as a part of that collective deciding), not whining about things that haven't happened yet.