For example, Elon's personal spending and the structure of his compensation. If he just wants to go to mars, why does the stock price and his share thereof matter so dang much?
Or Bill's net worth, which has curiously increased despite his mission of giving it all away.
Capitalism does optimise for property rights, that's true - but if you're taking a utilitarian perspective (as all good rationalists should) then you have to ask what the material outcomes of that are. What we see is accumulation of massive power in the hands of a few, many of whom are doing active harm to the planet (just look at oil billionaires' behaviour). Supporting them is a worldwide underclass of people working in sweatshop conditions or actual human slavery. Maybe optimising for private property isn't providing very good outcomes.
1. The owner of the company doesn't keep all the profit -- they share out the vast majority of it. (Look at the retained profits vs gross margins of any company you can think of. Look at the wages that you, yes I bet you yourself, are paid).
2. When we don't do it this way, we don't actually get the innovation. The thing that incents people to solve the problems of bringing new products to the public (the very large problems, the problems they can spend the majority of their lives attacking) is the possibility of reward.
2 isn't true. Not only is most innovation funded by taxes and conducted by public institutions, but it's the market that drives iterative and combinational improvement on new inventions and you can have markets without capitalism. Capitalism is the division of people into capitalists who own the means of production (traditionally factories but nowadays increasingly IP), and workers who work them for a wage. The first step in moving in the right direction, IMO (I'm keen on incrementally and cautiously implementing improvements) would be to ratchet up the number of cooperatives operating within the economy. They're more likely to survive than corporations and they're more empowering for workers.
Why did this not happen in India, a country which had many advantages over China starting from 30 years ago? It's because the Indian commercial economy is strangled by socialist inspired bureaucracy and anti-competitive special interests. For all the China claims to be communist, it's commercial sector is cut throat capitalism of a particularly raw form. Much rawer that I would like, Chinese people suffer from some of it's excesses, but nevertheless it's produced an incredibly powerful and dynamic economy.
My primary political concern is reducing suffering. Capitalism leads to less suffering than dictatorship, but also leads to things like 996 and Chinese factory conditions, and thus should be either moderated or replaced with market socialism.
This is of course all about goal setting. I think those fearful of maximisers are being rational, but if we’re going to build AIs then we need to make them part of our community. The things that stop us as individuals going off the rails and taking everyone else around us down with us is respect for community, for others rights and interests. The need to seek consensus and consent. Figuring out how to implement that is the real challenge.