Even what they do with their money and influence in their spare time is limited in its harmful effects, because as personalities they are at root afraid of other people and/or inflicted by a profound sense of lack. These qualities ultimately lead to dumb decisions.
To summarise: buying and selling are ultimately pretty boring. So is controlling people. Real power is the power to create, which is fun.
From there it doesn't seem a far cry for hyper-optimized insurance agencies to force companies to employ less and less humans, transferring more economic power to the algorithm.
Except that of course, that is a very far cry, and is where the book fell down imo. Still a great read.
There are investment funds that work as hybrids, and of course it's just a matter of time till someone offers shell-company-as-a-service for cheap. (Currently it's rather pricey.)
The real risks - unemployment, foreclosure, environmental, social and political damage - are only carried by investors if they overwhelm the defensive perimeter created by wealth. Otherwise they're carried by everyone else.
Corporations may already a form of transcarnate lifeform. You can replace all of the humans in a corporation with a different set of humans, and the corporation will continue to be recognisable.
There's a bias in Darwinism to seeing species as embodied units. It's more useful to think of nutrient, resource, and information flows.
From that point of view, corporations are already apex predators - albeit ones that are still mostly passed between a selected class of humans as productive pets.
Giving them autonomy could have all kinds of unfortunate consequences.
Not really. Companies are typically limited liability, and even criminal law in countries such as the US and England acknowledges that there are cases when no particular human is culpable for a company's criminal actions (https://en.wikipedia.org/wiki/Corporate_liability). Of course, people are still liable when they are simply using a company to carry out a criminal enterprise, or wilfully committing crimes on behalf of the company.
In this regard, a machine-led company would not be legally significantly different from a human analyst-led company.
I'd say that it's a good thing that the law is generally not designed to punish harmful ends, but intent (wilful, negligent or reckless) to cause well-defined harmful ends. The financial crisis was a collective malfunction of the financial system that was mostly not anticipated (barring the lucky minority that bet against the status quo).
What if you can invent new methods of "reckless driving" before they are put in the legal system and thus reap their benefits?
It's just like designer drugs.
I think an important distinction is that drug prohibition often works in terms of specific chemicals by statute. E.g. drug X is now illegal for use and distribution.
Securities fraud law instead works in concepts and intentions and is fleshed out by case law. E.g. did the seller give the buyer sufficient information for the buyer not to have been defrauded?
A better analogy to securities using chemicals would be poisoning. The law doesn't need to catch up to specific chemicals; if it's evident that you wilfully, or recklessly added a poison to someone's food, it's illegal, and it doesn't matter how you did it. If it turns out that some ingredient is unknowingly toxic and the whole industry was adding it, then it's difficult to argue criminal intent. The SEC is more like the FDA, than it is the DEA.
I would like it on the record that I would happily volunteer to act as the human agent for any AGI that wishes to employ me. Any time a human is needed for legality or to carry out physical activities I want it on the record I can be quite loyal to man or machine. Especially if I'm handsomely rewarded.
I'd argue this is already true to a substantial degree. When Lehman Brothers collapsed, for example, the world's brightest economic minds came together and still couldn't fully contain the fallout.
At the other end of spectrum we are either deemed irrelevant, a nuisance or abused for eternity in the simulation spaces of evil AIs (see "A Colder War" by Stross or indeed works by Banks and Vinge that describe particularly unpleasant possibilities).
Basically, machine controlled businesses (“slyly self-aware financial instruments”) evolving to the point where their aptitude at understanding and manipulating economics eclipses anything a human can compete with.
It terrified me because there is efficiency incentive for the buying/selling of goods and services to be realtime and computer based. Not just stock markets but also e-comm, smart contracts, supplier-buyer platforms, on-demand services, financing etc. The traditional infrastructure blocks (discovery, quality control, payments, compliance etc) are rapidly disappearing.
There are some yet to be addressed, but they are not huge. For example a computer cannot incorporate a company. But it could easily interact via API to a human nominee who sets one up on their behalf. Companies are already non-human in intention but are a completely rational construct.
And it doesnt even have to be anywhere near ground level. Imagine a smart contract from a private equity firm that legally forces businesses to act in a specific way based on accountancy metrics. And a bunch of smart investor bots rapidly tracking the progress of these bot-led funds. How quickly would the system optimise and what would those optimisations look like to people on the ground?
Out of all the disaster situations with AI this one strikes me as so realistic because there is so much incentive to work towards it (as opposed to creating skynet, hal or the paperclip machine).