I've actually pointed this out as a fallacy of the premise of current alignment theory. That being we simply need to apply humanities values for the AI to be properly aligned. However, every alignment problem can also be seen as an existing human problem for which we have never resolved.
Oil, plastic, electricity, natural gas, mines, wood, poultry, meat & fish, transportation etc. They're nice and super useful and when used in moderation or when there aren't that many humans to begin with you can easily absorb the consequences. But if you multiply the number of people while at the same time you increase the consumption/production on some or even all of these in tandem the effects are massively compounding.
We want AI to reason like humans and have humanities values. According alignment theory. Yet humans will also cheat to get a result. Our values also are what put us in conflict with each other.
FYI, I've elaborated in much more detail here on such issues with containment/alignment - https://dakara.substack.com/p/ai-singularity-the-hubris-trap
Yes yes yes holy crap yes.
Corporations are AI, and we are already in the AI apocalypse. Corporations exhibit emergent behavior that any individual human involved with them is powerless to stop, and they self-optimize towards inhuman goals. Coca Cola inc is a paperclip maximizer, except it maximizes the consumption of flavored sugar water. We are completely failing as a society to adequately rein them in.
It's scary how many of the things that are in that movie as caricatures you can now find IRL.
I've tried to watch it twice and I can't get more than 20 minutes in because it's my life. I've been living in the Idiocracy since I was eight years old.
Everyone seems so slow. Their problems are trivial. They only reason they don't live well is because they just aren't interested.
- - - - -
I have high hopes that the talking computers will make good rulers...
> The problem isn't thinking machines, it's letting machines do the thinking for you
...and then I like to point out that many of our social structures (including corporations) are, essentially, machines. They're just machines running on meat instead of silicon.
Goodhart's law in action. If "neatly composed and verbose documents" is the metric higher ups use to judge product managers, then documents will become neater and more verbose to the detriment of everything else.
In the end, the alignment problems that plague AI safety research are all playing out in company structures every day. It's hard to give a metric that measures your intended outcome, and any proxy leads to unintended consequences once there's an easier way to reach that proxy.