Yes, the "paperclip maximizer" already applies: corporations. I can't remember the post, but someone wrote that corporations are effectively an AI which maximizes profit. In many ways, we have already invented AI; its objective function is to maximize profits, often to the detriment of, well, everything.

Would that be Charlie Stross's essay (and the subsequent CCC talk): Dude, You Broke The Future?[0]

0: https://media.ccc.de/v/34c3-9270-dude_you_broke_the_future

That's the one! Thank you for finding it.

no - a corporation is not the same as an AI that effectively maximizes profits.

An AI that is more intelligent than a human may be able to come up with plans for profit maximization that no humans can come up with.

A corporation is only going to be able to come up with a plan that a human _could_ come up with.

A corporation is merely a sociopathic human intelligence that is hell bent on profit-maximization.

Think of what it would take to produce a phone, for example. A single person could not possibly know enough to do so. Our cleverest work their whole lives just to know enough to design some small part. There's just too much to know, too much to orchestrate, coordinate. Even to build one, let alone a million.

Yet, a single corporation is intelligent enough to do this.

I think we already have achieved super-human intelligence.

"I, Pencil" by Leonard Read is very relevant to this thought

https://en.wikipedia.org/wiki/I,_Pencil

Arguably, the ability to divide labour and ability is precisely human. That said yes, it is super-individual intelligence and it does scale incredibly well.

Perhaps a nitpick, but a corporation might act to combine the profit-maximising virtues of a number of different humans, to give a result that no single one of them is capable of. Combining the innovation of one with the amorality of another, for instance.

Related to this, on the morality dimension, corporations can introduce a diffusion of responsibility [0] and enable a collection of humans to commit to profit-maximising acts that no single one of them would do. Less ominously, teams of cooperating researchers may do something similar on the innovation dimension: their combined efforts act as a 'super researcher' beyond the ability of any single human researcher.

[0] https://en.wikipedia.org/wiki/Diffusion_of_responsibility

There are algorithms for combining many imperfect models into something much closer to reality than the best model - 'wisdom of crowd' effects etc. There are definitely human organisations that come up with smarter plans than could be come up with by the smartest human in those particular organisations.

If you're talking about AI with 'smarter than us the way we're smarter than chickens' as the standard I suppose I'd have to agree, but it seems like there's a big, very vaguely defined stretch between 'very smart human' and 'makes a very smart human look like an imbecile'

Universal Paperclips is a wonderful game:

http://www.decisionproblem.com/paperclips

well said, this is the premise of Das Kapital in modern words.