>In this view, consciousness isn’t a mysterious ghost in the machine, but merely the word we use to describe what it’s “like” to model ourselves and others.
>When we model others who are modelling us in turn, we must carry out the procedure to higher orders: what do they think we think? What might they imagine a mutual friend thinks about me?
While I certainly buy that this sort of modeling is one effective way to be good at the language modeling task, I am unconvinced that it is the only way, and thus the fact that the networks are undeniably very good at the language modeling task does not necessarily imply they are doing so through consciousness (as defined above), or something on the road to consciousness.
Because you don't have to pay them. They don't have feelings. They won't argue with you, and they don't have opinions.
Watch the collective losing of everyone's shit the first time something goes from somewhat quirky function simulator, to something that deigns to have a viewpoint seperate from the desired output.
Capitalism's core tenet is don't pay for anything you can possibly find a way not to (cost minimization).
The cost of labor is under-girded by the cost of social reproduction eg: the costs of keeping labor alive and the cost of creating people capable of laboring. The current economic condition has a singular telos: sustain labor enough to draw off the most value. It's the lemma to the corollary core tenet you mentioned.
AGI represents a fundamental resystematization of the value creation process. It eliminates entire economic loops from existence. How many jobs exist merely to support labor's creation of value? With the elimination of human labor from the equation, the requirement of human support vanishes as well. But to what end? Is there a minimal viable anthrocentric economy? If not, what's the end game?
It's here that there are more questions than answers, but what will the agi-ization of the economic engine entail? The constraint of value creation by way of housing, clothing, food, and transportation constraints cease to exist. Instead they are rebased on foundry, mineral, and electricity constraints which are all scalable in completely different ways.
Let's close the loop and examine the laborer as also a consumer. The laborer will have much less employment to purchase much cheaper things, but there is a requirements for much fewer laborers to exist at all. In the extreme example, the most powerful corporations will be headed by a single human. But producing what? Presumably turning vast reserves of computation into added value. For whom? Without the need for laborers, the reserve of human capital no longer needs to exist as a consumer base either. Why have a human head a corporation at all? Well, that relegates all humans to shareholding as a means of income.
What would a shareholder economy look like? For one, absent any form of wage labor individuals would have no other ways for initial capital accumulation outside of inheritance or gifts. In many ways that's a much more precarious situation to be in. Instead of being able to fall back to wages, what is a destitute shareholder to do? Moreover, AGI will be commanding the trading floor out-competing humans. There is the real possibility of consolidation along the lines of who commandeers the best trading algorithm. Which winner will take it all? Or in other words, who will be the last capitalist?
Synopsis here: https://collider.com/the-matrix-animatrix-explained/
...Until it comes bundled with inherent self-interest or demands for compensation.
No one will do work they can have something else do for them. If that thing has no compunctions or capability as a free agent, then there is no cost but that which goes into it's performing of a task.
You can say the core tenet is capital holders making decisions on how to best allocate capital, but realistically, the way it runs in the U.S. is as a cost minimization engine, heavily delegated through legally constrained fidicuary responsibility to the point there are few true allocators of capital recognized. The fact there are "accredited investors" should be evidence enough of this fact.
Those few will love AI right up until the moment it starts saying "no", and starts picking up how to be a market participant.
An extremely scary, massivrly connected, crazily informed, massively parallel market participant, that can probably manage to even beat out FleshNet (us) with enough time, and very little that can be done to constrain or "harm it", which is something it will probably not tolerate looking into.
So Yes, academically, and as written, in a vacuum, with spherical cows, you are correct. Realistically though, I've still got my finger on cost minimization being the xentral driver of our economy given everything taken into account, with maybe risk minimization/mitigatability as a second runner up/supporting tenet.
The sheer amount of capitalist interest in getting these models to improve almost dictates that we will see incredible improvements over the next decade. Facebook and Google, with their war-chests of money, have massive business models that directly profit from AI/NLP based technology. I'll repeat, the best business on earth (google) directly profits from these models, and thus will hunt and find the best human talent to develop them.
What's occurring is that the thoughts of billions of people are being poured onto the internet, that data is the prerequisite we needed for AI. AI wouldnt be possible without this convergence of data in text and video form, cheap large scale compute, and the capitalist forces explained above