News for hackers. Stuff that matters.
Re:What Comes After AGI? by lo_zamoyski on Friday November 20, 2026 @11:34AM (#50577371)
Words like "AGI" and "superintelligence" have caused by eye-rolling-muscles to become sore and swole.

But I would make a stronger and different set of claims here:

1. Computation as mathematical formalism is distinct from computation as it exists in the wild (paradigmatically, the human mind). Therefore, computation as mathematical formalism is not identical to computation as it exists in the human mind.

2. Computation as mechanizable mathematical formalism is a syntactic construct. This property is exactly what lends it to mechanical simulation. A purely formal system does not require semantics to operate, though it does require a human observer with semantics to design, recognize, and interpret it (see Searle's Chinese room argument).

3. Computation as mathematical formalism is not identical to computation as it exists in physical machines. The latter only simulate the former through a conventional correspondence between effectively arbitrarily chosen physical events and symbolic constructs locked away in the human observer. If a system is merely executing physical transitions under a mechanistic worldview, its "goals" are parasitic on external human users. There is just no way in which we may objectively ascribe computation to physical computing machines (see Kripke's quaddition argument, for instance). And strictly speaking, physical machines aren't even applying syntactic rules to symbols. A corollary of this is that computation as mechanical simulation is also not identical to computation in the human mind.

4. What is most characteristic of intelligence is not computation, but intentionality and semantics. These are absent from the mathematical formalism as mechanizable construct, precisely because semantics and intentionality cannot be mechanized. So the claim that "intelligence" is some broad and inclusive continuum ignores the fact that there is no alchemy that can transmute syntax into semantics. You can't be just a little bit pregnant. These are differences in kind, not degree. You can't retreat into "neutrality" here either. To call a system "intelligent" in a non-trivial sense rather than merely "capable of producing useful outputs" is already to take a stand on what intelligence is. Quietly adopting a"mechanistic/functionalist stance while leaning on concepts like "information", "representation", "optimization", and "capability" ignores how much heavy philosophical baggage these words carry.

There are no comments on this comment yet.