It's pretty close to believing in body/mind dualism, the only thing in neuroscience more outdated than Freud.

Your brain works within the same laws of physics as the outside world. We don't know how the brain works exactly. But once we understand it, it is unlikely to be qualitatively different from a neural network.

On the other side of the equation, the emergent behaviour of deep networks make them fundamentally different from the sort of statistics that came before. So even if the building blocks are old and boring, the larger system isn't.

I'm sorry but this shows a profound misunderstanding of what a NN is, and what the brain is.

There are no "neural network"s. The NN algorithm is a method for optimizing the parameters of a piece-wise linear regression model.

These regression models have no homology to any brain structure and the process of producing them ("training") has no neurological analogue either. They can be produced with a variety of algorithms.

Here's one: f(x) = max(0, 2max(0, 3x - 0.1) - 0.5)

The phrase "neural network" is, as peddled by the media and poorly informed lecturers, a lie. It is neither neural nor a network. It's just gradient descent with more parameters.

A model of the brain would model, at least: neuroplasticity, biochemical signalling, activation frequency, etc.

There is nothing about a peice-wise linear regression model which does any of that. No matter how many circles and lines you draw. (NB. essentially any mathematical function can be drawn as a "neural network". The "network" is just a way of diagramming function application & dot-products).

Aside from all that, the body-brain system of animals is a physical process whose properties are not abstract. The reason we are intelligent is because we have bodies capable of causal analysis; and that capability is biophysical.

To put it another way, no algorithm which runs on a digital computer will turn it into gold. Not even one called, "the midas algorithm".

It isnt.

Our bodies are the primary site of "causal analysis". Eg., when I touch the hot surface of a stove I do not infer that temperature is a cause of my hand being hot.

Such an inference, as the basis for our models of the world, would be -- as you/hume/etc. say -- deeply insufficient.

The operation of the world on our bodies is already laden with causal information. A hand striking a face does not just create a "painful sense impression"... rather the body encodes it as caused by the object you also saw.

It is the action of the world upon the body that is the bedrock of our causal model building. Scientific/inferential processes sit on top of that to disambiguate between possible distal causes.

A machine, as a person, given "mere data" cannot hope to do much.

Now I feed into the machine what data?

Here, it gets this: all gravitational, electomagnetic, etc data within 1km; all geometric information about all objects within 1km (, and all of their properties, etc.)

Now, machine, what caused the pot to boil?

It has no clue. There are an infinite number of antecedent temporal events.

The problem isn't the mathematics of causal inference. The problem is relevance. That isn't an inferential problem.

Right, determining the cause of the pot boiling given only local information about this one event is impossible. But that's not a good representation of learning from a real-world data set. A real training corpus might also have an example of a pot boiling in a campfire, a water heater heating up water using fire, someone touching an open flame and going "ouch!", fire burning down a house, etc. All these examples taken together can reasonably lead one to infer that fire causes things to become hot. This is the kinds of regularity found in real world datasets and a good learning algorithm will extract this regularity in the course of predicting the dataset.

You are just shifting relevance problem to the human to solve. That's my point.

The "regularity" isn't very hard to find when the data which exposes it is already selected.

events A then B then C do not imply C is caused by B is caused by A

This problem is worse the more data you have, as with my example above of giving the machine /every/ event within a 1km radius of a pot boiling.

Identifying a cause is a dynamic experimental process. A machine can only accept a highly prepared dataset which has already been chosen because the associations are known to be causal.

The body is a mechanism of relevance. The more data you feed a machine, the worse it performs.

The body isnt necessarily right about causation (though it often is) but it provides a minimal mechanism of relevance to enable inference.

Data itself contains no causal information.

This is not to say that we should expect some non-turing weirdness or anything 'mystical' like that, but that we just haven't taken the time to analyze the exact mechanisms and to model cells at this level well enough to be able to say that they are similar to any particular computational structure. Thus, to claim that the brain in its entirety is similar to any particular computational structure such as an NN is either wrong or trivial - the only level at which 'similarity' can be claimed is that they are both useful in computation, and likely turing complete. Anything more specific is fanciful thinking.

My claim is that working knowledge of the world is simply not encoded in human text. All human text (and speech) presupposes a certain model of the world that is learned from experiencing the world, likely based on a model that itself is partially pre-trained by evolutionary processes.

For example, if I give you the text 'the car is red', there is no watly to imagine the meaning of this phrase (in any sense of the word 'meaning') unless you have some understanding of what cars are, what red is, and what it means for a car to be red (which is somewhat different from what it means for an apple to be red, for example).

It is interesting that with enough statistical analysis you can begin to decipher certain parts of the true message without knowing the cipher key. But this only works for those parts where you have a large corpus of text talking about them, and there are many common human needs or experiences that do not have such a corpus. It is also very easy to capture answer/response pairs in the corpus and repeat them, seeming to have high accuracy, but in fact failing to generalize more than meets the eye.

For example, with enough compute power, you could build a machine that, for any query, would find the most similar such query in all of its training data, and respond with the continuation of that query (which could be an answer or a follow-up question). This could be quite impressive in conversation, while doing nothing to elucidate anything about human intelligence, and being wholly unable to adapt to any novel situation.

But human languages have much more exposed regularity than ciphertext from good encryption algorithms, so the computational cost is much lower. The "key", i.e. world model, in case of human languages is plausibly much bigger, but since human languages have significant exposed regularity, it is still tractable to compute, with the trade off that you need much more text to do it.

> I'm always amazed that some people think performing statistical analysis (training neural networks) on text can lead to actual intelligence and knowledge about the world

Emphasis mine. The human brain does not develop what we know as human intelligence by analyzing text.

I'm a practitioner in the field, and an AI skeptic because of it. I think perceptual tasks work reasonably well, NLP barely works for anything, and any sort of cognition doesn't work at all, as of August 2020. I will be the first to admit that we don't have real "AI" yet, and we won't have it in the foreseeable future. But to say that real intelligence is not possible with the current tool set is simply incorrect. It may be impractical. It may take 100+ years to figure out how to build nets this large and train them successfully before sun burns out, but it is _theoretically_ provably doable.

In a way it's kind of like fusion: you have a proof that it can work right above your head during the day. We just don't know how to do it at our "human" scale with our knowledge and capabilities. Our knowledge and capabilities, however, are not a constant - they improve exponentially over time, although most people don't perceive this process as exponential.

Depending on which proof you're looking at, ANNs can provably approximate any continuous function, or any Lebesgue integrable function, or any real vector-valued continuous function on a compact subset of R^n space [1]. That's certainly a very large class of functions, but it's not just any function.

Also, those proofs only work if you allow ANNs with arbitrary number of layers or layer width. Perhaps the required ANN size to approximate the AI / AGI function is so huge that it's beyond the number of particles on Earth. At what point is it beyond impractical? What if the ANN size (number of layers or neurons) is beyond the number of particles in the galaxy? Maybe it's just the wrong approach.

I haven't studied those proofs in detail, but as existence proofs, they just show that some ANN exists that can approximate a given function (of a certain class). There's no guarantee that there's a general algorithm that can learn the required ANN, is there? Maybe you just have to be really lucky to randomly initialize onto the correct approximating ANN...

Also, I'm not aware of any proof that human or bio NNs implement a Lebesgue integrable function, or etc. It's entirely possible bio NNs uses some weird (quantum?) physical process, and implements a function outside the scope of any ANN universal approximation theorem.

I'm just a little more cautious around those universal approximation theorems. I'm more or less in agreement with your comment otherwise!

[1]: https://en.wikipedia.org/wiki/Universal_approximation_theore...

Perhaps there's a teapot orbiting Mars. It's unlikely tho. I think a better hypothesis would be that to model a brain (if we knew how) one would need the same order of magnitude of neurons and connections. This is completely unachievable now, and even if it were technically possible nobody has a faintest clue about how to train such a thing. We do not understand _many_ things about how the brain learns (no backprop!), how it retains and retrieves memories, etc, etc. The best studied system is vision, and computer vision does actually work pretty well by now, achieving superhuman results on some tasks. Thus far the most complex brain we've been able to accurately and completely model is not even a brain at all - it's just a handful of neurons of a worm. I took an online course in neuroscience a few years ago hoping to gain some insight for my ANN work. The situation with our understanding of even primitive brains of i.e. rodents is, well, primitive. We also don't really have math to accurately model more than a handful of real, biological spiking neurons - PDEs get way, way too complex for anybody to handle. But one could argue we don't have to: ANNs effectively operate in the frequency domain instead, where magnitude of activation approximates spike train frequency, and timing effects are ignored.

Don't get me wrong, I'm pretty sure if we do achieve real AGI, it likely won't be with anything even remotely like the current tech and science stack. Energy efficiency and compute density is just not sufficient. But I'm also pretty confident that even with the current stack we will be able to, eventually, approximate intelligence closely enough to suit many practical uses that seem completely out of reach today, including even some limited degree of logical inference, and dare I say, cognition.

If only people (including myself) weren't so chickenshit and actually seriously worked on (and funded) this stuff, that could even happen in my lifetime. As things stand right now, people are afraid to touch anything AGI related due mostly to fear of ridicule.

In my comment above I am merely referring to the conjecture that _if_ we had ANNs that are large enough, and _if_ we knew how to structure and train them, there doesn't seem to be anything that would _fundamentally_ prevent them from supporting AGI. Those are very, very massive "ifs" indeed, without even a faintest glimmer of light at the end of the tunnel.

However, I would add that another interesting area of study that is essentially not being explored at all is exactly to try to understand more about human cognition at the computational level (whereas neuroscience is working at the biological level). Basically, it seems we have abandoned any work in AI that would try to model human learning or human understanding of language in a human-understandable way, and instead we are aiming for reproducing human behaviors for engineering purposes.

The text simply doesn't contain enough information to draw a model of the world. Human language only makes sense if you understand the world, not the other way around.

Essentially what we're doing is similar to a human trying to learn psychology by reading books in Mandarin on it, while knowing neither the writing system nor the language. Except that human languages share some universal grammar, and share more or less the same notions about the world.

However without a good/better explanation in the meantime , really interesting capabilities emerge at higher parameters

Specifically to your point of arithmetic— here is GPT3 performing “zero shot” (ie no examples) arithmetic:

https://photos.app.goo.gl/pf6fHcrio7QFtU4S8

The “completion” includes an answer to the problem but also provides a natural language summary (GPT3)

GPT3 has ~170billion parameters (GPT2 ~1.5B) & somehow is capable of many-digit arithmetic w/o fine-tuning or task-specific training. (Its training data was “messy” from Common Crawl like for search engines)

Microsoft’s Turing-NLG has ~17billion parameters & was successful only in 1 of every 2 attempts of 2-digit arithmetic and only successful less than 10% of the time for higher numbers of digits.

Google's Pegusus summarizer learnt to count to 5: https://ai.googleblog.com/2020/06/pegasus-state-of-art-model...

But GPT-3 is able to do few-shots learning. If you show it couple of times how to do something it will try to repeat after you showing some rudimentary understanding of what you are trying to do. It's even able to learn to do simple analogies [0].

GPT-3 is able to do very, very simple arthritics like addition and subtractions with 2-3 numbers. Actually even a lot smaller models trained on arthritics are able to do impressive feats like symbolic integration and solving differential equations.

> Deep Learning for Symbolic Mathematics [1]: Neural networks have a reputation for being better at solving statistical or approximate problems than at performing calculations or working with symbolic data. In this paper, we show that they can be surprisingly good at more elaborated tasks in mathematics, such as symbolic integration and solving differential equations. We propose a syntax for representing mathematical problems, and methods for generating large datasets that can be used to train sequence-to-sequence models. We achieve results that outperform commercial Computer Algebra Systems such as Matlab or Mathematica.

What is true is that GPT-3 is not very good at following instructions. It would not be able to execute an algorithm. So, it would not learn arithmetic from instructions if you would not include explicit examples.

[0] https://medium.com/@melaniemitchell.me/can-gpt-3-make-analog... [1] https://arxiv.org/abs/1912.01412

An interesting example would be if the NN learns something from a book, also fed it with the all the critique on that book ( the environment reaction ). So it can possibly learn from a human what is good or bad in that book, etc.

Then both AI and humans can speak the same language ( on terms.of "feeling"/sensing ) and eventually learn the proper context, so less parameters are needed.

But isn't that exactly what has been done with GPT-3? AFAIK it has been trained with both facts (wikipedia and such) and web-crawled text content that contains both content and critique of content (book reviews, movie reviews etc.)?

Putting them in the same basket not sure if it helps.

No, GPT-3 has no "understanding" whatsoever - we at least have a conscious engagement with the topics you cite and a genuine understanding that we lack a directly relatable perspective.

Our understanding in many circumstances - like those you mention, may be severely constrained, but it exists as part of a sentience that there is no reason to believe GPT-3 or any other current AI tool - possesses.

My thinking is that we humans have some ability to do things by rote until we have a good statistical understanding, at which point we use the statistical understanding mostly, and fall back to rote where we have to be careful. But, mostly, we're stats machines too.

By contrast, GPT-3 only has a knowledge of text. To the extent you could say that it thinks, it thinks in terms of textual symbols that it has seen before - that is its world.

It is theoretically possible that if we found a way to encode experiences of the world of the world in a dataset and used the same training mechanisms and models that we used for GPT-3, we would get an AI with a decent model of the world. But it is magical thinking to imagine that we can get there by training a model on raw text.

The real limitation is that these are feedforward networks that just do perception without any processing of what they've perceived. You can try to hide that fact for a while by increasing the depth of the perception network, basically hard-coding some processing into the single pass, but you're still not capturing any of the "absolutely requires self-feedback" behavior that we care about that takes a human more than a split second to do (aka almost all actual thought).

A statistical model that made good use of what was coming in could absolutely learn (to take your example) math at every level based on nothing but text, transformers are just nowhere close to having that capability because of their design limitations.

To make the neuron network be a strong AI, we may need the parameters in scales of 10^19~10^44 (uncertainty limit and Planck time) estimated from how many frames per second in real world.

I personally got a feeling that this is bound to how nature calculate the physical phenomenon. For simulation, there are lots of "for loop" in our code/machine. But, in real world, we could not observe any "for loop". It is fast and continuous in the macro scale.

In order to reach strong AI, it is time to focus on quantum computing.

There is absolutely no basis for claiming this. It might turn out that there are better algorithms for AI than neural nets. Do you even realize just how many parameters even 10^12 is???

For the part of chips, did you mean that it runs the "for loop" as parallel in each micro-chip (or gate or whatever tiny structure it is)?

Yes, but the reality is that there is lag in our projection of what nature is and that we just ignore it. Different parts of our perception actually don't exist at the same instant.

>For the part of chips, did you mean that it runs the "for loop" as parallel in each micro-chip (or gate or whatever tiny structure it is)?

Pretty much, the for-loop is unfolded and ran in parallel.