Especially in how people think machines work: the user facing interface is optimized for these naive interpretations, while the true operation is hidden because of complexity that "would confound us".
> [Good test takers] are prone to adopting shallow heuristics that succeed for the majority of [tests], instead of learning the underlying [facts] that they are intended to [assess].
Semi-related, I wonder where one draws the line on which heuristics are deep vs shallow in test taking. If a question assesses vocabulary knowledge by asking you to pick an appropriate word from several choices, "these are words I've seen before even though I don't know what they mean" && "here's a choice whose Latin roots plausibly give it the right meaning" seems shallow to me. It allows the test taker to select the right answer without actually knowing it. At the same time, it's not shallow in that you're incorporating quite a bit of knowledge of the language as well as the context you've developed over years of reading.
---
As soon as the conversation here turned to test taking, the thread immediately reminded me of the scene in The Wire where the students pick the right answer from the blackboard by seeing one of the choices has a lot more stray chalk marks beside it, indicating it was pointed at during the previous period.
"The answer is B five. B five's got all the dinks"
The point is, good test design is actually hard.
Paul Nation's "Vocabulary Size Test" has a good approach to the challenge of determining how many words someone knows. Even with this relatively simple-to-state metric, there's a good bit of subtlety in getting a defensible result.
https://www.victoria.ac.nz/lals/about/staff/publications/pau...
Yes, I think this is how the human mind works, kind of, but I don't think the part that provides the grid is like the verifiers or provers we have implemented. There is something that provides a substructure, and I think that you can see in mental illness where it's not functioning properly, but even when healthy, it's not that logical. When humans do logical reasoning, that's a very high level activity, superimposed on top of the other layers, IMO.
I think that the "snap to" part is going to be very difficult to develop because we are not conscious of it. Where to even begin? It might be fruitful to study instances where it isn't working properly - like thought disorders in schizophrenia.
So yeah, I think human thinking may have severe flaws that are rather similar to the ones discussed in the article, but that doesn't prove that current AI has all the components needed to match humans.
Certainly not. But it is similar in that confidence increases once one decides to believe in something.
> I think that the "snap to" part is going to be very difficult to develop because we are not conscious of it. Where to even begin?
Begin by becoming conscious of it. Learn to recognize when you have made a decision, or a judgment, learn to feel what that feels like when it is just about to happen, and then pay attention to what leads up to it. Is it some kind of subconscious ratiocination? Then slow it down and figure it out consciously. Is it a subconscious review of some sense data? Then review it with higher awareness. Intuition and introspection will teach more than trying to watch it break.
I adamantly agree, there is probably a lot of generally applicable (i.e. not domain-specific) "tricks" or "implicit insights" that mammal brains use which we haven't discovered yet.
Another good example is, how it's often stated that humans have no trouble learning things "in one shot", but how on closer inspection that may or may not be true and is very hard to verify:
Consider how when we have a clear negative or positive experience (say being thrown out of class or standing in a corner facing the wall versus getting a compliment on your work etc), we afterwards typically replay the sequence of events and how it led to the current situation. It is unclear if this replaying occurs only for the highest-level abstract thoughts possibly centralized into a couple of regions recording and replaying episodic memory, or if this is actually happening in a more decentralized way throughout the brain while we are only subjectively aware of this episodic aspect at the conscious level. If this self replaying of the last lessons that apply locally is spread and operating independently throughout the brain, could this be an explanation of the brain wave patterns? Is this origin of the "aha" signals or is that just a fantastic concoction of the reproducibility crisis?
Local feedback signals (numbering on the order of number of synapses) vs global feedback signals (numbering on the order of kinds of hormones, diffusing neurotransmitters, blood sugar etc) ===
We still know very little of the feedback mechanisms in the brain, is it a low number of global feedback signals, or a large number of local feedback signals?
A) global feedback: the hardcoded feedback mechanism is primitive wetware (listening to the low number of chemical signals) while the high-speed feedback is learned by neurons influencing each other in the prograde direction as emergent behaviour resulting from this primitive wetware (the only feedback being through axons literally feeding back as opposed to feedforward networks); or
B) local feedback: if the high-speed feedback is hardcoded wetware too, i.e. retrograde signalling of adjoint derivatives across the synapse, which would mean that the reverse accumulation automatic differentiation we use is closer to biology than currently accepted.
For example in A) suppose there is a low bandwidth (low frequency) feedback reward signal, say blood sugar rising after eating a given sweet for the first few times, then the shortest path between the taste buds and the chewing and swallowing motor neurons would improve their weights for intake, a first level anticipation, but then later seeing or touching the sweet or actively seizing one might cause the previously trained neurons to train the newly recruited signal paths by some other neurotransmitter reward by anticipation.
Fast-approach-but-imprecise adaptation vs Slow-approach-but-precise fine-tuning ===
Another facet is that we currently have "1 training regime" for digital neural networks, with which I mean: we use gradient descent both to 1) adapt the weights from a totally randomized initial state to a somewhat acceptable state, and then after a nonexistent pause, 2) to continue improving the weights near the top of the hill (or bottom of valley...) of the scoring landscape. We have no guarantee that nature uses a single regime, let me concoct a hypothetical (and thus improbable) guess: perhaps it uses gradient descent to get the weights approximately where they should be, but then a per-synapse lock-in amplifier correlates (multiplies over time 2 signals while summing) the positive / negative feedback with its own positive or negative variation on the weight, smoothes this product-sum correlation signal (low pass filtering) to get a very high precision feedback signal which has higher precision than a single instantaneous "sample" of feedback (local or global). Throughout the brain synapes some would be closer to the instantaneous feedback signal regime (in response to new lessons. or for short term memory), while others would be closer to the LIA regime, for longer term memory and / or precision. That would be trillions of lock-in amplifiers in a single human brain...
EDIT: in fact SGD stochastic gradient descent can already be seen as a global LIA mechanism, but with all weights / synapses using the same low pass filter. In theory we could give each synapse / weight not only a weight but also each their own timescale (or example count scale, the tau in (alpha) and (1-alpha) multiplier factors when expressed as a filter), and adapt the timescale depending on the local feedback signal (adjoint derivative) in the backpropagation algorithm. To vary the weight just use weight times (1 + 0.001 times the randombit bit r), and multiply it with the feedback signal (the component of the usual gradient, i.e. the derivative that corresponds to this synapse / weight). A similar trick could be used to vary the timescale. One could also hardcode the timescale for certain synapses as a hyperparameter to forcibly locate short term vs long term memory pathways.
EDIT2: As an example of the function of weights that would settle (or were forced) at long vs short timescales at low or high levels:
low level, short term: adapting to reverb entering or leaving a church, adapting to noise when entering or leaving a party,
low level, long term: associating spectral peaks among frequency bins (the fundamental and higher harmonics for sounds of different timbres would remain in roughly the same place, regardless of background noise)
high level, short term: unimportant small talk, or important but quickly dealt with information (did I already pay this drink or can I walk away?)
high level, long term: what is my pin code?
Speak for yourself! There are plenty of other ways. Gradient descent is just currently in fashion.
I think we're more likely to create AGI than to understand how the brain works in our lifetimes.
I love this idea, thank you for sharing it.
EDIT: if you are into ML and like this idea, certainly check out MetaMath, the book is very accessible, and the author (Norman Megill) is extremely friendly and helpful. It takes perhaps a few days to a week to learn, study, and reimplement the verifier. One does not at all need to actually study the whole of the set.mm database (i.e. all the theorems and proofs) to implement the snap to makes sense I propose above. It simultaneously gets rid of the expensive labeling for correct vs incorrect proofs on one hand, and is a path to self-explaining AI on the other. I can not at all exclude that "soon" AI will be proving math conjectures faster than humanity can conjure them. That would be quite an interesting world! At first it would be trained on generating proofs for propositional logic statements (the challenges could be generated by a prover doing a "random walk"), then first-order logic / set theory / numbers.
Word2vec results in similarity scores you can use to find analogies. Great! Now how do you know which analogies to look for? Which ones to remember? Which ones to pursue? You need some kind of intuition for where to look. So you need (1) a way to move around the search space and (2) a way to know when to backtrack, or when a line of inquiry is unfruitful, and this last is precisely the capacity for boredom. If you can't come up with new ideas and you can't get bored, you'll suck at finding proofs just like you'll suck at math or problem solving in general. There's also a danger in getting bored by the wrong things. So how do you develop this boredom intuition?
We choose what to think about, but until we make conscious machines, we are probably the only creatures that have to make this choice.
extracting the analogies is not that hard (a naive brute force is looping over combinations of 4 words and testing how close the relationship holds), but more importantly, one doesn't need to extract the analogies, the neural networks utilize these analogies implicit in their embedding.
I have never seen the boredom of neural networks investigated, the closest concept that comes to my mind is surprisal, which is widely understood since Shannon & information theory...
Neural networks can't get bored because they don't decide what to think about. They are like a total functional programming language with no control flow constructs. This is why they always give an answer in the same amount of time, regardless of the input vector.
Surprisal is only a measure of information (given a distribution) and is only distantly related to what I'm talking about. However, if a neural network could choose to think more, choose to get new data and reconsider, choose to go back and look at that one from a couple minutes ago... then you'd also want it to have the ability to get bored. AKA to recognize when it is spending resources on an unprofitable line of inquiry.
If one thinks about loose associations that newcomers make compared to experts in a domain, this seems very similar to me.
When designing a vector graphic one can enable "snap to grid", similarily at some point we will have to "snap to makes sense" by means of verifiers or provers. "Why" type questions ultimately ask for a proof or derivation, which in the past (before the advent of logic, to which a philosopher would have to carefully abide) was not objectively verifiable, but at some point it is foreseeable that neural networks will be asked to construct (or append) a formal belief system, predict a conclusion, and justify it by supplying a proof.