Re:Self-supervised learning: The dark matter of intelligence
by jtsiskin on Friday March 12, 2021 @05:12AM
(#26434025)
The person isn’t changed, just like the weights aren’t changed. You’re missing the training step - after the person fills in the blank, then we tell them what was supposed to go in the blank. If I had you try to paint the the rest of the face of a portrait, you may do a poor job, but then if I showed you exactly what it was supposed to look like, you can see what you did wrong, and next time you will draw it better.
Re:Self-supervised learning: The dark matter of intelligence
by fantod on Friday March 12, 2021 @12:03AM
(#26432378)
If the person who filled in the blanks already had a good grasp of the language, they probably didn't learn much. However, fill-in-the-blank exercises can be a good language learning tool for children as well as adults learning a foreign language.
Re:Self-supervised learning: The dark matter of intelligence
by just1morething on Friday March 12, 2021 @12:17AM
(#26432433)
There is something special about that "lock and key" moment when you feel that satisfying "click" as your response opens the next opportunity.
Re:Self-supervised learning: The dark matter of intelligence
by social_quotient on Friday March 12, 2021 @04:18AM
(#26433720)
Escape rooms for language models :)
(Context https://en.m.wikipedia.org/wiki/Escape_room )
Re:Self-supervised learning: The dark matter of intelligence
by visarga on Thursday March 11, 2021 @11:54PM
(#26432341)
One could make a similarly long list of poorly understood concepts in psychology.
Re:Self-supervised learning: The dark matter of intelligence
by just1morething on Friday March 12, 2021 @12:14AM
(#26432416)
I've found that plucking words from paragraphs has, on occasion, brought them more firmly under my control-- most of them remain mysterious to me when they pop up in papers on programming or psychology.
Rather than list "ideas" or "concepts" I try to make a record of what appeared and the response I had to its appearance.
How are we to solve difficult problems if we stop mentioning them or their difficulty?
Here are some words that I read but did not grasp firmly: learn, data, task, train, intelligence, general, model, skill, label, language, understanding, reality, observation, predictive, objects, concepts, act, hypotheses, knowledge, 'common sense', 'dark matter', 'artificial intelligence', teaching, classify, supervision, autonomous, 'self-supervised learning', recognize, patterns, representations, processing, systems, pretrained, vision, real-world, helpful, promising, 'energy-based models', prediction, uncertainty, 'joint embedding methods', 'latent-variable architectures', reasoning, 'predictive learning', 'supervisory signals', signals, structure, unobserved, property, input, 'co-occuring modalities', 'unsupervised learning', feedback, reinfrocement, 'downstream tasks', meaning, syntactic, word, associate, probability, vocabulary, 'convolutional network', network, 'prediction uncertainty', 'predicting missing words', computing, softmax layer, probability distribution, energy, incompatible, computer vision, and so on.