Maybe I'm asking for an explanation :)
Since you seem to understand the mechanism, can you do a 3 line summary please?
Make a bunch of neural nets to recognise every concept, the same way you would make them to recognise numbers or letters in handwiting recognition. Glue them together with more neural nets. Put another on the end to turn concepts back into words.
For a less wrong but still introductory summary that still glosses over stuff, about 1.5 hours of 3blue1brown videos, #4-#8 in this playlist: https://youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_...
... Oh interesting. And those concepts are hand picked or generated automatically somehow?
> For a less wrong but still introductory summary that still glosses over stuff, about 1.5 hours of 3blue1brown videos
Sorry, my religion forbids me from watching talking heads. I'll have to live with your summary for now. Until I run into someone who condensed those 1.5 hours in text that takes at most 30 min to read...
Fully automated.
> Sorry, my religion forbids me from watching talking heads.
What about professional maths communicators who created their own open sourced python library for creating video content and doesn't even show their face on most videos?
You're unlikely to get a better time-quality trade-off on any maths topic than a 3blue1brown video.
He's the kind of presenter that others try to mimic because he's so good at what he does — you may recognise the visuals from elsewhere because of the library he created[0] in order to visualise the topics he was discussing.
[0] https://docs.manim.community/en/stable/faq/installation.html
There's also a playback speed slider in YouTube. I use it a lot.
The precise mechanism LLMs use for reaching their probability distributions is why they are able to pass most undergraduate level exams, whereas the Markov chain projects I made 15-20 years ago were not.
Even as an intermediary, word2vec had to build a space in which the concept of "gender" exists such that "man" -> "woman" ~= "king" -> "queen".