LLMs don't have anywhere near that much code. The algorithms for training and inference are not that complicated; the "intelligent" behavior is entirely due to the weights.
Aside from annoying people who want to understand how things work, it also means you can't ever know if you have a fully optimal or correct solution, all you can do is keep throwing money into the training furnace and hope a better solution falls out next time. The whole nature of it gatekeeps out anyone who doesn't have enormous amounts of money to burn.
Totally agreed that the process of generating and evaluating weights is opaque and not very accessible.
You don't see any meaningful, understandable code. For example, if prompt begins with draw me a picture then jump to layer X.
I'm no expert but I can imagine that to be the problem when one attempts to debug an Algorithmic Intelligence black box.
>“If we open up ChatGPT or a system like it and look inside, you just see millions of numbers flipping around a few hundred times a second,” says AI scientist Sam Bowman. “And we just have no idea what any of it means.”
>To me as an engineer, that is just incredibly unsatisfying. Without understanding how something works, we are doomed to be just users.
AI aren't complicated. They aren't sophisticated math that you can poke at and understand.
They're fucking million dollar spaghetti code that happen to work (for values of 'work').
Those videos are teaching people "This is an if statement! This is a CPU!" And then you can look at 5.8 billion lines of spaghetti code and say "Gee! I understand how this works now! Yay!"