Re:StarCoder and StarCoderBase: 15.5B parameter models with ...
by toxik on Tuesday May 16, 2023 @03:52AM
(#35958819)
How do you know that isn’t emergent given enough embodied experience of the world? This is the big thing with LLMs for me, showing that yes, if you scale up, all sorts of emergent abilities pop up. And LLMs aren’t even embodied, it’s still a very primitive approach and we’re already seeing this level of competence. Truly makes me wonder if I’m not just a stochastic parrot underneath it all.
It's very hard for me to ignore the intuition that there's some higher level cognitive process going on that can pick out abstract concepts and use them to control and focus the lower level "training" that might look more similar to what we're doing in ML these days.