Re:The Unpredictable Abilities Emerging from Large AI Models
by truculent on Friday March 17, 2023 @08:20AM
(#35196103)
So it sounds like the mechanism is something like stochastic memoisation?
[ Parent
| on HN ]
Re:The Unpredictable Abilities Emerging from Large AI Models
by pedrovhb on Friday March 17, 2023 @08:46AM
(#35196382)
I don't think "memoisation" is an accurate word for this; it implies doing the computation once, and storing the result in cache to return later. It's more like replacing your Python code with a SQL query of the LLM's understanding of what it's supposed to do, which may or may not be correct, and executing it on its "database" of knowledge, which may or may not have the correct data.
Re:The Unpredictable Abilities Emerging from Large AI Models
by mistrial9 on Friday March 17, 2023 @08:50AM
(#35196425)
this is new to me but.. a quick read of Wikipedia [1] later, it appears that this decades-old method takes a goal, and then uses branching and recording to find paths that differ the least from the goal. The article mentions that in practice, the curse of dimensionality is so bad here that approximations are often used in practice. Does that capture it?
[1] https://en.wikipedia.org/wiki/Stochastic_dynamic_programming