Re:Towards a conversational agent that can chat about anything
by red75prime on Wednesday January 29, 2020 @03:30AM
(#22178542)
I prefer to think that we need relevant responses rather than new. And I'm not worried whether it's lookup table, given that lookup table that is required wouldn't fit our universe.
Such models have to make generalizations to fit into mere 2.6 billion parameters as ten word sentence presents a possibility of one billion billion different inputs.
Generalizations are limited, of course. Arithmetic doesn't seem to be generalized, for example, which isn't that surprising given that people need around 4 years to be taught to do it (as opposed to learning it by analyzing random texts), and then they usually need pen and paper in addition to their 100 billion neurons to do non-trivial calculations.
I'm not saying that current network architectures are capable of all generalization humans can do. And we can't teach them like we do with children.
Such models have to make generalizations to fit into mere 2.6 billion parameters as ten word sentence presents a possibility of one billion billion different inputs.
Generalizations are limited, of course. Arithmetic doesn't seem to be generalized, for example, which isn't that surprising given that people need around 4 years to be taught to do it (as opposed to learning it by analyzing random texts), and then they usually need pen and paper in addition to their 100 billion neurons to do non-trivial calculations.
I'm not saying that current network architectures are capable of all generalization humans can do. And we can't teach them like we do with children.