Re:Turing-NLG: A 17B-parameter language model
by gchq-7703 on Tuesday February 11, 2020 @04:28AM
(#22297174)
I'm fairly sure that these models don't work solely on the previous word, but instead are able to remember some level of information from history.
Otherwise, you'd reach a word like 'and' and couldn't possibly follow it with a logical statement that follows on from the previous part.
Re:Turing-NLG: A 17B-parameter language model
by ragebol on Tuesday February 11, 2020 @05:51AM
(#22297517)
This is why I said 'words', multiple :-).
My point being that these generation models should be conditioned on something more than just word history, like something they want/are instructed to express.
Does anything like this exist?