News for hackers. Stuff that matters.
Re:Better Language Models and Their Implications by DenisM on Thursday February 14, 2019 @08:31PM (#19167785)
It is uncanny.

I would expect a language model to pull up different person names for each time that one was called for. For a person name to be consistently used through several paragraphs it is not enough to rely on word co-occurrence.

If I had to produce a text like this I would simply take an existing text and replace randomly chosen words with other similar words (as hinted by amvalo). Similar as in - words that tend to occur in similar contexts. So John->Bob throughout entire text. But that would not be a language model product anymore, and where is fun in that?

I should set aside some time to read this paper.

Re:Better Language Models and Their Implications by dannyw on Friday February 15, 2019 @08:52PM (#19176477)
That’s because the transformer model used has memory.