News for hackers. Stuff that matters.

A comment on I do not agree with Github's use of copyrighted code as training for Copilot. Parent comment

Re:I do not agree with Github's use of copyrighted code as t... by robbedpeter on Saturday July 03, 2021 @06:23PM (#27725435)
Transformers do learn and abstract. Not as well as humans, but for whatever definitive of innovation or creativity you wanna run with, these gpt models have it. It's not magic, it's math, but these programs are approximating the human function of media synthesis across narrowly limited domains.

These aren't your crazy uncle's Markov chain chatbots. They're sophisticated bayesian models trained to approximate the functions that produced the content used in training.

[ Parent | on HN ]
Re:I do not agree with Github's use of copyrighted code as t... by astrange on Saturday July 03, 2021 @07:26PM (#27725865)
Are they Bayesian? It would be good if ML models were Bayesian (they'd be able to show uncertainty better) but they usually aren't.
[ Parent | on HN ]
Re:I do not agree with Github's use of copyrighted code as t... by robbedpeter on Sunday July 04, 2021 @03:27AM (#27727901)
The model and attention mechanism produces Bayesian properties, but transformers as a whole contain non-Bayesian aspects, depending on how rigorous you want to be in defining Bayesian.
[ Parent | on HN ]