News for hackers. Stuff that matters.

A comment on OpenAI Codex. Parent comment

Re:OpenAI Codex by nightski on Tuesday August 10, 2021 @04:15PM (#28133768)
Has this transfer of knowledge from one domain to another really been demonstrated by these models/learning processes? I know transfer learning is a thing (I have a couple books on my shelf on it). But it seems far from what you are describing.
[ Parent | on HN ]
Re:OpenAI Codex by sbierwagen on Tuesday August 10, 2021 @04:54PM (#28134385)
The AlphaZero algorithm swapped between board games pretty easily. OpenAI could also have been gesturing at this when they named the GPT paper "Language Models are Few-Shot Learners".
[ Parent | on HN ]
Re:OpenAI Codex by visarga on Tuesday August 10, 2021 @04:52PM (#28134355)
DALL-E + CLIP models show a deep understanding of the relation between images and text.
[ Parent | on HN ]
Re:OpenAI Codex by talor_a on Tuesday August 10, 2021 @06:03PM (#28135272)
they mention in the demo video that the inspiration for codex came from GPT-3 users training it to respond to queries with code samples. I saw some pretty impressive demos of the original model creating SQL queries from plain questions. I'm not sure if that counts as switching domains, but it's something?
[ Parent | on HN ]