News for hackers. Stuff that matters.

A comment on We Found an Neuron in GPT-2. Parent comment

Re:We Found an Neuron in GPT-2 by KhoomeiK on Thursday February 16, 2023 @07:37PM (#34828989)
Several papers have explored emergent linguistic structure in LMs. Here's some early introductory work in this space. Despite having explicit syntax parses etc as input, models seem to learn something like syntax.

https://arxiv.org/abs/1905.05950

https://aclanthology.org/N19-1419/

https://arxiv.org/abs/1906.04341

[ Parent | on HN ]
Re:We Found an Neuron in GPT-2 by compressedgas on Friday February 17, 2023 @08:57AM (#34834675)
I would add https://www.pnas.org/doi/10.1073/pnas.1907367117
[ Parent | on HN ]
Re:We Found an Neuron in GPT-2 by KhoomeiK on Friday February 17, 2023 @02:14PM (#34838982)
Yeah this is more or less a re-release of the second work I linked, except for a broader audience and more targeted at linguistics / cognitive science researchers.
[ Parent | on HN ]