News for hackers. Stuff that matters.
Re:We Found an Neuron in GPT-2 by letmevoteplease on Thursday February 16, 2023 @09:04PM (#34829751)
Reminds me of Stephen Krashen's input hypothesis of second-language acquisition. Krashen argues that consciously studying grammar is more or less useless, and only massive exposure to the language results in acquisition.[1] This is true in my experience.

[1] https://en.wikipedia.org/wiki/Input_hypothesis

Re:We Found an Neuron in GPT-2 by nmfisher on Thursday February 16, 2023 @09:46PM (#34830060)
I'm designing a Chinese learning app and I'd mostly agree with this. My working hypothesis though? Most adult learners don't have the time/patience for massive/lengthy exposure, so grammar lessons are a shortcut to "feeling" like they're making progress.

I think it's a mistake to discount the psychology of learning. It's like saying that "calories-in, calories-out" is all there is to weight-loss. Strictly true, but not helpful for 90% of people.

Re:We Found an Neuron in GPT-2 by charcircuit on Thursday February 16, 2023 @11:46PM (#34830896)
Grammar lessons are a shortcut. So are books and other corpuses of knowledge. People have spent time documenting patterns that exist and its useful to learn from them instead having to brute force everything yourself.
Re:We Found an Neuron in GPT-2 by naasking on Friday February 17, 2023 @08:19AM (#34834299)
This is assuming that the brain regions that learn rules overlap with the brain regions that develops fluency in a language. I think Krashen's hypothesis is that this is largely not the case. You can "fake" some degree of competence by learning the rules and using that brain region, but you're slow and not fluent until you expose the other region to enough real-world data.