News for hackers. Stuff that matters.
Re:We Found an Neuron in GPT-2 by varispeed on Thursday February 16, 2023 @06:55PM (#34828541)
My friend who is not a native English speaker told me that one thing he struggled the most while learning English was the "a" and "an". He couldn't grasp the concept how it is possible that person knows which determiner to use before saying the word, until he learned it as just the part of the word and then he uses it depending on the context which one can "feel". So when he sees an apple, he says this is an apple.

Exactly what you mean by bigram.

Re:We Found an Neuron in GPT-2 by caf on Thursday February 16, 2023 @07:08PM (#34828696)
And of course it is not unheard of for a native English speaker to correct themselves if they change their mind about which noun to use. You can imagine being asked to very rapidly verbally classify fruits ("it's an apple, it's an apple, it's a pear, it's an apple,..") that you might well find yourself stumbling like "it's a.. an apple".
Re:We Found an Neuron in GPT-2 by PeterisP on Thursday February 16, 2023 @07:30PM (#34828909)
This is also why a bunch of words have changed - e.g. apron was originally called napron, but when people were saying "a napron" it got mangled to "an apron"..
Re:We Found an Neuron in GPT-2 by caf on Thursday February 16, 2023 @07:44PM (#34829070)
Maybe one day "knife" will become "ife".
Re:We Found an Neuron in GPT-2 by hgsgm on Thursday February 16, 2023 @07:50PM (#34829120)
And narrow from "an arrow"
Re:We Found an Neuron in GPT-2 by chimpansteve on Thursday February 16, 2023 @09:26PM (#34829894)
It's also not unheard of for native English speakers to use it differently (in speech, at least) as a consequence of regional accents. I would use "a history book", because my accent pronounces the h sound. My wife says "an istory book", because her accent drops the h.

But yeah, it should be easier when you're just dealing with text I guess

Re:We Found an Neuron in GPT-2 by hgsgm on Thursday February 16, 2023 @07:50PM (#34829126)
Does his language have grammatical gender?
Re:We Found an Neuron in GPT-2 by sunk1st on Thursday February 16, 2023 @08:00PM (#34829228)
The rule is the rule because that’s what make sense phonetically. It’s why you would say “el agua” and “el hacha” in Spanish even though those articles don’t match the gender of those words.
Re:We Found an Neuron in GPT-2 by alain94040 on Thursday February 16, 2023 @08:26PM (#34829445)
Another interesting case is possessive in French:

  une hache   - ma hache
  une horloge - mon horloge
Re:We Found an Neuron in GPT-2 by BaseballPhysics on Thursday February 16, 2023 @09:03PM (#34829743)
Now really bake their noodle and ask what they put before the word 'history' or the acronym 'LLM'? :)
Re:We Found an Neuron in GPT-2 by SamBam on Thursday February 16, 2023 @10:23PM (#34830333)
Are there languages that don't have similar patterns?

Obviously any romance language includes gender, so you need to use the correct gendered article before the noun.

Japanese has different counting words depending on what you're counting.

I don't know if Mandarin has anything similar.

Re:We Found an Neuron in GPT-2 by renonce on Friday February 17, 2023 @06:41AM (#34833489)
Mandarin speaker here. There are indeed different counting words for different nouns, so for a bird (鸟) you say 一只鸟 and for a chair (椅子) you say 一把椅子.