>> That's because they are literally different model architectures. The 3-gram model is the same everywhere, it doesn't depend on dataset. A 3-gram "trained" on Russian would still be a 3-gram. Similarly, BERT trained on a German corpus would still be BERT, it wouldn't need a "new catchy name." That's my point.
N-gram models are not "the same everywhere". An n-gram model is a set of probabilities assigned to, well, n-grams. Different sets of n-grams, or probabilities trained from different datasets would be- different. The n-grams would be different and the probabilities would be different and the structure of the model would be different. A Russian n-gram model would represent the probabilities of Russian n-grams, an English one would reprsent the probabilities of English n-grams and so on. The type of model would be the same, an n-gram model; but the model itself would be different. Otherwise, why bother training more models?
Same goes for a model trained with a transformer architecture. BERT is an English language model trained with a transformer architecture and representing the probabilities of tokens in an English corpus. A model trained with a transformer architecture and representing the probabilities of tokens in a German corpus would be, again, a different model. You could also call it "BERT" but that would eventually become very confusing.
>> Honestly, quite confused this is the hill you want to die on.
I'm not dying on any hill. Could you please not do that?
I don't really think it's super worth quibbling over, since I think we're generally on the same page. It's worth noting that BERT trained on different corpora is generally still called BERT (you can look at the huggingface/transformers library, for instance).
I'm sorry that my message came across as more adversarial than intended.
Thanks. I enjoyed our conversation.
Actual NLP practitioners are not having a hype-fest every time someone spends a shitton to train a transformer LM on a new dataset. It's definitely cool, but I wouldn't cal it remarkable. What was cool about GPT-3 was the demonstration of the effectiveness of the "few-shot"/"conditioning" approach. I didn't find it surprising, but it was cool to push it to the next level.
> In any case, I don't remember anyone ever naming their n-gram models or HMMs etc
That's because they are literally different model architectures. The 3-gram model is the same everywhere, it doesn't depend on dataset. A 3-gram "trained" on Russian would still be a 3-gram. Similarly, BERT trained on a German corpus would still be BERT, it wouldn't need a "new catchy name." That's my point.
As a side remark, you are vastly overestimating the amount of dataset specific tuning that you need - you can often just use the exact same hyperparams and run it on a new dataset - there's been some recent literature showing that a lot of these architectures are actually quite resilient to different datasets and hyperparameter choices and arrive at similar local minima. Sure, you might be off by 1-2% from SOTA accuracy (let's say on a classification task) but in production systems you're generally not shooting for the SOTA that was released in the last 2 months but rather something close-ish to those results.
Honestly, quite confused this is the hill you want to die on.