Re:A cartel of influential datasets are dominating machine l...
by domenicrosati on Monday December 06, 2021 @07:24AM
(#29458858)
For brand name journals and conferences, yes they do. My experience with those is a very high rejection rate.
There is a good reason to reject if you are not using a standard dataset. How can you compare the results of two approaches, to say natural language inference of whether one sentence entails another, without results being tested on the same dataset?
I think one thing overlooked in the conversation is that many papers start with a standard baseline and then use another dataset to establish additional results.
In my experience in nlp also, journals and conferences tend to establish datasets of their own when they make a call for submission. Often these are called the "shared task" track. ACL has operated this way for decades.
There is a good reason to reject if you are not using a standard dataset. How can you compare the results of two approaches, to say natural language inference of whether one sentence entails another, without results being tested on the same dataset?
I think one thing overlooked in the conversation is that many papers start with a standard baseline and then use another dataset to establish additional results.
In my experience in nlp also, journals and conferences tend to establish datasets of their own when they make a call for submission. Often these are called the "shared task" track. ACL has operated this way for decades.