Re:Word2Vec Received 'Strong Reject' Four Times at ICLR2013
by johnfn on Monday December 18, 2023 @01:49PM
(#38686547)
Don't you think something is missing if we've defined "quality" as an independent and uncorrelated characteristic from importance or influentiality?
[ Parent
| on HN ]
Re:Word2Vec Received 'Strong Reject' Four Times at ICLR2013
by nicklecompte on Monday December 18, 2023 @02:02PM
(#38686711)
No, because "quality" means two different things here. I believe the main reason word2vec became important was purely on the software/engineering side, not because it was scientifically novel. Advancements in Python development, especially good higher-level constructs around numerical linear algebra, meant that the fairly shallow and simple tools of word2vec were available to almost every tech company. So I don't think word2vec was (or is) particularly good research, but it became good software for reasons beyond its own control
In 2016 or so it was proven that word2vec is equivalent to the pointwise mutual information between the words in its training set, after doing some preprocessing. This means that Claude Shannon had things mostly figured out in the 60s, and some reviewers were quite critical of the word2vec paper for not citing similar developments in the 70s.
Re:Word2Vec Received 'Strong Reject' Four Times at ICLR2013
by godelski on Monday December 18, 2023 @02:18PM
(#38686900)
Yes and no. I think the larger issue is about the ambiguity of what publications mean and should be. There's a lot of ways to optimize this, and none of those has optimal solutions. I don't think you should be down voted for your different framing because I think we just need to be more open about this chaos and consider other values than our own or the road we're on. I think it is very unclear what we are trying to optimize and it is quite apparent that you're going to have many opinions on this and your group of reviewers may all seek to optimize different things. The only solution I can see is to stop pretending as if we know what each other is trying to do and be a bit more explicit about it. Because if we argue based on different assumptions we'll talk past one another if we assume the other is working on the same set of assumptions.
Re:Word2Vec Received 'Strong Reject' Four Times at ICLR2013
by dr_kiszonka on Monday December 18, 2023 @02:46PM
(#38687232)
I believe many journals focusing on potentially influential papers is why we have a reproducibility crisis. Since it is very hard to publish null results, people often don't even bother trying. This leads to tons of wasted effort as multiple groups attempt the same thing not knowing that others before them have failed.
Also, predicting whether a paper will be influential is very hard and inherently subjective, unless you are reviewing something truly groundbreaking. Quality-based reviews are also subjective, but less so.
Re:Word2Vec Received 'Strong Reject' Four Times at ICLR2013
by nybsjytm on Monday December 18, 2023 @02:11PM
(#38686822)
Quality in the sense I meant it (cogency and intellectual depth/rigor) should certainly be correlated with importance and influence!
Re:Word2Vec Received 'Strong Reject' Four Times at ICLR2013
by SubiculumCode on Monday December 18, 2023 @02:00PM
(#38686684)
When an author refuses to address reasonable questions by the reviewers, what should you expect? There were legitimate questions and concerns about potential alternative explanations for the increase in accuracy raised by the reviewers, and the authors didn't play ball.