Agreed. Just because you don't compare your model to another model doesn't mean you're hiding something and it's irresponsible to pretend that's the case. Perhaps the author should have run char-cnn against data instead of making baseless accusations.

On the whole, a lot of the arguments are realistic of ML in general. There is far too much focus on individual problems and the accuracy measure without enough focus on the interpretation of the accuracy, how well it will really generalize, etc.

In general, you are right. But in this particular case, char-CNN is the standard used by many people, Stanford included. It is not sophisticated at all.

Look at the big list of exotic models they tried: http://moleculenet.ai/models

For me, this omission is a negligence. It is selective laziness.

The author (me) does not have 20 PhD students, postdocs and startuppers under his hand, to do the job for Stanford. With my limited resources, it is more cost-effective to do my job against Stanford ;)

It sounds like you have axe to grind against well-funded professors, not that you actually have a valid argument.

This professor gave an argument in his paper. I gave a refutation. If you agree with him that char-CNN was a sophisticated model in early 2017, then you are not well-informed about the situation in deep learning.

The MoleculeNet co-author gave another argument in the comments. My refutation is that you can't claim to lack time after 8*20 man-months have passed.

Author here

1. I agree that graph convolutions make a lot of sense. However, it is not shown yet that they are better than SMILES, although it might just be a matter of time. A lot of graph variants are possible, or maybe we should look at molecules not as graphs, but as quantum objects (See the other Stanford paper on atom convolutions).

2. Yes, Deepchem is built on top of Tensorflow. This additional layer looks useful, but is it really the case? Using standard NLP models is simpler than using deepchem models, and NLP models might still be the state-of-the-art for chemistry tasks, via SMILES (until we get a reasonably comprehensive benchmark, which MoleculeNet is not).

I am not buying into a chemistry-specific library, until it is shown to be really necessary.

Well my group and others are toying with applying RNNs to permuted SMILES strings; if it ends up being better, we will find out some time this year.