- In the paper you trained the property decoder separately but mention in passing that it could be a good idea to train the whole thing jointly. I haven't implemented the property decoder part yet and it seems like a pretty easy extension to just add that to the same loss function. Where is your thinking on independent vs joint training of the autoencoder/property prediction network now? Do you think non-obvious tricks will be required to get nice smoothness of the latent representation for things more complex than logP?
- Though learning directly to/from SMILES is weird, it's also kind of awesome. Do you think the future is an "inverse weave" module or something of that kind, or going further on learning more complex input representations?
- How do you feel about the synthetic accessibility score you used in the paper overall? Obviously it fell short on carbon ring sizes, but do you think we're close on this or we basically need to start over? What datasets would you think about using for "learning" synthetic accessibility as you imply in your post here?
Thanks! My vague intuition had been that this is a domain where I'd expect physics-based simulation should work better than machine learning but I have to say that the recent literature is changing my perspective.
- As you say, jointly training the autoencoder for reconstruction and prediction accuracy would be easy to code, but it might be tricky to get the tradeoff right since we usually have relatively little labeled data. A massively multi-task approach probably makes sense here. I'm not sure how hard it will be to get smoothness w.r.t. the latent representation, but the logP result was mildly encouraging.
- I agree that it's probably time to bring some of the recent augmented recurrent network architectures to bear on graph decoders. What makes is hard is that it requires differentiating through a cascading series of discrete choices, but people are making progress on this sort of problem.
- I do think we need to completely start over on the synthetic accessibility. There are just too many ways for molecules to be weird to try to write an explicit function for it. In fact I'd go so far as to say that this is the 'missing half' of this method.
- I view this method as a complement to physics-based simulation. A neural net is almost always going to be cheaper to evaluate than a physical simulation, so it's not a bad way to compile all the simulation results together.
Here is an example from our lab using virtual screening to develop PZM21 to treat pain [1]. where we screened 3M compounds. We would have liked to have screened 10^6 fold more compounds to cover easily synthesizable chemical space in this as well as other campaigns, but that is currently computationally infeasible. If molecular autoencoders could help us more efficiently screen this space, it would be huge.
I'm co-organizing a free, 1-day workshop for deep learning for chemoinformatics at Stanford Nov 11th. We've got ~75 mostly computational chemistry researchers coming. I would love to have more machine learning researchers come as well. The website is deepchemworkshop.docking.org, or PM if any of you think you may be interested.
[1] Manglik, et al. Structure-based discovery of opioid analgesics with reduced side effects (doi:10.1038/nature19112)
(Wallach, 2015, http://arxiv.org/pdf/1510.02855.pdf) AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery
(Duvenaud, 2015, http://papers.nips.cc/paper/5954-convolutional-networks-on-g...) Convolutional Networks on Graphs for Learning Molecular Fingerprints
(Kearnes, 2016, http://arxiv.org/abs/1606.08793v1) Modeling Industrial ADMET Data with Multitask Networks
(Gómez-Bombarelli, 2016, doi:10.1038/nmat4717) Design of efficient molecular organic light-emitting diodes by a high-throughput virtual screening and experimental approach
and of course
(Gómez-Bombarelli, 2016, https://arxiv.org/abs/1610.02415) Automatic chemical design using a data-driven continuous representation of molecules
Granularity might be some kind of "well we need some kind of phenol or phenol-derived ring here, why not just replace that with some sort of representation of 'phenol-like-ring-here'" or something to that effect.
Also, about PZM21 -- will it ever experience the same fate of U47700, or the likes of the orphaned opioids resurfacing from their watery graves?
For virtual screening it is possible to speed things up by say not taking into account receptor flexibility or ignoring explicit interactions with water.
As for lower resolution representations of small molecules, there is ROCS[1] and friends which represents small molecules with a set of gaussians.
One of challenges with low-resolution representations is that the aims of virtual screening is often to find novel backbones that may interact with the protein. So any low-resolution representation should mix different backbones into the same cluster, but finding such a representation is difficult, given the diversity of small molecules.
As for U47700, finding the mechanism of action for drugs that treat complex processes like pain is quite difficult. Also small molecules often interact with numerous targets so deconstructing how it works is non trivial. Part of the motivation for PZM21 is to try to separate out the downstream effects of hitting the mu-opioid receptor as a "biased" ligand. I think PZM21 with its new scaffold will help disentangle the effects of classical opioids.
http://deepchem.io/ is trying to set up standard data sets for chemoinformatics/machine learning.
ChEMBL and PubChem are the big public repositories though some care must be taken in curating data from these for machine learning.
You can contact me if you want a more formal version : )
Kind regards
From a machine learning point of view, we simply glued together two techniques: text autoencoders and Bayesian optimization. That is, we trained an autoencoder to transform a text representation of chemicals (SMILES) to and from continuous vectors. Then “chemical design” is just maximizing a function of a continuous variable, something that we already know a lot about.
We also showed off some of the nice things that one can do with continuous latent representations, such as interpolation. This had already been done for images by many people, and for text in https://arxiv.org/abs/1511.06349
Of course, our paper is just a proof of concept. For instance, instead of encoding to and from SMILES, it would be much better to encode to and from graphs directly. We know how to encode graphs into vectors, but I don’t know of a good way to decode a vector into a graph.
Another open problem is that it’s hard to know what to optimize. Our initial experiments optimizing for specific chemical properties produced suggested molecules with crazy structures, such as giant rings. Human chemists have a great intuition for what is easily synthesizable or stable, and it’s hard to articulate all the properties we want the molecule to have programmatically. Alternatively, we could enforce the optimizer to only look at molecules similar to ones we’ve already seen, but this is unsatisfying too - after all, the best result of exploration is when you find something unlike what you’ve seen before.