Engineering topics

Goh, Garrett B.

Publications and source records attributed to Goh, Garrett B..

6 records

DOE OSTI2018

Multiple-objective Reinforcement Learning for Inverse Design and Identification

Inverse chemical design using generative deep learning (DL) models is an emerging tool, which has early success in generating molecular structures with desired properties. We enhance this tool by developing heuristics for curriculum-learning based multiple-objective (20+ objectives) reinforcement learning, and apply it to the context of chemical identi?cation. We develop a generative DL framework that utilizes constraints pertaining to the unknown molecule’s mass and functional group composition, and show that multiple-objective RL-based generative DL models can correctly identify unknown molecules with a 80% success rate, compared to the baseline approach of 0%. Lastly, the heuristics developed are not limited to just chemistry research challenges; we anticipate any problem that utilizes reinforcement learning with multiple-objectives will bene?t from it.

Artificial Intelligence, Data Science, deep learni↗

DOE OSTI2018

IL-Net: Using Expert Knowledge to Guide the Design of Furcated Neural Networks

Deep neural networks (DNN) excel at extracting patterns. Through representation learning and automated feature engineering on large datasets, such models have been highly successful in computer vision and natural language applications. Designing optimal network architectures from a principled or rational approach however has been less than successful, with the best successful approaches utilizing an additional machine learning algorithm to tune the network hyperparameters. However, in many technical fields, there exist established domain knowledge and understanding about the subject matter. In this work, we develop a novel furcated neural network architecture that utilizes domain knowledge as high-level design principles of the network. We demonstrate proof-of-concept by developing IL-Net, a furcated network for predicting the properties of ionic liquids, which is a class of complex multi-chemicals entities. Compared to existing state-of-the-art approaches, we show that furcated networks can improve model accuracy by approximately 20-35%, without using additional labeled data. Lastly, we distill two key design principles for furcated networks that can be adapted to other domains.

Sakloth, Khushmeen↗

DOE OSTI2018

Using Rule-Based Models for Weak Supervised Learning: A ChemNet for Transferable Chemical Property Prediction

With access to large datasets, deep neural networks (DNN) have achieved human-level accuracy in image and speech recognition tasks. However, in chemistry, data is inherently small and fragmented. In this work, we develop an approach of integrating rule-based knowledge into the Chemception CNN model via transfer learning techniques. The resulting model, ChemNet, is a transferable and generalizable deep neural network for chemical property prediction that learns in a semi-supervised manner from large unlabeled chemical databases. When ChemNet is further fine-tuned on 3 smaller datasets to predict chemical properties that it was not originally trained on, we show that ChemNet exceeds the accuracy of existing Chemception CNN models and other contemporary DNN models that were trained using conventional supervised learning approaches. These results indicate that pre-training ChemNet on a large diverse chemical database while incorporating chemistry domain knowledge, enables the development of more generalizable deep neural networks for the prediction of novel chemical properties.

Artificial Intelligence, Cheminformatics, Computat↗

DOE OSTI2018

How Much Chemistry Does a Deep Neural Network Need to Know to Make Accurate Predictions?

In the last few years, we have seen the rise of deep learning applications in a broad range of computational chemistry research problems. Using human-engineered chemical features, such as molecular descriptors and fingerprints, deep learning models have shown similar, if not better performance that most traditional machine learning algorithms. Recently, we reported on the development of Chemception, a deep convolutional neural network (CNN) architecture for general-purpose small molecule property prediction. On average, Chemception matched the performance of expert-developed QSAR/QSPR models trained on chemical features (molecular fingerprints), despite that it was trained on just 2D images of molecular drawings with minimal chemical information. Here, we investigate the effects of systematically removing and adding basic chemical information to the image channels of the 2D images used to train Chemception. By augmenting our images with only 3 additional basic chemical information, we demonstrate the improvement of Chemception performance – that it is now more accurate than contemporary deep learning models trained on ECFP fingerprints for the prediction of toxicity, activity and solvation free energy, as well as physics-based free energy simulation methods for computing solvation properties. By altering the chemical information content in the image channels, and examining the resulting performance of Chemception, we also identify to two different “learning patterns” in toxicity/activity as compared to solvation free energy, and it parallels the fundamental differences in contemporary chemistry research for predicting toxicity/activity and solvation free energy.

Artificial Intelligence, Cheminformatics, Computat↗

DOE OSTI2017

ChemNet: A Transferable and Generalizable Deep Neural Network for Small-Molecule Property Prediction

With access to large datasets, deep neural networks through representation learning have been able to identify patterns from raw data, achieving human-level accuracy in image and speech recognition tasks. However, in chemistry, availability of large standardized and labelled datasets is scarce, and with a multitude of chemical properties of interest, chemical data is inherently small and fragmented. In this work, we explore transfer learning techniques in conjunction with the existing Chemception CNN model, to create a transferable and generalizable deep neural network for small-molecule property prediction. Our latest model, ChemNet learns in a semi-supervised manner from inexpensive labels computed from the ChEMBL database. When fine-tuned to the Tox21, HIV and FreeSolv dataset, which are 3 separate chemical tasks that ChemNet was not originally trained on, we demonstrate that ChemNet exceeds the performance of existing Chemception models, contemporary MLP models that trains on molecular fingerprints, and it matches the performance of the ConvGraph algorithm, the current state-of-the-art. Furthermore, as ChemNet has been pre-trained on a large diverse chemical database, it can be used as a universal “plug-and-play” deep neural network, which accelerates the deployment of deep neural networks for the prediction of novel small-molecule chemical properties.

Artificial Intelligence↗

DOE OSTI2017

Deep learning for computational chemistry

The rise and fall of artificial neural networks is well documented in the scientific literature of both the fields of computer science and computational chemistry. Yet almost two decades later, we are now seeing a resurgence of interest in deep learning, a machine learning algorithm based on “deep” neural networks. Within the last few years, we have seen the transformative impact of deep learning the computer science domain, notably in speech recognition and computer vision, to the extent that the majority of practitioners in those field are now regularly eschewing prior established models in favor of deep learning models. In this review, we provide an introductory overview into the theory of deep neural networks and their unique properties as compared to traditional machine learning algorithms used in cheminformatics. By providing an overview of the variety of emerging applications of deep neural networks, we highlight its ubiquity and broad applicability to a wide range of challenges in the field, including QSAR, virtual screening, protein structure modeling, QM calculations, materials synthesis and property prediction. In reviewing the performance of deep neural networks, we observed a consistent outperformance against non neural networks state-of-the-art models across disparate research topics, and deep neural network based models often exceeded the “glass ceiling” expectations of their respective tasks. Coupled with the maturity of GPU-accelerated computing for training deep neural networks and the exponential growth of chemical data on which to train these networks on, we anticipate that deep learning algorithms will be a useful tool and may grow into a pivotal role for various challenges in the computational chemistry field.

Computational Chemistry↗