pnnl/chemreasoner
ChemReasoner - Catalyst Discovery via Large Language Model-driven Reasoning
Choudhury, Sutanay↗
Engineering topics
Agarwal, Khushbu
Publications and source records attributed to Agarwal, Khushbu.
High dimensional predictions of suicide risk in 4.2 million US Veterans using ensemble transfer learning
We present an ensemble transfer learning method to predict suicide from Veterans Affairs (VA) electronic medical records (EMR). A diverse set of base models was trained to predict a binary outcome constructed from reported suicide, suicide attempt, and overdose diagnoses with varying choices of study design and prediction methodology. Each model used twenty cross-sectional and 190 longitudinal variables observed in eight time intervals covering 7.5 years prior to the time of prediction. Ensembles of seven base models were created and fine-tuned with ten variables expected to change with study design and outcome definition in order to predict suicide and combined outcome in a prospective cohort. The ensemble models achieved c-statistics of 0.73 on 2-year suicide risk and 0.83 on the combined outcome when predicting on a prospective cohort of ~4.2 M veterans. The ensembles rely on nonlinear base models trained using a matched retrospective nested case-control (Rcc) study cohort and show good calibration across a diversity of subgroups, including risk strata, age, sex, race, and level of healthcare utilization. In addition, a linear Rcc base model provided a rich set of biological predictors, including indicators of suicide, substance use disorder, mental health diagnoses and treatments, hypoxia and vascular damage, and demographics. Similar content being viewed by others
60 APPLIED LIFE SCIENCES↗
A Unification Framework for Euclidean and Hyperbolic Graph Neural Networks
Hyperbolic neural networks have recently gained significant attention due to their promising results on several graph problems including node classification and link prediction. The primary reason for this success is the effectiveness of hyperbolic space in capturing the inherent hierarchy of graph datasets. However, they are limited in terms of generalization, scalability, and have inferior performance when applied to non-hierarchical datasets. In this paper, we take a completely different perspective for modeling hyperbolic networks and answer the following question: is an Euclidean model able to approximate a function or behavior in the hyperbolic space? Extending the universal approximation theory developed for Euclidean models, We draw an analogy from the hyperbolic components to the Euclidean counterparts and conclude that, in order to capture hierarchical features, it is possible to generalize hyperbolic models to be a special case of Euclidean models with the proposed Pseudo-Poincaré technique. We applied our non-linear hyperbolic normalization to the current state-of-the-art homogeneous and multi-relational graph networks and demonstrate significant improvements in performance compared to both Euclidean and hyperbolic counterparts. The primary impact of this work lies in its ability to capture hierarchical features in the Euclidean space, and thus, can replace hyperbolic networks without any loss in performance metrics while simultaneously leveraging the power of Euclidean networks such as interpretability and efficient execution of various model components.
Khatir, Mehrdad↗
Learning Distributed Geometric Koopman Operator for Sparse Networked Dynamical Systems
Koopman operator theory provides an alternative to study nonlinear networked dynamical systems by mapping the state space to an abstract higher dimensional space where the system evolution is linear. Recent works show the application of graph neural networks (GNNs) to learn state to object-centric embeddings and achieve centralized block-wise computation of Koopman operator (KO) under additional assumptions on the underlying agents properties and constraints on the KO structure. However, the computational complexity of learning the Koopman increases exponentially for networked systems where the number of possible system states grows in a combinatorial fashion with the number of nodes. The learning challenge is further amplified for sparse networks by two factors: 1) sample sparsity for learning the Koopman operator in the non-linear space, and 2) the divergence in the dynamics of individual nodes or from one subgraph to another. Our work aims to address these challenge by formulating the representation learning of networked dynamical systems into a multi-agent paradigm and learning the Koopman operator in a distributive manner. The computational as well as performance advantages of distributed Koopman is predominant for sparse networks whereas for fully connected networks, it is shown to coincide with the centralized one. The empirical study on rope system, network of oscillators and a synthetic power system show comparable and superior performance along with computational benefits with the state-of-the-art methods.
Mukherjee, Sayak↗
Identification of Novel, Replicable Genetic Risk Loci for Suicidal Thoughts and Behaviors Among US Military Veterans
Importance: Suicide is a leading cause of death; however, the molecular genetic basis of suicidal thoughts and behaviors (SITB) remains unknown. Objective: To identify novel, replicable genomic risk loci for SITB. Design, Setting, and Participants: This genome-wide association study included 633 778 US military veterans with and without SITB, as identified through electronic health records. GWAS was performed separately by ancestry, controlling for sex, age, and genetic substructure. Cross-ancestry risk loci were identified through meta-analysis. Study enrollment began in 2011 and is ongoing. Data were analyzed from November 2021 to August 2022. Main Outcome and Measures: SITB. Results: A total of 633 778 US military veterans were included in the analysis (57 152 [9%] female; 121 118 [19.1%] African ancestry, 8285 [1.3%] Asian ancestry, 452 767 [71.4%] European ancestry, and 51 608 [8.1%] Hispanic ancestry), including 121 211 individuals with SITB (19.1%). Meta-analysis identified more than 200 GWS (P < 5 × 10 -8 ) cross-ancestry risk single-nucleotide variants for SITB concentrated in 7 regions on chromosomes 2, 6, 9, 11, 14, 16, and 18. Top single-nucleotide variants were largely intronic in nature; 5 were independently replicated in ISGC, including rs6557168 in ESR1, rs12808482 in DRD2, rs77641763 in EXD3, rs10671545 in DCC, and rs36006172 in TRAF3. Associations for FBXL19 and AC018880.2 were not replicated. Gene-based analyses implicated 24 additional GWS cross-ancestry risk genes, including FURIN, TSNARE1, and the NCAM1-TTC12-ANKK1-DRD2 gene cluster. Cross-ancestry enrichment analyses revealed significant enrichment for expression in brain and pituitary tissue, synapse and ubiquitination processes, amphetamine addiction, parathyroid hormone synthesis, axon guidance, and dopaminergic pathways. Seven other unique European ancestry–specific GWS loci were identified, 2 of which (POM121L2 and METTL15/LINC02758) were replicated. Two additional GWS ancestry-specific loci were identified within the African ancestry (PET112/GATB) and Hispanic ancestry (intergenic locus on chromosome 4) subsets, both of which were replicated. Further, no GWS loci were identified within the Asian ancestry subset; however, significant enrichment was observed for axon guidance, cyclic adenosine monophosphate signaling, focal adhesion, glutamatergic synapse, and oxytocin signaling pathways across all ancestries. Within the European ancestry subset, genetic correlations (r > 0.75) were observed between the SITB phenotype and a suicide attempt-only phenotype, depression, and posttraumatic stress disorder. Additionally, polygenic risk score analyses revealed that the Million Veteran Program polygenic risk score had nominally significant main effects in 2 independent samples of veterans of European and African ancestry.
60 APPLIED LIFE SCIENCES↗
pnnl/TransMED
Transfer Learning From Existing Diseases Via Hierarchical Multi-Modal BERT Models
Ham, Colby↗
Attention-based Aspect Reasoning for Knowledge Base Question Answering on Clinical Notes
Question Answering (QA) in clinical notes has gained a lot of attention in the past few years. Existing machine reading comprehension approaches in clinical domain can only handle questions about a single block of clinical texts and fail to retrieve information about different patients and clinical notes. To handle more complex questions, we aim at creating knowledge base from clinical notes to link different patients and clinical notes, and performing knowledge base question answering (KBQA). Based on the expert annotations in n2c2, we first created the ClinicalKBQA dataset that includes 8,952 QA pairs and covers questions about seven medical topics through 322 question templates. Then, we proposed an attention-based aspect reasoning (AAR) method for KBQA and investigated the impact of different aspects of answers (e.g., entity, type, path, and context) for prediction. The AAR method achieves better performance due to the well-designed encoder and attention mechanism. In the experiments, we find that both aspects, type and path, enable the model to identify answers satisfying the general conditions and produce lower precision and higher recall. On the other hand, the aspects, entity and context, limit the answers by node-specific information and lead to higher precision and lower recall.
Wang, Ping↗
Preparing for the next pandemic via transfer learning from existing diseases with hierarchical multi-modal BERT: a study on COVID-19 outcome prediction
Abstract Developing prediction models for emerging infectious diseases from relatively small numbers of cases is a critical need for improving pandemic preparedness. Using COVID-19 as an exemplar, we propose a transfer learning methodology for developing predictive models from multi-modal electronic healthcare records by leveraging information from more prevalent diseases with shared clinical characteristics. Our novel hierarchical, multi-modal model ( $${\textsc {TransMED}}$$ T R A N S MED ) integrates baseline risk factors from the natural language processing of clinical notes at admission, time-series measurements of biomarkers obtained from laboratory tests, and discrete diagnostic, procedure and drug codes. We demonstrate the alignment of $${\textsc {TransMED}}$$ T R A N S MED ’s predictions with well-established clinical knowledge about COVID-19 through univariate and multivariate risk factor driven sub-cohort analysis. $${\textsc {TransMED}}$$ T R A N S MED ’s superior performance over state-of-the-art methods shows that leveraging patient data across modalities and transferring prior knowledge from similar disorders is critical for accurate prediction of patient outcomes, and this approach may serve as an important tool in the early response to future pandemics.
59 BASIC BIOLOGICAL SCIENCES↗
DeepCare: Improving Patient Care using Deep Learning on Electronic Health Records
Coordinating patient care using electronic health records (EHR) data presents an exciting but formidable opportunity in data extraction, analysis and modeling. Traditional methods use a manual feature driven approach to model patients with age, family history and symptoms to predict disease outcomes. We propose a novel approach to model patients based on their streaming electronic health records data combined with information from medical knowledge bases, which has been gained over years of medical research. Using a combination of representation learning and long short term memory (LSTM) networks we plan to model patient evolution over time, leading to more accurate and individualized predictive models for patient’s diseases. Our approach will be transformative in providing critical decision support for patient care, enabling accurate understanding and evolution of diseases in patients.
60 APPLIED LIFE SCIENCES↗
Graph Neural Network and Koopman Models for Learning Networked Dynamics: A Comparative Study on Power Grid Transients Prediction
Not Available
24 POWER TRANSMISSION AND DISTRIBUTION↗
Data-driven Distributed Learning of Multi-agent Systems: A Koopman Operator Approach
Koopman operator theory provides a model-free technique for studying nonlinear dynamical systems purely from data. Since the Koopman operator is infinite-dimensional, researchers have developed several methods that provide a finite-dimensional approximation of the Koopman operator so that it can be applied for practical use cases. One common thing with most of the methods is that their solutions are obtained by solving a centralized minimization problem. In this work, we treat the dynamical system to be a multi-agent system and propose an algorithm to compute the finite-dimensional approximation of the Koopman operator in a distributed manner using the knowledge of the topology of the underlying multi-agent system. The proposed distributed approach is shown to be equivalent to the centralized learning problem and results in a sparse Koopman whose block structure mimics the Laplacian of the multi-agent system. Extensive simulation studies illustrate the proposed framework on the network of oscillators and the IEEE 68 bus system.
Nandanoori, Sai Pushpak↗
pnnl/SLiCE
Subgraph Based Learning of Contextual Embedding a framework bridging static representation learning methods using global information from the entire graph with localized attention driven mechanisms to learn contextual node representations.
Ham, Colby↗
pnnl/grid_prediction
Two datadriven predictive approaches, namely, {\em Koopman Operator Theoretic (KOT)-based} model and {\em Graph Neural Network (GNN)-based model}, to enable effective power system state predictions. The KOT-based approaches (Robust DMD, deepDMD) capture the power system evolution as a linear dynamical system on an abstract space. The GNNs model the spatio-temporal correlations using graph convolutional network and are called Spatio-Temporal Graph Convolutional Network (STGCN). These predictive models are trained, tested and compared rigorously based on their predictions of frequencies in the IEEE 68 bus system when subjected to a disturbance. GridSTAGE framework developed at Pacific Northwest National Laboratory is leveraged to generate multiple datasets (in the form of PMU measurements) for training and testing by strategically creating load changes across the spatial locations of the network
Nandanoor, Sai Pushpak↗
Model-Agnostic Algorithm for Real-Time Attack Identification in Power Grid using Koopman Modes
Malicious activities on measurements from sensors like Phasor Measurement Units (PMUs) can mislead the control center operator into taking wrong control actions resulting in disruption of operation, financial losses, and equipment damage. In particular, false data attacks initiated during power systems transients caused due to abrupt changes in load and generation can fool the conventional model-based detection methods relying on thresholds comparison to trigger an anomaly. In this paper, we propose a Koopman mode decomposition (KMD) based algorithm to detect and identify false data attacks in real-time. The Koopman modes (KMs) are capable of capturing the nonlinear modes of oscillation in the transient dynamics of the power networks and reveal the spatial embedding of both natural and anomalous modes of oscillations in the sensor measurements. The Koopman-based spatio-temporal nonlinear modal analysis is used to filter out the false data injected by an attacker. The performance of the algorithm is illustrated on the IEEE 68-bus test system using synthetic attack scenarios generated on GridSTAGE, a recently developed multivariate spatio-temporal data generation framework for simulation of adversarial scenarios in cyber-physical power systems.
Nandanoori, Sai Pushpak↗
System and methods for automated detection, reasoning and recommendations for resilient cyber systems
A method for securing an IT (information technology) system using a set of methods for knowledge extraction, event detection, risk estimation and explanation for ranking cyber-alerts which includes a method to explain the relationship (or an attack pathway) from an entity (user or host) and an event context to another entity (a high-value resource) and an event context (attack or service failure).
Choudhury, Sutanay↗
Performance and usability enhancements for continuous subgraph matching queries on graph-structured data
A query graph, which includes vertices and edges, represents a query on graph-structured data. The query graph is decomposed into query subgraphs. A network analysis tool performs continuous subgraph matching queries to facilitate analysis of computer network traffic, social media events, or other streams of data represented as a dynamic data graph (graph-structured data). This can help identify emerging trends in the data. Some features of the network analysis tool enhance performance by effectively utilizing distributed computing resources (including processing cores and memory at different nodes of a cluster) to speed up the process of updating the dynamic data graph and detecting matches of query subgraphs. Features of a query graph building tool enhance usability by providing intuitive ways to specify query graphs and their subgraphs. Features of a results visualization tool enhance usability by providing an intuitive way to present the results of continuous subgraph matching queries.
Choudhury, Sutanay↗
GridSTAGE (aka PowerDrone)
Simulation tool for electrical grid security. It is possible to select from pre-loaded power system topologies, enable different grid controls (like AGC, power system stabilizer), add faults, tweak different parameters, add pre-modeled attack scenarios, and run batch simulations to generate data needed for research.
Nandanoori, Sai Pushpak↗
Deep Learning for Scientific Discovery
Deep learning has positively impacted fields were perceptive tasks are paramount, such as computer vision and natural language processing. Tremendous strides have also been made in deep reinforcement learning towards robotics and gaming. Given the success in these fields, it is apparent that deep learning may also accelerate scientific discovery. Herein, we discuss several topic areas where modern representation learning is driving innovation. This article presents a) learning non-linear models from data, b) learning scientific laws from models, c) deep learning on graphs, and d) approaches to scalable deep learning.
deep learning, scientific computing, koopman, mate↗