Search NASA⌕ Search

SEARCH · Search NASA

Results for “Machine Learning Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55

Machine Learning Based Cavity Fault Classification and Prediction

We report on the development of machine learning models for classifying C100 superconducting radio frequency (SRF) cavity faults in the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab. CEBAF is a continuous-wave recirculating linac utilizing 418 SRF cavities to accelerate electrons up to 12 GeV through 5-passes. Of these, 96 cavities (12 cryomodules) are designed with a digital low-level RF system configured such that a cavity fault triggers waveform recordings of 17 RF signals for each of the eight cavities in the cryomodule. Subject matter experts (SME) can analyze the collected time-series data, identify which of the eight cavities faulted first, and classify the type of fault. This information is used to find trends and strategically deploy mitigations to problematic cryomodules. However, manually labeling the data is laborious and time-consuming. By leveraging machine learning, near real-time – rather than post-mortem – identification of the offending cavity and classification of the fault type has been implemented. We discuss the performance of the ML models during a recent physics run.

43 PARTICLE ACCELERATORS↗

Three dimensional cluster analysis for atom probe tomography using Ripley’s K-function and machine learning

The size and structure of spatial molecular and atomic clustering can significantly impact material properties and is therefore important to accurately quantify. Ripley’s K-function (K(r)), a measure of spatial correlation, can be used to perform such quantification when the material system of interest can be represented as a marked point pattern. This work demonstrates how machine learning models based on K (r)-derived metrics can accurately estimate cluster size and intra-cluster density in simulated three dimensional (3D) point patterns containing spherical clusters of varying size; over 90% of model estimates for cluster size and intra-cluster density fall within 11% and 18% error of the true values, respectively. These K (r)-based size and density estimates are then applied to an experimental APT reconstruction to characterize MgZn clusters in a 7000 series aluminum alloy. Here we find that the estimates are more accurate, consistent, and robust to user interaction than estimates from the popular maximum separation algorithm. Using K (r) and machine learning to measure clustering is an accurate and repeatable way to quantify this important material attribute.

36 MATERIALS SCIENCE↗

Detecting control system misbehavior by fingerprinting programmable logic controller functionality

In recent years, attacks such as the Stuxnet malware have demonstrated that cyberattacks against control systems cause extensive damage. These attacks can result in physical damage to the networked systems under their control. In this paper, we discuss our approach for detecting such attacks by distinguishing between programs running on a programmable logic controller (PLC) without having to monitor communications. Using power signatures generated by an attached, high-frequency power measurement device, we can identify what a PLC is doing and when an attack may have altered what the PLC should be doing. To accomplish this, we generated labeled data for testing our methods and applied feature engineering techniques and machine learning models. The results of this research demonstrate that Random Forests and Convolutional Neural Networks classify programs with up to 98% accuracy for major program differences and 84% accuracy for minor differences. Our results can be used for both online and offline applications.

97 MATHEMATICS AND COMPUTING↗

Optimization of simulated high-field side lower hybrid current drive coupling using machine learning predictions of scrape-off layer density

Lower hybrid current drive (LHCD) is a potential source of non-inductive off-axis current drive (CD) for tokamaks. Although LHCD has been successfully deployed on a number of tokamaks, it is highly sensitive to the scrape-off layer (SOL) conditions local to the LHCD launcher. Large gaps between the launcher and plasma core, SOL turbulence, or edge density perturbations due to edge-localized modes can hamper CD or cause large reflected power. These coupling issues in part motivated the installation of an LHCD launcher on the high-field side (HFS) of DIII-D. On the HFS, the SOL is less turbulent and more controllable compared to the low-field side. This quiescence may result in more predictable edge conditions and thus a more predictable CD. Here, in this work, HFS SOL reflectometry measurements are predicted from global plasma parameters using machine learning models. The SOL predictions coupled with the full-wave simulation of the LHCD launcher allow for the prediction of reflected power, directivity, and arcing risk before the discharge. Launcher performance is then optimized using multi-objective Bayesian optimization, finding the shot parameters that result in an optimal SOL density that maximizes CD while minimizing the risk of arcing. The predictions and optimizations of LHCD performance are then accelerated using a surrogate model of the full-wave LHCD simulation.

Bayesian optimization↗

Asynchronous Reciprocal Coupling of Martini 2.2 Coarse-Grained and CHARMM36 All-Atom Simulations in an Automated Multiscale Framework

The appeal of multiscale modeling approaches is predicated on the promise of combinatorial synergy. However, this promise can only be realized when distinct scales are combined with reciprocal consistency. Here, we consider multiscale molecular dynamics (MD) simulations that combine the accuracy and macromolecular flexibility accessible to fixed-charge all-atom (AA) representations with the sampling speed accessible to reductive, coarse-grained (CG) representations. AA-to-CG conversions are relatively straightforward because deterministic routines with unique outcomes are achievable. Conversely, CG-to-AA conversions have many solutions due to a surge in the number of degrees of freedom. While automated tools for biomolecular CG-to-AA transformation exist, we find that one popular option, called Backward, is prone to stochastic failure and the AA models that it does generate frequently have compromised protein structure and incorrect stereochemistry. Although these shortcomings can likely be circumvented by human intervention in isolated instances, automated multiscale coupling requires reliable and robust scale conversion. In this work, we detail an extension to Multiscale Machine-learned Modeling Infrastructure (MuMMI), including an improved CG-to-AA conversion tool called sinceCG. This tool is reliable (~98% weakly correlated repeat success rate), automatable (no unrecoverable hangs), and yields AA models that generally preserve protein secondary structure and maintain correct stereochemistry. We describe how the MuMMI framework identifies CG system configurations of interest, converts them to AA representations, and simulates them at the AA scale while on-the-fly analyses provide feedback to update CG parameters. Application to systems containing the peripheral membrane protein RAS and proximal components of RAF kinase on complex eight-component lipid bilayers with ~1.5 million atoms is discussed in the context of MuMMI.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Large-Scale Visualization of 3D Unstructured Groundwater Model Using Cave Automated Virtual Environment

The immersive three-dimensional (3D) virtual reality (VR) visualization of groundwater models allows us to deepen our understanding of aquifer systems and provide better solutions to present groundwater-related problems, such as groundwater recharge, water quality, and sustainability. Visualization assists in accurately developing groundwater models and revealing important subsurface features, including faulting, folding, and unconformity. However, assessing model accuracy poses challenges due to the complexity of geology and groundwater systems. This research demonstrates a workflow to visualize and analyze raw 3D unstructured groundwater model data using an immersive Cave Automated Virtual Environment (CAVE). To visualize the unstructured groundwater model data, the raw dataset is converted into interactive CAVE-compatible formats utilizing a set of tools: ParaView, Blender, and Unity. This enables researchers to immerse themselves in the data, identifying influential patterns and relationships. e resulting insights can inform the development of sophisticated machine-learning models for groundwater level prediction. The CAVE’s immersive capabilities allow intuitive exploration from various perspectives, providing a more holistic understanding of the factors affecting groundwater levels. These insights are crucial to improve predictive models. The CAVE results also facilitate collaborative analysis and have potential applications in training and education. is research demonstrates the value of immersive VR tools such as the CAVE for unraveling intricacies within high-dimensional scientific data to drive real-world forecasting and modeling applications.

54 ENVIRONMENTAL SCIENCES↗

CovTransformer: A transformer model for SARS-CoV-2 lineage frequency forecasting

With hundreds of SARS-CoV-2 lineages circulating in the global population, there is an ongoing need for predicting and forecasting lineage frequencies and thus identifying rapidly expanding lineages. Accurate prediction would allow for more focused experimental efforts to understand pathogenicity of future dominating lineages and characterize the extent of their immune escape. Here, we first show that the inherent noise and biases in lineage frequency data make a commonly-used regression-based approach unreliable. To address this weakness, we constructed a machine learning model for SARS-CoV-2 lineage frequency forecasting, called CovTransformer, based on the transformer architecture. We designed our model to navigate challenges such as a limited amount of data with high levels of noise and bias. We first trained and tested the model using data from the UK and the USA, and then tested the generalization ability of the model to many other countries and US states. Remarkably, the trained model makes accurate predictions two months into the future with high levels of accuracy both globally (in 31 countries with high levels of sequencing effort) and at the US-state level. Our model performed substantially better than a widely used forecasting tool, the multinomial regression model implemented in Nextstrain, demonstrating its utility in SARS-CoV-2 monitoring. Assuming a newly emerged lineage is identified and assigned, our test using retrospective data shows that our model is able to identify the dominating lineages 7 weeks in advance on average before they became dominant. Overall, our work demonstrates that transformer models represent a promising approach for SARS-CoV-2 forecasting and pandemic monitoring.

60 APPLIED LIFE SCIENCES↗

Improved accuracy and transferability of molecular-orbital-based machine learning: Organics, transition-metal complexes, non-covalent interactions, and transition states

We report molecular-orbital-based machine learning (MOB-ML) provides a general framework for the prediction of accurate correlation energies at the cost of obtaining molecular orbitals. The application of Nesbet’s theorem makes it possible to recast a typical extrapolation task, training on correlation energies for small molecules and predicting correlation energies for large molecules, into an interpolation task based on the properties of orbital pairs. We demonstrate the importance of preserving physical constraints, including invariance conditions and size consistency, when generating the input for the machine learning model. Numerical improvements are demonstrated for different datasets covering total and relative energies for thermally accessible organic and transition-metal containing molecules, non-covalent interactions, and transition-state energies. MOB-ML requires training data from only 1% of the QM7b-T dataset (i.e., only 70 organic molecules with seven and fewer heavy atoms) to predict the total energy of the remaining 99% of this dataset with sub-kcal/mol accuracy. This MOB-ML model is significantly more accurate than other methods when transferred to a dataset comprising of 13 heavy atom molecules, exhibiting no loss of accuracy on a size intensive (i.e., per-electron) basis. It is shown that MOB-ML also works well for extrapolating to transition-state structures, predicting the barrier region for malonaldehyde intramolecular proton-transfer to within 0.35 kcal/mol when only trained on reactant/product-like structures. Finally, the use of the Gaussian process variance enables an active learning strategy for extending the MOB-ML model to new regions of chemical space with minimal effort. We demonstrate this active learning strategy by extending a QM7b-T model to describe non-covalent interactions in the protein backbone–backbone interaction dataset to an accuracy of 0.28 kcal/mol.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Fast and Accurate Pixel Calibration of Tof Neutron Diffractometers with Machine Learning

At a spallation neutron source, neutron pulses of varying energies are generated, and the detection of neutrons by instrument detectors is recorded as time-of-flight from the emission of the neutron pulse to its arrival at specific detector pixels with high time resolution. The flight path of neutrons from the moderator to the sample and then to the detector must be precisely calibrated at the detector-pixel level using standard powders, so the neutron events from all pixels can be time-focused to produce high-resolution diffraction patterns. Modern time-of-flight neutron diffractometers at spallation neutron sources are equipped with two-dimensional detectors with millimeter-scale pixelations. The number of pixels in a diffraction instrument can reach millions, which makes a single-pixel-level calibration process time-consuming or even impossible with conventional refinement or fitting approaches. Here we present a machine-learning-aided calibration process using a train-and-predict approach, in which machine learning models are trained on the relationship between an individual pixel time-of-flight diffraction pattern and its diffraction constant. These models use a portion of the available pixels for training, and a good model then predicts the diffraction constants precisely and rapidly for large sets of pixel diffraction patterns.

detector pixel calibration↗

Quantifying mean, variability, and uncertainty in indoor radon exposure in Pennsylvania using random forest and quantile regression forest models

Radon is a naturally occurring radioactive gas that poses a serious health risk as the primary cause of lung cancer in non-smokers. Despite the well-known adverse association with health outcomes, current radon exposure assessments are limited to county-level or average-level estimates, which fail to capture regional variability. This study uses Machine Learning models, including Random Forest (RF) and Quantile Regression Forest (QRF), to estimate the indoor radon concentrations at the ZCTA (Zip code tabulation area)-level and characterize uncertainties in model estimates. Incorporating geological, meteorological, and building-specific data, the models aim to improve radon risk assessment by capturing mean exposure, variability, and extreme concentration levels. Processed radon test data (n = 718,111) were analyzed using average, variability, and quantile prediction methods. Models that estimate the average radon exposure at the ZCTA-level can yield promising model-fit results, but they do not capture the underlying variability of indoor radon exposure within a ZCTA. We utilize volatility analyses to identify characteristics indicative of high variability of indoor radon exposure. We also show that a QRF model can be used to estimate upper quantiles of residential radon exposure, thereby uncovering localized areas of elevated exposure that were not apparent in mean estimates. The results highlighted the need for a deep characterization of exposure risk and show that regions with moderate average exposure levels could still harbor extreme outliers with implications for evaluating health risks. Utilizing multiple radon exposure models allows for a deeper characterization of radon risk within a geographic area and can better identify high-risk areas. The results from this study provide a foundation for developing mitigation strategies and examining associations between radon exposure and health outcomes at fine scales. Future research should extend the geographic scope and incorporate additional environmental risk factors to establish a comprehensive framework for risk assessment.

Lee, Heechan [ORNL]↗

Integrating Machine Learning into a Methodology for Early Detection of Wellbore Failure [Slides]

Approximately 93% of US total energy supply is dependent on wellbores in some form. The industry will drill more wells in next ten years than in the last 100 years (King, 2014). Global well population is around 1.8 million of which approximately 35% has some signs of leakage (i.e. sustained casing pressure). Around 5% of offshore oil and gas wells “fail” early, more with age and most with maturity. 8.9% of “shale gas” wells in the Marcellus play have experienced failure (120 out of 1,346 wells drilled in 2012) (Ingraffea et al., 2014). Current methods for identifying wells that are at highest priority for increased monitoring and/or at highest risk for failure consists of “hand” analysis of multi-arm caliper (MAC) well logging data and geomechanical models. Machine learning (ML) methods are of interest to explore feasibility for increasing analysis efficiency and/or enhanced detection of precursors to failure (e.g. deformations). MAC datasets used to train ML algorithms and preliminary tests were run for “predicting” casing collar locations and performed above 90% in classification and identifying of casing collar locations.

54 ENVIRONMENTAL SCIENCES↗

Integrating Machine Learning into a Methodology for Early Detection of Wellbore Failure [Slides]

Approximately 93% of US total energy supply is dependent on wellbores in some form. The industry will drill more wells in next ten years than in the last 100 years (King, 2014). Global well population is around 1.8 million of which approximately 35% has some signs of leakage (i.e. sustained casing pressure). Around 5% of offshore oil and gas wells “fail” early, more with age and most with maturity. 8.9% of “shale gas” wells in the Marcellus play have experienced failure (120 out of 1,346 wells drilled in 2012) (Ingraffea et al., 2014). Current methods for identifying wells that are at highest priority for increased monitoring and/or at highest risk for failure consists of “hand” analysis of multi-arm caliper (MAC) well logging data and geomechanical models. Machine learning (ML) methods are of interest to explore feasibility for increasing analysis efficiency and/or enhanced detection of precursors to failure (e.g. deformations). MAC datasets used to train ML algorithms and preliminary tests were run for “predicting” casing collar locations and performed above 90% in classification and identifying of casing collar locations.

54 ENVIRONMENTAL SCIENCES↗

Privacy-Preserving Knowledge Transfer with Bootstrap Aggregation of Teacher Ensembles

There is a need to transfer knowledge among institutions and organizations to save effort in annotation and labeling or in enhancing task performance. However, knowledge transfer is difficult because of restrictions that are in place to ensure data security and privacy. Institutions are not allowed to exchange data or perform any activity that may expose personal information. With the leverage of a differential privacy algorithm in a high-performance computing environment, we propose a new training protocol, Bootstrap Aggregation of Teacher Ensembles (BATE), which is applicable to various types of machine learning models. The BATE algorithm is based on and provides enhancements to the PATE algorithm, maintaining competitive task performance scores on complex datasets with underrepresented class labels.We conducted a proof-of-the-concept study of the information extraction from cancer pathology report data from four cancer registries and performed comparisons between four scenarios: no collaboration, no privacy-preserving collaboration, the PATE algorithm, and the proposed BATE algorithm. The results showed that the BATE algorithm maintained competitive macro-averaged F1 scores, demonstrating that the suggested algorithm is an effective yet privacy-preserving method for machine learning and deep learning solutions.

Yoon, Hong-Jun↗

Charting the energy landscape of metal/organic interfaces via machine learning

The rich polymorphism exhibited by inorganic/organic interfaces is a major challenge for materials design. Here in this work, we present a method to efficiently explore the potential energy surface and predict the formation energies of polymorphs and defects. This is achieved by training a machine learning model on a list of only 100 candidate structures that are evaluated via dispersion-corrected density functional theory (DFT) calculations. Finally, we demonstrate the power of this approach for tetracyanoethylene on Ag(100) and explain the anisotropic ordering that is observed experimentally.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Two-level overlapping additive Schwarz preconditioner for training scientific machine learning applications

In this work we introduce a novel two-level overlapping additive Schwarz preconditioner for accelerating the training of scientific machine learning applications. The design of the proposed preconditioner is motivated by the nonlinear two-level overlapping additive Schwarz preconditioner. The neural network parameters are decomposed into groups (subdomains) with overlapping regions. In addition, the network’s feed-forward structure is indirectly imposed through a novel subdomain-wise synchronization strategy and a coarse-level training step. Through a series of numerical experiments, which consider physicsinformed neural networks and operator learning approaches, we demonstrate that the proposed two-level preconditioner significantly speeds up the convergence of the standard (LBFGS) optimizer while also yielding more accurate machine learning models. Moreover, the devised preconditioner is designed to take advantage of model-parallel computations, which can further reduce the training time.

97 MATHEMATICS AND COMPUTING↗

Multi-Omics integration can be used to rescue metabolic information for some of the dark region of the Pseudomonas putida proteome

In every omics experiment, genes or their products are identified for which even state of the art tools are unable to assign a function. In the biotechnology chassis organism Pseudomonas putida, these proteins of unknown function make up 14% of the proteome. This missing information can bias analyses since these proteins can carry out functions which impact the engineering of organisms. As a consequence of predicting protein function across all organisms, function prediction tools generally fail to use all of the types of data available for any specific organism, including protein and transcript expression information. Additionally, the release of Alphafold predictions for all Uniprot proteins provides a novel opportunity for leveraging structural information. We constructed a bespoke machine learning model to predict the function of recalcitrant proteins of unknown function in Pseudomonas putida based on these sources of data, which annotated 1079 terms to 213 proteins. Among the predicted functions supplied by the model, we found evidence for a significant overrepresentation of nitrogen metabolism and macromolecule processing proteins. These findings were corroborated by manual analyses of selected proteins which identified, among others, a functionally unannotated operon that likely encodes a branch of the shikimate pathway.

60 APPLIED LIFE SCIENCES↗

Extensive Attention Mechanisms in Graph Neural Networks for Materials Discovery

We present our research where attention mechanism is extensively applied to various aspects of graph neural net- works for predicting materials properties. As a result, surrogate models can not only replace costly simulations for materials screening but also formulate hypotheses and insights to guide further design exploration. We predict formation energy of the Materials Project and gas adsorption of crystalline adsorbents, and demonstrate the superior performance of our graph neural networks. Moreover, attention reveals important substructures that the machine learning models deem important for a material to achieve desired target properties. Our model is based solely on standard structural input files containing atomistic descriptions of the adsorbent material candidates. We construct novel methodological extensions to match the prediction accuracy of state-of-the-art models some of which were built with hundreds of features at much higher computational cost. We show that sophisticated neural networks can obviate the need for elaborate feature engineering. Our approach can be more broadly applied to optimize gas capture processes at industrial scale.

Cong, Guojing↗

Deep learning of experimental electrochemistry for battery cathodes across diverse compositions

Artificial intelligence (AI) has emerged as a tool for discovering and optimizing novel battery materials. However, the adoption of AI in battery cathode representation and discovery is still limited due to the complexity of optimizing multiple performance properties and the scarcity of high-fidelity data. Here, we present a machine learning model (DRXNet) for battery informatics and demonstrate the application in the discovery and optimization of disordered rocksalt (DRX) cathode materials. We have compiled the electrochemistry data of DRX cathodes over the past 5 years, resulting in a dataset of more than 19,000 discharge voltage profiles on diverse chemistries spanning 14 different metal species. Learning from this extensive dataset, our DRXNet model can capture critical features in the cycling curves of DRX cathodes under various conditions. Our approach offers a data-driven solution to facilitate the rapid identification of novel cathode materials, accelerating the development of next-generation batteries for carbon neutralization.

25 ENERGY STORAGE↗