Search NASA⌕ Search

SEARCH · Search NASA

Results for “Machine Learning Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41

Using feature importance as an exploratory data analysis tool on Earth system models

Abstract. Machine learning (ML) models are commonly used to generate predictions, but these models can also support the discovery of new science. Generating accurate predictions necessitates that a model captures the structure of the underlying data. If the structure is properly extracted, ML could be a useful exploratory and evidential tool. In this paper, we present a case study that demonstrates the use of ML for exploratory data analysis (EDA) in the climate space. We apply the ML explainability method of spatiotemporal zeroed feature importance (stZFI) to understand how climate-variable associations evolve over space and time. Our analyses focus on data from ensembles of Earth system models (ESMs) which provide data on different climate states and conditions. We elect to work with ESM ensembles since they allow us to compare feature importance across alternative scenarios not available with observed data. The ensembles also account for natural variability so that we can distinguish between signal and noise due to natural climate variability when computing feature importance. The use of perturbed initial condition ensembles introduces variability mimicking the natural variability in the atmosphere; thus the signals emerging using feature importance (FI) can be evaluated against the natural variability in the climate system. For our analyses, we consider the 1991 volcanic eruption of Mount Pinatubo, which was a large stratospheric aerosol injection. We explore the climate pathway associated with the eruption from aerosols to radiation to temperature at both the near-surface and stratospheric levels. In addition to applying the method to data generated from two different ESMs, we apply stZFI to reanalysis data to compare the associations identified by stZFI. We show how stZFI tracks the importance of aerosol optical depth over time on forecasting temperatures. This case study illustrates usefulness of an ML tool (stZFI) for EDA on a well-studied climate exemplar.

Ries, Daniel (ORCID:0000000250294647)↗

Power Electronics Materials and Bonded Interfaces - Reliability and Lifetime

Advanced packaging technologies are currently being designed and developed by the power electronics industry however, the maximum operating temperature is still limited to 175 degrees Celsius for the silicon carbide devices. Bonded materials such as sintered copper and polymeric materials are potential candidates for high temperature operation, but it is critical to characterize and evaluate its reliability under harsh operating conditions. In this project, we discuss the results of the accelerated experiments conducted on sintered copper and polymeric materials. Additionally, a novel framework to develop the lifetime prediction model of bonded interfaces through employing statistical and machine learning models are described. In this task, scanning acoustic microscope images of bonded interfaces obtained under thermal cycling experiments are used as the data.

ENGINEERING↗

Improving multiwell petrophysical interpretation from well logs via machine learning and statistical models

Well-log interpretation estimates in situ rock properties along well trajectory, such as porosity, water saturation, and permeability, to support reserve-volume estimation, production forecasts, and decision making in reservoir development. However, due to measurement errors, variability of well logs caused by multiple measurement vendors, different borehole tools, and nonuniform drilling/borehole conditions, estimations of rock properties with original well logs without proper preprocessing may not be accurate, especially in the context of multiwell estimation. Well-log normalization techniques such as two-point scaling and mean-variance normalization are commonly used to improve the robustness of multiwell rock-property estimation. However, these techniques do not consider the correlation between well logs and require subjective knowledge for their effective implementation. To reduce uncertainties and processing time associated with multiwell rock-property estimation from well logs, we develop discriminative adversarial (DA) and linear constraint models for well-log normalization and rock-property estimation. The DA neural network model developed for well-log normalization and interpretation can perform linear and nonlinear well-log normalization while considering the joint distribution of each well log and rock properties. However, the linear constraint model uses an ensemble of predictions from linear models to constrain well-log normalization and rock-property estimation. We also develop a divergence-based type well identification method to select type (training) wells for a test well based on the statistical similarity of associated well-log distributions instead of the interwell distance. We apply the DA model to perform well-log normalization and prediction of permeability for the Seminole San Andres Unit carbonate reservoir. Compared with the permeability predicted with the classical machine learning model without well-log normalization and models with two-point scaling normalization, the DA model yields the most accurate permeability prediction by decreasing the mean-squared error of permeability prediction by 20%–50%.

Geochemistry & Geophysics↗

What Machine Learning Can and Cannot Do for Inertial Confinement Fusion

Machine learning methodologies have played remarkable roles in solving complex systems with large data, well-defined input–output pairs, and clearly definable goals and metrics. The methodologies are effective in image analysis, classification, and systems without long chains of logic. Recently, machine-learning methodologies have been widely applied to inertial confinement fusion (ICF) capsules and the design optimization of OMEGA (Omega Laser Facility) capsule implosion and NIF (National Ignition Facility) ignition capsules, leading to significant progress. As machine learning is being increasingly applied, concerns arise regarding its capabilities and limitations in the context of ICF. ICF is a complicated physical system that relies on physics knowledge and human judgment to guide machine learning. Additionally, the experimental database for ICF ignition is not large enough to provide credible training data. Most researchers in the field of ICF use simulations, or a mix of simulations and experimental results, instead of real data to train machine learning models and related tools. They then use the trained learning model to predict future events. This methodology can be successful, subject to a careful choice of data and simulations. However, because of the extreme sensitivity of the neutron yield to the input implosion parameters, physics-guided machine learning for ICF is extremely important and necessary, especially when the database is small, the uncertain-domain knowledge is large, and the physical capabilities of the learning models are still being developed. In this work, we identify problems in ICF that are suitable for machine learning and circumstances where machine learning is less likely to be successful. This study investigates the applications of machine learning and highlights fundamental research challenges and directions associated with machine learning in ICF.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Final LDRD Report: Using Linkography of Cyber Attack Patterns to Inform Honeytoken Placement

The war to establish cyber supremacy continues, and the literature is crowded with strictly technical cyber security measures. We present the results of a three year LDRD project using Linkography, a methodology new to the field of cyber security, we establish the foundation necessary to track and profile the microbehavior of humans attacking cyber systems. We also propose ways to leverage this understanding to influence and deceive these attackers. We studied the science of linkography, applied it to the cyber security domain, implemented a software package to manage linkographs, generated the preprocessing blocks necessary to ingest raw data, produced machine learning models, created ontology refinement algorithms and prototyped a web application for researchers and practitioners to apply linkography. Machine learning produced some of our key results: We trained and validated multinomial classifiers with a real world data set and predicted the attacker's next category of action with 86 to 98% accuracy; dimension reduction techniques indicated that the linkography-based features were among the most powerful. We also discovered ontology refinement algorithms that advanced the state of the art in linkography in general and cyber security in particular. We conclude that linkography is a viable tool for cyber security; we look forward to expanding our work to other data sources and using our prediction results to enable adversary deception techniques.

97 MATHEMATICS AND COMPUTING↗

Self-Supervised and Interpretable Anomaly Detection Using Network Transformers

Machine learning and deep neural networks (DNNs) have been proposed as a tool to identify anomalies in computer network communications. However, due the obfuscated nature of off-the-shelf machine learning models, their output often does not provide enough information to isolate the source of the anomaly to take corrective measures. In this article, we introduce the network transformer (NeT), a DNN model for anomaly detection that incorporates the graph structure of the communication network in order to improve interpretability. Further, the presented approach has the following advantages: first, enhanced interpretability by incorporating the graph structure of computer networks; second, provides a hierarchical set of features that enables analysis at different levels of granularity; second, self-supervised training that does not require labeled data. The NeT model was evaluated on a set of anomalous scenarios executed in a real industrial control system. The presented approach successfully identified the anomalies, the devices affected, and the specific connections causing the anomalies, providing a data-driven hierarchical approach to analyze the behavior of a cyber network.

97 MATHEMATICS AND COMPUTING↗

Xanthos-Lake Model Source Code

This repository contains the source code for Xanthos-Lake, a lake-modeling extension of the Xanthos framework that introduces a coupled lake component comprising the Xanthos-Lake Snow and Ice Model (xLSIM) and the Xanthos-Lake Water Balance Model (xLWBM). xLSIM is a basin-aware machine-learning model for lake snow, ice, and thermal conditions. It predicts monthly lake ice thickness, snow depth, snow-cover fraction, mixing-layer temperature, and lake ice fraction from meteorological forcing and lake surface-area information. It uses sequence-based deep-learning architectures, including Transformer and hybrid Long Short-Term Memory–Transformer (LSTM–Transformer) models, together with seasonal encoding, multi-lake learning, physical masking, and basin-level cryospheric and non-cryospheric classification. The training workflow uses Ray for scalable execution and includes optional Ray Tune hyperparameter optimization. Model predictions, observations, diagnostics, and feature-importance outputs are written in NetCDF. xLWBM is the water-balance component of the new lake framework. It simulates monthly lake storage, surface area, evaporation, inflow, outflow, and lake–groundwater exchange. It combines physical water-balance equations with calibrated bathymetric relationships, weir-based outlet flow, modified Penman open-water evaporation, groundwater head relaxation, Penman–Monteith snow and ice sublimation, and snow, ice, and thermal conditions supplied by xLSIM. The model calibrates lake parameters against satellite-derived surface-area data, using evaporation-based calibration where surface-area data are unavailable, and supports small, medium, and large lake classes. For large lakes, xLWBM is integrated with the managed-routing workflow so that lake storage and outflow interact directly with downstream river routing and reservoir operations. Together, xLSIM and xLWBM provide Xanthos with a coupled lake-modeling capability. xLSIM supplies the snow, ice, and thermal conditions that affect lake evaporation and snow- and ice-related water exchanges, while xLWBM translates those conditions into dynamic lake storage, surface area, evaporation, and discharge. In return, xLWBM supplies evolving lake surface area to xLSIM. This coupling enables Xanthos to represent lakes as active hydrologic components within basin-scale water-availability and routing simulations.

Machine Learning↗

Machine-Learning-Based Predictive Modeling of Glass Transition Temperatures: A Case of Polyhydroxyalkanoate Homopolymers and Copolymers

Polyhydroxyalkanoate-based polymers—being ecofriendly, biosynthesizable, and economically viable and possessing a broad range of tunable properties—are currently being actively pursued as promising alternatives for petroleum-based plastics. The vast chemical complexity accessible within this class of polymers gives rise to challenges in the rational discovery of novel polymer chemistries for specific applications. Here, the burgeoning field of polymer informatics addresses this challenge via providing tools and strategies for accelerated property prediction and materials design via surrogate machine-learning models built on reliable past data. In this contribution, we use glass transition temperature T g as an example target property to demonstrate promise of the data-enabled route to accelerated learning of accurate structure–property mappings in PHA-based polymers. Our analysis uses a data set of experimentally measured T g values, polymer molecular weights, and a polydispersity index for PHA-based homo- and copolymers that was carefully assembled from the literature. A fingerprinting scheme that captures key properties based on topology, shape, and charge/polarity of specific chemical units or motifs forming the polymer backbone was devised to numerically represent the polymers. A validated statistical learning model is then developed to allow for a mapping of the polymer fingerprints onto the property space in a physically meaningful and reliable manner. Once developed, the model can not only rapidly predict the property of new PHA polymers but also provide uncertainties underlying the predictions. The model is further combined with an evolutionary-algorithm-based search strategy to efficiently identify multicomponent polymer compositions with a prespecified T g . While the present contribution is focused specifically on T g , the surrogate model development approach put forward here is general and can, in principle, be extended to a range of other properties.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Autonomous Synthesis and Inverse Design of Electrochromic Polymers with High Efficiency and Accuracy

Here, the design and synthesis of functional polymers, aimed at targeted properties through specific structures, have long been challenged by their complex and often nonlinear structure–property relationships. Key processes, including knowledge accumulation for predictive design and experimental refinement and validation, are traditionally labor-insensitive and time-consuming, making it difficult to balance accuracy and efficiency. Here, we introduce an accelerated, autonomous system for the on-demand synthesis of electronic polymers that achieves the desired electrochromic functionality with high accuracy and efficiency. Our approach leverages large language model-assisted data mining, a physics-informed copolymer machine learning model, and an AI-driven autonomous robotic workflow in the Polybot lab. Within 72 h, Polybot autonomously synthesized electrochromic polymers (ECPs) with targeted, previously-unreported color values, including green polymers with specific absorption profiles, precisely fine-tuning copolymer structures with a 5% step size in comonomer composition within a three-monomer system. A publicly accessible ECP informatics database has also been created to foster knowledge exchange.

AI-driven Robotic Lab↗

Strategies and Software for Machine Learning Accelerated Discovery in Transition Metal Chemistry

Machine learning the electronic structure of open shell transition metal complexes presents unique challenges, including robust and automated data set generation. In this report we introduce tools that simplify data acquisition from density functional theory (DFT) and validation of trained machine learning models using the molSimplify automatic design (mAD) workflow. We demonstrate this workflow by training and comparing the performance of LASSO, kernel ridge regression (KRR), and artificial neural network (ANN) models using heuristic, topological revised autocorrelation (RAC) descriptors we have recently introduced for machine learning inorganic chemistry. On a series of open shell transition metal complexes, we evaluate set aside test errors of these models for predicting the HOMO level and HOMO–LUMO gap. The best performing models are ANNs, which show 0.15 and 0.25 eV test set mean absolute errors on the HOMO level and HOMO–LUMO gap, respectively. Poor performing KRR models using the full 153-feature RAC set are improved to nearly the same performance as the ANNs when trained on down-selected subsets of 20–30 features. Analysis of the essential descriptors for HOMO level and HOMO–LUMO gap prediction as well as comparison to subsets previously obtained for other properties reveal the paramount importance of nonlocal, steric properties in determining frontier molecular orbital energetics. We demonstrate our model performance on diverse complexes and in the discovery of molecules with target HOMO–LUMO gaps from a large 15,000 molecule design space in minutes rather than days that full DFT evaluation would require.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Graph-based featurization methods for classifying small molecule compounds

For over a decade, drug-induced liver injury (DILI) has posed significant drawbacks in the synthesis and development of drugs and remains a consequential concern. With finite success within the existing preclinical models, DILI is one of the main causes of drug withdrawal or termination from the market. Particularly, this withdrawal occurs during the late stages of drug development (Kullak-Ublick, 2017). Since DILI is difficult to diagnose and treat, it has become an obstacle in the drug production market that in turn affects clinicians, pharmaceutical companies, and consumers. We propose a method for learning features of DILI-positive drugs based on the graphical relationships and patterns they possess within a network of biological databases. We also train various statistical and machine learning models on these learned features in order to classify the drugs as DILI-positive or negative. Our methods include Random Forest, Neural networks, and logistic regression classification. We utilize labeled DILI-positive and DILI-negative datasets, which were developed by the FDA and the National center for toxicological research, as well as additional literature datasets (Thakkar, 2020) in order to validate our results and assess our featurization and model accuracy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Better understanding and prediction of antiviral peptides through primary and secondary structure feature importance

The emergence of viral epidemics throughout the world is of concern due to the scarcity of available effective antiviral therapeutics. The discovery of new antiviral therapies is imperative to address this challenge, and antiviral peptides (AVPs) represent a valuable resource for the development of novel therapies to combat viral infection. We present a new machine learning model to distinguish AVPs from non-AVPs using the most informative features derived from the physicochemical and structural properties of their amino acid sequences. To focus on those features that are most likely to contribute to antiviral performance, we filter potential features based on their importance for classification. These feature selection analyses suggest that secondary structure is the most important peptide sequence feature for predicting AVPs. Our Feature-Informed Reduced Machine Learning for Antiviral Peptide Prediction (FIRM-AVP) approach achieves a higher accuracy than either the model with all features or current state-of-the-art single classifiers. Understanding the features that are associated with AVP activity is a core need to identify and design new AVPs in novel systems. The FIRM-AVP code and standalone software package are available at https://github.com/pmartR/FIRM-AVP with an accompanying web application at https://msc-viz.emsl.pnnl.gov/AVPR.

59 BASIC BIOLOGICAL SCIENCES↗

Kohn–Sham accuracy from orbital-free density functional theory via Δ-machine learning

Here, we present a Δ-machine learning model for obtaining Kohn–Sham accuracy from orbital-free density functional theory (DFT) calculations. In particular, we employ a machine-learned force field (MLFF) scheme based on the kernel method to capture the difference between Kohn–Sham and orbital-free DFT energies/forces. We implement this model in the context of on-the-fly molecular dynamics simulations and study its accuracy, performance, and sensitivity to parameters for representative systems. We find that the formalism not only improves the accuracy of Thomas–Fermi–von Weizsäcker orbital-free energies and forces by more than two orders of magnitude but is also more accurate than MLFFs based solely on Kohn–Sham DFT while being more efficient and less sensitive to model parameters. We apply the framework to study the structure of molten Al 0.88 Si 0.12 , the results suggesting no aggregation of Si atoms, in agreement with a previous Kohn–Sham study performed at an order of magnitude smaller length and time scales.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Transient Optimization of the Cryogenic Moderator System Controller at the Spallation Neutron Source for Improved Performance

The high-energy neutron beam generated at the Spallation Neutron Source (SNS) at Oak Ridge National Laboratory is moderated to use cold (slow) neutrons for scientific discoveries. The Cryogenic Moderator System (CMS) removes heat from the neutron beam using cryogenic hydrogen (H 2 ) moderators connected via heat exchangers to a helium (He) refrigeration loop that dissipates heat using a compressor-brake system. However, the CMS is affected by sporadic losses in beam power, referred to as "beam trips," as these events generate significant disturbances in cooling requirements. To accommodate the heat load transients during beam trips, the CMS uses a decentralized control strategy consisting of four flow valves and one electric heater adjusted by independent proportional-integral (PI) controllers. During the CMS’s initial commissioning, the PI gains were calibrated based only on tracking performance, overlooking their effectiveness in disturbance rejection. A data-driven, control-oriented closed-loop model was developed to recalibrate the PI gains and minimize the transient disturbances caused by beam trips. The model consists of three main components: (1) a physics-based model of the He refrigeration loop, (2) a machine-learning model of the cryogenic H 2 cooling trains, and (3) the control logic used for feedback set-point tracking. Experimental results showed that the recalibrated gains obtained in this study improved the CMS’s transient response during beam trips.

Maldonado Puente, Bryan↗

OptML

OptML is an open-source Python package that provides modeling and solution capabilities for optimization problems with embedded machine learning models. OptML is a collaborative open-source project that includes advanced algorithms from the academic community for efficient solution of these problems. Sandia contributions to this package include the development of interfaces between these algorithms and Pyomo. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2021-10861 O

Jalving, Jordan↗

A Hybrid Approach for Model Order Reduction of Barotropic Quasi-Geostrophic Turbulence

We put forth a robust reduced-order modeling approach for near real-time prediction of mesoscale flows. In our hybrid-modeling framework, we combine physics-based projection methods with neural network closures to account for truncated modes. We introduce a weighting parameter between the Galerkin projection and extreme learning machine models and explore its effectiveness, accuracy and generalizability. To illustrate the success of the proposed modeling paradigm, we predict both the mean flow pattern and the time series response of a single-layer quasi-geostrophic ocean model, which is a simplified prototype for wind-driven general circulation models. We demonstrate that our approach yields significant improvements over both the standard Galerkin projection and fully non-intrusive neural network methods with a negligible computational overhead.

42 ENGINEERING↗

Machine learning to classify animal species in camera trap images: Applications in ecology

Motion-activated cameras (“camera traps”) are increasingly used in ecological and management studies for remotely observing wildlife and are amongst the most powerful tools for wildlife research. However, studies involving camera traps result in millions of images that need to be analysed, typically by visually observing each image, in order to extract data that can be used in ecological analyses. We trained machine learning models using convolutional neural networks with the ResNet-18 architecture and 3,367,383 images to automatically classify wildlife species from camera trap images obtained from five states across the United States. We tested our model on an independent subset of images not seen during training from the United States and on an out-of-sample (or “out-of-distribution” in the machine learning literature) dataset of ungulate images from Canada. We also tested the ability of our model to distinguish empty images from those with animals in another out-of-sample dataset from Tanzania, containing a faunal community that was novel to the model. The trained model classified approximately 2,000 images per minute on a laptop computer with 16 gigabytes of RAM. The trained model achieved 98% accuracy at identifying species in the United States, the highest accuracy of such a model to date. Out-of-sample validation from Canada achieved 82% accuracy and correctly identified 94% of images containing an animal in the dataset from Tanzania. We provide an r package (Machine Learning for Wildlife Image Classification) that allows the users to (a) use the trained model presented here and (b) train their own model using classified images of wildlife from their studies. The use of machine learning to rapidly and accurately classify wildlife in camera trap images can facilitate non-invasive sampling designs in ecological studies by reducing the burden of manually analysing images. Our r package makes these methods accessible to ecologists.

54 ENVIRONMENTAL SCIENCES↗