Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural Networks Research”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Improving the Concrete Crack Detection Process via a Hybrid Visual Transformer Algorithm

Inspections of concrete bridges across the United States represent a significant commitment of resources, given their biannual mandate for many structures. With a notable number of aging bridges, there is an imperative need to enhance the efficiency of these inspections. This study harnessed the power of computer vision to streamline the inspection process. Our experiment examined the efficacy of a state-of-the-art Visual Transformer (ViT) model combined with distinct image enhancement detector algorithms. We benchmarked against a deep learning Convolutional Neural Network (CNN) model. These models were applied to over 20,000 high-quality images from the Concrete Images for Classification dataset. Traditional crack detection methods often fall short due to their heavy reliance on time and resources. This research pioneers bridge inspection by integrating ViT with diverse image enhancement detectors, significantly improving concrete crack detection accuracy. Notably, a custom-built CNN achieves over 99% accuracy with substantially lower training time than ViT, making it an efficient solution for enhancing safety and resource conservation in infrastructure management. These advancements enhance safety by enabling reliable detection and timely maintenance, but they also align with Industry 4.0 objectives, automating manual inspections, reducing costs, and advancing technological integration in public infrastructure management.

42 ENGINEERING↗
DOE OSTI2020

A comparison of computational models for predicting yield sooting index

Sooting propensity, a measurement of how much particulate matter is produced when a fuel is burned, is a property of significant interest among researchers who are striving to discover the next generation of cleaner, more efficient fuels and fuel additives. Many compounds are not viable as fuels and/or fuel additives, and as a result, designing cleaner-burning biofuels using only experimental techniques is inefficient. Predictive models have been instrumental in reducing this inherent difficulty, providing researchers with a tool to preemptively screen compounds before production and testing. The present work compares the accuracies and interpretabilities of existing models used to predict a particular measure of sooting propensity, Yield Sooting Index (YSI). These models include artificial neural networks, graph neural networks, and multivariate equations. A novel equation for predicting YSI based on atom path count and bond order is proposed, which can highlight key structural components that contribute to YSI. It was found that artificial neural networks slightly outperform graph neural networks and greatly outperform multivariate equations in blind (test set) prediction accuracy; however, graph neural networks and multivariate equations provide significantly more interpretability as to how compound structure relates to YSI. Predictions of YSI are compared to experimental measurements for previously un-tested compounds with cetane numbers comparable to diesel fuel (50-60) (butyl decanoate, ethyl decanoate, 1,4-bis(ethenoxymethyl)cyclohexane, and 5-heptyloxolan-2-one), and it was found that these compounds produce significantly less soot compared to diesel fuel.

09 BIOMASS FUELS↗
DOE OSTI2020

Deep learning of interface structures from simulated 4D STEM data: cation intermixing vs. roughening ∗

Abstract Interface structures in complex oxides remain an active area of condensed matter physics research, largely enabled by recent advances in scanning transmission electron microscopy (STEM). Yet the nature of the STEM contrast in which the structure is projected along the given direction precludes separation of possible structural models. Here, we utilize deep convolutional neural networks (DCNN) trained on simulated 4D STEM datasets to predict structural descriptors of interfaces. We focus on the widely studied interface between LaAlO 3 and SrTiO 3 , using dynamical diffraction theory and leveraging high performance computing to simulate thousands of possible 4D STEM datasets to train the DCNN to learn properties of the underlying structures on which the simulations are based. We test the DCNN on simulated data and show that it is possible (with >95% accuracy) to identify a physically rough from a chemically diffuse interface and create a DCNN regression model to predict step positions. We quantify the applicability of the model to different thicknesses and the transferability of the approach. The method shown here is general and can be applied for any inverse imaging problem where forward models are present.

42 ENGINEERING↗
DOE OSTI2024

Speeding up and reducing memory usage for scientific machine learning via mixed precision

Scientific machine learning (SciML) has emerged as a versatile approach to address complex computational science and engineering problems. Within this field, physics-informed neural networks (PINNs) and deep operator networks (DeepONets) stand out as the leading techniques for solving partial differential equations by incorporating both physical equations and experimental data. However, training PINNs and DeepONets require significant computational resources, including long computational times and large amounts of memory. In search of computational efficiency, training neural networks using half precision (float16) rather than the conventional single (float32) or double (float64) precision has gained substantial interest, given the inherent benefits of reduced computational time and memory consumed. However, we find that float16 cannot be applied to SciML methods, because of gradient divergence at the start of training, weight updates going to zero, and the inability to converge to a local minima. To overcome these limitations, we explore mixed precision, which is an approach that combines the float16 and float32 numerical formats to reduce memory usage and increase computational speed. Our experiments showcase that mixed precision training not only substantially decreases training times and memory demands but also maintains model accuracy. Here, we also reinforce our empirical observations with a theoretical analysis. The research has broad implications for SciML in various computational applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Search for Stable and Low-Energy Ce–Co–Cu Ternary Compounds Using Machine Learning

Cerium-based intermetallics have garnered significant research attention as potential new permanent magnets. In this study, we explore the compositional and structural landscape of Ce−Co−Cu ternary compounds using a machine learning (ML)- guided framework integrated with first-principles calculations. We employ a crystal graph convolutional neural network (CGCNN), which enables efficient screening for promising candidates, significantly accelerating the material discovery process. With this approach, we predict five stable compounds, Ce 3 Co 3 Cu, CeCoCu 2 , Ce 12 Co 7 Cu, Ce 11 Co 9 Cu, and Ce 10 Co 11 Cu 4 , with formation energies below the convex hull, along with hundreds of low-energy (possibly metastable) Ce−Co−Cu ternary compounds. Firstprinciples calculations reveal that several structures are both energetically and dynamically stable. Notably, two Co-rich low-energy compounds, Ce 4 Co 33 Cu and Ce 4 Co 31 Cu 3 , are predicted to have high magnetizations.

Chemical structure↗
DOE OSTI2024

Exploring Object Detection and Image Classification Tasks for Niche Use Case in Naturalistic Driving Studies

Naturalistic driving studies consist of drivers using their personal vehicles and provide valuable real-world data, but privacy issues must be handled very carefully. Drivers sign a consent form when they elect to participate, but passengers do not for a variety of practical reasons. However, their privacy must still be protected. One large study includes a blurred image of the entire cabin which allows reviewers to find passengers in the vehicle; this protects the privacy but still allows a means of answering questions regarding the impact of passengers on driver behavior. A method for automatically counting the passengers would have scientific value for transportation researchers. We investigated different image analysis methods for automatically locating and counting the non-drivers including simple face detection and fine-tuned methods for image classification and a published object detection method. We also compared the image classification using convolutional neural network and vision transformer backbones. Our studies show the image classification method appears to work the best in terms of absolute performance, although we note the closed nature of our dataset and nature of the imagery makes the application somewhat niche and object detection methods also have advantages. We perform some analysis to support our conclusion.

Peruski, Ryan↗
DOE OSTI2024

Computational Approaches for Clean Energy Materials

Currently, 80% of the global final energy consumption occurs in form of fuels and only 20% as electricity. On the other hand, renewable energy additions come almost exclusively in the form of electricity (dominantly photovoltaics and wind). Thus, a successful energy transition will require enormous growth in renewables, sufficient to convert excess electricity into fuels, as well as the development of non-electricity based solar fuel technologies. As much as photovoltaic capacities have grown over the past 20 years, it is far from clear that current technologies and materials are up to the task to grow from here by yet another factor 100 until 2050. Therefore, sustained research efforts on emerging inorganic semiconductors for solar electricity and fuels are essential for facing the double challenge of climate change and energy security. Computational materials science can make important contributions, guiding and supporting research activities through both materials search and discovery and through detailed studies that help to develop a mechanistic understanding of materials performance and bottlenecks. This presentation will highlight three recent computational projects with relevance for photovoltaics and solar fuels (1) Defect graph neural networks (dGNN) for materials discovery in solar thermochemical hydrogen (STCH) [1]. The dGNN approach facilitates broad and fast materials screening for defect properties. (2) Modeling highly off-stoichiometric systems by evaluating the free energy of defect interaction [2]. This approach allows quantitative prediction of H2 production in complex STCH oxides. (3) First-principles atomic structure prediction for interfaces [3]. This work showed how an atomically thin CdCl2 interlayer phase enables in principle ideal electron transport across the incommensurate SnO2/CdTe interface. [1] M.D. Witman, A. Goyal, T. Ogitsu, A.H. McDaniel, S. Lany, Nat. Comput. Sci. (2023). https://doi.org/10.1038/s43588-023-00495-2. [2] A. Goyal, M.D. Sanders, R.P. O'Hayre, S. Lany, PRX Energy 3, 013008 (2024). https://doi.org/10.1103/PRXEnergy.3.013008. [3] A. Sharan, M. Nardone, D. Krasikov, N. Singh, S. Lany, Appl. Phys. Rev. 9, 041411 (2022). https://doi.org/10.1063/5.0104008.

density functional theory↗
DOE OSTI2023

Emerging opportunities for hybrid perovskite solar cells using machine learning

While there are several bottlenecks in hybrid organic–inorganic perovskite (HOIP) solar cell production steps, including composition screening, fabrication, material stability, and device performance, machine learning approaches have begun to tackle each of these issues in recent years. Different algorithms have successfully been adopted to solve the unique problems at each step of HOIP development. Specifically, high-throughput experimentation produces vast amount of training data required to effectively implement machine learning methods. Here, we present an overview of machine learning models, including linear regression, neural networks, deep learning, and statistical forecasting. Experimental examples from the literature, where machine learning is applied to HOIP composition screening, thin film fabrication, thin film characterization, and full device testing, are discussed. These paradigms give insights into the future of HOIP solar cell research. As databases expand and computational power improves, increasingly accurate predictions of the HOIP behavior are becoming possible.

Hering, Abigail R. (ORCID:0000000270806953)↗
DOE OSTI2022

Liquid-Liquid Transition in Water from First Principles

A long-standing question in water research is the possibility that supercooled liquid water can undergo a liquid-liquid phase transition (LLT) into high- and low-density liquids. Here, we used several complementary molecular simulation techniques to evaluate the possibility of an LLT in an ab initio neural network model of water trained on density functional theory calculations with the SCAN exchange correlation functional. We conclusively show the existence of a first-order LLT and an associated critical point in the SCAN description of water, representing the first definitive computational evidence for an LLT in water from first principles.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Iterative ML and Experiments for Emerging VOCs

SAND2026-17074O Iterative ML and Experiments for Emerging VOCs is a tool that analyzes and predicts the behaviors of SARS-CoV-2 variants. It processes experimental data on ACE2 (the receptor for the SARS-CoV-2 virus that allows it to infect the cell) and antibody binding using machine learning models, including neural networks, to forecast ACE2 interactions and variant expression. The tool employs transfer learning and global epistasis modeling, integrating public datasets with proprietary data to enhance prediction accuracy. Additionally, it fits concentration-response curves to determine dissociation constants and generates visualizations to support research findings, thereby aiding in the identification of new antibodies for emerging variants of concern. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Sheffield, Thomas [Sandia National Lab. (SNL-NM), ↗
DOE OSTI2020

A Small, Low Cost Event-Driven Architecture for Spiking Neural Networks on FPGAs

Currently, there is a lack of availability of low cost, low power neuromorphic hardware. In this work, we introduce the μCaspian architecture along with an associated development PCB design which provides a low cost and SWaP (size, weight, and power) optimized neuromorphic hardware platform. Further, our proposed system only uses commercial off the shelf components and an open source FPGA workflow to maximize the accessibility of μCaspian to all researchers.

Mitchell, Parker↗
DOE OSTI2023

Uncertainty quantification of a deep learning fuel property prediction model

Deep learning models are being widely used in the field of combustion. Given the black-box nature of typical neural network based models, uncertainty quantification (UQ) is critical to ensure the reliability of predictions as well as the training datasets, and for a principled quantification of noise and its various sources. Deep learning surrogate models for predicting properties of chemical compounds and mixtures have been recently shown to be promising for enabling data-driven fuel design and optimization, with the ultimate goal of improving efficiency and lowering emissions from combustion engines. In this study, UQ is performed for a multi-task deep learning model that simultaneously predicts the research octane number (RON), Motor Octane Number (MON), and Yield Sooting Index (YSI) of pure components and multicomponent blends. The deep learning model is comprised of three smaller networks: Extractor 1, Extractor 2, and Predictor, and a mixing operator. The molecular fingerprints of individual components are encoded via Extractor 1 and Extractor 2, the mixing operator generates fingerprints for mixtures/blends based on linear mixing operation, and the predictor maps the fingerprint to the target properties. Two different classes of UQ methods, Monte Carlo ensemble methods and Bayesian neural networks (BNNs), are employed for quantifying the epistemic uncertainty. Combinations of Bernoulli and Gaussian distributions with DropConnect and DropOut techniques are explored as ensemble methods. All the DropConnect, DropOut and Bayesian layers are applied to the predictor network. Aleatoric uncertainty is modeled by assuming that each data point has an independent uncertainty associated with it. The results of the UQ study are further analyzed to compare the performance of BNN and ensemble methods. Although this study is confined to UQ of fuel property prediction, the methodologies are applicable to other deep learning frameworks that are being widely used in the combustion community.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

Neural Networks to Find the Optimal Forcing for Offsetting the Anthropogenic Climate Change Effects

Abstract Of great relevance to climate engineering is the systematic relationship between the radiative forcing to the climate system and the response of the system, a relationship often represented by the linear response function (LRF) of the system. However, estimating the LRF often becomes an ill-posed inverse problem due to high-dimensionality and nonunique relationships between the forcing and response. Recent advances in machine learning make it possible to address the ill-posed inverse problem through regularization and sparse system fitting. Here, we develop a convolutional neural network (CNN) for regularized inversion. The CNN is trained using the surface temperature responses from a set of Green’s function perturbation experiments as imagery input data together with data sample densification. The resulting CNN model can infer the forcing pattern responsible for the temperature response from out-of-sample forcing scenarios. This promising proof of concept suggests a possible strategy for estimating the optimal forcing to negate certain undesirable effects of climate change. The limited success of this effort underscores the challenges of solving an inverse problem for a climate system with inherent nonlinearity. Significance Statement Predicting the climate response for a given climate forcing is a direct problem, while inferring the forcing for a given desired climate response is often an inverse, ill-posed, problem, posing a new challenge to the climate community. This study makes the first attempt to infer the radiative forcing for a given target pattern of global surface temperature response using a deep learning approach. The resulting deeply trained convolutional neural network inversion model shows promise in capturing the forcing pattern corresponding to a given surface temperature response, with a significant implication on the design of an optimal solar radiation management strategy for curbing global warming. This study also highlights the technical challenges that future research should prioritize in seeking feasible solutions to the inverse climate problem.

Ren, Huiying↗
DOE OSTI2021

Surrogate Model of Flexible Research Platform EnergyPlus Models to Enable Sensitivity Analysis

This letter report describes the surrogate models developed from the EnergyPlus model of Oak Ridge National Laboratory’s Flexible Research Platform. Two data-driven black-box models were developed, and the outputs of the surrogate models were compared with the EnergyPlus model. The two models developed are a multilayer perceptron deep learning model, and a long short-term memory (LSTM) neural network model. The three factors for selecting the black-box models are scalability, computation time, and accuracy. A total of 107 input variables were the dominant variables in determining the outputs of building energy consumptions and thermal comfort. A total of 54 output variables were identified as the prediction targets, including the system- and zone-level outputs. The large set of the simulation cases were generated by integrating sensor errors into an emulator based on EnergyPlus and Python EMS, which includes advanced control sequences from ASHRAE Guideline 36-2018: High-Performance Sequences of Operation. The surrogate models were developed based on a set of large-scale simulation runs (i.e., 4,000 runs) on a cloud platform. The comparison analysis shows that the two black-box models had good accuracy for predicting new outputs for sensitivity analysis using the root mean square error metric. As a next step, the developed surrogate models will be used to perform sensitivity analysis for different sensor impacts (e.g., sensor types, sensor locations).

42 ENGINEERING↗
DOE OSTI2025

From tides to seasons: How cyclic tidal drivers and plant physiology interact to affect carbon cycling at the terrestrial-estuarine boundary (Final technical report)

Coastal ecosystems are among the most biologically and biogeochemically active and diverse systems on Earth. Because they act as important linkages between terrestrial ecosystems and the open ocean, their incorporation in Earth system models (ESMs) is critical to predict coastal and global responses to environmental changes. However, they vary greatly in the magnitude of tides and the volume and timing of freshwater input from land, making it challenging to model the major biogeochemical reactions that control productivity and greenhouse gas emissions across coastal terrestrial aquatic interfaces (TAIs). Our overall objective was to improve mechanistic process understanding and modeling of tidal wetland hydro-biogeochemistry in coastal TAIs. We established a new flux tower site (Ameriflux US-PLo) in the oligohaline part of the Parker River to continuously monitor ecosystem-scale carbon fluxes under temporally varying salinity conditions. The site is co-located with long-term monitoring plots of the Plum Island Ecosystems LTER project. We installed wells and redox sensors in the marsh interior and creek bank, established biomass monitoring plots and deployed novel optode sensors in both locations. We used this data to parameterize plant-mediated transport in PFLOTRAN and tested the impact of soil heterogeneity on porewater constituents and gas fluxes. We collected observations of root oxygen release with a novel planar optode system in the field. Flux data collected during the measurement period encompasses a large variation in salinity ranging from drought to record precipitation years. We developed a method to extract functional relationships from the flux data using artificial neural networks, identifying salinity thresholds for CH 4 fluxes. Finally, we are using the coupled ELM-PFLOTRAN model to test the impact of antecedent hydrological conditions on the salinity-CH 4 flux relationship. This grant contributed to the professional development of one postdoc, three research assistants and one graduate student. The sensor data has been shared with external collaborators.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Unveiling mesoscopic structures in distorted lamellar phases through deep learning-based small angle neutron scattering analysis

Hypothesis: The formation of distorted lamellar phases, distinguished by their arrangement of crumpled, stacked layers, is frequently accompanied by the disruption of long-range order, leading to the formation of interconnected network structures commonly observed in the sponge phase. Nevertheless, traditional scattering functions grounded in deterministic modeling fall short of fully representing these intricate structural characteristics. Our hypothesis posits that a deep learning method, in conjunction with the generalized leveled wave approach used for describing structural features of distorted lamellar phases, can quantitatively unveil the inherent spatial correlations within these phases. Experiments and Simulations: This report outlines a novel strategy that integrates convolutional neural networks and variational autoencoders, supported by stochastically generated density fluctuations, into a regression analysis framework for extracting structural features of distorted lamellar phases from small angle neutron scattering data. To evaluate the efficacy of our proposed approach, we conducted computational accuracy assessments and applied it to the analysis of experimentally measured small angle neutron scattering spectra of AOT surfactant solutions, a frequently studied lamellar system. Findings: The findings unambiguously demonstrate that deep learning provides a dependable and quantitative approach for investigating the morphology of wide variations of distorted lamellar phases. It is adaptable for deciphering structures from the lamellar to sponge phase including intermediate structures exhibiting fused topological features. In conclusion, this research highlights the effectiveness of deep learning methods in tackling complex issues in the field of soft matter structural analysis and beyond.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Scalable deep learning for watershed model calibration

Watershed models such as the Soil and Water Assessment Tool (SWAT) consist of high-dimensional physical and empirical parameters. These parameters often need to be estimated/calibrated through inverse modeling to produce reliable predictions on hydrological fluxes and states. Existing parameter estimation methods can be time consuming, inefficient, and computationally expensive for high-dimensional problems. In this paper, we present an accurate and robust method to calibrate the SWAT model (i.e., 20 parameters) using scalable deep learning (DL). We developed inverse models based on convolutional neural networks (CNN) to assimilate observed streamflow data and estimate the SWAT model parameters. Scalable hyperparameter tuning is performed using high-performance computing resources to identify the top 50 optimal neural network architectures. We used ensemble SWAT simulations to train, validate, and test the CNN models. We estimated the parameters of the SWAT model using observed streamflow data and assessed the impact of measurement errors on SWAT model calibration. We tested and validated the proposed scalable DL methodology on the American River Watershed, located in the Pacific Northwest-based Yakima River basin. Our results show that the CNN-based calibration is better than two popular parameter estimation methods (i.e., the generalized likelihood uncertainty estimation [GLUE] and the dynamically dimensioned search [DDS], which is a global optimization algorithm). For the set of parameters that are sensitive to the observations, our proposed method yields narrower ranges than the GLUE method but broader ranges than values produced using the DDS method within the sampling range even under high relative observational errors. The SWAT model calibration performance using the CNNs, GLUE, and DDS methods are compared using R 2 and a set of efficiency metrics, including Nash-Sutcliffe, logarithmic Nash-Sutcliffe, Kling-Gupta, modified Kling-Gupta, and non-parametric Kling-Gupta scores, computed on the observed and simulated watershed responses. The best CNN-based calibrated set has scores of 0.71, 0.75, 0.85, 0.85, 0.86, and 0.91. The best DDS-based calibrated set has scores of 0.62, 0.69, 0.8, 0.77, 0.79, and 0.82. The best GLUE-based calibrated set has scores of 0.56, 0.58, 0.71, 0.7, 0.71, and 0.8. The scores above show that the CNN-based calibration leads to more accurate low and high streamflow predictions than the GLUE and DDS sets. Our research demonstrates that the proposed method has high potential to improve our current practice in calibrating large-scale integrated hydrologic models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Attention-based 3D – convolutional neural network model for mechanical property predictions using visible light images in metal additive manufacturing

Additive manufacturing (AM), while commonly used for rapid prototyping and creating components with complex geometries, has not been widely adopted for critical applications across the aerospace, automotive, defense, energy, and medical industries. This is, in part, due to the challenges of controlling flaws and uncertainty in the mechanical behavior of additively manufactured components. In recent years, there has been an increase in research aimed at predicting the final mechanical properties of additively manufactured components during the printing process. To address these issues, a 3D-CNN model was trained using low-cost in situ visible-light camera data, anomaly classifications, and the chosen process parameters to predict the ultimate tensile strength (UTS), yield strength (YS), total elongation (TE), and uniform elongation (UE). The 3D-CNN layers of the model employed attention mechanisms to prioritize features in the data, thereby improving prediction accuracy. Furthermore, the effect of each process parameter and anomaly class is investigated using attention-based dynamic sigmoid weighted gates to interpret the influence each class has on the final prediction. Different combinations of the in situ data were fed into the 3D-CNN, with varying amounts of image layers, to determine the ideal combination for predicting mechanical properties in situ. Here, the 3D-CNN model achieved mean absolute percentage errors (MAPE) below 5% for both UTS and YS while using only a single camera input and under half of the available image layers.

36 MATERIALS SCIENCE↗