Automated and Autonomous Experiments in Electron and Scanning Probe Microscopy
Not Available
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
SEARCH · Search NASA
Results for “artificial neural network regression”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Machine Learning for Improving Surface-Layer-Flux Estimates
Abstract Flows in the atmospheric boundary layer are turbulent, characterized by a large Reynolds number, the existence of a roughness sublayer and the absence of a well-defined viscous layer. Exchanges with the surface are therefore dominated by turbulent fluxes. In numerical models for atmospheric flows, turbulent fluxes must be specified at the surface; however, surface fluxes are not known a priori and therefore must be parametrized. Atmospheric flow models, including global circulation, limited area models, and large-eddy simulation, employ Monin–Obukhov similarity theory (MOST) to parametrize surface fluxes. The MOST approach is a semi-empirical formulation that accounts for atmospheric stability effects through universal stability functions. The stability functions are determined based on limited observations using simple regression as a function of the non-dimensional stability parameter representing a ratio of distance from the surface and the Obukhov length scale (Obukhov in Trudy Inst Theor Geofiz AN SSSR 1:95–115, 1946), $$z/L$$ z / L . However, simple regression cannot capture the relationship between governing parameters and surface-layer structure under the wide range of conditions to which MOST is commonly applied. We therefore develop, train, and test two machine-learning models, an artificial neural network (ANN) and random forest (RF), to estimate surface fluxes of momentum, sensible heat, and moisture based on surface and near-surface observations. To train and test these machine-learning algorithms, we use several years of observations from the Cabauw mast in the Netherlands and from the National Oceanic and Atmospheric Administration’s Field Research Division tower in Idaho. The RF and ANN models outperform MOST. Even when we train the RF and ANN on one set of data and apply them to the second set, they provide more accurate estimates of all of the fluxes compared to MOST. Estimates of sensible heat and moisture fluxes are significantly improved, and model interpretability techniques highlight the logical physical relationships we expect in surface-layer processes.
Meteorology & Atmospheric Sciences↗
Assessment of Machine Learning for Ultrasonic Nondestructive Evaluation of Alkali–Silica Reaction in Concrete
Alkali–silica reaction (ASR) is a type of material degradation in concrete structures that leads to concrete cracking and rebar corrosion, thereby reducing the material’s structural integrity and the overall structure’s lifetime and raising safety concerns. Ultrasonic nondestructive evaluation (NDE) has been proven to be a valuable technique for assessing concrete properties and monitoring ASR progression in concrete. However, the deployment and analysis of ultrasonic NDE and its data requires specialized expertise, often relying on the engineer’s subjective interpretation. With the surge in computational power, artificial intelligence (AI) and machine learning (ML) algorithms have become popular in automating NDE data analysis. Various industrial sectors are increasingly adopting ML algorithms for NDE data analysis with a growing emphasis on AI–assisted automation. Regulatory agencies are also preparing for this technological shift, anticipating corresponding revisions in standards. Thus, there is an urgent need to identify the capabilities and limitations of current ML technologies for the evaluation of concrete material properties and damage status. Furthermore, the effects of various factors on ML model performance must be thoroughly investigated. The study summarized herein evaluated the effectiveness of two ML models (i.e., support vector regression (SVR) and deep neural network (DNN)) in predicting concrete material damage induced by ASR based on the long-term ultrasonic monitoring data. Four distinct concrete specimens were cast with artificially induced ASR, and over a period exceeding 500 days, ultrasonic signals and expansion data were continuously collected. For the SVR model, wave velocity and 12 other wave features were extracted from the ultrasonic signals, with 6 out of 13 features selected as input for the model. Different combinations of training and testing datasets were designed to explore factors influencing prediction performance, including the range of data within training and testing sets, in addition to various signal preprocessing methodologies. These findings suggest the importance of using a training dataset with a broader data range compared with testing datasets for improved model performance alongside consistent signal preprocessing across datasets.
36 MATERIALS SCIENCE↗
Data augmentation for disruption prediction via robust surrogate models
The goal of this work is to generate large statistically representative datasets to train machine learning models for disruption prediction provided by data from few existing discharges. Such a comprehensive training database is important to achieve satisfying and reliable prediction results in artificial neural network classifiers. Here, we aim for a robust augmentation of the training database for multivariate time series data using Student-t process regression. We apply Student-t process regression in a state space formulation via Bayesian filtering to tackle challenges imposed by outliers and noise in the training data set and to reduce the computational complexity. Thus, the method can also be used if the time resolution is high. We use an uncorrelated model for each dimension and impose correlations afterwards via coloring transformations. We demonstrate the efficacy of our approach on plasma diagnostics data of three different disruption classes from the DIII-D tokamak. To evaluate if the distribution of the generated data is similar to the training data, we additionally perform statistical analyses using methods from time series analysis, descriptive statistics, and classic machine learning clustering algorithms.
97 MATHEMATICS AND COMPUTING↗
Using Machine Learning Tools to Predict Compressor Stall
Clean energy has become an increasingly important consideration in today’s power systems. As the push for clean energy continues, many coal-fired power plants are being decommissioned in favor of renewable power sources such as wind and solar. However, the intermittent nature of renewables means that dynamic load following traditional power systems is crucial to grid stability. With high flexibility and fast response at a wide range of operating conditions, gas turbine systems are poised to become the main load following component in the power grid. Yet, rapid changes in load can lead to fluid flow instabilities in gas turbine power systems. These instabilities often lead to compressor surge and stall, which are some of the most critical problems facing the safe and efficient operation of compressors in turbomachinery today. Although the topic of compressor surge and stall has been extensively researched, no methods for early prediction have been proven effective. This study explores the utilization of machine learning tools to predict compressor stall. The long short-term memory (LSTM) model, a form of recurrent neural network (RNN), was trained using real compressor stall datasets from a 100 kW recuperated gas turbine power system designed for hybrid configuration. Two variations of the LSTM model, classification and regression, were tested to determine optimal performance. The regression scheme was determined to be the most accurate approach, and a tool for predicting compressor stall was developed using this configuration. Overall, results show that the tool is capable of predicting stalls 5–20 ms before they occur. With a high-speed controller capable of 5 ms time-steps, mitigating action could be taken to prevent compressor stall before it occurs.
42 ENGINEERING↗
Symbolic regression development of empirical equations for diffusion in Lennard-Jones fluids
Symbolic regression (SR) with a multi-gene genetic program has been used to elucidate new empirical equations describing diffusion in Lennard-Jones (LJ) fluids. Some examples include equations to predict self-diffusion in pure LJ fluids and equations describing the finite-size correction for self-diffusion in binary LJ fluids. The performance of the SR-obtained equations was compared to that of both the existing empirical equations in the literature and to the results from artificial neural net (ANN) models recently reported. It is found that the SR equations have improved predictive performance in comparison to the existing empirical equations, even though employing a smaller number of adjustable parameters, but show an overall reduced performance in comparison to more extensive ANNs.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Discovering a reaction–diffusion model for Alzheimer’s disease by combining PINNs with symbolic regression
Misfolded tau proteins play a critical role in the progression and pathology of Alzheimer's disease. Recent studies suggest that the spatio-temporal pattern of misfolded tau follows a reaction-diffusion type equation. However, the precise mathematical model and parameters that characterize the progression of misfolded protein across the brain remain incompletely understood. Here, we use deep learning and artificial intelligence to discover a mathematical model for the progression of Alzheimer's disease using longitudinal tau positron emission tomography from the Alzheimer's Disease Neuroimaging Initiative database. Specifically, we integrate physics informed neural networks (PINNs) and symbolic regression to discover a reaction-diffusion type partial differential equation for tau protein misfolding and spreading. First, we demonstrate the potential of our model and parameter discovery on synthetic data. Then, we apply our method to discover the best model and parameters to explain tau imaging data from 46 individuals who are likely to develop Alzheimer's disease and 30 healthy controls. Our symbolic regression discovers different misfolding models f(c) for two groups, with a faster misfolding for the Alzheimer's group, f(c) = 0.23c 3 – 1.34c 2 + 1.11c, than for the healthy control group, f(c) = –c 3 + 0.62c 2 + 0.39c. Our results suggest that PINNs, supplemented by symbolic regression, can discover a reaction-diffusion type model to explain misfolded tau protein concentrations in Alzheimer's disease. Furthermore, we expect our study to be the starting point for a more holistic analysis to provide image-based technologies for early diagnosis, and ideally early treatment of neurodegeneration in Alzheimer's disease and possibly other misfolding-protein based neurodegenerative disorders.
60 APPLIED LIFE SCIENCES↗
Analytical gradients for molecular-orbital-based machine learning
We report molecular-orbital-based machine learning (MOB-ML) enables the prediction of accurate correlation energies at the cost of obtaining molecular orbitals. Here, we present the derivation, implementation, and numerical demonstration of MOB-ML analytical nuclear gradients, which are formulated in a general Lagrangian framework to enforce orthogonality, localization, and Brillouin constraints on the molecular orbitals. The MOB-ML gradient framework is general with respect to the regression technique (e.g., Gaussian process regression or neural networks) and the MOB feature design. We show that MOB-ML gradients are highly accurate compared to other ML methods on the ISO17 dataset while only being trained on energies for hundreds of molecules compared to energies and gradients for hundreds of thousands of molecules for the other ML methods. The MOB-ML gradients are also shown to yield accurate optimized structures at a computational cost for the gradient evaluation that is comparable to a density-corrected density functional theory calculation.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Evaluating the Trustworthiness of Explainable Artificial Intelligence (XAI) Methods Applied to Regression Predictions of Arctic Sea Ice Motion
Abstract Recent advances in explainable artificial intelligence (XAI) methods show promise for understanding predictions made by machine learning (ML) models. XAI explains how the input features are relevant or important for the model predictions. We train linear regression (LR) and convolutional neural network (CNN) models to make 1-day predictions of sea ice velocity in the Arctic from inputs of present-day wind velocity and previous-day ice velocity and concentration. We apply XAI methods to the CNN and compare explanations to variance explained by LR. We confirm the feasibility of using a novel XAI method [i.e., global layerwise relevance propagation (LRP)] to understand ML model predictions of sea ice motion by comparing it to established techniques. We investigate a suite of linear, perturbation-based, and propagation-based XAI methods in both local and global forms. Outputs from different explainability methods are generally consistent in showing that wind speed is the input feature with the highest contribution to ML predictions of ice motion, and we discuss inconsistencies in the spatial variability of the explanations. Additionally, we show that the CNN relies on both linear and nonlinear relationships between the inputs and uses nonlocal information to make predictions. LRP shows that wind speed over land is highly relevant for predicting ice motion offshore. This provides a framework to show how knowledge of environmental variables (i.e., wind) on land could be useful for predicting other properties (i.e., sea ice velocity) elsewhere. Significance Statement Explainable artificial intelligence (XAI) is useful for understanding predictions made by machine learning models. Our research establishes trustability in a novel implementation of an explainable AI method known as layerwise relevance propagation for Earth science applications. To do this, we provide a comparative evaluation of a suite of explainable AI methods applied to machine learning models that make 1-day predictions of Arctic sea ice velocity. We use explainable AI outputs to understand how the input features are used by the machine learning to predict ice motion. Additionally, we show that a convolutional neural network uses nonlinear and nonlocal information in making its predictions. We take advantage of the nonlocality to investigate the extent to which knowledge of wind on land is useful for predicting sea ice velocity elsewhere.
Hoffman, Lauren [Scripps Institution of Oceanograp↗
Machine learning to accelerate screening for Marcus reorganization energies
Understanding and predicting the charge transport properties of π-conjugated materials is an important challenge for designing new organic electronic devices, such as solar cells, plastic transistors, light-emitting devices, and chemical sensors. A key component of the hopping mechanism of charge transfer in these materials is the Marcus reorganization energy which serves as an activation barrier to hole or electron transfer. While modern density functional methods have proven to accurately predict trends in intramolecular reorganization energy, such calculations are computationally expensive. In this work, we outline active machine learning methods to predict computed intramolecular reorganization energies of a wide range of polythiophenes and their use toward screening new compounds with low internal reorganization energies. Our models have an overall root mean square error (RMSE) of ±0.113 eV, but a much smaller RMSE of only ±0.036 eV on the new screening set. Since the larger error derives from high-reorganization energy compounds, the new method is highly effective to screen for compounds with potentially efficient charge transport parameters.
14 SOLAR ENERGY↗
What drives the scatter of local star-forming galaxies in the BPT diagrams? A Machine Learning based analysis
ABSTRACT We investigate which physical properties are most predictive of the position of local star forming galaxies on the BPT diagrams, by means of different Machine Learning (ML) algorithms. Exploiting the large statistics from the Sloan Digital Sky Survey (SDSS), we define a framework in which the deviation of star-forming galaxies from their median sequence can be described in terms of the relative variations in a variety of observational parameters. We train artificial neural networks (ANN) and random forest (RF) trees to predict whether galaxies are offset above or below the sequence (via classification), and to estimate the exact magnitude of the offset itself (via regression). We find, with high significance, that parameters primarily associated to variations in the nitrogen-over-oxygen abundance ratio (N/O) are the most predictive for the [N ii]-BPT diagram, whereas properties related to star formation (like variations in SFR or EW(H α)) perform better in the [S ii]-BPT diagram. We interpret the former as a reflection of the N/O–O/H relationship for local galaxies, while the latter as primarily tracing the variation in the effective size of the S+ emitting region, which directly impacts the [S ii] emission lines. This analysis paves the way to assess to what extent the physics shaping local BPT diagrams is also responsible for the offsets seen in high redshift galaxies or, instead, whether a different framework or even different mechanisms need to be invoked.
79 ASTRONOMY AND ASTROPHYSICS↗
Intelligent Prediction of States in Multi-port Autonomous Reconfigurable Solar power plant (MARS)
In power electronics, prediction of states may be used for identification of faults, determination of aging of components, identification of bad data measurements, among others. Prediction of states in power electronics have broadly been based on: (a) physics-based models, (b) data-driven models, and (c) hybrid models. In this paper, data-driven approaches are presented for intelligent prediction of states in multi-port autonomous reconfigurable solar power plant (MARS) and compared. The data-set needed to train the data-driven models based on artificial intelligence (AI) algorithms has been identified and the trained models are evaluated under different extrapolated normal and abnormal operating conditions. The AI algorithms include nonlinear auto-regressive exogenous model (NARX), spiking neural networks (SNN), and decision tree. The models are compared and contrasted. The best model (NARX) is evaluated under different normal and abnormal operating conditions that have indicated accurate prediction.
Debnath, Suman↗
Materials representation and transfer learning for multi-property prediction
The adoption of machine learning in materials science has rapidly transformed materials property prediction. Hurdles limiting full capitalization of recent advancements in machine learning include the limited development of methods to learn the underlying interactions of multiple elements as well as the relationships among multiple properties to facilitate property prediction in new composition spaces. To address these issues, we introduce the Hierarchical Correlation Learning for Multi-property Prediction (H-CLMP) framework that seamlessly integrates: (i) prediction using only a material's composition, (ii) learning and exploitation of correlations among target properties in multi-target regression, and (iii) leveraging training data from tangential domains via generative transfer learning. The model is demonstrated for prediction of spectral optical absorption of complex metal oxides spanning 69 three-cation metal oxide composition spaces. H-CLMP accurately predicts non-linear composition-property relationships in composition spaces for which no training data are available, which broadens the purview of machine learning to the discovery of materials with exceptional properties. This achievement results from the principled integration of latent embedding learning, property correlation learning, generative transfer learning, and attention models. The best performance is obtained using H-CLMP with transfer learning [H-CLMP(T)] wherein a generative adversarial network is trained on computational density of states data and deployed in the target domain to augment prediction of optical absorption from composition. H-CLMP(T) aggregates multiple knowledge sources with a framework that is well suited for multi-target regression across the physical sciences.
36 MATERIALS SCIENCE↗
Predicting Nugget Size of Resistance Spot Welds Using Infrared Thermal Videos With Image Segmentation and Convolutional Neural Network
Resistance spot welding (RSW) is a widely adopted joining technique in automotive industry. Recent advancement in sensing technology makes it possible to collect thermal videos of the weld nugget during RSW using an infrared (IR) camera. The effective and timely analysis of such thermal videos has the potential of enabling in situ nondestructive evaluation (NDE) of the weld nugget by predicting nugget thickness and diameter. Deep learning (DL) has demonstrated to be effective in analyzing imaging data in many applications. However, the thermal videos in RSW present unique data-level challenges that compromise the effectiveness of most pre-trained DL models. We propose a novel image segmentation method for handling the RSW thermal videos to improve the prediction performance of DL models in RSW. The proposed method transforms raw thermal videos into spatial-temporal instances in four steps: video-wise normalization, removal of uninformative images, watershed segmentation, and spatial-temporal instance construction. The extracted spatial-temporal instances serve as the input data for training a DL-based NDE model. The proposed method is able to extract high-quality data with spatial-temporal correlations in the thermal videos, while being robust to the impact of unknown surface emissivity. Overall, our case studies demonstrate that the proposed method achieves better prediction of nugget thickness and diameter than predicting without the transformation.
42 ENGINEERING↗
Artificial neural networks estimate evapotranspiration for Miscanthus × giganteus as effectively as empirical model but with fewer inputs
Estimating actual evapotranspiration (ET) is particularly crucial for addressing how vegetation affects the water balance of ecosystems. ET estimation can be complex with empirical models due to their many parameters and reliance on aridity. In contrast, artificial neural networks (ANNs) could potentially estimate ET with fewer and more common meteorological parameters. In this study, we trained two ANNs, one using a feed-forward approach (FFN) and the other a nonlinear auto-regressive network (NARX), to predict ET and compared them to the commonly used empirical model Granger and Gray (GG). We trained our models on a nine-year eddy covariance (EC) dataset for Miscanthu s × giganteus ( M . × giganteus ) from Illinois (UIEF), then tested them using out-of-sample data from both UIEF and a different location in Iowa (SABR) to compare the accuracy of FFN, NARX, and GG models in estimating daily ET. A combination of air temperature (T a ) and solar radiation (R s ) was chosen as inputs due to the highest R 2 for FFN (R 2 = 0.79, 0.81, and 0.79 for training, testing, and validation, respectively) and only T a for NARX (R 2 = 0.70 for out-of-sample validation). The predictive power of the FFN model was superior to the NARX and GG models at the UIEF site (R 2 = 0.84, 0.70, and 0.83 for out-of-sample validation, respectively). Our analysis showed that ANN approaches are as accurate as empirical approaches for estimating ET but use fewer inputs.
54 ENVIRONMENTAL SCIENCES↗
Computational optimal transport for molecular spectra: The fully discrete case
In this study, the use of computational optimal transport is investigated as a tool for comparing two molecular spectra. Unlike other techniques for comparing molecular spectra in a pattern-recognition framework, transport distances simultaneously encode information about line positions and intensities. In addition, it is shown that transport distances are a useful alternative to Euclidean distances as Euclidean distances are based on line-by-line comparisons, while transport distances reflect broader features of molecular spectra and adequately compare spectra with different resolutions. This paper includes a tutorial on the use of optimal transport and investigates several well-chosen examples to illustrate the utility of computational optimal transport for comparing molecular spectra.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Explaining drivers of housing prices with nonlinear hedonic regressions
Housing markets play a critical role in shaping the spatial and demographic evolution of urban areas. Simulating housing price dynamics can enhance projections of future urban development outcomes. However, traditional hedonic regressions for housing prices, which neglect nonlinear interactions among explanatory variables, often exhibit limited predictive performance. While machine learning (ML) methods can provide a more flexible representation of the relationships between predictors, they are often regarded as “black boxes” due to their complexity and lack of transparency. Interpretable ML techniques provide a promising route by combining the flexibility of ML methods with approaches to analyze the relationships between inputs and outputs. In this study, we employ interpretable ML to analyze the patterns driving the housing market in Baltimore, Maryland, USA. We train an Artificial Neural Network (ANN) to predict Baltimore housing prices based on structural characteristics (e.g., home size, number of stories) and locational attributes (e.g., distance to the city center). We then conduct sensitivity and Partial Dependence Plot (PDP) analyses to interpret the fitted ANN model. We find that the ML model achieves higher predictive accuracy and explains 16 % more of housing price variance than a traditional linear regression model. The interpretable ML model also reveals more nuanced and realistic nonlinear relationships between housing sales price and predictors as well as interactive effects underlying Baltimore home price dynamics. For instance, while the linear model indicates a steady housing price increase over time, our interpretable ML model detects a post-2008 decline, with smaller properties experiencing the sharpest drop.
97 MATHEMATICS AND COMPUTING↗
Artificial to Spiking Neural Networks Conversion with Calibration in Scientific Machine Learning
Here, we introduce a method to convert physics-informed neural networks (PINNs), commonly used in scientific machine learning, to spiking neural networks (SNNs), which are expected to have higher energy efficiency compared to traditional artificial neural networks (ANNs). We first extend the calibration technique of SNNs to arbitrary activation functions beyond ReLU, making it more versatile, and we prove a theorem that ensures the effectiveness of the calibration. We successfully convert PINNs to SNNs, enabling computational efficiency for diverse regression tasks in solving multiple differential equations, including the unsteady Navier–Stokes equations. We demonstrate great gains in terms of overall efficiency, including separable PINNs (SPINNs), which accelerate the training process. Overall, this is the first work of this kind and the proposed method achieves relatively good accuracy with low spike rates.
PINN↗