Imbalanced Regression Artificial Neural Network Model for Auroral Electrojet Indices (IRANNA): Can We Predict Strong Events?
Explore the source record for details and available documents.
Xiangning Chu↗
SEARCH · Search NASA
Results for “artificial neural network regression”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Predicting nepheline precipitation in waste glasses using ternary submixture model and machine learning
Nepheline precipitation in nuclear waste glasses during vitrification can be detrimental due to its negative effect on chemical durability. Developing models to accurately predict nepheline precipitation from compositions is important to increase waste loading since existing models can be overly conservative. In this study, an expanded dataset containing 955 glasses was compiled from literature data, where 355 glasses are for high-level waste (HLW). Previously developed submixture models were refitted using the new dataset, where a misclassification rate of 7.8% was achieved. Nine machine learning (ML) algorithms (e.g., k-nearest neighbor, Gaussian process regression, artificial neural network, support vector machine, decision tree, etc.) were applied to evaluate their ability of predicting nepheline precipitation from compositions. Model accuracy, precision, recall/sensitivity, and F1 score were systemically compared between different ML algorithms and modeling protocols. Good model prediction with an accuracy ~0.9 (misclassification rate of ~10%) was observed with different algorithms under certain protocol. This study evaluated various ML models to predict nepheline precipitations in waste glasses, highlighting the importance of data preparation, modeling protocol, and their effect on model stability and reproducibility. The results provide insights into applying ML to predict glass properties and suggest areas for future research on modeling nepheline precipitations.
Lu, Xiaonan↗
Development of a Filtered CFD-DEM Drag Model with Multiscale Markers Using an Artificial Neural Network and Nonlinear Regression
Here, the accuracy of coarse-grained Euler-Lagrangian simulations of fluidized beds heavily depends on the mesoscale drag models to account for the influences of the unresolved sub-grid structures. Traditional filtered drag models are regressed with mesoscale markers such as voidage and slip velocities. In this research, a filtered drag was regressed with both mesoscale and macro-scale markers using fine grid Computational Fluid Dynamics - Discrete Element Method (CFD-DEM) simulations. The traditional non-linear regression method was compared with machine learning regression using an Artificial Neural Network (ANN) implemented in PyTorch and coupled with MFiX. The new drag showed higher accuracy than the Wen-Yu drag and another filtered drag derived from the two-fluid model. The nonlinear regression shows slightly better results than ANN regression in cases with similar R 2 values. The utilization of the gas inlet velocity as an additional macro-scale marker reduced the errors by up to 55.3% in the tested cases.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Machine Learning-Assisted High-Temperature Reservoir Thermal Energy Storage Optimization: Numerical Modeling and Machine Learning Input and Output Files
This data set includes the numerical modeling input files and output files used to synthesize data, and the reduced-order machine learning models trained from the synthesized data for reservoir thermal energy storage site identification. In this study, a machine-learning-assisted computational framework is presented to identify High-Temperature Reservoir Thermal Energy Storage (HT-RTES) site with optimal performance metrics by combining physics-based simulation with stochastic hydrogeologic formation and thermal energy storage operation parameters, artificial neural network regression of the simulation data, and genetic algorithm-enabled multi-objective optimization. A doublet well configuration with a layered (aquitard-aquifer-aquitard) generic reservoir is simulated for cases of continuous operation and seasonal-cycle operation scenarios. Neural network-based surrogate models are developed for the two scenarios and applied to generate the Pareto fronts of the HT-RTES performance for four potential HT-RTES sites. The developed Pareto optimal solutions indicate the performance of HT-RTES is operation-scenario (i.e., fluid cycle) and reservoir-site dependent, and the performance metrics have competing effects for a given site and a given fluid cycle. The developed neural network models can be applied to identify suitable sites for HT-RTES, and the proposed framework sheds light on the design of resilient HT-RTES systems. All the simulations and the neural network model were done by Idaho National Laboratory. A detailed description of the work was reported in publication linked below.
15 GEOTHERMAL ENERGY↗
Advancing Multiscale Simulation of Plasma-Surface Interfaces
We report the development of an atomistic-informed, surface-state-dependent predictive model for particle exchange in a carbon-tungsten plasma-surface interface. The predictive model uses machine learning (ML) techniques to learn the energy and angular distributions for particle exchange and rate functions for surface state evolution from molecular dynamics simulations of cumulative bombardment of tungsten by energetic carbon ions. Each predictive component is sensitive to the energy and trajectory of incident plasma species and the surface state. The surface state is represented by a set of surface state descriptors, which were derived from the atomistic surface state for each independent carbon bombardment event. These descriptors are representative of the composition and degree of amorphization of the outermost angstrom of surface material and were chosen to optimize predictive performance for particle exchange at the interface. The distributions for particle exchange (reflection/sputtering) are demonstrated to vary with each surface state descriptor, motivating the development of surface-state-dependent particle exchange models for plasma simulations. The performance of various ML methods was compared, including polynomial quantile regression, artificial neural networks, k-nearest neighbors, and random forest algorithms, with polynomial regression performing the best for interpolation and extrapolation of learned relationships. In addition to the particle exchange model, a neutral network was developed and used to identify data sufficiency throughout surface descriptor space, which will enable real-time feedback during future data production to ensure data is produced where it is most needed, and we provide commentary on improvements to the data production workflow for future endeavors.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Gap-filling eddy covariance methane fluxes: Comparison of machine learning model predictions and uncertainties at FLUXNET-CH4 wetlands
Time series of methane fluxes measured by eddy-covariance require gap-filling to estimate annual emissions. Gap-filling methane fluxes is challenging because of high variability and complex responses to multiple drivers. To date, there is no widely established gap-filling standard for methane, with regards both to the best model algorithms and predictors. In this study, we address the need for standardization by synthesizing results of gap-filling methods applied at 17 wetland sites spanning boreal to tropical regions including all major wetlands classes and two rice paddies. We introduce new procedures for: 1) creating realistic artificial gap scenarios, 2) training and evaluating gap-filling models without overstating performance, and 3) predicting half-hourly methane fluxes and annual emissions with robust uncertainty estimates. We tested a conventional method (marginal distribution sampling) and four machine learning algorithms - penalized linear regression, artificial neural networks, random forests, and boosted decision trees - and four predictor sets, including temporal, meteorological, ecosystem carbon and energy flux, and soil predictors. We find that the conventional method can achieve similar median performance to the machine learning models but is worse than the best machine learning models and relatively insensitive to predictor choices. Of the machine learning models, decision tree algorithms performed the best in cross-validation experiments, even with a baseline predictor set, and artificial neural networks showed comparable performance when using all predictors. Soil temperature was frequently the most important predictor whilst water table depth was important at sites with substantial water table fluctuations, highlighting the value of data on soil conditions. Raw gap-filling uncertainties from the machine learning models were underestimated and we propose a method to calibrate uncertainties to observations. Finally, we gap-fill and provide summary evaluation metrics for all 81 sites in the FLUXNET-CH4 community dataset and publicly release the python code for model development, evaluation, and uncertainty estimation.
42 ENGINEERING↗
Machine-learning-assisted high-temperature reservoir thermal energy storage optimization
High-temperature reservoir thermal energy storage (HT-RTES) has the potential to become an indispensable component in achieving the goal of the net-zero carbon economy, given its capability to balance the intermittent nature of renewable energy generation. In this study, a machine-learning-assisted computational framework is presented to co-optimize the performance metrics of HT-RTES by combining physics-based simulation with stochastic hydrogeologic formation and thermal energy storage operation parameters, artificial neural network regression of the simulation data, and genetic algorithm-enabled multi-objective optimization. A doublet well configuration with a layered (aquitard-aquifer-aquitard) generic reservoir is simulated for cases of continuous operation and seasonal-cycle operation scenarios. Further, neural network-based surrogate models are developed for the two scenarios and applied to generate the Pareto fronts of the HT-RTES performance for four potential HT-RTES sites. The developed Pareto optimal solutions indicate the performance of HT-RTES is operation-scenario (i.e., fluid cycle) and reservoir-site dependent, and the performance metrics have competing effects for a given site and a given fluid cycle. The developed neural network models can be applied to identify suitable sites for HT-RTES, and the proposed framework sheds light on the design of resilient HT-RTES systems.
15 GEOTHERMAL ENERGY↗
Predicting battery capacity from impedance at varying temperature and state of charge using machine learning
Prediction of battery health from electrochemical impedance spectroscopy (EIS) data can enable rapid measurement of battery state in real-world applications without using additional sensors or time-consuming performance measurements. However, deconvoluting the effect of capacity, state of charge, and temperature on EIS response is complicated analytically. Here, various machine-learning models, such as linear, Gaussian process, random forest, and artificial neural network regression, are utilized to predict capacity from EIS using hundreds of capacity, direct current (DC) resistance, and EIS measurements recorded under varying conditions of health, temperature, and state of charge (SOC). Several feature extraction and selection methods from traditional electrochemical analysis and statistical modeling are explored using machine-learning pipelines. EIS data from just two frequencies can accurately predict capacity, and interrogation shows that the optimal set of frequencies is not usually intuitive. Best results are achieved with an ensemble model, which predicts battery capacity with a mean absolute error of 1.9% on data from unobserved cells.
25 ENERGY STORAGE↗
Evaluating Algorithm Performance Metrics Tailored for Prognostics
Prognostics has taken a center stage in Condition Based Maintenance (CBM) where it is desired to estimate Remaining Useful Life (RUL) of the system so that remedial measures may be taken in advance to avoid catastrophic events or unwanted downtimes. Validation of such predictions is an important but difficult proposition and a lack of appropriate evaluation methods renders prognostics meaningless. Evaluation methods currently used in the research community are not standardized and in many cases do not sufficiently assess key performance aspects expected out of a prognostics algorithm. In this paper we introduce several new evaluation metrics tailored for prognostics and show that they can effectively evaluate various algorithms as compared to other conventional metrics. Specifically four algorithms namely; Relevance Vector Machine (RVM), Gaussian Process Regression (GPR), Artificial Neural Network (ANN), and Polynomial Regression (PR) are compared. These algorithms vary in complexity and their ability to manage uncertainty around predicted estimates. Results show that the new metrics rank these algorithms in different manner and depending on the requirements and constraints suitable metrics may be chosen. Beyond these results, these metrics offer ideas about how metrics suitable to prognostics may be designed so that the evaluation procedure can be standardized. 1
Saxena, Abhinav↗
Machine-Learning-Driven, Site-Specific Weather Forecasting for Grid-Interactive Efficient Buildings: Preprint
Emerging grid-interactive efficient buildings (GEBs) have great potential to provide much-needed demand flexibility to electric grids while fulfilling their own control targets by co-optimizing smart appliances, solar photovoltaics, electric vehicles, and energy storage at buildings. To enable the optimal operation of GEBs, site-specific weather information—such as temperature, solar irradiance, relative humidity, and wind speed—is crucial; however, this information is generally unavailable or expensive to obtain. This paper develops advanced machine learning methods to provide precise weather forecasts for individual building sites using readily available weather station data. Support vector regression and artificial neural networks have been employed to learn the spatiotemporal correlations between the weather conditions at nearby weather stations and the individual building site. The proposed site-specific weather forecasting methods have been validated using 1-year actual weather measurement data collected in the Denver metro area. Results show that the developed machine-learning-driven methods can accurately forecast the temperature at the target building site 1 hour ahead with mean absolute error less than 0.72°C and a 48% improvement over the persistence method. Site-specific weather forecasts will improve the understanding of the microclimate effect and its impact on building energy consumption. This information will drive efficiency upgrades and adjustments of building control strategies to improve energy savings and increase flexibility in building loads.
30 DIRECT ENERGY CONVERSION↗
Experiments to Determine Whether Recursive Partitioning (CART) or an Artificial Neural Network Overcomes Theoretical Limitations of Cox Proportional Hazards Regression
New computationally intensive tools for medical survival analyses include recursive partitioning (also called CART) and artificial neural networks. A challenge that remains is to better understand the behavior of these techniques in effort to know when they will be effective tools. Theoretically they may overcome limitations of the traditional multivariable survival technique, the Cox proportional hazards regression model. Experiments were designed to test whether the new tools would, in practice, overcome these limitations. Two datasets in which theory suggests CART and the neural network should outperform the Cox model were selected. The first was a published leukemia dataset manipulated to have a strong interaction that CART should detect. The second was a published cirrhosis dataset with pronounced nonlinear effects that a neural network should fit. Repeated sampling of 50 training and testing subsets was applied to each technique. The concordance index C was calculated as a measure of predictive accuracy by each technique on the testing dataset. In the interaction dataset, CART outperformed Cox (P less than 0.05) with a C improvement of 0.1 (95% Cl, 0.08 to 0.12). In the nonlinear dataset, the neural network outperformed the Cox model (P less than 0.05), but by a very slight amount (0.015). As predicted by theory, CART and the neural network were able to overcome limitations of the Cox model. Experiments like these are important to increase our understanding of when one of these new techniques will outperform the standard Cox model. Further research is necessary to predict which technique will do best a priori and to assess the magnitude of superiority.
Kattan, Michael W.↗
Development of advanced machine learning models for analysis of plutonium surrogate optical emission spectra
This work investigates and applies machine learning paradigms seldom seen in analytical spectroscopy for quantification of gallium in cerium matrices via processing of laser-plasma spectra. Ensemble regressions, support vector machine regressions, Gaussian kernel regressions, and artificial neural network techniques are trained and tested on cerium-gallium pellet spectra. A thorough hyperparameter optimization experiment is conducted initially to determine the best design features for each model. The optimized models are evaluated for sensitivity and precision using the limit of detection (LoD) and root mean-squared error of prediction (RMSEP) metrics, respectively. Gaussian kernel regression yields the superlative predictive model with an RMSEP of 0.33% and an LoD of 0.015% for quantification of Ga in a Ce matrix. This study concludes that these machine learning methods could yield robust prediction models for rapid quality control analysis of plutonium alloys.
Rao, Ashwin P. (ORCID:0000000319312568)↗
Local primordial non-Gaussianity from the large-scale clustering of photometric DESI luminous red galaxies
ABSTRACT We use angular clustering of luminous red galaxies from the Dark Energy Spectroscopic Instrument (DESI) imaging surveys to constrain the local primordial non-Gaussianity parameter fNL. Our sample comprises over 12 million targets, covering 14 000 deg2 of the sky, with redshifts in the range 0.2 < z < 1.35. We identify Galactic extinction, survey depth, and astronomical seeing as the primary sources of systematic error, and employ linear regression and artificial neural networks to alleviate non-cosmological excess clustering on large scales. Our methods are tested against simulations with and without fNL and systematics, showing superior performance of the neural network treatment. The neural network with a set of nine imaging property maps passes our systematic null test criteria, and is chosen as the fiducial treatment. Assuming the universality relation, we find $f_{\rm NL} = 34^{+24(+50)}_{-44(-73)}$ at 68 per cent (95 per cent) confidence. We apply a series of robustness tests (e.g. cuts on imaging, declination, or scales used) that show consistency in the obtained constraints. We study how the regression method biases the measured angular power spectrum and degrades the fNL constraining power. The use of the nine maps more than doubles the uncertainty compared to using only the three primary maps in the regression. Our results thus motivate the development of more efficient methods that avoid overcorrection, protect large-scale clustering information, and preserve constraining power. Additionally, our results encourage further studies of fNL with DESI spectroscopic samples, where the inclusion of 3D clustering modes should help separate imaging systematics and lessen the degradation in the fNL uncertainty.
79 ASTRONOMY AND ASTROPHYSICS↗
Multiscale CFD simulation of biomass fast pyrolysis with a machine learning derived intra-particle model and detailed pyrolysis kinetics
Coupling particle and reactor scale models is as essential as reactor fluid dynamics and particle motion for accurate Computational Fluid Dynamic (CFD) simulations of biomass fast pyrolysis reactors due to intraparticle heat transfer and chemical reactions controlling conversion time and product distributions. Direct online coupling of a particle model with a reactor model is computationally expensive, while offline coupling is case-dependent. In this research, solutions from a series of particle pyrolysis simulations were regressed with Artificial Neural Network (ANN). Furthermoer, this machine learning-derived model predicted the same temperature and conversion profiles compared with particle resolved simulation while the isothermal approach overpredicted the temperature by 130 K and underpredicted the conversion time by 30 s. The ANN model was then integrated into CFD simulations of fluidized bed biomass fast pyrolysis with varied feedstocks via coupling PyTorch and MFiX. The averaged error of simulation predicted bio-oil yields with four feedstocks is 6.4%. This multi-scale approach provides an efficient tool for the coupled particle and reactor scale simulations of biomass pyrolysis.
09 BIOMASS FUELS↗
Automated and efficient local adaptive regression for principal component-based reduced-order modeling of turbulent reacting flows
Principal Component Analysis can be used to reduce the cost of Computational Fluid Dynamics simulations of turbulent reacting flows by reducing the dimensionality of the transported variables through projection of the thermochemical state onto a lower-dimensional manifold. However, because of the nonlinearity of the principal component source terms, nonlinear regression techniques must be utilized for the source terms in terms of the principal components. Unfortunately, widely available and utilized nonlinear regression techniques can have prohibitive computational requirements and/or accuracy that is highly dependent on user experience in ad hoc tuning of model architecture and hyperparameters. Here, in this work, a new nonlinear regression approach is proposed that is both computationally efficient and automated so does not require any user input. The approach is evaluated through a priori prediction of principal component source terms using data from a Direct Numerical Simulation of a turbulent nonpremixed n-heptane/air jet flame. In particular, the proposed framework consists of local regressions whose complexity is adapted according to the local nonlinearity of the data: local linear regression when accurate enough and local Artificial Neural Networks when nonlinear regression is required. The number of local clusters for local regression is determined automatically using the Davies-Bouldin index. In addition, Bayesian optimization is utilized for model training (i.e., to select the best architectures and hyperparameters of the nonlinear regressions in an unsupervised fashion), eliminating ad hoc hand-tuning and/or expensive grid searches. Overall, compared to a single, global neural network, the new local adaptive regression approach is shown to have comparable accuracy but 69% less training time due to the utilization of local linear regression and faster training of local neural networks.
42 ENGINEERING↗
Rapid quantitative analysis of trace elements in plutonium alloys using a handheld laser-induced breakdown spectroscopy (LIBS) device coupled with chemometrics and machine learning
Here, we present the first reported quantification of trace elements in plutonium via a portable laser-induced breakdown spectroscopy (LIBS) device and demonstrate the use of chemometric analysis to enhance the handheld device's sensitivity and precision. Quantification of trace elements such as iron and nickel in plutonium metal via LIBS is a challenging problem due to the complex nature of the plutonium optical emission spectra. While rapid analysis of plutonium alloys has been demonstrated using portable LIBS devices, such as the SciAps Z300, their detection limits for trace elements are severely constrained by their achievable pulse power and length, light collection optics, and detectors. In this paper, analytical methods are evaluated as a means to circumvent the detection constraints. Three chemometric methods often used in analytical spectroscopy are evaluated; principal component regression, partial least-squares regression, and artificial neural networks. These models are evaluated based on goodness-of-fit metrics, root mean-squared error, and their achievable limits of detection (LoDs). Partial least squares proved superior for determining content of iron and nickel in plutonium metal, yielding LoDs of 15 and 20 ppm, respectively. These results of identifying the undesirable trace elements in plutonium components are critical for applications such as fabricating radioisotope thermoelectric generators or nuclear fuel.
11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
A Comparison of Machine Learning Methods for Frequency Nadir Estimation in Power Systems
An increasing penetration level of inverter-based renewable energy resources changes the inertia of power systems, posing challenges for maintaining the desired system frequency stability. An accurate frequency nadir estimation is crucial for power system operators to prepare preventive actions against large frequency excursions. In this paper, five machine learning methods - linear regression, gradient boosting, support vector regression, an artificial neural network, and XGBoost - are applied to two different datasets, i.e., 1) the unit generation dataset and 2) the system total inertia and headroom dataset, for the prediction of the frequency nadir. The training and testing datasets are generated through extensive generation scheduling simulations using Multi-timescale Integrated Dynamic and Scheduling (MI-DAS) toolbox on the Western Electricity Coordinating Council 240-bus system with high renewable penetration levels. Numerical results show that all five machine learning methods perform well in predicting the nadir frequency of the system. Among them, the gradient boosting and the XGBoost are clear winners yielding the best prediction accuracy in terms of four evaluation metrics.
data driven↗
Gaussian Process Regression for Aggregate Baseline Load Forecasting
Demand response (DR) is one of the most effective ways to maintain the reliability and improve the flexibility of power systems. Accurate forecasts of baseline loads are essential for DR programs. In the era of big data, machine learning-based approaches present a unique opportunity for baseline load forecasting. Thus, this paper presents a machine learning-based approach using a relatively less explored algorithm, Gaussian process regression (GPR), to forecast aggregate baseline loads. As such, a dataset was generated using a set of EnergyPlus simulations. Using the generated dataset, a GPR-based forecasting model was developed. In addition, support vector regression (SVR)-, artificial neural network (ANN)-, and averaging-based models were developed as baseline models for comparison. These models were compared in terms of accuracy, simplicity, and integrity. The prediction performance of the models showed that the GPR-based model is more accurate and reliable than the others. Such high performance shows the potential of the GPR in baseline load forecasting. GPR, therefore, can be used for DR applications.
Amasyali, Kadir↗