Search NASA⌕ Search

SEARCH · Search NASA

Results for “artificial neural network regression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Advanced Cross Section Library Generation using Reduced Order Models

Deterministic neutronics calculations rely on multigroup neutron cross section libraries, which consist of databases of tabulated values, used to calculate the neutron cross sections through multivariate linear interpolation. However, interpolation of the multidimensional cross section data becomes memory inefficient and time consuming as the number of tabulations increases, significantly slowing down the neutronics calculation, especially in the case of microscopic cross section libraries where every isotope (on the order of hundreds) has its own set of specific reactions and cross sections. In order to address this challenge, this work constructs efficient and robust reduced-order models (ROMs) of the multi-group cross sections to support the Griffin simulation of high-temperature gas-cooled reactors (HTGRs). The first part of the study investigates the linearity of the multigroup cross section data across isotopes, reaction types, and energy groups on pre-generated datasets for the purpose of dimensionality reduction. Secondly, a down-selection of ROM techniques is presented on representative classical machine learning (ML) techniques, including variants of linear regression, kernel-based methods, tree-based algorithms, and artificial neural networks. The selection criteria jointly consider the memory efficiency, predictive accuracy, prediction speed, and scalability in comparison to the multidimensional interpolation. Among all the ML techniques, deep neural networks (DNNs) have proven to be the best selection with sufficient accuracy, high robustness, good memory efficiency, great scalability, and superior flexibility. DNNs have been trained for all isotopes in this work and systematic Griffin testing is ongoing to ensure the feasibility of this ROM technique for predicting cross section and reducing memory requirements without a significant sacrifice in computational performance.

42 - ENGINEERING↗
DOE OSTI2025

Unifying simulation and inference with normalizing flows

There have been many applications of deep neural networks to detector calibrations and a growing number of studies that propose deep generative models as automated fast detector simulators. We show that these two tasks can be unified by using maximum likelihood estimation (MLE) from conditional generative models for energy regression. Unlike direct regression techniques, the MLE approach is prior independent and non-Gaussian resolutions can be determined from the shape of the likelihood near the maximum. Using an ATLAS-like calorimeter simulation, we demonstrate this concept in the context of calorimeter energy calibration. Published by the American Physical Society 2025

Hadronic calorimiters↗
DOE OSTI2024

Machine learning without a processor: Emergent learning in a nonlinear analog network

Standard deep learning algorithms require differentiating large nonlinear networks, a process that is slow and power-hungry. Electronic contrastive local learning networks (CLLNs) offer potentially fast, efficient, and fault-tolerant hardware for analog machine learning, but existing implementations are linear, severely limiting their capabilities. These systems differ significantly from artificial neural networks as well as the brain, so the feasibility and utility of incorporating nonlinear elements have not been explored. Here, we introduce a nonlinear CLLN—an analog electronic network made of self-adjusting nonlinear resistive elements based on transistors. We demonstrate that the system learns tasks unachievable in linear systems, including XOR (exclusive or) and nonlinear regression, without a computer. We find our decentralized system reduces modes of training error in order (mean, slope, curvature), similar to spectral bias in artificial neural networks. The circuitry is robust to damage, retrainable in seconds, and performs learned tasks in microseconds while dissipating only picojoules of energy across each transistor. This suggests enormous potential for fast, low-power computing in edge systems like sensors, robotic controllers, and medical devices, as well as manufacturability at scale for performing and studying emergent learning.

Science & Technology - Other Topics↗
DOE OSTI2022

Using soil library hyperspectral reflectance and machine learning to predict soil organic carbon: Assessing potential of airborne and spaceborne optical soil sensing

Soil organic carbon (SOC) is a key variable to determine soil functioning, ecosystem services, and global carbon cycles. Spectroscopy, particularly optical hyperspectral reflectance coupled with machine learning, can provide rapid, efficient, and cost-effective quantification of SOC. However, how to exploit soil hyperspectral reflectance to predict SOC concentration, and the potential performance of airborne and satellite data for predicting surface SOC at large scales remain relatively underknown. Here, this study utilized a continental-scale soil laboratory spectral library (37,540 full-pedon 350–2500 nm reflectance spectra with SOC concentration of 0–780 g·kg –1 across the US) to thoroughly evaluate seven machine learning algorithms including Partial-Least Squares Regression (PLSR), Random Forest (RF), K-Nearest Neighbors (KNN), Ridge, Artificial Neural Networks (ANN), Convolutional Neural Networks (CNN), and Long Short-Term Memory (LSTM) along with four preprocessed spectra, i.e. original, vector normalization, continuum removal, and first-order derivative, to quantify SOC concentration. Furthermore, by using the coupled soil-vegetation-atmosphere radiative transfer model, we simulated twelve airborne and spaceborne hyper/multi-spectral remote sensing data from surface bare soil laboratory spectra to evaluate their potential for estimating SOC concentration of surface bare soils. Results show that LSTM achieved best predictive performance of quantifying SOC concentration for the whole data sets (R 2 = 0.96, RMSE = 30.81 g·kg –1 ), mineral soils (SOC ≤ 120 g·kg –1 , R 2 = 0.71, RMSE = 10.60 g·kg –1 ), and organic soils (SOC > 120 g·kg –1 , R 2 = 0.78, RMSE = 62.31 g·kg –1 ). Spectral data preprocessing, particularly the first-order derivative, improved the performance of PLSR, RF, Ridge, KNN, and ANN, but not LSTM or CNN. We found that the SOC models of mineral and organic soils should be distinguished given their distinct spectral signatures. Finally, we identified that the shortwave infrared is vital for airborne and spaceborne hyperspectral sensors to monitor surface SOC. This study highlights the high accuracy of LSTM with hyperspectral/multispectral data to mitigate a certain level of noise (soil moisture <0.4 m 3 ·m –3 , green leaf area < 0.3 m 2 ·m –2 , plant residue <0.4 m 2 ·m –2 ) for quantifying surface SOC concentration. Forthcoming satellite hyperspectral missions like Surface Biology and Geology (SBG) have a high potential for future global soil carbon monitoring, while high-resolution satellite multispectral fusion data can be an alternative.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

IoT Intrusion Detection Taxonomy, Reference Architecture, and Analyses

This paper surveys the deep learning (DL) approaches for intrusion-detection systems (IDSs) in Internet of Things (IoT) and the associated datasets toward identifying gaps, weaknesses, and a neutral reference architecture. A comparative study of IDSs is provided, with a review of anomaly-based IDSs on DL approaches, which include supervised, unsupervised, and hybrid methods. All techniques in these three categories have essentially been used in IoT environments. To date, only a few have been used in the anomaly-based IDS for IoT. For each of these anomaly-based IDSs, the implementation of the four categories of feature(s) extraction, classification, prediction, and regression were evaluated. We studied important performance metrics and benchmark detection rates, including the requisite efficiency of the various methods. Four machine learning algorithms were evaluated for classification purposes: Logistic Regression (LR), Support Vector Machine (SVM), Decision Tree (DT), and an Artificial Neural Network (ANN). Therefore, we compared each via the Receiver Operating Characteristic (ROC) curve. The study model exhibits promising outcomes for all classes of attacks. The scope of our analysis examines attacks targeting the IoT ecosystem using empirically based, simulation-generated datasets (namely the Bot-IoT and the IoTID20 datasets).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Do Machine Learning Approaches Offer Skill Improvement for Short-Term Forecasting of Wind Gust Occurrence and Magnitude?

Abstract Wind gusts, and in particular intense gusts, are societally relevant but extremely challenging to forecast. This study systematically assesses the skill enhancement that can be achieved using artificial neural networks (ANNs) for forecasting of wind gust occurrence and magnitude. Geophysical predictors from the ERA5 reanalysis are used in conjunction with an autoregressive term in regression and ANN models with different predictors, and varying model complexity. Models are derived and assessed for the warm (April–September) and cold (October–March) seasons for three high passenger volume airports in the United States. Model uncertainty is assessed by deriving models for 1000 different randomly selected training (70%) and testing (30%) subsets. Gust prediction fidelity in independent test samples is critically dependent on inclusion of an autoregressive term. Gust occurrence probabilities derived using five-layer ANNs exhibit consistently higher fidelity than those from regression models and shallower ANNs. Inclusion of the autoregressive term and increasing the number of hidden layers in ANNs from 1 to 5 also improve the model performance for gust magnitudes (lower RMSE, increased correlation, and model standard deviations that more closely approximate observed values). Deeper ANNs (e.g., 20 hidden layers) exhibit higher skill in forecasting strong (17–25.7 m s −1 ) and damaging (≥25.7 m s −1 ) wind gusts. However, such deep networks exhibit evidence of overfitting and still substantially underestimate (by 50%) the frequency of strong and damaging wind gusts at the three airports considered herein. Significance Statement Improved short-term forecasting of wind gusts will enhance aviation safety and logistics and may offer other societal benefits. Here we present a rigorous investigation of the relative skill of models of wind gust occurrence and magnitude that employ different statistical methods. It is shown that artificial neural networks (ANNs) offer considerable skill enhancement over regression methods, particularly for strong and damaging wind gusts. For wind gust magnitudes in particular, application of deeper learning networks (e.g., five or more hidden layers) offers tangible improvements in forecast accuracy. However, deeper networks are vulnerable to overfitting and exhibit substantial variability with the specific training and testing data subset used. Also, even deep ANNs reproduce only half of strong and damaging wind gusts. These results indicate the need for future work to elucidate the dynamical mechanisms of intense wind gusts and advance solutions to their prediction.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

A comparison of computational models for predicting yield sooting index

Sooting propensity, a measurement of how much particulate matter is produced when a fuel is burned, is a property of significant interest among researchers who are striving to discover the next generation of cleaner, more efficient fuels and fuel additives. Many compounds are not viable as fuels and/or fuel additives, and as a result, designing cleaner-burning biofuels using only experimental techniques is inefficient. Predictive models have been instrumental in reducing this inherent difficulty, providing researchers with a tool to preemptively screen compounds before production and testing. The present work compares the accuracies and interpretabilities of existing models used to predict a particular measure of sooting propensity, Yield Sooting Index (YSI). These models include artificial neural networks, graph neural networks, and multivariate equations. A novel equation for predicting YSI based on atom path count and bond order is proposed, which can highlight key structural components that contribute to YSI. It was found that artificial neural networks slightly outperform graph neural networks and greatly outperform multivariate equations in blind (test set) prediction accuracy; however, graph neural networks and multivariate equations provide significantly more interpretability as to how compound structure relates to YSI. Predictions of YSI are compared to experimental measurements for previously un-tested compounds with cetane numbers comparable to diesel fuel (50-60) (butyl decanoate, ethyl decanoate, 1,4-bis(ethenoxymethyl)cyclohexane, and 5-heptyloxolan-2-one), and it was found that these compounds produce significantly less soot compared to diesel fuel.

09 BIOMASS FUELS↗
DOE OSTI2022

Automatic Waveform Quality Control for Surface Waves Using Machine Learning

Surface-wave seismograms are widely used by researchers to study Earth’s interior and earthquakes. To extract information reliably and robustly from a suite of surface waveforms, the signals require quality control screening to reduce artifacts from signal complexity and noise. This process has usually been completed by human experts labeling each waveform visually, which is time consuming and tedious for large data sets. We explore automated approaches to improve the efficiency of waveform quality control processing by investigating logistic regression, support vector machines, K-nearest neighbors, random forests (RF), and artificial neural networks (ANN) algorithms. To speed up signal quality assessment, we trained these five machine learning (ML) methods using nearly 400,000 human-labeled waveforms. The ANN and RF models outperformed other algorithms and achieved a test accuracy of 92%. We evaluated these two best-performing models using seismic events from geographic regions not used for training. The results show that the two trained models agree with labels from human analysts but required only 0.4% of the time. Although the original (human) quality assignments assessed general waveform signal-to-noise, the ANN or RF labels can help facilitate detailed waveform analysis. Our investigations demonstrate the capability of the automated processing using these two ML models to reduce outliers in surface-wave-related measurements without human quality control screening.

58 GEOSCIENCES↗
DOE OSTI2023

Predicting Biomass Yields of Advanced Switchgrass Cultivars for Bioenergy and Ecosystem Services Using Machine Learning

The production of advanced perennial bioenergy crops within marginal areas of the agricultural landscape is gaining interest due to its potential to sustainably produce feedstocks for biofuels and bioproducts while also improving the sustainability and resilience of commodity crop production. However, predicting the biomass yields of this production system is challenging because marginal areas are often relatively small and spread around agricultural fields and are typically associated with various abiotic conditions that limit crop production. Machine learning (ML) offers a viable solution as a biomass yield prediction tool because it is suited to predicting relationships with complex functional associations. The objectives of this study were to (1) evaluate the accuracy of commonly applied ML algorithms in agricultural applications for predicting the biomass yields of advanced switchgrass cultivars for bioenergy and ecosystem services and (2) determine the most important biomass yield predictors. Datasets on biomass yield, weather, land marginality, soil properties, and agronomic management were generated from three field study sites in two U.S. Midwest states (Illinois and Iowa) over three growing seasons. The ML algorithms evaluated in the study included random forests (RFs), gradient boosting machines (GBMs), artificial neural networks (ANNs), K-neighbors regressor (KNR), AdaBoost regressor (ABR), and partial least squares regression (PLSR). Coefficient of determination (R 2 ) and mean absolute error (MAE) were used to evaluate the predictive accuracy of the tested algorithms. Results showed that the ensemble methods, RF (R 2 = 0.86, MAE = 0.62 Mg/ha), GBM (R 2 = 0.88, MAE = 0.57 Mg/ha), and GBM (R 2 = 0.78, MAE = 0.66 Mg/ha), were the most accurate in predicting biomass yields of the Independence, Liberty, and Shawnee switchgrass cultivars, respectively. This is in agreement with similar studies that apply ML to multi-feature problems where traditional statistical methods are less applicable and datasets used were considered to be relatively small for ANNs. Consistent with previous studies on switchgrass, the most important predictors of biomass yield included average annual temperature, average growing season temperature, sum of the growing season precipitation, field slope, and elevation. This study helps pave the way for applying ML as a management tool for alternative bioenergy landscapes where understanding agronomic and environmental performance of a multifunctional cropping system seasonally and interannually at the sub-field scale is critical.

09 BIOMASS FUELS↗
DOE OSTI2023

When less is more: How increasing the complexity of machine learning strategies for geothermal energy assessments may not lead toward better estimates

Previous moderate- and high-temperature geothermal resource assessments of the western United States utilized data-driven methods and expert decisions to estimate resource favorability. Although expert decisions can add confidence to the modeling process by ensuring reasonable models are employed, expert decisions also introduce human and, thereby, model bias. This bias can present a source of error that reduces the predictive performance of the models and confidence in the resulting resource estimates. Our study aims to develop robust data-driven methods with the goals of reducing bias and improving predictive ability. We present and compare nine favorability maps for geothermal resources in the western United States using data from the U.S. Geological Survey's 2008 geothermal resource assessment. Two favorability maps are created using the expert decision-dependent methods from the 2008 assessment (i.e., weight-of-evidence and logistic regression). With the same data, we then create six different favorability maps using logistic regression (without underlying expert decisions), XGBoost, and support-vector machines paired with two training strategies. The training strategies are customized to address the inherent challenges of applying machine learning to the geothermal training data, which have no negative examples and severe class imbalance. We also create another favorability map using an artificial neural network. We demonstrate that modern machine learning approaches can improve upon systems built with expert decisions. We also find that XGBoost, a non-linear algorithm, produces greater agreement with the 2008 results than linear logistic regression without expert decisions, because the expert decisions in the 2008 assessment rendered the otherwise linear approaches non-linear despite the fact that the 2008 assessment used only linear methods. The F1 scores for all approaches appear low (F1 score < 0.10), do not improve with increasing model complexity, and, therefore, indicate the fundamental limitations of the input features (i.e., training data). Until improved feature data are incorporated into the assessment process, simple non-linear algorithms (e.g., XGBoost) perform equally well or better than more complex methods (e.g., artificial neural networks) and remain easier to interpret.

15 GEOTHERMAL ENERGY↗
DOE OSTI2025

Decoding diffraction and spectroscopy data with machine learning: A tutorial

This Tutorial provides a step-by-step guide on how to apply supervised machine-learning techniques to analyze diffraction and spectroscopy data. This Tutorial details four models—a reconstruction-focused model, a regression-focused model, a hybrid reconstruction/regression model, and a multimodal model—that use x-ray diffraction profiles and vibrational density of states spectra to predict various microstructural descriptors. In this Tutorial, we cover data pre-processing steps, constructions of the models via dimensionality reduction and regression, training, and analysis of these models. Comparisons of the model’s performance are provided, highlighting the strength and weakness of the various approaches utilized.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Day-ahead photovoltaic power production forecasting methodology based on machine learning and statistical post-processing

A main challenge towards ensuring large-scale and seamless integration of photovoltaic systems is to improve the accuracy of energy yield forecasts, especially in grid areas of high photovoltaic shares. The scope of this paper is to address this issue by presenting a unified methodology for hourly-averaged day-ahead photovoltaic power forecasts with improved accuracy, based on data-driven machine learning techniques and statistical post-processing. More specifically, the proposed forecasting methodology framework comprised of a data quality stage, data-driven power output machine learning model development (artificial neural networks), weather clustering assessment (K-means clustering), post-processing output optimisation (linear regressive correction method) and the final performance accuracy evaluation. The results showed that the application of linear regression coefficients to the forecasted outputs of the developed day-ahead photovoltaic power production neural network improved the performance accuracy by further correcting solar irradiance forecasting biases. The resulting optimised model provided a mean absolute percentage error of 4.7% when applied to historical system datasets. Finally, the model was validated both, at a hot as well as a cold semi-arid climatic location, and the obtained results demonstrated close agreement by yielding forecasting accuracies of mean absolute percentage error of 4.7% and 6.3%, respectively. Finally, the validation analysis provides evidence that the proposed model exhibits high performance in both forecasting accuracy and stability.

42 ENGINEERING↗
DOE OSTI2020

Global ocean dimethyl sulfide climatology estimated from observations and an artificial neural network

Marine dimethyl sulfide (DMS) is important to climate due to the ability of DMS to alter Earth's radiation budget. Knowledge of the global-scale distribution, seasonal variability, and sea-to-air flux of DMS is needed in order to improve understanding of atmospheric sulfur, aerosol/cloud dynamics, and albedo. Here we examine the use of an artificial neural network (ANN) to extrapolate available DMS measurements to the global ocean and produce a global climatology with monthly temporal resolution. A global database of 82 996 ship-based DMS measurements in surface waters was used along with a suite of environmental parameters consisting of latitude–longitude coordinates, time of day, time of year, solar radiation, mixed layer depth, sea surface temperature, salinity, nitrate, phosphate, and silicate. Linear regressions of DMS against the environmental parameters show that on a global-scale mixed layer depth and solar radiation are the strongest predictors of DMS. These parameters capture ~9 % and ~7 % of the raw DMS data variance, respectively. Multilinear regression can capture more of the raw data variance (~39 %) but strongly underestimates DMS in high-concentration regions. In contrast, the artificial neural network captures ~66 % of the raw data variance in our database. Like prior climatologies our results show a strong seasonal cycle in surface ocean DMS with the highest concentrations and sea-to-air fluxes in the high-latitude summertime oceans. We estimate a lower global sea-to-air DMS flux (20.12±0.43 Tg S yr –1 ) than the prior estimate based on a map interpolation method when the same gas transfer velocity parameterization is used. Our sensitivity test results show that DMS concentration does not change unidirectionally with each of the environmental parameters, which emphasizes the interactions among these parameters. The ANN model suggests that the flux of DMS from the ocean to the atmosphere will increase with global warming. Given that larger DMS fluxes induce greater cloud albedo, this corresponds to a negative climate feedback.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Artificial Intelligence Applications in Renewable Energy

Addressing new methodologies in deep learning (DL), machine learning (ML) and artificial intelligence (AI), the webinar speakers will provide an overview of the literature spanning these three overlapping fields as applied to energy systems research. The audience will learn how developments in these areas have added new capabilities for pattern recognition and predictive modeling that are complementary to more traditional modeling approaches used in energy systems research. The speakers will illustrate several use cases of these new approaches in the energy space, such as physics-guided neural networks to improve ML regressions for solar data and how AI agents can be used to explore power plant operations.

40 EE - Geothermal Technologies Office (EE-4G)↗
DOE OSTI2021

Beyond optimization—supervised learning applications in relativistic laser-plasma experiments

We explore the applications of machine learning techniques in relativistic laser-plasma experiments beyond optimization purposes. We predict the beam charge of electrons produced in a laser wakefield accelerator given the laser wavefront change caused by a deformable mirror. Machine learning enables feature analysis beyond merely searching for an optimal beam charge, showing that specific aberrations in the laser wavefront are favored in generating higher beam charges. Supervised learning models allow characterizing the measured data quality as well as recognizing irreproducible data and potential outliers. Furthermore, we also include virtual measurement errors in the experimental data to examine the model robustness under these conditions. This work demonstrates how machine learning methods can benefit data analysis and physics interpretation in a highly nonlinear problem of relativistic laser-plasma interaction.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Nested active learning for efficient model contextualization and parameterization: pathway to generating simulated populations using multi-scale computational models

There is increasing interest in the use of mechanism-based multi-scale computational models (such as agent-based models (ABMs)) to generate simulated clinical populations in order to discover and evaluate potential diagnostic and therapeutic modalities. The description of the environment in which a biomedical simulation operates (model context) and parameterization of internal model rules (model content) requires the optimization of a large number of free parameters. In this work, we utilize a nested active learning (AL) workflow to efficiently parameterize and contextualize an ABM of systemic inflammation used to examine sepsis. Contextual parameter space was examined using four parameters external to the model’s rule set. The model’s internal parameterization, which represents gene expression and associated cellular behaviors, was explored through the augmentation or inhibition of signaling pathways for 12 signaling mediators associated with inflammation and wound healing. We have implemented a nested AL approach in which the clinically relevant (CR) model environment space for a given internal model parameterization is mapped using a small Artificial Neural Network (ANN). The outer AL level workflow is a larger ANN that uses AL to efficiently regress the volume and centroid location of the CR space given by a single internal parameterization. We have reduced the number of simulations required to efficiently map the CR parameter space of this model by approximately 99%. In addition, we have shown that more complex models with a larger number of variables may expect further improvements in efficiency.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

AI-Based EMT Dynamic Model of PV Systems

Several electromagnetic transient (EMT) dynamic modeling methods are available to model systems like photovoltaic (PV) plants, wind power plants, variable-speed drives, among others. The methods include: (a) physics-based models and (b) data-driven models. The physics-based dynamic models may include high-fidelity switched system model and average-value model that both require the control algorithms included in the models. However, manufacturers typically prefer to provide black-box models to avoid disclosing proprietary. One of the solutions to prevent disclosing control algorithms is the use of data-driven dynamic EMT models of PV systems. In this paper, data-driven dynamic EMT model based on artificial intelligence (AI) algorithms are presented. The AI algorithms evaluated include convolutional neural networks, recurrent neural networks, and nonlinear auto-regressive exogenous model. Automation in generating data and training these models is also discussed in this paper. The results generated by the best AI algorithms have been observed to be greater than 95 % accurate.

Debnath, Suman↗