Search NASA⌕ Search

SEARCH · Search NASA

Results for “Hyperparameter tuning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Machine Learning based Aircraft Performance Model Estimation for Trajectory Prediction

The accurate prediction of aircraft trajectory by ground-based decision support tools is a critical component of air traffic management in the US National Airspace System (NAS). Accurate predictions of where the aircraft will be in the future or when they will arrive at specific locations (e.g., fixes) is a key enabler for sequencing and efficient arrival management of flights. Traditional physics based aircraft trajectory prediction relies on a simplified point-mass total energy model whose parameters are referred to as Aircraft Performance Model (APM) parameters. Even though the performance coefficients and weight of an aircraft are a vital part of the aircraft performance model’s predictions and accuracy, these coefficients are proprietary in nature and therefore, unavailable to decision-support tools. Current approaches freeze some coefficients to default base of aircraft data (BADA) values and optimize others. However, the APM parameters are highly coupled by the flight dynamics and prioritizing one parameter over others leads to bias and skewed predictions. To alleviate this problem, we provide a combined optimization framework to predict all the critical (thrust, drag and weight) APM parameters. This paper is focused on training Machine Learning (ML) models that map historical flights to optimized APM parameters that provide the best fit (in terms of prediction error). Our dataset obtained from NASA’s Sherlock data warehouse is comprised of thousands of historical flights and includes weather and track data collected from 2019. Using different subsets of relevant features (e.g., aircraft type), we trained several ML models to estimate the aircraft’s take off weight, drag polar coefficients (both parasitic and lift induced), and thrust settings (multiplier applied to the maximum engine thrust). The chosen flights are from three of the most common aircraft types (B738, B737, and A320) arriving at four airports (LAX, DEN, MSP, and DFW). Our ML approach is comprised of two different solutions: 1- using a subset of features that are known prior to the flight departure and do not change during flight (such as engine type, current temperature at departure & destination airports, aircraft type) and 2 - using a subset of temporal features of the flight trajectory (such as cruise altitude, Mach, airspeed, and rate of climb) in addition to the pre-departure features from the first solution. The labels or target variables are the APM parameters that were obtained by an optimized ordinary differential equations (ODE) fitting process (applied to individual flights). The ODE-fitting is very time intensive and is therefore performed offline. Thus, training an ML model to learn the relationship between the flight features and ODE-generated labels enables faster estimation of the APM parameters and is therefore amenable to real-time prediction. Various ML models including linear regression, random forest, XGBoost, and neural network were trained, and the results are compared. After model validation and hyperparameter-tuning, we observed that the Random Forest model outperformed the other three models by the overall mean square error (MSE) of 2% for the first solution and 1.5% for the second solution. Finally, the ML-derived parameters are compared against default BADA APM parameters using NASA’s Autonomy Development toolkit (ADK) simulation software. The simulation results for one of each aircraft type is shown and discussed.

Aida Sharif Rohani↗
NASA NTRSDate not supplied

Predicting Airport Runway Configurations for Decision-Support Using Supervised Learning

One of the most challenging tasks for air traffic controllers is runway configuration management (RCM). It deals with the optimal selection of runways to operate on (for arrivals and departures) based on current and forecast of traffic, surface wind speed, wind direction, other environmental variables, noise constraints, and several other airport-specific factors. In this paper, a methodology using supervised learning is developed to build a predictive model for RCM decision-support from large volumes of historical data. Data from two full years (2018 and 2019) related to current and forecast weather, demand/capacity, etc. is collected, analyzed, and fused together. A variety of supervised learning algorithms are tested for predicting runway configuration and hyperparameter tuning is carried out to select the best performing model. The validation process involves two airports of low (Charlotte Douglas International Airport, CLT) and high (Denver International Airport, DEN) complexity of configuration decision-making. The results show significant promise for the two airports with test accuracy of 93% (CLT) and 73% (DEN). The methodology is scalable and generalizable to other airports across the U.S. National Airspace System.

air traffic management↗
NASA NTRSDate not supplied

Predicting Airport Runway Configuration for Decision-Support Using Supervised Learning

One of the most challenging tasks for air traffic controllers is runway configuration management (RCM). It deals with the optimal selection of runways to operate on (for arrivals and departures) based on current and forecast of traffic, surface wind speed, wind direction, other environmental variables, noise constraints, and several other airport-specific factors. In this paper, a methodology using supervised learning is developed to build a predictive model for RCM decision-support from large volumes of historical data. Data from two full years (2018 and 2019) related to current and forecast weather, demand/capacity, etc. is collected, analyzed, and fused together. A variety of supervised learning algorithms are tested for predicting runway configuration and hyperparameter tuning is carried out to select the best performing model. The validation process involves two airports of low (Charlotte Douglas International Airport, CLT) and high (Denver International Airport, DEN) complexity of configuration decision-making. The results show significant promise for the two airports with test accuracy of 93% (CLT) and 73% (DEN). The methodology is scalable and generalizable to other airports across the U.S. National Airspace System.

air traffic management↗
NASA NTRSDate not supplied

A Machine Learning Ready Dataset of Acoustic Power Maps for Detection of Active Region Emergence

The development of an accurate forecast for solar eruptive activity has become increasingly important in order to prevent any potential impact on activities in space and the Earth's environment. It is therefore crucial to detect active regions before they appear on the solar surface and create early warning capabilities for upcoming Space Weather disturbances. In this work, 9TB of solar data (SDO/HMI dopplergrams, magnetograms and continuum intensity maps) involving the emergence of 61 NOAA solar active regions since 2010 were processed using the NASA HECC capabilities. An acoustic power maps time-series dataset was created (for four different frequency ranges and processed to take into account the solar sphere geometric effect ) which can be used for understanding the dynamics of the solar surface and train a variety of ML models. The calculated acoustic power maps carry precursor information associated with the decrease in continuum intensity on the solar surface, verifying older helioseismology research. Our results show that a Long Short-Term Memory (LSTMs) model, with a modest layer depth and the right hyperparameters tuned, when trained on this solar acoustic power maps dataset can predict without false negatives a drop in intensity (associated with the emergence of the active region), up to 18 hours in advance.

SMD↗
DOE OSTI2021

Failure prediction and estimation of failure parameters

Machine-learning methods and apparatus are disclosed to determine frictional state or other parameters in an earthquake zone or other failing medium, using acoustic emission, seismic waves, or other detectable indicators of microscopic processes. Predictions of future failures are demonstrated in different regimes. A classifier is trained using time series of acoustic emission data along with historic data of frictional state or failure events. In disclosed examples, random forests and gradient boost trees are used, and grid-search or EGO procedures are used for hyperparameter tuning. Once trained, the classifier can be applied to testing or live data in order to assess a frictional state, assess seismic hazard, or make predictions regarding a future failure event. The technology has been developed in a double direct shear apparatus, but can be widely applied to seismic faults, other terrestrial failures, or failures in man-made structures. Variations are disclosed.

Johnson, Paul Allan↗
DOE OSTI2022

Subsurface stress criticality associated with fluid injection and determined using machine learning

Machine-learning methods and apparatus are disclosed to determine critical state or other parameters related to fluid-driven failure of a terrestrial locale impacted by anthropogenic activities such as hydraulic fracturing, hydrocarbon extraction, wastewater disposal, or geothermal harvesting. Acoustic emission, seismic waves, or other detectable indicators of microscopic processes are sensed. A classifier is trained using time series of microscopic data along with corresponding data of critical state or failure events. In disclosed examples, random forests and artificial neural networks are used, and grid-search or EGO procedures are used for hyperparameter tuning. Once trained, the classifier can be applied to live data from a fluid injection locale in order to assess a frictional state, assess seismic hazard, assess permeability, make predictions regarding a future fluid-driven failure event, or drive engineering solutions for mitigation or remediation. Variations are disclosed.

Johnson, Paul Allan↗
DOE OSTI2023

Deep Learning for Modeling Enhanced Geothermal Systems

Enhanced Geothermal Systems (EGS) offer a vast potential to expand the use of geothermal energy. Heat is extracted from this engineered system by injecting cold water into a subsurface fractures, which are in contact with the hot dry rock, and pulled through the production wells. Creating EGS requires improving the natural permeability of hot crystalline rocks. To develop economically viable EGS reservoirs, significant technical barriers (e.g., better stimulation technologies without adequate water and/or permeability) and non-technical barriers (e.g., land access and permitting) must be overcome. In this short conference paper, we present a workflow to address a part of this challenge – “How to develop economically viable EGS using existing technologies?”. Our workflow called the GeoThermalCloud (GTC) for EGS, leverages recent advances in machine learning, deep learning, and cloud computing. This GTC framework is open-source and available at https://github.com/SmartTensors/GeoThermalCloud.jl. The GTC framework provides trained deep learning (DL) models to estimate the net present value of a given EGS design scenario. The Geothermal Design Tool (https://github.com/GeoDesignTool/GeoDT.git), a fast and simplified multi-physics solver, is used to develop a database for training DL models. The database consists of EGS design parameters (inputs to DL model) and their net present value (output of DL model) in uncertain geologic systems. The EGS design parameters for constructing this training database are based on Utah FORGE but include the options of more wells and deeper depths. The DL models are trained by ingesting the EGS design parameters and estimating the corresponding net present value. Such an emulation allows us to screen various EGS designs quickly and identify good development strategies by coupling them with optimization techniques. Our preliminary results show promise in DL emulation of net present value. However, a lot more work is needed to improve the predictive capability of DL models (i.e., extensive hyperparameter tuning is necessary). This will be the primary focus of our future work.

artificial neural networks, geothermal↗
DOE OSTI2025

Neural Network Analysis of Nuclear Magnetic Resonance and Infrared Spectra

Nuclear magnetic resonance (NMR) spectroscopy and infrared (IR) spectroscopy are powerful chemical characterization techniques with broad general usage. However, the manual evaluation of the resulting spectra is time-consuming and requires significant expertise, preventing insights from being used in real-time applications. With recent advances in computation and artificial intelligence (AI), new tools are available for automating spectral interpretation. In this work, machine learning (ML) algorithms using 1-dimensional convolutional neural networks (CNNs) were applied to identify common functional groups from spectral information. Raw spectra were collected virtually from the Human Metabolome Database (HMDB) and National Institute of Standards and Technology (NIST) Chemistry WebBook and processed into a suitable standard. Algorithm design was tailored to best fit the nature of the problem, with built-in flexibility to accommodate relevant parameters beyond the raw spectral input, specifically solvent identity and magnetic frequency for NMR. The predictive capability of the algorithm in identifying functional groups is displayed in several examples. This methodology has been compiled into a code repository and could easily be modified to adapt alternative data sources, including other spectrum types. To mitigate overfitting, a common problem in mathematical modeling where overfamiliarity with training data produces trends that are not representative of the general data, a novel metric was developed, referred to as Accufit. Accufit includes a parameter that penalizes substantial differences in the training accuracy and the accuracy of an independent validation set. Examples are presented showing the effectiveness of Accufit in maintaining the model’s predictive capability while controlling the overfitting when used as a custom metric for hyperparameter tuning.

Sturgill, James↗
DOE OSTI2025

Predicting Initial Trans-Membrane Pressure for Optimized Operations in UF Unit Using Random Forest

With the growing scarcity of freshwater, innovative process design mechanisms like Reverse Osmosis (RO) are increasingly gaining attention among water treatment utilities to address the rising demand. Ensuring reliable water production necessitates efficient resource utilization, minimizing downtime in (ultra-filtration) UF systems. Recent advancements in machine learning (ML) have enabled the development of accurate data-driven models for Model Predictive Control (MPC), often requiring minimal prior knowledge of underlying physical processes. In this study, we present predictive regression models based on Random Forest (RF) and Auto-Regressive (AR) approaches to forecast the initial Trans-Membrane Pressure (TMP) for each filtration cycle in data generated by Direct Potable Reuse (DPR) systems. The proposed RF-based model demonstrates superior performance compared to baseline methods, including historical mean, Last Observation Carried Forward (LOCF), and naïve AR models, across various forecasting horizons in terms of root mean square error (RMSE) metric. To evaluate how different classes of process variables contribute to TMP dynamics over time, we examine the feature importance of independent covariates across multiple forecast horizons. This analysis provides insight into the temporal relevance of operational and sensor-derived features, guiding control and monitoring strategies. Additionally, the impact of hyperparameter tuning on TMP prediction performance is studied for both direct and recursive RF modelling approaches across increasing forecast horizons. Accurate prediction of initial TMP is critical for optimizing RO operations, as it enables the development of robust modelling frameworks by accurately estimating membrane fouling trends, thereby enhancing process efficiency and long-term reliability. The demonstrated efficacy of the RF-based approach highlights its potential as a tool for real-time decision-making in water treatment systems, paving the way for advanced process optimization and sustainable water resource management.

Mukherjee, Subrata [ORNL] (ORCID:0000000309930338)↗
DOE OSTI2025

Investigation of the Performance and Explainability Tradeoffs for Machine-Learning Models for Predictive Maintenance of Circulating Water Systems in Nuclear Power Plants

Predictive maintenance (PdM) has shown great potential for achieving substantial cost savings and enhancing the economic competitiveness of nuclear power plants (NPPs) in today's energy market. Among the different modeling approaches that exist, machine learning (ML) tools in particular have a demonstrated ability to handle high dimensional and multivariate data and to extract hidden relationships within data in industrial environments. While ML methods show great potential, their lack of explainability---especially for black-box models---is a major hurdle to their adoption. Moreover, considering the supposed trade-off between explainability and performance challenges, careful consideration must be made as to which of these quality aspects takes precedence in light of multiple modeling options, resource availability, and domain characteristics. The present work evaluates the performance of six ML models, each with a different degree of explainability, in classifying the conditions of circulating water pumps (CWPs) by utilizing sensor data from nuclear power plants. To determine the drivers behind the trade-offs presented by this array of models, this work also tests different combinations of CWP units as the training and testing data, degrees of data imbalance, and objective functions for hyperparameter tuning. It was found that black-box models tend to afford superior performance in cases where there are far more instances of one type of labeled data than of any other type. It is recommended that a guided procedure be followed for designing and delivering an ML system that is sufficiently explainable to all involved stakeholders.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

Real-Time Detection of Charge Jumps in Superconducting Qubits with a Convolutional Neural Network

Ionizing radiation from cosmic rays and gammas can induce discontinuous jumps in the environmental charge of superconducting qubits (charge jumps), causing correlated errors that challenge fault-tolerant quantum computing while simultaneously providing a detection signature for quantum sensing applications. Current detection methods operate offline, introducing latency incompatible with in-the-loop qubit control. In this paper, an online detector of charge jumps for superconducting qubits, based on a dilated causal convolutional neural network (DCCNN) designed for in-the-loop deployment on the Quantum Instrumentation Control Kit (QICK) platform, is presented. The network is trained on synthetic Ramsey tomography scans generated from qubit templates measured at the Northwestern Experimental Underground Site (NEXUS) at Fermilab, and translated to FPGA firmware via hls4ml with ap_fixed$\langle 16,6 \rangle$ quantization, reaching a per-inference latency of $6.19 μ$s on the Zynq UltraScale+ RFSoC ZCU216. At this operating point the DCCNN matches the detection efficiency of the established offline $χ^2$ algorithm ($0.843 \pm 0.022$ vs. $0.866 \pm 0.020$ on $|Δq| \in [0.1, 0.5] e$ at matched false-positive rate), while requiring no per-qubit hyperparameter tuning. This shifts charge-jump detection from a post-hoc diagnostic to a control-loop primitive, enabling adaptive protocols that respond to radiation-induced events in situ, with applications to quantum-computing error mitigation and to the use of superconducting qubits as particle detectors.

Gaytan-Villarreal, Daniel [Carnegie Mellon U.]↗
DOE OSTI2021

Multi-Objective Hyperparameter Optimization for Spiking Neural Network Neuroevolution

Neuroevolution has had significant success over recent years, but there has been relatively little work applying neuroevolution approaches to spiking neural networks (SNNs). SNNs are a type of neural network that includes temporal processing component, are not easily trained using other methods, and can be deployed into energy-efficient neuromorphic hardware. In this work, we investigate two evolutionary approaches for training SNNs. We explore the impact of the hyperparameters of the evolutionary approaches, including tournament size, population size, and representation type, on the performance of the algorithms. We present a multi-objective Bayesian-based hyperparameter optimization approach to tune the hyperparameters to produce the most accurate and smallest SNNs. We show that the hyperparameters can significantly affect the performance of these algorithms. We also perform sensitivity analysis and demonstrate that every hyperparameter value has the potential to perform well, assuming other hyperparameter values are set correctly.

Parsa, Maryam↗
DOE OSTI2021

A Software Framework for Comparing Training Approaches for Spiking Neuromorphic Systems

There are a wide variety of training approaches for spiking neural networks for neuromorphic deployment. However, it is often not clear how these training algorithms perform or compare when applied across multiple neuromorphic hardware platforms and multiple datasets. In this work, we present a software framework for comparing performance across four neuromorphic training algorithms across three neuromorphic simulators and four simple classification tasks. We introduce an approach for training a spiking neural network using a decision tree, and we compare this approach to training algorithms based on evolutionary algorithms, back-propagation, and reservoir computing. We present a hyperparameter optimization approach to tune the hyperparameters of the algorithm, and show that these optimized hyperparameters depend on the processor, algorithm, and classification task. Finally, we compare the performance of the optimized algorithms across multiple metrics, including accuracy, training time, and resulting network size, and we show that there is not one best training algorithm across all datasets and performance metrics.

Schuman, Catherine↗
DOE OSTI2021

HYPPO: A Surrogate-Based Multi-Level Parallelism Tool for Hyperparameter Optimization

We present a new software, HYPPO, that enables the automatic tuning of hyperparameters of various deep learning (DL) models. Unlike other hyperparameter optimization (HPO) methods, HYPPO uses adaptive surrogate models and directly accounts for uncertainty in model predictions to find accurate and reliable models that make robust predictions. Using asynchronous nested parallelism, we are able to significantly alleviate the computational burden of training complex architectures and quantifying the uncertainty. HYPPO is implemented in Python and can be used with both TensorFlow and PyTorch libraries. We demonstrate various software features on time-series prediction and image classification problems as well as a scientific application in computed tomography image reconstruction. Finally, we show that (1) we can reduce by an order of magnitude the number of evaluations necessary to find the most optimal region in the hyperparameter space and (2) we can reduce by two orders of magnitude the throughput for such HPO process to complete.

adaptation models↗
DOE OSTI2023

Surrogate Model Based Optimization for Finding Robust Deep Learning Model Architectures

Deep Learning (DL) models are increasingly used throughout the sciences. However, their performance and usefulness depend greatly on their architecture which is defined by hyperparameters such as the number of nodes, layers, the learning rate, etc. Tuning these hyperparameters is time-consuming because evaluating their performance requires a lengthy training step. Stochastic optimizers used in training lead to performance variability and potentially prediction reliability issues. In this talk, we will describe an automated optimization method based on surrogate models and active learning strategies for tuning DL model architectures. We take into account the prediction variability with the goal to identify architectures that make reliable and robust predictions. We demonstrate our developments on an application arising in particle physics.

deep learning↗
DOE OSTI2023

Multiobjective Hyperparameter Optimization for Deep Learning Interatomic Potential Training Using NSGA-II

Deep neural network (DNN) potentials are an emerging tool for simulation of dynamical atomistic systems, with the promise of quantum mechanical accuracy at speedups of 10000$\times$. As with other DNN methods, hyperparameters used during training can make a substantial difference in model accuracy, and optimal settings vary with dataset. To enable rapid tuning of hyperparameters for DNN potential training, we developed a scalable multiobjective optimization evolutionary algorithm for supercomputers and tested it on the Summit system at the Oak Ridge Leadership Computing Facility (OLCF). The multiobjective approach is required due to the coupling of two learned values defining the potential: the energy and force. Using a large-scale implementation of the NSGA-II algorithm adapted for training DNN potentials, we discovered several optimal multiobjective combinations, including best choices of activation functions, learning rate scaling scheme, and pairing of the two radial cutoffs used in the three dimensional descriptor function.

Coletti, Mark↗
DOE OSTI2022

An Accuracy-Maximization Approach for Claims Classifiers in Document Content Analytics for Cybersecurity

This paper presents our research approach and findings towards maximizing the accuracy of our classifier of feature claims for cybersecurity literature analytics, and introduces the resulting model ClaimsBERT. Its architecture, after extensive evaluations of different approaches, introduces a feature map concatenated with a Bidirectional Encoder Representation from Transformers (BERT) model. We discuss deployment of this new concept and the research insights that resulted in the selection of Convolution Neural Networks for its feature mapping aspects. We also present our results showing ClaimsBERT to outperform all other evaluated approaches. This new claims classifier represents an essential processing stage within our vetting framework aiming to improve the cybersecurity of industrial control systems (ICS). Furthermore, in order to maximize the accuracy of our new ClaimsBERT classifier, we propose an approach for optimal architecture selection and determination of optimized hyperparameters, in particular the best learning rate, number of convolutions, filter sizes, activation function, the number of dense layers, as well as the number of neurons and the drop-out rate for each layer. Fine-tuning these hyperparameters within our model led to an increase in classification accuracy from 76% obtained with BertForSequenceClassification’s original model to a 97% accuracy obtained with ClaimsBERT.

Ameri, Kimia (ORCID:0000000328791871)↗