Search NASA⌕ Search

SEARCH · Search NASA

Results for “overfitting”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

RADEMACHER COMPLEXITY REGULARIZATION FOR CORRELATION-BASED MULTIVIEW REPRESENTATION LEARNING

Deep correlation-based multiview representation learning techniques have become increasingly popular methods for extracting highly correlated representations from multiview data. However, their ability to find highly complex mappings between the views can also lead to overfitting and overly correlated representations. In this work, we propose a regularizer for this specific problem, based on the Rademacher complexity of the DNNs, tailored for multiview correlation maximization. We demonstrate that the proposed regularization leads to less noisy representations in synthetic data and improved performance of downstream tasks in real-world multiview datasets.

Kuschel, Maurice↗
DOE OSTI2026

FFTSF: Revisiting Sub-Seasonal Streamflow Forecasting with Simple Feedforward Network

Accurate short-to-subseasonal streamflow forecasts are vital for water management, including flood preparedness, drought mitigation, hydropower scheduling, and ecosystem protection. However, extending a forecast beyond a few days remains challenging due to complexity of hydrological processes. While recent self-attention based transformer architectures such as iTransformer have gained traction in time-series forecasting, these models suffer from several critical limitations: (1) significant computational overhead that scales quadratically with sequence length, (2) vulnerability to overfitting on limited hydrological datasets, (3) degraded performance on long-horizon forecasts due to attention decay, and (4) excessive architectural complexity that hampers interpretability and operational deployment. In this study, we propose a simple Feedforward Time Series Forecasting (FFTSF) network that directly addresses these limitations through its lightweight architecture and long-range forecasting capabilities. We evaluate FFTSF across 178 USGS stream gauges spanning diverse climate regimes by forecasting lead times of 1-, 7-, 14-, and 30-days. Our results demonstrate that FFTSF achieves competitive performance at short lead times (NSE of 0.778 for 1-day forecasts) while substantially outperforming complex baselines at longer forecast period, achieving the highest NSE (0.271) at 30-day forecasts with greater robustness and stability. For 30-day forecasts, FFTSF achieves a 71% improvement over NLinear, 57% improvement over DLinear and 12% improvement over the computationally intensive iTransformer while requiring fewer computational resources. Our findings reveal that architectural complexity is not necessary for hydrological forecasting, demonstrating that well-designed simple models can outperform attention mechanisms for subseasonal streamflow forecasting. The computational efficiency and consistent long-range performance of FFTSF make it suitable for water management applications where reliable extended forecasts are essential.

Krishnan Kutty Ambika, Anukesh [ORNL] (ORCID:00000↗
DOE OSTI2020

Non-Intrusive Appliance Identification with Appliance-Specific Networks

The problem of noninstrusive load monitoring (NILM) is usually formulated as a single-channel blind source separation task, whose successful solution enable fast and convenient load identification and energy disaggregation. When applied at test time, NILM algorithms aim to identify the operating characteristics of individual appliances from an aggregate power measurement of the entire house. Recent advances in deep learning gave rise to many methods that mostly focus on learning a direct mapping from aggregate measurement to individual appliance power. However, these methods are not only computationally expensive, but they often suffer from overfitting and do not generalize very well. In this article, we propose a novel NILM method that leverages advances in statistical learning that have not been properly applied in this domain before. The proposed method consists of three stages: first, a Bayesian nonparametric learning-based approach for appliance state extraction; second, synthetic minority oversampling technique for data augmentation and mitigating the heavy imbalance in switching events; and third, appliance-specific lightweight long short-term memory networks for status classification for each appliance. Here, we adopt a “differential” input (the difference before and after the switching event) to reduce the complexity of network training and make the proposed method robust to multiappliance switching events. Experiments are conducted to demonstrate the effectiveness of the proposed method, achieving superior performance when compared to recent methods. An ablation study is conducted to demonstrate the effectiveness of each module of our method. Finally, we investigate the quality of generated synthetic samples.

42 ENGINEERING↗
DOE OSTI2022

Hanford low-activity waste glass composition-temperature-melt viscosity relationships

This study developed a model for predicting viscosity of alkali-alumino-borosilicate glass melts as functions of composition and temperature. The model is based on a total of 3935 viscosity-temperature data from 574 glasses with viscosity values ranging from 2.53 to 7260 Poise (P) in the temperature range of 900–1260°C. Several different model forms were surveyed, including those based on Arrhenius, Vogel-Fulcher-Tammann, Avramov-Milchev, and Mauro-Yue-Ellison-Gupta-Allen. For each of these models, combinations of the temperature-independent parameters were fitted to composition. It was found that generally fitting more than one temperature-independent parameter as functions of composition resulted in overfitting. The Avramov-Milchev-based model was found to best represent the Hanford low-activity waste glass melt viscosity data based on model fit and validation statistics. A 21-term partial quadratic mixture model was recommended for use. This model predicts melt viscosity with a root-mean square error of .1736 ln(P), which is similar to the error in viscosity measurements from replicate glass analyses of .1383 ln(P). Viscosity was found to be most increased by SiO 2 > Al 2 O 3 > ZrO 2 > SnO 2 and most decreased by Li 2 O > Na 2 O > B 2 O 3 > CaO > K 2 O > MgO, at temperatures from 900 to 1260°C.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2026

Boron Coordination in Multicomponent Glasses: Analytical Models and Machine Learning With Uncertainty

Borosilicate glasses are extensively used in a variety of applications from kitchenware to nuclear waste immobilization due to the strong network formed by the Si-O-B bond that makes it resistant to chemical corrosion and gives it a low thermal expansion. Boron, however, exists in both trigonal BO3 and tetrahedral BO4 bonds in glass systems, which impacts the chemical durability and thermal resistance of the glass, amongst other properties. Boron coordination (N4), or the ratio of the amount of BO4 to BO3 within a glass, may aid in predicting these properties but is difficult to derive without experimental data due to the complexity of impacts from varied glass compositions and processing factors. For this reason, compositional models have been developed to predict boron coordination, but the models typically include a limited number of glass components. To help fill this gap in the models, in this work, a diverse multicomponent glass dataset of 809 glasses is compiled from a literature search, and then a number of analytical and machine learning (ML) models are trained on the dataset. Previously developed modified Bernstein and modified Du Stebbins analytical models were fitted to update parameters with the new dataset. Then, partially Bayesian neural networks, Gaussian process regressor, and heteroskedastic deterministic neural networks were evaluated. The ML models examined all have different strategies to overcome the potential for overfitting as a result of a limited training dataset, and return results that account for model uncertainty, which can be valuable for understanding model reliability. For the first time, cooling rate is introduced as an input parameter for ML models, showing consistent improvements in performance and solidifying the importance of including parameters outside of composition alone for N4 prediction. The machine learning models examined here show promise in accurate predictions of boron coordination in borosilicate glasses, all achieving R2 values of 0.91.

boron coordination↗
DOE OSTI2021

Evaluation of U-net-based Image Segmentation Model to Digital Mammography

Detecting suspicious lesions in medical imaging is the important first step in computer-aided detection (CAD) systems. However, detecting abnormalities in breast tissue is difficult due to the lesion's varying size, shape, margin, and contrast with the background tissue. We focused on mass segmentation, a method that provides notable morphological features by outlining contours of masses. Accurate segmentation is crucial for correct diagnosis. Recent advancements in deep learning have improved object detection and segmentation, and these techniques are also being applied to medical imaging studies. We focused on U-net, which is a recently developed mass segmentation algorithm based on a fully convolutional network. The U-net architecture consists of (1) a contracting path to increase the resolution of the output and (2) a symmetric expanding path to better locate the region of interest. The performance of a U-net model was tested with 63 digital mammograms from INbreast, a publicly available database. We trained the model with images resized to 40x40 pixels and conducted 10-fold cross-validation to prevent overfitting. The model's performance with respect to breast density and the lesion's BI-RADS rating was also investigated. Dice coefficients (DC) were used as a performance measure to compare the predicted segmentation of the model with the ground truth. Logistic regression and an analysis of variance were performed to determine the significance of the DCs with regards to breast density and lesion behavior and to calculate the 95% confidence interval. The average DC was 0.80. The difference between DCs for BI-RADS 2 and 4c and for BI-RADS 2 and 5 were significant, suggesting that the model has more difficulty in segmenting benign lesions.

Cho, Priscilla↗
DOE OSTI2022

Comprehensive compartmental model and calibration algorithm for the study of clinical implications of the population-level spread of COVID-19: a study protocol

The complex dynamics of the coronavirus disease 2019 (COVID-19) pandemic has made obtaining reliable long-term forecasts of the disease progression difficult. Simple mechanistic models with deterministic parameters are useful for short-term predictions but have ultimately been unsuccessful in extrapolating the trajectory of the pandemic because of unmodelled dynamics and the unrealistic level of certainty that is assumed in the predictions. We propose a 22-compartment epidemiological model that includes compartments not previously considered concurrently, to account for the effects of vaccination, asymptomatic individuals, inadequate access to hospital care, post-acute COVID-19 and recovery with long-term health complications. Additionally, new connections between compartments introduce new dynamics to the system and provide a framework to study the sensitivity of model outputs to several concurrent effects, including temporary immunity, vaccination rate and vaccine effectiveness. Subject to data availability for a given region, we discuss a means by which population demographics (age, comorbidity, socioeconomic status, sex and geographical location) and clinically relevant information (different variants, different vaccines) can be incorporated within the 22-compartment framework. Considering a probabilistic interpretation of the parameters allows the model's predictions to reflect the current state of uncertainty about the model parameters and model states. We propose the use of a sparse Bayesian learning algorithm for parameter calibration and model selection. This methodology considers a combination of prescribed parameter prior distributions for parameters that are known to be essential to the modelled dynamics and automatic relevance determination priors for parameters whose relevance is questionable. This is useful as it helps prevent overfitting the available epidemiological data when calibrating the parameters of the proposed model. Population-level administrative health data will serve as partial observations of the model states.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?

Physics-informed neural networks (PINNs) have become a popular choice for solving high-dimensional partial differential equations (PDEs) due to their excellent approximation power and generalization ability. Recently, extended PINNs (XPINNs) based on domain decomposition methods have attracted considerable attention due to their effectiveness in modeling multiscale and multiphysics problems and their parallelization. However, theoretical understanding of their convergence and generalization properties remains unexplored. In this study, we take an initial step towards understanding how and when XPINNs outperform PINNs. Specifically, for general multilayer PINNs and XPINNs, we first provide a prior generalization bound via the complexity of the target functions in the PDE problem and a posterior generalization bound via the posterior matrix norms of the networks after optimization. Moreover, based on our bounds, we analyze the conditions under which XPINNs improve generalization. Concretely, our theory shows that the key building block of XPINN, namely, the domain decomposition, introduces a tradeoff for generalization. On the one hand, XPINNs decompose the complex PDE solution into several simple parts, which decreases the complexity needed to learn each part and boosts generalization. On the other hand, decomposition leads to less training data being available in each subdomain, and hence such a model is typically prone to overfitting and may become less generalizable. Empirically, we choose five PDEs to show when XPINNs perform better than, similar to, or worse than PINNs, hence demonstrating and justifying our new theory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

TNet: A Model-Constrained Tikhonov Network Approach for Inverse Problems

Deep learning (DL), in particular deep neural networks, by default is purely data-driven and in general does not require physics. This is the strength of DL but also one of its key limitations when applied to science and engineering problems in which underlying physical properties—such as stability, conservation, and positivity—and accuracy are required. DL methods in their original forms are often not capable of respecting the underlying mathematical models or achieving desired accuracy even in big-data regimes. On the other hand, many data-driven science and engineering problems, such as inverse problems, typically have limited experimental or observational data, and DL would overfit the data in this case. Leveraging information encoded in the underlying mathematical models, we argue, not only compensates for missing information in low data regimes but also provides opportunities to equip DL methods with the underlying physics, hence promoting better generalization. This paper develops a model-constrained DL approach and its variant TNet—a Tikhonov neural network—which are capable of learning not only information hidden in the training data but also in the underlying mathematical models to solve inverse problems governed by partial differential equations in low data regimes. We provide the constructions and some theoretical results for the proposed approaches for both linear and nonlinear inverse problems. Since TNet is designed to learn inverse solutions with Tikhonov regularization, it is interpretable: in fact it recovers Tikhonov solutions for linear cases while potentially approximating Tikhonov solutions for nonlinear inverse problems. We also prove that data randomization can enhance not only the smoothness of the networks but also their generalizations. Comprehensive numerical results confirm the theoretical findings and show that with even as little as 1 training data sample for one-dimensional (1D) deconvolution, 5 for an inverse 2D heat conductivity problem, 100 for inverse initial conditions for a time-dependent 2D Burgers’s equation, and 50 for inverse initial conditions for 2D Navier–Stokes equations, TNet solutions can be as accurate as Tikhonov solutions while being several orders of magnitude faster. Furthermore, this is possible owing to the model-constrained term, replications, and randomization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Discovery of Probabilistic Dirichlet-to-Neumann Maps on Graphs

Dirichlet-to-Neumann maps enable the coupling of multiphysics simulations across computational subdomains by ensuring continuity of state variables and fluxes at artificial interfaces. We present a novel method for learning Dirichlet-to-Neumann maps on graphs using Gaussian processes, specifically for problems where the data obey a conservation law arising from an underlying partial differential equation. Our approach combines discrete exterior calculus and nonlinear optimal recovery to infer relationships between vertex and edge values. This framework yields data-driven predictions with uncertainty quantification across the entire graph, even when observations are limited to a subset of vertices and edges. By minimizing the reproducing kernel Hilbert space norm while penalizing kernel complexity through maximum likelihood estimation, our method ensures that the resulting surrogate strictly enforces conservation laws without overfitting. We demonstrate our method on two representative applications: subsurface flow in fracture networks and arterial blood flow. Finally, the results demonstrate that the method maintains high accuracy and well-calibrated uncertainty estimates even under severe data scarcity, highlighting its potential for scientific applications where limited data and reliable uncertainty quantification are critical.

Dirichlet-to-Neumann map↗
DOE OSTI2022

ASP (Adaptive Splines for Prediction) [SWR-22-66]

This software package provides functionality to fit piecewise cubic splines to model the relationship between a univariate X (independent variable) and a univariate Y (dependent variable). The output is akin to linear regression but has higher "capacity" in that it can learn more complicated relationships despite leveraging only a single predictor. It thus can learn very reasonable relationships between X and Y with minimal effort. The canonical intended use case is for predicting electric demand (Y) from temperature (X). There can be complicated, non-linear relationships between these two variables, but these can be reasonably approximated by the sum of: 1) an "aggregate" relationship between daily average temperature and daily average demand, and 2) intra-day patterns represented as hourly offsets from the daily average load. The spline functionality is used for learning a robust "aggregate" relationship without the inconvenience of manual feature engineering while still ensuring robust results. The model fitting workhorse is R's built-in smooth.spline. This package provides a suite of options for controlling how smooth.spline fits a spline to the data. In particular, it is designed to return a model with a single critical point (i.e., a single location along the domain/support where the first derivative is zero). This is to ensure that predicted changes in electric demand are always positive as temperatures become more extreme. This helps to prevent overfitting, particularly when the input data has few observations or has significant uncertainty from other sources.

Murphy, Sinnott↗
DOE OSTI2026

Using Machine Learning to Understand Electric and Hybrid Vehicles Ownership in Burdened and Nonburdened Communities

Transitioning to electric and hybrid vehicles (EHVs) for all communities is a pivotal step toward sustainable transportation and environmental conservation. This paper aims to understand the adoption of EHVs, focusing on burdened communities (BCs) in the United States. The EHV ownership-based analysis combines two datasets—behavioral data from the Puget Sound Regional Travel Survey integrated with BCs (Justice40) data covering transportation insecurity, environmental burden, social vulnerability, health vulnerability, and climate and disaster risk burden. After creating this unique database, descriptive analysis and modeling are used to analyze the data and predict EHV ownership in the future. Specifically, we use a new method that combines particle swarm optimization (PSO) with a stacking model named PSO-Stacking, which incorporates heterogeneous base learners of machine learning and deep learning. PSO applies a customized objective function to select the optimal hyperparameters for heterogeneous learners within the stacking model, effectively addressing challenges such as multicollinearity, data imbalance, nonlinearity, and overfitting. The proposed solution covers more accurate results than standard benchmark models for EHV ownership in BCs and non-BCs. In addition, the results of the PSO-Stacking method are explained using the local interpretable model-agnostic explanations technique. Results show a negative correlation between the BCs indicators, that is, higher transportation insecurity associated with lower EHV ownership. Furthermore, BCs have higher future climate risk scores, diesel particulate matter levels, and PM2.5 in the air than non-BCs because of higher conventional vehicle ownership. These communities are at higher risk and can benefit from electrification, EV infrastructure, and EV policies to address environmental challenges.

Aslam, Zeeshan [ORNL]↗
DOE OSTI2022

Towards a robust out-of-the-box neural network model for genomic data

The accurate prediction of biological features from genomic data is paramount for precision medicine and sustainable agriculture. For decades, neural network models have been widely popular in fields like computer vision, astrophysics and targeted marketing given their prediction accuracy and their robust performance under big data settings. Yet neural network models have not made a successful transition into the medical and biological world due to the ubiquitous characteristics of biological data such as modest sample sizes, sparsity, and extreme heterogeneity. Here, we investigate the robustness, generalization potential and prediction accuracy of widely used convolutional neural network and natural language processing models with a variety of heterogeneous genomic datasets. Mainly, recurrent neural network models outperform convolutional neural network models in terms of prediction accuracy, overfitting and transferability across the datasets under study. While the perspective of a robust out-of-the-box neural network model is out of reach, we identify certain model characteristics that translate well across datasets and could serve as a baseline model for translational researchers.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Evaluation of the 2022 West Nile virus forecasting challenge, USA

Abstract Background West Nile virus (WNV) is the most common cause of mosquito-borne disease in the continental USA, with an average of ~1200 severe, neuroinvasive cases reported annually from 2005 to 2021 (range 386–2873). Despite this burden, efforts to forecast WNV disease to inform public health measures to reduce disease incidence have had limited success. Here, we analyze forecasts submitted to the 2022 WNV Forecasting Challenge, a follow-up to the 2020 WNV Forecasting Challenge. Methods Forecasting teams submitted probabilistic forecasts of annual West Nile virus neuroinvasive disease (WNND) cases for each county in the continental USA for the 2022 WNV season. We assessed the skill of team-specific forecasts, baseline forecasts, and an ensemble created from team-specific forecasts. We then characterized the impact of model characteristics and county-specific contextual factors (e.g., population) on forecast skill. Results Ensemble forecasts for 2022 anticipated a season at or below median long-term WNND incidence for nearly all (> 99%) counties. More counties reported higher case numbers than anticipated by the ensemble forecast median, but national caseload (826) was well below the 10-year median (1386). Forecast skill was highest for the ensemble forecast, though the historical negative binomial baseline model and several team-submitted forecasts had similar forecast skill. Forecasts utilizing regression-based frameworks tended to have more skill than those that did not and models using climate, mosquito surveillance, demographic, or avian data had less skill than those that did not, potentially due to overfitting. County-contextual analysis showed strong relationships with the number of years that WNND had been reported and permutation entropy (historical variability). Evaluations based on weighted interval score and logarithmic scoring metrics produced similar results. Conclusions The relative success of the ensemble forecast, the best forecast for 2022, suggests potential gains in community ability to forecast WNV, an improvement from the 2020 Challenge. Similar to the previous challenge, however, our results indicate that skill was still limited with general underprediction despite a relative low incidence year. Potential opportunities for improvement include refining mechanistic approaches, integrating additional data sources, and considering different approaches for areas with and without previous cases. Graphical Abstract

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Learning genetic perturbation effects with variational causal inference

Advances in sequencing technologies have enhanced the understanding of gene regulation in cells. In particular, Perturb-seq has enabled high-resolution profiling of the transcriptomic response to genetic perturbations at the single-cell level. This understanding has implications in functional genomics and potentially for identifying therapeutic targets. Various computational models have been developed to predict perturbational effects. While deep learning models excel at interpolating observed perturbational data, they tend to overfit in the lack of enough data and may not generalize well to unseen perturbations. In contrast, mechanistic models, such as linear causal models based on gene regulatory networks, hold greater potential for extrapolation, as they encapsulate regulatory information that can predict responses to unseen perturbations. However, their application has been limited to small studies due to overly simplistic assumptions, making them less effective in handling noisy, large-scale single-cell data. We propose a hybrid approach that combines a mechanistic causal model with variational deep learning, termed Single Cell Causal Variational Autoencoder (SCCVAE). The mechanistic model employs a learned regulatory network to represent perturbational changes as shift interventions that propagate through the learned network. SCCVAE integrates this mechanistic causal model into a variational autoencoder, generating rich, comprehensive transcriptomic responses. Our results indicate that SCCVAE exhibits superior performance over current state-of-the-art baselines for extrapolating to predict unseen perturbational responses. Additionally, for the observed perturbations, the latent space learned by SCCVAE allows for the identification of functional perturbation modules and simulation of single-gene knockdown experiments of varying penetrance, presenting a robust tool for interpreting and interpolating perturbational responses at the single-cell level.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

BASS: An R Package for Fitting and Performing Sensitivity Analysis of Bayesian Adaptive Spline Surfaces

In this work, we present the R package BASS as a tool for nonparametric regression. The primary focus of the package is fitting fully Bayesian adaptive spline surface (BASS) models and performing global sensitivity analyses of these models. The BASS framework is similar to that of Bayesian multivariate adaptive regression splines (BMARS) from Denison, Mallick, and Smith (1998), but with many added features. The software is built to efficiently handle significant amounts of data with many continuous or categorical predictors and with functional response. Under our Bayesian framework, most priors are automatic but these can be modified by the user to focus on parsimony and the avoidance of overfitting. If directed to do so, the software uses parallel tempering to improve the reversible jump Markov chain Monte Carlo (RJMCMC) methods used to perform inference. We discuss the implementation of these features and present the performance of BASS in a number of analyses of simulated and real data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

TBASS: A Robust Adaptation of Bayesian Adaptive Spline Surfaces

The R package TBASS is an extension of the BASS package created by Francom and Sansó (2019). The package is used to fit a Bayesian multivariate adaptive spline to a dataset that either follows a Student’s t-distribution or has outliers. Much of the framework for TBASS is adapted from the concepts of Bayesian Multivariate Adaptive Regression Splines (BMARS), specifically the work done by Denison, Mallick, and Smith (1998). The spline function is fit using a Reversible-Jump Markov Chain Monte Carlo algorithm,. By including this more robust generalization, a dataset with outliers can be accurately fit using the BMARS model, without the possibility of overfitting or variance inflation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Limited-view Cone Beam CT reconstruction using 3D Patch-based Supervised and Adversarial Learning [Slides]

We present a novel machine learning CNN architecture that can learn from limited data combined appropriately with physics and statistical priors (e.g., forward models and noise models). To address the limited availability of training data we adopt a 3D patch-based approach for our models. Patch-based learning is central to several image reconstruction methods and demands fewer training data than DL approaches, as a single data volume can be broken into several millions of overlapping 3D sub-volumes or patches. This creates a very large number of training sub-volumes from a limited number of overall image volumes. A 3D Generative Adversarial Networks (GAN) is then trained to remove artifacts at the sub-volume level. The combination of a sub-volume-based approach with DL allows us to exploit the richness of the latter in extracting and representing image features, while avoiding risks associated with overfitting due to limited training data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗