Multi-level structural damage characterization using sparse acoustic sensor networks and knowledge transferred deep learning
Not Available
36 MATERIALS SCIENCE↗
SEARCH · Search NASA
Results for “Sparsity of training data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Toward a Generalizable Prediction Model of Molten Salt Mixture Density with Chemistry‐Informed Transfer Learning
Optimally designing applications of molten salts requires knowledge of their thermophysical properties over a wide range of temperatures and compositions. There exist significant gaps in existing databases and this data can be challenging to experimentally measure due to high temperatures, salt corrosivity, and salt hygroscopicity. Existing databases have been used to create Redlich–Kister (RK) models for mixture density showing improved accuracy with respect to ideal mixing assumptions, but these models require subcomponent data measurements for each new system, therefore lacking generality. In order to address generalizability and data sparsity, a transfer learning procedure is proposed to train deep neural networks (DNNs) using a combination of semi‐empirical relationships (RK), data from the thermophysical arm of the molten salt thermal properties database and universal ab initio properties of component mixtures taken from the joint automated repository for various integrated simulations (JARVIS) classical force‐field inspired descriptors database to predict density in molten salts. Herein, it is shown that DNNs predict molten salt density with an r 2 over 0.99 and a mean absolute percentage error under 1%, outperforming alternative methods.
inorganic materials↗
Data Quality Monitoring for the Hadron Calorimeters Using Transfer Learning for Anomaly Detection
The proliferation of sensors brings an immense volume of spatio-temporal (ST) data in many domains, including monitoring, diagnostics, and prognostics applications. Data curation is a time-consuming process for a large volume of data, making it challenging and expensive to deploy data analytics platforms in new environments. Transfer learning (TL) mechanisms promise to mitigate data sparsity and model complexity by utilizing pre-trained models for a new task. Despite the triumph of TL in fields like computer vision and natural language processing, efforts on complex ST models for anomaly detection (AD) applications are limited. In this study, we present the potential of TL within the context of high-dimensional ST AD with a hybrid autoencoder architecture, incorporating convolutional, graph, and recurrent neural networks. Motivated by the need for improved model accuracy and robustness, particularly in scenarios with limited training data on systems with thousands of sensors, this research investigates the transferability of models trained on different sections of the Hadron Calorimeter of the Compact Muon Solenoid experiment at CERN. The key contributions of the study include exploring TL’s potential and limitations within the context of encoder and decoder networks, revealing insights into model initialization and training configurations that enhance performance while substantially reducing trainable parameters and mitigating data contamination effects.
47 OTHER INSTRUMENTATION↗
Evaluating the Efficacy of Conditional Variational Autoencoders in Generating Synthetic Single Nuclei RNA-Seq Data for Space Biology Research
Astronauts are subject to unique stressors during spaceflight, leading to changes in their cellular function. However, neither astronauts nor model organisms respond the same to spaceflight, and research implicates a contribution of omics components in differential responses. Understanding how gene expression affects astronaut health is critical for the success of long-term space missions, prompting interest in developing personalized predictive models leveraging artificial intelligence (AI) and machine learning (ML) techniques. Developing such models requires extensive data, which is challenging to obtain and share. This study explores the use of conditional variational autoencoders (CVAEs) to synthetically generate single-nuclei RNA-seq (snRNA-seq) data. CVAEs build on standard variational autoencoders (VAEs) by conditioning data generation on covariates like sample identity and mission parameters, enhancing the relevance of generated data for specific contexts. For our work, we built two CVAEs with varying degrees of sparsity to optimize both interpretability and generative power. We train and validate models on existing snRNA-seq data collected from the brain tissue of mice subjected to spaceflight conditions and their ground control counterparts. We evaluate model performance using statistical tests and visualizations to compare synthetic data to real data. We aim to demonstrate that these prototype CVAE architectures could be used in future space biology work and that this is a method worth further exploring.
Sarah Golts↗
Sparse measurement medical CT reconstruction using multi-fused block matching denoising priors
A major challenge for medical X-ray CT imaging is reducing the number of X-ray projections to lower radiation dosage and reduce scan times without compromising image quality. However these under-determined inverse imaging problems rely on the formulation of an expressive prior model to constrain the solution space while remaining computationally tractable. Traditional analytical reconstruction methods like Filtered Back Projection (FBP) often fail with sparse measurements, producing artifacts due to their reliance on the Shannon-Nyquist Sampling Theorem. Consensus Equilibrium, which is a generalization of Plug and Play, is a recent advancement in Model-Based Iterative Reconstruction (MBIR), has facilitated the use of multiple denoisers are prior models in an optimization free framework to capture complex, non-linear prior information. However, 3D prior modelling in a Plug and Play approach for volumetric image reconstruction requires long processing time due to high computing requirement. Instead of directly using a 3D prior, this work proposes a BM3D Multi Slice Fusion (BM3D-MSF) prior that uses multiple 2D image denoisers fused to act as a fully 3D prior model in Plug and Play reconstruction approach. Our approach does not require training and are thus able to circumvent ethical issues related with patient training data and are readily deployable in varying noise and measurement sparsity levels. In addition, reconstruction with the BM3D-MSF prior achieves similar reconstruction image quality as fully 3D image priors, but with significantly reduced computational complexity. We test our method on clinical CT data and demonstrate that our approach improves reconstructed image quality.
Hossain, Maliha [ORNL]↗
Bayesian Model Selection for Reducing Bloat and Overfitting in Genetic Programming for Symbolic Regression
When performing symbolic regression using genetic programming, overfitting and bloat can negatively impact generalizability and interpretability of the resulting equations as well as increase computation times. A Bayesian fitness metric is introduced and its impact on bloat and overfitting during population evolution is studied and compared to common alternatives in the literature. The proposed approach was found to be more robust to noise and data sparsity in numerical experiments, guiding evolution to a level of complexity appropriate to the dataset. Further evolution of the population resulted not in overfitting or bloat, but rather in slight simplifications in model form. The ability to identify an equation of complexity appropriate to the scale of noise in the training data was also demonstrated. In general, the Bayesian model selection algorithm was shown to be an effective means of regularization which resulted in less bloat and overfitting when any amount of noise was present in the training data.
G F Bomarito↗
Bayesian Model Selection for Reducing Bloat and Overfitting in Genetic Programming for Symbolic Regression
When performing symbolic regression using genetic programming, overfitting and bloat can negatively impact generalizability and interpretability of the resulting equations as well as increase computation times. A Bayesian fitness metric is introduced and its impact on bloat and overfitting during population evolution is studied and compared to common alternatives in the literature. The proposed approach was found to be more robust to noise and data sparsity in numerical experiments, guiding evolution to a level of complexity appropriate to the dataset. Further evolution of the population resulted not in overfitting or bloat, but rather in slight simplifications in model form. The ability to identify an equation of complexity appropriate to the scale of noise in the training data was also demonstrated. In general, the Bayesian model selection algorithm was shown to be an effective means of regularization which resulted in less bloat and overfitting when any amount of noise was present in the training data.
Uncertainty quantification↗
Multi-head physics-informed neural networks for learning functional priors and uncertainty quantification
In numerous applications, the integration of prior knowledge and historical information is essential, particularly for tasks requiring the solution of ordinary or partial differential equations (ODEs/PDEs) in data-sparse or noisy environments. For instance, achieving accurate solutions to time-dependent PDEs with limited initial condition measurements necessitates an effective strategy for embedding prior knowledge. Hard-parameter sharing architectures in neural networks (NNs) have demonstrated success in both traditional and scientific machine learning domains, facilitating the learning of informative representations. Here, in this study, we introduce a novel, yet efficient, method to enhance physics-informed neural networks (PINNs) by incorporating a multi-head structure that enables the learning of functional priors from both empirical data and governing physical laws. This prior information can then be used to address data sparsity and high-level noise in solving ODE/PDE problems with uncertainty quantification (UQ). The approach, termed Multi-Head PINN (MH-PINN), consists of a shared body NN and multiple head NNs, each corresponding to an individual PINN instance. Our framework for functional prior learning is carried out in two stages: (1) training the MH-PINNs to develop a shared body NN alongside multiple head NNs, and (2) employing these trained head NNs to estimate a prior distribution through a normalizing flow-based density estimator. The learned functional prior can then be applied as a regularization mechanism in deterministic contexts or as an informative prior within a Bayesian inference framework, aiding in the resolution of subsequent ODE/PDE tasks. We evaluate the efficacy of MH-PINNs across five benchmark problems, including a high-dimensional parametric PDE, all characterized by data sparsity or substantial noise levels. Our findings reveal that MH-PINNs deliver accurate solutions and robust UQ, demonstrating adaptability across a range of complex and challenging scenarios.
Bayesian inference↗
Efficient Dimension Reduction of Complex Three-dimensional CO2 Saturation using Deep Learning Models
In the domain of deep learning (DL), dimension reduction is crucial for enhancing training efficiency and mitigating overfitting, particularly when managing complex data such as three-dimensional (3D) saturation data. The 3D saturation data in the context of geological carbon storage (GCS) presents unique challenges due to its inherent sparsity and the abrupt transitions at plume boundaries, known as shock fronts. To address the challenges, we proposed a novel DL framework that integrates dimension reduction with advanced 3D reconstruction techniques. Our model leveraged latent variables derived from 2D average saturation data, offering a robust and efficient solution tailored to the intricate dynamics of 3D saturation fields. The proposed framework can extract the critical features of the high-dimensional data while reducing the variable numbers, which is more tractable for DL models and enhances the model robustness and accuracy. Therefore, it provides a novel approach for modeling and analyses in complex geological scenarios, which finds great potential applications in environmental monitoring and energy storage.
Wang, Hongsheng↗
Point cloud-based diffusion models for the Electron-Ion Collider
At high-energy collider experiments, generative models can be used for a wide range of tasks, including fast detector simulations, unfolding, searches of physics beyond the Standard Model, and inference tasks. In particular, it has been demonstrated that score-based diffusion models can generate high-fidelity and accurate samples of jets or collider events. This work expands on previous generative models in three distinct ways. First, our model is trained to generate entire collider events, including all particle species with complete kinematic information. We quantify how well the model learns event-wide constraints such as the conservation of momentum and discrete quantum numbers. We focus on the events at the future Electron-Ion Collider, but we expect that our results can be extended to proton-proton and heavy-ion collisions. Second, previous generative models often relied on image-based techniques. The sparsity of the data can negatively affect the fidelity and sampling time of the model. We address these issues using point clouds and a novel architecture combining edge creation with transformer modules called Point Edge Transformers. Third, we adapt the foundation model OmniLearn, to generate full collider events. This approach may indicate a transition toward adapting and fine-tuning foundation models for downstream tasks instead of training new models from scratch.
Araz, Jack Y. [Stony Brook Univ., NY (United State↗
An adaptive and stability-promoting layerwise training approach for sparse deep neural network architecture
This work presents a two-stage adaptive framework for progressively developing deep neural network (DNN) architectures that generalize well for a given training data set. In the first stage, a layerwise training approach is adopted where a new layer is added each time and trained independently by freezing parameters in the previous layers. We impose desirable structures on the DNN by employing manifold regularization, sparsity regularization, and physics-informed terms. We introduce a ε – δ – stability-promoting concept as a desirable property for a learning algorithm and show that employing manifold regularization yields a ε – δ stability-promoting algorithm. Further, we also derive the necessary conditions for the trainability of a newly added layer and investigate the training saturation problem. In the second stage of the algorithm (post-processing), a sequence of shallow networks is employed to extract information from the residual produced in the first stage, thereby improving the prediction accuracy. Numerical investigations on prototype regression and classification problems demonstrate that the proposed approach can outperform fully connected DNNs of the same size. Moreover, by equipping the physics-informed neural network (PINN) with the proposed adaptive architecture strategy to solve partial differential equations, we numerically show that adaptive PINNs not only are superior to standard PINNs but also produce interpretable hidden layers with provable stability. As a result, we also apply our architecture design strategy to solve inverse problems governed by elliptic partial differential equations.
42 ENGINEERING↗
Moltensaltpropnet
MoltenSaltPropnet is a physics-informed machine learning framework that aims to predict the thermophysical properties of molten fluoride and chloride salt mixtures, which are crucial for the design and safety of Generation IV molten salt reactors. The code processes data from the Molten-Salt Thermal Properties Database (MSTDB-TP) and the Janz compendium, converting critically evaluated correlations into fast, differentiable surrogate models for density, viscosity, thermal conductivity, and heat capacity across 448 distinct salt systems. The implementation consists of several key components: 1. Data Curation: The code parses and cleans the raw data, normalizing elemental mole fractions and extracting relevant regression coefficients for various thermophysical properties. 2. Feature Engineering: It generates fixed-length numerical descriptors that encapsulate the composition and temperature, incorporating polynomial interaction terms and dimensionality-reduction techniques to optimize model performance. 3. Coefficient Learning: Four different machine learning architectures are employed: a deep residual network (ResNet), a Kolmogorov–Arnold network (KAN), a sparsity-inducing neural network (SNN), and classical regression models. Each model learns to predict coefficients that define the temperature-dependent correlations for the thermophysical properties. 4. Property Reconstruction: The predicted coefficients are used to compute temperature-dependent property values, ensuring positivity and monotonic trends through a composite loss function that enforces physical constraints. 5. User Interface: An open-source web application enables users to filter the database, train task-specific models, and visualize the results, allowing for rapid exploration of candidate salt mixtures. MoltenSaltPropnet bridges the gap between limited experimental data and high-fidelity reactor simulations, providing a powerful tool for researchers in the field of molten salt reactors and advanced nuclear energy systems.
Retamales, Mauricio Eduardo Tano [Idaho National L↗
Deep nonparametric estimation of operators between infinite dimensional spaces
Learning operators between infinitely dimensional spaces is an important learning task arising in machine learning, imaging science, mathematical modeling and simulations, etc. This paper studies the nonparametric estimation of Lipschitz operators using deep neural networks. Non-asymptotic upper bounds are derived for the generalization error of the empirical risk minimizer over a properly chosen network class. Under the assumption that the target operator exhibits a low dimensional structure, our error bounds decay as the training sample size increases, with an attractive fast rate depending on the intrinsic dimension in our estimation. Our assumptions cover most scenarios in real applications and our results give rise to fast rates by exploiting low dimensional structures of data in operator estimation. We also investigate the influence of network structures (e.g., network width, depth, and sparsity) on the generalization error of the neural network estimator and propose a general suggestion on the choice of network structures to maximize the learning efficiency quantitatively.
97 MATHEMATICS AND COMPUTING↗
Rocket Launch Detection with Smartphone Audio and Transfer Learning
Rocket launches generate infrasound signatures that have been detected at great distances. Due to the sparsity of the networks that have made these detections, however, most signals are detected tens of minutes to hours after the rocket launch. In this work, a method of near-real-time detection of rocket launches using data from a network of smartphones located 10–70 km from launch sites is presented. A machine learning model is trained and tested on the open-access Aggregated Smartphone Timeseries of Rocket-generated Acoustics (ASTRA), Smartphone High-explosive Audio Recordings Dataset (SHAReD), and ESC-50 datasets, resulting in a final accuracy of 97% and a false positive rate of <1%. The performance and behavior of the model are summarized, and its suitability for persistent monitoring applications is discussed.
acoustics↗
Synergistic learning with multi-task DeepONet for efficient PDE problem solving
Multi-task learning (MTL) is an inductive transfer mechanism designed to leverage useful information from multiple tasks to improve generalization performance compared to single-task learning. It has been extensively explored in traditional machine learning to address issues such as data sparsity and overfitting in neural networks. In this work, we apply MTL to problems in science and engineering governed by partial differential equations (PDEs). However, implementing MTL in this context is complex, as it requires task-specific modifications to accommodate various scenarios representing different physical processes. To this end, we present a multi-task deep operator network (MT-DeepONet) to learn solutions across various functional forms of source terms in a PDE and multiple geometries in a single concurrent training session. We introduce modifications in the branch network of the vanilla DeepONet to account for various functional forms of a parameterized coefficient in a PDE. Additionally, we handle parameterized geometries by introducing a binary mask in the branch network and incorporating it into the loss term to improve convergence and generalization to new geometry tasks. Our approach is demonstrated on three benchmark problems: (1) learning different functional forms of the source term in the Fisher equation; (2) learning multiple geometries in a 2D Darcy Flow problem and showcasing better transfer learning capabilities to new geometries; and (3) learning 3D parameterized geometries for a heat transfer problem and demonstrate the ability to predict on new but similar geometries. Finally, our MT-DeepONet framework offers a novel approach to solving PDE problems in engineering and science under a unified umbrella based on synergistic learning that reduces the overall training cost for neural operators.
42 ENGINEERING↗
Adaptive Pressure Profile Method to Locate the Isolator Shock Train Leading Edge Given Limited Pressure Information
To maximize the performance of high-speed air-breathing engines, such as dual-mode scramjets, the streamwise location of the shock train leading edge (STLE) is ideally placed as far upstream in the isolator as possible while avoiding engine unstart. Thus, it is of interest to quantify and control the STLE location as the vehicle travels along its flight trajectory. The STLE location is typically quantified using wall static pressure measurements but there are often restrictions on the number and placement of transducers, thus reducing the accuracy and overall capability of STLE detection methods. In this work, the Adaptive Pressure Profile (APP) method is introduced to address such sparsity concerns. This method is data driven and does not heavily rely on prior information about the flow regime or engine model. Instead, the \mname method uses real-time pressure measurements from a small number of transducers to adaptively learn the isolator pressure profile. This adaptively-learned profile is fit to the pressure data at each time instance to estimate STLE location. The \mname method produces accurate estimates even when (1) the STLE location is not bounded by two or more transducers or (2) when the STLE location is between two transducers that are situated several duct heights apart. Data from two direct-connect isolator models are used to evaluate the accuracy of the \mname method and demonstrate its robustness for different back-pressure scenarios and transducer configurations.
Robin L Hunt↗
Adaptive Pressure Profile Method to Locate the Isolator Shock Train Leading Edge Given Limited Pressure Information
To maximize the performance of high-speed air-breathing engines, such as dual-mode scramjets, the streamwise location of the shock train leading edge (STLE) is ideally placed as far upstream in the isolator as possible while avoiding engine unstart. Thus, it is of interest to quantify and control the STLE location as the vehicle travels along its flight trajectory. The STLE location is typically quantified using wall static pressure measurements but there are often restrictions on the number and placement of transducers, thus reducing the accuracy and overall capability of STLE detection methods. In this work, the Adaptive Pressure Profile (APP) method is introduced to address such sparsity concerns. This method is data driven and does not heavily rely on prior information about the flow regime or engine model. Instead, the \mname method uses real-time pressure measurements from a small number of transducers to adaptively learn the isolator pressure profile. This adaptively-learned profile is fit to the pressure data at each time instance to estimate STLE location. The \mname method produces accurate estimates even when (1) the STLE location is not bounded by two or more transducers or (2) when the STLE location is between two transducers that are situated several duct heights apart. Data from two direct-connect isolator models are used to evaluate the accuracy of the \mname method and demonstrate its robustness for different back-pressure scenarios and transducer configurations.
Robin Hunt↗
Contextualizing Air Traffic Management Conversations using Natural Language Understanding
Efficient management of air traffic and mitigation of delays depend on extracting actionable information from unstructured data, such as dialogues from the Federal Aviation Administration’s (FAA’s) Air Traffic Control System Command Center (ATCSCC) telecons. This study presents a pipeline utilizing Natural Language Processing (NLP) methods for Intent Classification (IC) and Slot Filling (SF) to identify and extract Traffic Management Initiatives (TMIs) from aviation-specific dialogues. We leveraged DeBERTa, a pre-trained transformer model, and fine-tuned it to the nuances of the aviation domain. Despite challenges posed by annotation complexities, the IC model achieved promising results with a weighted average F1-score of 0.81. Our results are close to those of human annotators, which demonstrates the model’s strong alignment with human-level performance. The SF model also showed strong performance, achieving a weighted F1-score of 0.97, which demonstrates its effectiveness in accurately predicting key slots. Our analysis revealed limitations in handling less frequent intents and slot labels due to data sparsity, motivating future efforts to adopt joint IC-SF modeling and data augmentation strategies. This research highlights the potential of domain-specific NLP to streamline decision-making in the aviation industry and improve the management of TMIs.
Air Traffic Control Management↗