Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural network embeddings”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Track Seeding and Labelling with Embedded-space Graph Neural Networks

To address the unprecedented scale of HL-LHC data, the Exa.TrkX project is investigating a variety of machine learning approaches to particle track reconstruction. The most promising of these solutions, graph neural networks (GNN), process the event as a graph that connects track measurements (detector hits corresponding to nodes) with candidate line segments between the hits (corresponding to edges). Detector information can be associated with nodes and edges, enabling a GNN to propagate the embedded parameters around the graph and predict node-, edge- and graph-level observables. Previously, message-passing GNNs have shown success in predicting doublet likelihood, and we here report updates on the state-of-the-art architectures for this task. In addition, the Exa.TrkX project has investigated innovations in both graph construction, and embedded representations, in an effort to achieve fully learned end-to-end track finding. Hence, we present a suite of extensions to the original model, with encouraging results for hitgraph classification. In addition, we explore increased performance by constructing graphs from learned representations which contain non-linear metric structure, allowing for efficient clustering and neighborhood queries of data points. We demonstrate how this framework fits in with both traditional clustering pipelines, and GNN approaches. The embedded graphs feed into high-accuracy doublet and triplet classifiers, or can be used as an end-to-end track classifier by clustering in an embedded space. A set of post-processing methods improve performance with knowledge of the detector physics. Finally, we present numerical results on the TrackML particle tracking challenge dataset, where our framework shows favorable results in both seeding and track finding.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Embedded training of neural-network subgrid-scale turbulence models

We report that the weights of a deep neural-network model are optimized in conjunction with the governing flow equations to provide a model for subgrid-scale stresses in a temporally developing plane turbulent jet at Reynolds number Re 0 = 6000 . The objective function for training is first based on the instantaneous filtered velocity fields from a corresponding direct numerical simulation, and the training is by a stochastic gradient descent method, which uses the adjoint Navier-Stokes equations to provide the end-to-end sensitivities of the model weights to the velocity fields. In-sample and out-of-sample testing on multiple dual-jet configurations show that its required mesh density in each coordinate direction for prediction of mean flow, Reynolds stresses, and spectra is half that needed by the dynamic Smagorinsky model for comparable accuracy. The same neural-network model trained directly to match filtered subgrid-scale stresses, without the constraint of being embedded within the flow equations during the training, fails to provide a qualitatively correct prediction. The coupled formulation is generalized to train based only on mean-flow and Reynolds stresses, which are more readily available in experiments. The mean-flow training provides a robust model, which is important, though a somewhat less accurate prediction for the same coarse meshes, as might be anticipated due to the reduced information available for training in this case. The anticipated advantage of the formulation is that the inclusion of resolved physics in the training increases its capacity to extrapolate. This is assessed for the case of passive scalar transport, for which it outperforms established models due to improved mixing predictions.

42 ENGINEERING↗

NSDann2BS, a neutron spectrum unfolding code based on neural networks technology and two bonner spheres

In this work a neutron spectrum unfolding code, based on artificial intelligence technology is presented. The code called ''Neutron Spectrometry and Dosimetry with Artificial Neural Networks and two Bonner spheres'', (NSDann2BS), was designed in a graphical user interface under the LabVIEW programming environment. The main features of this code are to use an embedded artificial neural network architecture optimized with the ''Robust design of artificial neural networks methodology'' and to use two Bonner spheres as the only piece of information. In order to build the code here presented, once the net topology was optimized and properly trained, knowledge stored at synaptic weights was extracted and using a graphical framework build on the LabVIEW programming environment, the NSDann2BS code was designed. This code is friendly, intuitive and easy to use for the end user. The code is freely available upon request to authors. To demonstrate the use of the neural net embedded in the NSDann2BS code, the rate counts of {sup 252}Cf, {sup 241}AmBe and {sup 239}PuBe neutron sources measured with a Bonner spheres system.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Data-Augmented Modeling of Transition to Turbulence in Rayleigh-Taylor Mixing Layers

Existing turbulence models based on the Reynolds-averaged Navier–Stokes (RANS) equations are ill-equipped to represent transitional phenomena that are characteristic of problems such as those involved in Rayleigh-Taylor mixing layers. In this work, large eddy simulation(LES) data is used to augment and improve the predictive capability of RANS models. In-verse models are used to extract temporally varying embedded discrepancy functions from a number of LES datasets. These discrepancies are generalized into functional forms using neural networks and embedded inside the RANS solver to augment predictions. Specifically, the data-augmented models are demonstrated to provide improved predictions in Rayleigh-Taylor mixing problems at low Atwood numbers.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Deep Analysis Net with Causal Embedding for Coal-fired Power Plant Fault Detection and Diagnosis (DANCE4CFDD)

Fault detection and diagnosis is critical to power plant operation to ensure attaining high reliability while reducing operation cost. As more renewable power is introduced to the power grid, traditional fossil power plants take on the extra burden of excessive load cycling to compensate the generation variability from renewable power. Such load cycling will pose more reliability challenges to power plant operation. There are a number of challenges faced by today’s asset health management system in coal- fired (or gas) power plants: 1) high-dimensional nonlinear interaction among multiple time series measurements; 2) high measurement variance induced by operational conditions/modes; 3) variation among asset types and plant configurations; and 4) a small number of faulty events to learn from. To cope with these challenges, today’s fielded asset health management systems rely heavily on manual efforts from domain experts and hand-crafted features or rules based on domain knowledge. Despite its role in plant reliability, such a practice is costly and hinders its scalability and sustainability, particularly when a plant undergoes modifications. The objective of this project is to develop a novel end-to-end AI learning system that is trainable (i.e., the AI representation of a complex system behavior can be directly learned from properly labeled data) for accurate fault detection and root cause analysis. The ability to create a fault detection model directly from time series could alleviate the efforts associated with today’s asset management solution development. In the course of this project, we have achieved the following: Created an AI model development environment incorporating state-of-the-art neural network architectures for rapid model development and evaluation; Developed novel learning strategies for training of fault detection model; Developed special-purpose neural network architecture embedded with variable association graph aiming for better interpretability; Developed a learning strategy to leverage a small number of faulty events for enhanced fault detection capability; Conducted detailed experimental study based on public benchmark datasets and demonstrated the effectiveness of the proposed solution; and Validated the developed system with data from both a coal-fired plant boiler dynamic simulation model and real-world coal-fired power plant covering multiple asset and fault types. Overall, the project attained a technology readiness level of TRL 5 from TRL 2 at the beginning of the project.

20 FOSSIL-FUELED POWER PLANTS↗

Model-Form Epistemic Uncertainty Quantification for Modeling with Differential Equations: Application to Epidemiology

Modeling real-world phenomena to any degree of accuracy is a challenge that the scientific research community has navigated since its foundation. Lack of information and limited computational and observational resources necessitate modeling assumptions which, when invalid, lead to model-form error (MFE). The work reported herein explored a novel method to represent model-form uncertainty (MFU) that combines Bayesian statistics with the emerging field of universal differential equations (UDEs). The fundamental principle behind UDEs is simple: use known equational forms that govern a dynamical system when you have them; then incorporate data-driven approaches – in this case neural networks (NNs) – embedded within the governing equations to learn the interacting terms that were underrepresented. Utilizing epidemiology as our motivating exemplar, this report will highlight the challenges of modeling novel infectious diseases while introducing ways to incorporate NN approximations to MFE. Prior to embarking on a Bayesian calibration, we first explored methods to augment the standard (non-Bayesian) UDE training procedure to account for uncertainty and increase robustness of training. In addition, it is often the case that uncertainty in observations is significant; this may be due to randomness or lack of precision in the measurement process. This uncertainty typically manifests as “noisy” observations which deviate from a true underlying signal. To account for such variability, the NN approximation to MFE is endowed with a probabilistic representation and is updated using available observational data in a Bayesian framework. By representing the MFU explicitly and deploying an embedded, data-driven model, this approach enables an agile, expressive, and interpretable method for representing MFU. In this report we will provide evidence that Bayesian UDEs show promise as a novel framework for any science-based, data-driven MFU representation; while emphasizing that significant advances must be made in the calibration of Bayesian NNs to ensure a robust calibration procedure.

97 MATHEMATICS AND COMPUTING↗

Low power and privacy preserving sensor platform for occupancy detection

A low-cost, low-power, stand-alone sensor platform having a visible-range camera sensor, a thermopile array, a microphone, a motion sensor, and a microprocessor that is configured to perform occupancy detection and counting while preserving the privacy of occupants. The platform is programmed to extract shape/texture from images in spatial domain; motion from video in time domain; and audio features in frequency domain. Embedded binarized neural networks are used for efficient object of interest detection. The platform is also programmed with advanced fusion algorithms for multiple sensor modalities addressing dependent sensor observations. The platform may be deployed for (i) residential use in detecting occupants for autonomously controlling building systems, such as HVAC and lighting systems, to provide energy savings, (ii) security and surveillance, such as to detect loitering and surveil places of interest, (iii) analyzing customer behavior and flows, (iv) identifying high performing stores by retailers.

Velipasalar, Senem↗

A case study in contrastive learning information combination: Application to technical forensics of additive manufacturing filament source identification

Combination of information from disparate data sources into a single decision is a core challenge in many fields, including the field of technical forensics. Technical forensics (TF) utilizes technical characterization of questioned samples to determine properties of that sample; these properties are then used to infer information of forensic interest, such as provenance, age, or attribution. TF is utilized in traditional forensic applications, such as the attribution of material fragments from an explosive, and in nuclear forensic applications, such as the attribution of actinides which have been interdicted out of regulatory control. The challenge of combining information from disparate sources, described alternately by many terms including “Data Fusion” and “Data Integration”, is exacerbated in the technical forensics domain due to at least two factors: the challenge of interpreting each information source singularly, and the relatively small data set sizes available. Extensive literature exists attempting to combine technical forensics information sources, both in manual and automated processes. These attempts are often bespoke to the specific information sources (such as the bi-, tri-, or quad-isotope chart (Moody, Grant, and Hutcheon 2005)), with some emerging examples of simple early- and late- fusion (, respectively). Simultaneous to the information combination efforts described in the previous paragraph, the field of natural language processing attempted (and largely succeeded) in combining information from multiple non-technical information sources. The ecosystem of “multi-modal” language models, which can take text and images as input, and generate text and images as output, became large and diverse by 2025 (Khan et al. 2025). In a generalized sense, many of these methods are trained by learning neural networks which can convert raw text or images into a vector of numbers describing the text or image, hereafter called “embeddings” and the neural networks performing the conversion are called “embedders”. By using a separate embedder for text and images, finding coincident text and images (such as images with their captions), and optimizing the parameters of the embedders such that the embeddings for the text and the image are similar, the field has found a bridge between text and images (Girdhar et al. 2023). It is the contention of the authors of this report that this insight is not limited to text and images but instead can be extended to any modality which can be found coincidently. The subject of the rest of this report is the application of this method to example multi-modal technical forensic data. Some details about the data used in this report are not appropriate for this report, and are included in a companion report (PNNL-38669).

36 MATERIALS SCIENCE↗

SNNVis: Visualizing Graph Embedding of Evolutionary Optimization for Spiking Neural Networks

While Spiking Neural Networks (SNNs) show a lot of promise, it is difficult to optimize them because applying traditional gradient-based optimization techniques is difficult. Even though evolutionary algorithms (EAs) have been shown to promise to optimize SNNs, understanding the relationship between evolving the characteristics of SNNs and their performance to improve the optimization algorithm is challenging because of the complex characteristics and huge population size. We propose visual analytics with novel graph embedding for evolutionary SNNs to address the challenges. While existing graph embedding techniques have limitations in preserving the specific features of the nodes and edges, our approach maintains them. Also, we develop visual analytics for understanding the relationship between the network performance and the features of nodes and edges and exploring and analyzing the evolving SNNs to build insights into improving the EA.

Chae, Junghoon [ORNL] (ORCID:0000000206016746)↗

Classification of Wildfires from MODIS Data Using Neural Networks

Wildfires are destructive to both life and property, which necessitates an approach to quickly and autonomously detect these events from orbital observatories. This talk will introduce a neural network based approach for classifying wildfires in MODIS multispectral data, and will show how it could be applied to a constellation of low-cost CubeSats. The approach combines training a deep neural network on the ground using high performance consumer GPUs, with a highly optimized inference system running on a flight-proven embedded processor. Normally neural networks execute on hardware orders of magnitude more powerful than anything found in a space-based computer, therefore the inference system is designed to be performance even on the most modest of platforms. This implementation is able to be significantly more accurate than previous neural network implementations, while also approaching the accuracy of the state-of-the-art MODFIRE data products.

Artificial Intelligence↗

Method and system for training dynamic nonlinear adaptive filters which have embedded memory

Described herein is a method and system for training nonlinear adaptive filters (or neural networks) which have embedded memory. Such memory can arise in a multi-layer finite impulse response (FIR) architecture, or an infinite impulse response (IIR) architecture. We focus on filter architectures with separate linear dynamic components and static nonlinear components. Such filters can be structured so as to restrict their degrees of computational freedom based on a priori knowledge about the dynamic operation to be emulated. The method is detailed for an FIR architecture which consists of linear FIR filters together with nonlinear generalized single layer subnets. For the IIR case, we extend the methodology to a general nonlinear architecture which uses feedback. For these dynamic architectures, we describe how one can apply optimization techniques which make updates closer to the Newton direction than those of a steepest descent method, such as backpropagation. We detail a novel adaptive modified Gauss-Newton optimization technique, which uses an adaptive learning rate to determine both the magnitude and direction of update steps. For a wide range of adaptive filtering applications, the new training algorithm converges faster and to a smaller value of cost than both steepest-descent methods such as backpropagation-through-time, and standard quasi-Newton methods. We apply the algorithm to modeling the inverse of a nonlinear dynamic tracking system 5, as well as a nonlinear amplifier 6.

Rabinowitz, Matthew↗

Thermodynamic Consistent Neural Networks for Learning Material Interfacial Mechanics

For multilayer materials in thin substrate systems, interfacial failure is one of the most challenges. The traction-separation relations (TSR) quantitatively describe the mechanical behavior of a material interface undergoing openings, which is critical to understand and predict interfacial failures under complex loadings. However, existing theoretical models have limitations on enough complexity and flexibility to well learn the real-world TSR from experimental observations. A neural network can fit well along with the loading paths but often fails to obey the laws of physics, due to a lack of experimental data and understanding of the hidden physical mechanism. In this paper, we propose a thermodynamic consistent neural network (TCNN) approach to build a data-driven model of the TSR with sparse experimental data. The TCNN leverages recent advances in physics-informed neural networks (PINN) that encode prior physical information into the loss function and efficiently train the neural networks using automatic differentiation. We investigate three thermodynamic consistent principles, i.e., positive energy dissipation, steepest energy dissipation gradient, and energy conservative loading path. All of them are mathematically formulated and embedded into a neural network model with a novel defined loss function. A real-world experiment demonstrates the superior performance of TCNN, and we find that TCNN provides an accurate prediction of the whole TSR surface and significantly reduces the violated prediction against the laws of physics.

Zhang, Jiaxin↗

TEMPEST

This repository solves the problem of driver identification through vehicular and biometric data. Through an embedding-based approach and a novel loss function, we're able to distinguish between different drivers' behaviors. This also provides preprocessing for reproducibility of results.The code preprocesses vehicular data, trains neural networks, and outputs predictions.This code introduces a novel embedding-based neural network with a 91% rank-1 accuracy, as well as all code to reproduce training and results.

Musgrove, Kyle↗

Understanding Generative AI Content with Embedding Models

The construction of high-quality numerical features is critical to any quantitative data analysis. Feature engineering has been historically addressed by carefully hand-crafting data representations based on domain expertise. This work views the internal representations of modern deep neural networks (DNNs), called embeddings, as an implicit form of traditional feature engineering. For trained DNNs, we show that these embeddings can reveal interpretable, high-level concepts in unstructured sample data. We use these embeddings in natural language and computer vision tasks to uncover both inherent heterogeneity in the underlying data and human-understandable explanations for it. In particular, we find empirical evidence that there is inherent separability between real data and those generated from AI models.

Vargas, Max↗

A Graph-Net with Node Embeddings to Detect False Data Injection Attacks in Photovoltaic Systems

Distributed energy resources (DER) contribute to the operational stability of the larger power grid both at utility-scale as well as commercial and residential scales in aggregated forms. These DER in-turn are susceptible to increasing cyber threats. An adversary can plug into the same local network that a field photovoltaic (PV) system uses to interconnect its data loggers and inverters and manipulate certain measurements collected from the network or trick existing irradiance and inverter readings through false data injection attacks (FDIA). Control routines that rely on these measurements can propagate the false data, impacting critical decisions that result in a suboptimal operation or even cause intentional harm leading to inverter-tripping or unscheduled loads that need to be shed. To detect FDIA in PV systems, the paper introduces an attention-based graph neural network with node embeddings and applied it to a simple prototypical DC-coupled microgrid with PV, energy storage, and load. The algorithm shows a detection accuracy of up to 98.95%. The proposed FDIA detection technique will provide micro-grid operators with an effective method to safeguard their systems, guaranteeing the secure and reliable operation.

Parvez, Imtiaz [Utah Valley University]↗