SEARCH · Search NASA

Results for “stream networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Differentiable vertex fitting for jet flavor tagging

This work explores the use of differentiable programming to integrate domain knowledge, in the form of domain specific software, into neural networks to develop scientific machine learning systems. We propose a differentiable vertex fitting algorithm that estimates the crossing point of multiple curves. In the high energy physics setting, these curves are defined by particle equations of motion and the crossing point represents the origin of particle production. This differentiable vertex fitting algorithm can be seamlessly integrated into neural networks, and we show its utility and efficacy in the high energy physics application of the classification of jets, i.e., collimated streams of particles in particle detectors whose originating parent particle we aim to classify. We demonstrate how differentiable vertex fitting can be integrated into larger transformer-based models for jet flavor tagging and show improvements in heavy flavor jet classification when compared to baseline models. Published by the American Physical Society 2024

Smith, Rachel E. C. (ORCID:0000000335851262)↗

DOE OSTI2023

The Distributed Grid Sensing Services (DGSS) platform: A scalable and decentralized sensor dissemination network

In this paper, a scalable, decentralized sensor-oriented, communication blueprint will be introduced. The proposed architecture has been specifically designed to support the mass deployment of sensors while enabling data consumers to access the underlying data streams using a distributed systems approach. With the proposed approach, sensors and data clients can be decoupled from the physical communication infrastructure and migrated into a modern, software-defined infrastructure ecosystem that can be configured to suit the end application’s demands. In its current iteration, the blueprint defines the interactions, processes, and expected outcomes that each individual component within the architecture must fulfill in order to support the overall ecosystem’s needs. Within this work, such assemble of services is referred the databus, and represents the heart of the Distributed Grid Sensing Services (DGSS) architecture, a multi-year project that aims to design, implement and test a dedicated sensor dissemination network that can support the needs of the extended grid state. It is our expectation, that the proposed blueprint presented in this paper will serve as a future guide to our implementation efforts

Sebastian Cardenas, David J.↗

DOE OSTI2024

Integrating multi-modal remote sensing, deep learning, and attention mechanisms for yield prediction in plant breeding experiments

In both plant breeding and crop management, interpretability plays a crucial role in instilling trust in AI-driven approaches and enabling the provision of actionable insights. The primary objective of this research is to explore and evaluate the potential contributions of deep learning network architectures that employ stacked LSTM for end-of-season maize grain yield prediction. A secondary aim is to expand the capabilities of these networks by adapting them to better accommodate and leverage the multi-modality properties of remote sensing data. In this study, a multi-modal deep learning architecture that assimilates inputs from heterogeneous data streams, including high-resolution hyperspectral imagery, LiDAR point clouds, and environmental data, is proposed to forecast maize crop yields. The architecture includes attention mechanisms that assign varying levels of importance to different modalities and temporal features that, reflect the dynamics of plant growth and environmental interactions. The interpretability of the attention weights is investigated in multi-modal networks that seek to both improve predictions and attribute crop yield outcomes to genetic and environmental variables. This approach also contributes to increased interpretability of the model's predictions. The temporal attention weight distributions highlighted relevant factors and critical growth stages that contribute to the predictions. The results of this study affirm that the attention weights are consistent with recognized biological growth stages, thereby substantiating the network's capability to learn biologically interpretable features. Accuracies of the model's predictions of yield ranged from 0.82-0.93 R 2 ref in this genetics-focused study, further highlighting the potential of attention-based models. Further, this research facilitates understanding of how multi-modality remote sensing aligns with the physiological stages of maize. The proposed architecture shows promise in improving predictions and offering interpretable insights into the factors affecting maize crop yields, while demonstrating the impact of data collection by different modalities through the growing season. By identifying relevant factors and critical growth stages, the model's attention weights provide valuable information that can be used in both plant breeding and crop management. The consistency of attention weights with biological growth stages reinforces the potential of deep learning networks in agricultural applications, particularly in leveraging remote sensing data for yield prediction. To the best of our knowledge, this is the first study that investigates the use of hyperspectral and LiDAR UAV time series data for explaining/interpreting plant growth stages within deep learning networks and forecasting plot-level maize grain yield using late fusion modalities with attention mechanisms.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2026

FFTSF: Revisiting Sub-Seasonal Streamflow Forecasting with Simple Feedforward Network

Accurate short-to-subseasonal streamflow forecasts are vital for water management, including flood preparedness, drought mitigation, hydropower scheduling, and ecosystem protection. However, extending a forecast beyond a few days remains challenging due to complexity of hydrological processes. While recent self-attention based transformer architectures such as iTransformer have gained traction in time-series forecasting, these models suffer from several critical limitations: (1) significant computational overhead that scales quadratically with sequence length, (2) vulnerability to overfitting on limited hydrological datasets, (3) degraded performance on long-horizon forecasts due to attention decay, and (4) excessive architectural complexity that hampers interpretability and operational deployment. In this study, we propose a simple Feedforward Time Series Forecasting (FFTSF) network that directly addresses these limitations through its lightweight architecture and long-range forecasting capabilities. We evaluate FFTSF across 178 USGS stream gauges spanning diverse climate regimes by forecasting lead times of 1-, 7-, 14-, and 30-days. Our results demonstrate that FFTSF achieves competitive performance at short lead times (NSE of 0.778 for 1-day forecasts) while substantially outperforming complex baselines at longer forecast period, achieving the highest NSE (0.271) at 30-day forecasts with greater robustness and stability. For 30-day forecasts, FFTSF achieves a 71% improvement over NLinear, 57% improvement over DLinear and 12% improvement over the computationally intensive iTransformer while requiring fewer computational resources. Our findings reveal that architectural complexity is not necessary for hydrological forecasting, demonstrating that well-designed simple models can outperform attention mechanisms for subseasonal streamflow forecasting. The computational efficiency and consistent long-range performance of FFTSF make it suitable for water management applications where reliable extended forecasts are essential.

Krishnan Kutty Ambika, Anukesh [ORNL] (ORCID:00000↗

DOE OSTI2023

AmeriFlux FLUXNET-1F US-xDL NEON Dead Lake (DELA)

This is the AmeriFlux Management Project (AMP) created FLUXNET-1F version of the carbon flux data for the site US-xDL NEON Dead Lake (DELA). This is the FLUXNET version of the carbon flux data for the site US-xDL NEON Dead Lake (DELA) produced by applying the standard ONEFlux (1F) software. Site Description - The Dead Lake site is just northeast of Demopolis, AL near the Black Warrior River. The site is in Major Land Resource Area (MLRA) 133A–Southern Coastal Plains, near the boundary of MLRA 135A–Alabama and Mississippi Blackland Prairie. Surrounded by woody wetlands and evergreen forest, DELA sits on the flood plain steps and stream terraces of an inside meander of the Black Warrior River in Greene Co., AL.

Network), NEON (National Ecological Observatory↗

DOE OSTI2026

A Brief Survey of Data Streaming Technologies

Streaming data is data that is emitted at variable volumes in a continuous, incremental manner with the goal of low-latency processing often at a different physical location. Network infrastructure is used to facilitate the connection between data sources and sinks, and must be robust to handle the requirements of the workflow. The U.S. Department of Energy Office of Science (DOE SC) a federal agency supporting fundamental scientific research for energy and the Nation’s largest supporter of basic research in the physical sciences. DOE SC has the responsibility for operating $\mathbf{1 0}$ National Laboratories, and 28 scientific user facilities supporting advanced supercomputers, particle accelerators, large x-ray light sources, neutron scattering sources, and other specialized facilities for nanoscience and genomics. This paper investigates the state of streaming data workfows, and details some of the approaches to this challenging problem.

Kissel, Ezra↗

DOE OSTI2022

Analysis of the Source Physics Experiment Dry Alluvium Geology Ball Drop Sources

Two balls were dropped approximately 150 m down an open borehole as part of an investigation into the aftershocks from a 51 ton TNT-equivalent chemical explosion. The ball drop sources, a concrete-filled basketball and a bowling ball, were intended to mimic falling debris within an underground explosion borehole and cavity. Falling debris or sloughing of the borehole and cavity walls is thought to be one of the sources of explosion aftershocks. The signals from the ball drop sources were recorded by the same seismic network that monitored the aftershocks of the chemical explosion, so the ball drop recordings were used as templates to search through the continuous stream of aftershocks and detect correlated signals as a proportion of all aftershocks. The search resulted in only four similar event recordings out of the thousands of detected aftershocks. A synthetic test showed that the small, high-frequency ball drop templates were limited to producing highly correlated detections for events occurring in a small region around the impact location at 150 m depth. The high-frequency templates were necessary since the small ball drop signals were only above the noise at frequencies greater than 10 Hz. Unfortunately, the impact location is where the borehole was filled with sand in preparation for the subsequent chemical explosion in the test series, making it 150 m above the explosion created cavity centered at 300 m depth. This puts the ball drop templates 150 m away from the cavity, where many more events from falling debris would be expected. Further synthetic tests confirmed events emanating from the cavity 150 m below the impact site would have correlated with the lower frequency template and been detected had the template been usable below 10 Hz. Therefore, we conclude that the four detected events were most probably falling debris within the open borehole that impacted the floor at 150 m depth just as the ball drop sources had, and that this type of explosion aftershock mechanism comprises less than 1% of all explosion aftershocks.

58 GEOSCIENCES↗

DOE OSTI2023

Vehicle Localization in 3D World Coordinates Using Single Camera at Traffic Intersection

Optimizing traffic control systems at traffic intersections can reduce the network-wide fuel consumption, as well as emissions of conventional fuel-powered vehicles. While traffic signals have been controlled based on predetermined schedules, various adaptive signal control systems have recently been developed using advanced sensors such as cameras, radars, and LiDARs. Among these sensors, cameras can provide a cost-effective way to determine the number, location, type, and speed of the vehicles for better-informed decision-making at traffic intersections. In this research, a new approach for accurately determining vehicle locations near traffic intersections using a single camera is presented. For that purpose, a well-known object detection algorithm called YOLO is used to determine vehicle locations in video images captured by a traffic camera. YOLO draws a bounding box around each detected vehicle, and the vehicle location in the image coordinates is converted to the world coordinates using camera calibration data. During this process, a significant error between the center of a vehicle’s bounding box and the real center of the vehicle in the world coordinates is generated due to the angled view of the vehicles by a camera installed on a traffic light pole. As a means of mitigating this vehicle localization error, two different types of regression models are trained and applied to the centers of the bounding boxes of the camera-detected vehicles. The accuracy of the proposed approach is validated using both static camera images and live-streamed traffic video. Based on the improved vehicle localization, it is expected that more accurate traffic signal control can be made to improve the overall network-wide energy efficiency and traffic flow at traffic intersections.

47 OTHER INSTRUMENTATION↗

DOE OSTI2021

Anomaly Detection, Localization and Classification using Drifting Synchrophasor Data Streams

With ongoing automation and digitization of the electric power system, several Phasor Measurement Units(PMUs) have been deployed for monitoring and control. PMU data can have multiple anomalies, and many of the researchers in the past have concentrated on training machine/deep learning algorithms offline for anomaly detection over PMU data (i.e., not in real time). These machine/deep learning algorithms, when trained offline on a sample rather than a population of the dataset, fail to consider the dynamic behavior of the power grid in real-time, resulting in low accuracy. Considering the dynamic behavior of the power grid (e.g., change in load, generation, distributed energy resources (DERs) switching, network, controls), the definition of data anomalies varies in time and requires online training. A fundamental challenge is to enable online (i.e., real-time) training of machine/deep learning algorithms for anomaly detection over streaming PMU data. While machine/deep learning is often desirable to manage data streams, training a deep learning algorithm over streaming PMU data is nontrivial due to changes in data statistics caused by dynamic streaming data. This paper proposes PMUNET: a novel device-level deep learning-based data-driven approach for anomaly detection, localization, and classification over streaming PMU data, using online learning and multivariate data-drift detection algorithm .Two variants of PMUNET, Dynamic data Change Driven Learning (DCDL) and Continuity Driven Learning (CDL), are proposed and compared. DCDL aims to train the deep learning algorithm whenever the definition of anomaly changes due to the power grid dynamics. On the other hand, CDL continuously trains the deep learning algorithm over the PMU data-stream. The experimental results verify that DCDL outperforms CDL and other efficient anomaly detection methods over multiple events such as faults and load/ generator/capacitor/DERs variations/switching for IEEE 14 and 39 Bus test system as well as real PMU industrial data. The result verifies that DCDL variant of PMUNET improves over existing approach with a gain of 2% - 10% in terms of accuracy, false-positive rate, and false-negative rate.

adversarial deep learning↗

DOE OSTI2020

My vehicle is a data mine

In this talk we explore how analysis of vehicle data provides information of individual vehicle behaviors, information of other vehicles in the flow of traffic, and insights into the behavior of drivers. Over the last two decades traditional passenger vehicles have been transformed from integrated two-port electrical nodes to cyber-physical systems of communicating computational nodes whose individual state and control variables are shared on a standard controller area network (CAN) bus. As driver assistance systems have crept into vehicles as safety features, driver behaviors can be observed through analysis of the data streams on the CAN bus as these new nodes communicate with one another. The properties of these data streams, as well as architectures and approaches to gather the data, are important to consider when drawing conclusions on the relevance of the data in making decisions at varying levels of the information hierarchy. We will demonstrate several technical challenges associated with these data collection processes, as well as preliminary results that demonstrate application relevance of the data to behavior, traffic, and systems domains.

42 ENGINEERING↗

DOE OSTI2023

Real-Time Monitoring of Fracture Dynamics with a Contrast Agent-Assisted Electromagnetic Method

In collaboration with the Advanced Energy Consortium, our team has previously demonstrated that the placement of electrically active proppants (EAPs) in a hydraulic fracture surveyed by electromagnetic (EM) methods can enhance the imaging of the stimulated reservoir volumes during hydraulic fracturing. That work culminated in constructing a well-characterized EAP-filled fracture anomaly at the Devine field pilot site (DFPS). In subsequent laboratory studies, we observed that the electrical conductivity of our EAP correlates with changes in pressure, salinity, and flow. Thus, we postulated that the EAP could be used as an in-situ sensor for the remote monitoring of these changes in previously EAP-filled fractures. This paper presents our latest field data from the DFPS to demonstrate such correlations at an intermediate pilot scale. We conducted surface-based EM surveys during freshwater (200 ppm) and saltwater (2,500 ppm) slug injections while running surfaced-based EM surveys. Simultaneously, we measured the following: 1) bottomhole pressure and salinity in five monitoring wells; 2) injection rate using high-precision data loggers; 3) distributed acoustic sensors in four monitoring wells; and 4) tiltmeter data on the survey area. We demonstrated that injections into an EAP-filled fracture could be successfully coupled with real-time electric field measurements on the surface, leading to remote monitoring of dynamic changes within the EAP-filled fracture. Furthermore, by comparing the electrical field traces with the bottomhole pressure, flow rate, and salinity, we concluded that the observed electric field in our study is influenced by fracture dilation and flow rate. Salinity effect was observed when saltwater was injected. EM simulations solely based on assumptions of fracture conductivity changes during injection did not reproduce all of the measured electric field magnitudes. Preliminary estimates showed that including streaming potential in our geophysical model may be needed to reduce the simulation mismatch. The methods developed and demonstrated during this study will lead to a better understanding of the extent of fracture networks, formation stress states, fluid leakoff and invasion, characterizations of engineered fracture systems, and other applications where monitoring subsurface flow tracking is deemed important.

02 PETROLEUM↗

DOE OSTI2021

Mitigating Catastrophic Forgetting in Deep Learning in a Streaming Setting Using Historical Summary

Recent advancements in scientific equipment and the adaptation of electronics and the Internet of Things (IoT) in our everyday lives resulted in large and complex data production at a high rate. Making meaningful and timely knowledge discovery at a modest cost from this big data is difficult for computing power and storage limitations. Training deep learning models incrementally in a streaming setting can help us with overcoming these limitations. However, in a well-known phenomenon named catastrophic forgetting, incrementally trained models increasingly perform poorly on the past data. To mitigate catastrophic forgetting in training in a streaming setting, we propose constructing a historical summary over time and use the summary with newly arrived data during incremental training. We propose various data summarization techniques such as random sampling, micro clustering, coreset computation, and Auto Encoders to counteract catastrophic forgetting. We built a pipeline for incremental training with a historical summary for training deep learning models for streaming data. We demonstrate the effectiveness of historical summary in mitigating catastrophic forgetting using three case studies involving three different deep learning applications: an Artificial Neural Network (ANN) for classification task on MNIST dataset, a language model (RNN-LM) on the WikiText2 dataset, and a Convolutional Neural Network (CNN), ResNet50 to classify the ImageNet dataset. Through the training of the models, we observe that catastrophic forgetting is evident in ANN and CNN but not in an RNN. For the first task, our method recovers up to 47.9% lost accuracy due to catastrophic forgetting. For the third task, the historical summary recovers classification accuracy by up to 25%. For the second task, though there is not proof of catastrophic forgetting, the training performance (PPL) improves by up to 26% with historical summary.

Dash, Sajal↗

DOE OSTI2025

Investigation of the Mechanical Degradation of Zinc-Based Cold Spray Coatings for Steel Pipelines

adhesion↗

DOE OSTI2026

Intelligent Experiments Through Real-time AI: Fast Data Processing and Autonomous Detector Control for sPHENIX and Future EIC Detectors (Final Report)

The overall vision of this project was to integrate real-time artificial intelligence (AI) directly into the data acquisition and detector-control systems of nuclear physics experiments, including both fast online event selection and an autonomous detector-control feedback loop. The work carried out under the award focused on the fast online event-selection half of that vision: the efficient recording of low-momentum heavy-flavor (HF) hadron decays in proton-proton collisions at the sPHENIX experiment at the Relativistic Heavy Ion Collider (RHIC)—an observable that requires fast tracking and topological trigger selection not previously demonstrated at RHIC, and that is essential for QCD studies at future facilities such as the Electron-Ion Collider (EIC). The autonomous detector-control (GPU-based feedback) component named in the project title remained a design concept and was not implemented under this award. The Massachusetts Institute of Technology (MIT) group led the offline simulation and data processing needed to train the machine-learning (ML) models, the translation of trained models to Field-Programmable Gate Array (FPGA) firmware using the hls4ml framework, and the physics validation of heavy-flavor reconstruction. Over the award period, the team developed and hardware-tested the principal components of an AI-based heavy-flavor trigger on simulated and recorded sPHENIX tracker data: a software Bipartite Graph Attention Network (BiGAT) trigger model reaching > 95% signal efficiency at 99% background rejection; an FPGA-native hit clusterizer matching the offline clustering; smaller networks synthesized to FPGA within the required sub-10 µs latency; and an assembled decoder–clusterizer–inference firmware chain exercised on the FELIX readout board. A complete, fully integrated hardware demonstrator was not finished within the award period. This report documents the project goals, the MIT group’s contributions, the technical accomplishments, and the outlook toward applications at the future EIC ePIC detector.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2025

Investigation of the Mechanical Degradation of Zinc-Based Cold Spray Coatings for Steel Pipelines

Internal corrosion in wet natural gas is a big challenge in the oil and gas industry due to corrosive constituents such as carbon dioxide (CO2), hydrogen sulfide (H2S), other forms of sulfur, and water in the gas stream. To mitigate internal corrosion, zinc-based cold spray coatings were designed for use in natural gas pipelines to increase the lifespan of the pipeline network. However, one of the requirements in designing internal coatings is the resistance of the coatings to mechanical forces applied on the pipeline's internal wall during pigging operations. These forces are primarily compressive and shear/friction forces. This study examines material properties that must be considered when evaluating mechanical considerations. To determine the viability of the developed coatings with respect to the shear/friction forces, the shear adhesion and wear resistance of the materials have been evaluated. Shear adhesion testing was performed with modified clevises to determine the shear stress required to cause adhesion failure at the coating/substrate interface. Scanning electron microscopy was utilized to characterize the interfaces after failure. Wear testing was performed on coatings and pipeline materials utilizing pin-on-drum testing. These tests display the stress limitations and wear that the coatings can tolerate from pigging.

mechanical degradation↗

DOE OSTI2020

Seismology in the Cloud: A New Streaming Workflow

Data-intensive research in seismology is experiencing a recent boom, driven in part by large volumes of available data and advances in the growing field of data science. However, there are significant barriers to processing large data volumes, such as long retrieval times from data repositories, complex data management, and limited computational resources. New tools and platforms have reduced the barriers to entry for scientific cluster computing, including the maturation of the commercial cloud as an accessible instrument for research. Here, we build a customized research cluster in the cloud to test a new workflow for large-scale seismic analysis, in which data are processed as a stream (retrieved on-the-fly and acted upon without storing), with data from the Incorporated Research Institutions for Seismology Data Management Center. We use this workflow to deploy a spectral peak detection algorithm over 5.6 TB of compressed continuous seismic data from 2074 stations of the USArray Transportable Array EarthScope network. Using a 50-node cluster in the cloud, we completed the noise survey in 80 hr, with an average data throughput of 1.7 GB per minute. By varying cluster sizes, we find the scaling of our analysis to be sublinear, due to a combination of algorithmic limitations and data center response times. The cloud-based streaming workflow represents an order-of-magnitude increase in acquisition and processing speed compared to a traditional download-store-process workflow, and offers the additional benefits of employing a flexible, accessible, and widely used computing architecture. It is limited, however, due to its reliance on Internet transfer speeds and data center service capacity, and may not work well for repeated analyses or those for which even higher data throughputs are needed. These research applications will require a new class of cloud-native approaches in which both data and analysis are in the cloud.

58 GEOSCIENCES↗

DOE OSTI2021

A Framework for International Collaboration on ITER Using Large-Scale Data Transfer to Enable Near-Real-Time Analysis

The global nature of the ITER project along with its projected ~ petabyte per day data generation presents a unique challenge, but also an opportunity for the fusion community to rethink, optimize, and enhance our scientific discovery process. Recognizing this, collaborative research with computational scientists was undertaken over the past several years to create a framework for large-scale data movement across wide-area networks (WANs), to enable global near-real time analysis of fusion data. This would broaden the available computational resources for analysis/simulation, and increase the number of researchers actively participating in experiments. An official demonstration of this framework for fast, large data transfer and real-time analysis was carried out between the KSTAR tokamak in Daejeon, Korea and PPPL in Princeton, USA. Streaming large data transfer, with near real-time movie creation and analysis of the KSTAR Electron Cyclotron Emission imaging (ECEI) data, was performed using the I/O framework ADIOS, and comparisons made at PPPL with simulation results from the XGC1 code. These demonstrations were made possible utilizing an optimized network configuration at PPPL, which achieved over 8.8 Gbps (88% utilization) in throughput tests from NFRI to PPPL. This demonstration showed the feasibility for large-scale data analysis of KSTAR data, and provides a nascent framework to enable use of globally distributed computational and personnel resources in pursuit of scientific knowledge from the ITER experiment.

43 PARTICLE ACCELERATORS↗

DOE OSTI2022

Exploring Physics of Ferroelectric Domain Walls in Real Time: Deep Learning Enabled Scanning Probe Microscopy

The functionality of ferroelastic domain walls in ferroelectric materials is explored in real-time via the in situ implementation of computer vision algorithms in scanning probe microscopy (SPM) experiment. The robust deep convolutional neural network (DCNN) is implemented based on a deep residual learning framework (Res) and holistically nested edge detection (Hed), and ensembled to minimize the out-of-distribution drift effects. The DCNN is implemented for real-time operations on SPM, converting the data stream into the semantically segmented image of domain walls and the corresponding uncertainty. Further the pre-defined experimental workflows perform piezoresponse spectroscopy measurement on thus discovered domain walls, and alternating high- and low-polarization dynamic (out-of-plane) ferroelastic domain walls in a PbTiO 3 (PTO) thin film and high polarization dynamic (out-of-plane) at short ferroelastic walls (compared with long ferroelastic walls) in a lead zirconate titanate (PZT) thin film is reported. This work establishes the framework for real-time DCNN analysis of data streams in scanning probe and other microscopies and highlights the role of out-of-distribution effects and strategies to ameliorate them in real time analytics.

36 MATERIALS SCIENCE↗