Search NASA⌕ Search

SEARCH · Search NASA

Results for “Streaming Analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Near real-time streaming analysis of big fusion data

Experiments on fusion plasmas produce high-dimensional data time series with ever-increasing magnitude and velocity, but turn-around times for analysis of this data have not kept up. For example, many data analysis tasks are often performed in a manual, ad-hoc manner some time after an experiment. In this article, we introduce the Delta framework that facilitates near real-time streaming analysis of big and fast fusion data. By streaming measurement data from fusion experiments to a high-performance compute center, Delta allows computationally expensive data analysis tasks to be performed in between plasma pulses. This article describes the modular and expandable software architecture of Delta and presents performance benchmarks of individual components as well as of an example workflow. Focusing on a streaming analysis workflow where electron cyclotron emission imaging (ECEi) data is measured at KSTAR on the National Energy Research Scientific Computing Center's (NERSC's) supercomputer we routinely observe data transfer rates of about 4 Gigabit per second. In NERSC, a demanding turbulence analysis workflow effectively utilizes multiple nodes and graphical processing units and executes them in under 5 min. We further discuss how Delta uses modern database systems and container orchestration services to provide web-based real-time data visualization. For the case of ECEi data we demonstrate how data visualizations can be augmented with outputs from machine learning models. Here, by providing session leaders and physics operators, results of higher-order data analysis using live visualizations may make more informed decisions on how to configure the machine for the next shot.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Leveraging In-Network Computing and Programmable Switches for Streaming Analysis of Scientific Data

With the emergence of programmable network devices that match the performance of fixed function devices, several recent projects have explored in-network computing, where the processing that is traditionally done outside the network is offloaded to the network devices. In-network computing has typically been applied to network functions (e.g., load balancing, NAT, and DNS), caching, data reduction/aggregation, and coordination/consensus functions. In some cases it has been used to accelerate stream-processing tasks that involve small payloads and simple operations. In this work we focus on leveraging in-network computing for stream processing of scientific datasets with large payloads that require complex operations such as floating-point computations and logarithmic functions. We demonstrate in-network computing for a real-world scientific application performing streaming normalization of a 2-D image from a light source experiment. We discuss the challenges we encountered and potential approaches to address them.

Sankaran, Ganesh↗

Empirical Study of Focus-Plus-Context and Aggregation Techniques for the Visualization of Streaming Data

Analysis of streaming data often involves both real-time monitoring of incoming data as well as contextual awareness of data history. A focus-plus-context approach can support both goals, with variable levels of visual aggregation making it possible to provide a high level of detail for incoming and recent data while providing contextual information about recent history. Visual aggregation reduces data resolution in order to show the context of data over large periods of time within a limited display space. With a controlled experiment, we evaluated the effectiveness of different types of aggregation for four types of stream-analysis tasks. Overall, the results show that a focus-plus-context design has little negative impact on the ability to successfully monitor and analyze streaming data, making it possible to show longer periods of time than other approaches. However, visual aggregation can be problematic for trend recognition tasks. This research demonstrates how the effectiveness of the visualization depends on the specifics of the analysis task.

Ragan, Eric↗

Biomass Electrochemical Reactor for Upgrading Biorefinery Waste to Industrial Chemicals and Hydrogen

The goal of this project was to develop an electrochemical process for depolymerization of biorefinery lignin to industrial chemicals, most notably as replacements in resins. The project team consisted of Ohio University (lead organization), Hexion, Inc. and Lakehead University. There were six distinct project tasks, consisting of Initial Validation, Development of TiO 2 Nanowire-supported Ni-Co Electrocatalysts, Electrochemical Characterization of TiO 2 Nanowire-supported Ni-Co Electrocatalysts, Development of Electrochemical Flow Reactor for Production of Industrial Chemicals and Hydrogen, Product Formulations and Integrated Biorefinery Analysis. The project was divided into two budget periods. After intermediate validation near the end of Budget Period 1, the project was discontinued. This final report is intended to highlight major project objectives and results. Technical metrics were developed between the project team and the DOE validation team during the initial validation period. Critical technical metrics included conversion of lignin to products and product selectivity. Desired products were low molecular weight aromatic compounds in the 150-300 MW range. We targeted this class of compounds because of their potential to replace petroleum derivatives in resin formulations. Electrochemical oxidation of biorefinery lignin can lead to significant lignin depolymerization and conversion, with >40% of the lignin reacted. However, product stream analysis is extremely difficult. We were not able to adequately quantify the concentration of the primary oxidation products, or to determine with any degree of certainty what the product selectivity was. The project team initially proposed analytical techniques such as GC-MS to identify product distributions. However, due to 1) difficulty in extracting products into an organic phase suitable for GC analysis and 2) the wide range of product compounds with similar structures, GC-MS analysis was difficult to perform. The project team did have some success with NMR spectroscopic analysis, but not enough so to adequately identify product selectivity. By UV-vis spectroscopy and statistical analysis, we were able to confirm achieving the technical milestone for lignin conversion, and achieved at least 40% conversion by the electrochemical oxidation technique. However, we were not able to confirm product selectivity using GC-MS analysis, as originally postulated. Despite these difficulties, we were able to incorporate the product stream into a resin, and observed better resin synthesis behavior using the oxidized lignin versus the unoxidized lignin. However, no resin synthesized with oxidized biorefinery lignin met the commercial standards for resin quality. One potential problem with electrochemical oxidation of biomass is the competing oxygen evolution reaction (OER) at the reactor anode. We observed that, especially at higher anode potentials, the OER can consume up to 50% of the energy input to the system, meaning that much of the energy goes toward generation of an essentially worthless product (O 2 ). However, we also observed that the OER can be avoided almost entirely at lower potentials. These uncertainties made assigning a value to the electrochemical reactor product stream difficult. Any product generated electrochemically would have to be valuable enough to offset the additional cost of electricity incurred, including electricity to run the electrochemical reactor and the additional electricity that must be purchased because some of the lignin (which is currently burned in biorefineries to recover energy) is diverted to the electrochemical reactor. It is not clear at this time whether the product stream has sufficient economic value to offset these additional electricity costs. The two most promising areas for this project in terms of demonstrating economic feasibility are product stream analysis and resin synthesis. In particular, product stream analysis has proven to be difficult. With additional time, we would refine analytical techniques to better understand key product stream characteristics, such as aromatic compound content. Evaluating the product streams in resin synthesis would likely provide more information on the economic value of the product stream. If we could determine that value, then we would be able to better evaluate whether the electrochemical conversion approach is feasible. The details of this work are presented in the following final report. The report is organized into individual tasks for clarity, although all tasks were integrated and many occurred simultaneously.

09 BIOMASS FUELS↗

Experimental nitrogen and phosphorus enrichment stimulates multiple trophic levels of algal and detrital-based food webs: a global meta-analysis from streams and rivers

Anthropogenic increases in nitrogen (N) and phosphorus (P) concentrations can strongly influence the structure and function of ecosystems. Even though lotic ecosystems receive cumulative inputs of nutrients applied to and deposited on land, no comprehensive assessment has quantified nutrient-enrichment effects within streams and rivers. We conducted a meta-analysis of published studies that experimentally increased concentrations of N and/or P in streams and rivers to examine how enrichment alters ecosystem structure (state: primary producer and consumer biomass and abundance) and function (rate: primary production, leaf breakdown rates, metabolism) at multiple trophic levels (primary producer, microbial heterotroph, primary and secondary consumers, and integrated ecosystem). Our synthesis included 184 studies, 885 experiments, and 3497 biotic responses to nutrient enrichment. We documented widespread increases in organismal biomass and abundance (mean response = +48%) and rates of ecosystem processes (+54%) to enrichment across multiple trophic levels, with no large differences in responses among trophic levels or between autotrophic or heterotrophic food-web pathways. Responses to nutrient enrichment varied with the nutrient added (N, P, or both) depending on rate versus state variable and experiment type, and were greater in flume and whole-stream experiments than in experiments using nutrient-diffusing substrata. Generally, nutrient-enrichment effects also increased with water temperature and light, and decreased under elevated ambient concentrations of inorganic N and/or P. Overall, increased concentrations of N and/or P altered multiple food-web pathways and trophic levels in lotic ecosystems. Finally, our results indicate that preservation or restoration of biodiversity and ecosystem functions of streams and rivers requires management of nutrient inputs and consideration of multiple trophic pathways.

59 BASIC BIOLOGICAL SCIENCES↗

Comprehensive Analysis of Streaming and Shutdown Dose Rate Experiments at JET with ORNL Fusion Neutronics Workflows

Current experimental fusion systems and conceptual designs of fusion pilot plants (FPPs) are growing in complexity and size. Several radiation metrics are crucial to the safe operation of fusion machines, including neutron flux streaming through openings and the shutdown dose rate (SDDR). Most current designs of advanced experimental fusion systems—and the most probable candidates for FPPs—are based on the tokamak concept, which is prone to neutron streaming through the myriad openings needed for diagnostic and support systems. SDDR is caused by decay gamma rays from radionuclides that become activated by neutrons during the operation of a fusion system that use deuterium-deuterium (DD), tritium-tritium, or deuterium-tritium plasma. Because computational tools have become essential for determining these radiation metrics, they must be validated against reliable and applicable experimental data. Experiments at the Joint European Torus (JET) provide a unique source of experimental data for validating computational tools and nuclear data used to determine SDDR and neutron fluxes in streaming-dominated geometries. Here, this paper presents the comprehensive analysis of the high-performance DD JET SDDR, and streaming experiments performed using Oak Ridge National Laboratory (ORNL) fusion workflows. The computational results were compared with experimental results that consist of online SDDR measurements with ionization chambers and neutron fluence streaming measurements using thermoluminescent detectors. The ratio of calculated-to-experimental SDDR values ranges from 0.6 to 2.5, and the streaming results range from 0.5 to 8.0. Future work will include analyzing the JET 2021 DTE2 campaign alongside the integration of the Shift Monte Carlo transport code into all ORNL fusion neutronics workflows.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

GraphTango: A Hybrid Representation Format for Efficient Streaming Graph Updates and Analysis

Abstract Streaming graph processing performs batched updates and analytics on a time-evolving graph. The underlying representation format of the graph largely determines the throughputs of these updates and analytics phases. Existing representation formats usually employ variations of hash tables or adjacency lists. However, a recent study showed that the adjacency-list-based approaches perform poorly on heavy-tailed graphs, and the hash table-based approaches suffer on short-tailed graphs. We propose GraphTango, a hybrid representation format that provides excellent update and analytics throughput regardless of the graph’s degree distribution. GraphTango dynamically switches among three different formats based on a vertex’s degree: (i) Low-degree vertices store the edges directly with the neighborhood metadata, confining accesses to a single cache line, (2) Medium-degree vertices use adjacency lists, and (3) High-degree vertices use hash tables as well as adjacency lists. In this case, the adjacency list provides fast traversal during the analytics phase, while the hash table provides constant-time lookups during the update phase. We further optimized the performance by designing an open-addressing-based hash table that fully utilizes every fetched cache line. In addition, we developed a thread-local lock-free memory pool that allows fast growing/shrinking of the adjacency lists and hash tables in a multi-threaded environment. We evaluated GraphTango with the help of the SAGA-Bench framework and compared it with four other representation formats: Stinger, Degree-aware Robin Hood Hashing, and two adjacency list-based formats with different workload balancing scheme. On average, GraphTango provides 4.5x higher insertion throughput, 3.2x higher deletion throughput, and 1.1x higher analytics throughput over the next best format. Furthermore, we integrated GraphTango with the state-of-the-art graph processing frameworks DZiG and RisGraph. Compared to the vanilla DZiG and vanilla RisGraph , [ GraphTango + DZiG ] and [ GraphTango + RisGraph ] reduces the average batch processing time by 2.3x and 1.5x, respectively.

Ahmed, Alif↗

Data and scripts associated with a manuscript on a meta-analysis synthesizing stream biogeochemical response to wildfires across space and time (v2)

This data package is associated with the publication “Catchment characteristics modulate the influence of wildfires on nitrate and dissolved organic carbon in lotic systems across space and time: A meta-analysis” submitted to Global Biogeochemical Cycles (Cavaiani et al. 2025). This study uses meta-analytical techniques to evaluate the effect of wildfire on in-stream responses in burned and unburned watersheds. The study aims to provide additional insight into the range of responses and net influences that wildfires have on hydro-biogeochemistry across broad spatial scales, burn extents, and the persistence of water-quality change. This study compiles data and metadata from 18 total publications that includes 1) surface water geochemistry data (dissolved organic carbon; nitrate), 2) climate classifications, 3) year of the wildfire, 4) the time lag between when the fire occurred and when the sampling occurred, and 5) study design of the publication. In total, this meta-analysis draws data that spans 8 climate guilds, 3 biomes, 62 watersheds, and 20 unique wildfires. See Sites_meta_data.csv for citations of the papers used in this meta-analysis. All R scripts and the associated data can also be found on GitHub at This data package was originally published in March 2024. It was updated in April 2025 (v2; new and modified files). See the change history section in the readme for more details. This data package contains five primary folders that include the following: (1) inputs; (2) output for analysis; (3) initial plots; (4) R scripts; and (5) GIS data. The data package also contains a data dictionary (dd) that provides column header definitions and a file-level metadata (flmd) file that describes every file. The “inputs” folder contains a list of all publications identified during the formal web search and an indication of whether each publication was included in the final analysis. Additionally, it includes site-level metadata, catchment characteristics, and GIS data for all publications included in the final analysis. The “Output_for_analysis” folder contains all data frames and figures generated from each R script used for additional data analysis. The “initial_plots” folder includes all exploratory figures that will be included in a supplemental and figures that will be submitted with the manuscript for publication. The “R_scripts” folder contains the scripts that perform all the data manipulations, statistical analyses, and plots. The “gis_data” folder includes shape files for each fire included in this meta-analysis. This data package contains the following file types: csv, pdf, jpeg, cpg, dbf, prj, shp, shp.ea.iso.xml, shp.iso.xml, shx.

54 ENVIRONMENTAL SCIENCES↗

Phoenix: A Scalable Streaming Hypergraph Analysis Framework

We present Phoenix, a scalable hypergraph analytics framework for data analytics and knowledge discovery that was implemented on the leadership class computing platforms at Oak Ridge National Laboratory (ORNL). Our software framework comprises a distributed implementation of a streaming server architecture which acts as a gateway for various hypergraph generators/external sources to connect. Phoenix has the capability to utilize diverse hypergraph generators, including HyGen, a very large-scale hypergraph generator developed by ORNL. Phoenix incorporates specific algorithms for efficient data representation by exploiting hidden structures of the hypergraphs. Our experimental results demonstrate Phoenix’s scalable and stable performance on massively parallel computing platforms. Phoenix’s superior performance is due to the merging of high-performance computing with data analytic.

Kurte, Kuldeep↗

System, algorithm, and method using short pulse interrogation with neutrons to detect and identify matter

A system, method, and algorithm for detecting and identifying special nuclear material (SNM) (including fissionable material), explosives, or drugs is disclosed. This material detection system relies on active interrogation of material using a short, intense neutron pulse, and characterization of the resulting prompt gamma response in two short time windows, the first concurrent with and the second immediately following the end of the neutron pulse, optionally subject to a total transit delay time of the neutrons from the neutron source to the target and from the target to the gamma detector. A high data rate analysis system implements data stream analysis and statistical correlations of the two short time window gamma responses to rapidly detect or identify SNM, explosives, or drugs. The duration of the neutron pulse is so short that minimal dose results. Various shields help to minimize the background signal falling on the gamma detector to improve sensitivity.

Derzon, Mark S.↗

Dynamic Approach to Dependency Analysis in Human Reliability Analysis: Application in a Stream Generator Tube Rupture Scenario

Dependency analysis in human reliability analysis (HRA) is a method of adjusting the failure probability of a given action by considering the impact of the action preceding it. It plays a role in reasonably accounting for human actions in the context of probabilistic safety assessments (PSAs), preventing PSA results from being estimated too optimistically based on the HRA results. Nevertheless, the existing dependency methods present a couple of challenges in that the quantification approaches rarely explain the adjustment of human error probabilities (HEPs). For this reason, the authors’ previous research has pointed out challenges of the existing dependency approaches and conceptually, theoretically proposed a performance shaping factor (PSF)-based dynamic dependency analysis method for HRA in order to complement the existing dependency methods. The current paper explores the latest version of the method and guidance for applying it to a steam generator tube rupture (SGTR) scenario.

99 GENERAL AND MISCELLANEOUS↗

Refining HPCToolkit for application performance analysis at exascale

As part of the US Department of Energy’s Exascale Computing Project (ECP), Rice University has been refining its HPCToolkit performance tools to better support measurement and analysis of applications executing on exascale supercomputers. To efficiently collect performance measurements of GPU-accelerated applications, HPCToolkit employs novel non-blocking data structures to communicate performance measurements between tool threads and application threads. To attribute performance information in detail to source lines, loop nests, and inlined call chains, HPCToolkit performs parallel analysis of large CPU and GPU binaries involved in the execution of an exascale application to rapidly recover mappings between machine instructions and source code. To analyze terabytes of performance measurements gathered during executions at exascale, HPCToolkit employs distributed-memory parallelism, multithreading, sparse data structures, and out-of-core streaming analysis algorithms. To support interactive exploration of profiles up to terabytes in size, HPCToolkit’s hpcviewer graphical user interface uses out-of-core methods to visualize performance data. The result of these efforts is that HPCToolkit now supports collection, analysis, and presentation of profiles and traces of GPU-accelerated applications at exascale. These improvements have enabled HPCToolkit to efficiently measure, analyze and explore terabytes of performance data for executions using as many as 64K MPI ranks and 64K GPU tiles on ORNL’s Frontier supercomputer. HPCToolkit’s support for measurement and analysis of GPU-accelerated applications has been employed to study a collection of open-science applications developed as part of ECP. This paper reports on these experiences, which provided insight into opportunities for tuning applications, strengths and weaknesses of HPCToolkit itself, as well as unexpected behaviors in executions at exascale.

Adhianto, Laksono↗

Impact of cyberattacks on safety and stability of connected and automated vehicle platoons under lane changes

Connected and automated vehicles (CAVs) offer a huge potential to improve the operations and safety of transportation systems. However, the use of smart devices and communications in CAVs introduce new risks. CAVs would leverage vehicle to vehicle (V2V) and vehicle to infrastructure (V2I) communication, thus providing additional system access points compared to traditional systems. Automation makes these systems more vulnerable and increases the consequences of cyberattacks. This study utilizes an infrastructure-based communication platform consisting of cooperative adaptive cruise control and lane control advisories developed by the authors to perform cyber risk assessment of CAVs. The study emulates three types of cyberattacks (message falsification, dedicated denial of service, and spoofing attacks) in a representative traffic environment consisting of multiple CAV platoons and lane change events to analyze the safety and stability impacts of the cyberattacks. Simulation experiments using VISSIM reveals that traffic stream and CAV string is unstable under all three types of cyberattacks. The worst case is represented by the message falsification attack. Increases in volatility are observed over a no attack case, with variations increasing by an average of 43%–51% along with an increase of over 3000 crash conflicts. Similarly, lane change crash conflicts are observed to be more severe compared to rear end crash conflicts, showing a higher probability of severe injuries. Further, the case of slight cyberattack on a single CAV also creates significant disruption in the traffic stream. Analysis of variance (ANOVA) reveals the statistical significance of the results. Furthermore, these results pave the way for future design of secure systems from a monitoring perspective.

97 MATHEMATICS AND COMPUTING↗

Multi-objective sizing and dispatch for building thermal and battery storage towards economic and environmental synergy

The role of building thermal and battery storage is pivotal in advancing smart cities and achieving sustainability goals through effective energy management. Despite their significance, there are several limitations in the sizing approach and value stream analysis with various objectives for their widespread adoption in buildings. This work proposes a flexible and scalable multi-objective optimization framework for optimal sizing and dispatch of building thermal and battery storage, addressing conflicting objectives simultaneously using mixed-integer linear programming. The weighted-sum method is adapted, combining multiple objectives into a single function. The two-stage procedure iterates over different weights, generating optimal solutions and forming the Pareto front. Case studies are performed to assess the energy, economic, and environmental benefits of building energy storage systems for a large office building in three climate locations. The results demonstrate that the proposed framework efficiently determines optimal sizing and dispatch strategies, addressing the balance between economic viability and emission reduction. The dynamic relationship between time-of-use energy charges and emission factors leads to significantly different strategies based on whether economic or environmental concerns are prioritized. This research enhances our understanding of the benefits of TES and BES systems in buildings, providing valuable guidance to stakeholders.

25 ENERGY STORAGE↗

Beyond Ohm's law: Bernoulli effect and streaming in electron hydrodynamics

Recent observations of nonlocal transport in ultraclean two-dimensional materials raised the tantalizing possibility of accessing hydrodynamic correlated transport of a many-electron state. However, it has been pointed out that nonlocal transport can also arise from impurity scattering rather than interaction. At the crux of the ambiguity is the focus on linear effects, i.e., Ohm's law, which cannot easily differentiate among different modes of transport. Furthermore we propose experiments that can reveal rich hydrodynamic features in the system by tapping into the nonlinearity of the Navier-Stokes equation. Three experiments we propose will each manifest a unique phenomenon that is well known in classical fluids: the Bernoulli effect, Eckart streaming, and Rayleigh streaming. Analysis of known parameters confirms that the proposed experiments are feasible and the hydrodynamic signatures are within reach of graphene-based devices. Experimental realization of any one of the three phenomena will provide a stepping stone to formulating and exploring the notions of nonlinear electron fluid dynamics with an eye to celebrated examples from classical nonlaminar flows, e.g., pattern formation and turbulence.

2-dimensional systems↗

High Resolution Fission Fragment Spectroscopy with Superconducting Microcalorimeters

Sub-1 AMU mass determination is important for determining fission yields and neutron multiplicity, which are necessary inputs for fission models. Fission models can improve spent nuclear waste stream analysis and nuclear fuel burnup determination. Here, to achieve this goal, we have used superconducting microcalorimeter detectors to directly measure the energy of fission fragments from the spontaneous fission of 252 Cf. With a fiber coupled LED pulser setup we demonstrate that we can reach a relative energy resolution of 0.1% and better for photon pulses with energies above 60 MeV. This instrument, in conjunction with time-of-flight (TOF) measurement, would allow for sub-1 atomic mass unit (AMU) mass determination of fission fragments in a future beamline application.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗