Search NASA⌕ Search

SEARCH · Search NASA

Results for “Streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

MODAQ-BB (Modular Offshore Data Acquisition System - Blackbox) [SWR-24-72]

MODAQ-BB is a compact, rapid deployment data acquisition system that can withstand water depths up to 400m. Codenamed "BlackBox" (or simply BB), since its initial purpose was to track marine assets and record vital data streams that could later be recovered in the event of a mishap - much like a traditional black box used in aviation and shipping, the name has stuck. MODAQ BlackBox is a battery-operable microcontroller platform with internal inertial sensing, GPS, satellite communications, and additional I/O (input/output) support in a depth-rated pressure enclosure. Since BB is self-contained and relatively compact, it can be quickly deployed with minimal effort. The enclosure can be clamped to a tube (such as part of a railing) or mast in a location with unobstructed view of the sky using the available clamp accessory or a common hose clamp. While BB is designed to operate unattended, users can configure what data are uploaded and the frequency of satellite transmissions. Once data are uploaded, they can be relayed to an email distribution list, the MODAQ:Web operational dashboard, or a custom destination. BB has found utility in the National Laboratory of the Rockies' (NLR's) Waterpower projects beyond its original vision and has been configured and successfully deployed in more traditional data-gathering applications where a simple, battery-operated solution was indicated. As part of the MODAQ family, BB fills a space in the spectrum of missions that can be supported that were previously impractical using the traditional MODAQ hardware architecture due to factors such as weight, size, and cost.

Raye, Robert↗
DOE OSTI2026

The high level trigger and express data production at STAR

To meet the demands of the Beam Energy Scan phase-II (BES-II) program, the STAR experiment at the Relativistic Heavy Ion Collider (RHIC) developed a dual real-time framework consisting of a High Level Trigger (HLT) and an Express Data Production system (xProduction). The HLT operates online within the Data Acquisition (DAQ) chain on a dedicated multi-core CPU cluster with the option to offload compute-intensive kernels to Xeon Phi coprocessors. It uses parallelized algorithms, such as the Cellular Automaton (CA) Track Finder, to perform rapid tracking, vertexing, and event filtering. This allows it to select events of interest in real time and provide immediate feedback on detector and beam conditions. In contrast, the xProduction workflow runs concurrently and independently of the DAQ loop. It applies near offline-quality calibration and reconstruction within hours of data collection. The xProduction input is the express data stream, whose content can be enriched by HLT trigger/priority selections under DAQ/HLT resource constraints, and it uses the STAR calibration/conditions framework, incorporating online calibration/QA information when available. This enables early preliminary physics analysis, including the reconstruction of rare signals, such as hyperons and hypernuclei. It also provides collaboration-wide access to analysis-ready datasets. Together, the HLT and xProduction systems form a complementary architecture: the HLT performs online event selection while the xProduction chain delivers high-quality results within a short amount of time. This integrated framework has enabled the prompt reconstruction of the $^5_Λ$ He hypernucleus with high statistical significance and the efficient processing of hundreds of millions of heavy-ion collision events. In conclusion, its demonstrated scalability and robustness establish a model for future high-luminosity experiments requiring both online event filtering and rapid access to analysis-quality data.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

SIDM Concerto: Compilation and Data Release of Self-interacting Dark Matter Zoom-in Simulations

We present SIDM Concerto: 14 cosmological zoom-in simulations in cold dark matter (CDM) and self-interacting dark matter (SIDM) models based on the Symphony and Milky Way-est suites. SIDM Concerto includes one Large Magellanic Cloud– (LMC-) mass system (host mass ∼10 11 M ⊙ ), two Milky Way (MW) analogs (∼10 12 M ⊙ ), two group-mass hosts (∼10 13 M ⊙ ), and one low-mass cluster (∼10 14 M ⊙ ). Each host contains ≈2 × 10 7 particles and is run in CDM and one or more strong, velocity-dependent SIDM models. Our analysis of SIDM (sub)halo populations over seven subhalo mass decades reveals that (1) the fraction of core-collapsed isolated halos and subhalos peaks at a maximum circular velocity corresponding to the transition of the SIDM cross section from a v −4 to v 0 scaling; (2) SIDM subhalo mass functions are suppressed by ≈50% relative to CDM in LMC, MW, and group-mass hosts but are consistent with CDM in the low-mass cluster host; (3) subhalos’ inner density profile slopes, which are more diverse in SIDM than in CDM, are sensitive to both the amplitude and shape of the SIDM cross section. Our simulations provide a benchmark for testing SIDM predictions with astrophysical observations of field and satellite galaxies, strong lensing systems, and stellar streams. Data products are publicly available at doi:10.5281/zenodo.14933624.

dark matter↗
DOE OSTI2025

Homomorphic data compression for real time photon correlation analysis

The construction of highly coherent X-ray sources, combined with next-generation detectors that are larger and faster, has enabled new research opportunities across the scientific landscape. Among the techniques that benefit most from these advancements is X-ray photon correlation spectroscopy (XPCS), where faster acquisition unlocks the ability to study faster dynamics within samples. However, faster acquisition on larger detectors also introduces unprecedented challenges for online data processing and offline data storage. Such challenges are particularly prominent for XPCS, where real time analyses require simultaneous calculation of all the previously acquired data in the time series. We present a homomorphic compression scheme to effectively reduce the computational time and memory space required for XPCS analysis. Leveraging similarities in the mathematical expression between a matrix-based compression algorithm and the correlation calculation, our approach allows direct operation on the compressed data without their decompression. The offline compression scheme extends storage capacity by a factor of 40 while preserving key features in the lossy compressed data. Meanwhile, the online compression scheme reduces the computational time to below 1 ms, enabling real time calculation of the correlation functions at kHz framerate. Our demonstration of a homomorphic compression of scientific data provides an effective solution to the big data challenge at coherent light sources. Beyond the example shown in this work, the framework can be extended to facilitate real-time operations directly on a compressed data stream for other techniques.

36 MATERIALS SCIENCE↗
DOE OSTI2025

A virtual Frisch-grid geometry-based CZT gamma detector for in-field radioisotope identification

Here, we present a Virtual Frisch-Grid geometry-based CZT gamma detector developed for identifying different radioisotopes over an energy range from a few keV up to 2 MeV, and useful for efficient characterization of CZT crystals. The detector is built with a 3 x 3 matrix of CZT crystals, each measuring approximately 6 mm x 6 mm x 15 mm. The charge generated within the sensor’s active volume is read out via an anode connected directly to the AVG3_Dev integrated circuit. A current signal induced by charge drift is collected on side pads of the crystals, enabling reconstruction of a 3D interaction position. This paper discusses the design, development, and performance of the standalone, mobile detector system, which integrates the AVG3_Dev readout IC developed at Brookhaven National Laboratory, a high-speed FPGA-based with per-channel digital signal processing, and embedded system capabilities. The device is compact, battery-powered, and supports wireless data streaming, making it suitable for field operations for radioisotope identification.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Integrating Characteristic Arctic Vegetation in a Land Surface Model Improves Representation of Carbon Dynamics Across a Tundra Landscape

Arctic warming is altering vegetation and carbon dynamics with global implications, yet Earth System Model (ESM) predictions in the Arctic remain highly uncertain, in part due to historically limited data for model parameterization and validation. As such, ESMs typically represent Arctic ecosystems in an oversimplified manner. Recently, nine plant functional types (PFTs) designed to realistically represent tundra vegetation were integrated into the Energy Exascale Earth System Model (E3SM) Land Model (ELM) and parameterized using plot-scale observations from a single site. Additional evaluation was needed to determine their transferability across the Arctic. Here, in this study, we evaluated whether refined representation of tundra vegetation improved model accuracy by conducting spatially explicit 100 × 100 m resolution ELM simulations on Alaska's Seward Peninsula. Simulations with the default two-PFT configuration and with the nine Arctic-specific PFTs were benchmarked against observations of net ecosystem exchange, gross primary production, and aboveground biomass from multiple data streams including an eddy covariance flux tower, flux chambers, and aircraft and unoccupied aerial system hyperspectral remote sensing. Evaluation revealed that Arctic-specific PFT simulations produced more realistic landscape-level carbon exchanges, and better captured observed heterogeneity in biomass and productivity, explaining 60%–70% of spatial variance (R 2 = 0.6–0.7) compared to just 12%–18% (R 2 = 0.12–0.18) with the default configuration. However, the refined model failed to reproduce observed aboveground biomass for highly productive alder-willow communities, requiring further evaluation of carbon allocation parameterizations for tall shrubs that are increasingly expanding across tundra landscapes. Our results demonstrate that enhanced representation of vegetation heterogeneity boosts predictive understanding of tundra carbon dynamics, facilitating regional to pan-Arctic model and remote-sensing scaling.

Murphy, Bailey A. [Oak Ridge National Laboratory (↗
DOE OSTI2024

Demystifying group-4 polyolefin hydrogenolysis catalysis: Gaseous propane hydrogenolysis mechanism over the same catalysts

A kinetic/mechanistic investigation of gaseous propane hydrogenolysis over the single-site heterogeneous polyolefin depolymerization catalysts AlS/ZrNp 2 and AlS/HfNp 2 (AlS = sulfated alumina, Np = neopentyl), is use to probe intrinsic catalyst properties without the complexities introduced by time- and viscosity-dependent polymer medium effects. In a polymer-free automated plug-flow catalytic reactor, propane hydrogenolysis turnover frequencies approach 3,000 h −1 at 150 °C. Both catalysts exhibit approximately linear relationships between rate and [H 2 ] at substoichiometric [H 2 ] with rate law orders of 0.66 ± 0.09 and 0.48 ± 0.07 for Hf and Zr, respectively; at higher [H 2 ], the rates approach zero-order in [H 2 ]. Reaction orders in [C 3 H 8 ] and [catalyst] are essentially zero-order under all conditions, with the former implying rapid, irreversible alkane binding/activation. This rate law, activation parameter, and DFT energy span analysis support a scenario in which [H 2 ] is pivotal in one of two plausible and competing rate-determining transition states—bimolecular metal-alkyl bond hydrogenolysis vs. unimolecular β-alkyl elimination. The Zr and Hf catalyst activation parameters, ΔH ‡ = 16.8 ± 0.2 kcal mol −1 and 18.2 ± 0.6 kcal mol −1 , respectively, track the relative turnover frequencies, while ΔS ‡ = −19.1 ± 0.8 and −16.7 ± 1.4 cal mol −1 K −1 , respectively, imply highly organized transition states. These catalysts maintain activity up to 200 °C, while time-on-stream data indicate multiday activities with an extrapolated turnover number ~92,000 at 150 °C for the Zr catalyst. This methodology is attractive for depolymerization catalyst discovery and process optimization.

03 NATURAL GAS↗
DOE OSTI2025

Modeling the differential rate for signal interactions in coincidence with noise fluctuations or large rate backgrounds

The characteristic energy of a relic dark matter interaction with a detector scales strongly with the putative dark matter mass. Consequently, experimental search sensitivity at the lightest masses will always come from interactions whose size is similar to noise fluctuations and low energy backgrounds in the detector. In this paper, we correctly calculate the net change in measured differential event rate due to low rate signal interactions that overlap in time with noise and backgrounds, accounting for both periods of time when the signal is coincident with noise/backgrounds and for the decreased amount of time in which only noise/backgrounds occur and we show that the introduction of random simulated signal events into the continuous raw data stream (a form of “salting”) provides a correct and practical implementation to estimate this net linear signal sensitivity. Unfortunately this does not apply to the situation with non negligible pileups as we show through explicit examples. Consequently, this exclusion technique should be complemented by less sensitive techniques that can separately exclude large rate, high pileup signal parameter space. An analysis threshold above the Gaussian-like noise component of the measured differential rate spectrum should also likely produce conservative limits. Though previous light mass dark matter searches did not correctly account for decreased background only live time effects in their signal sensitivity, conservative analysis choices likely protected their published limits from being nonconservative.

Li, Xinran↗
DOE OSTI2025

Automation for Grid Interconnected Laboratory Emulation

As computational capabilities improve, digital twins are becoming vital for evaluating equipment realistically in laboratories. This paper outlines a digital twin architecture for the power grid, employing electromagnetic transient (EMT) simulation alongside real-time simulation of power hardware and hierarchical control systems. EMT simulation occurs on a high-performance computing server for scalability. Additionally, the paper describes a workflow and real-time data streaming software facilitating connectivity among EMT simulation, hierarchical control systems, and power hardware. This software enables automated equipment connectivity in the laboratory for realistic evaluations, aiding in identifying necessary upgrades for both equipment control systems and the power grid.

Marthi, Phani Ratna Vanamali [ORNL] (ORCID:0000000↗
DOE OSTI2025

Free-Space Optical Channel Turbulence Prediction: A Machine Learning Approach

Channel turbulence is a formidable obstacle for freespace optical (FSO) communication. Anticipation of turbulence levels is highly important for mitigating disruptions but has not been demonstrated without dedicated, auxiliary hardware. We show that machine learning (ML) can be applied to raw FSO data streams to rapidly predict channel turbulence levels with no additional sensing hardware. FSO was conducted through a controlled channel in the lab under six distinct turbulence levels, and the efficacy of using ML to classify turbulence levels was examined. ML-based turbulence level classification was found to be > 98% accurate with multiple ML training parameters. Classification effectiveness was found to depend on the timescale of changes between turbulence levels but converges when turbulence stabilizes over about a one minute timescale.

Zobaer Islam, Md [School of Electrical and Compute↗
DOE OSTI2024

Enabling DER visibility using a distributed dissemination network

The electrical grid is currently undergoing a series of rapid transformational changes that have resulted in the introduction of new actors and operational schemes that have fragmented the data and control planes. To help address the issue, this paper describes the implementation of a sensor-oriented, distributed data dissemination network that seeks to eliminate data silos. The implementation is based on the DGSS architecture previously described in [1]. The developed product seeks to facilitate the seamless integration of multi-operator, multi-origin, multi-domain sensor data by using a distributed systems approach. The proposed solution decouples the sensor’s data streams from the application-specific infrastructure and migrates them into a software-defined databus that can be configured to suit the end application’s demands. To further validate DGSS capabilities, a DER oriented use case has been developed.

Sensor Dissemination Networks, Enhanced DER visibi↗
DOE OSTI2024

Real-Time Ethernet Interface for NSTX-U’s Thomson Scattering Diagnostic (2023)

Here, the multipoint Thomson scattering (MPTS) diagnostic system at the National Spherical Torus Experiment Upgrade (NSTX-U) facility is undergoing an upgrade to operate in real-time and interface with the plasma control system (PCS) for NSTX-U. Previous prototyping efforts have shown that spectral analysis and rapid calculations of electron temperature and density are possible on a real-time Linux machine when using up to a 100-Hz laser pulse repetition rate. A remaining challenge was transferring the real-time data to NSTX-U’s PCS, which utilizes the front panel data port (FPDP) protocol. The original proposed method was to convert the real-time data into analog values, but a new solution was developed to keep the output format digital by using an Ethernet controller with a field-programmable gate array (FPGA). This article focuses on a new input module that has been developed to accept incoming user datagram protocol (UDP) packets sent over Ethernet, convert into FPDP format, and integrate into the existing data stream under NSTX-U’s real-time framework.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Proximal remote sensing: an essential tool for bridging the gap between high‐resolution ecosystem monitoring and global ecology

Summary A new proliferation of optical instruments that can be attached to towers over or within ecosystems, or ‘proximal’ remote sensing, enables a comprehensive characterization of terrestrial ecosystem structure, function, and fluxes of energy, water, and carbon. Proximal remote sensing can bridge the gap between individual plants, site‐level eddy‐covariance fluxes, and airborne and spaceborne remote sensing by providing continuous data at a high‐spatiotemporal resolution. Here, we review recent advances in proximal remote sensing for improving our mechanistic understanding of plant and ecosystem processes, model development, and validation of current and upcoming satellite missions. We provide current best practices for data availability and metadata for proximal remote sensing: spectral reflectance, solar‐induced fluorescence, thermal infrared radiation, microwave backscatter, and LiDAR. Our paper outlines the steps necessary for making these data streams more widespread, accessible, interoperable, and information‐rich, enabling us to address key ecological questions unanswerable from space‐based observations alone and, ultimately, to demonstrate the feasibility of these technologies to address critical questions in local and global ecology.

Plant Sciences↗
DOE OSTI2025

Development of Emergency Vehicle Preemption Strategies on Smart Corridors in a Digital Twin Environment

Emergency Response Vehicles (ERVs), such as firetrucks, ambulances, etc. operate with the purpose of saving lives and mitigating property damage. As such, ERV travel-time reductions may result in significant benefits to the community. A common strategy to improve travel times is Emergency Vehicle Preemption (EVP). EVP seeks to reduce ERV delays by providing the right-of-way to ERVs as they approach an intersection. This study proposes a new Dynamic Preemption strategy that determines the need for preemption prior to the ERV reaching the vicinity of the intersection, utilizing real-time data streams. This paper evaluates the effectiveness of some existing and proposed preempt control strategies using a digital twin testbed consisting of a series of signalized intersections on an urban arterial in Georgia. The best EVP strategy maximizes the improvement in ERV travel time while minimizing the adverse effect of preemption on the traffic in conflicting directions. Therefore, this study evaluates both the positive impact of EVP on the ERV as well as the adverse impact on the cross-street traffic. The study found that the potential exists for significant improvements in ERV travel time with the proposed Dynamic Preemption strategy, with minimal impact to the conflicting traffic. For the simulation corridor there was a 20% reduction in the ERV travel times with the implementation of the Dynamic Preemption strategy, compared to traditional EVP practices.

Roy, Somdut [Georgia Institute of Technology]↗
DOE OSTI2025

Accelerating Advanced Light Source Science Through Multi-Facility HPC Workflows

Synchrotron light sources support a wide array of techniques to investigate materials, often producing complex, high-volume data that challenge traditional workflows. At the Advanced Light Source (ALS), we developed infrastructure to move microtomography data over ESnet to ALCF and NERSC, where CPU- and GPU-based algorithms generate 3D reconstructed volumes of experimental samples. We employ two data movement and reconstruction models: real-time processing as data streams directly to NERSC compute nodes, and automated file transfer to NERSC and ALCF file systems. The streaming pipeline provides users with feedback in under ten seconds, while the file-based workflow produces high-quality reconstructions suitable for deeper analysis in 20-30 minutes. This infrastructure enables users to utilize HPC resources without direct access to backend systems. We plan to extend this architecture to more endstations, supporting our beamline scientists and users.

Abramov, David↗
DOE OSTI2026

COMPASS-FME Terrestrial Ecosystem Manipulation to Probe the Effects of Storm Treatments (TEMPEST) Experiment Level 2 Sensor Data v2-1

This is the version v2-1 Level 2 (L2) data release for COMPASS-FME environmental sensors located at our Terrestrial Ecosystem Manipulation to Probe the Effects of Storm Treatments (TEMPEST) experimental site. This manipulative, ecosystem-scale TEMPEST experiment addresses the potential for freshwater and estuarine-water disturbance events to alter tree function, species composition, and ecosystem processes in a deciduous coastal forest in MD, USA. The experiment uses a large-unit (2000 m2), un-replicated experimental design, with three 50 m × 40 m plots serving as control, freshwater, and estuarine-water treatments. Level 2 (L2) data consist of sensor observations from the COMPASS-FME synoptic sites, TEMPEST, and DELUGE. Compared to the L1 data, these are more consistent (always 15-minute timestamps for the entire year); better QA/QC’d (out of bounds, out of service, and extreme outlier values are removed); and more complete, with a gap-filled time series available alongside the main observations, and additional derived (calculated) variables. L2 data are intended to be rapidly and easily usable in analyses and simulations. However, algorithmic outlier identification always carries the risk of removing valid data, and Level 1 data may be more suitable for analyses that focus on variability or extreme events. This dataset includes: - An overall dataset README file that describes the current version, gives citation and contact information, etc. - Site- and year-specific folders, each holding variable-specific Parquet (a high performance, space efficient format; see https://parquet.apache.org) data files for each site and plot in that year. - Metadata files within each site-year folder provide full information on data units, expected ranges, contact information, detailed flood times, as well as a general description of the site. - Environmental sensor types that appear in the data files include weather (ClimaVUE50, CS, RM Young, and LI instruments in the graphs below); soil conditions (TEROS12); soil redox state (Redox); groundwater variables (AquaTROLL200 and AquaTROLL600); open water sondes (Exo); tree sap velocity (Sapflow); and system voltage and state (Datalogger). Data are reported every 15 minutes. Please see v2-1 TEMPEST L2 Sensor Package Quick Start.pdf for detailed information on data package structure, temporal coverage, and versioning. Data files are in Apache Parquet, a high performance, space efficient format for tabular data. These files can be read using R's `arrow` package (https://arrow.apache.org/docs/r/), with similar tools available in other languages. The TEMPEST flood events occurred on the following dates. They lasted for ~10 hours each day and delivered ~80,000 gallons to each plot; many data streams are available at 1 or 5 minute frequency during these periods. * Tests: Aug 25 (fresh plot) and Sep 9 (salt plot), 2021 * TEMPEST 1: June 22, 2022 * TEMPEST 2: June 6-7, 2023 * TEMPEST 3: June 11-13, 2024

EARTH SCIENCE > ATMOSPHERE > ATMOSPHERIC TEMPERATU↗
DOE OSTI2026

Machine learning at the Spallation Neutron Source accelerator and target

We describe the ongoing efforts to apply Machine Learning techniques to improve the performance of our accelerator and target. Specially, we are looking to minimize halo beam losses in the absence of a proper physics model, automatically detect and log anomalies in the target support systems such as cooling, and detect and prevent errant beam pulses in the linac. We also describe the infrastructure we use to acquire and stream data to the GPU cluster for training, our code development cycle, and edge computing for model inference. To minimize halo beam losses, we use a Reinforcement Learning technique tested on a virtual accelerator. The target anomaly detection is trained on archived data using incomplete physics models and is made part of the existing target reporting system. The errant beam prevention analyzes beam current and beam phase waveforms as well as accelerator configuration data to predict errant pulses. We also develop continual learning to adapt to changes in the accelerator.

Accelerator Physics↗
DOE OSTI2025

Graph-Learning-Assisted State and Event Tracking for Solar-Penetrated Power Grids with Heterogeneous Data Sources

Unlike transmission systems, distribution systems do not typically contain sufficient metering to enable real-time state estimation. The lack of sufficient real-time measurements prohibits accurate and timely monitoring of the state of distribution systems. As a result, control and optimal operation of distribution systems, especially those containing large numbers of renewable generation units are not possible without proper data and information about the current state of the system. The main motivation of this project is to address this shortcoming by developing an approach which provides “predicted” real-time measurements so that they can be used to execute a distribution system state estimator. Thus, the objective of the project is to make the distribution systems fully observable, such that the hosting capacity for solar generation can be accurately estimated, and unnecessary solar curtailments can be avoided. In order to accomplish this goal, the project investigated the use of a grid-model-informed machine learning (ML) tool which integrates heterogeneous data streams obtained from AMI meters, SCADA as well as PMU measurements and created synchronous measurement snapshots for the state estimator (SE); and developed a hybrid robust SE which provides not only accurate state estimates but also real-time feedback for the ML model refinement.

14 SOLAR ENERGY↗