Search NASA⌕ Search

SEARCH · Search NASA

Results for “Streaming Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Data Federation Challenges in Remote Near-Real-Time Fusion Experiment Data Processing

Fusion energy experiments and simulations provide critical information needed to plan future fusion reactors. As next-generation devices like ITER move toward long-pulse experiments, analyses, including AI and ML, should be performed in a wide range of time and computing constraints, from near-real-time constraints, between-shot analysis, and to campaign-wide long-term analysis. However, the data volume, velocity, and variety make it extremely challenging for analyses using only local computational resources. Researchers need the ability to compose and execute workflows spanning edge resources to large-scale high-performance computing facilities.We present Delta, a system to address data analysis challenges, including AI/ML, in fusion science, by leveraging the ADIOS I/O library and middleware, to support executing science workflows over the wide area network for near-real-time streaming. We discuss the data federation challenges in performing remote workflows, focusing on on-going research work in (1) managing, reducing, and streaming data to minimize I/O and data movement overheads, (2) decompressing and reorganizing data for analysis, and (3) executing workflows for automated data analysis. We introduce examples for deep-learning based data analysis for the fusion domain and demonstrate how we use Delta to construct end-to-end workflows for a fusion device in Korea, connecting a remote DOE facility in the USA. The capability demonstrated by this project is the basis for improving the state of the art for near-real-time data federation amongst remote facilities.

Choi, Jong Youl↗
NASA NTRS2014

Evaluating LSM-Based Water Budgets Over a West African Basin Assisted with a River Routing Scheme

Within the framework of the African Monsoon Multidisciplinary Analysis (AMMA) Land Surface Model Intercomparison Project phase 2 (ALMIP-2), this study evaluates the water balance simulated by the Interactions between Soil, Biosphere, and Atmosphere (ISBA) over the upper Oum River basin, in Benin, using a mesoscale river routing scheme (RRS). The RRS is based on the nonlinear Muskingum Cunge method coupled with two linear reservoirs that simulate the time delay of both surface runoff and base flow that are produced by land surface models. On the basis of the evidence of a deep water-table recharge in that region,a reservoir representing the deep-water infiltration (DWI) is introduced. The hydrological processes of the basin are simulated for the 2005-08 AMMA field campaign period during which rainfall and stream flow data were intensively collected over the study area. Optimal RRS parameter sets were determined for three optimization experiments that were performed using daily stream flow at five gauges within the basin. Results demonstrate that the RRS simulates stream flow at all gauges with relative errors varying from -22% to 3% and Nash-Sutcliffe coefficients varying from 0.62 to 0.90. DWI varies from 24% to 67% of the base flow as a function of the sub-basin. The relatively simple reservoir DWI approach is quite robust, and further improvements would likely necessitate more complex solutions (e.g., considering seasonality and soil type in ISBA); thus, such modifications are recommended for future studies. Although the evaluation shows that the simulated stream flows are generally satisfactory, further field investigations are necessary to confirm some of the model assumptions.

Multidisciplinary↗
DOE OSTI2024

An Incremental Tensor Train Decomposition Algorithm

We present a new algorithm for incrementally updating the tensor train decomposition of a stream of tensor data. This new algorithm, called the tensor train incremental core expansion (TT-ICE) improves upon the current state-of-the-art algorithms for compressing in tensor train format by developing a new adaptive approach that incurs significantly slower rank growth and guarantees compression accuracy. This capability is achieved by limiting the number of new vectors appended to the TT-cores of an existing accumulation tensor after each data increment. These vectors represent directions orthogonal to the span of existing cores and are limited to those needed to represent a newly arrived tensor to a target accuracy. We provide two versions of the algorithm: TT-ICE and TT-ICE accelerated with heuristics (TT-ICE*). Here, we provide a proof of correctness for TT-ICE and empirically demonstrate the performance of the algorithms in compressing large-scale video and scientific simulation datasets. Compared to existing approaches that also use rank adaptation, TT-ICE* achieves 57× higher compression and up to 95% reduction in computational time.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1987

Synchronization tracking in pulse position modulation receiver

A clock pulse generator for decoding pulse position modulation in an optical communication receiver is synchronized by a delay tracking loop which multiplies impulses of a data pulse by the square wave clock pulses from the generator to produce positive impulses when the clock pulse is of one level, and negative impulses when the clock pulse is of another level. A delay tracking loop integrates the impulses and produces an error signal that adjusts the delay so the clock pulses will be synchronized with data pulses. A dead-time tau sub d is provided between data pulses of an interval tau sub p in the data pulse period tau. When synchronized, the average number of positive impulses integrated will equal the average number of negative impulses over the continuous stream of data pulses.

Vilnrotter, Victor A.↗
DOE OSTI2025

Remote sensing images, DEM, and point clouds associated with “Accuracy evaluation of cost-effective 3D reconstruction approaches for hydrobiogeochemical processes in non-perennial stream riverbeds”

This data package is associated with the publication “Accuracy evaluation of cost-effective 3D reconstruction approaches for hydrobiogeochemical processes in non-perennial stream riverbeds” published in Frontiers in Environmental Science, Environmental Informatics and Remote Sensing (Bao et al., 2026; doi: 10.3389/fenvs.2026.1725258). This data package includes the drone photos for a section of Umtanum Creek in Washington, Unted States. The photos were used to reconstruct the 3-dimensional (3D) digital elevation model (DEM) of the riverbed for the investigated stream section. The reconstruction results from four approaches are provided: (1) unoccupied aerial vehicle (UAV, colloquially known as drone) imagery-based Structure-from-Motion (SfM), (2) a machine learning-based 3D reconstruction model, Visual Geometry Grounded Deep Structure from Motion (VGGSfM), (3) Visual Geometry Grounded Transformer for long sequence of images (VGGT-Long), and (4) handheld smartphone LiDAR scanning. The ground truth measurements by tripod-mounted optical level kit and ground control points GPS locations for evaluating the accuracy of the four reconstruction approaches are also provided in this data package. A preliminary version of this data package was published in October 2025 at the time of manuscript submission. It was updated in March 2026, at the time of manuscript acceptance, to include additional metadata (this readme, data dictionary, and file level metadata). The data did not change. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. In addition to a readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. This dataset is comprised of (1) 8 folders; (2) the detailed flight configuration html files; (3) field metadata; (4) a readme; (5) a data dictionary; and (6) file-level metadata. The folders “2024_10_18_d01” and “2024_10_18_d02” contain the original drone photos for the two drone flights (d01 and d02) on October 18, 2024. The reconstruction results from each of the approaches are in the folders called “ODM_SfM”, “VGGSfM”, “VGGTLong”, and “LiDAR”. The ground truth measurements are in the folder called “optical_level_kit”. Lastly, results comparing the different approaches are in the folder called “comparisons”. All files are .csv, .html, .jpg, .obj, .txt, and .npy. For information on using the .obj and .npy files, see the readme files within the same folder as the files.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

Data Architecture for Security Monitoring (Project Summary)

The research has 2 thrusts: 1. new data architectures and tech-transfer-ready prototype tools that use write optimized data structures (WODS) to track real-world events, provide value to analysts, and support our cyber missions; and 2. algorithm research to address infinite streams of data, including expiration and sustainability.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2020

Gila Water Resources II: Using Earth Observations to Identify Wildfire Impacts on Hydrologic Functions and Recovery in the Gila National Forest

Wildfires have the potential to cause devastating and long-lasting impacts on ecological systems. In the Gila NF, wildfire events have occurred with increasing frequency and severity over recent years. These disturbances such as the historic Whitewater-Baldy Complex Fire (2012) and Silver Fire (2013) have raised concerns over post-fire soil erosion, flooding, debris flows, and vegetation recovery. Understanding the connection between burn events and ecological functions is crucial for developing effective land management practices within the Gila NF that ensure the conservation of the watershed. Our Gila Water Resources II team worked in partnership with the US Department of Agriculture (USDA) US Forest Service’s (USFS) Gila National Forest and Region 3. This project was designed to provide insight into the influence of wildfires on increased flooding events and to determine if restoration efforts in the Gila NF are having a beneficial and noticeable impact on recovery. The goals of this project included generating data-supported end products to inform land management decisions, including the prioritization of specific regions and time scales for post-fire restoration efforts. To understand recovery trends and hydrologic impact in the Gila NF between 2000 and 2020, this project used NASA EO and ancillary data, including, but not limited to, Landsat 5 Thematic Mapper (TM), Landsat 7 Enhanced Thematic Mapper Plus (ETM+), Landsat 8 Operational Land Imager (OLI), USGS stream gauge data, and data provided by USDA USFS’s Gila National Forest and Region 3.

Water Resources↗
DOE OSTI2023

SPLENDAQ: A Detector-Agnostic Data Acquisition System for Small-Scale Physics Experiments

Many scientific applications from rare-event searches to condensed matter system characterization to high-rate nuclear experiments require time-domain triggering on a raw stream of data, where the triggering is generally threshold-based or randomly acquired. When carrying out detector R &D, there is a need for a general data acquisition (DAQ) system to quickly and efficiently process such data. In the SPLENDOR collaboration, we are developing the Python-based SPLENDAQ package for this exact purpose—it offers two main features for offline analysis of continuous data: a threshold triggering algorithm based on the time-domain optimal filter formalism and an algorithm for randomly choosing nonoverlapping segments for noise measurements. Further, combined with the commercially available Moku platform, developed by Liquid Instruments, we have a full pipeline of event building off raw data with minimal setup. Here, we review the underlying principles of this detector-agnostic DAQ package and give concrete examples of its utility in various applications.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
NASA NTRS2009

Telemetry Boards Interpret Rocket, Airplane Engine Data

For all the data gathered by the space shuttle while in orbit, NASA engineers are just as concerned about the information it generates on the ground. From the moment the shuttle s wheels touch the runway to the break of its electrical umbilical cord at 0.4 seconds before its next launch, sensors feed streams of data about the status of the vehicle and its various systems to Kennedy Space Center s shuttle crews. Even while the shuttle orbiter is refitted in Kennedy s orbiter processing facility, engineers constantly monitor everything from power levels to the testing of the mechanical arm in the orbiter s payload bay. On the launch pad and up until liftoff, the Launch Control Center, attached to the large Vehicle Assembly Building, screens all of the shuttle s vital data. (Once the shuttle clears its launch tower, this responsibility shifts to Mission Control at Johnson Space Center, with Kennedy in a backup role.) Ground systems for satellite launches also generate significant amounts of data. At Cape Canaveral Air Force Station, across the Banana River from Kennedy s location on Merritt Island, Florida, NASA rockets carrying precious satellite payloads into space flood the Launch Vehicle Data Center with sensor information on temperature, speed, trajectory, and vibration. The remote measurement and transmission of systems data called telemetry is essential to ensuring the safe and successful launch of the Agency s space missions. When a launch is unsuccessful, as it was for this year s Orbiting Carbon Observatory satellite, telemetry data also provides valuable clues as to what went wrong and how to remedy any problems for future attempts. All of this information is streamed from sensors in the form of binary code: strings of ones and zeros. One small company has partnered with NASA to provide technology that renders raw telemetry data intelligible not only for Agency engineers, but also for those in the private sector.

Source record↗
DOE OSTI2024

Repurposing of the Run 2 CMS High Level Trigger Infrastructure as a Cloud Resource for Offline Computing

The former CMS Run 2 High Level Trigger (HLT) farm is one of the largest contributors to CMS compute resources, providing about 25k job slots for offline computing. This CPU farm was initially employed as an opportunistic resource, exploited during inter-fill periods, in the LHC Run 2. Since then, it has become a nearly transparent extension of the CMS capacity at CERN, being located on-site at the LHC interaction point 5 (P5), where the CMS detector is installed. This resource has been configured to support the execution of critical CMS tasks, such as prompt detector data reconstruction. It can therefore be used in combination with the dedicated Tier 0 capacity at CERN, in order to process and absorb peaks in the stream of data coming from the CMS detector. The initial configuration for this resource, based on statically configured VMs, provided the required level of functionality. However, regular operations of this cluster revealed certain limitations compared to the resource provisioning and use model employed in the case of WLCG sites. A new configuration, based on a vacuum-like model, has been implemented for this resource in order to solve the detected shortcomings. This paper reports about this redeployment work on the permanent cloud for an enhanced support to CMS offline computing, comparing the former and new models’ respective functionalities, along with the commissioning effort for the new setup.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Online Event Detection in Synchrophasor Data with Graph Signal Processing

Online detection of anomalies is crucial to enhancing the reliability and resiliency of power systems. We propose a novel data-driven online event detection algorithm with synchrophasor data using graph signal processing. In addition to being extremely scalable, our proposed algorithm can accurately capture and leverage the spatio-temporal correlations of the streaming PMU data. This paper also develops a general technique to decouple spatial and temporal correlations in multiple time series. Finally, we develop a unique framework to construct a weighted adjacency matrix and graph Laplacian for product graph. Case studies with real-world, large-scale synchrophasor data demonstrate the scalability and accuracy of our proposed event detection algorithm. Compared to the state-of-the-art benchmark, the proposed method not only achieves higher detection accuracy but also yields higher computational efficiency.

Event detection↗
DOE OSTI2020

Online Event Detection in Synchrophasor Data with Graph Signal Processing

Online detection of anomalies is crucial to enhancing the reliability and resiliency of power systems. We propose a novel data-driven online event detection algorithm with synchrophasor data using graph signal processing. In addition to being extremely scalable, our proposed algorithm can accurately capture and leverage the spatio-temporal correlations of the streaming PMU data. This paper also develops a general technique to decouple spatial and temporal correlations in multiple time series. Finally, we develop a unique framework to construct a weighted adjacency matrix and graph Laplacian for product graph. Case studies with real-world, large-scale synchrophasor data demonstrate the scalability and accuracy of our proposed event detection algorithm. Compared to the state-of-the-art benchmark, the proposed method not only achieves higher detection accuracy but also yields higher computational efficiency.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2019

An Integrated Hydrological Model for Long-Term Monitoring in Tims Branch Watershed, SC - 19435

The Tims Branch watershed in Savannah River Site (SRS), South Carolina is a small-scale watershed that has been impacted by contaminated runoff and discharge wastewater from nuclear research facilities located within the watershed over the past century. Innovative remediation technologies, such as tin injection and pump and treat, have been implemented to remove heavy metals and volatile organic compounds (VOCs) from the watershed. Today, the Tims Branch watershed serves as a test-bed for monitoring the performance of remediation activities. This study describes the development of the two-dimensional integrated surface water/groundwater model for Tims Branch watershed to assess the impact of cleanup activities on hydrological processes in this watershed. This model will serve as a tool to simulate the effect of climatic events on overland flow distribution and further on will be used to assess contaminant transport. This paper illustrates the application of the developed model for simulation of overland flow depth and velocity in Tims Branch watershed and streamflow in A-014 outfall tributary. The watershed includes the second order Tims Branch stream, several outfalls, and a network of tributaries. Simulation of non-calibrated overland flow was conducted for the years 2016 using available stream flow data within the watershed. Recent flow data from a newly deployed stage gauge was also used to assess the model results. The results show the model ability to simulate overland flow and stream flow and can be used as a tool to represent the hydrological conditions in the Tims Branch watershed. The results of this model simulation will be further used to develop contaminant transport model. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
NASA NTRS1992

Hydrostatic pressure mimics gravitational pressure in characean cells

Hydrostatic pressure applied to one end of a horizontal Chara cell induces a polarity of cytoplasmic streaming, thus mimicking the effect of gravity. A positive hydrostatic pressure induces a more rapid streaming away from the applied pressure and a slower streaming toward the applied pressure. In contrast, a negative pressure induces a more rapid streaming toward and a slower streaming away from the applied pressure. Both the hydrostatic pressure-induced and gravity-induced polarity of cytoplasmic streaming respond identically to cell ligation, UV microbeam irradiation, external Ca2+ concentrations, osmotic pressure, neutral red, TEA Cl-, and the Ca2+ channel blockers nifedipine and LaCl3. In addition, hydrostatic pressure applied to the bottom of a vertically-oriented cell can abolish and even reverse the gravity-induced polarity of cytoplasmic streaming. These data indicate that both gravity and hydrostatic pressure act at the same point of the signal transduction chain leading to the induction of a polarity of cytoplasmic streaming and support the hypothesis that characean cells respond to gravity by sensing a gravity-induced pressure differential between the cell ends.

NASA Discipline Number 40-50↗
NASA NTRS2020

Ellicott City Disasters II: Enhancing a Statistical Flood Risk Model to Continue Improving Early Warning Systems and Public Safety in Ellicott City, Maryland

As flooding events in the United States grow in frequency and intensity, the use of technological advancements and applied science are increasingly necessary for effective flood monitoring and warning systems. The NASA DEVELOP Ellicott City Disasters II project investigated the use of machine learning for applications in flood risk detection to support the improvement of early warning systems. To strengthen the efforts of the Howard County Office of Emergency Management (OEM) in building a more robust flood monitoring system, the project improved the original statistical flood risk model, FLuME (Flood Learning Model Environment), programmed by the first DEVELOP term. The enhancements incorporated an additional six years of precipitation and soil moisture data from the North American Land Data Assimilation System (NLDAS), modeled using Aqua Advanced Microwave Scanning Radiometer for EOS and Tropical Rainfall Measuring Mission (TRMM) Microwave Imager. These Earth observations were supplemented by stream gauge data from the OEM and the US Geological Survey. The resultant flood risk model FLASH (Flood Learning Environment and Severity Assessment Hub) was trained to evaluate input variables and predict stage height in Ellicott City in real time. The addition of an advanced deep learning framework known as long short-term memory improved the model’s ability to capture relationships between variables. To assess the effectiveness of the new model, FLASH produced a model efficiency metric of 0.99, a significant improvement over the 0.85 value produced by the previous model. The project assisted the OEM in pursuing the integration of open data and NASA Earth observations into a threat matrix capable of informing near real-time decision making.

Disasters↗
NASA NTRS2020

Ellicott City Disasters II: Enhancing a Statistical Flood Risk Model to Continue Improving Early Warning Systems and Public Safety in Ellicott City, Maryland

As flooding events in the United States grow in frequency and intensity, the use of technological advancements and applied science are increasingly necessary for effective flood monitoring and warning systems. The NASA DEVELOP Ellicott City Disasters II project investigated the use of machine learning for applications in flood risk detection to support the improvement of early warning systems. To strengthen the efforts of the Howard County Office of Emergency Management (OEM) in building a more robust flood monitoring system, the project improved the original statistical flood risk model, FLuME (Flood Learning Model Environment), programmed by the first DEVELOP term The enhancements incorporated an additional six years of precipitation and soil moisture data from the North American Land Data Assimilation System (NLDAS), modeled using Aqua Advanced Microwave Scanning Radiometer for EOS and Tropical Rainfall Measuring Mission TRMM Microwave Imager. These Earth observations were supplemented by stream gauge data from the OEM and the US Geological Survey. The resultant flood risk model FLASH (Flood Learning Environment and Severity Assessment Hub) was trained to evaluate input variables and predict stage height in Ellicott City in real time. The addition of an advanced deep learning framework known as long short-term memory improved the model’s ability to capture relationships between variables. To assess the effectiveness of the new model, FLASH produced a model efficiency metric of 0.99, a significant improvement over the 0.85 value produced by the previous model. The project assisted the OEM in pursuing the integration of open data and NASA Earth observations into a threat matrix capable of informing near real-time decision making.

Disasters↗
NASA NTRS1992

Deep near-infrared survey of the Southern Sky (DENIS)

DENIS (Deep Near-Infrared Survey of the Southern Sky) will be the first complete census of astronomical sources in the near-infrared spectral range. The challenges of this novel survey are both scientific and technical. Phenomena radiating in the near-infrared range from brown dwarfs to galaxies in the early stages of cosmological evolution, the scientific exploitation of data relevant over such a wide range requires pooling expertise from several of the leading European astronomical centers. The technical challenges of a project which will provide an order of magnitude more sources than given by the IRAS space mission, and which will involve advanced data-handling and image-processing techniques, likewise require pooling of hardware and software resources, as well as of human expertise. The DENIS project team is composed of some 40 scientists, computer specialists, and engineers located in 5 European Community countries (France, Germany, Italy, The Netherlands, and Spain), with important contributions from specialists in Australia, Brazil, Chile, and Hungary. DENIS will survey the entire southern sky in 3 colors, namely in the I band at a wavelength of 0.8 micron, in the 1.25 micron J band, and in the 2.15 micron K' band. The sensitivity limits will be 18th magnitude in the I band, 16th in the J band, and 14.5th in the K' band. The angular resolution achieved will be 1 arcsecond in the I band, and 3.0 arcseconds in the J and K' bands. The European Southern Observatory 1 m telescope on La Silla will be dedicated to survey use during operations expected to last four years, commencing in late 1993. DENIS aims to provide the astronomical community with complete digitized infrared images of the full southern sky and a catalogue of extracted objects, both of the best quality and in readily accessible form. This will be achieved through dedicated software packages and specialized catalogues, and with assistance from the Leiden and Paris Data Analysis Centers. The data will be mailed on DAT tapes from La Silla to the two Data Analysis Centers for further processing. Two centers are necessary because of the shear quantity of data and because of the complementary roles the Centers will develop, each exploiting its own particular expertise. The Leiden Data Analysis Center (LDAC) will extract objects, establish their parameters, and archive them into a source catalogue. The LDAC will collaborate with the Groningen Space Research group that has gained experience in infrared image handling from the IRAS satellite. The Paris Data Analysis Center (PDAC) will be responsible for archiving and preprocessing the raw data to provide a homogeneous set of data suitable for further reduction in both the Leiden and Paris data analysis streams. The PDAC will also extract and archive images for the sources flagged by the LDAC as extended, and create a catalogue of galaxies. In exploiting the DENIS data we foresee the collaboration with other data analysis centers, such as the Observatoire de Lyon where the relevant DENIS catalogue of galaxies can be incorporated into their extragalactic database. The Point Sources and the Small Extended Sources catalogues could be incorporated in the Late Type Star database at Montpellier, and in the SIMBAD database as CDS. At Groningen the IRAS Point Source catalogue and/or image data can be merged with the DENIS catalogues. At Meudon algorithms and software will be developed with main goal assessing the limits reachable for the homogeneity and intrinsic consistency between the ensemble of the images in the data base (flat-fielding, relative positioning of the fields, bootstrapped flux calibration) but also for the data analysis.

Deul, E.↗