SEARCH · Search NASA

Results for “Streaming Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,063 records · Page 59

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Real-time Multi-granular Analytics Framework for HIT Systems

Streaming analytics is the process of ingesting and digesting live data from multiple data sources. In the healthcare domain, as the importance of extracting immediate insights while data are streaming into the system grows, the focus is shifting from batch processing to streaming analytics. With data increasing dramatically at high speeds, many informatics designs have been proposed to adapt healthcare domain into this new environment. In our previous work, we introduced a prototype of health informatics technology (HIT) framework that aims to address challenges in adopting state-of-the-art technologies to enable advanced healthcare analytic tasks in new streaming environments. We recently made major updates to the framework so that anomaly from multiple streaming data sources at different granularity levels can be detected in near real-time. In this paper, we detail the implementation and deployment of the framework in Kubernetes clusters and report its performances when tested on electronic health record (EHR) data of Veterans Affairs.

Park, Byung↗

NASA NTRS1981

Variations in Transport Derived from Satellite Altimeter Data over the Gulf Stream

Variations in total change of sea surface height (delta h) across the Gulf Stream are observed using Seasat radar altimeter data. The sea surface height is related to transport within the stream by a two layer model. Variations in delta h are compared with previously observed changes in transport found to increase with distance downstream. No such increase is apparent since the satellite transports show no significant dependence on distance. Though most discrepancies are less than 50 percent, a few cases differ by about 100 percent and more. Several possible reasons for these discrepancies are advanced, including geoid error, but only two oceanographic contributions to the variability are examined, namely, limitations in the two layer model and meanders in the current. It is concluded that some of the discrepancies could be explained as changes in the density structure not accounted for by the two layer model.

Molinelli, Eugene↗

DOE OSTI2026

Stream discharge and temperature data collected within the East and Taylor Watershed, Colorado for the Lawrence Berkeley National Laboratory Watershed Function Science Focus Area (water years 2019 to 2025)

This dataset contains stream discharge and temperature data for water years 2019 to 2025 from the East and Taylor Watersheds in Colorado, United States. This data was collected to understand hydrological processes occurring in the East River and Taylor River Watersheds, Colorado, which is part of the Lawrence Berkeley National Laboratory Watershed Function Scientific Focus Area. Data includes instantaneous observed discharge using salt dilution and acoustic doppler velocimeter techniques, raw pressure transducer downloaded data, sub-hourly temperature as well as corrected water level and associated stream discharge and mean daily values. Notes on water level corrections, rating curve development and metadata provided. A rating curve is the translation of depth to streamflow. The rating curve can be used as a quantitative measure of the “quality of the data.” Data within this dataset is formatted using ESS-DIVE’s Hydrological Monitoring Reporting Format. This data package contains (1) a zip file (Stream_Discharge_Data_WY19-WY25.zip) containing stream discharge and temperature data organized by location; (2) an InstallationMethods file (InstallationMethods.csv) describing metadata about the installation; (3) a file-level metadata (flmd.csv) file that lists each file contained in the dataset with associated metadata; (4) a data dictionary (dd.csv) file that contains terms/column_headers used throughout the files along with a definition, units, and data type; (5) a locations metadata file (locations.csv); (6) and a sensor metadata file (sensors.csv). All data files are in non-proprietary formats (csv, png, or pdf formats). Please contact Rosemary Carroll, Curtis Beutler, or Austin Shirley for any support in accessing the files. Update on 2023-05-12: Additional data from WYs 2021 and 2022 were added. Additionally, the dataset was converted using ESS-DIVE’s Hydrological Monitoring Reporting Format. Data files were reformatted to match reporting format guidance, new metadata files were added, and files were converted from excel to CSV. Update on 2025-05-16: Additional data from WYs 2022 (for locations not previously included), 2023, and 2024 were added. An additional descriptive PDF (WFSFA_Streamflow_Hydrograph_Disclaimer.pdf) was added. Metadata files were updated to reflect the addition of new data and locations. Update on 2026-05-18: Additional data from WY 2025 were added, including a new location Upper Trail Creek (TR-TCG2). Metadata files were updated to reflect the addition of new data.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1980

Digital optical recorder-reproducer system

A mass archival optical recording and reproduction system includes a recording light source such as a laser beam focussed and directed upon an acousto-optic linear modulator array (or page composer) that receives parallel blocks of data converted from a serial stream of digital data to be stored. The page composer imparts to the laser beam modulation representative of a plurality of parallel channels of data and through focussing optics downstream of the page composer parallel arrays of optical spots are recorded upon a suitable recording medium such as a photographic film floppy disc. The recording medium may be substantially frictionlessly and stably positioned for recording at a record/read station by an air-bearing platen arrangement which is preferably thermodynamically non-throttling so that the recording film may be positioned in the path of the information-carrying light beam in a static or dynamic mode. During readout, the page composer is bypassed and a readout light beam is focussed directly upon the recording medium containing an array of previously recorded digital spots, a sync bit, data positioning bits, and a tracking band. The readout beam which has been directed through the recording medium is then imaged upon a photodetector array, the output of which may be coupled to suitable electronic processing circuitry, such as a digital multiplexer, whereby the parallel spot array is converted back into the original serial data stream.

Reddersen, Brad R.↗

NASA NTRS2014

A Model-Based Anomaly Detection Approach for Analyzing Streaming Aircraft Engine Measurement Data

This paper presents a model-based anomaly detection architecture designed for analyzing streaming transient aircraft engine measurement data. The technique calculates and monitors residuals between sensed engine outputs and model predicted outputs for anomaly detection purposes. Pivotal to the performance of this technique is the ability to construct a model that accurately reflects the nominal operating performance of the engine. The dynamic model applied in the architecture is a piecewise linear design comprising steady-state trim points and dynamic state space matrices. A simple curve-fitting technique for updating the model trim point information based on steadystate information extracted from available nominal engine measurement data is presented. Results from the application of the model-based approach for processing actual engine test data are shown. These include both nominal fault-free test case data and seeded fault test case data. The results indicate that the updates applied to improve the model trim point information also improve anomaly detection performance. Recommendations for follow-on enhancements to the technique are also presented and discussed.

Propulsion System Performance↗

NASA NTRS2015

A Model-Based Anomaly Detection Approach for Analyzing Streaming Aircraft Engine Measurement Data

Propulsion System Performance↗

NASA NTRS1991

A high performance FDDI NIU for streaming voice, video, and data

A 100-Mbit/s FDDI network interface unit (NIU) is described that supports real-time data, voice and video. Its high-speed interrupt-driven hardware architecture efficiently manages stream and packet data transfers to the FDDI network. Other enhancements include modular single-mode laser-dioce fiber optic links to maximize node spacing, optic bypass switches for increased fault tolerance, and a hardware performance monitor to gather real-time network diagnostics.

Bergman, L. A.↗

NASA NTRS2004

Viewing ISS Data in Real Time via the Internet

EZStream is a computer program that enables authorized users at diverse terrestrial locations to view, in real time, data generated by scientific payloads aboard the International Space Station (ISS). The only computation/communication resource needed for use of EZStream is a computer equipped with standard Web-browser software and a connection to the Internet. EZStream runs in conjunction with the TReK software, described in a prior NASA Tech Briefs article, that coordinates multiple streams of data for the ground communication system of the ISS. EZStream includes server components that interact with TReK within the ISS ground communication system and client components that reside in the users' remote computers. Once an authorized client has logged in, a server component of EZStream pulls the requested data from a TReK application-program interface and sends the data to the client. Future EZStream enhancements will include (1) extensions that enable the server to receive and process arbitrary data streams on its own and (2) a Web-based graphical-user-interface-building subprogram that enables a client who lacks programming expertise to create customized display Web pages.

Myers, Gerry↗

DOE OSTI2025

Levoglucosan data from five coastal streams impacted by the 2020 CZU Lightning Complex Fires, California, United States

This dataset includes levoglucosan data for five coastal California (United States) streams impacted by the 2020 CZU Lightning Complex Fires which burned from August 16th through September 22nd. Levoglucosan is a highly soluble and biolabile fraction of pyrogenic carbon. The five watersheds (San Lorenzo River, Pescadero Creek, Majors Creek, Laguna Creek, and Scott Creek) were impacted by the fires with watersheds experiencing a range of burn severity and extents. Grab samples were collected from each stream between October 2020 and May 2021, targeting both baseflow and event flow hydrologic conditions. Additional biogeochemistry data (i.e., organic and black carbon concentrations) can be found in a separate data package (https://doi.org/10.4211/hs.421c0226bb38460c8393d67fe0c4f802). This data package consists of one main data folder that contains (1) readme; (2) file-level metadata; (3) data dictionary; (4) field metadata with international generic sample numbers (IGSN); (5) methods codes; and (6) levoglucosan data. All files are .csv or .pdf.

2020 CZU Lightning Complex Fires↗

NASA NTRS1997

Monthly Maps of Sea Surface Height in the North Atlantic and Zonal Indices for the Gulf Stream Using TOPEX/Poseidon Altimeter Data

Monthly Maps of sea surface height are constructed for the North Atlantic Ocean using TOPEX/Poseidon altimeter data. Mean sea surface height is reconstructed using a weighted combination of historical, hydrographic data and a synthetic mean obtained by fitting a Gaussian model of the Gulf Stream jet to altimeter data. The resultant mean shows increased resolution over the hydrographic mean, and incorporates recirculation information that is absent in the synthetic mean. Monthly maps, obtained by adding the mean field to altimeter sea surface height residuals, are used to derive a set of zonal indices that describe the annual cycle of meandering as well as position and strength of the Gulf Stream.

GULF STREAM↗

DOE OSTI2022

Near real-time streaming analysis of big fusion data

Experiments on fusion plasmas produce high-dimensional data time series with ever-increasing magnitude and velocity, but turn-around times for analysis of this data have not kept up. For example, many data analysis tasks are often performed in a manual, ad-hoc manner some time after an experiment. In this article, we introduce the Delta framework that facilitates near real-time streaming analysis of big and fast fusion data. By streaming measurement data from fusion experiments to a high-performance compute center, Delta allows computationally expensive data analysis tasks to be performed in between plasma pulses. This article describes the modular and expandable software architecture of Delta and presents performance benchmarks of individual components as well as of an example workflow. Focusing on a streaming analysis workflow where electron cyclotron emission imaging (ECEi) data is measured at KSTAR on the National Energy Research Scientific Computing Center's (NERSC's) supercomputer we routinely observe data transfer rates of about 4 Gigabit per second. In NERSC, a demanding turbulence analysis workflow effectively utilizes multiple nodes and graphical processing units and executes them in under 5 min. We further discuss how Delta uses modern database systems and container orchestration services to provide web-based real-time data visualization. For the case of ECEi data we demonstrate how data visualizations can be augmented with outputs from machine learning models. Here, by providing session leaders and physics operators, results of higher-order data analysis using live visualizations may make more informed decisions on how to configure the machine for the next shot.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

NASA NTRS1991

Burst-Compression And -Expansion For TDMA Communication

Burst-compression and -expansion technique enables interconnection of users transmitting and receiving data at rates asynchronous with respect to clocks within ground terminals of satellite-switched, time-division-multiple-access (TDMA) communication network. Matrix switch aboard satellite routes bursts of data from source users received on uplink antennas to downlink antennas illuminating ground areas containing destination users. TDMA ground terminal compresses streams of data from source users into rapid bursts for transmission and reexpands bursts of received data into slower streams of data for delivery to destination users. Greater flexibility in interconnecting widely dispersed users achieved by use of hopping beams.

Budinger, James M.↗

DOE OSTI2016

Walker Branch Watershed: Temperature Response of Organic-Matter Decomposition in Headwater Stream

This data set reports the results of a field study investigating the effect of temperature on organic-matter decomposition in the West Fork of Walker Branch, a headwater stream on the Oak Ridge Reservation in east Tennessee. The goal of this project was to evaluate the effect of temperature on organic-matter decomposition in a stream ecosystem to better understand how carbon dynamics across terrestrial-aquatic carbon interfaces may change with warming. The West Fork is a spring-fed headwater stream and the spring inputs create a natural longitudinal temperature gradient. In winter, the upstream sites near the springs are warmer due to the spring input. In summer, the temperature gradient reverses and the upstream sites are cooler. One experiment examined the decomposition of senesced leaves from 3 tree species (red maple [Acer rubrum], tulip poplar [Liriodendron tulipifera], and white oak [Quercus alba]) along the temperature gradient in winter (December 2011 - March 2012). The second experiment examined the breakdown of cotton strips, a substrate of consistent quality (95% cellulose), along the temperature gradient with approximately month-long incubations over ~2 years (August 2011 to October 2013). This dataset contains six data files in comma separate (*.csv) format.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2005

Predictive Interfaces for Long-Distance Tele-Operations

We address the development of predictive tele-operator interfaces for humanoid robots with respect to two basic challenges. Firstly, we address automating the transition from fully tele-operated systems towards degrees of autonomy. Secondly, we develop compensation for the time-delay that exists when sending telemetry data from a remote operation point to robots located at low earth orbit and beyond. Humanoid robots have a great advantage over other robotic platforms for use in space-based construction and maintenance because they can use the same tools as astronauts do. The major disadvantage is that they are difficult to control due to the large number of degrees of freedom, which makes it difficult to synthesize autonomous behaviors using conventional means. We are working with the NASA Johnson Space Center's Robonaut which is an anthropomorphic robot with fully articulated hands, arms, and neck. We have trained hidden Markov models that make use of the command data, sensory streams, and other relevant data sources to predict a tele-operator's intent. This allows us to achieve subgoal level commanding without the use of predefined command dictionaries, and to create sub-goal autonomy via sequence generation from generative models. Our method works as a means to incrementally transition from manual tele-operation to semi-autonomous, supervised operation. The multi-agent laboratory experiments conducted by Ambrose et. al. have shown that it is feasible to directly tele-operate multiple Robonauts with humans to perform complex tasks such as truss assembly. However, once a time-delay is introduced into the system, the rate of tele\ioperation slows down to mimic a bump and wait type of activity. We would like to maintain the same interface to the operator despite time-delays. To this end, we are developing an interface which will allow for us to predict the intentions of the operator while interacting with a 3D virtual representation of the expected state of the robot. The predictive interface anticipates the intention of the operator, and then uses this prediction to initiate appropriate sub-goal autonomy tasks.

Wheeler, Kevin R.↗

NASA NTRS2007

Integrated inertial stellar attitude sensor

An integrated inertial stellar attitude sensor for an aerospace vehicle includes a star camera system, a gyroscope system, a controller system for synchronously integrating an output of said star camera system and an output of said gyroscope system into a stream of data, and a flight computer responsive to said stream of data for determining from the star camera system output and the gyroscope system output the attitude of the aerospace vehicle.

Brady, Tye M.↗

DOE OSTI2021

Closing the Gap between FAIR Data Repositories and Hierarchical Data Formats

Many in the scientific community, particularly in publicly funded research, are pushing to adhere to more accessible data standards to maximize the findability, accessibility, interoperability, and reusability (FAIR) of scientific data, especially with the growing prevalence of machine learning augmented research. Online FAIR data repositories, such as the Open Science Framework (OSF), help facilitate the adoption of these standards by providing frameworks for storage, access, search, APIs, and other features that create organized hubs of scientific data. However, the wider acceptance of such repositories is hindered by the lack of support of hierarchical data formats, such as Technical Data Management Streaming (TDMS) and Hierarchical Data Format 5 (HDF5), that many researchers rely on to organize their datasets. Various tools and strategies should be used to allow hierarchical data formats, FAIR data repositories, and scientific organizations to work more seamlessly together. A pilot project at Los Alamos National Laboratory (LANL) addresses the disconnect between them by integrating the OSF FAIR data repository with hierarchical data renderers, extending support for additional file types in their framework. The multifaceted interactive renderer displays a tree of metadata alongside a table and plot of the data channels in the file. This allows users to quickly and efficiently load large and complex data files directly in the OSF webapp. Users who are browsing files can quickly and intuitively see the files in the way they or their colleagues structured the hierarchical form and immediately grasp their contents. This solution helps bridge the gap between hierarchical data storage techniques and FAIR data repositories, making both of them more viable options for scientific institutions like LANL which have been put off by the lack of integration between them.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1992

Fast Vector-Quantizing Data Compressor

Proposed encoding digital electronic data-processing system compresses 7-Mb/s input stream of data into 1-Mb/s output stream. Intended to reduce flow of data from orbiting synthetic-aperture-radar (SAR) system to rate low enough for transmission to ground. Features include reliability, light weight, and low power. Proves advantageous in such terrestrial applications as compression of speech and of video signals.

Chang, Chi-Yung↗

DOE OSTI2020

Seismology in the Cloud: A New Streaming Workflow

Data-intensive research in seismology is experiencing a recent boom, driven in part by large volumes of available data and advances in the growing field of data science. However, there are significant barriers to processing large data volumes, such as long retrieval times from data repositories, complex data management, and limited computational resources. New tools and platforms have reduced the barriers to entry for scientific cluster computing, including the maturation of the commercial cloud as an accessible instrument for research. Here, we build a customized research cluster in the cloud to test a new workflow for large-scale seismic analysis, in which data are processed as a stream (retrieved on-the-fly and acted upon without storing), with data from the Incorporated Research Institutions for Seismology Data Management Center. We use this workflow to deploy a spectral peak detection algorithm over 5.6 TB of compressed continuous seismic data from 2074 stations of the USArray Transportable Array EarthScope network. Using a 50-node cluster in the cloud, we completed the noise survey in 80 hr, with an average data throughput of 1.7 GB per minute. By varying cluster sizes, we find the scaling of our analysis to be sublinear, due to a combination of algorithmic limitations and data center response times. The cloud-based streaming workflow represents an order-of-magnitude increase in acquisition and processing speed compared to a traditional download-store-process workflow, and offers the additional benefits of employing a flexible, accessible, and widely used computing architecture. It is limited, however, due to its reliance on Internet transfer speeds and data center service capacity, and may not work well for repeated analyses or those for which even higher data throughputs are needed. These research applications will require a new class of cloud-native approaches in which both data and analysis are in the cloud.

58 GEOSCIENCES↗