SEARCH · Search NASA

Results for “data streaming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Streaming Large-Scale Microscopy Data to a Supercomputing Facility

Data management is a critical component of modern experimental workflows. As data generation rates increase, transferring data from acquisition servers to processing servers via conventional file-based methods is becoming increasingly impractical. The 4D Camera at the National Center for Electron Microscopy generates data at a nominal rate of 480 Gbit s -1 (87,000 frames s -1 ⁠), producing a 700 GB dataset in 15 s. To address the challenges associated with storing and processing such quantities of data, we developed a streaming workflow that utilizes a high-speed network to connect the 4D Camera’s data acquisition system to supercomputing nodes at the National Energy Research Scientific Computing Center, bypassing intermediate file storage entirely. In this work, we demonstrate the effectiveness of our streaming pipeline in a production setting through an hour-long experiment that generated over 10 TB of raw data, yielding high-quality datasets suitable for advanced analyses. Additionally, we compare the efficacy of this streaming workflow against the conventional file-transfer workflow by conducting a postmortem analysis on historical data from experiments performed by real users. Our findings show that the streaming workflow significantly improves data turnaround time, enables real-time decision-making, and minimizes the potential for human error by eliminating manual user interactions.

4D-STEM↗

NASA NTRS1981

Variations in Transport Derived from Satellite Altimeter Data over the Gulf Stream

Variations in total change of sea surface height (delta h) across the Gulf Stream are observed using Seasat radar altimeter data. The sea surface height is related to transport within the stream by a two layer model. Variations in delta h are compared with previously observed changes in transport found to increase with distance downstream. No such increase is apparent since the satellite transports show no significant dependence on distance. Though most discrepancies are less than 50 percent, a few cases differ by about 100 percent and more. Several possible reasons for these discrepancies are advanced, including geoid error, but only two oceanographic contributions to the variability are examined, namely, limitations in the two layer model and meanders in the current. It is concluded that some of the discrepancies could be explained as changes in the density structure not accounted for by the two layer model.

Molinelli, Eugene↗

DOE OSTI2026

Stream discharge and temperature data collected within the East and Taylor Watershed, Colorado for the Lawrence Berkeley National Laboratory Watershed Function Science Focus Area (water years 2019 to 2025)

This dataset contains stream discharge and temperature data for water years 2019 to 2025 from the East and Taylor Watersheds in Colorado, United States. This data was collected to understand hydrological processes occurring in the East River and Taylor River Watersheds, Colorado, which is part of the Lawrence Berkeley National Laboratory Watershed Function Scientific Focus Area. Data includes instantaneous observed discharge using salt dilution and acoustic doppler velocimeter techniques, raw pressure transducer downloaded data, sub-hourly temperature as well as corrected water level and associated stream discharge and mean daily values. Notes on water level corrections, rating curve development and metadata provided. A rating curve is the translation of depth to streamflow. The rating curve can be used as a quantitative measure of the “quality of the data.” Data within this dataset is formatted using ESS-DIVE’s Hydrological Monitoring Reporting Format. This data package contains (1) a zip file (Stream_Discharge_Data_WY19-WY25.zip) containing stream discharge and temperature data organized by location; (2) an InstallationMethods file (InstallationMethods.csv) describing metadata about the installation; (3) a file-level metadata (flmd.csv) file that lists each file contained in the dataset with associated metadata; (4) a data dictionary (dd.csv) file that contains terms/column_headers used throughout the files along with a definition, units, and data type; (5) a locations metadata file (locations.csv); (6) and a sensor metadata file (sensors.csv). All data files are in non-proprietary formats (csv, png, or pdf formats). Please contact Rosemary Carroll, Curtis Beutler, or Austin Shirley for any support in accessing the files. Update on 2023-05-12: Additional data from WYs 2021 and 2022 were added. Additionally, the dataset was converted using ESS-DIVE’s Hydrological Monitoring Reporting Format. Data files were reformatted to match reporting format guidance, new metadata files were added, and files were converted from excel to CSV. Update on 2025-05-16: Additional data from WYs 2022 (for locations not previously included), 2023, and 2024 were added. An additional descriptive PDF (WFSFA_Streamflow_Hydrograph_Disclaimer.pdf) was added. Metadata files were updated to reflect the addition of new data and locations. Update on 2026-05-18: Additional data from WY 2025 were added, including a new location Upper Trail Creek (TR-TCG2). Metadata files were updated to reflect the addition of new data.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1980

Digital optical recorder-reproducer system

A mass archival optical recording and reproduction system includes a recording light source such as a laser beam focussed and directed upon an acousto-optic linear modulator array (or page composer) that receives parallel blocks of data converted from a serial stream of digital data to be stored. The page composer imparts to the laser beam modulation representative of a plurality of parallel channels of data and through focussing optics downstream of the page composer parallel arrays of optical spots are recorded upon a suitable recording medium such as a photographic film floppy disc. The recording medium may be substantially frictionlessly and stably positioned for recording at a record/read station by an air-bearing platen arrangement which is preferably thermodynamically non-throttling so that the recording film may be positioned in the path of the information-carrying light beam in a static or dynamic mode. During readout, the page composer is bypassed and a readout light beam is focussed directly upon the recording medium containing an array of previously recorded digital spots, a sync bit, data positioning bits, and a tracking band. The readout beam which has been directed through the recording medium is then imaged upon a photodetector array, the output of which may be coupled to suitable electronic processing circuitry, such as a digital multiplexer, whereby the parallel spot array is converted back into the original serial data stream.

Reddersen, Brad R.↗

NASA NTRS2014

A Model-Based Anomaly Detection Approach for Analyzing Streaming Aircraft Engine Measurement Data

This paper presents a model-based anomaly detection architecture designed for analyzing streaming transient aircraft engine measurement data. The technique calculates and monitors residuals between sensed engine outputs and model predicted outputs for anomaly detection purposes. Pivotal to the performance of this technique is the ability to construct a model that accurately reflects the nominal operating performance of the engine. The dynamic model applied in the architecture is a piecewise linear design comprising steady-state trim points and dynamic state space matrices. A simple curve-fitting technique for updating the model trim point information based on steadystate information extracted from available nominal engine measurement data is presented. Results from the application of the model-based approach for processing actual engine test data are shown. These include both nominal fault-free test case data and seeded fault test case data. The results indicate that the updates applied to improve the model trim point information also improve anomaly detection performance. Recommendations for follow-on enhancements to the technique are also presented and discussed.

Propulsion System Performance↗

NASA NTRS2015

A Model-Based Anomaly Detection Approach for Analyzing Streaming Aircraft Engine Measurement Data

Propulsion System Performance↗

NASA NTRS1991

A high performance FDDI NIU for streaming voice, video, and data

A 100-Mbit/s FDDI network interface unit (NIU) is described that supports real-time data, voice and video. Its high-speed interrupt-driven hardware architecture efficiently manages stream and packet data transfers to the FDDI network. Other enhancements include modular single-mode laser-dioce fiber optic links to maximize node spacing, optic bypass switches for increased fault tolerance, and a hardware performance monitor to gather real-time network diagnostics.

Bergman, L. A.↗

NASA NTRS2004

Viewing ISS Data in Real Time via the Internet

EZStream is a computer program that enables authorized users at diverse terrestrial locations to view, in real time, data generated by scientific payloads aboard the International Space Station (ISS). The only computation/communication resource needed for use of EZStream is a computer equipped with standard Web-browser software and a connection to the Internet. EZStream runs in conjunction with the TReK software, described in a prior NASA Tech Briefs article, that coordinates multiple streams of data for the ground communication system of the ISS. EZStream includes server components that interact with TReK within the ISS ground communication system and client components that reside in the users' remote computers. Once an authorized client has logged in, a server component of EZStream pulls the requested data from a TReK application-program interface and sends the data to the client. Future EZStream enhancements will include (1) extensions that enable the server to receive and process arbitrary data streams on its own and (2) a Web-based graphical-user-interface-building subprogram that enables a client who lacks programming expertise to create customized display Web pages.

Myers, Gerry↗

DOE OSTI2025

Levoglucosan data from five coastal streams impacted by the 2020 CZU Lightning Complex Fires, California, United States

This dataset includes levoglucosan data for five coastal California (United States) streams impacted by the 2020 CZU Lightning Complex Fires which burned from August 16th through September 22nd. Levoglucosan is a highly soluble and biolabile fraction of pyrogenic carbon. The five watersheds (San Lorenzo River, Pescadero Creek, Majors Creek, Laguna Creek, and Scott Creek) were impacted by the fires with watersheds experiencing a range of burn severity and extents. Grab samples were collected from each stream between October 2020 and May 2021, targeting both baseflow and event flow hydrologic conditions. Additional biogeochemistry data (i.e., organic and black carbon concentrations) can be found in a separate data package (https://doi.org/10.4211/hs.421c0226bb38460c8393d67fe0c4f802). This data package consists of one main data folder that contains (1) readme; (2) file-level metadata; (3) data dictionary; (4) field metadata with international generic sample numbers (IGSN); (5) methods codes; and (6) levoglucosan data. All files are .csv or .pdf.

2020 CZU Lightning Complex Fires↗

NASA NTRS1997

Monthly Maps of Sea Surface Height in the North Atlantic and Zonal Indices for the Gulf Stream Using TOPEX/Poseidon Altimeter Data

Monthly Maps of sea surface height are constructed for the North Atlantic Ocean using TOPEX/Poseidon altimeter data. Mean sea surface height is reconstructed using a weighted combination of historical, hydrographic data and a synthetic mean obtained by fitting a Gaussian model of the Gulf Stream jet to altimeter data. The resultant mean shows increased resolution over the hydrographic mean, and incorporates recirculation information that is absent in the synthetic mean. Monthly maps, obtained by adding the mean field to altimeter sea surface height residuals, are used to derive a set of zonal indices that describe the annual cycle of meandering as well as position and strength of the Gulf Stream.

GULF STREAM↗

NASA NTRS1991

Burst-Compression And -Expansion For TDMA Communication

Burst-compression and -expansion technique enables interconnection of users transmitting and receiving data at rates asynchronous with respect to clocks within ground terminals of satellite-switched, time-division-multiple-access (TDMA) communication network. Matrix switch aboard satellite routes bursts of data from source users received on uplink antennas to downlink antennas illuminating ground areas containing destination users. TDMA ground terminal compresses streams of data from source users into rapid bursts for transmission and reexpands bursts of received data into slower streams of data for delivery to destination users. Greater flexibility in interconnecting widely dispersed users achieved by use of hopping beams.

Budinger, James M.↗

NASA NTRS2005

Predictive Interfaces for Long-Distance Tele-Operations

We address the development of predictive tele-operator interfaces for humanoid robots with respect to two basic challenges. Firstly, we address automating the transition from fully tele-operated systems towards degrees of autonomy. Secondly, we develop compensation for the time-delay that exists when sending telemetry data from a remote operation point to robots located at low earth orbit and beyond. Humanoid robots have a great advantage over other robotic platforms for use in space-based construction and maintenance because they can use the same tools as astronauts do. The major disadvantage is that they are difficult to control due to the large number of degrees of freedom, which makes it difficult to synthesize autonomous behaviors using conventional means. We are working with the NASA Johnson Space Center's Robonaut which is an anthropomorphic robot with fully articulated hands, arms, and neck. We have trained hidden Markov models that make use of the command data, sensory streams, and other relevant data sources to predict a tele-operator's intent. This allows us to achieve subgoal level commanding without the use of predefined command dictionaries, and to create sub-goal autonomy via sequence generation from generative models. Our method works as a means to incrementally transition from manual tele-operation to semi-autonomous, supervised operation. The multi-agent laboratory experiments conducted by Ambrose et. al. have shown that it is feasible to directly tele-operate multiple Robonauts with humans to perform complex tasks such as truss assembly. However, once a time-delay is introduced into the system, the rate of tele\ioperation slows down to mimic a bump and wait type of activity. We would like to maintain the same interface to the operator despite time-delays. To this end, we are developing an interface which will allow for us to predict the intentions of the operator while interacting with a 3D virtual representation of the expected state of the robot. The predictive interface anticipates the intention of the operator, and then uses this prediction to initiate appropriate sub-goal autonomy tasks.

Wheeler, Kevin R.↗

NASA NTRS2007

Integrated inertial stellar attitude sensor

An integrated inertial stellar attitude sensor for an aerospace vehicle includes a star camera system, a gyroscope system, a controller system for synchronously integrating an output of said star camera system and an output of said gyroscope system into a stream of data, and a flight computer responsive to said stream of data for determining from the star camera system output and the gyroscope system output the attitude of the aerospace vehicle.

Brady, Tye M.↗

NASA NTRS1992

Fast Vector-Quantizing Data Compressor

Proposed encoding digital electronic data-processing system compresses 7-Mb/s input stream of data into 1-Mb/s output stream. Intended to reduce flow of data from orbiting synthetic-aperture-radar (SAR) system to rate low enough for transmission to ground. Features include reliability, light weight, and low power. Proves advantageous in such terrestrial applications as compression of speech and of video signals.

Chang, Chi-Yung↗

NASA NTRS1994

Preliminary estimates of Gulf Stream characteristics from TOPEX data and a precise gravimetric geoid

TOPEX sea surface height data has been used, with a gravimetric geoid, to calculate sea surface topography across the Gulf Stream. This topography was initially computed for nine tracks on cycles 21 to 29. Due to inaccurate geoid undulations on one track, results for eight tracks are reported. The sea surface topography estimates were used to calculate parameters that describe Gulf Stream characteristics from two models of the Gulf Stream. One model was based on a Gaussian representation of the velocity while the other was a hyperbolic representation of velocity or the sea surface topography. The parameters of the Gaussian velocity model fit were a width parameter, a maximum velocity value, and the location of the maximum velocity. The parameters of the hyperbolic sea surface topography model were the width, the height jump, position, and sea surface topography at the center of the stream. Both models were used for the eight tracks and nine cycles studied. Comparisons were made between the width parameters, the maximum velocities, and the height jumps. Some of the parameter estimates were found to be highly (0.9) correlated when the hyperbolic sea surface topography fit was carried out, but such correlations were reduced for either the Gaussian velocity fits or the hyperbolic velocity model fit. A comparison of the parameters derived from 1-year TOPEX data showed good agreement with values derived by Kelly (1991) using 2.5 years of Geosat data near 38 deg N, 66 deg W longitude. Accuracy of the geoid undulations used in the calculations was of order of +/- 16 cm with the accuracy of a geoid undulation difference equal to +/- 15 cm over a 100-km line in areas with good terrestrial data coverage. This paper demonstrates that our knowledge or geoid undulations and undulation differences, in a portion of the Gulf Stream region, is sufficiently accurate to determine characteristics of the jet when used with TOPEX altimeter data. The method used here has not been shown to be more accurate than methods that average altimeter data to form a reference surface used in analysis to obtain the Gulf Stream characteristics. However, the results show the geoid approach may be used in areas where lack of current meandering reduces the accuracy of the average surface procedure.

Rapp, Richard H.↗

NASA NTRS1989

Jet stream winds - Enhanced aircraft data acquisition and analysis over Southwest Asia

A project is described for providing the accurate initial and verification analyses for the jet stream in regions where general circulation models are known to have large systematic errors, either due to the extreme sparsity of data or to incorrect physical parameterizations. For this purpose, finely spaced aircraft-based meteorological data for the Southwest Asian regions collected for three 10-day periods in the winter of 1988-1989 will be used, together with corresponding data for the North American regions used as a control, to rerun the assimilation cycles and forecast models of the NMC and the NASA Goddard Laboratory for Atmospheres. Data for Southeast Asia will be collected by three carriers with extensive wide-body routes crossing the total region, while data for the North American region will be obtained from the archives of ACARS and GTS.

Tenenbaum, J.↗

NASA NTRSDate not supplied

Continuous Spacecraft Communications via Make-Before-Break Antenna Array Beam Steering

Active phased array (APA) antennas can electronically form multiple beams to track several targets simultaneously. Spacecraft equipped with these antennas can achieve continuous communications through a constellation of relay satellites by forming a second beam to an upcoming relay satellite before the spacecraft moves beyond the current relay’s line of sight. These make-before-break operations have utility for streaming critical data without gaps. Using characterization data from a commercially-available APA-based satellite communications terminal, we simulate performance from low-Earth orbit to a representative geosynchronous relay satellite system at Ka-band. Average data rates of 3 Mbps are achievable with a minimum rate of 0.8 Mbps during worst-case handovers. We prototype the beam-splitting algorithm in the terminal hardware and evaluate its performance in an antenna range. In an over-the-air test emulating a handover between two relays we observe error-free data in the combined telemetry stream.

active phased array antennas↗

DOE OSTI2024

An Active Learning-Based Streaming Pipeline for Reduced Data Training of Structure Finding Models in Neutron Diffractometry

Structure determination workloads in neutron diffractometry are computationally expensive and routinely require several hours to many days to determine the structure of a material from its neutron diffraction patterns. The potential for machine learning models trained on simulated neutron scattering patterns to significantly speed up these tasks have been reported recently. However, the amount of simulated data needed to train these models grows exponentially with the number of structural parameters to be predicted and poses a significant computational challenge. To overcome this challenge, we introduce a novel batch-mode active learning (AL) policy that uses uncertainty sampling to simulate training data drawn from a probability distribution that prefers labelled examples about which the model is least certain. We confirm its efficacy in training the same models with ∼ 75% less training data while improving the accuracy. We then discuss the design of an efficient stream-based training workflow that uses this AL policy and present a performance study on two heterogeneous platforms to demonstrate that, compared with a conventional training workflow, the streaming workflow delivers ∼ 20% shorter training time without any loss of accuracy.

Wang, Tianle [Brookhaven National Laboratory (BNL)↗