SEARCH · Search NASA

Results for “data streaming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Artificial Intelligence for Data Center Operations (AIOps): Cooperative Research and Development (Final Report)

High performance computing data centers will increasingly need to rely on automation to keep pace with exascale growth in compute capability and to manage and optimize the data center environment and facility resources. Artificial intelligence and machine learning approaches provide the means to improve HPC data center operational efficiency, by learning historical trends and training models to operate on real-time data collected from both IT and facilities sources. NREL has developed methods of real-time collection, aggregation and streaming of these data in the ESIF HPC Data Center and has collected a significant dataset of relevant metrics across computer systems, racks, environmental, building and utility sources for research into various predictive analytics problems. HPE's Advanced Technology Group (ATG) is doing comprehensive research into exascale monitoring and management for High Performance Computing (HPC) systems (hereinafter HPE's Data Monitoring/ Management Technology). NREL and HPE will collaborate to add Artificial Intelligence (AI) to NREL's real-time data collection/ aggregation/ streaming system and HPE's Data Monitoring/ Management System, with the goal of improving the operational efficiency of NREL's Energy Systems Integration Facility (ESIF) HPC Data Center through data analytics on both historical and real-time data from IT systems and facilities operations. This collaboration will consist of efforts in Data Management, Data Analytics, and AI/ML Optimization for both manual and autonomous intervention in data center operations. This will be a multi-year, multi-staged effort with a goal towards building capabilities for an Advanced Smart Facility, and demonstration of these techniques in the NREL ESIF HPC Data Center.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2007

Bayesian Analysis of the Cosmic Microwave Background

There is a wealth of cosmological information encoded in the spatial power spectrum of temperature anisotropies of the cosmic microwave background! Experiments designed to map the microwave sky are returning a flood of data (time streams of instrument response as a beam is swept over the sky) at several different frequencies (from 30 to 900 GHz), all with different resolutions and noise properties. The resulting analysis challenge is to estimate, and quantify our uncertainty in, the spatial power spectrum of the cosmic microwave background given the complexities of "missing data", foreground emission, and complicated instrumental noise. Bayesian formulation of this problem allows consistent treatment of many complexities including complicated instrumental noise and foregrounds, and can be numerically implemented with Gibbs sampling. Gibbs sampling has now been validated as an efficient, statistically exact, and practically useful method for low-resolution (as demonstrated on WMAP 1 and 3 year temperature and polarization data). Continuing development for Planck - the goal is to exploit the unique capabilities of Gibbs sampling to directly propagate uncertainties in both foreground and instrument models to total uncertainty in cosmological parameters.

methods - statistical↗

NASA NTRSDate not supplied

A Preliminary Development of The Intelligent Change Detection System (ICDS): Using Machine Learning to Combat Change Blindness in Remote Operation Environments

The emergence of Advance Air Mobility (AAM) will increase the number and types of aerial vehicles operating in shared airspaces, which will subsequently affect the amount of actionable data that ground control station operators (GCSOs) will be expected to manage. In this environment, GCSOs are particularly susceptible to the visual perceptual phenomenon known as change blindness, in which a stimulus undergoes a change without the change being noticed by its observer. A machine agent designed to detect change blindness and mitigate the effects could improve human agent performance in a data-saturated remote operations environment. The objective of this paper is to describe a proof-of-concept system architecture that integrates real-time eye tracking and vehicle telemetry data to prevent human-agent errors resulting from change blindness while operating aircraft from a remote ground control station (GCS).

Real-Time Streaming Data Architecture↗

NASA NTRSDate not supplied

A Preliminary Development of The Intelligent Change Detection System (ICDS): Using Machine Learning to Combat Change Blindness in Remote Operation Environments

Real-Time Streaming Data Architecture↗

NASA NTRS2003

Computer Program Recognizes Patterns in Time-Series Data

A computer program recognizes selected patterns in time-series data like digitized samples of seismic or electrophysiological signals. The program implements an artificial neural network (ANN) and a set of N clocks for the purpose of determining whether N or more instances of a certain waveform, W, occur within a given time interval, T. The ANN must be trained to recognize W in the incoming stream of data. The first time the ANN recognizes W, it sets clock 1 to count down from T to zero; the second time it recognizes W, it sets clock 2 to count down from T to zero, and so forth through the Nth instance. On the N + 1st instance, the cycle is repeated, starting with clock 1. If any clock has not reached zero when it is reset, then N instances of W have been detected within time T, and the program so indicates. The program can readily be encoded in a field-programmable gate array or an application-specific integrated circuit that could be used, for example, to detect electroencephalographic or electrocardiographic waveforms indicative of epileptic seizures or heart attacks, respectively.

Hand, Charles↗

NASA NTRS1996

Ocean observations with EOS/MODIS: Algorithm development and post launch studies

Several significant accomplishments were made during the present reporting period. We have completed our basic study of using the 1.38 micron MODIS band for removal of the effects of thin cirrus clouds and stratospheric aerosol. The results suggest that it should be possible to correct imagery for thin cirrus clouds with optical thicknesses as large as 0.5 to 1.0. We have also acquired reflectance data for oceanic whitecaps during a cruise on the RV Malcolm Baldrige in the Gulf of Mexico. The reflectance spectrum of whitecaps was found to be similar to that for breaking waves in the surf zone measured by Frouin, Schwindling and Deschamps. We installed a CIMEL sun photometer at Fort Jefferson on the Dry Tortugas off Key West in the Gulf of Mexico. The instrument has yielded a continuous stream of data since February. It shows that the aerosol optical thickness at 669 nm is often less than 0.1 in winter. This suggests that the Southern Gulf of Mexico will be an excellent winter site for vicarious calibration. In addition, we completed a study of the effect of vicarious calibration, i.e., the accuracy with which the radiance at the top of the atmosphere (TOA) can be predicted from measurement of the sky radiance at the bottom of the atmosphere (BOA). The results suggest that the neglect of polarization in the aerosol optical property inversion algorithm and in the prediction code for the TOA radiances is the largest error associated with the radiative transfer process. Overall, the study showed that the accuracy of the TOA radiance prediction is now limited by the racliometric calibration error in the sky radiometer. Finally, considerable coccolith light scattering data were obtained in the Gulf of Maine with a flow-through instrument, along with data relating to calcite concentration and the rate of calcite production.

Gordan, Howard R.↗

NASA NTRS2004

Low-Cutoff, High-Pass Digital Filtering of Neural Signals

The figure depicts the major functional blocks of a system, now undergoing development, for conditioning neural signals acquired by electrodes implanted in a brain. The overall functions to be performed by this system can be summarized as preamplification, multiplexing, digitization, and high-pass filtering. Other systems under development for recording neural signals typically contain resistor-capacitor analog low-pass filters characterized by cutoff frequencies in the vicinity of 100 Hz. In the application for which this system is being developed, there is a requirement for a cutoff frequency of 5 Hz. Because the resistors needed to obtain such a low cutoff frequency would be impractically large, it was decided to perform low-pass filtering by use of digital rather than analog circuitry. In addition, it was decided to timemultiplex the digitized signals from the multiple input channels into a single stream of data in a single output channel. The signal in each input channel is first processed by a preamplifier having a voltage gain of approximately 50. Embedded in each preamplifier is a low-pass anti-aliasing filter having a cutoff frequency of approximately 10 kHz. The anti-aliasing filters make it possible to couple the outputs of the preamplifiers to the input ports of a multiplexer. The output of the multiplexer is a single stream of time-multiplexed samples of analog signals. This stream is processed by a main differential amplifier, the output of which is sent to an analog-to-digital converter (ADC). The output of the ADC is sent to a digital signal processor (DSP).

Mojarradi,Mohammad↗

DOE OSTI2026

Model Data Archive Associated with Manuscript "Fire-altered Carbon Pools Create Disturbance Memory in Stream Dissolved Organic Carbon"

This data package supports the publication “Fire-altered Carbon Pools Create Disturbance Memory in Stream Dissolved Organic Carbon” by Li et al. (2026). The package contains processed model inputs, configuration files, restart files, simulation outputs, scripts, and visualization products used to evaluate post-fire dissolved organic carbon (DOC) dynamics in the Naches River Watershed, Washington, USA, following the 2021 Schneider Springs Fire. The modeling workflow couples ELM-BGC, the biogeochemistry-enabled Energy Exascale Earth System Model Land Model; ATS, the Advanced Terrestrial Simulator for integrated surface-subsurface hydrology; and PFLOTRAN, a reactive transport model for multicomponent aqueous geochemistry. Together, these models simulate how wildfire-induced changes in vegetation, litter, coarse woody debris, and soil organic matter influence DOC production, transport, and reaction from burned hillslopes to stream networks. The archive includes preprocessed meteorological, geospatial, hydrologic, and biogeochemical forcing data; ELM-BGC-derived DOC source terms; ATS mesh files; PFLOTRAN reactive-transport inputs; model configuration files; spin-up and transient restart files; watershed-scale diagnostic outputs; stream concentration time series; and figures or visualization files used to inspect and reproduce key results. File types include Hierarchical Data Format 5 (HDF5) files for gridded forcing and model-coupling data, model input and configuration files for ELM-BGC, ATS, and PFLOTRAN, restart and simulation-output files generated by the modeling workflow, tabular or time-series diagnostic outputs, scripts for post-processing and figure generation, and image or visualization products associated with the manuscript. Use of the package depends on the intended task. Re-running the simulations requires the relevant modeling software, including ELM-BGC, ATS, and PFLOTRAN as ATS's geochemical engine. Inspecting outputs and reproducing figures requires Python with scientific plotting libraries such as Matplotlib, and three-dimensional model outputs may be viewed with ParaView. Geographic information system files or maps may be inspected with ArcGIS Pro or comparable GIS software. The data package is intended to enable traceability, reuse, and partial reproduction of the coupled land-to-watershed hydro-biogeochemical modeling workflow used to test how wildfire disturbance affects terrestrial carbon pools and downstream DOC dynamics.

ATS↗

NASA NTRS1993

A high-speed lossless data compression system for space applications

This paper reports on the integration of a lossless data compression/decompression chipset into a space data system architecture. For its compression engine, the data system incorporates the Universal Source Encoder (USE) designed for the NASA/Goddard Space Flight Center. Currently, the data compression testbed generates video frames consisting of 512 lines of 512 pixels having 8-bit resolution. Each image is passed through the USE where the lines are internally partitioned into 16-word blocks. These blocks are adaptively encoded across widely varying entropy levels using a Rice 12-option set coding algorithm. The current system operates at an Input/Output rate of 10 Msamples/s or 80 Mbits/s for each buffered input line. Frame and line synchronization for each image are maintained through the use of uniquely decodable command words. Length information of each variable length compressed image line is also included in the output stream. The data and command information are passed to the next stage of the system architecture through a serial fiber-optic transmitter. The initial segment of this stage consists of packetizer hardware which adds an appropriate CCSDS header to the received source data. An uncompressed mode is optionally available to pass image lines directly to the packetizer hardware. A data decompression testbed has also been developed to confirm the data compression operation.

Miko, Joe↗

NASA NTRS2002

REACH: Real-Time Data Awareness in Multi-Spacecraft Missions

Missions have been proposed that will use multiple spacecraft to perform scientific or commercial tasks. Indeed, in the commercial world, some spacecraft constellations already exist. Aside from the technical challenges of constructing and flying these missions, there is also the financial challenge presented by the tradition model of the flight operations team (FOT) when it is applied to a constellation mission. Proposed constellation missions range in size from three spacecraft to more than 50. If the current ratio of three-to-five FOT personnel per spacecraft is maintained, the size of the FOT becomes cost prohibitive. The Advanced Architectures and Automation Branch at the Goddard Space Flight Center (GSFC Code 588) saw the potential to reduce the cost of these missions by creating new user interfaces to the ground system health-and-safety data. The goal is to enable a smaller FOT to remain aware and responsive to the increased amount of ground system information in a multi-spacecraft environment. Rather than abandon the tried and true, these interfaces were developed to run alongside existing ground system software to provide additional support to the FOT. These new user interfaces have been combined in a tool called REACH. REACH-the Real-time Evaluation and Analysis of Consolidated Health-is a software product that uses advanced visualization techniques to make spacecraft anomalies easy to spot, no matter how many spacecraft are in the constellation. REACH reads a real-time stream of data from the ground system and displays it to the FOT such that anomalies are easy to pick out and investigate. Data visualization has been used in ground system operations for many years. To provide a unique visualization tool, we developed a unique source of data to visualize: the REACH Health Model Engine. The Health Model Engine is rule-based software that receives real-time telemetry information and outputs "health" information related to the subsystems and spacecraft that the telemetry belong to. The Health Engine can run out-of-the-box or can be tailored with a scripting language. Out of the box, it uses limit violations to determine the health of subsystems and spacecraft; when tailored, it determines health using equations combining the values and limits of any telemetry in the spacecraft. The REACH visualizations then "roll up" the information from the Health Engine into high level, summary displays. These summary visualizations can be "zoomed" into for increasing levels of detail. Currently REACH is installed in the Small Explorer (SMEX) lab at GSFC, and is monitoring three of their five spacecraft. We are scheduled to install REACH in the Mid-sized Explorer (MIDEX) lab, which will allow us to monitor up to six more spacecraft. The process of installing and using our "research" software in an operational environment has provided many insights into which parts of REACH are a step forward and which of our ideas are missteps. Our paper explores both the new concepts in spacecraft health-and-safety visualization, the difficulties of such systems in the operational environment, and the cost and safety issues of multi-spacecraft missions.

Maks, Lori↗

NASA NTRS2002

REACH: Real-Time Data Awareness in Multi-Spacecraft Missions

NASA's Advanced Architectures and Automation Branch at the Goddard Space Flight Center (Code 588) saw the potential to reduce the cost of constellation missions by creating new user interfaces to the ground system health-and-safety data. The goal is to enable a small Flight Operations Team (FOT) to remain aware and responsive to the increased amount of ground system information in a multi-spacecraft environment. Rather than abandon the tried and true, these interfaces were developed to run alongside existing ground system software to provide additional support to the FOT. These new user interfaces have been combined in a tool called REACH. REACH-the Real-time Evaluation and Analysis of Consolidated Health-is a software product that uses advanced visualization techniques to make spacecraft anomalies easy to spot, no matter how many spacecraft are in the constellation. REACH reads numerous real-time streams of data from the ground system(s) and displays synthesized information to the FOT such that anomalies are easy to pick out and investigate.

Maks, Lori↗

NASA NTRS1991

Jet stream winds - Comparisons of analyses with independent aircraft data over Southwest Asia

Cruise-level wind data from commercial aircraft are obtained, and these data are compared with operational jet stream analyses over southwest Asia, an area of limited conventional data. Results from an ensemble of 11 cases during January 1989 and individual cases during December 1988-March 1989 are presented. The key results are: (1) European Centre for Medium-Range Weather Forecasts (ECMWF), National Meteorological Center, and United Kingdom Meteorological Office analyses of the subtropical jet in southwest Asia are 11 percent, 17 percent, and 17 percent weaker, respectively, than aircraft observations; (2) analyzed poleward shears range up to 1 f (0.00007/s) compared with up to 3 f (0.00021/s) in the aircraft observations where f is the local Coriolis parameters; (3) the ECMWF errors are largest at the base of the jet; (4) the mean ECMWF core location is latitudinally correct but has an rms latitude variance of 1.5 deg; (5) isolated erroneous radiosondes produce unmeteorological structures in the analyzed subtropical jet stream; and (6) the increased utilization of automated aircraft reports is likely to produce a spurious secular increase in the apparent strength of the jets. The magnitude and spatial extent of the errors seen are near limits of current GCM resolution (100 km) but should be resolvable. The results imply that studies of GCM systematic jet stream wind errors in weather and climate forecasts must be interpreted with caution in this region.

Tenenbaum, J.↗

DOE OSTI2026

Design and simulation of a muon detector to characterize geological overburden

This study presents the design, construction, and simulation of a mobile muon detector tailored for geological overburden characterization. The detector employs plastic scintillator paddles with silicon photomultipliers (SiPMs) and a QuarkNet data acquisition system, offering a portable solution suitable for remote field deployment. The simulator’s modular aluminum frame allows for adjustable geometry and directional sensitivity, while its battery system supports over a week of autonomous operation. Preliminary experimental tests confirmed that its muon flux measurements were consistent with theoretical expectations. A comprehensive simulation framework using Geant4 and CORSIKA was developed to model detector response and overburden effects. Analytical and Monte Carlo methods were used to assess quadrant resolution and infer muon directionality. This work lays the foundation for future overburden mapping and supports the development of reconstruction algorithms for geological applications.

72 - PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2026

A novel method for thermal noise reduction, enabling measurements of broadband, low-amplitude electron temperature fluctuations using individual radiometer channels

A new analysis method has been developed for measurements of broadband, low-amplitude turbulent electron temperature fluctuations in fusion plasmas using individual radiometer channels of a correlation electron cyclotron emission diagnostic. This method takes advantage of differences in the correlation time of thermal noise compared to the correlation time of plasma fluctuations in fusion reactors. The validation of this single-channel method is demonstrated using comparisons with the standard dual-channel radiometer spectral decorrelation method for measurements of turbulent electron temperature fluctuations in the core and edge of low confinement (L), improved confinement (I), and high confinement (H)-mode plasmas at the ASDEX Upgrade tokamak.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2026

Gaia: segmented germanium detector for high-energy X-ray fluorescence and spectroscopic imaging

We present Gaia, a monolithic array of 96 high-purity germanium pixel detectors integrated with a custom low-noise application-specific integrated circuit (ASIC) and a field-programmable gate array (FPGA)-based data acquisition system. The sensor operates at ∼100 K using a commercial closed-cycle cryocooler, with the in-vacuum electronics thermally isolated from the cold finger to ensure thermal stability. The system demonstrates an average energy resolution of 711 eV at 122 keV, measured using a 57 Co source, and 253 eV at 5.89 keV, measured with 55 Fe across all channels. The readout architecture incorporates a high-performance FPGA paired with a dual-core ARM processor, forming a complete embedded Linux-based computing platform. Communication between the processor and FPGA is handled via memory-mapped I/O, and data are streamed over high-speed gigabit Ethernet. A full-scale 384-pixel Gaia detector, based on this 96-element module, is currently under fabrication.

36 MATERIALS SCIENCE↗

DOE OSTI2024

ICED: An Integrated CGRA Framework Enabling DFVS-Aware Acceleration

oarse-grained reconfigurable arrays (CGRAs) are a promising solution to enable energy-efficient acceleration of applications from different domains. By leveraging reconfiguration at the functional level, they can adapt to significantly different computational patterns. Existing CGRA mapping approaches extract instruction-level parallelism, exploit loop-pipelining opportunities, guarantee the data dependency, and target high throughput of a given loop. However, the recurrence data-dependency in the DFG and the mismatch between required and available computing/communication resources complicate the mapping, and might lead to significant unbalances in the utilization of the CGRA's tiles. This results in wasted power for tiles with low utilization. Applying dynamic voltage and frequency scaling (DVFS) can potentially solve this challenge and improve energy efficiency by adjusting voltage and frequency of different tiles independently. CGRAs have also been successful in accelerating data-dependent streaming applications. However, in these applications, the execution time of each kernel in the pipeline might dynamically vary depending on the characteristics of the input. This also leads to under-utilization of resources for the dynamically changing kernels that do not limit the application throughput. DVFS can also improve energy efficiency for these applications by dynamically changing the voltage and frequency levels of tiles that host non performance-constraining kernels. This paper proposes ICEDTEA -- an integrated DVFS-aware framework to map applications on CGRAs that support power islands. ICEDTEA proposes a CGRA architecture supporting DVFS islands at varying granularity (from a single tile to a group of tiles) and the related DVFS-aware compilation and mapping toolchain. ICEDTEA is the first work that introduces DVFS support for spatio-temporal CGRAs at power-island levels. The experimental evaluation shows that ICEDTEA improves average utilization by 2.3$\times$ and energy-efficiency by 1.32$\times$ over a conventional CGRA. With streaming applications, ICEDTEA improves energy efficiency by 1.12$\times$ over a state-of-the-art CGRA that introduces partial dynamic reconfiguration to adapt to variations in kernels' throughput.

Tan, Cheng↗

DOE OSTI2024

interactEM v1.0

An interactive, container-based workflow tool for creating and spawning directed acyclic graphs (DAGs) of operators in a distributed environment. It has a microservices architecture, and flow-based programming model. Current tools like this do not enable streaming of data directly between operators.

Welborn, Sam [Lawrence Berkeley National Laborator↗

DOE OSTI2025

Neutron Source Localization Using the NoMAD He-3 Detector [Slides]

With the goal of advancing criticality safety with neutron localization, the following tasks were successful: (1) demonstrated neutron source localization using a NoMAD detector and ML; and (2) developed a real-time system for streaming in data from NoMAD, verifying that the system works in experimental settings.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗