SEARCH · Search NASA

Results for “Streaming Data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

All sources NASA NTRS DOE OSTI

DOE OSTI2024

A Computational Review of Privacy-Preserving Mechanisms for the Smart Grid

Smart grid technologies have rapidly become one of the largest and most comprehensive sources of data for the modern utility. For the most part, data streams are seen as an essential tool that enable utilities to carry their day-to-day business operations, but they also create the need for efficient and secure data management strategies. In the context of the smart grid, ensuring data privacy is becoming an increasing concern due to a combination of factors that range from shifts in operational paradigms and rapid technology evolution to changes in legislation. Furthermore, researchers have highlighted the risks associated with improperly protected energy records. For example, energy consumption data from homes could be used to infer the behaviors and habits of home occupants through activity recognition or user profiling (Fan, 2017), which may lead to unfair service pricing, targeted advertising, or other personal security violations. Similarly, Electric Vehicles’ (EVs) charging metadata could be used to reveal private information about the owner such as their payment methods, preferred charging stations, and other locational and timing information that could be used to reconstruct the vehicle owner’s behaviors. The privacy of user data, even when used for statistical analysis or machine learning training processes, also needs to be carefully considered, as an individual’s private traits may still be vulnerable if their inclusion/exclusion greatly impacts the result or could be linked to a public dataset through cross-reference. The breach of user privacy also has severe impacts for organizations that store, transmit, or work on the data in the form of diminishing the public’s trust in them while potentially incurring legal consequences (e.g., fines and suspensions under the European Union General Data Protection Regulation, Health Insurance Portability and Accountability Act, etc.). Because of these risks, several privacy-preserving mechanisms are available to help organizations comply with privacy legislations and prevent the unauthorized and malicious use of user data. In light of these concerns, this report focuses on performing a computational review of privacy-preserving mechanisms that have received a significant amount of interest in literature. It specifically focuses on 1) homomorphic encryption, 2) zero-knowledge proofs, 3) differential privacy, and 4) federated learning. It is worth noting that although many of the methods presented in this document rely on cryptographic primitives, their intent is not to provide perfect secrecy, but rather to enable users to maintain privacy, and thus they shall not be compared or equated to other constructs that are aimed to address cybersecurity constructs.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS1982

Concepts for on board satellite image registration. Volume 4: Impact of data set selection on satellite on board signal processing

The NASA NEEDS program goals present a requirement for on-board signal processing to achieve user-compatible, information-adaptive data acquisition. This volume addresses the impact of data set selection on data formatting required for efficient telemetering of the acquired satellite sensor data. More specifically, the FILE algorithm developed by Martin-Marietta provides a means for the determination of those pixels from the data stream effects an improvement in the achievable system throughput. It will be seen that based on the lack of statistical stationarity in cloud cover, spatial distribution periods exist where data acquisition rates exceed the throughput capability. The study therefore addresses various approaches to data compression and truncation as applicable to this sensor mission.

Ruedger, W. H.↗

NASA NTRS2000

A Study of Cloud Radiative Forcing and Feedback

The main objective of the grant proposal was to participate in the CERES (Cloud and Earth's Radiant Energy System) Satellite experiment and perform interdisciplinary investigation of NASA's Earth Observing System (EOS). During the grant period, massive amounts of scientific data from diverse platforms have been accessed, processed and archived for continuing use; several software packages have been developed for integration of different data streams for performing scientific evaluation; extensive validation studies planned have been completed culminating in the development of important algorithms that are being used presently in the operational production of data from the CERES. Contributions to the inter-disciplinary science investigations have been significantly more than originally envisioned. The results of these studies have appeared in several refereed journals and conference proceedings. They are listed at the end of this report.

Ramanathan, Veerabhadran↗

DOE OSTI2023

Integrating LEO and GEO Observations: Toward Optimal Summertime Satellite Precipitation Retrieval

Abstract Reliable quantitative precipitation estimation with a rich spatiotemporal resolution is vital for understanding the Earth’s hydrological cycle. Precipitation estimation over land and coastal regions is necessary for addressing the high degree of spatial heterogeneity of water availability and demand, and for resolving the extremes that modulate and amplify hazards such as flooding and landslides. Advancements in computation power along with unique high spatiotemporal and spectral resolution data streams from passive meteorological sensors aboard geosynchronous Earth-orbiting (GEO) and low Earth-orbiting (LEO) satellites offer exciting opportunities to retrieve information about surface precipitation phenomena using data-driven machine learning techniques. In this study, the capabilities of U-Net–like architecture are investigated to map instantaneous, summertime surface precipitation intensity at the spatial resolution of 2 km. The calibrated brightness temperature products from the Global Precipitation Measurement (GPM) Microwave Imager (GMI) radiometer are combined with multispectral images (visible, near-infrared, and infrared bands) from the Advanced Baseline Imager (ABI) aboard the GOES-R satellites as main inputs to the U-Net–like precipitation algorithm. Total precipitable water and 2-m temperature from the Global Forecast System (GFS) model are also used as auxiliary inputs to the model. The results show that the U-Net–like algorithm can capture fine-scale patterns and intensity of surface precipitation at high spatial resolution over stratiform and convective precipitation regimes. The evaluations reveal the potential of extracting relevant, high spatial features over complex surface types such as mountainous regions and coastlines. The algorithm allows users to interpret the inputs’ importance and can serve as a starting point for further exploration of precipitation systems within the field of hydrometeorology.

Meteorology & Atmospheric Sciences↗

DOE OSTI2023

Fusing time-varying mosquito data and continuous mosquito population dynamics models

Climate change is arguably one of the most pressing issues affecting the world today and requires the fusion of disparate data streams to accurately model its impacts. Mosquito populations respond to temperature and precipitation in a nonlinear way, making predicting climate impacts on mosquito-borne diseases an ongoing challenge. Data-driven approaches for accurately modeling mosquito populations are needed for predicting mosquito-borne disease risk under climate change scenarios. Many current models for disease transmission are continuous and autonomous, while mosquito data is discrete and varies both within and between seasons. This study uses an optimization framework to fit a non-autonomous logistic model with periodic net growth rate and carrying capacity parameters for 15 years of daily mosquito time-series data from the Greater Toronto Area of Canada. The resulting parameters accurately capture the inter-annual and intra-seasonal variability of mosquito populations within a single geographic region, and a variance-based sensitivity analysis highlights the influence each parameter has on the peak magnitude and timing of the mosquito season. This method can easily extend to other geographic regions and be integrated into a larger disease transmission model. This method addresses the ongoing challenges of data and model fusion by serving as a link between discrete time-series data and continuous differential equations for mosquito-borne epidemiology models.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1989

DNET: A communications facility for distributed heterogeneous computing

This document describes DNET, a heterogeneous data communications networking facility. DNET allows programs operating on hosts on dissimilar networks to communicate with one another without concern for computer hardware, network protocol, or operating system differences. The overall DNET network is defined as the collection of host machines/networks on which the DNET software is operating. Each underlying network is considered a DNET 'domain'. Data communications service is provided between any two processes on any two hosts on any of the networks (domains) that may be reached via DNET. DNET provides protocol transparent, reliable, streaming data transmission between hosts (restricted, initially to DECnet and TCP/IP networks). DNET also provides variable length datagram service with optional return receipts.

Tole, John↗

DOE OSTI2023

Dissolved Oxygen and Temperature Data from the Hyporheic Zone of the East River Watershed July 2017 to October 2018

Dissolved oxygen (DO) is critical for aquatic ecosystems. Our focus is on the long-term DO dynamics in hyporheic zone of rivers, which are a function of both transport (hydrologic exchange between river and hyporheic zone) and uptake by biogeochemical reactions or respiration. The study site is the alpine East River watershed in Colorado, USA, meander A downstream from the pump house. Opti O2 probes were deployed in the water column and directly within the river-bed at 10, 20, and 35 cm depth (38°55'23.38"N, 106°57'4.21"W, 9046.88m) to monitor DO and temperature. A continuous data stream from July 24, 2017 to Oct 24, 2018 was autonomously telemetered to the cloud. This 14-month DO and temperature time series were obtained without any servicing for maintenance or data downloads; additionally the ability to remotely verify probe performance during field deployment was essential to confirm data validity during winter freeze-in and hydrological/weather events, such as spring melt and summer monsoons. We investigate the variations in dissolved oxygen dynamics of this snow-pack dominated watershed during a comparatively low flow water year (2018) and a relatively normal water year (2017), enabled by distinctive, in-situ, high frequency (∆t = 5min) sensors that provided a continuous time-series from the undisturbed study site over multiple seasons.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2002

High-Rate Communications Outage Recorder Operations for Optimal Payload and Science Telemetry Management Onboard the International Space Station

All International Space Station (ISS) Ku-band telemetry transmits through the High-Rate Communications Outage Recorder (HCOR). The HCOR provides the recording and playback capability for all payload, science, and International Partner data streams transmitting through NASA's Ku-band antenna system. The HCOR is a solid-state memory recorder that provides recording capability to record all eight ISS high-rate data during ISS Loss-of-Signal periods. NASA payloads in the Destiny module are prime users of the HCOR; however, NASDA and ESA will also utilize the HCOR for data capture and playback of their high data rate links from the Kibo and Columbus modules. Marshall Space Flight Center's Payload Operations Integration Center manages the HCOR for nominal functions, including system configurations and playback operations. The purpose of this paper is to present the nominal operations plan for the HCOR and the plans for handling contingency operations affecting payload operations. In addition, the paper will address HCOR operation limitations and the expected effects on payload operations. The HCOR is manifested for ISS delivery on flight 9A with the HCOR backup manifested on flight 11A. The HCOR replaces the Medium-Rate Communications Outage Recorder (MCOR), which has supported payloads since flight 5A.1.

Shell, Michael T.↗

NASA NTRS2024

Artificial Intelligence Medical Support for Long-Duration Space Missions

We envision an artificial intelligence (AI) based system that will provide support and recommendations to the crew medical officer (CMO) and ground flight surgeon during long-duration space missions. Such a system would be pretrained on the knowledgebase of clinical knowledge on Earth, minimizing the amount of Earth data that needs to be transferred into space. Then during deployment, the system would be constantly refined through active learning from diverse streams of data from sensors in the spacecraft, data collected daily from individual astronauts, and human-in-the-loop feedback from the crew. The model could be interrogated for predictions and recommendations on personalized crew health based on the overall status of the spacecraft, medicinal stores, and status of other crew members. Adaptation techniques would be used to incorporate spaceflight data that have very different distributions from the training data due to the extreme environment. Edge computing and the most advanced neuromorphic processing would enable computation in scenarios with low power and bandwidth, while dimensionality reduction would be employed to ensure that the input data streams from spaceflight are as small as possible. In order to realize this long-term vision, several hardware and software aspects need to be developed and assembled. First, models pretrained on Earth biomedical data would need to be evaluated for predictive accuracy, and the best one selected. That model would need to be adapted to learn from diverse, sparse, and inconsistently measured data streams, as well as human-in-the-loop feedback. A data integration, standardization, and dimensionality reduction methodology would need to be developed to handle all data types and feed them into the model. Once the software and data infrastructure is developed, it would need to be integrated with small footprint compute processors and tested in high-radiation, high-vibration, unregulated temperature situations. As a short-term goal, we recommend to focus on the development of the data and model software structure. Several large language models (LLM) already exist that have been trained on Earth biomedical and clinical knowledgebases, including BioMedLLM, Med-PaLM, SPOKE LLM, and Foresight. These models need to be evaluated for accuracy and the best one chosen for a proof-of-concept structure, while maintaining awareness of the accelerating AI field and incorporating any newly improved model architectures as needed. Then, we recommend to develop a database of synthetic data types to mimic the diverse data streams that are expected in a long-duration space mission. This should include environmental and microbial data from the spacecraft, non-invasive data from wearables and point-of-care devices employed by astronauts, and more invasive molecular and physiological monitoring of clinical and biomarker data from astronauts. The data standardization methodology should be developed, and these data streams used to refine the clinical LLM. Several scenarios should be developed that could plausibly come up in a long-duration space mission, and changes or aberrations introduced to the data at specific times to mimic these scenarios. Then, question and answer tasks should be designed to interrogate the model for predictions and recommendations, with acceptable answers already identified.

Artificial Intelligence↗

DOE OSTI2024

Congruity of genomic and epidemiological data in modelling of local cholera outbreaks

Cholera continues to be a global health threat. Understanding how cholera spreads between locations is fundamental to the rational, evidence-based design of intervention and control efforts. Traditionally, cholera transmission models have used cholera case-count data. More recently, whole-genome sequence data have qualitatively described cholera transmission. Integrating these data streams may provide much more accurate models of cholera spread; however, no systematic analyses have been performed so far to compare traditional case-count models to the phylodynamic models from genomic data for cholera transmission. Here, we use high-fidelity case-count and whole-genome sequencing data from the 1991 to 1998 cholera epidemic in Argentina to directly compare the epidemiological model parameters estimated from these two data sources. We find that phylodynamic methods applied to cholera genomics data provide comparable estimates that are in line with established methods. Our methodology represents a critical step in building a framework for integrating case-count and genomic data sources for cholera epidemiology and other bacterial pathogens.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2019

Monitoring for Additive Manufacturing Technologies: Report on Progress, Achievements, and Limitations of Monitoring Techniques

The Transformational Challenge Reactor (TCR) program is leveraging additive manufacturing (AM) technologies to produce multiple nuclear components to be assembled into a fully functional microreactor core. AM was selected as the main manufacturing technology of choice for TCR because it has the potential to disrupt the nuclear industry on two fronts: (1) it enables the manufacturing of very complex geometries with optimized and tailored material properties for the intended use of the component, opening up new options for reactor designs, and (2) it allows for a better understanding of the manufacturing process through real-time in situ monitoring, data analytics, and artificial intelligence that can lead to a streamlined qualification and certification process. Within TCR, the development and deployment of a digital platform aims at addressing the latter opportunity. As part of this effort, the collection of pedigree data sets is vital, should these data be generated before, during, or after the manufacturing process. This report focuses on the collection of manufacturing process data using systems instrumentation and monitoring. It presents an inventory of the various monitoring modalities installed on the manufacturing systems by the machine manufacturer or specifically designed by our team to fill information gaps. It also provides details on other relevant data streams available and leverageable to evaluate the manufacturing process on all classes of AM systems investigated during the first year of this program. Through examples, the report gives an overview of the data collected and presents a path forward for the digital platform.

42 ENGINEERING↗

NASA NTRS1983

Tracking and data relay satellite system (TDRSS) - A worldwide view from space

The development, performance levels, and operational use of the TDRSS satellite system are outlined. The TDRSS spacecraft were conceived in the mid-1960s by NASA as a means of using GEO-positioned satellites to eliminate existing ground stations. The main ground terminal becomes Goddard Space Flight Center, through which users other than the Shuttle can also gain access. The TDRSS functions as a relay vehicle, with very little on-board processing except for status reports inserted into the data stream. Use of the TDRSS system by nonNASA agencies currently costs $110/min for forwards, return, and tracking, $24/min for forward service alone, and $8/min for return service only. The spacecraft can store data on board and dump it to the ground station during the limited hours of operation.

Macoughtry, W. O.↗

NASA NTRS2021

Future SAR Imaging Systems: Goals, Plans, Challenges and Opportunities

Synthetic Aperture Radar (SAR) Earth observation data are becoming increasingly ubiquitous as new spaceborne systems become operational and their data are made available to scientists and applications users. The characteristic of active sensors like SAR to be able to observe Earth independent of weather or solar illumination, coupled with regular data acquisition, fosters reliability and encourages the investment in algorithm and product development toward a beneficial result. As SAR systems typically contain proprietary or nationally important technologies, civilian SAR systems are typically developed with a national focus, or in the case of the European Union, with the Union’s focus. As a result, when viewed from a global perspective, SAR programs can be generally viewed as independent developments, each with their own requirements, schedules, development approaches, and data policies. At the same time, these systems can be expensive, and particularly in an era of increasingly open data policies, coordination of programs could reduce redundancy in observations, increase sampling density and measurement diversity, and improve dependability of data streams in the long term. Since 2018, agency representatives from NASA, ESA, DLR, JAXA, ISRO, ASI, and CONAE have been evaluating the possibilities for programmatic and technical coordination of future SAR systems, data sharing, and scientific exploitation. In this paper, we describe the work in discovering trends and possibilities associated with flight systems, by evaluating current and future plans for SAR systems around the world, and identifying opportunities for coordination.

Zink, Manfred↗

DOE OSTI2018

New Los Alamos space instrument goes for a spin

Scientists and engineers at Los Alamos National Laboratory are using a unique centrifuge facility to evaluate a flight-ready telemetry system for evaluating a nuclear weapons test missile launch. "The telemetry unit was designed and manufactured by Los Alamos' Intelligence and Space Research Division," said Telemetry Program Manager Myles Fitzgerald. "Telemetry is a data stream of information on temperature, acceleration, vibration, strain, all the conditions a system encounters during a test flight." Because a test missile flight ultimately ends with the test system completely destroyed, telemetry data from the flight must be collected by receiving stations along the flight path. "The purpose of the centrifuge test is to subject the electronics to a high gravitational load (G-load) that's representative of what the system will experience when re-entering the Earth's atmosphere," said weapons test engineer Alex Cusick.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

DOE OSTI2018

New Los Alamos space instrument goes for a spin

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

DOE OSTI2020

Automated Integration of Continental-Scale Observations in Near-Real Time for Simulation and Analysis of Biosphere–Atmosphere Interactions

The National Ecological Observatory Network (NEON) is a continental-scale observatory with sites across the US collecting standardized ecological observations that will operate for multiple decades. To maximize the utility of NEON data, we envision edge computing systems that gather, calibrate, aggregate, and ingest measurements in an integrated fashion. Edge systems will employ machine learning methods to cross-calibrate, gap-fill and provision data in near-real time to the NEON Data Portal and to High Performance Computing (HPC) systems, running ensembles of Earth system models (ESMs) that assimilate the data. For the first time gridded EC data products and response functions promise to offset pervasive observational biases through evaluating, benchmarking, optimizing parameters, and training new machine learning parameterizations within ESMs all at the same model-grid scale. Leveraging open-source software for EC data analysis, we are already building software infrastructure for integration of near-real time data streams into the International Land Model Benchmarking (ILAMB) package for use by the wider research community. We will present a perspective on the design and integration of end-to-end infrastructure for data acquisition, edge computing, HPC simulation, analysis, and validation, where Artificial Intelligence (AI) approaches are used throughout the distributed workflow to improve accuracy and computational performance.

Durden, David J.↗

DOE OSTI2016

Adaptive Sequential Monte Carlo for Multiple Changepoint Analysis

Process monitoring and control requires detection of structural changes in a data stream in real time. This paper introduces an efficient sequential Monte Carlo algorithm designed for learning unknown changepoints in continuous time. The method is intuitively simple: new changepoints for the latest window of data are proposed by conditioning only on data observed since the most recent estimated changepoint, as these observations carry most of the information about the current state of the process. The proposed method shows improved performance over the current state of the art. Another advantage of the proposed algorithm is that it can be made adaptive, varying the number of particles according to the apparent local complexity of the target changepoint probability distribution. This saves valuable computing time when changes in the changepoint distribution are negligible, and enables re-balancing of the importance weights of existing particles when a significant change in the target distribution is encountered. The plain and adaptive versions of the method are illustrated using the canonical continuous time changepoint problem of inferring the intensity of an inhomogeneous Poisson process, although the method is generally applicable to any changepoint problem. Performance is demonstrated using both conjugate and non-conjugate Bayesian models for the intensity. Lastly, appendices to the article are available online, illustrating the method on other models and applications.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Machine learning on FPGA for event selection

Real-time data processing is a frontier field in experimental particle physics. The application of FPGAs at the trigger level is used by many current and planned experiments (CMS, LHCb, Belle2, PANDA). Usually they use conventional processing algorithms. LHCb has implemented Machine Learning (ML) elements for real-time data processing with a triggered readout system that runs most of the ML algorithms on a computer farm. The work described in this article aims to test the ML-FPGA algorithms for streaming data acquisition. Herein, there are many experiments working in this area and they have a lot in common, but there are many specific solutions for detector and accelerator parameters that are worth exploring further. This report describes the purpose of the work and progress in evaluating the ML-FPGA application.

47 OTHER INSTRUMENTATION↗