Search NASA⌕ Search

SEARCH · Search NASA

Results for “data streaming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Generation of Land Surface Reflectance with Combined Geo-KOMPSAT-2A AMI and Himawari 8 AHI Observations

The latest generation of geostationary satellites has opened a new era of Earth observations with unprecedented spatiotemporal resolution and spectral range. Together with GOES 16/17 ABI, FY4-A AGRI, and Himawari-8 AHI, a new Korean geostationary satellite (Geo-KOMPSAT-2A AMI) has operationally collected a full-disk image in 16 channels every ten minutes since July 2019, allowing diurnal land surface monitoring over a large proportion of Asia and all of Oceania. Retrieving accurate surface reflectance (SR) over land from GK-2A/AMI is a challenging but high priority objective. One of the challenges is the absence of a spectral band in the 2.2 m SWIR range from AMI, which is required by many atmospheric correction algorithms to retrieve atmospheric aerosol properties. To remedy this issue, we adopt a strategy that combines concurrent GK-2A/AMI and Himawari 8/AHI observations in order to derive AMI SR. We have adapted the NASA Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm to process the data stream from Himawari 8/AHI. The advantages of the MAIAC algorithm is its capability to exploit the high temporal frequency and varying illumination geometry of the geostationary data for advanced cloud/snow detection, aerosol retrieval, and characterization of surface bidirectional reflectance properties. Leveraging the similarities of spectral bands and the sun-target-sensor geometry between AMI and AHI, we are able to create denser time series of observations and enhanced BRDF samples over most of the spatial coverage of AMI (and AHI). The combined stereo-type observations not only help derive SR for AMI but also enhance retrievals of the corresponding AHI surface products. We evaluate the resulting AMI SR using ground (AERONET) observations and corresponding MODIS products. Further, we discuss potential challenges in utilizing the geostationary satellite data for land surface monitoring.

geostationary satellite↗
NASA NTRSDate not supplied

Actor-based Runtime Verification with MESA

This work presents a runtime verification approach implemented in the tool MESA (MEssage-based System Analysis) which allows for using concurrent monitors to check for properties specified in data parameterized temporal logic and state machines. The tool is implemented as an internal Scala DSL. We employ the actor programming model to implement MESA where monitors are captured by concurrent actors that communicate via messaging. The paper presents a case study in which MESA is used to effectively monitor a large number of flights from live US airspace data streams. We also perform an empirical study by conducting experiments using monitoring systems with different numbers of concurrent monitors and different layers of indexing on the data contained in events. The paper describes the experiments, evaluates the results, and discusses challenges faced during the study. The evaluation shows the value of combining concurrency with indexing to handle data rich events.

runtime verification↗
NASA NTRS1984

Revised radiometric calibration technique for LANDSAT-4 Thematic Mapper data by the Canada Centre for Remote Sensing

Observations of raw image data, raw radiometric calibration data, and background measurements extracted from the raw data streams on high density tape reveal major shortcomings in a technique proposed by the Canadian Center for Remote Sensing in 1982 for the radiometric correction of TM data. Results are presented which correlate measurements of the DC background with variations in both image data background and calibration samples. The effect on both raw data and data corrected using the earlier proposed technique is explained and the correction required for these factors as a function of individual scan line number for each detector is described. How the revised technique can be incorporated into an operational environment is demonstrated.

Murphy, J.↗
NASA NTRS2011

Validation of Real-Time Data Processing for the Ground and Air-MSPI Systems

JPL is currently developing the multi-angle spectro-polarimetric imager (MSPI), targeted for the Aerosol-Cloud-Ecosystems (ACE) mission, as defined in the National Academies 2007 Decadal Survey. In preparation for the space instrument, the MSPI team has built two incremental camera systems (Ground- and Air-MSPI) to improve understanding of the proposed architecture. Ground-MSPI is a gimballed instrument used primarily for stationary observation and characterization of the imager and optics. The ER-2 based Air-MSPI operates in a step-and-stare mode, providing multi-angle imaging of a static target. This mode-of-operation simulates the observation scenario of the space instrument. Physically, MSPI is a pushbroom camera with a specialized frontend. Before imaging, light entering the camera passes through a pair of photoelastic modulators and a set of pattern polarizers. These optical elements act on the light to make polarimetric extraction computationally feasible. Calculating polarimetric parameters from the imager's data stream requires a real-time least-squares computation that produces coefficients of a truncated time-series expansion of the image. As reported in, the data processing algorithm can operate in real-time on a Xilinx Virtex-5 FPGA. Moving beyond verification with an onboard data source, the algorithm has been validated on a commercial development board interfaced with the ground camera. In addition, the algorithm has been instantiated within the Air-MSPI electronics board's FPGA, and in situ first-light has been achieved.

real-time data processing↗
NASA NTRS2020

Actor-based Runtime Verification with MESA

This work presents a runtime verification approach implemented in the tool MESA (MEssage-based System Analysis) which allows for using concurrent monitors to check for properties specified in data parameterized temporal logic and state machines. The tool is implemented as an internal Scala DSL. We employ the actor programming model to implement MESA where monitors are captured by concurrent actors that communicate via messaging. The paper presents a case study in which MESA is used to effectively monitor a large number of flights from live US airspace data streams. We also perform an empirical study by conducting experiments using monitoring systems with different num- bers of concurrent monitors and different layers of indexing on the data contained in events. The paper describes the experiments, evaluates the results, and discusses challenges faced during the study. The evaluation shows the value of combining concurrency with indexing to handle data rich events.

Mehlitz, Peter↗
NASA NTRS2017

Framework for Processing Citizens Science Data for Applications to NASA Earth Science Missions

Citizen science (or crowdsourcing) has drawn much high-level recent and ongoing interest and support. It is poised to be applied, beyond the by-now fairly familiar use of, e.g., Twitter for natural hazards monitoring, to science research, such as augmenting the validation of NASA earth science mission data. This interest and support is seen in the 2014 National Plan for Civil Earth Observations, the 2015 White House forum on citizen science and crowdsourcing, the ongoing Senate Bill 2013 (Crowdsourcing and Citizen Science Act of 2015), the recent (August 2016) Open Geospatial Consortium (OGC) call for public participation in its newly-established Citizen Science Domain Working Group, and NASA's initiation of a new Citizen Science for Earth Systems Program (along with its first citizen science-focused solicitation for proposals). Over the past several years, we have been exploring the feasibility of extracting from the Twitter data stream useful information for application to NASA precipitation research, with both "passive" and "active" participation by the twitterers. The Twitter database, which recently passed its tenth anniversary, is potentially a rich source of real-time and historical global information for science applications. The time-varying set of "precipitation" tweets can be thought of as an organic network of rain gauges, potentially providing a widespread view of precipitation occurrence. The validation of satellite precipitation estimates is challenging, because many regions lack data or access to data, especially outside of the U.S. and in remote and developing areas. Mining the Twitter stream could augment these validation programs and, potentially, help tune existing algorithms. Our ongoing work, though exploratory, has resulted in key components for processing and managing tweets, including the capabilities to filter the Twitter stream in real time, to extract location information, to filter for exact phrases, and to plot tweet distributions. The key step is to process the "precipitation" tweets to be compatible with satellite-retrieved precipitation data. These key components for processing and managing "precipitation" tweets (and additional ones to be developed) are not limited to precipitation, nor are they limited to the Twitter social medium. Indeed, to maximize the value of our work for NASA earth science programs, these components should be generalized and be part of an overall framework for processing citizen science data for science research. In this paper, we outline such a framework.

earth science satellite data↗
NASA NTRS1974

The engineering design integration (EDIN) system

A digital computer program complex for the evaluation of aerospace vehicle preliminary designs is described. The system consists of a Univac 1100 series computer and peripherals using the Exec 8 operating system, a set of demand access terminals of the alphanumeric and graphics types, and a library of independent computer programs. Modification of the partial run streams, data base maintenance and construction, and control of program sequencing are provided by a data manipulation program called the DLG processor. The executive control of library program execution is performed by the Univac Exec 8 operating system through a user established run stream. A combination of demand and batch operations is employed in the evaluation of preliminary designs. Applications accomplished with the EDIN system are described.

Glatt, C. R.↗
DOE OSTI2024

A Computational Review of Privacy-Preserving Mechanisms for the Smart Grid

Smart grid technologies have rapidly become one of the largest and most comprehensive sources of data for the modern utility. For the most part, data streams are seen as an essential tool that enable utilities to carry their day-to-day business operations, but they also create the need for efficient and secure data management strategies. In the context of the smart grid, ensuring data privacy is becoming an increasing concern due to a combination of factors that range from shifts in operational paradigms and rapid technology evolution to changes in legislation. Furthermore, researchers have highlighted the risks associated with improperly protected energy records. For example, energy consumption data from homes could be used to infer the behaviors and habits of home occupants through activity recognition or user profiling (Fan, 2017), which may lead to unfair service pricing, targeted advertising, or other personal security violations. Similarly, Electric Vehicles’ (EVs) charging metadata could be used to reveal private information about the owner such as their payment methods, preferred charging stations, and other locational and timing information that could be used to reconstruct the vehicle owner’s behaviors. The privacy of user data, even when used for statistical analysis or machine learning training processes, also needs to be carefully considered, as an individual’s private traits may still be vulnerable if their inclusion/exclusion greatly impacts the result or could be linked to a public dataset through cross-reference. The breach of user privacy also has severe impacts for organizations that store, transmit, or work on the data in the form of diminishing the public’s trust in them while potentially incurring legal consequences (e.g., fines and suspensions under the European Union General Data Protection Regulation, Health Insurance Portability and Accountability Act, etc.). Because of these risks, several privacy-preserving mechanisms are available to help organizations comply with privacy legislations and prevent the unauthorized and malicious use of user data. In light of these concerns, this report focuses on performing a computational review of privacy-preserving mechanisms that have received a significant amount of interest in literature. It specifically focuses on 1) homomorphic encryption, 2) zero-knowledge proofs, 3) differential privacy, and 4) federated learning. It is worth noting that although many of the methods presented in this document rely on cryptographic primitives, their intent is not to provide perfect secrecy, but rather to enable users to maintain privacy, and thus they shall not be compared or equated to other constructs that are aimed to address cybersecurity constructs.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1982

Concepts for on board satellite image registration. Volume 4: Impact of data set selection on satellite on board signal processing

The NASA NEEDS program goals present a requirement for on-board signal processing to achieve user-compatible, information-adaptive data acquisition. This volume addresses the impact of data set selection on data formatting required for efficient telemetering of the acquired satellite sensor data. More specifically, the FILE algorithm developed by Martin-Marietta provides a means for the determination of those pixels from the data stream effects an improvement in the achievable system throughput. It will be seen that based on the lack of statistical stationarity in cloud cover, spatial distribution periods exist where data acquisition rates exceed the throughput capability. The study therefore addresses various approaches to data compression and truncation as applicable to this sensor mission.

Ruedger, W. H.↗
NASA NTRS2000

A Study of Cloud Radiative Forcing and Feedback

The main objective of the grant proposal was to participate in the CERES (Cloud and Earth's Radiant Energy System) Satellite experiment and perform interdisciplinary investigation of NASA's Earth Observing System (EOS). During the grant period, massive amounts of scientific data from diverse platforms have been accessed, processed and archived for continuing use; several software packages have been developed for integration of different data streams for performing scientific evaluation; extensive validation studies planned have been completed culminating in the development of important algorithms that are being used presently in the operational production of data from the CERES. Contributions to the inter-disciplinary science investigations have been significantly more than originally envisioned. The results of these studies have appeared in several refereed journals and conference proceedings. They are listed at the end of this report.

Ramanathan, Veerabhadran↗
NASA NTRS1989

DNET: A communications facility for distributed heterogeneous computing

This document describes DNET, a heterogeneous data communications networking facility. DNET allows programs operating on hosts on dissimilar networks to communicate with one another without concern for computer hardware, network protocol, or operating system differences. The overall DNET network is defined as the collection of host machines/networks on which the DNET software is operating. Each underlying network is considered a DNET 'domain'. Data communications service is provided between any two processes on any two hosts on any of the networks (domains) that may be reached via DNET. DNET provides protocol transparent, reliable, streaming data transmission between hosts (restricted, initially to DECnet and TCP/IP networks). DNET also provides variable length datagram service with optional return receipts.

Tole, John↗
NASA NTRS2002

High-Rate Communications Outage Recorder Operations for Optimal Payload and Science Telemetry Management Onboard the International Space Station

All International Space Station (ISS) Ku-band telemetry transmits through the High-Rate Communications Outage Recorder (HCOR). The HCOR provides the recording and playback capability for all payload, science, and International Partner data streams transmitting through NASA's Ku-band antenna system. The HCOR is a solid-state memory recorder that provides recording capability to record all eight ISS high-rate data during ISS Loss-of-Signal periods. NASA payloads in the Destiny module are prime users of the HCOR; however, NASDA and ESA will also utilize the HCOR for data capture and playback of their high data rate links from the Kibo and Columbus modules. Marshall Space Flight Center's Payload Operations Integration Center manages the HCOR for nominal functions, including system configurations and playback operations. The purpose of this paper is to present the nominal operations plan for the HCOR and the plans for handling contingency operations affecting payload operations. In addition, the paper will address HCOR operation limitations and the expected effects on payload operations. The HCOR is manifested for ISS delivery on flight 9A with the HCOR backup manifested on flight 11A. The HCOR replaces the Medium-Rate Communications Outage Recorder (MCOR), which has supported payloads since flight 5A.1.

Shell, Michael T.↗
NASA NTRS2024

Artificial Intelligence Medical Support for Long-Duration Space Missions

We envision an artificial intelligence (AI) based system that will provide support and recommendations to the crew medical officer (CMO) and ground flight surgeon during long-duration space missions. Such a system would be pretrained on the knowledgebase of clinical knowledge on Earth, minimizing the amount of Earth data that needs to be transferred into space. Then during deployment, the system would be constantly refined through active learning from diverse streams of data from sensors in the spacecraft, data collected daily from individual astronauts, and human-in-the-loop feedback from the crew. The model could be interrogated for predictions and recommendations on personalized crew health based on the overall status of the spacecraft, medicinal stores, and status of other crew members. Adaptation techniques would be used to incorporate spaceflight data that have very different distributions from the training data due to the extreme environment. Edge computing and the most advanced neuromorphic processing would enable computation in scenarios with low power and bandwidth, while dimensionality reduction would be employed to ensure that the input data streams from spaceflight are as small as possible. In order to realize this long-term vision, several hardware and software aspects need to be developed and assembled. First, models pretrained on Earth biomedical data would need to be evaluated for predictive accuracy, and the best one selected. That model would need to be adapted to learn from diverse, sparse, and inconsistently measured data streams, as well as human-in-the-loop feedback. A data integration, standardization, and dimensionality reduction methodology would need to be developed to handle all data types and feed them into the model. Once the software and data infrastructure is developed, it would need to be integrated with small footprint compute processors and tested in high-radiation, high-vibration, unregulated temperature situations. As a short-term goal, we recommend to focus on the development of the data and model software structure. Several large language models (LLM) already exist that have been trained on Earth biomedical and clinical knowledgebases, including BioMedLLM, Med-PaLM, SPOKE LLM, and Foresight. These models need to be evaluated for accuracy and the best one chosen for a proof-of-concept structure, while maintaining awareness of the accelerating AI field and incorporating any newly improved model architectures as needed. Then, we recommend to develop a database of synthetic data types to mimic the diverse data streams that are expected in a long-duration space mission. This should include environmental and microbial data from the spacecraft, non-invasive data from wearables and point-of-care devices employed by astronauts, and more invasive molecular and physiological monitoring of clinical and biomarker data from astronauts. The data standardization methodology should be developed, and these data streams used to refine the clinical LLM. Several scenarios should be developed that could plausibly come up in a long-duration space mission, and changes or aberrations introduced to the data at specific times to mimic these scenarios. Then, question and answer tasks should be designed to interrogate the model for predictions and recommendations, with acceptable answers already identified.

Artificial Intelligence↗
NASA NTRS1983

Tracking and data relay satellite system (TDRSS) - A worldwide view from space

The development, performance levels, and operational use of the TDRSS satellite system are outlined. The TDRSS spacecraft were conceived in the mid-1960s by NASA as a means of using GEO-positioned satellites to eliminate existing ground stations. The main ground terminal becomes Goddard Space Flight Center, through which users other than the Shuttle can also gain access. The TDRSS functions as a relay vehicle, with very little on-board processing except for status reports inserted into the data stream. Use of the TDRSS system by nonNASA agencies currently costs $110/min for forwards, return, and tracking, $24/min for forward service alone, and $8/min for return service only. The spacecraft can store data on board and dump it to the ground station during the limited hours of operation.

Macoughtry, W. O.↗
NASA NTRS2021

Future SAR Imaging Systems: Goals, Plans, Challenges and Opportunities

Synthetic Aperture Radar (SAR) Earth observation data are becoming increasingly ubiquitous as new spaceborne systems become operational and their data are made available to scientists and applications users. The characteristic of active sensors like SAR to be able to observe Earth independent of weather or solar illumination, coupled with regular data acquisition, fosters reliability and encourages the investment in algorithm and product development toward a beneficial result. As SAR systems typically contain proprietary or nationally important technologies, civilian SAR systems are typically developed with a national focus, or in the case of the European Union, with the Union’s focus. As a result, when viewed from a global perspective, SAR programs can be generally viewed as independent developments, each with their own requirements, schedules, development approaches, and data policies. At the same time, these systems can be expensive, and particularly in an era of increasingly open data policies, coordination of programs could reduce redundancy in observations, increase sampling density and measurement diversity, and improve dependability of data streams in the long term. Since 2018, agency representatives from NASA, ESA, DLR, JAXA, ISRO, ASI, and CONAE have been evaluating the possibilities for programmatic and technical coordination of future SAR systems, data sharing, and scientific exploitation. In this paper, we describe the work in discovering trends and possibilities associated with flight systems, by evaluating current and future plans for SAR systems around the world, and identifying opportunities for coordination.

Zink, Manfred↗
DOE OSTI2025

Modular Autonomous Experimentation for Biological Applications (Full Report)

The Modular Autonomous Research System (MARS) was developed to address the pressing need for faster, more reliable, and more adaptable scientific discovery. Traditional experimentation is limited by manual labor, long cycle times, and fragmented data streams, which constrain the ability to explore complex chemical and materials design spaces. To overcome these limitations, we created an integrated, modular platform that combines laboratory robotics, diverse measurement instruments, and a central data infrastructure with artificial intelligence–driven decision-making. The system links liquid handling robots, robotic arms, and optical plate readers into a closed loop where experiments are executed automatically, data is analyzed in real time, and subsequent experimental conditions are adaptively chosen to maximize information gain. Over the course of the project, MARS was validated on two primary test cases—spectroscopic metal–ligand binding assays and peptide-directed mineralization—which highlighted the system’s ability to handle uncertainty and variability in experimental measurements. To further demonstrate modularity and extensibility, we also established additional testbeds in electrochemistry for catalyst discovery and electrolyte formulation for advanced batteries. The results show that MARS can reliably conduct autonomous campaigns with minimal human intervention, adapt to distinct scientific domains, and provide a scalable model for future self-driving laboratories. This work establishes new capabilities for modular, uncertainty-aware automation and directly supports the need for advanced, data-driven research platforms capable of accelerating discovery across a wide range of scientific and national security missions.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRSDate not supplied

Leveraging VIPER Geotechnical and Mobility Insights for Endurance Rover and Traverse Development

A new generation of lunar surface rover exploration is rapidly approaching. This is evidenced by the rise of commercially developed rovers from private companies (e.g., Lunar Outpost’s MAPP) and government agencies, like NASA’s Volatiles Investigating Polar Exploration Rover (VIPER), the Lunar Terrain Vehicle (LTV), and the Endurance/Intrepid rover concepts, and the INSPIRE mission concept. The drive distance requirements for off-world vehicles are also rapidly increasing. For example, LTV driving requirements dictate a 20 km range on a single battery charge, with an overall lifetime requirement of 13,000 km over a minimum 10-year service life. Similarly, NASA's Endurance-A mission concept requires a 2000 km vehicle traverse distance, which is nearly 50 times further than the USSR's Lunokhod 2, the longest lunar rover mission to date. Previous planetary surface missions on the Moon and Mars have revealed harsh mobility conditions: aggressive rocky terrains on Mars severely damaged the Curiosity rover’s wheels, soft lunar regolith caused the Lunokhod 2 rover to sink and NASA’s Apollo 15 LRV (Lunar Roving Vehicle) spun out due to poor traction, which required manual correction. This demonstrates that mobility challenges like wheel degradation and soft soil driving performance must be accounted for in a mission’s design phase to meet increasingly aggressive mission durations and mobility goals. The success of these long duration missions requires highly reliable and durable mobility platforms in conjunction with a more developed understanding of the geotechnical properties of the lunar surface. The VIPER mission is uniquely posed to inform future missions about mobility performance, wheel durability, and lunar surface soil mechanics of sunlit and shadowed regions through use of data streams that already exist for other mission purposes. VIPER’s mission to the lunar south pole will provide a new perspective into off-world mobility challenges including traversal into permanently shadowed regions. Mobility data from VIPER’s mission will represent a first glimpse into the surface mobility challenges of the lunar South Pole. Utilization of VIPER’s existing capabilities including cameras, inertial measurement unit (IMU) data, motor currents/torque, drill data, and the rover’s suspension state and kinematics provide the opportunity to determine potentially useful information for future rover and traverse designs, as well as on-Earth testing practices. Data products collected during VIPER’s mission could inform design considerations for the Endurance rover much earlier than NASA’s standard “lessons learned” approach as VIPER has a highly tactical mission operations structure. This work outlines the specific links between possible data products from the VIPER mission and their potential impact on the Endurance/Intrepid mission concepts in their early design phases. From VIPER, we will also gain insights for future development of onboard mobility software as VIPER is a way-point driven rover which requires more constant human operation on Earth than some other rover missions (non-lunar surface). Primary VIPER data products and insights of interest include wheel sinkage, wheel slip, wheel degradation, surface soil shear strength, and terrain slope estimation. VIPER data products will be made available via the Planetary Data System’s Geosciences and Cartography and Imaging Sciences nodes, conforming to the PDS4 standard.

mobility↗
DOE OSTI2025

Unsupervised anomaly clustering via offset alignment in multivariate grid sensing data

Modern industries increasingly rely on multi-sensor technologies to acquire complex, high-dimensional data streams, enabling advanced monitoring and control systems. One critical application is online anomaly detection in electrical smart grids, where multivariate and multimodal sensing technologies play a vital role. However, detecting anomalies in such time-series data is challenging due to their inherent temporal dependencies and stochastic behavior. Traditional approaches based on supervised and semi-supervised learning methods depend on labeled datasets, which are often unavailable in real-world scenarios. While unsupervised methods have emerged as promising alternatives, these methods are highly susceptible to noise and outliers commonly present in sensing applications. Furthermore, deep learning-based anomaly detection methods, despite their performance, are often criticized for their black-box nature, limiting their applicability in safety-critical and online environments where interpretability and explainability are paramount. In this work, we propose an unsupervised anomaly clustering method leveraging a cyclic alignment-based offset detection algorithm for multivariate time-series signals. The proposed method is applied to multivariate data collected from vibrational, voltage, and magnetic field sensors deployed in a local grid substation. Our results demonstrate the robustness of the algorithm in accurately clustering various anomalies/events across different sensing modalities. Additionally, we compare the effectiveness of the proposed approach against a simple pattern-based anomaly detection method, which performs well for univariate data but fails to generalize to multivariate and multimodal time-series data.

Mukherjee, Subrata [ORNL] (ORCID:0000000309930338)↗