Results for “data streaming” — Page 4

NASA NTRS2022

A Central Asia Hydrologic Monitoring Dataset for Food and Water Security Applications in Afghanistan

From the Hindu Kush mountains to the Registan Desert, Afghanistan is a diverse landscape where droughts, floods, conflict, and economic market accessibility pose challenges for agricultural livelihoods and food security. The ability to remotely monitor environmental conditions is critical to support decision making for humanitarian assistance. The Famine Early Warning Systems Network (FEWS NET) Land Data Assimilation System (FLDAS) global and Central Asia data streams provide information on hydrologic states for routine integrated food security analysis. While developed for a specific project, these data are publicly available and useful for other applications that require hydrologic estimates of the water and energy balance. These two data streams are unique because of their suitability for routine monitoring, as well as for being a historical record for computing relative indicators of water availability. The global stream is available at ∼ 1-month latency, and monthly average outputs are on a 10 km grid from 1982–present. The second data stream, Central Asia (21–56°N, 30–100°E), at ∼ 1 d latency, provides daily average outputs on a 1 km grid from 2000–present. This paper describes the configuration of the two FLDAS data streams, background on the software modeling framework, selected meteorological inputs and parameters, and results from previous evaluation studies. We also provide additional analysis of precipitation and snow cover over Afghanistan. We conclude with an example of how these data are used in integrated food security analysis. For use in new and innovative studies that will improve understanding of this region, these data are hosted by U.S. Geological Survey data portals and the National Aeronautics and Space Administration (NASA). The Central Asia data described in this paper can be accessed via the NASA repository at https://doi.org/10.5067/VQ4CD3Y9YC0R (Jacob and Slinski, 2021), and the global data described in this paper can be accessed via the NASA repository at https://doi.org/10.5067/5NHC22T9375G (McNally, 2018).

Amy McNally↗

NASA NTRS1991

SAR data compression: Application, requirements, and designs

The feasibility of reducing data volume and data rate is evaluated for the Earth Observing System (EOS) Synthetic Aperture Radar (SAR). All elements of data stream from the sensor downlink data stream to electronic delivery of browse data products are explored. The factors influencing design of a data compression system are analyzed, including the signal data characteristics, the image quality requirements, and the throughput requirements. The conclusion is that little or no reduction can be achieved in the raw signal data using traditional data compression techniques (e.g., vector quantization, adaptive discrete cosine transform) due to the induced phase errors in the output image. However, after image formation, a number of techniques are effective for data compression.

Curlander, John C.↗

NASA NTRS1950

Penetration of Liquid Jets into a High-velocity Air Stream

Data are presented showing the penetration characteristics of liquid jets directed approximately perpendicular to a high-velocity air stream for jet-nozzle-throat diameters from 0.0135 to 0.0625 inch, air stream densities from 0.0805 to 0.1365 pound per cubic foot, liquid jet velocities from 168.1 to 229.0 feet per second and a liquid jet density of approximately 62 pounds per cubic foot. The data were analyzed and a correlation was developed that permitted the determination of the penetration length of the liquid jet for any operation condition within the range of variables investigated.

TURBINES, GAS - LIQUID INJECTION↗

NASA NTRS2002

Operational Limitations of the High Rate Frame Multiplexer (HRFM) Onboard the International Space Station(ISS)-and How These Limitations Affect Payload Developers (PDs) and International Partners (IPs)

The data system onboard the United States Operating Segment (USOS) of the ISS is currently used to capture, route, record and downlink high-rate science data from experiments inside the US Lab. Once NASDA's Japanese Experiment Module (JEM) and ESA's Attached Pressurized Module (APM) are launched - in the 2004 to 2005 timeframe - data from these facilities will also be routed to the ground using the USOS data system. A critical component of the USOS data system is the High Rate Frame Multiplexer (HRFM). The HRFM combines multiple data/video inputs and combines them into one data stream. This Ku-band data stream is then routed through the Tracking and Data Relay Satellite (TDRS) system to the ground. The Data Management Coordinator (DMC) - located at the Marshall Space Flight Center's Payload Operations Center (POC) - is responsible for commanding and controlling the HRFM. The HRFM can multiplex a maximum of eight digital data sources and four digital video sources. Thus far, this limitation has not been constraining to operations. However, once the JEM and APM are integrated, the HRFM limitations will become a major constraint to science operations onboard. The purpose of this paper is to characterize the limitations of the HRFM and to explain how these limitations can be successfully managed. With this information, Payload Developers and International Partners will be able to more effectively utilize the data systems onboard the ISS. Ultimately, more science data can be captured and downlinked to Flight Controllers and Scientists on the ground.

Mixson, Charles D.↗

NASA NTRS2013

Contemplating Synergistic Algorithms for the NASA ACE Mission

ACE is a proposed Tier 2 NASA Decadal Survey mission that will focus on clouds, aerosols, and precipitation as well as ocean ecosystems. The primary objective of the clouds component of this mission is to advance our ability to predict changes to the Earth's hydrological cycle and energy balance in response to climate forcings by generating observational constraints on future science questions, especially those associated with the effects of aerosol on clouds and precipitation. ACE will continue and extend the measurement heritage that began with the A-Train and that will continue through Earthcare. ACE planning efforts have identified several data streams that can contribute significantly to characterizing the properties of clouds and precipitation and the physical processes that force these properties. These include dual frequency Doppler radar, high spectral resolution lidar, polarimetric visible imagers, passive microwave and submillimeter wave radiometry. While all these data streams are technologically feasible, their total cost is substantial and likely prohibitive. It is, therefore, necessary to critically evaluate their contributions to the ACE science goals. We have begun developing algorithms to explore this trade space. Specifically, we will describe our early exploratory algorithms that take as input the set of potential ACE-like data streams and evaluate critically to what extent each data stream influences the error in a specific cloud quantity retrieval.

precipitation↗

NASA NTRSDate not supplied

Earth Independent Medical Operations (EIMO) Datascope: Challenges and Potential Solutions

Data flows and storage/retrieval capacity are severely constrained during missions in space and challenges will become even greater during exploration class missions. There is a need for an artificial intelligence (AI)-based clinical decision support system (CDSS) to monitor and analyze data to provide real-time consultative support for crew medical officer (CMO) decision-making. EIMO is defined as the gradual transition of medical care and decision making from terrestrial to space-based assets, enabling support of astronaut health and performance and reducing overall mission risk. While a hallmark of this paradigm shift from low-earth orbit is that on-board care will increasingly become the responsibility of the astronauts for primary management and decision making, terrestrial assets will continue to be paramount in pre-mission screening and planning, as well as prevention, health maintenance and long-term care contingencies. New capabilities and systems that enable progressively more robust and resilient systems and crews will be necessary to reduce risk and increase probability of deep space exploration mission success. An aspiration for EIMO is to develop AI-enhanced solutions for analysis of crew health & performance data and to facilitate clinical decision support for autonomous medical operations. A “system of systems” approach is envisioned whereby EIMO will deploy AI-supported natural language processing and machine learning (ML) techniques to utilize embedded reference databases and real-time data streams [input vectors] from multiple data sources. Constituent input vectors may include environmental controls, countermeasures data, behavioral data, physiologic wearables, point-of-care laboratory tests, personalized medical records, inventory trade space risk assessments, COTS medical databases, and ground support inputs. An ideal AI capability would possess trained fusion algorithms to cross reference input vectors with medical ‘knowledge’ [cultivated database] to stratify relevant data streams for predictive and actionable capabilities. In addition, EIMO will feature mobility, in that it can be accessed and can push/pull data within and between multiple vehicles/habitats. Large amounts and variable sources of data can be leveraged to diagnose, inform treatment strategies, and potentially predict medical events and performance decrements. Inclusion of advanced training tools using extended reality will enable increasingly autonomous medical care to aid a CMO when ground support is unavailable or time-delayed beyond required action window, e.g., emergent medical situations. EIMO CDSS would require very large datasets to train pre-flight and significant amounts of data are needed to support ML via in-flight CDSS operations. An additional challenge will be to find sufficient data to train a model relevant to astronaut demographics. The rapid, accelerating evolution of this field creates a propitious solution space to leverage multi-modal AI through public-private partnership(s). The status of multi-modal AI systems today would preclude their use for long duration missions as they remain unreliable and are subject to “digital hallucinations” and other errors that could pose operational risk. A federated labs structure is being considered to test and optimize data flow from the multiple input vectors leading to field testing in suitable ground/flight analogs. Critical to the success of an EIMO CDSS will be integration and interoperability and success will be defined by a system that can serve as an in-flight medical consult for the CMO providing critical support during medical contingencies. Benefits to terrestrial medicine may be significant as an outflow of the EIMO medical system, particularly for remote areas and communities lacking significant infrastructure, personnel and resources.

J Lemery↗

NASA NTRS2024

Earth Independent Medical Operations (EIMO) Datascope: Challenges and Potential Solutions

Data flows and storage/retrieval capacity are severely constrained during missions in space and challenges will become even greater during exploration class missions. There is a need for an artificial intelligence (AI)-based clinical decision support system (CDSS) to monitor and analyze data to provide real-time consultative support for crew medical officer (CMO) decision-making. EIMO is defined as the gradual transition of medical care and decision making from terrestrial to space-based assets, enabling support of astronaut health and performance and reducing overall mission risk. While a hallmark of this paradigm shift from low-earth orbit is that on-board care will increasingly become the responsibility of the astronauts for primary management and decision making, terrestrial assets will continue to be paramount in pre-mission screening and planning, as well as prevention, health maintenance and long-term care contingencies. New capabilities and systems that enable progressively more robust and resilient systems and crews will be necessary to reduce risk and increase probability of deep space exploration mission success. An aspiration for EIMO is to develop AI-enhanced solutions for analysis of crew health & performance data and to facilitate clinical decision support for autonomous medical operations. A “system of systems” approach is envisioned whereby EIMO will deploy AI-supported natural language processing and machine learning (ML) techniques to utilize embedded reference databases and real-time data streams [input vectors] from multiple data sources. Constituent input vectors may include environmental controls, countermeasures data, behavioral data, physiologic wearables, point-of-care laboratory tests, personalized medical records, inventory trade space risk assessments, COTS medical databases, and ground support inputs. An ideal AI capability would possess trained fusion algorithms to cross reference input vectors with medical ‘knowledge’ [cultivated database] to stratify relevant data streams for predictive and actionable capabilities. In addition, EIMO will feature mobility, in that it can be accessed and can push/pull data within and between multiple vehicles/habitats. Large amounts and variable sources of data can be leveraged to diagnose, inform treatment strategies, and potentially predict medical events and performance decrements. Inclusion of advanced training tools using extended reality will enable increasingly autonomous medical care to aid a CMO when ground support is unavailable or time-delayed beyond required action window, e.g., emergent medical situations. EIMO CDSS would require very large datasets to train pre-flight and significant amounts of data are needed to support ML via in-flight CDSS operations. An additional challenge will be to find sufficient data to train a model relevant to astronaut demographics. The rapid, accelerating evolution of this field creates a propitious solution space to leverage multi-modal AI through public-private partnership(s). The status of multi-modal AI systems today would preclude their use for long duration missions as they remain unreliable and are subject to “digital hallucinations” and other errors that could pose operational risk. A federated labs structure is being considered to test and optimize data flow from the multiple input vectors leading to field testing in suitable ground/flight analogs. Critical to the success of an EIMO CDSS will be integration and interoperability and success will be defined by a system that can serve as an in-flight medical consult for the CMO providing critical support during medical contingencies. Benefits to terrestrial medicine may be significant as an outflow of the EIMO medical system, particularly for remote areas and communities lacking significant infrastructure, personnel and resources.

Medical Operations↗

NASA NTRS2024

Earth Independent Medical Operations (EIMO) DATASCOPE Technical Interchange Meeting 21st August 2023: Background and Summary of Discussion

An aspiration for EIMO datascope is to realize artificial intelligence-enhanced solutions for analysis of crew health & performance data and to facilitate clinical decision support for autonomous medical operations. A vision proposed to the meeting participants was that of a “system of systems,” whereby EIMO will utilize AI-supported natural language processing and machine learning techniques to synthesize embedded reference databases and real-time data streams [input vectors] from multiple data sources to continuously and seamlessly assess crew health & performance. Constituent input vectors may include environmental controls, countermeasures data, behavioral data, physiologic wearables, point-of-care laboratory tests, personalized medical records, inventory trade space risk assessments, COTS medical databases, and ground support inputs. An ideal AI capability would possess trained fusion algorithms to cross reference input vectors with medical ‘knowledge’ [cultivated database] to stratify relevant data streams for predictive and actionable capabilities. In addition, EIMO will ideally have a degree of mobility, in that it can be accessed and can push/pull data within and between multiple vehicles/habitats.

Artificial Intelligence↗

NASA NTRS2012

Integrated rate isolation sensor

In one embodiment, a system for providing fault-tolerant inertial measurement data includes a sensor for measuring an inertial parameter and a processor. The sensor has less accuracy than a typical inertial measurement unit (IMU). The processor detects whether a difference exists between a first data stream received from a first inertial measurement unit and a second data stream received from a second inertial measurement unit. Upon detecting a difference, the processor determines whether at least one of the first or second inertial measurement units has failed by comparing each of the first and second data streams to the inertial parameter.

Brady, Tye↗

NASA NTRS1973

A variable-data-rate, multimode quadriphase modem.

This paper describes the design and performance of a highly versatile modulator and demodulator recently developed to facilitate the evaluation of various digital communications links. The modem is capable of either PSK or QPSK operation and can accommodate a very wide range of continuously tunable data rates (1 kbps to 30 Mbps in each of two channels). In the QPSK mode, operation is possible using either a single serial data stream (single channel operation) or using two mutually independent, unrelated, and asynchronous data streams (dual-channel operation). Integrate and dump detectors are used at the demodulator for regeneration of the data stream(s). Measurements indicate that the performance of the overall system (including the bit detectors) is within 2 dB of the theoretically optimum performance of either PSK or QPSK at any rate within the range of rates provided by the modem, and is within 1 dB of theoretical over most of the range of rates.

Allen, R. W.↗

NASA NTRS1983

The magnetic-field investigation on ISPM

The International Solar Polar Mission (ISPM) onboard instrumentation for the magnetic field experiment to establish, on the basis of in-situ observations, the heliolatitude dependence of the interplanetary magnetic field, is described. The prime output consists of vector measurements, made by two triaxial magnetometers, of the ambient magnetic field along the orbit of the spacecraft. The onboard data processor generates two data streams to be transmitted through the spacecraft telemetry. The low speed, analog data stream consists of averaged and despun vector measurements, digitized in the spacecraft analog to digital converter (ADC). The despinning algorithm used in the analog processor is described. The high speed, digital data stream consists of vector measurements digitized in the instrument ADCs, generating up to two vector samples per sec. Multiple data-path switching is used to increase system reliability and to allow cross calibration of the ADCs. On board facilities for inflight calibration are described.

Balogh, A.↗

DOE OSTI2025

A Data-Agnostic, Continuous Machine Learning Framework for Application in High Energy Physics and Beyond: Phase 1 Final Scientific/Technical Report

This Phase 1 effort has focused on the development of continual learning frameworks for use in machine learning, specifically in the applied context of High Energy Physics (HEP). Machine learning (ML) is a transformative technology by which computers, typically through the use of neural networks, are able to perform tasks with proficiency that rivals or surpasses that of human users. Model Degradation & Catastrophic Forgetting are two undesired phenomena which can occur in ML where the performance of a model degrades when either deployed on novel data streams, or trained on novel data which are sufficiently different than the data the models were initially trained on. A natural example where these sorts of effects can be observed is in the performance of detectors in harsh environments, where the detector signature may change over the lifetime of the detector as it ages and deteriorates — precisely what occurs in the experiments conducted in HEP. Real world HEP data is therefore an excellent test-ground and use-case for Continual Learning paradigms, which are techniques used in ML to counteract these problems. Ensemble learning is one such technique, where multiple smaller models are trained on subsets of the overall data and are ensembled together during inference. The intuition behind this technique is that, although there are shifts in the distributions which govern the incoming data streams, these shifts are not expected to be homogeneous or global. If a sufficient diversity in solutions within the various sub-models has been achieved, then at least one sub-model is expected to retain its performance within the overall ensemble. One further strength of this approach is that the architectures of the various models do not need to be identical, and in fact even different modalities of data can naturally be combined in this way. This work focused on applying ensemble learning techniques to derive results using two main datasets, anomaly detection in HEP data & time-series forecasting in semiconductor manufacturing data. Semiconductor manufacturing involves data with surprising similarity to that of HEP (e.g. wafer maps look very similar to digi-occupancy maps) and Cerium Lab’s prominence within the semiconductor industry makes semiconductor manufacturing a natural opportunity for commercialization of this work. Our efforts have led to two strong results. The first is that we evaluated the proposed ensembling techniques using previously proposed machine learning architectures for use in anomaly detection, namely AutoEncoder based models and their derivatives. We also developed new architectures which have not been evaluated in this context before. In fact, this work marks the first use of Vision Transformers for anomaly detection in HEP. Second, we demonstrated that ensemble learning significantly improves model performance in scenarios prone to degradation, validating its effectiveness across both HEP and semiconductor datasets. These results further support ensemble learning as a powerful strategy for mitigating catastrophic forgetting and maintaining robust performance in evolving data environments.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2024

StOKeDMD: Streaming Occupation kernel dynamic mode decomposition

Dynamic mode decomposition (DMD) has become a common technique for constructing surrogate models for dynamical systems from observed system states. The Occupation Kernel DMD (OKDMD) method proposed in (Rosenfeld et al., 2022) and (Rosenfeld et al., 2024) is a Liouville operator based method that builds surrogate models from system state trajectories. Here, this paper proposes an extension of OKDMD to the case when the system states are observed in a streaming fashion, i.e., only a small fraction of the state trajectory is available at a given time. The developed method, Streaming Occupation Kernel DMD (StOKeDMD), accommodates the streaming data input by leveraging properties of specific choices of kernel functions and occupation kernels. We apply the StoKeDMD method as a compression method for streaming data, analyze the memory complexity, and demonstrate the performance of StoKeDMD in the compression of streaming data generated from a Lorenz system and a fluid flow simulation.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1991

The IRAS faint source survey

The principal features of the IRAS Faint Source Survey (FSS), a new product resulting from the extended IRAS mission, are reviewed. The FSS has achieved an increase in sensitivity of about a factor of 2.5 relative to the IRAS Point Source Catalog by coadding the data before extracting sources. The FSS was produced by point-source filtering the individual detector data streams and then coadding the data streams using a trimmed-average algorithm. The discussion covers FSS production methods; reliability, completeness, and positional accuracy of the FSS; and FSS view of the IR sky.

Moshir, Mehrdad↗

NASA NTRSDate not supplied

ARM Data as A Resource for Validation of NASA PACE Cloud Retrievals

NASA’s Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) mission will launch in January 2024 and continue and improve upon satellite data records in its eponymous domains. PACE will carry a broad-swath hyperspectral imager, OCI, which will provide MODIS/VIIRS-type cloud data products (i.e. a cloud mask, top height, visible optical thickness, droplet effective radius, phase, and derived water path). It will also carry two multi-angle polarimeters (HARP2 and SPEXone) which will not only provide the above but also enable retrievals of additional cloud properties (e.g. droplet effective variance, ice crystal asymmetry parameter). Validating satellite-based cloud retrievals is challenging. We plan to use several ARM data streams to evaluate PACE cloud data products and are prototyping our analyses using retrievals from MODIS on the Aqua satellite and OLCI on the Sentinel-3A satellite. This poster shows how we plan to use ARM data to evaluate liquid water path (via MWRRET) and cloud top height retrievals (via KARZASRCL), with example results from these proxy sensors and ARM data from the SGP, ENA, and NSA sites. We seek comments from and collaborations with the ARM community to get the most out of our respective data streams.

ARM↗

NASA NTRS2008

Distributing Data to Hand-Held Devices in a Wireless Network

ADROIT is a developmental computer program for real-time distribution of complex data streams for display on Web-enabled, portable terminals held by members of an operational team of a spacecraft-command-and-control center who may be located away from the center. Examples of such terminals include personal data assistants, laptop computers, and cellular telephones. ADROIT would make it unnecessary to equip each terminal with platform- specific software for access to the data streams or with software that implements the information-sharing protocol used to deliver telemetry data to clients in the center. ADROIT is a combination of middleware plus software specific to the center. (Middleware enables one application program to communicate with another by performing such functions as conversion, translation, consolidation, and/or integration.) ADROIT translates a data stream (voice, video, or alphanumerical data) from the center into Extensible Markup Language, effectuates a subscription process to determine who gets what data when, and presents the data to each user in real time. Thus, ADROIT is expected to enable distribution of operations and to reduce the cost of operations by reducing the number of persons required to be in the center.

Hodges, Mark↗

DOE OSTI2025

EJFAT Scientific Perspective

Presented new computing model to the test by deploying the EJFAT system alongside a data-stream processing framework running the production-level CLAS12 event reconstruction application. In this experiment, a continuous stream of CLAS12 Level-1 identified events was processed in real-time using the EJFAT load balancer, distributing the workload across 90 computing nodes located across the U.S. This marks the first-ever large-scale, real-time distributed data stream processing experiment, demonstrating that scientific data-streaming pipelines can efficiently scale across four dimensions, thanks to EJFAT’s advanced hardware and software capabilities.

Gyurjyan, Vardan [Thomas Jefferson National Accele↗

NASA NTRS1977

PN ranging/telemetry transmission

System can transmit range-indicating pseudonoise (PN) codes and simultaneously transmit auxiliary information as binary data at a rate at least on order of pseudonoise chipping rate. PN code is modulated by data stream with relatively low bit rate. Data stream with high bit rate can be transmitted in same frequency band as PN ranging code.

Deerkoski, L. F.↗