Results for “Streaming Data” — Page 32

NASA NTRS2001

Level 1 Processing of MODIS Direct Broadcast Data at the GSFC DAAC

The GSFC DAAC is working to test and package the MODIS Level 1 Processing software for Aqua Direct Broadcast data. This entails the same code base, but different lookup tables for Aqua and Terra. However, the most significant change is the use of ancillary attitude and ephemeris files instead of orbit/attitude information within the science data stream (as with Terra). In addition, we are working on Linux: ports of the algorithms, which could eventually enable processing on PC clusters. Finally, the GSFC DAAC is also working with the GSFC Direct Readout laboratory to ingest Level 0 data from the GSFC DB antenna into the main DAAC, enabling level 1 production in near real time in support of applications users, such as the Synergy project. The mechanism developed for this could conceivably be extended to other participating stations.

Lynnes, Christopher↗

NASA NTRS2017

Evaluation of the Ozone Fields in NASA's MERRA-2 Reanalysis

The assimilated ozone product from the Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2), produced at NASAs Global Modeling and Assimilation Office (GMAO) is summarized. The reanalysis begins in 1980 with the use of retrieved partial-column ozone concentrations from a series of Solar Backscatter Ultraviolet Radiometer (SBUV) instruments on NASA and NOAA spacecraft. Beginning in October 2004, retrieved ozone profiles from the Microwave Limb Sounder (MLS) and total column ozone from the Ozone Monitoring Instrument (OMI) on NASAs EOS Aura satellite are assimilated. While this change in data streams does lead to a discontinuity in the assimilated ozone fields in MERRA-2, making it not useful for studies in decadal (secular) trends in ozone, this choice was made to prioritize demonstrating the value NASAs high-quality research data in the reanalysis context. The MERRA-2 ozone is compared with independent satellite and ozonesonde data, focusing on the representation of the spatial and temporal variability of stratospheric and upper-tropospheric ozone. The comparisons show agreement within 10 (standard deviation of the difference) between MERRA-2 profiles and independent satellite data in most of the stratosphere. The agreement improves after 2004, when EOS Aura data are assimilated. The standard deviation of the differences between the lower-stratospheric and upper-tropospheric MERRA-2 ozone and ozonesondes is 11.2 and 24.5, respectively, with correlations of 0.8 and above. This is indicative of a realistic representation of the UTLS ozone variability in MERRA-2. After 2004, the upper tropospheric ozone in MERRA-2 shows a low bias compared to the sondes, but the covariance with independent observations is improved compared to earlier years. Case studies demonstrate the integrity of MERRA-2 analyses in representing important features such as tropopause folds.

Ozone↗

NASA NTRS2023

Interchangeable Use of GNSS and Seismic Data for Rapid Earthquake Characterization: 2021 Chignik Earthquake, Alaska

Earthquake magnitude estimation using peak ground velocities (PGV) derived from 13 Global Navigation Satellite Systems (GNSS) data has shown promise for rapid 14 characterization of damaging earthquakes. Here we examine the feasibility of using 15 GNSS-derived velocity waveforms as interchangeable data for ground motion estimation 16 and other products that typically rely on strong-motion seismic records. Our study 17 compares PGVs derived from high-rate GNSS to those computed from high-rate seismic 18 records (strong-motion and velocity), at co- and closely-located stations. The recent 2021 Manuscript Click here to access/download;Manuscript;Manuscript_Final.docx 2 19 Mw 8.2 Chignik earthquake in Alaska that was recorded on co-located GNSS and strong20 motion sensors provides the perfect opportunity to compare the two data streams and 21 their application in rapid response. The Chignik velocity records appear almost identical 22 at co-located GNSS and strong-motion stations when observed at frequencies < 0.25 Hz. 23 GNSS and strong-motion derived velocity data are further employed to generate rapid 24 estimates of PGV-derived moment magnitudes for the earthquake. The moment 25 magnitude estimates from GNSS and joint GNSS/seismic data are within ~ ±0.4 26 magnitude units (Fang et al., 2020) of the final magnitude (Mw 8.2). ShakeMaps 27 generated for the 2021 Chignik earthquake using GNSS and seismic PGVs show notable 28 agreement between them, and show negligible shifts in PGV contours when co-/closely 29 located GNSS and seismic stations are substituted for one another. Therefore, we posit 30 that GNSS is a powerful alternative or addition to seismic data and vice versa.

Earthquake rapid response↗

DOE OSTI2024

An Accurate and Automated Convective Vortex Detection Method for Long-Duration Infrasound Microbarometer Data

Heating of the surficial layer of the atmosphere often generates convective vortices, known as “dust devils” when they entrain visible debris. Convective vortices are common on both Earth and Mars, where they affect the climate via dust loading, contribute to wind erosion, impact the efficiency of photovoltaic systems, and potentially result in injury and property damage. However, long-duration terrestrial convective vortex activity records are rare. We have developed a high-precision and high-recall method to extract convective vortex signatures from infrasound microbarometer data streams. The techniques utilizes a wavelet-based detector to capture potential events and then a template matching system to extract the duration of the vortex. Since permanent and temporary infrasound sensors networks are present throughout the globe (many with open data), our method unlocks a vast new convective vortex dataset without requiring the deployment of specialized instrumentation.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

INITIAL STUDIES OF CAVITY FAULT PREDICTION AT JEFFERSON LABORATORY

The Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Laboratory is a CW recirculating linac that utilizes over 400 superconducting radio-frequency (SRF) cavities to accelerate electrons up to 12 GeV through 5-passes. Recent work has shown that, given RF signals from a cavity during a fault as input, machine learning approaches can accurately classify the fault type. In this paper we report on initial results of predicting a fault onset using only data prior to the failure event. A data set was constructed using time-series data immediately before a fault (’unstable’) and 1.5 seconds prior to a fault (’stable’) gathered from over 5,000 saved fault events. The data was used to train a binary classifier. The results gave key insights into the behavior of several fault types and provided motivation to investigate whether data prior to a failure event could also predict the type of fault. We discuss our method using a sliding window approach and report on initial results. Recent modifications to the low-level RF control system will provide access to streaming signals and we outline a path forward for leveraging deep learning on streaming data

Vidyaratne, Lasitha↗

DOE OSTI2021

INITIAL STUDIES OF CAVITY FAULT PREDICTION AT JEFFERSON LABORATORY

The Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Laboratory is a CW recirculating linac that utilizes over 400 superconducting radio-frequency (SRF) cavities to accelerate electrons up to 12 GeV through 5-passes. Recent work has shown that, given RF signals from a cavity during a fault as input, machine learning approaches can accurately classify the fault type. In this paper we report on initial results of predicting a fault onset using only data prior to the failure event. A data set was constructed using time-series data immediately before a fault (’unstable’) and 1.5 seconds prior to a fault (’stable’) gathered from over 5,000 saved fault events. The data was used to train a binary classifier. The results gave key insights into the behavior of several fault types and provided motivation to investigate whether data prior to a failure event could also predict the type of fault. We discuss our method using a sliding window approach and report on initial results. Recent modifications to the low-level RF control system will provide access to streaming signals and we outline a path forward for leveraging deep learning on streaming data

Vidyaratne, Lasitha↗

NASA NTRS1978

Data processing in infrared astronomy

Infrared astronomy is often carried out with rocket probes or orbiting satellite telescopes in order to escape the effects of atmospheric absorption. The data returned from such missions is a highly abstracted digital representation of measurements made by analog detectors. The ability to extract infrared-emission information from these data streams depends on a thorough understanding of the information flow from the telescope aperture to the computer center. This paper reviews the primary elements of this end-to-end concept and the impact of each of these elements on the data processing algorithms, including the division between onboard and ground processing for scientific measurements.

Pelzmann, R. F., Jr.↗

NASA NTRSDate not supplied

Datascope to Enable Earth Independent Medical Operations (EIMO)

BACKGROUND: NASA has amassed sixty years of knowledge and experience relevant to maintenance of crew health and performance in low earth orbit. The Apollo Program introduced the importance of ensuring progressively autonomous operational capability. Earth Independent Medical Operations (EIMO) will require a gradual shift in the balance of medical responsibility, management, and authority from terrestrial to space-based assets. Terrestrial assets will continue to be essential for pre-mission screening and planning in addition to maintenance of crew health and performance. However, new capabilities are needed to enable EIMO and the amount of data required to support these systems, and mitigate the impacts of data transmission delays and reduced bandwidth coupled with lack of cloud-like resources and on-board computing capacity that is currently unclear or operationally insufficient. OVERVIEW: The overall goal of EIMO is to develop artificial intelligence (AI)-based solutions to analyze crew health and performance data utilizing a clinical decision support system (CDSS) to provide crew medical officers (CMO) with the equivalent of real-time, on-board medical consults. The EIMO ecosystem is envisioned as a “system of systems” where embedded reference databases and real-time data streams from multiple input vectors continuously and seamlessly assess crew health and performance. EIMO will be designed to make recommendations to the CMO using multi-modal AI-based natural language processing and machine learning methods with interoperability to push/pull data within and between multiple vehicle and habitat architectures. DISCUSSION: Data flows and storage/retrieval capacity are severely constrained during space missions and the challenges will become even greater during exploration missions. Just as each past program from Mercury to the International Space Station (ISS) required rethinking the interaction between ground-based controllers and space-based crew, so too will future missions to the Moon and Mars. While the NASA High-Performance Spaceflight Computing Processor project aims to increase computational capacity by 100 times over current spaceflight computers, the projected deliverable still lags considerably behind what will be needed to enable an AI-driven CDSS. Restrictions in processing speed and data storage capacity, coupled with transmission bottlenecks and delays, necessitate definition and optimization of an integrated data architecture to enable a progressively autonomous medical capability.

Medical operations↗

DOE OSTI2021

Data Analysis Approach for Large Data Volumes in a Connected Community

Recent advancements within smart neighborhoods where utilities are enabling automatic control of appliances such as heating, ventilation, and air conditioning (HVAC) and water heater (WH) systems are providing new opportunities to minimize energy costs through reduced peak load. This requires systematic collection, storage, management, and in-memory processing of large volumes of streaming data for fast performance. In this paper, we propose a multi-tier layered IoT software framework that enables effective descriptive and predictive data analysis for understanding live operation of the neighborhood, fault identification, and future opportunities for further optimization of load curves. We then demonstrate how we achieve live situational awareness of the connected neighborhood through a suite of visualization components. Finally, we discuss a few analytic dashboards that address questions such as peak load reductions obtained due to optimization, customer preference for automatic control of appliances (do they override the automatic control of HVAC?, etc.). 1 1 This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a nonexclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

Chinthavali, Supriya↗

NASA NTRS1989

Prototype architecture for a VLSI level zero processing system

The prototype architecture and implementation of a high-speed level zero processing (LZP) system are discussed. Due to the new processing algorithm and VLSI technology, the prototype LZP system features compact size, low cost, high processing throughput, and easy maintainability and increased reliability. Though extensive control functions have been done by hardware, the programmability of processing tasks makes it possible to adapt the system to different data formats and processing requirements. It is noted that the LZP system can handle up to 8 virtual channels and 24 sources with combined data volume of 15 Gbytes per orbit. For greater demands, multiple LZP systems can be configured in parallel, each called a processing channel and assigned a subset of virtual channels. The telemetry data stream will be steered into different processing channels in accordance with their virtual channel IDs. This super system can cope with a virtually unlimited number of virtual channels and sources. In the near future, it is expected that new disk farms with data rate exceeding 150 Mbps will be available from commercial vendors due to the advance in disk drive technology.

Shi, Jianfei↗

NASA NTRS2005

CDEP Consortium on Ocean Data Assimilation for Seasonal-to-Interannual Prediction (ODASI)

The ODASI consortium is focused activity of the NOAA/OGP/Climate Diagnostics and Experimental Prediction Program with the goal of improving ocean data assimilation methods and their implementations in support of seasonal forecasts with coupled general circulation models. The consortium is undertaking coordinated assimilation experiments, with common forcing data sets and common input data streams. With different assimilation systems and different models, we aim to understand what approach works best in improving forecast skill in the equatorial Pacific. The presentation will provide an overview of the consortium goals and plans and recent results focused towards evaluating data impacts.

Rienecker, Michele↗

NASA NTRS2020

Global Data Assembly Center (gdac) Report to the GHRSST Science Team

In 2019-2020 the Global Data Assembly Center (GDAC) at NASA’s Physical Oceanography Distributed Active Archive Center (PO.DAAC) provided ingest, archive, distribution and user services for GHRSST operational data streams with improved and evolved tools, services, and tutorials, and interfaced with the user community to address technical inquiries. Several new GHRSST datasets including reprocessed MODIS Aqua/Terra L2P with improved cloud screening were made available. The PO.DAAC participated in the evolving GHRSST data management re architecture and development activities. GHRSST Science Team member Edward Armstrong assumed the role of co-leader of the CEOS SST Virtual Constellation. The following sections summarize and document the specific achievements of the GDAC to the GHRSST community.

Finch, Chris↗

DOE OSTI2025

Enabling Seamless Transitions from Experimental to Production HPC for Interactive Workflows

The evolving landscape of scientific computing requires seamless transitions from experimental to production HPC environments for interactive workflows. This paper presents a structured transition pathway developed at OLCF that bridges the gap between development testbeds and production systems. We address both technological and policy challenges, introducing frameworks for data streaming architectures, secure service interfaces, and adaptive resource scheduling for time-sensitive workloads and improved HPC interactivity. Our approach transforms traditional batch-oriented HPC into a more dynamic ecosystem capable of supporting modern scientific workflows that require near real-time data analysis, experimental steering, and cross-facility integration.

Etz, Brian [ORNL] (ORCID:0000000208554863)↗

DOE OSTI2024

Atmospheric Radiation Measurement (ARM) airborne field campaign data products between 2013 and 2018

Airborne measurements are pivotal for providing detailed, spatiotemporally resolved information about atmospheric parameters and aerosol and cloud properties, thereby enhancing our understanding of dynamic atmospheric processes. For 30 years, the US Department of Energy (DOE) Office of Science supported an instrumented Gulfstream 1 (G-1) aircraft for atmospheric field campaigns. Data from the final decade of G-1 operations were archived by the Atmospheric Radiation Measurement (ARM) Data Center and made publicly available at no cost to all registered users. To ensure a consistent data format and to improve the accessibility of the ARM airborne data, an integrated dataset was recently developed covering the final 6 years of G-1 operations (2013 to 2018, https://doi.org/10.5439/1999133; Mei and Gaustad, 2024). The integrated dataset includes data collected from 236 flights (766.4 h), which covered the Arctic, the US Southern Great Plains (SGP), the US West Coast, the eastern North Atlantic (ENA), the Amazon Basin in Brazil, and the Sierras de Córdoba range in Argentina. These comprehensive data streams provide much-needed insight into spatiotemporal variability in the thermodynamic quantities and aerosol and cloud properties for addressing essential science questions in Earth system process studies. This paper describes the DOE ARM merged G-1 datasets, including information on the acquisition, data collection challenges and future potentials, and quality control processes. It further illustrates the usage of this merged dataset to evaluate the Energy Exascale Earth System Model (E3SM) with the Earth System Model Aerosol–Cloud Diagnostics (ESMAC Diags) package.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2006

Wireless augmented reality communication system

The system of the present invention is a highly integrated radio communication system with a multimedia co-processor which allows true two-way multimedia (video, audio, data) access as well as real-time biomedical monitoring in a pager-sized portable access unit. The system is integrated in a network structure including one or more general purpose nodes for providing a wireless-to-wired interface. The network architecture allows video, audio and data (including biomedical data) streams to be connected directly to external users and devices. The portable access units may also be mated to various non-personal devices such as cameras or environmental sensors for providing a method for setting up wireless sensor nets from which reported data may be accessed through the portable access unit. The reported data may alternatively be automatically logged at a remote computer for access and viewing through a portable access unit, including the user's own.

Devereaux, Ann↗

NASA NTRS2012

NASA/SPoRT's GOES-R Activities in Support of Product Development, Management, and Training

The NASA Short-term Prediction Research and Transition (SPoRT) Center supports many activities within the GOES-R Proving Grounds (PG). These include the development of imagery from existing instrumentation as a proxy to future Advanced Baseline Imager (ABI) capabilities on GOES-R. The Moderate Resolution Imaging Spectroradiometer (MODIS) and the Visible/Infrared Imager/Radiometer Suite (VIIRS) instruments are used to provide a glimpse of the multi-spectral capabilities that will become the norm as the number of channels and data rate dramatically increase with GOES-R. The NOAA/NWS has plans to provide operational users with all ABI channels at the highest resolution. Data fusion of individual channels into composite red, green, and blue imagery products will assist the end user with this future wave of information. While increasing the efficiency in the operational use of ABI channels, these composites provide only qualitative information. Within the GOES-R PG, SPoRT and other partners are exploring ways to include quantitative information as part of the composite imagery. However, limitations in local hardware processing and/or data bandwidth for users of the GOES-R data stream are challenges to overcome. This presentation will discuss the creation of these composite images as well as possible solutions to address these processing challenges. In a similar manner the Geostationary Lightning Mapper (GLM) to be launched on GOES-R presents several data management challenges. The GLM is a pioneering instrument to quantify total lightning from a geostationary platform. The expected data frequency from the GLM is to be at a sub-minute interval. Users of such a data set may have little experience in handling such a rapid update of information. To assist users, SPoRT is working with the NWS to develop tools within the user fs decision support system to allow tracking and analysis of total lightning from a storm-based perspective. This presentation will discuss the challenges and progress of this tool development work. With new data and products comes the need for user Training. Within the GOES-R PG SPoRT is supporting the demonstration of these future products by providing various training materials to end users. A summary of training provided to operational users will be discussed.

Fuell, Kevin K.↗

NASA NTRS2014

Wireless Augmented Reality Communication System

The system of the present invention is a highly integrated radio communication system with a multimedia co-processor which allows true two-way multimedia (video, audio, data) access as well as real-time biomedical monitoring in a pager-sized portable access unit. The system is integrated in a network structure including one or more general purpose nodes for providing a wireless-to-wired interface. The network architecture allows video, audio and data (including biomedical data) streams to be connected directly to external users and devices. The portable access units may also be mated to various non-personal devices such as cameras or environmental sensors for providing a method for setting up wireless sensor nets from which reported data may be accessed through the portable access unit. The reported data may alternatively be automatically logged at a remote computer for access and viewing through a portable access unit, including the user's own.

Devereaux, Ann↗

NASA NTRS2014

Wireless Augmented Reality Communication System

The system of the present invention is a highly integrated radio communication system with a multimedia co-processor which allows true two-way multimedia (video, audio, data) access as well as real-time biomedical monitoring in a pager-sized portable access unit. The system is integrated in a network structure including one or more general purpose nodes for providing a wireless-to-wired interface. The network architecture allows video, audio and data (including biomedical data) streams to be connected directly to external users and devices. The portable access units may also be mated to various non-personal devices such as cameras or environmental sensors for providing a method for setting up wireless sensor nets from which reported data may be accessed through the portable access unit. The reported data may alternatively be automatically logged at a remote computer for access and viewing through a portable access unit, including the user's own.

Devereaux, Ann↗