Search NASA⌕ Search

SEARCH · Search NASA

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38

Prototype architecture for a VLSI level zero processing system

The prototype architecture and implementation of a high-speed level zero processing (LZP) system are discussed. Due to the new processing algorithm and VLSI technology, the prototype LZP system features compact size, low cost, high processing throughput, and easy maintainability and increased reliability. Though extensive control functions have been done by hardware, the programmability of processing tasks makes it possible to adapt the system to different data formats and processing requirements. It is noted that the LZP system can handle up to 8 virtual channels and 24 sources with combined data volume of 15 Gbytes per orbit. For greater demands, multiple LZP systems can be configured in parallel, each called a processing channel and assigned a subset of virtual channels. The telemetry data stream will be steered into different processing channels in accordance with their virtual channel IDs. This super system can cope with a virtually unlimited number of virtual channels and sources. In the near future, it is expected that new disk farms with data rate exceeding 150 Mbps will be available from commercial vendors due to the advance in disk drive technology.

Shi, Jianfei↗

CDEP Consortium on Ocean Data Assimilation for Seasonal-to-Interannual Prediction (ODASI)

The ODASI consortium is focused activity of the NOAA/OGP/Climate Diagnostics and Experimental Prediction Program with the goal of improving ocean data assimilation methods and their implementations in support of seasonal forecasts with coupled general circulation models. The consortium is undertaking coordinated assimilation experiments, with common forcing data sets and common input data streams. With different assimilation systems and different models, we aim to understand what approach works best in improving forecast skill in the equatorial Pacific. The presentation will provide an overview of the consortium goals and plans and recent results focused towards evaluating data impacts.

Rienecker, Michele↗

Global Data Assembly Center (gdac) Report to the GHRSST Science Team

In 2019-2020 the Global Data Assembly Center (GDAC) at NASA’s Physical Oceanography Distributed Active Archive Center (PO.DAAC) provided ingest, archive, distribution and user services for GHRSST operational data streams with improved and evolved tools, services, and tutorials, and interfaced with the user community to address technical inquiries. Several new GHRSST datasets including reprocessed MODIS Aqua/Terra L2P with improved cloud screening were made available. The PO.DAAC participated in the evolving GHRSST data management re architecture and development activities. GHRSST Science Team member Edward Armstrong assumed the role of co-leader of the CEOS SST Virtual Constellation. The following sections summarize and document the specific achievements of the GDAC to the GHRSST community.

Finch, Chris↗

Enabling Seamless Transitions from Experimental to Production HPC for Interactive Workflows

The evolving landscape of scientific computing requires seamless transitions from experimental to production HPC environments for interactive workflows. This paper presents a structured transition pathway developed at OLCF that bridges the gap between development testbeds and production systems. We address both technological and policy challenges, introducing frameworks for data streaming architectures, secure service interfaces, and adaptive resource scheduling for time-sensitive workloads and improved HPC interactivity. Our approach transforms traditional batch-oriented HPC into a more dynamic ecosystem capable of supporting modern scientific workflows that require near real-time data analysis, experimental steering, and cross-facility integration.

Etz, Brian [ORNL] (ORCID:0000000208554863)↗

Atmospheric Radiation Measurement (ARM) airborne field campaign data products between 2013 and 2018

Airborne measurements are pivotal for providing detailed, spatiotemporally resolved information about atmospheric parameters and aerosol and cloud properties, thereby enhancing our understanding of dynamic atmospheric processes. For 30 years, the US Department of Energy (DOE) Office of Science supported an instrumented Gulfstream 1 (G-1) aircraft for atmospheric field campaigns. Data from the final decade of G-1 operations were archived by the Atmospheric Radiation Measurement (ARM) Data Center and made publicly available at no cost to all registered users. To ensure a consistent data format and to improve the accessibility of the ARM airborne data, an integrated dataset was recently developed covering the final 6 years of G-1 operations (2013 to 2018, https://doi.org/10.5439/1999133; Mei and Gaustad, 2024). The integrated dataset includes data collected from 236 flights (766.4 h), which covered the Arctic, the US Southern Great Plains (SGP), the US West Coast, the eastern North Atlantic (ENA), the Amazon Basin in Brazil, and the Sierras de Córdoba range in Argentina. These comprehensive data streams provide much-needed insight into spatiotemporal variability in the thermodynamic quantities and aerosol and cloud properties for addressing essential science questions in Earth system process studies. This paper describes the DOE ARM merged G-1 datasets, including information on the acquisition, data collection challenges and future potentials, and quality control processes. It further illustrates the usage of this merged dataset to evaluate the Energy Exascale Earth System Model (E3SM) with the Earth System Model Aerosol–Cloud Diagnostics (ESMAC Diags) package.

54 ENVIRONMENTAL SCIENCES↗

Wireless augmented reality communication system

The system of the present invention is a highly integrated radio communication system with a multimedia co-processor which allows true two-way multimedia (video, audio, data) access as well as real-time biomedical monitoring in a pager-sized portable access unit. The system is integrated in a network structure including one or more general purpose nodes for providing a wireless-to-wired interface. The network architecture allows video, audio and data (including biomedical data) streams to be connected directly to external users and devices. The portable access units may also be mated to various non-personal devices such as cameras or environmental sensors for providing a method for setting up wireless sensor nets from which reported data may be accessed through the portable access unit. The reported data may alternatively be automatically logged at a remote computer for access and viewing through a portable access unit, including the user's own.

Devereaux, Ann↗

NASA/SPoRT's GOES-R Activities in Support of Product Development, Management, and Training

The NASA Short-term Prediction Research and Transition (SPoRT) Center supports many activities within the GOES-R Proving Grounds (PG). These include the development of imagery from existing instrumentation as a proxy to future Advanced Baseline Imager (ABI) capabilities on GOES-R. The Moderate Resolution Imaging Spectroradiometer (MODIS) and the Visible/Infrared Imager/Radiometer Suite (VIIRS) instruments are used to provide a glimpse of the multi-spectral capabilities that will become the norm as the number of channels and data rate dramatically increase with GOES-R. The NOAA/NWS has plans to provide operational users with all ABI channels at the highest resolution. Data fusion of individual channels into composite red, green, and blue imagery products will assist the end user with this future wave of information. While increasing the efficiency in the operational use of ABI channels, these composites provide only qualitative information. Within the GOES-R PG, SPoRT and other partners are exploring ways to include quantitative information as part of the composite imagery. However, limitations in local hardware processing and/or data bandwidth for users of the GOES-R data stream are challenges to overcome. This presentation will discuss the creation of these composite images as well as possible solutions to address these processing challenges. In a similar manner the Geostationary Lightning Mapper (GLM) to be launched on GOES-R presents several data management challenges. The GLM is a pioneering instrument to quantify total lightning from a geostationary platform. The expected data frequency from the GLM is to be at a sub-minute interval. Users of such a data set may have little experience in handling such a rapid update of information. To assist users, SPoRT is working with the NWS to develop tools within the user fs decision support system to allow tracking and analysis of total lightning from a storm-based perspective. This presentation will discuss the challenges and progress of this tool development work. With new data and products comes the need for user Training. Within the GOES-R PG SPoRT is supporting the demonstration of these future products by providing various training materials to end users. A summary of training provided to operational users will be discussed.

Fuell, Kevin K.↗

Wireless Augmented Reality Communication System

The system of the present invention is a highly integrated radio communication system with a multimedia co-processor which allows true two-way multimedia (video, audio, data) access as well as real-time biomedical monitoring in a pager-sized portable access unit. The system is integrated in a network structure including one or more general purpose nodes for providing a wireless-to-wired interface. The network architecture allows video, audio and data (including biomedical data) streams to be connected directly to external users and devices. The portable access units may also be mated to various non-personal devices such as cameras or environmental sensors for providing a method for setting up wireless sensor nets from which reported data may be accessed through the portable access unit. The reported data may alternatively be automatically logged at a remote computer for access and viewing through a portable access unit, including the user's own.

Devereaux, Ann↗

Wireless Augmented Reality Communication System

The system of the present invention is a highly integrated radio communication system with a multimedia co-processor which allows true two-way multimedia (video, audio, data) access as well as real-time biomedical monitoring in a pager-sized portable access unit. The system is integrated in a network structure including one or more general purpose nodes for providing a wireless-to-wired interface. The network architecture allows video, audio and data (including biomedical data) streams to be connected directly to external users and devices. The portable access units may also be mated to various non-personal devices such as cameras or environmental sensors for providing a method for setting up wireless sensor nets from which reported data may be accessed through the portable access unit. The reported data may alternatively be automatically logged at a remote computer for access and viewing through a portable access unit, including the user's own.

Devereaux, Ann↗

Wireless Augmented Reality Communication System

The system of the present invention is a highly integrated radio communication system with a multimedia co-processor which allows true two-way multimedia (video, audio, data) access as well as real-time biomedical monitoring in a pager-sized portable access unit. The system is integrated in a network structure including one or more general purpose nodes for providing a wireless-to-wired interface. The network architecture allows video, audio and data (including biomedical data) streams to be connected directly to external users and devices. The portable access units may also be mated to various non-personal devices such as cameras or environmental sensors for providing a method for setting up wireless sensor nets from which reported data may be accessed through the portable access unit. The reported data may alternatively be automatically logged at a remote computer for access and viewing through a portable access unit, including the user's own.

Devereaux, Ann↗

Application of Machine Learning to Rotorcraft Health Monitoring

Machine learning is a powerful tool for data exploration and model building with large data sets. This project aimed to use machine learning techniques to explore the inherent structure of data from rotorcraft gear tests, relationships between features and damage states, and to build a system for predicting gear health for future rotorcraft transmission applications. Classical machine learning techniques are difficult, if not irresponsible to apply to time series data because many make the assumption of independence between samples. To overcome this, Hidden Markov Models were used to create a binary classifier for identifying scuffing transitions and Recurrent Neural Networks were used to leverage long distance relationships in predicting discrete damage states. When combined in a workflow, where the binary classifier acted as a filter for the fatigue monitor, the system was able to demonstrate accuracy in damage state prediction and scuffing identification. The time dependent nature of the data restricted data exploration to collecting and analyzing data from the model selection process. The limited amount of available data was unable to give useful information, and the division of training and testing sets tended to heavily influence the scores of the models across combinations of features and hyper-parameters. This work built a framework for tracking scuffing and fatigue on streaming data and demonstrates that machine learning has much to offer rotorcraft health monitoring by using Bayesian learning and deep learning methods to capture the time dependent nature of the data. Suggested future work is to implement the framework developed in this project using a larger variety of data sets to test the generalization capabilities of the models and allow for data exploration.

machine learning↗

Parallel Algorithms for Computing the Tensor-Train Decomposition

The tensor-train (TT) decomposition expresses a tensor in a data-sparse format used in molecular simulations, high-order correlation functions, and optimization. In this paper, we propose four parallelizable algorithms that compute the TT format from various tensor inputs: (1) Parallel-TTSVD for traditional format, (2) PSTT and its variants for streaming data, (3) Tucker2TT for Tucker format, and (4) TT-fADI for solutions of Sylvester tensor equations. We provide theoretical guarantees of accuracy, parallelization methods, scaling analysis, and numerical results. For example, for a d-dimension tensor in $\mathbb{R}$ $n\times∙∙∙$$\times$$n$ a two-sided sketching algorithm PSTT2 is shown to have a memory complexity of $O(n^{[d/2]})$, improving upon $O(n^{d—1})$ from previous algorithms.

97 MATHEMATICS AND COMPUTING↗

Challenges at the Frontiers of Matter and Energy: Transformative Opportunities for Discovery Science

FIVE TRANSFORMATIVE OPPORTUNITIES FOR DISCOVERY SCIENCE As a result of this effort, it has become clear that the progress made to date on the five Grand Challenges has created a springboard for seizing five new Transformative Opportunities that have the potential to further transform key technologies involving matter and energy. These five new Transformative Opportunities and the evidence supporting them are discussed in this new report, “Challenges at the Frontiers of Matter and Energy: Transformative Opportunities for Discovery Science.” Mastering Hierarchical Architectures and Beyond-Equilibrium Matter Complex materials and chemical processes transmute matter and energy, for example from CO2 and water to chemical fuel in photosynthesis, from visible light to electricity in solar cells and from electricity to light in light emitting diodes (LEDs) Such functionality requires complex assemblies of heterogeneous materials in hierarchical architectures that display time-dependent away-from-equilibrium behaviors. Much of the foundation of our understanding of such transformations however, is based on monolithic single- phase materials operating at or near thermodynamic equilibrium. The emergent functionalities enabling next-generation disruptive energy technologies require mastering the design, synthesis, and control of complex hierarchical materials employing dynamic far-from-equilibrium behavior. A key guide in this pursuit is nature, for biological systems prove the power of hierarchical assembly and far- from-equilibrium behavior. The challenges here are many: a description of the functionality of hierarchical assemblies in terms of their constituent parts, a blueprint of atomic and molecular positions for each constituent part, and a synthesis strategy for (a) placing the atoms and molecules in the proper positions for the component parts and (b) arranging the component parts into the required hierarchical structure. Targeted functionality will open the door to significant advances in the harvesting, transforming (e.g., reducing CO2, splitting water, and fixing nitrogen), storing, and use of energy to create new materials, manufacturing processes, and technologies—the lifeblood of human societies and economic growth. Beyond Ideal Materials and Systems: Understanding the Critical Roles of Heterogeneity, Interfaces, and Disorder Real materials, both natural ones and those we engineer, are usually a complex mixture of compositional and structural heterogeneities, interfaces, and disorder across all spatial and temporal scales. It is the fluctuations and disorderly states of these heterogeneities and interfaces that often determine the system’s properties and functionality. Much of our fundamental scientific knowledge is based on “ideal” systems, meaning materials that are observed in “frozen” states or represented by spatially or temporally averaged states. Too often, this approach has yielded overly simplistic models that hide important nuances and do not capture the complex behaviors of materials under realistic conditions. These behaviors drive vital chemical transformations such as catalysis, which initiates most industrial manufacturing processes, and friction and corrosion, the parasitic effects of which cost the U.S. economy billions of dollars annually. Expanding our scientific knowledge from the relative simplicity of ideal, perfectly ordered, or structurally averaged materials to the true complexity of real-world heterogeneities, interfaces, and disorder should enable us to realize enormous benefits in the materials and chemical sciences, which translates to the energy sciences, including solar and nuclear power, hydraulic fracturing, power conversion, airframes, and batteries. Harnessing Coherence in Light and Matter Quantum coherence in light and matter is a measure of the extent to which a wave field vibrates in unison with itself at neighboring points in space and time. Although this phenomenon is expressed at the atomic and electronic scales, it can dominate the macroscopic properties of materials and chemical reactions such as superconductivity and efficient photosynthesis. In recent years, enormous progress has been made in recognizing, manipulating, and exploiting quantum coherence. This progress has already elucidated the role that symmetry plays in protecting coherence in key materials, taught us how to use light to manipulate atoms and molecules, and provided us with increasingly sophisticated techniques for controlling and probing the charges and spins of quantum coherent systems. With the arrival of new sources of coherent light and electron beams, thanks in large part to investments by the U.S. Department of Energy’s Office of Basic Energy Sciences (BES), there is now an opportunity to engineer coherence in heterostructures that incorporate multiple types of materials and to control complex, multistep chemical transformations. This approach will pave the way for quantum information processing and next-generation photovoltaic cells and sensors. Revolutionary Advances in Models, Mathematics, Algorithms, Data, and Computing Science today is benefiting from a convergence of theoretical, mathematical, computational, and experimental capabilities that put us on the brink of greatly accelerating our ability to predict, synthesize, and control new materials and chemical processes, and to understand the complexities of matter across a range of scales. Imagine being able to chart a path through a vast sea of possible new materials to find a select few with desired properties. Instead of the time-honored forward approach, in which materials with desired properties are found through either trial-and-error experiments or lucky accidents, we have the opportunity to inversely design and create new materials that possess the properties we desire. The traditional approach has allowed us to make only a tiny fraction of all the materials that are theoretically possible. The inverse design approach, through the harmonious convergence of theoretical, mathematical, computational, and experimental capabilities, could usher in a virtual cornucopia of new materials with functionalities far beyond what nature can provide. Similarly, enhanced mathematical and computational capabilities significantly enhance our ability to extract physical and chemical insights from vastly larger data streams gathered during multimodal and multidimensional experiments using advanced characterization facilities. Exploiting Transformative Advances in Imaging Capabilities across Multiple Scales Historically, improvements in imaging capabilities have always resulted in improved understanding of scientific phenomena. A prime challenge today is finding ways to reconstruct raw data, obtained by probing and mapping matter across multiple scales, into analyzable images. BES investments in new and improved imaging facilities, most notably synchrotron x-ray sources, free-electron lasers, electron microscopes, and neutron sources, have greatly advanced our powers of observation, as have substantial improvements in laboratory- scale technologies. Furthermore, BES is now planning or actively discussing exciting new capabilities. Taken together, these advances in imaging capabilities provide an opportunity to expand our ability to observe and study matter from the 3D spatial perspectives of today to true “4D” spatially and temporally resolved maps of dynamics that allow quantitative predictions of time-dependent material properties and chemical processes. The knowledge gained will impact data storage, catalyst design, drug delivery, structural materials, and medical implants, to name just a few key technologies. ENABLING SUCCESS Seizing each of these five Transformative Opportunities, as well as accelerating further progress on Grand Challenge research, will require specific, targeted investments from BES in the areas of synthesis, meaning the ability to make the materials and architectures that are envisioned; instrumentation and tools, a category that includes theory and computation; and human capital, the most important asset for advancing the Grand Challenges and Transformative Opportunities. While “Challenges at the Frontiers of Matter and Energy: Transformative Opportunities for Discovery Science” could be viewed as a sequel to the original Grand Challenges report, it breaks much new ground in its assessment of the scientific landscape today versus the scientific landscape just a few years ago. In the original Grand Challenges report, it was noted that if the five Grand Challenges were met, our ability to direct matter and energy would be measured only by the limits of human imagination. This new report shows that, prodded by those challenges, the scientific community is positioned today to seize new opportunities whose impacts promise to be transformative for science and society, as well as dramatically accelerate progress in the pursuit of the original Grand Challenges.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Fractal image compression

Fractals are geometric or data structures which do not simplify under magnification. Fractal Image Compression is a technique which associates a fractal to an image. On the one hand, the fractal can be described in terms of a few succinct rules, while on the other, the fractal contains much or all of the image information. Since the rules are described with less bits of data than the image, compression results. Data compression with fractals is an approach to reach high compression ratios for large data streams related to images. The high compression ratios are attained at a cost of large amounts of computation. Both lossless and lossy modes are supported by the technique. The technique is stable in that small errors in codes lead to small errors in image data. Applications to the NASA mission are discussed.

Barnsley, Michael F.↗

AUTOMATIC DETECTION ALGORITHM OF DYNAMIC PRESSURE PULSES IN THE SOLAR WIND

Dynamic pressure pulses (DPPs) in the solar wind are a significant phenomenon closely related to the solar-terrestrial connection and physical processes of solar wind dynamics. In order to automatically identify DPPs from solar wind measurements, we develop a procedure with a three-step detection algorithm that is able to rapidly select DPPs from the plasma data stream and simultaneously define the transition region where large dynamic pressure variations occur and demarcate the upstream and downstream region by selecting the relatively quiet status before and after the abrupt change in dynamic pressure. To demonstrate the usefulness, efficiency, and accuracy of this procedure, we have applied it to the Wind observations from 1996 to 2008 by successfully obtaining the DPPs. The procedure can also be applied to other solar wind spacecraft observation data sets with different time resolutions.

79 ASTRONOMY AND ASTROPHYSICS↗

Support for the Harvard University Water Vapor and Total Water Instruments for the 2004 NASA WB57 Middle Latitude Cirrus Experiment

In order to improve our understanding of the role clouds play in the climate system, NASA is investing considerable effort in characterizing clouds with instruments ranging from passive remote sensors on board the EOS platforms, to the forthcoming active remote sensors on Cloudsat and Calipso. These missions, when taken together, have the capacity to advance our understanding of the coupling between various components of the hydrologic cycle and the atmospheric circulation, and hold the additional potential of leading to significant improvements in the characterization of cloud feedbacks in global models. This is especially true considering that several of these platforms will be flown in an identical orbit within several minutes of one another-a constellation of satellites known as the A-Train. The algorithms that are being implemented and developed to convert these new data streams from radiance and reflectivity measurements into geophysical parameters invariably rely on some set of simplifymg assumptions and empirical constants. Uncertainties in these relationships lead to poorly understood random and systematic errors in the retrieved properties. This lack of understanding introduces ambiguity in interpreting the data and in using the global data sets for their intended purposes. In light of this, a series of flights with the W57F was proposed to address certain specific issues related to the basic properties of mid latitude cirrus clouds: the NASA WE357 Middle Latitude Cirrus Experiment ("MidCiX"). The science questions addressed are: 1) Can cloud property retrieval algorithms developed for A-Train active and passive remote sensing measurements accurately characterize the microphysical properties of synoptic and convectively generated cirrus cloud systems? 2) What are the relationships between the cirrus particle mass, projected area, and particle size spectrum in various genre of cirrus clouds? 3) Does the present compliment of state of the art in situ cloud probes provide the level of precision and accuracy needed to develop and validate algorithms and to contribute to our understanding of the characteristics and microphysical processes operating in cirrus clouds?

Anderson, James G.↗

Improving an Acoustic Vehicle Detector Using an Iterative Self-Supervision Procedure

In many non-canonical data science scenarios, obtaining, detecting, attributing, and annotating enough high-quality training data is the primary barrier to developing highly effective models. Moreover, in many problems that are not sufficiently defined or constrained, manually developing a training dataset can often overlook interesting phenomena that should be included. To this end, we have developed and demonstrated an iterative self-supervised learning procedure, whereby models are successfully trained and applied to new data to extract new training examples that are added to the corpus of training data. Successive generations of classifiers are then trained on this augmented corpus. Using low-frequency acoustic data collected by a network of infrasound sensors deployed around the High Flux Isotope Reactor and Radiochemical Engineering Development Center at Oak Ridge National Laboratory, we test the viability of our proposed approach to develop a powerful classifier with the goal of identifying vehicles from continuously streamed data and differentiating these from other sources of noise such as tools, people, airplanes, and wind. Using a small collection of exhaustively manually labeled data, we test several implementation details of the procedure and demonstrate its success regardless of the fidelity of the initial model used to seed the iterative procedure. Finally, we demonstrate the method’s ability to update a model to accommodate changes in the data-generating distribution encountered during long-term persistent data collection.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Statistical Decomposition and Machine Learning to Clean In Situ Spaceflight Magnetic Field Measurements

Robust in situ magnetic field measurements are critical to understanding the various mechanisms that couple mass, momentum, and energy throughout our solar system. However, the spacecraft on which magnetometers are often deployed contaminate the magnetic field measurements via onboard subsystems including reaction wheels and magnetorquers. Two magnetometers can be deployed at different distances from the spacecraft to determine an approximation of the interfering field for subsequent removal, but constant data streams from both magnetometers can be impractical due to power and telemetry limitations. Here we propose a method to identify and remove time-varying magnetic interference from sources such as reaction wheels using statistical decomposition and convolutional neural networks, providing high-fidelity magnetic field data even in cases where dual-sensor measurements are not constantly available. For example, a measurement interval from the Parker Solar Probe outboard magnetometer experienced a 95.1% reduction in reaction wheel interference following application of the proposed technique.

magnetic field data↗