Search NASA⌕ Search

SEARCH · Search NASA

Results for “datastreams”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

An architecture for consolidating multidimensional time-series data onto a common coordinate grid

Consolidating measurement data for use by data models or in inter-comparison studies frequently requires transforming the data onto a common grid. Standard methods for interpolating multidimensional data are often not appropriate for data with non-homogenous dimensionality, and are hard to implement in a consistent manner for different datastreams. These challenges are increased when dealing with the automated procedures necessary for use with continuous, operational datastreams. In this paper we introduce a method of applying a series of one-dimensional transformations to merge data onto a common grid, examine the challenges of ensuring consistent application of data consolidation methods, present a framework for addressing those challenges, and describe the implementation of such a framework for the Atmospheric Radiation Measurement (ARM) program.

coordinate transformation↗
DOE OSTI2024

Online Learning of Effective Turbine Wind Speed in Wind Farms

To develop better wind farm controllers that can meet more complex objectives, methods of modeling the wind turbine wakes at low computational expense are needed. Gaussian process (GP) regression offers a computationally inexpensive framework for learning complex functions from noisy measurements with very few datapoints. In this work, an online learning approach is presented to learn the rotor-averaged wind velocity at downstream wind turbines with GPs, using the available datastream of wind field measurements and wind turbine control set-points. This framework can readily be integrated into model-based controls methods because the model a) is updated online at low computational expense, b) assumes a mathematically favorable Gaussian form, and c) explicitly quantifies the stochastic nature of the wake field so that the trade-off between exploration and exploitation, and the uncertainty in the prediction, can be utilized. We show that a GP-learned model can match true values with errors within 0.5% on average, with as few as 5 training data points.

Gaussian process↗
DOE OSTI2020

Scalable Linked Dynamic Equilibrium (SLiDE)

SLiDE (Scalable Linked Dynamic Equilibrium) replicates the blueNOTE (National Open source Tools for general Equilibrium analysis) datastream from the Wisconsin National Data Consortium. From there, it produces a recursive dynamic computable general equilibrium model of the US states and eventually the global/national economies. The datastream operations have been set up to allow for scalability of both regions and sectors represented. The model has the capability to flexibly define functional forms similar to other general equilibrium model construction software.

Hughes, Caroline↗
DOE OSTI2020

Description of the LASSO Data Bundles Product

The U. S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) user facility began a pilot project in May 2015 to design a routine, high-resolution modeling capability to complement ARM’s extensive suite of measurements. This modeling capability has evolved into the Large-Eddy Simulation (LES) ARM Symbiotic Simulation and Observation (LASSO) datastream. The datastream, broadly termed data bundles, contains high-resolution model output, input files, observations for evaluation, and skill scores for the simulations. The initial focus of LASSO is on shallow convection at the ARM Southern Great Plains (SGP) atmospheric observatory. The availability of LES simulations with concurrent observations serves many purposes. LES helps bridge the scale gap between DOE ARM observations and models, and the use of routine LES adds value to observations. It provides a self-consistent representation of the atmosphere and a dynamical context for the observations. Further, it elucidates unobservable processes and properties. LASSO generates a simulation library for researchers that enables statistical approaches beyond a single-case mentality. It also provides tools necessary for modelers to reproduce the LES and conduct their own sensitivity experiments. The LASSO library of data bundles is designed to facilitate a wide range of research. For an observationalist, LASSO can help inform instrument remote-sensing retrievals, conduct observation system simulation experiments (OSSEs), and test implications of radar scan strategies or flight paths. For a theoretician, LASSO can help calculate estimates of fluxes and co-variability of values, and test relationships without having to run the model personally. For a modeler, LASSO can help one know ahead of time which days have good forcing, have co-registered observations at high-resolution scales, and have simulation inputs and corresponding outputs to test parameterizations. Further details on the overall LASSO project are available at https://www.arm.gov/capabilities/modeling/lasso.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Network Models of Active Degradation Mechanisms and Pathways for Service Life Prediction of Indoor and Outdoor PV Modules

ct: PV service lifetime prediction (SLP) enables accurate calculation of levelized cost of energy (LCOE), which is crucial to rationalizing PV investment and installation. However, SLP is challeging since PV reliability in the field is affected by many combined factors, including various environmental stresses and module quality. In order to map out the active degradation mechanisms and pathways that best resemble real world conditions, we introduce the framework of a study protocol and use network models fitted to data, to enable analysis and SLP of complex PV systems with multiple active degradation mechanisms. The study protocol is the experimental design, including module variants and different exposure conditions, selection of evaluation methods, time-series data acquisition and training of network models to these data. We present SLP of minimodules in the lab and PV systems in the field. For lab SLP, minimodules with 8 variants based on manufacturer, architecture, and encapsulation were prepared and aged in modified damp heat with or without full spectrum light exposure. Stepwise I-V and Suns-Voc data acquisition tracks changes in electrical properties including Rs,IV, Isc,IV, Vmp,PIV providing insights into power loss of minimodules. Network structural equation modeling (netSEM) was utilized to construct degradation pathway models that identify active degradation mechanisms and predict power loss over time. For field SLP, datastreams of Pmp values and I-V curve datastreams of two types of modules installed in three distinctly different Köppen-Geiger climate zones for 9 years were acquired. With power loss modes corresponding to uniform current loss (ΔPIsc), recombination (ΔPVoc), series resistance (ΔPRs), and current mismatch (ΔPImis) determined, the performance loss rates (PLR) were determined using PVplr. We show how to establish a study protocol framework to ensure appropriate parametric variations and valid data collection from the variants of your complex systems. Then the data-driven netSEM model fitting provides a comprehensive mapping of multiple active degradation mechanisms, and accurate service life prediction.

network model, degradation, photovoltaic, solar↗
DOE OSTI2023

MOSAiC Radar b1 Processing: Corrections, Calibrations, and Processing Report

The U.S. Department of Energy’s (DOE) Atmospheric Radiation Measurement (ARM) user facility deployed many instruments on board a German ice breaker, the Research Vessel (RV) Polarstern, for one year from October 2019 to October 2020. The purpose of the Multidisciplinary Drifting Observatory for the Study of Arctic Climate (MOSAiC) field campaign was to study the decline in the sea-ice pack around the North Pole, and what factors may be at play. After the campaign ended, efforts were undertaken to provide a calibrated radar data set for future studies. MOSAiC presented new challenges to this process, as existing methodologies often were not applicable for the frozen environment with little to no ground clutter, and concurrent engineering updates or calibrations could not be accomplished once the ship set off. Like the previous Cloud, Aerosol, and Complex Terrain Interactions (CACTI) and Cold-air Outbreaks in the Marine Boundary Layer Experiment (COMBLE) documentation (Hardin et al. 2020, Matthews et al. 2023), the data correction and calibration process is known in ARM as creating a “b1” datastream. This means that the radar datastreams have been well characterized to the best possible quality. This report will detail the status of the raw “a1” level data sets during the MOSAiC campaign, the corrections that were applied to create the “b1” data files, and the details of the applied methods.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Merged Aerosol Value-Added Product Report

The Merged Aerosol Value-Added Product (VAP) simplifies scientists’ use of Atmospheric Radiation Measurement (ARM) User Facility aerosol data by performing several tedious, time-consuming tasks for the users. First, the VAP identifies the best data available when multiple datastreams exist for a single geophysical quantity so that ARM users do not have to research this for themselves. Second, the VAP consolidates multiple ARM aerosol datastreams into a single file for ARM data users so that they do not have to download, open, and read multiple files for their analysis. Next, the VAP transforms all measurements onto a common one-hour timestamp. The one-hour resolution matches the time resolution of the slowest instrument. Instruments with faster sampling rates than one measurement per hour are averaged over the time interval. Finally, the VAP reads the QA/QC variables and marks data with known issues as missing, so that users do not have to spend excessive time cleaning data. This includes incorporating Data Quality Reports (DQRs) that exist at the time when the VAP data is generated. DQRs are reports filed by instrument mentors or data users that indicate a problem with the output data of individual instruments.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Performance and Reliability Assessment of the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) Data Advisor (ADA)

The Atmospheric Radiation Measurement (ARM) User Facility provides one of the world's largest openly accessible repositories of atmospheric observations through the ARM Data Discovery platform. Although the repository contains more than three decades of measurements collected from permanent observatories, mobile facilities, aircraft campaigns, and field experiments, identifying appropriate datasets can be challenging, particularly for new users unfamiliar with ARM instrumentation and datastream organization. To improve data accessibility, the ARM Data Center developed the ARM Data Advisor (ADA), an artificial intelligence-powered assistant designed to facilitate scientific data discovery, dataset interpretation, and user guidance. This report evaluates ADA's performance as a domain-specific scientific assistant using realistic atmospheric science workflows. The evaluation examines five key capabilities: data retrieval and curation efficiency, hallucination resistance, scientific reasoning, response to ambiguous queries, and content retention and session continuity. Representative prompts were developed to simulate typical interactions between researchers and the ARM Data Discovery platform, and ADA's responses were assessed for retrieval completeness, scientific accuracy, consistency, and practical usefulness. In these representative tests, ADA reduced the complexity of discovering and accessing ARM datasets by recommending appropriate datastreams, explaining instrumentation, interpreting metadata, and assisting with data processing workflows. ADA also exhibits strong domain knowledge of atmospheric science terminology and generally resists hallucination by acknowledging unavailable datasets and requesting clarification when appropriate. Overall, the results indicate that ADA represents a promising advancement in scientific data discovery within the ARM User Facility and has considerable potential to improve researcher productivity, particularly for new users and interdisciplinary scientists seeking efficient access to ARM observations.

Salvador, Christian [ORNL] (ORCID:0000000283287777↗
DOE OSTI2017

pgscoeff.b1

pgscoeff.b1 datastream necessary for callibration corrections to pgs.b1 datastream

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

kasacrgridppi.c1

This datastream is made by kasacrgridppi VAP. It has (time, Y, X) gridded Ka-band PPI radar data, using nearest neighbor algorithm, with calibrated (.b1) input. This datastream is very similar to kasacrgridrhi, with the output grid changed from (time, height, h_distance) to (time, Y, X). output c1 level data use calibrated b1 data as input.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1990

Highly parallel computation

Highly parallel computing architectures are the only means to achieve the computation rates demanded by advanced scientific problems. A decade of research has demonstrated the feasibility of such machines and current research focuses on which architectures designated as multiple instruction multiple datastream (MIMD) and single instruction multiple datastream (SIMD) have produced the best results to date; neither shows a decisive advantage for most near-homogeneous scientific problems. For scientific problems with many dissimilar parts, more speculative architectures such as neural networks or data flow may be needed.

Denning, Peter J.↗
NASA NTRS1995

Highly-Parallel, Highly-Compact Computing Structures Implemented in Nanotechnology

In this paper, we describe work in which we are evaluating how the evolving properties of nano-electronic devices could best be utilized in highly parallel computing structures. Because of their combination of high performance, low power, and extreme compactness, such structures would have obvious applications in spaceborne environments, both for general mission control and for on-board data analysis. However, the anticipated properties of nano-devices mean that the optimum architecture for such systems is by no means certain. Candidates include single instruction multiple datastream (SIMD) arrays, neural networks, and multiple instruction multiple datastream (MIMD) assemblies.

Crawley, D. G.↗
NASA NTRS2001

Advances in Global Water Cycle Science Made Possible by Global Precipitation Mission (GPM)

Within this decade the internationally sponsored Global Precipitation Mission (GPM) will take an important step in creating a global precipitation observing system from space. One perspective for understanding the nature of GPM is that it will be a hierarchical system of datastreams from very high caliber combined dual frequency radar/passive microwave (PMW) rain-radiometer retrievals, to high caliber PMW rain-radiometer only retrievals, and on to blends of the former datastreams with other less-high caliber PMW-based and IR-based rain retrievals. Within the context of NASA's role in global water cycle science and its own Global Water & Energy Cycle (GWEC) program, GPM is the centerpiece mission for improving our understanding of the global water cycle from a space-based measurement perspective. One of the salient problems within our current understanding of the global water and energy cycle is determining whether a change in the rate of the water cycle is accompanying changes in global temperature. As there are a number of ways in which to define a rate-change of the global water cycle, it is not entirely clear as to what constitutes such a determination, This paper presents an overview of the Global Precipitation Mission and how its datasets can be used in a set of quantitative tests within the framework of the oceanic and continental water budget equations to determine comprehensively whether substantive rate changes do accompany perturbations in global temperatures and how such rate changes manifest themselves in both water storage and water flux transport processes.

Smith, Eric A.↗
NASA NTRS2002

Advances in Understanding Global Water Cycle with Advent of Global Precipitation Measurement (GPM) Mission

Within this decade the internationally organized Global Precipitation Measurement (GPM) Mission will take an important step in creating a global precipitation observing system from space. One perspective for understanding the nature of GPM is that it will be a hierarchical system of datastreams beginning with very high caliber combined dual frequency radar/passive microwave (PMW) rain-radiometer retrievals, to high caliber PMW rain-radiometer only retrievals, and then on to blends of the former datastreams with additional lower-caliber PMW-based and IR-based rain retrievals. Within the context of the now emerging global water & energy cycle (GWEC) programs of a number of research agencies throughout the world, GPM serves as a centerpiece space mission for improving our understanding of the global water cycle from a global measurement perspective. One of the salient problems within our current understanding of the global water and energy cycle is determining whether a change in the rate of the water cycle is accompanying changes in climate, e.g., climate warming. As there are a number of ways in which to define a rate-change of the global water cycle, it is not entirely clear as to what constitutes such a determination. This paper presents an overview of the GPM Mission and how its observations can be used within the framework of the oceanic and continental water budget equations to determine whether a given perturbation in precipitation is indicative of an actual rate change in the global water cycle, consistent with required responses in water storage and/or water flux transport processes, or whether it is the natural variability of a fixed rate cycle.

Smith, Eric A.↗
NASA NTRS2002

Precipitation Measurement from Space: The Next Generation

During the coming decade, the internationally organized Global Precipitation Measurement (GPM) Mission will take an important step in creating a global rain observing system from space based on an international fleet of satellites operated as a constellation of opportunity. One perspective for understanding the nature of the mission is that it will be a hierarchical system of datastreams beginning with very high caliber combined dual frequency rain-radar/passive microwave rain-radiometer retrievals, to high caliber rain-radiometer only retrievals, and then on to blends of the former datastreams with additional lower-caliber microwave- and infrared-based rain retrievals. Within the context of the now emerging global water & energy cycle (GWEC) research programs of a number of research agencies throughout the world, the GPM mission serves as a centerpiece space mission for improving our understanding of the Earth's water cycle from global scale and on down to regional scales and below. One of the salient problems within our current understanding of the global water and energy cycle is determining whether a change in the rate of the water cycle is accompanying changes in climate, e.g., global water cycle acceleration accompanying global warming. As there are a number of ways in which to define a rate-change of the water cycle, it has not always been clear as to what constitutes a conclusive determination. This seminar presents an overview of the GPM Mission and how its overriding scientific objectives for climate, weather, and hydrology flow from the anticipated improvements that are being planned for the constellation-based measuring system, and how this mission may offer, over the long run, a more fundamental means to ascertain water cycle accelerations.

Smith, Eric A.↗
NASA NTRS2002

Relationship Between GPM Mission and NASA's Global Water Cycle Research Program

During the coming decade, the internationally organized Global Precipitation Measurement (GPM) Mission will take an important step in creating a global precipitation observing system from space based on an international fleet of satellites operated as a constellation of opportunity. One perspective for understanding the nature of GPM is that it will be a hierarchical system of datastreams beginning with very high caliber combined dual frequency radar/passive microwave (PMW) rain-radiometer retrievals, to high caliber PMW rain- radiometer only retrievals, and then on to blends of the former datastreams with additional lower-caliber PMW-based and IR-based rain retrievals. Within the context of the now emerging global water & energy cycle (GWEC) programs of a number of research agencies throughout the world, GPM serves as a centerpiece space mission for improving our understanding of the Earth's water cycle from a global measurement perspective and on down to regional scales and below. One of the salient problems within our current understanding of the global water and energy cycle is determining whether a change in the rate of the water cycle is accompanying changes in climate, e.g., climate warming. As there are a number of ways in which to define a rate- change of the global water cycle, it is not entirely clear as to what constitutes such a determination. This paper first presents an overview of the GPM Mission and how its overriding scientific objectives for climate, weather, and hydrology flow from the anticipated improvements that are being planned for the constellation-based measuring system. Next, the paper shows how the GPM observations can be used within the framework of the oceanic and continental water budget equations to determine whether a given perturbation in precipitation is indicative of an actual rate change in the water cycle, consistent with required responses in water storage and/or water flux transport processes, or whether it is simply part of the natural variability of a fixed rate cycle.

Smith, Eric A.↗
NASA NTRS2002

Advances In Understanding Global Water Cycle With Advent of GPM Mission

During the coming decade, the internationally organized Global Precipitation Measurement (GPM) Mission will take an important step in creating a global precipitation observing system from space based on an international fleet of satellites operated as a constellation. One perspective for understanding the nature of GPM is that it will be a hierarchical system of datastreams beginning with very high caliber combined dual frequency radar/passive microwave (PMW) rain-radiometer retrievals, to high caliber PMW rain-radiometer only retrievals, and then on to blends of the former datastreams with additional lower-caliber PMW-based and IR-based rain retrievals. Within the context of the now emerging global water & energy cycle (GWEC) programs of a number of research agencies throughout the world, GPM serves as a centerpiece space mission for improving our understanding of the Earth's water cycle from a global measurement perspective and on down to regional scales and below. One of the salient problems within our current understanding of the global water and energy cycle is determining whether a change in the rate of the water cycle is accompanying changes in climate, e.g., climate warming. As there are a number of ways in which to define a rate-change of the global water cycle, it is not entirely clear as to what constitutes such a determination. This paper first presents an overview of the GPM Mission and how its overriding scientific objectives for climate, weather, and hydrology flow from the anticipated improvements that are being planned for the constellation-based measuring system. Next, the paper shows how the GPM observations can be used within the framework of the oceanic and continental water budget equations to determine whether a given perturbation in precipitation is indicative of an actual rate change in the water cycle, consistent with required responses in water storage and/or water flux transport processes, or whether it is simply part of the natural variability of a fixed rate cycle.

Smith, Eric A.↗
DOE OSTI2017

Atmospheric Radiation Measurement Climate Research Facility Operations Quarterly Report October 1-December 31, 2016

Individual datastreams from instrumentation at the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Climate Research Facility fixed and mobile research observatories (sites) are collected and routed to the ARM Data Center (ADC). The Data Management Facility (DMF), a component of the ADC, executes datastream processing in near-real time. Processed data are then delivered approximately daily to the ARM Data Archive, also a component of the ADC, where they are made freely available to the research community. For each instrument, ARM calculates the ratio of the actual number of processed data records received daily at the ARM Data Archive to the expected number of data records. DOE requires national user facilities to report time-based operating data.

54 ENVIRONMENTAL SCIENCES↗