Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Distribution Service”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Mesh-free data transfer algorithms for partitioned multiphysics problems: Conservation, accuracy, and parallelism

In this study we analyze and extend mesh-free algorithms for three-dimensional data transfer problems in partitioned multiphysics simulations. We first provide a direct comparison between a mesh-based weighted residual method using the common-refinement scheme and two mesh-free algorithms leveraging compactly supported radial basis functions: one using a spline interpolation and one using a moving least square reconstruction. Through the comparison we assess both the conservation and accuracy of the data transfer obtained from each of the methods. We do so for a varying set of geometries with and without curvature and sharp features and for functions with and without smoothness and with varying gradients. Our results show that the mesh-based and mesh-free algorithms are complementary with cases where each was demonstrated to perform better than the other. We then focus on the mesh-free methods by developing a set of algorithms to parallelize them based on sparse linear algebra techniques. This includes a discussion of fast parallel radius searching in point clouds and restructuring the interpolation algorithms to leverage data structures and linear algebra services designed for large distributed computing environments. The scalability of our new algorithms is demonstrated on a leadership class computing facility using a set of basic scaling studies. Finally, these scaling studies show that for problems with reasonable load balance, our new algorithms for both spline interpolation and moving least square reconstruction demonstrate both strong and weak scalability using more than 100,000 MPI processes with billions of degrees of freedom in the data transfer operation.

97 MATHEMATICS AND COMPUTING↗

Information systems and services, user services

The following topics were discussed: (1) data availability and distribution, (2) complete processing systems, (3) subsystems, (4) applications, (5) research for future technology, and (6) education, training opportunities, and materials. Evidence was given that remote sensing technology is being increasingly utilized. Therefore, it was concluded that a second stage of remote sensing technology should be developed.

Landgrebe, D. A.↗

QuakeSim and the Solid Earth Research Virtual Observatory

We are developing simulation and analysis tools in order to develop a solid Earth science framework for understanding and studying active tectonic and earthquake processes. The goal of QuakeSim and its extension, the Solid Earth Research Virtual Observatory (SERVO), is to study the physics of earthquakes using state-of-the-art modeling, data manipulation, and pattern recognition technologies. We are developing clearly defined accessible data formats and code protocols as inputs to simulations, which are adapted to high-performance computers. The solid Earth system is extremely complex and nonlinear resulting in computationally intensive problems with millions of unknowns. With these tools it will be possible to construct the more complex models and simulations necessary to develop hazard assessment systems critical for reducing future losses from major earthquakes. We are using Web (Grid) service technology to demonstrate the assimilation of multiple distributed data sources (a typical data grid problem) into a major parallel high-performance computing earthquake forecasting code. Such a linkage of Geoinformatics with Geocomplexity demonstrates the value of the Solid Earth Research Virtual Observatory (SERVO) Grid concept, and advances Grid technology by building the first real-time large-scale data assimilation grid.

virtual observatory↗

Satellite provided customer premise services: A forecast of potential domestic demand through the year 2000. Volume 3: Appendices

Voice applications, data applications, video applications, impacted baseline forecasts, market distribution, potential CPS (customers premises services) user classes, net long haul forecasts, CPS cost analysis, overall satellite forecast, CPS satellite market, Ka-band CPS satellite forecast, nationwide traffic distribution model, and intra-urban topology are discussed.

Kratochvil, D.↗

PV O&M Cost Model [SWR-21-14]

This tool presents a method for calculating costs associated with the operation and maintenance (O&M) of photovoltaic (PV) systems. The tool compiles details regarding the cost and frequency of multiple O&M services to estimate annual O&M costs ($/year) for each year of an analysis period, the net present value ($) of life cycle costs accumulated over the analysis period, and the reserve account amount ($) that might be required to fund unexpected repairs. The reserve account includes parts inventory and is important for providing a source of funds to make repairs quickly and avoid lost production. This method allows a detailed selection of services to perform based on system size, market served (e.g., residential, commercial, or utility), type and configuration of system components (e.g., micro-, string, or central inverter), and site and environmental conditions (e.g., pollen, bird populations) which is an improvement over simple per unit valuations of O&M costs ($/kW/year). This model also distinguishes costs that vary from year to year and increase at different rates over time as modeled by heuristic failure distributions (e.g., Weibull or Lognormal distribution) based on actuarial data for many of the services.

Walker, Andy↗

Obtaining and Managing Answer Quality for Online Data-Intensive Services

Online data-intensive (OLDI) services use anytime algorithms to compute over large amounts of data and respond quickly. Interactive response times are a priority, so OLDI services parallelize query execution across distributed software components and return best effort answers based on the data so far processed. Omitted data from slow components could lead to better answers, but tracing online how much better the answers could be is difficult. Here, we introduce Ubora, a design method to measure the effect of slow-running components on the quality of answers. Ubora randomly samples online queries and executes them a second time. The first online execution omits data from slow components and provides interactive answers. The second execution uses mature results from intermediate components completed after the online execution finishes. Ubora uses memoization to speed up mature executions by replaying network messages exchanged between components. Our systems-level implementation works for a wide range of services, including Hadoop/Yarn, Apache Lucene, the EasyRec Recommendation Engine, and the OpenEphyra question-answering system. Ubora computes answer quality with more mature executions per second than competing approaches that do not use memoization. With Ubora, we illustrate that answer quality is effective at guiding online admission control. While achieving the same answer quality on high-priority queries, our adaptive controller had 55% higher peak throughput on low-priority queries than a competing controller guided by the rate of timeouts.

answer quality↗

What (and How) MERRA-2 Reanalysis Data are Used in Applied Sciences

The Modern Era Retrospective-analysis for Research and Applications, Version 2 (MERRA-2) is the global atmospheric data reanalysis for the satellite era produced by NASA’s Global Modeling and Assimilation Office (GMAO), using the Goddard Earth Observing System Model (GEOS)version 5.12.4. The data are officially distributed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). MERRA-2 data have been widely used by the Earth sciences and application community. Since MERRA-2 data were released in early 2016, the number of registered data users has grown steadily from 1,252 in 2016 to 6477 in 2020. By the end of October 2021, ~16 petabytes (over 360 million files) of data have been distributed to more than 18,900 users. Searching in Google Scholar (https://scholar.google.com/), we have found over 7,000 articles, published between January 2017 and May 2021, involving the use ofMERRA-2 data. The figure shows the numbers for various application areas in which theMERRA-2 data have been used, covering almost all of the application areas defined in NASA Applied Sciences (http://appliedsciences.nasa.gov). The largest number of articles are found in disaster research, with the subcategories ordered in flood, wildfires, hurricanes and cyclones, and other forms of severe weather. In this presentation, we will discuss the preliminary findings from a review of the selected literature that uses MERRA-2 data in applied sciences. The current analytic and interoperable data services at GES DISC are listed, such as the on-the-fly subset and analysis service, NASA Giovanni; THREDDS Data Server(TDS); and Python Jupyter notebooks. In addition, we will introduce two new services for supporting the open sciences: My Dashboard and Related Publications.

data management↗

Social Networking Adapted for Distributed Scientific Collaboration

Share is a social networking site with novel, specially designed feature sets to enable simultaneous remote collaboration and sharing of large data sets among scientists. The site will include not only the standard features found on popular consumer-oriented social networking sites such as Facebook and Myspace, but also a number of powerful tools to extend its functionality to a science collaboration site. A Virtual Observatory is a promising technology for making data accessible from various missions and instruments through a Web browser. Sci-Share augments services provided by Virtual Observatories by enabling distributed collaboration and sharing of downloaded and/or processed data among scientists. This will, in turn, increase science returns from NASA missions. Sci-Share also enables better utilization of NASA s high-performance computing resources by providing an easy and central mechanism to access and share large files on users space or those saved on mass storage. The most common means of remote scientific collaboration today remains the trio of e-mail for electronic communication, FTP for file sharing, and personalized Web sites for dissemination of papers and research results. Each of these tools has well-known limitations. Sci-Share transforms the social networking paradigm into a scientific collaboration environment by offering powerful tools for cooperative discourse and digital content sharing. Sci-Share differentiates itself by serving as an online repository for users digital content with the following unique features: a) Sharing of any file type, any size, from anywhere; b) Creation of projects and groups for controlled sharing; c) Module for sharing files on HPC (High Performance Computing) sites; d) Universal accessibility of staged files as embedded links on other sites (e.g. Facebook) and tools (e.g. e-mail); e) Drag-and-drop transfer of large files, replacing awkward e-mail attachments (and file size limitations); f) Enterprise-level data and messaging encryption; and g) Easy-to-use intuitive workflow.

Karimabadi, Homa↗

Advancing User Supports with a Structured How-To Knowledge Base for Earth Science Data

It is a challenge to access and process fast growing Earth science data from satellites and numerical models, which may be archived in very different data format and structures. NASA data centers, managed by the Earth Observing System Data and Information System (EOSDIS), have developed a rich and diverse set of data services and tools with features intended to simplify finding, downloading, and working with these data. Although most data services and tools have user guides, many users still experience difficulties with accessing or reading data due to varying levels of familiarity with data services, tools, and/or formats. A type of structured online document, data recipe, were created in beginning 2013 by Goddard Earth Science Data and Information Services Center (GES DISC). A data recipe is the How-To document created by using the fixed template, containing step-by-step instructions with screenshots and examples of accessing and working with real data. The recipes has been found to be very helpful, especially to first-time-users of particular data services, tools, or data products. Online traffic to the data recipe pages is significant to some recipes. In 2014, the NASA Earth Science Data System Working Group (ESDSWG) for data recipes was established, aimed to initiate an EOSDIS-wide campaign for leveraging the distributed knowledge within EOSDIS and its user communities regarding their respective services and tools. The ESDSWG data recipe group started with inventory and analysis of existing EOSDIS-wide online help documents, and provided recommendations and guidelines and for writing and grouping data recipes. This presentation will overview activities of creating How-To documents at GES DISC and ESDSWG. We encourage feedback and contribution from users for improving the data How-To knowledge base.

how-to↗

North American Land Data Assimilation (NLDAS) Data and Services at NASA GES DISC

The North American Land Data Assimilation System (NLDAS, https://ldas.gsfc.nasa.gov/nldas) integrates a large quantity of observation-based and model reanalysis data to drive offline (not coupled to the atmosphere) land surface models (LSMs) to produce fields of soil moisture, snow, and surface fluxes. The NLDAS-2 data sets currently archived at, and distributed by, the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC, https://disc.gsfc.nasa.gov) have been updated to NLDAS Version 2.0 (NLDAS-2.0). This presentation provides an overview of the updated NLDAS-2.0 data at NASA GES DISC and discusses the methods for the public to access the data as well as the data services available.

Ashley Heath↗

GLDAS-2 Land Surface Model Data and Data Services at NASA GES DISC

The goal of the NASA Global Land Data Assimilation System (GLDAS, https://ldas.gsfc.nasa.gov/gldas(https://ldas.gsfc.nasa.gov/gldas)) is to generate optimal fields of land surface states and fluxes by ingesting satellite- and ground-based observational data products, using advanced land surface modeling and data assimilation techniques (Rodell et al., 2004).The GLDAS dataset currently archived at and distributed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC, https://disc.gsfc.nasa.gov/ (https://disc.gsfc.nasa.gov/)) is GLDAS Version 2 (GLDAS-2). It contains a series of output fields from the upgraded Noah-3.6, Catchment-F2.5, and VIC-4.1.2 Land Surface Models (LSMs) in the Land Information System (LIS-V7, https://lis.gsfc.nasa.gov/ (https://lis.gsfc.nasa.gov/)). GLDAS-2 has three components:GLDAS-2.0, GLDAS-2.1, and GLDAS-2.2. GLDAS-2.0 is forced entirely with the upgraded Princeton Meteorological ForcingV2.2 Dataset and provides a temporally consistent series from 1948 through 2014. GLDAS-2.1 is forced with a combination of model and observation data, with data spanning from 2000 to the present. The GLDAS-2.2 product suite uses data assimilation(DA), whereas the GLDAS-2.0 and GLDAS-2.1 products are "open-loop" (i.e., no data assimilation). The choice of forcing data, as well as DA observation source, variable, and scheme, varies for different GLDAS-2.2 products. The currently availableGLDAS-2.2 data contain a daily 0.25-degree output from the Catchment-F2.5 LSM in LIS-V7. The data are forced with the meteorological analysis fields from the operational European Centre for Medium-Range Weather Forecasts Integrated Forecasting System (ECMWF-IFS) and assimilated with GRACE and GRACE-FO data, ranging from February 1, 2003 to the present. The current GLDAS-2.0 and 2.1 Noah LSM data were reprocessed in November 2019 and January 2020 respectively and their data from Catchment and VIC LSMs are new to the GLDAS-2 collection. This presentation provides a summary of theGLDAS-2 data products, their land surface fields, and their related data services at the GES DISC; and a description of the majorGLDAS-2 climatological characteristics as well as the intercomparison with the data of the previous version.

Hydrology↗

Science data processing in the Mission to Planet Earth era

The data-processing systems developed to support NASA's Mission to Planet Earth (MPE) are described in an overview of the system architectures. The MPE program is discussed in terms of flight segments and data types to examine the required elements of the information systems. The data-processing segments are described for the Atmospheric Laboratory for Applications and Science, the Tropical Rainfall Measuring Mission, the Earth Observing System, and the Geostationary Earth Observatory. The data and information systems for the separate segments provide some redundant processing services. Distributed active archive centers are employed for each segment composed of three subelements: (1) a product-generation system, (2) a data archive and distribution system, and (3) an information management system. The data processing design for the MPE is expected to facilitate the understanding of the entire earth system on a global scale.

Goodman, H. M.↗

Use of Schema on Read in Earth Science Data Archives

Traditionally, NASA Earth Science data archives have file-based storage using proprietary data file formats, such as HDF and HDF-EOS, which are optimized to support fast and efficient storage of spaceborne and model data as they are generated. The use of file-based storage essentially imposes an indexing strategy based on data dimensions. In most cases, NASA Earth Science data uses time as the primary index, leading to poor performance in accessing data in spatial dimensions. For example, producing a time series for a single spatial grid cell involves accessing a large number of data files. With exponential growth in data volume due to the ever-increasing spatial and temporal resolution of the data, using file-based archives poses significant performance and cost barriers to data discovery and access. Storing and disseminating data in proprietary data formats imposes an additional access barrier for users outside the mainstream research community. At the NASA Goddard Earth Sciences Data Information Services Center (GES DISC), we have evaluated applying the schema-on-read principle to data access and distribution. We used Apache Parquet to store geospatial data, and have exposed data through Amazon Web Services (AWS) Athena, AWS Simple Storage Service (S3), and Apache Spark. Using the schema-on-read approach allows customization of indexing spatially or temporally to suit the data access pattern. The storage of data in open formats such as Apache Parquet has widespread support in popular programming languages. A wide range of solutions for handling big data lowers the access barrier for all users. This presentation will discuss formats used for data storage, frameworks with This presentation will discuss formats used for data storage, frameworks with support for schema-on-read used for data access, and common use cases covering data usage patterns seen in a geospatial data archive.

cloud applications↗

Comparison between the Strength Levels of Baseline Nuclear-Grade Graphite and Graphite Irradiated in AGC-2

This report details the initial comparison of mechanical strength properties between the cylindrical nuclear-grade graphite specimens irradiated in the second Advanced Graphite Creep (AGC-2) experiment with the established baseline, or unirradiated, mechanical properties compiled in the Baseline Graphite Characterization program. The overall comparative analysis will describe the development of an appropriate test protocol for irradiated specimens, the execution of the mechanical tests on the AGC-2 sample population, and will further discuss the data in terms of developing an accurate irradiated property distribution in the limited amount of irradiated data by leveraging the considerably larger property datasets being captured in the Baseline Graphite Characterization program. Integrating information on the inherent variability in nuclear-grade graphite with more complete datasets is one of the goals of the VHTR Graphite Materials program. Between “sister” specimens, or specimens with the same geometry machined from the same sub-block of graphite from which the irradiated AGC specimens were extracted, and the Baseline datasets, a comprehensive body of data will exist that can provide both a direct and indirect indication of the full irradiated property distributions that can be expected of irradiated nuclear-grade graphite while in service in a VHTR system. While the most critical data will remain the actual irradiated property measurements, expansion of this data into accurate distributions based on the inherent variability in graphite properties will be a crucial step in qualifying graphite for nuclear use as a structural material in a VHTR environment.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Modern Era Retrospective Restrospective-Analysis for Research and Applications (MERRA) Data and Services at the GES DISC

The Modern Era Retrospective-analysis for Research and Applications (MERRA) dataset is a NASA satellite era, 30 year (1979 - present), reanalysis using the Goddard Earth Observing System Data Assimilation System, Version 5 (GEOS-5). The project, run out of NASA's Global Modeling and Assimilation Office at Goddard Space Flight Center, provides the science and application communities with a state-of-the-art global analysis with emphasis on improved estimates of the hydrological cycle over a broad range of weather and climate time scales. MERRA products are generated as a long-term synthesis that places the NASA EOS suite of observations in a climate context. The MERRA analysis is performed at a horizontal resolution of 2/3 longitude x 1/2 latitude (540x361 global gridpoints) with observational analyses every 6 hours. The MERRA output data will include 3 dimensional state fields for every 6 hourly analysis cycle on 42 pressure levels (or 72 terrain following model coordinate levels) from the surface through the stratosphere. Several data products are specifically designed to support chemistry and stratosphere transport modeling. The 2 dimensional surface and atmospheric diagnostics (numbering 259) are being stored on the native grid at 1 hourly intervals. These include radiation and vertical integrals of the atmosphere for water and energy budget studies and also surface diagnostics where the diurnal cycle is important. The one hourly surface and near surface data product will also facilitate research on the integrated analysis of Earth system observations in the land, ocean and cryosphere. The MERRA products are archived and distributed by the Goddard Earth Sciences Data and Information Services Center (GES DISC) through its Modeling DISC Web (MDISC) portal. Multiple data access methods and services are available for MERRA data through MDISC: (1) Mirador offers a quick, comprehensive search of MERRA and all GES DISC archived data holdings, allowing searches on keywords, location names or latitude/longitude box, and date/time, with responses within a few seconds. (2) Giovanni is a GES DISC developed Web application that provides data visualization and analysis online. Giovanni features popular visualizations such as latitude-longitude maps, animations, cross sections, profiles, time series, etc. and some basic statistical analysis functions such as scatter plots and correlation coefficient maps. Users are able to download results in several different formats, including Google Earth. (3) On-the-fly parameter subsetting of data within a spatial/temporal window is provided through a simple select and click Web page. (4) MERRA data are also available via OPeNDAP, GrADS Data Server (GDS) and can be converted to netCDF on the fly.

Berrick, Stephen W.↗

Time-Based Data Streams: Fundamental Concepts for a Data Resource for Streams

Real time data, which we call data streams, are readings from instruments, environmental, bodily or building sensors that are generated at regular intervals and often, due to their volume, need to be processed in real time. Often a single pass is all that can be made on the data, and a decision to discard or keep the instance is made on the spot. Too, the stream is for all practical purposes indefinite, so decisions must be made on incomplete knowledge. This notion of data streams has a different set of issues from a file, for instance, that is byte streamed to a reader. The file is finite, so the byte stream is becomes a processing convenience more than a fundamentally different kind of data. Through the duration of the project we examined three aspects of streaming data: the first, techniques to handle streaming data in a distributed system organized as a collection of web services, the second, the notion of the dashboard and real time controllable analysis constructs in the context of the Fermi Tevatron Beam Position Monitor, and third and finally, we examined provenance collection of stream processing such as might occur as raw observational data flows from the source and undergoes correction, cleaning, and quality control. The impact of this work is severalfold. We were one of the first to advocate that streams had little value unless aggregated, and that notion is now gaining general acceptance. We were one of the first groups to grapple with the notion of provenance of stream data also.

97 MATHEMATICS AND COMPUTING↗

Software Framework for Peer Data-Management Services

Object Oriented Data Technology (OODT) is a software framework for creating a Web-based system for exchange of scientific data that are stored in diverse formats on computers at different sites under the management of scientific peers. OODT software consists of a set of cooperating, distributed peer components that provide distributed peer-to-peer (P2P) services that enable one peer to search and retrieve data managed by another peer. In effect, computers running OODT software at different locations become parts of an integrated data-management system.

Hughes, John↗

Catastrophic optical degradation of the output facet of high-power single-transverse-mode diode lasers. 2. Calculation of the spatial temperature distribution and threshold of the catastrophic optical degradation

The temperature distribution and the power threshold during the catastrophic optical degradation are calculated within the framework of the developed model of the COD of the output facet in high-power single-transverse-mode diode lasers. Comparison of the calculation results and the experiment show the model adequacy. The contribution of different physical mechanisms into the heating of the laser output facet is analysed. It is shown that the model under study can help to develop the method for predicting the laser lifetime by the accelerated ageing tests. (lasers)

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗