Search NASA⌕ Search

SEARCH · Search NASA

Results for “data access”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

Single-Point Access to Data Distributed on Many Processors

A description of the functions and data structures is defined that would be necessary to implement the Chapel concept of distributions, domains, allocation, access, and interfaces to the compiler for transformations from Chapel source to their run-time implementation for these concepts. A complete set of object-oriented operators is defined that enables one to access elements of a distributed array through regular arithmetic index sets, giving the programmer the illusion that all the elements are collocated on a single processor. This means that arbitrary regions of the arrays can be fragmented and distributed across multiple processors with a single point of access. This is important because it can significantly improve programmer productivity by allowing the programmers to concentrate on the high-level details of the algorithm without worrying about the efficiency and communication details of the underlying representation.

James, Mark↗

Long-Term Preservation of NASA Heliophysics Data and Access: Where We Were and Where We're Going

The importance of ensuring preservation and useful access to the unique science potential of past, present and future NASA solar and space physics (i.e. heliophysics) data has been recognized since the inception of NASA but remains challenging. In this talk, I will briefly review the history of this topic and and then discuss the present NASA model for heliophysics science data management, including key current resources for finding and using data projects like the Space Physics Data Facility. I will highlight expected future directions, building on working elements of the present program and exploiting new technology, to further improve the data environment, address existing issues and anticipate emerging challenges.

McGuire, Robert E.↗

A review of machine learning in building load prediction

The surge of machine learning in recent years has been empowering engineer modeling in various fields. The decreasing hardware cost, increasing data accessibility, and advances of building automation system (BAS) allow the collection and storage of a significant amount of building operation data. The two facts provide great opportunities of applying machine learning to building energy systems modeling and analysis. There are a great number of research papers on this topic but there lacks a comprehensive and general review to summarize the current development, limitations, gaps and future trend. In this review paper series, machine learning techniques in building energy system modeling and analysis are reviewed under the organization and logic of the machine learning definition by Tom M. Mitchell: a computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E. This paper is the first part of the review paper series, which focuses on building load prediction. First, the applications of building load prediction model (task T) are reviewed. Then, the modeling algorithms improving machine learning performance and accuracy (performance P) are reviewed. At the same time, the literature on the data perspective for modeling (experience E), including data engineering from sensors level to data level, pre-processing, feature extraction and selection, is reviewed. Finally, what is well-studied and what is lacking but with great potential are concluded; the gaps between present and future utilization of machine learning techniques are identified; the future trend and development are also predicted. The target readers of this paper are not only researchers from the building side who can get exposed to cutting edge machine learning tools, but also those from machine learning side who can understand the potential and challenge to apply machine learning in buildings.

Liang, Zhang↗

Efficient accesses of data structures using processing near memory

Systems, apparatuses, and methods for implementing efficient queues and other data structures. A queue may be shared among multiple processors and/or threads without using explicit software atomic instructions to coordinate access to the queue. System software may allocate an atomic queue and corresponding queue metadata in system memory and return, to the requesting thread, a handle referencing the queue metadata. Any number of threads may utilize the handle for accessing the atomic queue. The logic for ensuring the atomicity of accesses to the atomic queue may reside in a management unit in the memory controller coupled to the memory where the atomic queue is allocated.

97 MATHEMATICS AND COMPUTING↗

The Heliophysics Data Environment, Virtual Observatories, NSSDC, and SPASE

Heliophysics (the study of the Sun and its effects on the Solar System, especially the Earth) has an interesting data environment in that the data are often to be found in relatively small data sets widely scattered in archives around the world. Within the last decade there have been more concentrated efforts to organize the data access methods and create a Heliophysics Data and Model Consortium (HDMC). To provide data search and access capability a number of Virtual Observatories (VO's) have been established both via funding from the U.S. National Aeronautics and Space Administration (NASA) and through other funding agencies in the U.S. and worldwide. At least 15 systems can be labeled as Heliophysics Virtual Observatories, 9 of them funded by NASA. Other parts of this data environment include Resident Archives, and the final, or "deep" archive at the National Space Science Data Center (NSSDC). The problem is that different data search and access approaches are used by all of these elements of the HDMC and a search for data relevant to a particular research question can involve consulting with multiple VO's - needing to learn a different approach for finding and acquiring data for each. The Space Physics Archive Search and Extract (SPASE) project is intended to provide a common data model for Heliophysics data and therefore a common set of metadata for searches of the VO's and other data environment elements. The SPASE Data Model has been developed through the common efforts of the HDMC representatives over a number of years. We currently have released Version 2.1. of the Data Model. The advantages and disadvantages of the Data Model will be discussed along with the plans for the future. Recent changes requested by new members of the SPASE community indicate some of the directions for further development.

Thieman, James↗

Cheetah: A Framework for Scalable Hierarchical Collective Operations

Collective communication operations, used by many scientific applications, tend to limit overall parallel application performance and scalability. Computer systems are becoming more heterogeneous with increasing node and core-per-node counts. Also, a growing number of data-access mechanisms, of varying characteristics, are supported within a single computer system. We describe a new hierarchical collective communication framework that takes advantage of hardware-specific data-access mechanisms. It is flexible, with run-time hierarchy specification, and sharing of collective communication primitives between collective algorithms. Data buffers are shared between levels in the hierarchy reducing collective communication management overhead. We have implemented several versions of the Message Passing Interface (MPI) collective operations, MPI Barrier() and MPI Bcast(), and run experiments using up to 49, 152 processes on a Cray XT5, and a small InfiniBand based cluster. At 49, 152 processes our barrier implementation outperforms the optimized native implementation by 75%. 32 Byte and one Mega-Byte broadcasts outperform it by 62% and 11%, respectively, with better scalability characteristics. Improvements relative to the default Open MPI implementation are much larger.

99 GENERAL AND MISCELLANEOUS↗

Virtual Observatories for Space Physics Observations and Simulations: New Routes to Efficient Access and Visualization

New tools for data access and visualization promise to make the analysis of space plasma data both more efficient and more powerful, especially for answering questions about the global structure and dynamics of the Sun-Earth system. We will show how new existing tools (particularly the Virtual Space Physics Observatory-VSPO-and the Visual System for Browsing, Analysis and Retrieval of Data-ViSBARD; look for the acronyms in Google) already provide rapid access to such information as spacecraft orbits, browse plots, and detailed data, as well as visualizations that can quickly unite our view of multispacecraft observations. We will show movies illustrating multispacecraft observations of the solar wind and magnetosphere during a magnetic storm, and of simulations of 3 0-spacecraft observations derived from MHD simulations of the magnetosphere sampled along likely trajectories of the spacecraft for the MagCon mission. An important issue remaining to be solved is how best to integrate simulation data and services into the Virtual Observatory environment, and this talk will hopefully stimulate further discussion along these lines.

Roberts, Aaron↗

Characterization of bandwidth requirements of algorithms for extreme scale science

Technology trends have resulted in widely different rates of improvement in computational throughput as compared to data movement rates in computer systems. With future systems, the cost of data movement through the memory hierarchy is expected to become even more dominant relative to the cost of performing arithmetic operations, both in terms of time and energy. Therefore, optimizing data access costs will become ever more critical in the coming years. However, unlike the well understood model of computational complexity in terms of the number of elementary operations to execute an algorithm, the inherent data-movement complexity of computations is unknown for most algorithms. Given the crucial importance of optimizing data access costs in scalable parallel systems, it is of great interest to develop techniques for understanding the inherent data movement complexity of an algorithm. In this project, we focus on this fundamental and significant problem.

97 MATHEMATICS AND COMPUTING↗

Improving the Effectiveness of Context-based Prefetching with Multi-order Analysis

Data prefetching is an effective way to accelerate data access in high-end computing systems and to bridge the increasing performance gap between processor and memory. In recent years, the context-based data prefetching has received intensive attention because of its general applicability. In this study, we provide a preliminary analysis of the impact of orders on the effectiveness of the context-based prefetching. Motivated by the observations from the analytical results, we propose a new context-based prefetching method named Multi-Order Context-based (MOC) prefetching to adopt multi-order context analysis to increase the context-based prefetching effectiveness. We have carried out simulation testing with the SPEC-CPU2006 benchmarks via an enhanced CMP$im simulator. The simulation results show that the proposed MOC prefetching method outperforms the existing single-order prefetching and reduces the data-access latency effectively.

99 GENERAL AND MISCELLANEOUS↗

Implementing Access to Data Distributed on Many Processors

A reference architecture is defined for an object-oriented implementation of domains, arrays, and distributions written in the programming language Chapel. This technology primarily addresses domains that contain arrays that have regular index sets with the low-level implementation details being beyond the scope of this discussion. What is defined is a complete set of object-oriented operators that allows one to perform data distributions for domain arrays involving regular arithmetic index sets. What is unique is that these operators allow for the arbitrary regions of the arrays to be fragmented and distributed across multiple processors with a single point of access giving the programmer the illusion that all the elements are collocated on a single processor. Today's massively parallel High Productivity Computing Systems (HPCS) are characterized by a modular structure, with a large number of processing and memory units connected by a high-speed network. Locality of access as well as load balancing are primary concerns in these systems that are typically used for high-performance scientific computation. Data distributions address these issues by providing a range of methods for spreading large data sets across the components of a system. Over the past two decades, many languages, systems, tools, and libraries have been developed for the support of distributions. Since the performance of data parallel applications is directly influenced by the distribution strategy, users often resort to low-level programming models that allow fine-tuning of the distribution aspects affecting performance, but, at the same time, are tedious and error-prone. This technology presents a reusable design of a data-distribution framework for data parallel high-performance applications. Distributions are a means to express locality in systems composed of large numbers of processor and memory components connected by a network. Since distributions have a great effect on the performance of applications, it is important that the distribution strategy is flexible, so its behavior can change depending on the needs of the application. At the same time, high productivity concerns require that the user be shielded from error-prone, tedious details such as communication and synchronization.

James, Mark↗

Spatially coordinated airborne data and complementary products for aerosol, gas, cloud, and meteorological studies: the NASA ACTIVATE dataset

The NASA Aerosol Cloud meTeorology Interactions oVer the western ATlantic Experiment (ACTIVATE) produced a unique dataset for research into aerosol–cloud–meteorology interactions, with applications extending from process-based studies to multi-scale model intercomparison and improvement as well as to remote-sensing algorithm assessments and advancements. ACTIVATE used two NASA Langley Research Center aircraft, a HU-25 Falcon and King Air, to conduct systematic and spatially coordinated flights over the northwest Atlantic Ocean, resulting in 162 joint flights and 17 other single-aircraft flights between 2020 and 2022 across all seasons. Data cover 574 and 592 cumulative flights hours for the HU-25 Falcon and King Air, respectively. The HU-25 Falcon conducted profiling at different level legs below, in, and just above boundary layer clouds (< 3 km) and obtained in situ measurements of trace gases, aerosol particles, clouds, and atmospheric state parameters. Under cloud-free conditions, the HU-25 Falcon similarly conducted profiling at different level legs within and immediately above the boundary layer. The King Air (the high-flying aircraft) flew at approximately ~ 9 km and conducted remote sensing with a lidar and polarimeter while also launching dropsondes (785 in total). Collectively, simultaneous data from both aircraft help to characterize the same vertical column of the atmosphere. In addition to individual instrument files, data from the HU-25 Falcon aircraft are combined into “merge files” on the publicly available data archive that are created at different time resolutions of interest (e.g., 1, 5, 10, 15, 30, 60 s, or matching an individual data product's start and stop times). This paper describes the ACTIVATE flight strategy, instrument and complementary dataset products, data access and usage details, and data application notes. The data are publicly accessible through https://doi.org/10.5067/SUBORBITAL/ACTIVATE/DATA001 (ACTIVATE Science Team, 2020).

54 ENVIRONMENTAL SCIENCES↗

Spatially-Coordinated Airborne Data and Complementary Products for Aerosol, Gas, Cloud, and Meteorological Studies: the Nasa Activate Dataset

The NASA Aerosol Cloud meTeorology Interactions oVer the western ATlantic Experiment (ACTIVATE) produced a unique dataset for research into aerosol–cloud–meteorology interactions, with applications extending from process-based studies to multi-scale model intercomparison and improvement as well as to remote-sensing algorithm assessments and advancements. ACTIVATE used two NASA Langley Research Center aircraft, a HU-25 Falcon and King Air, to conduct systematic and spatially coordinated flights over the northwest Atlantic Ocean, resulting in 162 joint flights and 17 other single-aircraft flights between 2020 and 2022 across all seasons. Data cover 574 and 592 cumulative flights hours for the HU-25 Falcon and King Air, respectively. The HU-25 Falcon conducted profiling at different level legs below, in, and just above boundary layer clouds (< 3 km) and obtained in situ measurements of trace gases, aerosol particles, clouds, and atmospheric state parameters. Under cloud-free conditions, the HU-25 Falcon similarly conducted profiling at different level legs within and immediately above the boundary layer. The King Air (the high-flying aircraft) flew at approximately ∼ 9 km and conducted remote sensing with a lidar and polarimeter while also launching dropsondes (785 in total). Collectively, simultaneous data from both aircraft help to characterize the same vertical column of the atmosphere. In addition to individual instrument files, data from the HU-25 Falcon aircraft are combined into “merge files” on the publicly available data archive that are created at different time resolutions of interest (e.g., 1, 5, 10, 15, 30, 60 s, or matching an individual data product's start and stop times). This paper describes the ACTIVATE flight strategy, instrument and complementary dataset products, data access and usage details, and data application notes. The data are publicly accessible through https://doi.org/10.5067/SUBORBITAL/ACTIVATE/DATA001 (ACTIVATE Science Team, 2020).

Aerosol Cloud meTeorology Interactions oVer the we↗

Simplifying NASA Earth Science Data and Information Access Through Natural Language Processing Based Data Analysis and Visualization

NASA Earth science data collected from satellites, model assimilation, airborne missions, and field campaigns, are large, complex and evolving. Such characteristics pose great challenges for end users (e.g., Earth science and applied science users, students, citizen scientists), particularly for those who are unfamiliar with NASA's EOSDIS and thus unable to access and utilize datasets effectively. For example, a novice user may simply ask: what is the total rainfall for a flooding event in my county yesterday? For an experienced user (e.g., algorithm developer), a question can be: how did my rainfall product perform, compared to ground observations, during a flooding event? Nonetheless, with rapid information technology development such as natural language processing, it is possible to develop simplified Web interfaces and back-end processing components to handle such questions and deliver answers in terms of text, data, or graphic results directly to users.In this presentation, we describe the main challenges for end users with different levels of expertise in accessing and utilizing NASA Earth science data. Surveys reveal that most non-professional users normally do not want to download and handle raw data as well as conduct heavy-duty data processing tasks. Often they just want some simple graphics or data for various purposes. To them, simple and intuitive user interfaces are sufficient because complicated ones can be difficult and time-consuming to learn. Professionals also want such interfaces to answer many questions from datasets. One solution is to develop a natural language based search box like Google and the search results can be text, data, graphics and more. Now the challenge is, with natural language processing, can we design a system to process a scientific question typed in by a user? In this presentation, we describe our plan for such a prototype. The workflow is: 1) extract needed information (e.g., variables, spatial and temporal information, processing methods, etc.) from the input, 2) process the data in the backend, and 3) deliver the results (data or graphics) to the user.

Liu, Zhong↗

WMS Server 2.0

This software is a simple, yet flexible server of raster map products, compliant with the Open Geospatial Consortium (OGC) Web Map Service (WMS) 1.1.1 protocol. The server is a full implementation of the OGC WMS 1.1.1 as a fastCGI client and using Geospatial Data Abstraction Library (GDAL) for data access. The server can operate in a proxy mode, where all or part of the WMS requests are done on a back server. The server has explicit support for a colocated tiled WMS, including rapid response of black (no-data) requests. It generates JPEG and PNG images, including 16-bit PNG. The GDAL back-end support allows great flexibility on the data access. The server is a port to a Linux/GDAL platform from the original IRIX/IL platform. It is simpler to configure and use, and depending on the storage format used, it has better performance than other available implementations. The WMS server 2.0 is a high-performance WMS implementation due to the fastCGI architecture. The use of GDAL data back end allows for great flexibility. The configuration is relatively simple, based on a single XML file. It provides scaling and cropping, as well as blending of multiple layers based on layer transparency.

Plesea, Lucian↗

Parallel performance optimizations on unstructured mesh-based simulations

This paper addresses two key parallelization challenges the unstructured mesh-based ocean modeling code, MPAS-Ocean, which uses a mesh based on Voronoi tessellations: (1) load imbalance across processes, and (2) unstructured data access patterns, that inhibit intra- and inter-node performance. Our work analyzes the load imbalance due to naive partitioning of the mesh, and develops methods to generate mesh partitioning with better load balance and reduced communication. Furthermore, we present methods that minimize both inter- and intranode data movement and maximize data reuse. Our techniques include predictive ordering of data elements for higher cache efficiency, as well as communication reduction approaches. We present detailed performance data when running on thousands of cores using the Cray XC30 supercomputer and show that our optimization strategies can exceed the original performance by over 2×. Additionally, many of these solutions can be broadly applied to a wide variety of unstructured grid-based computations.

97 MATHEMATICS AND COMPUTING↗

Characterizing Impacts of Storage Faults on HPC Applications: A methodology and insights

In recent years, the increasing complexity in scientific simulations and emerging demands for training heavy artificial intelligence models require massive and fast data accesses, which urges high-performance computing (HPC) platforms to equip with more advanced storage infrastructures such as solid-state disks (SSDs). While SSDs offer high-performance I/O, it remains unclear about the reliability challenges faced by the HPC applications under the SSD-related failures, in particular, failures resulting in data corruptions. The goal of this paper is to understand the impact of SSD-related data corruptions on the behaviors of complex HPC applications. To this end, we propose FFIS, a FUSE-based fault injection framework that systematically introduces storage faults into the application layer to model the errors originated from SSDs. FFIS is able to plant different I/O related faults into the data returned from underlying file systems, which also enables the investigation on the error resilience characteristics of the scientific file format for the first time. We demonstrate the use of FFIS with three representative real HPC applications, show how each application reacts to the data corruptions, and provide insights on the error resilience of the widely-adopted HDF5 file format for the HPC applications.

Fang, Bo↗

Hurricane Data Analysis Tool

In order to facilitate Earth science data access, the NASA Goddard Earth Sciences Data Information Services Center (GES DISC) has developed a web prototype, the Hurricane Data Analysis Tool (HDAT; URL: http://disc.gsfc.nasa.gov/HDAT), to allow users to conduct online visualization and analysis of several remote sensing and model datasets for educational activities and studies of tropical cyclones and other weather phenomena. With a web browser and few mouse clicks, users can have a full access to terabytes of data and generate 2-D or time-series plots and animation without downloading any software and data. HDAT includes data from the NASA Tropical Rainfall Measuring Mission (TRMM), the NASA Quick Scatterometer(QuikSCAT) and NECP Reanalysis, and the NCEP/CPC half-hourly, 4-km Global (60 N - 60 S) IR Dataset. The GES DISC archives TRMM data. The daily global rainfall product derived from the 3-hourly multi-satellite precipitation product (3B42 V6) is available in HDAT. The TRMM Microwave Imager (TMI) sea surface temperature from the Remote Sensing Systems is in HDAT as well. The NASA QuikSCAT ocean surface wind and the NCEP Reanalysis provide ocean surface and atmospheric conditions, respectively. The global merged IR product, also known as, the NCEP/CPC half-hourly, 4-km Global (60 N -60 S) IR Dataset, is one of TRMM ancillary datasets. They are globally-merged pixel-resolution IR brightness temperature data (equivalent blackbody temperatures), merged from all available geostationary satellites (GOES-8/10, METEOSAT-7/5 & GMS). The GES DISC has collected over 10 years of the data beginning from February of 2000. This high temporal resolution (every 30 minutes) dataset not only provides additional background information to TRMM and other satellite missions, but also allows observing a wide range of meteorological phenomena from space, such as, hurricanes, typhoons, tropical cyclones, mesoscale convection system, etc. Basic functions include selection of area of interest and time, single imagery, overlay of two different products, animation,a time skip capability and different image size outputs. Users can save an animation as a file (animated gif) and import it in other presentation software, such as, Microsoft PowerPoint. Since the tool can directly access the real data, more features and functionality can be added in the future.

Liu, Zhong↗

National Geothermal Data System: Transforming the Discovery, Access, and Analytics of Data for Geothermal Exploration

Compendium of Papers from the 38th Workshop on Geothermal Reservoir Engineering Stanford University, Stanford, California February 11-13, 2013 The National Geothermal Data System (NGDS) is a distributed, interoperable network of data collected from state geological surveys across all fifty states and the nation’s leading academic geothermal centers. The system serves as a platform for sharing consistent, reliable, geothermal-relevant technical data with users of all types, while supplying tools relevant for their work. As aggregated data supports new scientific findings, this content-rich linked data ultimately broadens the pool of knowledge available to promote discovery and development of commercial-scale geothermal energy production. Most of the up-front risks associated with geothermal development stem from exploration and characterization of subsurface resources. Wider access to distributed data will, therefore, result in lower costs for geothermal development. NGDS is on track to become fully operational by 2014 and will provide a platform for custom applications for accessing geothermal relevant data in the U.S. and abroad. It is being built on the U.S. Geoscience Information Network (USGIN) data integration framework to promote interoperability across the Earth sciences community. The basic structure of the NGDS employs state-of-the art informatics to advance geothermal knowledge. The following four papers comprising this Open-File Report are a compendium of presentations, from the 38th Annual Workshop on Geothermal Reservoir Engineering, taking place February 11-13, 2013 at Stanford University, Stanford, California. “NGDS Geothermal Data Domain: Assessment of Geothermal Community Data Needs,” outlines the efforts of a set of nationwide data providers to supply data for the NGDS. In particular, data acquisition, delivery, and methodology are discussed. The paper addresses the various types of data and metadata required and why simple links to existing data are insufficient for promoting geothermal exploration. Authors of this paper are Arlene Anderson, US DOE Geothermal Technologies Office, David Blackwell, Southern Methodist University (SMU), Cathy Chickering (SMU), Toni Boyd, Oregon Institute of Technology’s GeoHeat Center, Roland Horne, Stanford University, Matthew MacKenzie, Uberity, Joe Moore, University of Utah, Duane Nickull, Uberity, Stephen Richard, Arizona Geological Survey, and Lisa Shevenell, University of Nevada, Reno. “NGDS User Centered Design: Meeting the Needs of the Geothermal Community,” discusses the user- centered design approach taken in the development of a user interface solution for the NGDS. The development process is research based, highly collaborative, and incorporates state-of-the-art practices to ensure a quality user interface for the widest and greatest utility. Authors of this paper are Harold Blackman, Boise State University, Suzanne Boyd, Anthro-Tech, Kim Patten, Arizona Geological Survey, and Sam Zheng, Siemens Corporate Research. “Fueling Innovation and Adoption by Sharing Data on the DOE Geothermal Data Repository Node on the National Geothermal Data System,” describes the motivation behind the development of the Geothermal Data Repository (GDR) and its role in the NGDS. This includes the benefits of using the GDR to share geothermal data of all types and DOE’s data submission process. Authors of this paper are Jon Weers, National Renewable Energy Laboratory and Arlene Anderson, US DOE Geothermal Technologies Office. Finally, “Developing the NGDS Adoption of CKAN for Domestic & International Data Deployment,” provides an overview of the “Node-In-A-Box” software package designed to provide data consumers with a highly functional interface to access the system, and to ease the burden on data providers who wish to publish data in the system. It is important to note that this software package constitutes a reference implementation and that the NGDS architecture is based on open standards, which means other server software can make resources available, and other client applications can utilize NGDS data. Authors of this paper are Ryan Clark, Arizona Geological Survey (AZGS), Christoph Kuhmuench, Siemens Corporate Research, and Stephen Richard, AZGS.

15 GEOTHERMAL ENERGY↗