Search NASA⌕ Search

SEARCH · Search NASA

Results for “data access”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,351 records · Page 75

A Versatile Simulated Data Transport Layer for in Situ Workflows Performance Evaluation

In situ processing does not only allow scientific applications to face the explosion in data volume and velocity but also to address the time constraints of many simulation-analysis workflows by providing scientists with early insights about their applications at runtime. Multiple frameworks implement the concept of a data transport layer (DTL) to enable such in situ workflows. These tools are very versatile, directly or indirectly access the data generated on the same node, another node of the same compute cluster, or a completely distinct node, and allow data publishers and subscribers to run on the same computing resources or not. This versatility puts on researchers the onus of taking key decisions related to resource allocation and how to transport data to ensure the most efficient execution of their in situ workflows. However, domain scientists and workflow practitioners lack the appropriate tools to assess the respective performance of particular design and deployment options. In this paper we introduce a versatile simulated DTL designed to provide researchers with insights on the respective performance of different execution scenarios of in situ workflows. This open-source, standalone library builds on the SimGrid toolkit and can be linked to any SimGrid-based simulator. It facilitates the evaluation of the performance behavior, at scale, of different data transport configurations and the study of the effects of resource allocation strategies. We demonstrate the scalability, versatility, and accuracy of this simulated DTL by reproducing the execution of two synthetic benchmarks and of a real-world in situ workflow composed of an MPI application and a parallel data analysis. Results of simulations run on a single core show that the proposed library can simulate the interactions of tens of thousands of simulated processes deployed on two interconnected commodity clusters in a few seconds, and the execution by a thousand simulated processes of an in situ workflow in less than three minutes.

Suter, Fred [ORNL] (ORCID:0000000319021955)↗

CDDIS Support of IGS LEO Activities

The Crustal Dynamics Data Information System (CDDIS) has served as a global data center for the International GPS Service (IGS) since its start in June 1992, providing on-line access to data from over 175 sites on a daily and hourly basis. This paper will present an overview about the current status of the CDDIS GPS data and products archive with a look to the future support of LEO (Low Earth Orbiting missions), including the archive of high-rate data and on-board GPS receiver data.

Noll, Carey E.↗

Restoration of APOLLO Data by the NSSDC and PDS Lunar Data Node

The Apollo Lunar Surface Experiment Packages (ALSEPs), suites of instruments deployed by the Apollo 12. 14, 15, 16 and 17 astronauts on the lunar surface, still represent the only in-situ measurements of the Moon's environment taken over long time periods, Much of these data are housed at the National Space Science Data Center (NSSDC) at Goddard Space Flight Center but are in forms that are not readily usable, such as microfilm, hardcopy, and magnetic tapes with older, obsolete formats. The Lunar Data Node (LDN) has been formed under the auspices of the Planetary Data System (PDS) Geosciences Node to put relevant, scientifically important Apollo data into accessible digital form for use by researchers and mission planners. The LDN has prioritized the restoration of these data based on their scientific and engineering value and the level of effort required. We will report on progress made and plans for future data restorations.

Williams, David R.↗

The Open Data Repositorys Data Publisher

Data management and data publication are becoming increasingly important components of researcher's workflows. The complexity of managing data, publishing data online, and archiving data has not decreased significantly even as computing access and power has greatly increased. The Open Data Repository's Data Publisher software strives to make data archiving, management, and publication a standard part of a researcher's workflow using simple, web-based tools and commodity server hardware. The publication engine allows for uploading, searching, and display of data with graphing capabilities and downloadable files. Access is controlled through a robust permissions system that can control publication at the field level and can be granted to the general public or protected so that only registered users at various permission levels receive access. Data Publisher also allows researchers to subscribe to meta-data standards through a plugin system, embargo data publication at their discretion, and collaborate with other researchers through various levels of data sharing. As the software matures, semantic data standards will be implemented to facilitate machine reading of data and each database will provide a REST application programming interface for programmatic access. Additionally, a citation system will allow snapshots of any data set to be archived and cited for publication while the data itself can remain living and continuously evolve beyond the snapshot date. The software runs on a traditional LAMP (Linux, Apache, MySQL, PHP) server and is available on GitHub (http://github.com/opendatarepository) under a GPLv2 open source license. The goal of the Open Data Repository is to lower the cost and training barrier to entry so that any researcher can easily publish their data and ensure it is archived for posterity.

Astrobiology data↗

Issues and Solutions for Bringing Heterogeneous Water Cycle Data Sets Together

The water cycle research community has generated many regional to global scale products using data from individual NASA missions or sensors (e.g., TRMM, AMSR-E); multiple ground- and space-based data sources (e.g., Global Precipitation Climatology Project [GPCP] products); and sophisticated data assimilation systems (e.g., Land Data Assimilation Systems [LDAS]). However, it is often difficult to access, explore, merge, analyze, and inter-compare these data in a coherent manner due to issues of data resolution, format, and structure. These difficulties were substantiated at the recent Collaborative Energy and Water Cycle Information Services (CEWIS) Workshop, where members of the NASA Energy and Water cycle Study (NEWS) community gave presentations, provided feedback, and developed scenarios which illustrated the difficulties and techniques for bringing together heterogeneous datasets. This presentation reports on the findings of the workshop, thus defining the problems and challenges of multi-dataset research. In addition, the CEWIS prototype shown at the workshop will be presented to illustrate new technologies that can mitigate data access roadblocks encountered in multi-dataset research, including: (1) Quick and easy search and access of selected NEWS data sets. (2) Multi-parameter data subsetting, manipulation, analysis, and display tools. (3) Access to input and derived water cycle data (data lineage). It is hoped that this presentation will encourage community discussion and feedback on heterogeneous data analysis scenarios, issues, and remedies.

Acker, James↗

Electron microscopy holdings of the Protein Data Bank: the impact of the resolution revolution, new validation tools, and implications for the future

Abstract As a discipline, structural biology has been transformed by the three-dimensional electron microscopy (3DEM) “Resolution Revolution” made possible by convergence of robust cryo-preservation of vitrified biological materials, sample handling systems, and measurement stages operating a liquid nitrogen temperature, improvements in electron optics that preserve phase information at the atomic level, direct electron detectors (DEDs), high-speed computing with graphics processing units, and rapid advances in data acquisition and processing software. 3DEM structure information (atomic coordinates and related metadata) are archived in the open-access Protein Data Bank (PDB), which currently holds more than 11,000 3DEM structures of proteins and nucleic acids, and their complexes with one another and small-molecule ligands (~ 6% of the archive). Underlying experimental data (3DEM density maps and related metadata) are stored in the Electron Microscopy Data Bank (EMDB), which currently holds more than 21,000 3DEM density maps. After describing the history of the PDB and the Worldwide Protein Data Bank (wwPDB) partnership, which jointly manages both the PDB and EMDB archives, this review examines the origins of the resolution revolution and analyzes its impact on structural biology viewed through the lens of PDB holdings. Six areas of focus exemplifying the impact of 3DEM across the biosciences are discussed in detail (icosahedral viruses, ribosomes, integral membrane proteins, SARS-CoV-2 spike proteins, cryogenic electron tomography, and integrative structure determination combining 3DEM with complementary biophysical measurement techniques), followed by a review of 3DEM structure validation by the wwPDB that underscores the importance of community engagement.

Burley, Stephen K. (ORCID:0000000224879713)↗

GridOPTICS Software System

GridOPTICS Software System (GOSS) is a middleware that facilitates creation of new, modular and flexible operational and planning platforms that can meet the challenges of the next generation power grid. GOSS enables Department of Energy, power system utilities, and vendors to build better tools faster. GOSS makes it possible to integrate Future Power Grid Initiative software products/prototypes into existing power grid software systems, including the PNNL PowerNet and EIOC environments. GOSS is designed to allow power grid applications developed for different underlying software platforms installed in different utilities to communicate with ease. This can be done in compliance with existing security and data sharing policies between the utilities. GOSS not only supports one-to-one data transfer between applications, but also publisher/subscriber scheme. To support interoperability requirements of future EMS, GOSS is designed for CIM compliance. In addition to this, it supports authentication and authorization capabilities to protect the system from cyber threats. In summary, the contributions of the GOSS middleware are as follows: A platform to support future EMS development. A middleware that promotes interoperability between power grid applications. A distributed architecture that separates data sources from power grid applications. Support for data exchange with either one-to-one or publisher/subscriber interfaces. An authentication and authorization scheme for limiting the access to data between utilities.

Akyol, BoraA↗

WMC Database Evaluation. Case Study Report

The WMC Database is ultimately envisioned to hold a collection of experimental data, design information, and information from computational models. This project was a first attempt at using the Database to access experimental data and extract information from it. This evaluation shows that the Database concept is sound and robust, and that the Database, once fully populated, should remain eminently usable for future researchers.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Department of Energy’s Atmospheric System Research (ASR) Program’s Workshop on the Future of Atmospheric Large Eddy Simulation (LES): Workshop Report

Large-eddy simulation (LES) is used as a tool to understand physical processes such as turbulence, aerosols, clouds, precipitation, radiation, the interactions among all these, and their interactions with the underlying surface. Over the next 10 years, LES will drive fundamental progress in open scientific questions in these areas as LES is increasingly used to gain understanding of complex interacting physical processes involving atmospheric turbulence. This growth will be driven both by scientific demand and the expansion of computational resources needed to conduct LES, and the form that the growth takes will largely be determined by how computational resources are leveraged for scientific gain. In particular, we suggest that computational resources are likely to be leveraged in two separate but not necessarily distinct ways. On one hand, growth in computational resources will allow LES to be made more routine, that is, performed more frequently, while on the other hand, the computational expense (measured in total floating point operations) afforded to individual LES will expand dramatically, allowing simulations to increase in both domain size and resolution as well as physical detail. Current U.S. Department of Energy (DOE) projects such as LES ARM Symbiotic Simulation and Observation Activity (LASSO) are leading the way in conducting routine LES, building large, public databases that are accessible for data science, sensitivity studies, and training for machine learning. LES will also become more routine as it becomes more accessible for individual researchers to address their scientific questions of interest. Scientific questions addressed by LES over the next 10 years are likely to include cloud organization and aggregation; aerosol cloud interactions and atmospheric chemistry (including geo-engineering); urban-scale LES; atmospheric extreme events, ranging from small-scale severe weather to wildfires; and ocean-wave-atmosphere interactions. Further LES-related research will likely grow significantly in areas related to societal impact studies of air quality and extreme weather events, applications to renewable energy forecasting and resource assessment, and aid in decision-making processes. The growth in the use of LES in atmospheric science research will drive the need for better physical process representations (e.g., cloud aerosol microphysics, radiation, and atmospheric chemistry) at the scales resolved by LES. To date, many of the process representations used by LES have been taken directly from coarser-resolution models. Promising methods for LES process representations include superdroplet and quadrature methods for microphysics, 3D approaches for radiation, and better representation of chemistry and aerosol processes. At LES resolution, land-atmosphere interactions for complex terrains, land cover/types, biogeochemistry, and plant canopy models are needed as an improvement beyond traditional and widely used Monin-Obuhkov similarity theory.

54 ENVIRONMENTAL SCIENCES↗

Department of Energy’s Atmospheric System Research (ASR) Program’s Workshop on the Future of Atmospheric Large Eddy Simulation (LES) (Workshop Report)

Large-eddy simulation (LES) is used as a tool to understand physical processes such as turbulence, aerosols, clouds, precipitation, radiation, the interactions among all these, and their interactions with the underlying surface. Over the next 10 years, LES will drive fundamental progress in open scientific questions in these areas as LES is increasingly used to gain understanding of complex interacting physical processes involving atmospheric turbulence. This growth will be driven both by scientific demand and the expansion of computational resources needed to conduct LES, and the form that the growth takes will largely be determined by how computational resources are leveraged for scientific gain. In particular, we suggest that computational resources are likely to be leveraged in two separate but not necessarily distinct ways. On one hand, growth in computational resources will allow LES to be made more routine, that is, performed more frequently, while on the other hand, the computational expense (measured in total floating point operations) afforded to individual LES will expand dramatically, allowing simulations to increase in both domain size and resolution as well as physical detail. Current U.S. Department of Energy (DOE) projects such as LES ARM Symbiotic Simulation and Observation Activity (LASSO) are leading the way in conducting routine LES, building large, public databases that are accessible for data science, sensitivity studies, and training for machine learning. LES will also become more routine as it becomes more accessible for individual researchers to address their scientific questions of interest. Scientific questions addressed by LES over the next 10 years are likely to include cloud organization and aggregation; aerosol cloud interactions and atmospheric chemistry (including geo-engineering); urban-scale LES; atmospheric extreme events, ranging from small-scale severe weather to wildfires; and ocean-wave-atmosphere interactions. Further LES-related research will likely grow significantly in areas related to societal impact studies of air quality and extreme weather events, applications to renewable energy forecasting and resource assessment, and aid in decision-making processes.

54 ENVIRONMENTAL SCIENCES↗

Ada and knowledge-based systems: A prototype combining the best of both worlds

A software architecture is described which facilitates the construction of distributed expert systems using Ada and selected knowledge based systems. This architecture was utilized in the development of a Knowledge-based Maintenance Expert System (KNOMES) prototype for the Space Station Mobile Service Center (MSC). The KNOMES prototype monitors a simulated data stream from MSC sensors and built-in test equipment. It detects anomalies in the data and performs diagnosis to determine the cause. The software architecture which supports the KNOMES prototype allows for the monitoring and diagnosis tasks to be performed concurrently. The basic concept of this software architecture is named ACTOR (Ada Cognitive Task ORganization Scheme). An individual ACTOR is a modular software unit which contains both standard data processing and artificial intelligence components. A generic ACTOR module contains Ada packages for communicating with other ACTORs and accessing various data sources. The knowledge based component of an ACTOR determines the role it will play in a system. In this prototype, an ACTOR will monitor the MSC data stream.

Brauer, David C.↗

Prototyping a large-scale distributed system for the Great Observatories era - NASA Astrophysics Data System (ADS)

The NASA Astrophysics Data System (ADS) is a distributed information system intended to support research in the Great Observatories era, to simplify access to data, and to enable simultaneous analyses of multispectral data sets. Here, the user agent and interface, its functions, and system components are examined, and the system architecture and infrastructure is addressed. The present status of the system and related future activities are examined.

Shames, Peter↗

GridOPTICS Software System

GridOPTICS Software System (GOSS) is a middleware that facilitates creation of new, modular and flexible operational and planning platforms that can meet the challenges of the next generation power grid. GOSS enables Department of Energy, power system utilities, and vendors to build better tools faster. GOSS makes it possible to integrate Future Power Grid Initiative software products/prototypes into existing power grid software systems, including the PNNL PowerNet and EIOC environments. GOSS is designed to allow power grid applications developed for different underlying software platforms installed in different utilities to communicate with ease. This can be done in compliance with existing security and data sharing policies between the utilities. GOSS not only supports one-to-one data transfer between applications, but also publisher/subscriber scheme. To support interoperability requirements of future EMS, GOSS is designed for CIM compliance. In addition to this, it supports authentication and authorization capabilities to protect the system from cyber threats. In summary, the contributions of the GOSS middleware are as follows: A platform to support future EMS development. A middleware that promotes interoperability between power grid applications. A distributed architecture that separates data sources from power grid applications. Support for data exchange with either one-to-one or publisher/subscriber interfaces. An authentication and authorization scheme for limiting the access to data between utilities.

Akyol, Bora A↗

RE Data Explorer: Supporting Renewable Energy Zones to Enable Low Emission Development

This fact sheet overviews the benefits of using the RE Data Explorer tool to analyze and develop renewable energy zones. Renewable energy zones are developed through a transmission planning and approval process customized for renewable energy. RE Data Explorer analysis can feed into broader stakeholder discussions and allow stakeholders to easily visualize potential zones. Stakeholders can access pertinent data to inform transmission planning and enable investment.

54 ENVIRONMENTAL SCIENCES↗

The CRUTEM4 land-surface air temperature data set: construction, previous versions and dissemination via Google Earth

The CRUTEM4 (Climatic Research Unit Temperature, version 4) land-surface air temperature data set is one of the most widely used records of the climate system. Here we provide an important additional dissemination route for this data set: online access to monthly, seasonal and annual data values and time series graphs via Google Earth. This is achieved via an interface written in Keyhole Markup Language (KML) and also provides access to the underlying weather station data used to construct the CRUTEM4 data set. A mathematical description of the construction of the CRUTEM4 data set (and its predecessor versions) is also provided, together with an archive of some previous versions and a recommendation for identifying the precise version of the data set used in a particular study. The CRUTEM4 data set used here is available from doi:10.5285/EECBA94F-62F9-4B7C-88D3-482F2C93C468.

54 ENVIRONMENTAL SCIENCES↗

Performance and Reliability Assessment of the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) Data Advisor (ADA)

The Atmospheric Radiation Measurement (ARM) User Facility provides one of the world's largest openly accessible repositories of atmospheric observations through the ARM Data Discovery platform. Although the repository contains more than three decades of measurements collected from permanent observatories, mobile facilities, aircraft campaigns, and field experiments, identifying appropriate datasets can be challenging, particularly for new users unfamiliar with ARM instrumentation and datastream organization. To improve data accessibility, the ARM Data Center developed the ARM Data Advisor (ADA), an artificial intelligence-powered assistant designed to facilitate scientific data discovery, dataset interpretation, and user guidance. This report evaluates ADA's performance as a domain-specific scientific assistant using realistic atmospheric science workflows. The evaluation examines five key capabilities: data retrieval and curation efficiency, hallucination resistance, scientific reasoning, response to ambiguous queries, and content retention and session continuity. Representative prompts were developed to simulate typical interactions between researchers and the ARM Data Discovery platform, and ADA's responses were assessed for retrieval completeness, scientific accuracy, consistency, and practical usefulness. In these representative tests, ADA reduced the complexity of discovering and accessing ARM datasets by recommending appropriate datastreams, explaining instrumentation, interpreting metadata, and assisting with data processing workflows. ADA also exhibits strong domain knowledge of atmospheric science terminology and generally resists hallucination by acknowledging unavailable datasets and requesting clarification when appropriate. Overall, the results indicate that ADA represents a promising advancement in scientific data discovery within the ARM User Facility and has considerable potential to improve researcher productivity, particularly for new users and interdisciplinary scientists seeking efficient access to ARM observations.

Salvador, Christian [ORNL] (ORCID:0000000283287777↗

An Adaptive Memory Interface Controller for Improving Bandwidth Utilization of Hybrid and Reconfigurable Systems

Emerging applications such as data mining, bioinformatics, knowledge discovery, social network analysis are irregular. They use data structures based on pointers or linked lists, such as graphs, unbalanced trees or unstructures grids, which generates unpredictable memory accesses. These data structures usually are large, but difficult to partition. These applications mostly are memory bandwidth bounded and have high synchronization intensity. However, they also have large amounts of inherent dynamic parallelism, because they potentially perform a task for each one of the element they are exploring. Several efforts are looking at accelerating these applications on hybrid architectures, which integrate general purpose processors with reconfigurable devices. Some solutions, which demonstrated significant speedups, include custom-hand tuned accelerators or even full processor architectures on the reconfigurable logic. In this paper we present an approach for the automatic synthesis of accelerators from C, targeted at irregular applications. In contrast to typical High Level Synthesis paradigms, which construct a centralized Finite State Machine, our approach generates dynamically scheduled hardware components. While parallelism exploitation in typical HLS-generated accelerators is usually bound within a single execution flow, our solution allows concurrently running multiple execution flow, thus also exploiting the coarser grain task parallelism of irregular applications. Our approach supports multiple, multi-ported and distributed memories, and atomic memory operations. Its main objective is parallelizing as many memory operations as possible, independently from their execution time, to maximize the memory bandwidth utilization. This significantly differs from current HLS flows, which usually consider a single memory port and require precise scheduling of memory operations. A key innovation of our approach is the generation of a memory interface controller, which dynamically maps concurrent memory accesses to multiple ports. We present a case study on a typical irregular kernel, Graph Breadth First search (BFS), exploring different tradeoffs in terms of parallelism and number of memories.

BFS↗