Search NASA⌕ Search

SEARCH · Search NASA

Results for “data access”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55

Executive and communications services to support the IPAD environment

The principal purposes of the prototype executive software are to provide a system independent interface to the underlying host system and to allow for extension to full IPAD executive services as described in the preliminary design. A basic set of functions is included in the prototype to meet the requirements of the other components of the prototype, principally IPID, the IPAD data management system. The functions were chosen so that they would be readily built on any of the proposed host systems with minimal redesign and execution overhead. The functions fall into five categories: access to host data, access to data files, access to communication services, data transformation, and instrumentation for performance measurement. Communication services provide message delivery between processes in a network of heterogeneous computers. Data transformation services and communication services ensure data type validity and data integrity of messages exchanged between processes.

Tanner, J. G.↗

Intelligent Systems Technologies to Assist in Utilization of Earth Observation Data

With the launch of several Earth observing satellites over the last decade, we are now in a data rich environment. From NASA's Earth Observing System (EOS) satellites alone, we are accumulating more than 3 TB per day of raw data and derived geophysical parameters. The data products are being distributed to a large user community comprising scientific researchers, educators and operational government agencies. Notable progress has been made in the last decade in facilitating access to data. However, to realize the full potential of the growing archives of valuable scientific data, further progress is necessary in the transformation of data into information, and information into knowledge that can be used in particular applications. Sponsored by NASA s Intelligent Systems Project within the Computing, Information and Communication Technology (CICT) Program, a conceptual architecture study has been conducted to examine ideas to improve data utilization through the addition of intelligence into the archives in the context of an overall knowledge building system. Potential Intelligent Archive concepts include: 1) Mining archived data holdings using Intelligent Data Understanding algorithms to improve metadata to facilitate data access and usability; 2) Building intelligence about transformations on data, information, knowledge, and accompanying services involved in a scientific enterprise; 3) Recognizing the value of results, indexing and formatting them for easy access, and delivering them to concerned individuals; 4) Interacting as a cooperative node in a web of distributed systems to perform knowledge building (i.e., the transformations from data to information to knowledge) instead of just data pipelining; and 5) Being aware of other nodes in the knowledge building system, participating in open systems interfaces and protocols for virtualization, and collaborative interoperability. This paper presents some of these concepts and identifies issues to be addressed by research in future intelligent systems technology.

Ramapriyan, Hampapuram K.↗

Open Data and the Free Access to Law Movement in the United States

US President Barack Obama issued Executive Order 13642-Making Open and Machine Readable the New Default for Government - on May, 9 2013, mandating, wherever legally permissible and possible, that US Government information be made open to the public.[1] This edict accelerated the construction of and framework for data repositories, and data citation principles and practices, such as data.gov. As a corollary, researchers across the country's national laboratories found themselves creating data management plans, applying data set metadata standards, and ensuring the long-term access of data for federally funded scientific research.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Improved Access and Analysis of Data Provides New Opportunities for Smart Cities

Making smart, informed, data-driven energy decisions for cities requires large amounts of high-value data and analysis. Cities’ energy data has traditionally been difficult to access and costly to store and manage for city administrators. When data is available, many cities lack the expertise to properly analyze and model large data sets. NREL’s transparency and knowledge about data availability, integration, and interpretation is critical to decision-making for cities.

cities' data↗

MERRA-2 Data and Analytic Services at NASA GES DISC for Climate Extremes Study

NASA's climate reanalysis datasets from the Modern Era Retrospective-analysis for Research and Applications, Version 2 (MERRA-2) contains numerous long-term atmosphere, land, and ocean data products from 1980-present. MERRA-2 datasets, such as precipitation, soil moisture, and temperature, have been used widely to study extreme events. The native archived MERRA-2 data files are day-file (hourly time interval) and month-file, containing up to 125 parameters in one file. Due to the large number of data files and volumes, it is challenging for users, especially the applications research community, to handle the original hourly data files for long time periods to analyze extreme events. In this presentation, we review MERRA-2 data for studies of extreme conditions, and demonstrate analytic services at the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). One of the current operational services, 'subsetter', allows users to download only specific data of interest, i.e. data selected by parameter, region, and time period. New services are under development that will provide more 'on-the-fly' statistical calculations when downloading data; improve efficiency when accessing long time-series data. We will provide additional "How-to" resources that include step-by-step instructions on data access and usage. We have tested restructuring of day-files in an optimized data cube, which has significantly improved system performance for accessing long time-series. Overall performance is associated with cube size and structure, data compression method, and how the data are accessed. The optimized data cube structure will enable better online analytic services for statistical analysis and extreme events mining. To demonstrate the service, we use an extreme drought associated with the anomalous 2016 monsoon over southern Asia. This prototype time-series service may be augmented in the cloud infrastructure in the future.

data access↗

NASA Global Satellite and Model Data Products and Services for Tropical Meteorology and Climatology

Satellite remote sensing and model data play an important role in research and applications of tropical meteorology and climatology over vast, data sparse oceans and remote continents. Since the first weather satellite was launched by NASA in 1960, a large collection of NASA's Earth science data is freely available to the research and application communities around the world, significantly improving our overall understanding of the Earth system and environment. Established in the mid-80s, the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), located in Maryland, USA, is a data archive center for multidisciplinary, satellite and model assimilation data products. As one of the 12 NASA data centers in Earth Sciences, GES DISC hosts several important NASA satellite missions for tropical meteorology and climatology such as the Tropical Rainfall Measuring Mission (TRMM), the Global Precipitation Measurement (GPM) mission and the Modern-Era Retrospective analysis for Research and Applications (MERRA). Over the years, GES DISC has developed data services to facilitate data discovery, access, distribution, analysis and visualization, including Giovanni, an online analysis and visualization tool without the need to download data and software. Despite many efforts for improving data access, still quite a number of challenges remain, such as finding datasets and services for a specific research topic or project, especially for inexperienced users or users outside the remote sensing community. In this article, we list and describe major NASA satellite remote sensing and model datasets and services for tropical meteorology and climatology along with examples of using the data and services, in hope that may help users better utilize the information in their research and applications.

tropical meteorology and climatology, data, servic↗

Improving the Accessibility and Usability of Geothermal Information with Data Lakes and Data Pipelines on the Geothermal Data Repository

The Geothermal Data Repository (GDR) provides universal access to data and information resulting from research and development activities funded by the Department of Energy (DOE). The GDR has extended this universal access to big data through integration with data lakes developed by the Open Energy Data Initiative (OEDI). Previously, large datasets such as seismic waveform or distributed acoustic sensing (DAS) data could only be accessed by institutions with high performance data storage and compute capabilities, effectively limiting the accessibility of big data to national labs, larger universities, and major corporations. Moreover, the time and resources needed to transport big data and configure them can produce additional barriers to use. Many of the standard formats used for structured data models (also known as content models) are incapable of handling big data and can introduce additional usability problems, often requiring data to be reformatted prior to use. This paper will explore how recent integrations between the GDR and the OEDI data lake have improved the accessibility and usability of geothermal data in a big way, making the data available to a broader audience, and enabling collaborative analysis and innovation across the greater geothermal industry.

access↗

Improving the Accessibility and Usability of Geothermal Information with Data Lakes and Data Pipelines on the Geothermal Data Repository: Preprint

The Geothermal Data Repository (GDR) provides universal access to data and information resulting from research and development activities funded by the Department of Energy (DOE). The GDR has extended this universal access to big data through integration with data lakes developed by the Open Energy Data Initiative (OEDI). Previously, large datasets such as seismic waveform or distributed acoustic sensing (DAS) data could only be accessed by institutions with high performance data storage and compute capabilities, effectively limiting the accessibility of big data to national labs, larger universities, and major corporations. Moreover, the time and resources needed to transport big data and configure them can produce additional barriers to use. Many of the standard formats used for structured data models (also known as content models) are incapable of handling big data and can introduce additional usability problems, often requiring data to be reformatted prior to use. This paper will explore how recent integrations between the GDR and the OEDI data lake have improved the accessibility and usability of geothermal data in a big way, making the data available to a broader audience, and enabling collaborative analysis and innovation across the greater geothermal industry.

access↗

A Conditions Data Management System for HEP Experiments

Conditions data infrastructure for both ATLAS and CMS have to deal with the management of several Terabytes of data. Distributed computing access to this data requires particular care and attention to manage request-rates of up to several tens of kHz. Thanks to the large overlap in use cases and requirements, ATLAS and CMS have worked towards a common solution for conditions data management with the aim of using this design for data-taking in Run 3. In the meantime other experiments, including NA62, have expressed an interest in this cross- experiment initiative. For experiments with a smaller payload volume and complexity, there is particular interest in simplifying the payload storage. The conditions data management model is implemented in a small set of relational database tables. A prototype access toolkit consisting of an intermediate web server has been implemented, using standard technologies available in the Java community. Access is provided through a set of REST services for which the API has been described in a generic way using standard Open API specications, implemented in Swagger. Such a solution allows the automatic generation of client code and server stubs and further allows changes in the backend technology transparently. An important advantage of using a REST API for conditions access is the possibility of caching identical URLs, addressing one of the biggest challenges that large distributed computing solutions impose on conditions data access, avoiding direct DB access by means of standard web proxy solutions.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Modifying the Heliophysics Data Policy to Better Enable Heliophysics Research

The Heliophysics (HP) Science Data Management Policy, adopted by HP in June 2007, has helped to provide a structure for the HP data lifecycle. It provides guidelines for Project Data Management Plans and related documents, initiates Resident Archives to maintain data services after a mission ends, and outlines a route to the unification of data finding, access, and distribution through Virtual observatories. Recently we have filled in missing pieces that assure more coherence and a home for the VxOs (through the 'Heliophsyics Data and Model Consortium'), and provide greater clarity with respect to long term archiving. In particular, the new policy which has been vetted with many community members, details the 'Final Archives' that are to provide long-term data access. These are distinguished from RAs in that they provide little additional service beyond servicing data, but critical to their success is that the final archival materials include calibrated data in useful formats such as one finds in CDAWeb and various ASCII or FITS archives. Having a clear goal for legacy products, to be detailed as part of the Mission Archives Plans presented at Senior Reviews, will help to avoid the situation so common in the past of having archival products that preserve bits well but not readily usable information. We hope to avoid the need for the large numbers of 'data upgrade' projects that have been necessary in recent years.

Hayes, Jeffrey↗

Generalized Nuclear Data: A New Structure (with Supporting Infrastructure) for Handling Nuclear Data

The Evaluated Nuclear Data File (ENDF) format was designed in the 1960s to accommodate neutron reaction data to support nuclear engineering applications in power, national security and criticality safety. Over the years, the scope of the format has been extended to handle many other kinds of data including charged particle, decay, atomic, photo-nuclear and thermal neutron scattering. Although ENDF has wide acceptance and support for many data types, its limited support for correlated particle emission, limited numeric precision, and general lack of extensibility mean that the nuclear data community cannot take advantage of many emerging opportunities. More generally, the ENDF format provides an unfriendly environment that makes it difficult for new data evaluators and users to create and access nuclear data. The Cross Section Evaluation Working Group (CSEWG) has begun the design of a new Generalized Nuclear Data (or 'GND') structure, meant to replace older formats with a hierarchy that mirrors the underlying physics, and is aligned with modern coding and database practices. In support of this new structure, Lawrence Livermore National Laboratory (LLNL) has updated its nuclear data/reactions management package Fudge to handle GND structured nuclear data. Fudge provides tools for converting both the latest ENDF format (ENDF-6) and the LLNL Evaluated Nuclear Data Library (ENDL) format to and from GND, as well as for visualizing, modifying and processing (i.e., converting evaluated nuclear data into a form more suitable to transport codes) GND structured nuclear data. GND defines the structure needed for storing nuclear data evaluations and the type of data that needs to be stored. But unlike ENDF and ENDL, GND does not define how the data are to be stored in a file. Currently, Fudge writes the structured GND data to a file using the eXtensible Markup Language (XML), as it is ASCII based and can be viewed with any text editor. XML is a meta-language, meaning that it has a primitive set of definitions for representing hierarchical data/text in a file. Other meta-languages, like HDF5 which stores the data in binary form, can also be used to store GND in a file. In this paper, we will present an overview of the new GND data structures along with associated tools in Fudge.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Advance Network Reservation and Provisioning for Science

We are witnessing a new era that offers new opportunities to conduct scientific research with the help of recent advancements in computational and storage technologies. Computational intensive science spans multiple scientific domains, such as particle physics, climate modeling, and bio-informatics simulations. These large-scale applications necessitate collaborators to access very large data sets resulting from simulations performed in geographically distributed institutions. Furthermore, often scientific experimental facilities generate massive data sets that need to be transferred to validate the simulation data in remote collaborating sites. A major component needed to support these needs is the communication infrastructure which enables high performance visualization, large volume data analysis, and also provides access to computational resources. In order to provide high-speed on-demand data access between collaborating institutions, national governments support next generation research networks such as Internet 2 and ESnet (Energy Sciences Network). Delivering network-as-a-service that provides predictable performance, efficient resource utilization and better coordination between compute and storage resources is highly desirable. In this paper, we study network provisioning and advanced bandwidth reservation in ESnet for on-demand high performance data transfers. We present a novel approach for path finding in time-dependent transport networks with bandwidth guarantees. We plan to improve the current ESnet advance network reservation system, OSCARS [3], by presenting to the clients, the possible reservation options and alternatives for earliest completion time and shortest transfer duration. The Energy Sciences Network (ESnet) provides high bandwidth connections between research laboratories and academic institutions for data sharing and video/voice communication. The ESnet On-Demand Secure Circuits and Advance Reservation System (OSCARS) establishes guaranteed bandwidth of secure virtual circuits at a certain time, for a certain bandwidth and length of time. Though OSCARS operates within the ESnet, it also supplies end-to-end provisioning between multiple autonomous network domains. OSCARS gets reservation requests through a standard web service interface, and conducts a Quality-of-service (QoS) path for bandwidth guarantees. Multi-protocol Label Switching (MPLS) and the Resource Reservation Protocol (RSVP) enable to create a virtual circuit using Label Switched Paths (LSP's). It contains three main components: a reservation manager, a bandwidth scheduler, and a path setup subsystem. The bandwidth scheduler needs to have information about the current and future states of the network topology in order to accomplish end-to-end bandwidth guaranteed paths.

97 MATHEMATICS AND COMPUTING↗

Dynamic Virtual Chunks: On Supporting Efficient Accesses to Compressed Scientific Data

Data compression could ameliorate the I/O pressure of data-intensive scientific applications. Unfortunately, the conventional wisdom of naively applying data compression to the file or block brings the dilemma between efficient random accesses and high compression ratios. File-level compression barely supports efficient random accesses to the compressed data: any retrieval request need trigger the decompression from the beginning of the compressed file. Block-level compression provides flexible random accesses to the compressed blocks, but introduces extra overhead when applying the compressor to each and every block that results in a degraded overall compression ratio. This paper extends our prior work that introduces virtual chunks offering efficient random accesses to the compressed scientific data without sacrificing the compression ratio. Virtual chunks are logical blocks pointed at by appended references without breaking the physical continuity of the file content. These references allow the decompression to start from an arbitrary position (efficient random accesses), while no per-block overhead is introduced because the file’s physical entirety is retained (high compression ratio). One limitation of virtual chunk is it only supports static references. This paper presents the algorithms, analysis, and evaluations of dynamic virtual chunks to deal with the cases where the references are updated dynamically.

I/O compression performance optimization↗

VIEWCACHE: An incremental pointer-based access method for autonomous interoperable databases

One of biggest problems facing NASA today is to provide scientists efficient access to a large number of distributed databases. Our pointer-based incremental database access method, VIEWCACHE, provides such an interface for accessing distributed data sets and directories. VIEWCACHE allows database browsing and search performing inter-database cross-referencing with no actual data movement between database sites. This organization and processing is especially suitable for managing Astrophysics databases which are physically distributed all over the world. Once the search is complete, the set of collected pointers pointing to the desired data are cached. VIEWCACHE includes spatial access methods for accessing image data sets, which provide much easier query formulation by referring directly to the image and very efficient search for objects contained within a two-dimensional window. We will develop and optimize a VIEWCACHE External Gateway Access to database management systems to facilitate distributed database search.

Roussopoulos, N.↗

RCSB Protein Data Bank: supporting research and education worldwide through explorations of experimentally determined and computationally predicted atomic level 3D biostructures

The Protein Data Bank (PDB) was established as the first open-access digital data resource in biology and medicine in 1971 with seven X-ray crystal structures of proteins. Today, the PDB houses >210 000 experimentally determined, atomic level, 3D structures of proteins and nucleic acids as well as their complexes with one another and small molecules ( e.g. approved drugs, enzyme cofactors). These data provide insights into fundamental biology, biomedicine, bioenergy and biotechnology. They proved particularly important for understanding the SARS-CoV-2 global pandemic. The US-funded Research Collaboratory for Structural Bioinformatics Protein Data Bank (RCSB PDB) and other members of the Worldwide Protein Data Bank (wwPDB) partnership jointly manage the PDB archive and support >60 000 `data depositors' (structural biologists) around the world. wwPDB ensures the quality and integrity of the data in the ever-expanding PDB archive and supports global open access without limitations on data usage. The RCSB PDB research-focused web portal at https://www.rcsb.org/ (RCSB.org) supports millions of users worldwide, representing a broad range of expertise and interests. In addition to retrieving 3D structure data, PDB `data consumers' access comparative data and external annotations, such as information about disease-causing point mutations and genetic variations. RCSB.org also provides access to >1 000 000 computed structure models (CSMs) generated using artificial intelligence/machine-learning methods. To avoid doubt, the provenance and reliability of experimentally determined PDB structures and CSMs are identified. Related training materials are available to support users in their RCSB.org explorations.

59 BASIC BIOLOGICAL SCIENCES↗