Search NASA⌕ Search

SEARCH · Search NASA

Results for “data access”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Querying Large Scientific Data Sets with Adaptable IO System ADIOS

When working with a large dataset, a relatively small fraction of data records are of interest in each analysis operation. For example, while examining a billion-particle dataset from an accelerator model, the scientists might focus on a few thousand fastest particles, or on the particle farthest from the beam center. In general, this type of selective data access is challenging because the selected data records could be anywhere in the dataset and require a significant amount of time to locate and retrieve. In this paper, we report our experience of addressing this data access challenge with the Adaptable IO System ADIOS. More specifically, we design a query interface for ADIOS to allow arbitrary combinations of range conditions on known variables, implement a number of different mechanisms for resolving these selection conditions, and devise strategies to reduce the time needed to retrieve the scattered data records. In many cases, the query mechanism can retrieve the selected data records orders of magnitude faster than the brute-force approach.Our work relies heavily on the in situ data processing feature of ADIOS to allow user functions to be executed in the data transport pipeline. This feature allows us to build indexes for efficient query processing, and to perform other intricate analyses while the data is in memory.

Gu, Junmin↗

Using Xrootd to federate regional storage

While the LHC data movement systems have demonstrated the ability to move data at the necessary throughput, we have identified two weaknesses: the latency for physicists to access data and the complexity of the tools involved. To address these, both ATLAS and CMS have begun to federate regional storage systems using Xrootd. Xrootd, referring to a protocol and implementation, allows us to provide data access to all disk-resident data from a single virtual endpoint. This redirector discovers the actual location of the data and redirects the client to the appropriate site. The approach is particularly advantageous since typically the redirection requires much less than 500 milliseconds and the Xrootd client is conveniently built into LHC physicists analysis tools. Currently, there are three regional storage federations - a US ATLAS region, a European CMS region, and a US CMS region. The US ATLAS and US CMS regions include their respective Tier 1, Tier 2 and some Tier 3 facilities, a large percentage of experimental data is available via the federation. Additionally, US ATLAS has begun studying low-latency regional federations of close-by sites. From the base idea of federating storage behind an endpoint, the implementations and use cases diverge. The CMS software framework is capable of efficiently processing data over high-latency links, so using the remote site directly is comparable to accessing local data. The ATLAS processing model allows a broad spectrum of user applications with varying degrees of performance with regard to latency, a particular focus has been optimizing n-tuple analysis. Both VOs use GSI security. ATLAS has developed a mapping of VOMS roles to specific file system authorizations, while CMS has developed callouts to the site's mapping service. Each federation presents a global namespace to users. For ATLAS, the global-to-local mapping is based on a heuristic-based lookup from the site's local file catalog, while CMS does the mapping based on translations given in a configuration file. We will also cover the latest usage statistics and interesting use cases that have developed over the previous 18 months.

Bauerdick, L.↗

A Brief Survey of Media Access Control, Data Link Layer, and Protocol Technologies for Lunar Surface Communications

This paper surveys and describes some of the existing media access control and data link layer technologies for possible application in lunar surface communications and the advanced wideband Direct Sequence Code Division Multiple Access (DSCDMA) conceptual systems utilizing phased-array technology that will evolve in the next decade. Time Domain Multiple Access (TDMA) and Code Division Multiple Access (CDMA) are standard Media Access Control (MAC) techniques that can be incorporated into lunar surface communications architectures. Another novel hybrid technique that is recently being developed for use with smart antenna technology combines the advantages of CDMA with those of TDMA. The relatively new and sundry wireless LAN data link layer protocols that are continually under development offer distinct advantages for lunar surface applications over the legacy protocols which are not wireless. Also several communication transport and routing protocols can be chosen with characteristics commensurate with smart antenna systems to provide spacecraft communications for links exhibiting high capacity on the surface of the Moon. The proper choices depend on the specific communication requirements.

Wallett, Thomas M.↗

Performance of Julia for High Energy Physics Analyses

We argue that the Julia programming language is a compelling alternative to currently more common implementations in Python and C++ for common data analysis workflows in high energy physics. We compare the speed of implementations of different workflows in Julia with those in Python and C++. Furthermore, our studies show that the Julia implementations are competitive for tasks that are dominated by computational load rather than data access. For work that is dominated by data access, we demonstrate an application with concurrent file reading and parallel data processing.

97 MATHEMATICS AND COMPUTING↗

A review of privacy in energy applications

As the distribution system continues to experience an increase in distributed energy resource (DER) and electric vehicle (EV) penetration, so does the need for new solutions that can help grid operators manage and leverage their capabilities. This will undoubtedly lead to new operational schemes and business opportunities that will transform the traditional consumer into a prosumer who will be more actively engaged in grid operations. Although the field is still under active development, many of the potential use cases presented in literature or industry are built upon edge computing, two-way communications, and other innovative computational constructs to attain their goals. However, at their core, many use cases assume a great level of data access to aid with the decision-making process, an assumption that may need to be revised to ensure fair and equitable operational processes are maintained. This may be particularly true as edge resources are predicted to participate in retail-side, many-to-many, or peer-to-peer markets and thus may lead to financial impacts if data access considerations are ignored. The need to revise data access mechanisms can be further justified by the introduction of new participants into the operational process, who do not have the same level of trust, nor the incentives to focus on energy delivery as their primary objective. At the same time, more consumers are becoming aware of their own data, and the potential impacts of its abuse. To help solution developers better understand these risks, this report has been developed to offer an initial introduction to the topic of privacy. This is achieved by 1) Highlighting the need for privacy-aware solutions; 2) Encouraging system designers to be inquisitive about the status quo; 3) Documenting the existing threat space; 4) Presenting and evaluating tools that may be helpful towards enabling better privacy postures; and 5) Making recommendations to encourage the adoption of better practices. From a technical perspective, the report focuses on evaluating two potential techniques by applying them to the Transactive Energy Space. Based on the obtained results, it can be established that differential privacy (DP) methods may have limited applicability when highly correlated, time-series data records need to be protected. However, DP may be a powerful tool when it is used to aggregate and analyze mid-size and large-size data sets in a more traditional statistical environment. The second tool under evaluation is threshold cryptography, which can guarantee complete secrecy (and thus privacy) but requires the establishment of key management procedures and dedicated communication channels for key coordination. Therefore, due to its increased computational overhead, the use of threshold cryptography must be weighted using a cost/benefit analysis on a per-application basis.

24 POWER TRANSMISSION AND DISTRIBUTION↗

PIMS Data Storage, Access, and Neural Network Processing

The Principal Investigator Microgravity Services (PIMS) project at NASA's Lewis Research Center has supported microgravity science Principal Investigator's (PIs) by processing, analyzing, and storing the acceleration environment data recorded on the NASA Space Shuttles and the Russian Mir space station. The acceleration data recorded in support of the microgravity science investigated on these platforms has been generated in discrete blocks totaling approximately 48 gigabytes for the Orbiter missions and 50 gigabytes for the Mir increments. Based on the anticipated volume of acceleration data resulting from continuous or nearly continuous operations, the International Space Station (ISS) presents a unique set of challenges regarding the storage of and access to microgravity acceleration environment data. This paper presents potential microgravity environment data storage, access, and analysis concepts for the ISS era.

McPherson, Kevin M.↗

Key Accomplishments & Results: Energy Data Accelerator

This fact sheet outlines the results of DOE’s Better Buildings Energy Data Accelerator. Key accomplishments of partners are outlined in this fact sheet, profiling the historic expansion of data accessibility and increases in building energy benchmarking.

data accessibility, data access, utilities, buildi↗

Accessing global data from accelerator devices

An aspect includes a table of contents (TOC) that was generated by a compiler being received at an accelerator device. The TOC includes an address of global data in a host memory space. The global data is copied from the address in the host memory space to an address in the device memory space. The address in the host memory space is obtained from the received TOC. The received TOC is updated to indicate that global data is stored at the address in the device memory space. A kernel that accesses the global data from the address in the device memory space is executed. The address in the device memory space is obtained based on contents of the updated TOC. When the executing is completed, the global data from the address in the device memory space is copied to the address in the host memory space.

97 MATHEMATICS AND COMPUTING↗

Accessing global data from accelerator devices

An aspect includes a table of contents (TOC) that was generated by a compiler being received at an accelerator device. The TOC includes an address of global data in a host memory space. The global data is copied from the address in the host memory space to an address in the device memory space. The address in the host memory space is obtained from the received TOC. The received TOC is updated to indicate that global data is stored at the address in the device memory space. A kernel that accesses the global data from the address in the device memory space is executed. The address in the device memory space is obtained based on contents of the updated TOC. When the executing is completed, the global data from the address in the device memory space is copied to the address in the host memory space.

97 MATHEMATICS AND COMPUTING↗

Improving NASA Earth Science Data and Information Access Through Natural Language Processing Based Data Analysis and Visualization

NASA: The Research Access initiative is part of the agency's framework for increasing public access to scientific publications and digital scientific data. The initiative follows the release of White House Office of Science and Technology Policy's (OSTP) memorandum "Increasing Access to the Results of Federally Funded Research," to ensure federally funded research is available to the public within one year of publication. NASA answered the mandate by creating an agency plan entitled "NASA Plan for Increasing Access to the Results of Scientific Research" and associated policy, NPD 2230.1, Research Data and Publication Access. Principles in NASA SMD Strategic Plan for Scientific Data and Computing: Continued free and open access to scientific data for any use. Improved ease of use and discoverability. Enhanced science applications and new use cases. Incorporates best practices and "state of the art" through partnerships. Earth Data and Systems are Evolving: Increasing archive and file sizes. More complicated data structures. More user-friendly and data services. What is the future direction?

Liu, Zhong↗

Efficient Unsteady Flow Visualization with High-Order Access Dependencies

We present a novel high-order access dependencies based model for efficient pathline computation in unsteady flow visualization. By taking longer access sequences into account to model more sophisticated data access patterns in particle tracing, our method greatly improves the accuracy and reliability in data access prediction. In our work, high-order access dependencies are calculated by tracing uniformly-seeded pathlines in both forward and backward directions in a preprocessing stage. The effectiveness of our proposed approach is demonstrated through a parallel particle tracing framework with high-order data prefetching. Results show that our method achieves higher data locality and hence improves the efficiency of pathline computation.

Zhang, Jiang↗

Solar Bankability Data to Advance Transactions and Access (SB-DATA) Final Technical Report

The DOE project “Solar Bankability Data to Advance Transactions and Access”, known in the solar industry and referred to in this report as “Orange Button”, had project objectives of convening key stakeholder participants in the solar project lifecycle to develop a data taxonomy that meets to the solar industry’s needs for standardized data types for use in solar projects. Additional Orange Button deliverables included the development of an online data exchange to act as a clearinghouse of data that conformed to the data standard and additional solar industry data, and a Solar Data Translation Platform that could transform legacy solar project data to conform to the Orange Button data standard. All of these objectives were achieved. The Smart Electric Power Alliance (SEPA) provided program management, industry participant recruiting and convening, worked with the solar industry to define the scope of the Orange Button data taxonomy, and documented and delivered the taxonomy data requirements to funded project partners to develop both the taxonomy and the data conversion tools. Further efforts included performing extensive industry outreach and awareness to encourage adoption of the Orange Button taxonomy by the solar industry by enumerating the cost saving benefits of using the taxonomy and the public benefit of the increased use of solar energy due to increased bankability and viability of solar development projects. Solar PV panel and component prices have fallen in the last decade, while soft costs have remained stubbornly high and can account for as much as 64% of the total cost of a new solar system. Part of the U.S. Department of Energy SunShot Initiative, Orange Button enabled the creation and adoption of industry-led open data standards for rapid and seamless information exchange across the solar value chain, from origination to decommissioning. Standardizing data will reduce soft costs by making it easier to share solar data and speeding up processes. The taxonomy is a product of solar industry participants convening to identify the pain points caused by incompatible data, and to agree on and standardize data types to be used by the actors in the solar project lifecycle. This report focuses on the SEPA led first phase of Orange Button project that included industry outreach and recruiting participants for five industry working groups, widespread industry outreach to increase awareness and encourage eventual adoption of the taxonomy, and the definition of the scope, use cases, and supporting data types of the Orange Button taxonomy. The solar industry has been receptive of the completed Orange Button taxonomy. In SEPA’s work, 401 solar industry participants facilitated by SEPA defined the scope, use cases, and requirements for the taxonomy leading to over 150 taxonomy early adopters incorporating the taxonomy into their software products serving the solar industry, and several large investors and banks have started projects to enable them to utilize the taxonomy to gain back-office efficiencies. Moving forward, as the solar industry more widely embraces the Orange Button taxonomy and begins realizing the tangible benefits of adoption at scale, we expect further pressure on solar soft costs and increases in the bankability of solar projects.

14 SOLAR ENERGY↗

Monthly Fossil-Fuel CO2 Emissions: Isomass of Emissions Gridded by One Degree Latitude by One Degree Longitude (1950 - 2007) (V. 2010)

The basic data provided in these data files are derived from time series of Global, Regional, and National Fossil-Fuel CO2 Emissions (http://cdiac.ess-dive.lbl.gov/trends/emis/overview_2013.html), the references therein, and the methodology described in Andres et al. (2011). The data accessible here take these tabular, national, mass-emissions data, multiply them by stable carbon isotopic signature (del 13C) as described in Andres et al. (2000), and distribute them spatially on a one degree latitude by one degree longitude grid. The within-country spatial distribution is achieved through a fixed population distribution as reported in Andres et al. (1996) for years prior to 1990 and a variable population distribution for later years (Andres et al. 2016). Note that the mass-emissions data used here are based on fossil-fuel consumption estimates as these are more representative of within country emissions than fossil-fuel production estimates (see http://cdiac.ess-dive.lbl.gov/faq.html#Q10 for a description why emission totals based upon consumption differ from those based upon production). The monthly, isotopic (δ 13C) fossil-fuel CO2 emissions estimates from 1950-2013 provided in this database are derived from time series of global, regional, and national fossil-fuel CO2 emissions (Boden et al. 2016), the references therein, and the methodology described in Andres et al. (2011). The data accessible here take these tabular, national, mass-emissions data, multiply them by stable carbon isotopic signatures (δ 13C) as described in Andres et al. (2000), and distribute them spatially on a one degree latitude by one degree longitude grid. The within-country spatial distribution is achieved through a fixed population distribution as reported in Andres et al. (1996). Note that the mass-emissions data used here are based on fossil-fuel consumption estimates as these are more representative of within country emissions than fossil-fuel production estimates (see http://cdiac.ess-dive.lbl.gov/faq.html#Q10 for a description why emission totals based upon consumption differ from those based upon production).

54 ENVIRONMENTAL SCIENCES↗

Closing the data life cycle: using information management in macrosystems ecology research

An important goal of macrosystems ecology research is to advance understanding of ecological systems at both fine and broad temporal and spatial scales. Our premise in this paper is that such projects require information management that is integrated into projects from their inception. Such efforts will lead to improved communication and sharing of knowledge among diverse project participants, better science outcomes, and more open science. We promote "closing the data life cycle" by publishing well-documented data sets, which allows for re-use of data to answer new and different questions from the ones conceived by the original projects. The practice of documenting and submitting data sets to publicly accessible data repositories ensures that research results and data are accessible to and useable by other researchers, thus fostering open science. Ecologists are often not familiar with the information management tools and requirements to effectively preserve data, however, and receive little institutional or professional incentive to do so. This paper describes recommended steps to these ends, and gives examples from current macrosystem ecology projects of why information management is so critical to ensuring that scientific results can be both reproduced and data shared for future use.

Ruegg, Janine↗

ESS-DIVE Unoccupied Aerial Systems (UAS) Reporting Format v1

Here we present documentation of the ESS-DIVE reporting format for Unoccupied Aerial System (UAS) data and metadata. This reporting format provides guidance to data contributors on how to store data to maximize their discoverability, facilitate their efficient reuse, and add value to individual datasets. For data users, the reporting format will better allow data repositories to optimize data search and extraction, and more readily integrate similar data into harmonized synthesis products. The reporting format provides templates and guidance for the reporting of metadata for UAS experimental campaigns, individual flights, platform and sensor description. To improve data access and discoverability, the reporting format proposes a data description scheme of Levels based on the degree of processing, where Level 0 includes raw data, through to Level 3 being derived data end products. A range of examples of data types for each Level are given, with suggested file naming schemes. The reporting format presented here is intended to form a foundation for future development that will accommodate new UAS technologies and approaches to data access and use in the future. The reporting format documentation is maintained and updated on the ESS-DIVE Community Space GitHub at https://github.com/ess-dive-community/essdive-uas. This data package is the first published version of this reporting format, and comprises a zip file of the complete content of https://github.com/ess-dive-community/essdive-uas v1.0. The zip contains the reporting format description, instructions and variable definitions in GitHub markdown language (*.md) and metadata templates in csv format. The reporting format is designed to be compatible with other ESS-DIVE formats, and it is specifically recommended that this reporting format be used in conjunction with the File-level metadata (FLMD) and comma separated values (csv) reporting formats for submission to the ESS-DIVE repository.

54 ENVIRONMENTAL SCIENCES↗

Advanced techniques and technology for efficient data storage, access, and transfer

Advanced techniques for efficiently representing most forms of data are being implemented in practical hardware and software form through the joint efforts of three NASA centers. These techniques adapt to local statistical variations to continually provide near optimum code efficiency when representing data without error. Demonstrated in several earlier space applications, these techniques are the basis of initial NASA data compression standards specifications. Since the techniques clearly apply to most NASA science data, NASA invested in the development of both hardware and software implementations for general use. This investment includes high-speed single-chip very large scale integration (VLSI) coding and decoding modules as well as machine-transferrable software routines. The hardware chips were tested in the laboratory at data rates as high as 700 Mbits/s. A coding module's definition includes a predictive preprocessing stage and a powerful adaptive coding stage. The function of the preprocessor is to optimally process incoming data into a standard form data source that the second stage can handle.The built-in preprocessor of the VLSI coder chips is ideal for high-speed sampled data applications such as imaging and high-quality audio, but additionally, the second stage adaptive coder can be used separately with any source that can be externally preprocessed into the 'standard form'. This generic functionality assures that the applicability of these techniques and their recent high-speed implementations should be equally broad outside of NASA.

Rice, Robert F.↗

The Virtual Solar Observatory: What Are We Up To Now?

In the nearly ten years of a functional Virtual Solar Observatory (VSO), http://virtualsolar.org/ we have made it possible to query and access sixty-seven distinct solar data products and several event lists from nine spacecraft and fifteen observatories or observing networks. We have used existing VSO technology, and developed new software, for a distributed network of sites caching and serving SDO HMI and/ or AlA data. We have also developed an application programming interface (API) that has enabled VSO search and data access capabilities in IDL, Python, and Java. We also have quite a bit of work yet to do, including completion of the implementation of access to SDO EVE data, and access to some nineteen other data sets from space- and ground-based observatories. In addition, we have been developing a new graphic user interface that will enable the saving of user interface and search preferences. We solicit advice from the community input prioritizing our task list, and adding to it

Gurman, J. B.↗

Exposing the Strategies that Can Reduce the Obstacles: Improving the Science User Experience

It is now well established that pursuing generic solutions to what seem are common problems in Earth science data access and use can often lead to disappointing results for both system developers and the intended users. This presentation focuses on real-world experience of managing a large and complex data system, NASAs Earth Science Data and Information Science System (EOSDIS), whose mission is to serve both broad user communities and those in smaller niche applications of Earth science data and services. In the talk, we focus on our experiences with known data user obstacles characterizing EOSDIS approaches, including various technological techniques, for engaging and bolstering, where possible, user experiences with EOSDIS. For improving how existing and prospective users discover and access NASA data from EOSDIS we introduce our cross-archive tool: Earthdata Search. This new search and order tool further empowers users to quickly access data sets using clever and intuitive features. The Worldview data visualization tool is also discussed highlighting how many users are now performing extensive data exploration without necessarily downloading data. Also, we explore our EOSDIS data discovery and access webinars, data recipes and short tutorials, targeted technical and data publications, user profiles and social media as additional tools and methods used for improving our outreach and communications to a diverse user community. These efforts have paid substantial dividends for our user communities by allowing us to target discipline specific community needs. The desired take-away from this presentation will be an improved understanding of how EOSDIS has approached, and in several instances achieved, removing or lowering the barriers to data access and use. As we look ahead to more complex Earth science missions, EOSDIS will continue to focus on our user communities, both broad and specialized, so that our overall data system can continue to serve the needs of science and applications users.

communications↗