Search NASA⌕ Search

SEARCH · Search NASA

Results for “data access”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,243 records · Page 69

A crops and soils data base for scene radiation research

Management and planning activities with respect to food production require accurate and timely information on crops and soils on a global basis. The needed information can be obtained with the aid of satellite-borne sensors, if the relations between the spectral properties and the important biological-physical parameters of crops and soils are known. In order to obtain this knowledge, the development of a crops and soils scene radiation research data base was initiated. Work related to the development of this data base is discussed, taking into account details regarding the conducted experiments, the performed measurements, the calibration of spectral data, questions of data base access, and the expansion of the crops and soils scene radiation data base for 1982.

Biehl, L. L.↗

State of the Data Union, 1992

This is the first report on the State of the Data Union (SDU) for the NASA Office of Space Science and Applications (OSSA). OSSA responsibilities include the collection, analysis, and permanent archival of data critical to space science research. The nature of how this is done by OSSA is evolving to keep pace with changes in space research. Current and planned missions have evolved to be more complex and multidisciplinary, and are generating much more data and lasting longer than earlier missions. New technologies enable global access to data, transfer of huge volumes of data, and increasingly complex analysis. The SDU provides a snapshot of this dynamic environment, identifying trends in capabilities and requirements. The current space science data environment is described and parameters which capture the pulse of key functions within that environment are presented. Continuous efforts of OSSA to improve the availability and quality of data provided to the scientific community are reported, highlighting efforts such as the Data Management Initiative.

Source record↗

Design of a Mission Data Storage and Retrieval System for NASA Dryden Flight Research Center

The Western Aeronautical Test Range (WATR) at the NASA Dryden Flight Research Center (DFRC) employs the WATR Integrated Next Generation System (WINGS) for the processing and display of aeronautical flight data. This report discusses the post-mission segment of the WINGS architecture. A team designed and implemented a system for the near- and long-term storage and distribution of mission data for flight projects at DFRC, providing the user with intelligent access to data. Discussed are the legacy system, an industry survey, system operational concept, high-level system features, and initial design efforts.

Lux, Jessica↗

Design of a Mission Data Storage and Retrieval System for NASA Dryden Flight Research Center

The Western Aeronautical Test Range (WATR) at the NASA Dryden Flight Research Center (DFRC) employs the WATR Integrated Next Generation System (WINGS) for the processing and display of aeronautical flight data. This report discusses the post-mission segment of the WINGS architecture. A team designed and implemented a system for the near- and long-term storage and distribution of mission data for flight projects at DFRC, providing the user with intelligent access to data. Discussed are the legacy system, an industry survey, system operational concept, high-level system features, and initial design efforts.

Lux, Jessica↗

A Web of Data Analytics Services

Cloud Computing has become the ubiquitous approach to our Big Data challenge. However, one will quickly discover that moving (a.k.a. forklifting) existing on-premise data analytics solutions to the Cloud doesn’t always translate to costing saving and performance boost. The Cloud’s elasticity, its availability, and its wide selection of computing options and selections of costing models making Cloud an attractive environment to tackle our Big Data challenge. The fact is Cloud, on its own, is not the silver bullet to our daunting challenge need for analyze and derive scientific inferences through vast collections of multi-sensor measurements. We would like to have all scientific data in one easy to access environment, but getting the world of scientific data in one analytic system is immensely difficult to achieve. This paper describes the data analytics web architecture NASA is developing by infusing instances of Integrated Data Analytics systems next to the data. The goal is to minimize unnecessary data movement through collection of data access and analytics webservices for researchers to interact with and analyze measurements without have to download data to their local computer. These services are RESTful and provisioned by the data centers with the help from subject matter and science experts. These services encapsulate the physical computing infrastructure, which could local computing cluster, on-premise or public Cloud environment.

Huang, Thomas↗

Review of Grey Box/Black Box Data Contamination Metrics on Open and Commercial Models

Dataset contamination is a problem where benchmarks and tasks used to evaluate the capabilities of Large Language Models (LLMs) have been incorporated into the training dataset of the models. This gives a false sense of performance that can overestimate how these models will function on truly unseen data. This problem becomes worse with commercial LLMs with larger and non-accessible training data, so techniques have been developed to try to measure the degree to which a model is contaminated with a benchmark’s data. To understand the effectiveness of these techniques, particularly when evaluating contamination on coding tasks, we review trends and categorize techniques by the degree of access to the model that is required. The research literature on this topic has reported mixed effectiveness of these techniques, so we select a set of black box (text access only) and grey box (access to model loss/probabilities required) techniques and apply them to both commercial and non-commercial models. We implement these metrics as part of a framework to test the contamination of Python code in LLMs to see to what extent we can replicate the effectiveness (or ineffectiveness) of these contamination detection techniques. Though we find mixed results in the capabilities of these metrics to identify contamination, we do observe evidence that they can identify contamination (broadly) in fine-tuned models when both a baseline and fine-tuned model is present. Additionally, similarity metrics were able to identify between contaminated and uncontaminated data even in situations where the data is distributionally similar (e.g., drawn from the same set of code projects).

97 MATHEMATICS AND COMPUTING↗

An Overview of NASA’s Airborne and Field Data Resource Center

A key recommendation from NASA’s 2022 Airborne and Field Data Workshop called for the development of a virtual Resource Center for all stakeholders across the data lifecycle of airborne and field Earth observations. The agency’s Earth Science Data and Information System (ESDIS) Project and Airborne Data Management Group (ADMG) have worked in concert to establish the newly launched NASA Airborne and Field Data Resource Center (AFDRC) to provide a single entry point for a wide assortment of information on and the effective, responsible stewardship of non-satellite observational data. The AFDRC compiles access to many existing resources, but does so in a newly organized way that integrates availability to increase efficiency and holistic understanding while simplifying users’ experience. Initially launched in fall of 2023, the AFDRC is a NASA Earthdata domain website that clarifies several previously disparate resources and provides newly updated information, including: Learning Resources: Educational resources to broaden understanding of the role airborne and field observations play in advancing understanding of our planet and NASA’s role in collecting and archiving these data. Support for data users to Find and Access Data: Advanced contextual browse/search capabilities that efficiently link researchers to data products suitable for their science objectives - this includes linking to NASA’s Catalog of Archived Suborbital Earth Science Investigations (CASEI). Working with Data: Tools specific to individual types of suborbital Earth Science data and their (inter-)disciplinary communities to provide access as well as guidance for their application. Stewardship Responsibilities: Resources for data producers with to lessen requirement burdens at the time of data transfer, and information for data stewards with consistent, authoritative guidance on best practices and agency- and/or community- specific archival procedures. This presentation will give an overview of the motivation for NASA’s AFDRC, approach for the design and content, iterative community-driven improvements, promote the use of the AFDRC, and solicit additional feedback from airborne and field data user communities.

Sara Lubkin↗

Data Management and Science at DOE BES User Facilities – Past, Present, and Future

The primary mission of user facilities operated by Basic Energy Sciences under the Department of Energy is to produce data for users in support of open science and basic research. We trace back almost 30 years of history across selected user facilities illustrating the evolution of facility data management practices and how these practices have related to performing scientific research. The facilities cover multiple techniques such as X-ray and neutron scattering, imaging and tomography sciences. Over time, detector and data acquisition technologies have dramatically increased the ability to produce prolific volumes of data challenging the traditional paradigm of users taking data home upon completion of their experiments to process and publish their results. During this time, computing capacity has also increased dramatically, though the size of the data has grown significantly faster than the capacity of one's laptop to manage and process this new facility produced data. Trends indicate that this will continue to be the case for yet some time. Thus users face a quandary for how to manage today's data complexity and size as these may exceed the computing resources users have available to themselves. This same quandary can also stifle collaboration and sharing. Realizing this, some facilities are already providing web portal access to data and computing thereby providing users access to resources they need. Portal based computing is now driving researchers to think about how to use the data collected at multiple facilities in an integrated way to perform their research, and also how to collaborate and share data. In the future, inter-facility data management systems will enable next tier cross-instrument-cross facility scientific research fuelled by smart applications residing upon user computer resources. We can learn from the medical imaging community that has been working since the early 1990's to integrate data from across multiple modalities to achieve better diagnoses - similarly, data fusion across BES facilities will lead to new scientific discoveries.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Pipelining RDP Data to the "Taxomatic" (Final Report)

This project builds on the results of previously funded research by integrating data and software that had been previously used in building resources used in the preparation of Bergey's Manual of Systematic Bacteriology, 2 nd Edition (Volumes 1 & 2A-C) and the Ribosomal Database Project-II (RDP-II) so as to both enhance the value of the data and create a pipeline approach to keeping the data current. Earlier, we demonstrated the value of using exploratory data analysis (EDA) to visualize large sets of sequence data (notably SSU rRNA gene sequences used in constructing a comprehensive phylogeny of prokaryotes. While the Self- Organizing Self-Correcting Classification (SOSCC) algorithms we developed were computationally efficient and useful for unraveling problems within the underlying data (e.g., identification of annotation errors, detection of unresolved synonymies, taxonomic and nomenclatural errors), bottlenecks at the preprocessing stage limited deployment of our applications as tools for end-users. To overcome the bottlenecks (which included hand alignment and computation of large matrices of pair-wise evolutionary distances), we proposed building a data pipeline between the "Taxomatic" application and RDP-II. The objectives were to accelerate the production and distribution of the updated versions of the prokaryotic taxonomy in lock-step with publication of new taxa and rearrangement of existing taxa, and to distribute these data more readily with the RDP-II and and other stakeholders in the community. A related goal of the current project is to deploy our visualization techniques as an interactive web application by which end-users can view manipulate, and select datasets of particular interest based upon phylogenetic and genomic information, access sequence data, and ultimately the scientific literature where the original observations were made and those that build on the original observations. The Taxomatic is a web-based tool to visualize distance matrices. The tool accepts raw distance matrices or aligned sequence information as data sources. When sequence information is provided the distance matrix is computed using the uncorrected distance model. Users can upload files to the Taxomatic website or sequences can be submitted by a SOAP service. This SOAP service is used by RDP to streamline Taxomatic use with RDP data. In addition to supplying source information, users can either supply their own taxonomic information by uploading it in XML, retrieve data taxonomic information from the RDP using either RDP or Genbank identifiers as source data, with or without classification by the RDP Classifier web service, or completely omit taxonomic data. In the latter case, the input distance matrix can be viewed in the order in which it was loaded.

59 BASIC BIOLOGICAL SCIENCES↗

Data Management and Its Role in Delivering Science at DOE BES User Facilities – Past, Present, and Future

Abstract. The primary mission of user facilities operated by Basic Energy Sciences under the Department of Energy is to produce data for users in support of open science and basic research [1]. We trace back almost 30 years of history across selected user facilities illustrating the evolution of facility data management practices and how these practices have related to performing scientific research. The facilities cover multiple techniques such as X-ray and neutron scattering, imaging and tomography sciences. Over time, detector and data acquisition technologies have dramatically increased the ability to produce prolific volumes of data challenging the traditional paradigm of users taking data home upon completion of their experiments to process and publish their results. During this time, computing capacity has also increased dramatically, though the size of the data has grown significantly faster than the capacity of one’s laptop to manage and process this new facility produced data. Trends indicate that this will continue to be the case for yet some time. Thus users face a quandary for how to manage today’s data complexity and size as these may exceed the computing resources users have available to themselves. This same quandary can also stifle collaboration and sharing. Realizing this, some facilities are already providing web portal access to data and computing thereby providing users access to resources they need [2]. Portal based computing is now driving researchers to think about how to use the data collected at multiple facilities in an integrated way to perform their research, and also how to collaborate and share data. In the future, inter-facility data management systems will enable next tier cross-instrument-cross facility scientific research fuelled by smart applications residing upon user computer resources. We can learn from the medical imaging community that has been working since the early 1990’s to integrate data from across multiple modalities to achieve better diagnoses [3] – similarly, data fusion across BES facilities will lead to new scientific discoveries.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Introduction to the special issue on smart transportation

Transportation is getting smarter and smarter, with the prominence of connected automated vehicle technologies in the global auto industry’s near-term growth strategies, of big data analytics and unprecedented access to sensing data of mobility, and of integration of this analytics into the optimization of mobility and transport. Further, these developments are setting off a wave of smart transportation innovations, which are featured by new methods and applications driven by various forms of sensor data such as GPS, CAN bus, LIDA, images, etc. At the same time, complexities surrounding the use, conflation, and processing of disparate data in near real-time is of essence for the design and development of futuristic smart transportation.

33 ADVANCED PROPULSION SYSTEMS↗

A new CDF model for data movement based on SRM

Being a large international collaboration established well before the full development of the Grid as the main computing tool for High Energy Physics, CDF has recently changed and improved its computing model, decentralizing some parts of it in order to be able to exploit the rising number of distributed resources available nowadays. Despite those efforts, while the large majority of CDF Monte Carlo production has moved to the Grid, data processing is still mainly performed in dedicated farms hosted at FNAL, requiring a centralized management of data and Monte Carlo samples needed for physics analysis. This rises the question on how to manage the transfer of produced Monte Carlo samples from remote Grid sites to FNAL in an efficient way; up to now CDF has relied on a non scalable centralized solution based on dedicated data servers accessed through rcp protocol, which has proven to be unsatisfactory. A new data transfer model has been designed that uses SRMs as local caches for remote Monte Carlo production sites, interfaces them with SAM, the experiment data catalog, and finally realizes the file movement exploiting the features provided by the data catalog transfer layer. We describe here the model and its integration within the current CDF computing architecture.

99 GENERAL AND MISCELLANEOUS↗

Enhancing the Impact of Science Data: Toward Data Discovery and Reuse

The amount of data produced in support of scientific research continues to grow rapidly. Despite the accumulation and demand for scientific data, relatively little data are actually made available for the broader scientific community. We surmise that one root of this problem is the perceived difficulty of electronically publishing scientific data and associated metadata in a way that makes it discoverable. We propose exploiting Semantic Web technologies and best practices to make metadata both discoverable and easy to publish. We share experiences in curating metadata to illustrate the cumbersome nature of data reuse in the current research environment. We also make recommendations with a real-world example of how data publishers can provide their metadata by adding limited additional markup to HTML pages on the Web. With little additional effort from data publishers, the difficulty of data discovery, access, and sharing can be greatly reduced and the impact of research data greatly enhanced.

Data Curation↗

Effective Tooling for Linked Data Publishing in Scientific Research

Challenges that make it difficult to find, share, and combine published data, such as data heterogeneity and resource discovery, have led to increased adoption of semantic data standards and data publishing technologies. To make data more accessible, interconnected and discoverable, some domains are being encouraged to publish their data as Linked Data. Consequently, this trend greatly increases the amount of data that semantic web tools are required to process, store, and interconnect. In attempting to process and manipulate large data sets, tools–ranging from simple text editors to modern triplestores– eventually breakdown upon reaching undefined thresholds. This paper offers a systematic approach that data publishers can use to categorize suitable tools to meet their data publishing needs. We present a real-world use case, the Resource Discovery for Extreme Scale Collaboration (RDESC), which features a scientific dataset(maximum size of 1.4 billion triples) used to evaluate a toolbox for data publishing in climate research. This paper also introduces a semantic data publishing software suite developed for the RDESC project.

Purohit, Sumit↗

Developer Network

NREL's Developer Network, developer.nrel.gov, provides data that users can access to provide data to their own analyses, mobile and web applications. Developers can retrieve the data through a Web services API (application programming interface). The Developer Network handles overhead of serving up web services such as key management, authentication, analytics, reporting, documentation standards, and throttling in a common architecture, while allowing web services and APIs to be maintained and managed independently.

Source record↗

MyEMSL

MyEMSL provides the following features: user-assisted upload of data from scientific instruments to an archive, tracking data though the upload process, providing metadata for recovery of uploaded data, providing users with access to the data. These features are supplied by a mixture of open source software and internally developed software. MyESML also provides a notification system that allows the system to communicate with users and other parts of the MyEMSL system. This allows the system to support automated workflows and user interaction. Reprocessing of already ingested data is also supported.

Glass, Kevin↗

zorder-lib: Library API for Z-Order Memory Layout

This document describes the motivation for, elements of, and use of the zorder-lib, a library API that implements organization of and access to data in memory using either a-order (also known as "row-major" order) or z-order memory layouts. The primary motivation for this work is to improve the performance of many types of data- intensive codes by increasing both spatial and temporal locality of memory accesses. The basic idea is that the cost associated with accessing a datum is less when it is nearby in either space or time.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗