Search NASA⌕ Search

SEARCH · Search NASA

Results for “data provenance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2024

Analyzing EOSDIS Dataset Research Outputs using Knowledge Graphs and Large Language Models

Datasets, unlike publications, can be updated over time, with each new version receiving a DOI but not always being linked to previous ones. This complicates tracking citations across a dataset’s lifecycle. We address this by integrating dataset versions and citations into a knowledge graph (KG), which helps trace dataset citations and analyze dataset usage in applied research. To categorize publications from various journals, we fine-tuned NASA IMPACT INDUS Large Language Model (LLM) on a labeled publication set, assigning publications to one of twenty applied research areas. By linking datasets to these research areas, we improved dataset searchability and discovery through these domains.

open-source↗
NASA NTRSDate not supplied

Discovering Research Areas in Dataset Applications Through Knowledge Graphs and Large Language Models

Scientific datasets are increasingly cited in peer-reviewed journal publications, facilitating easy access to research utilizing those datasets. Datasets undergo a life cycle where older versions of datasets are replaced by newer versions often due to improvements in data resolution, algorithms, and other factors. Unlike peer reviewed documents registered with a single Digital Unique Identifier (DOI), datasets can be updated over time and the newer version of the datasets are registered with a new DOI which is not necessarily linked to the previous version of the dataset. It is challenging when publications citing a dataset need to be traced over the entire life cycle of that dataset. We provide an innovative approach to link the dataset versions and publications using a knowledge graph (KG). KG can help to trace the dataset cited in publications over the entire dataset life cycle and shed light into dataset usage in various applied research areas. We fine-tuned the pretrained NASA IMPACTINDUS Large Language Model (LLM) on a set of labeled publications abstracts. Our results showed that 87% of the publications were classified into one of twenty applied research areas, while the remaining 13% were classified into non-applied research areas. By linking datasets to applied research areas through the KG and employing Global Change Master Directory(GCMD), a well-established controlled vocabulary of scientific keywords describing Earth science datasets, we contribute to a transparent and advanced search and discovery mechanism for datasets across the Earth data ecosystem. The integrated KG and LLM approach is now incorporated and operational in dataset publication management at one of NASA’s Earth science data archival centers.

data provenance↗
NASA NTRSDate not supplied

Enhancing Dataset Discovery and Usage Tracking in Earth Sciences: Integrating Knowledge Graphs and Large Language Models

NASA's Data Active Archive Centers (DAACs) have played a crucial role in supporting a wide range of applied research in Earth and Environmental sciences. To date, over 20,000 publications have been collected, citing more than 3,000 NASA Earth science datasets. We present an innovative approach that links datasets and collected publications through a knowledge graph (KG). This KG enables the tracking of dataset citations throughout the dataset's lifecycle, revealing patterns of dataset usage across various applied research areas. We fine-tuned the pre-trained NASA IMPACT INDUS-Base Retriever Large Language Model (LLM) using a set of labeled publication abstracts. Our results indicate that 87% of the publications were classified into one of twenty applied research areas, while the remaining 13% were categorized into non-applied research areas. The classified publications linked to datasets are used to discover datasets by users interested in specific applied research and by dataset providers to determine dataset usage for applications.

open-source↗
NASA NTRSDate not supplied

Expanding Biological Repository Data Available for Sharing and Knowledge Discovery

Biology has developed next-generation data science and alternative analytical approaches with methodologies which require principal investigator (PI) experimental assay data be re-used. This new approach involves mining multiple datasets at once from various hierarchical organizations of biological complexity, while concurrently evaluating how experimental factors affect endpoints of standard assays. The purpose of the NASA Ames Life Sciences Data Archive (ALSDA) is to collect, curate, and make findable, accessible, interoperable, and reusable (FAIR) all non-human space-relevant biological data. These data include mission metadata, subject metadata, assay metadata (parameters), raw and processed assay data, assay imagery, and subject-experienced telemetry (radiation, temperature, humidity, acoustics, vibrations). ALSDA has transformed to bring current biological repository data and all future collected data into this new scientific data mining reality. It has integrated into the ‘NASA Open Science’ group of projects to facilitate a suite of new tools and workflows to improve data accessibility and reusability by implementing data management plans, automating data submission agreements, and adopting the single-point-of-entry data submission portal, originally developed by NASA GeneLab. These systems required ALSDA to develop science assay configurations for the submission portal, capturing essential assay parameters according to established norms in each sub-field within biology. The submission portal expedites data collection by enhancing ease of PI data submission, providing a user interface and specificity for which data is to be submitted. ALSDA datasets are curated to maintain rich metadata, accuracy of datasets, data transparency, provenance, and additionally ensure data are machine-readable (e.g., R and Python languages). ALSDA integration with GeneLab and its analysis portals enable higher-order physiological-level datasets be mined in conjunction with -omics datasets. As ALSDA physiological-level datasets are published (micro-computed tomography, histology, intraocular pressure, hormonal assays, immunostaining, ultrasonography), the merging of hierarchical organizations of biological complexity from spaceflight will enable new knowledge discovery approaches.

Ryan T Scott↗
NASA NTRS2014

Formal Provenance Representation of the Data and Information Supporting the National Climate Assessment

The Global Change Information System (GCIS) provides a framework for the formal representation of structured metadata about data and information about global change. The pilot deployment of the system supports the National Climate Assessment (NCA), a major report of the U.S. Global Change Research Program (USGCRP). A consumer of that report can use the system to browse and explore that supporting information. Additionally, capturing that information into a structured data model and presenting it in standard formats through well defined open inter- faces, including query interfaces suitable for data mining and linking with other databases, the information becomes valuable for other analytic uses as well.

Provenance↗
NASA NTRS2004

Application of Bayesian Classification to Content-Based Data Management

The high volume of Earth Observing System data has proven to be challenging to manage for data centers and users alike. At the Goddard Earth Sciences Distributed Active Archive Center (GES DAAC), about 1 TB of new data are archived each day. Distribution to users is also about 1 TB/day. A substantial portion of this distribution is MODIS calibrated radiance data, which has a wide variety of uses. However, much of the data is not useful for a particular user's needs: for example, ocean color users typically need oceanic pixels that are free of cloud and sun-glint. The GES DAAC is using a simple Bayesian classification scheme to rapidly classify each pixel in the scene in order to support several experimental content-based data services for near-real-time MODIS calibrated radiance products (from Direct Readout stations). Content-based subsetting would allow distribution of, say, only clear pixels to the user if desired. Content-based subscriptions would distribute data to users only when they fit the user's usability criteria in their area of interest within the scene. Content-based cache management would retain more useful data on disk for easy online access. The classification may even be exploited in an automated quality assessment of the geolocation product. Though initially to be demonstrated at the GES DAAC, these techniques have applicability in other resource-limited environments, such as spaceborne data systems.

Lynnes, Christopher↗
NASA NTRS2011

Addressing and Presenting Quality of Satellite Data via Web-Based Services

With the recent attention to climate change and proliferation of remote-sensing data utilization, climate model and various environmental monitoring and protection applications have begun to increasingly rely on satellite measurements. Research application users seek good quality satellite data, with uncertainties and biases provided for each data point. However, different communities address remote sensing quality issues rather inconsistently and differently. We describe our attempt to systematically characterize, capture, and provision quality and uncertainty information as it applies to the NASA MODIS Aerosol Optical Depth data product. In particular, we note the semantic differences in quality/bias/uncertainty at the pixel, granule, product, and record levels. We outline various factors contributing to uncertainty or error budget; errors. Web-based science analysis and processing tools allow users to access, analyze, and generate visualizations of data while alleviating users from having directly managing complex data processing operations. These tools provide value by streamlining the data analysis process, but usually shield users from details of the data processing steps, algorithm assumptions, caveats, etc. Correct interpretation of the final analysis requires user understanding of how data has been generated and processed and what potential biases, anomalies, or errors may have been introduced. By providing services that leverage data lineage provenance and domain-expertise, expert systems can be built to aid the user in understanding data sources, processing, and the suitability for use of products generated by the tools. We describe our experiences developing a semantic, provenance-aware, expert-knowledge advisory system applied to NASA Giovanni web-based Earth science data analysis tool as part of the ESTO AIST-funded Multi-sensor Data Synergy Advisor project.

Leptoukh, Gregory↗
NASA NTRS2007

Designing Rolling-Element Bearings

Bearing Analysis Tool (BAT) is a computer program for designing rolling-element bearings for cryogenic turbomachines. BAT provides a graphical user interface (GUI) that guides the entry of data to develop mathematical models of bearings. The GUI breaks model data into logical subsets that are entered through logic-driven input screens. The software generates a threedimensional graphical model of a bearing as the data are entered. Most dataentry errors become immediately obvious in the graphical model. BAT provides for storage of all the data on a shaft/bearing system, enabling the creation of a library of proven designs. Data from the library can be transferred to subsequent projects by use of simple cut-and-paste routines. BAT includes a library of temperature- dependent cryogenic bearing-material properties for use in the mathematical models. BAT implements algorithms that (1) enable the user to select combinations of design and/or operating-condition parameters, and then (2) automatically optimize the design by performing trade studies over all of the parameter combinations. This feature enables optimization over a large trade space in a fraction of the time taken when using prior bearingmodel software.

Moore, James D., Jr.↗
NASA NTRS2013

Shuttle Topography Data Inform Solar Power Analysis

The next time you flip on a light switch, there s a chance that you could be benefitting from data originally acquired during the Space Shuttle Program. An effort spearheaded by Jet Propulsion Laboratory (JPL) and the National Geospatial-Intelligence Agency (NGA) in 2000 put together the first near-global elevation map of the Earth ever assembled, which has found use in everything from 3D terrain maps to models that inform solar power production. For the project, called the Shuttle Radar Topography Mission (SRTM), engineers at JPL designed a 60-meter mast that was fitted onto Shuttle Endeavour. Once deployed in space, an antenna attached to the end of the mast worked in combination with another antenna on the shuttle to simultaneously collect data from two perspectives. Just as having two eyes makes depth perception possible, the SRTM data sets could be combined to form an accurate picture of the Earth s surface elevations, the first hight-detail, near-global elevation map ever assembled. What made SRTM unique was not just its surface mapping capabilities but the completeness of the data it acquired. Over the course of 11 days, the shuttle orbited the Earth nearly 180 times, covering everything between the 60deg north and 54deg south latitudes, or roughly 80 percent of the world s total landmass. Of that targeted land area, 95 percent was mapped at least twice, and 24 percent was mapped at least four times. Following several years of processing, NASA released the data to the public in partnership with NGA. Robert Crippen, a member of the SRTM science team, says that the data have proven useful in a variety of fields. "Satellites have produced vast amounts of remote sensing data, which over the years have been mostly two-dimensional. But the Earth s surface is three-dimensional. Detailed topographic data give us the means to visualize and analyze remote sensing data in their natural three-dimensional structure, facilitating a greater understanding of the features and processes taking place on Earth."

Source record↗
NASA NTRS2024

Bridging the Gap: Enhancing Prominence and Provenance of NASA Datasets in Research Publications

Attribution of datasets that were used to generate research results described in peer-reviewed publications to the original source of these datasets (which are often archived at NASA Earth Science data centers) has been very challenging. Even though the data citation standard of citing datasets as research artifacts and citing them with Digital Object Identifiers (DOIs) was introduced over a decade ago, most authors do not properly reference the data used in their studies and merely mention them in the text. The lack of proper citations of datasets makes the peer-reviewed publication less transparent, imperils reproducibility, and impedes open science. We offer an open-source publication management methodology and a tool that can help to enhance usage-based data discovery, prominence, and provenance of the data; reproducibility of the research results; and potentially increase the return on investment on NASA-funded research.

open-source↗
NASA NTRS2014

Analyzing a 35-Year Hourly Data Record: Why So Difficult?

At the Goddard Distributed Active Archive Center, we have recently added a 35-Year record of output data from the North American Land Assimilation System (NLDAS) to the Giovanni web-based analysis and visualization tool. Giovanni (Geospatial Interactive Online Visualization ANd aNalysis Infrastructure) offers a variety of data summarization and visualization to users that operate at the data center, obviating the need for users to download and read the data themselves for exploratory data analysis. However, the NLDAS data has proven surprisingly resistant to application of the summarization algorithms. Algorithms that were perfectly happy analyzing 15 years of daily satellite data encountered limitations both at the algorithm and system level for 35 years of hourly data. Failures arose, sometimes unexpectedly, from command line overflows, memory overflows, internal buffer overflows, and time-outs, among others. These serve as an early warning sign for the problems likely to be encountered by the general user community as they try to scale up to Big Data analytics. Indeed, it is likely that more users will seek to perform remote web-based analysis precisely to avoid the issues, or the need to reprogram around them. We will discuss approaches to mitigating the limitations and the implications for data systems serving the user communities that try to scale up their current techniques to analyze Big Data.

computational performance↗
NASA NTRS2019

Reusability in NASA's Earth Observation System Data and Information System (EOSDIS)

NASA's Earth Observation System Data and Information System (EOSDIS) has been operational since 1994. The term FAIR is relatively recent compared to the operational life of EOSDIS. However, given the evolutionary nature of EOSDIS in response to both the state of the art and community expectations, it is useful to assess where EOSDIS stands relative to the FAIR data management. In this presentation we evaluate EOSDIS against the Reusability principle, which calls for a clear and accessible data usage license, rich description of data, detailed provenance, and compliance with domain-relevant community standards.Achieving consistent compliance for datasets is particularly challenging with the diverse community of missions and scientists providing data to EOSDIS. To address this, EOSDIS's constituent Distributed Active Archive Centers work with data producers to utilize the applicable standards and conventions. In addition, a guidebook is under development to provide comprehensive and understandable guidance on how to construct compliant, and thus more reusable, data products.

Hampapuram (Rama) Ramapriyan↗
NASA NTRS2010

Use of Remote Sensing Products for the SERVIR Project

The United Nations University (UNU) estimates that floods presently impacts greater than 520 million people per year worldwide, resulting in up to 25,000 annual deaths, extensive homelessness, disaster-induced disease, crop and livestock damage, famine, and other serious harm. Meanwhile, aid agencies such as the International Federation of Red Cross and Red Crescent Societies (IFRC) are increasingly seeking better information concerning flood hazards in order to plan for and help mitigate the effects of damaging floods. There is fertile ground to continue development of better remote sensing and modeling techniques to help manage flood related disasters. Disaster management and humanitarian aid organizations need accurate and timely information for making decisions regarding deployment of relief teams and emergency supplies during major floods. Flood maps based on the use of satellite data have proven extremely valuable to such organizations for identifying the location, extent, and severity of these events. However, despite extraordinary efforts on the part of remote sensing data providers to rapidly deliver such maps, there is typically a delay of several days or even weeks from the on-set of flooding until such maps are available to the disaster management community. This paper summarizes efforts at NASA to address this problem through development of an integrated and automated process of a) flood forecasting b) flood detection, c) satellite data acquisition, d) rapid flood mapping and distribution, and e) validation of flood forecasting and detection products.

Policelli, Frederick S.↗
NASA NTRS2010

Application of Satellite Observations to Manage Natural Disasters in the Lake Victoria Basin

Lake Victoria, the second largest fresh water lake in the Eastern part of Africa is a vital natural resource for the economic well being and prosperity of over 30 million people located in riparian regions of Uganda, Kenya and Tanzania. It covers a large area of about 68,870 km2 and produces a GDP of about US $30 billion per year. The region is also very much prone to natural disasters such as severe floods during heavy precipitation periods in the Eastern part of Africa. In addition to floods, the precipitation also produces large infestations of mosquito larvae due to the standing water in many areas. This further causes multiple vector borne diseases such as Malaria, Rift Valley Fever and more. These problems are of serious concern and require active and aggressive surveillance and management to minimize the loss of human and animal lives and property damage. Satellite imagery and observations along with the in situ measurements provide a great tool to analyze and study this area and inform the policy makers to make calculated policy decisions which are more beneficial to the environment. Recently, NASA and USAID have joined forces with the Regional Center for Mapping of Resources for Development (RCMRD) located in Nairobi, Kenya to utilize multiple NASA sensors such as TRMM, SRTM and MODIS to develop flood potential maps for the Lake Victoria Basin. The idea is to generate a flood forecasts and "nowcasts" that can be sent to the disaster management organizations of Uganda, Kenya, and Tanzania. Post flood event satellite imagery is becoming a common tool to assess the areas inundated by flooding. However, this work is unique undertaking by utilizing land imaging and atmospheric satellites to build credible flood potential maps. At same time, we are also studying the potential occurrence and spread of Rift Valley Fever disease based on the short term climate records and precipitation data. These activities require multi-nation coordination and agreements and multiple operational agencies within each respective country. It also requires credible in situ data such as precipitation, river flow rates and lake levels to further validate the global and regional Hood models and algorithms. This also requires a considerable amount of training and capacity building for the RCMRD experts who will help us validate the model results and eventually transition it for operational use. In a final analysis, Disaster management and humanitarian aid organizations need accurate and timely information for making decisions regarding deployment of relief teams and emergency supplies during major floods. Flood maps based on the use of satellite data have proven extremely valuable to such organizations for identifying the location, extent, and severity of these events. However, despite extraordinary efforts on the part of remote sensing data providers to rapidly deliver such maps, there is typically a delay of several days or even weeks from the on-set of flooding until such maps are available to the disaster management community. This paper summarizes efforts at NASA to address this problem through development of an integrated and automated process of a) flood forecasting b) flood detection, c) satellite data acquisition, d) rapid Hood mapping and distribution, and e) validation of Hood forecasting and detection products.

Habib, Shahid↗
NASA NTRS2012

From EXOSAT to the High Energy Astrophysics Science Archive (HEASARC): X-ray Astronomy Comes of Age

In May 1983 the European Space Agency launched EXOSAT, its first X-ray astronomy observatory. Even though it lasted only 3 short years, this mission brought not only new capabilities that resulted in unexpected discoveries, but also a pioneering approach to operations and archiving that changed X-ray astronomy from observations led by small instrument teams, to an observatory approach open to the entire community through a guest observer program. The community use of the observatory was supported by a small dedicated team of scientists, the precursor to the data center activities created to support e.g. Chandra and XMM-Newton. The new science capabilities of EX OS AT included a 90 hr highly eccentric high earth orbit that allow unprecedented continuous coverage of sources as well as direct communication with the satellite that allowed real time decisions to respond to unexpected events through targets of opportunity. The advantages of this orbit demonstrated by EXOSAT resulted in Chandra and XMM-Newton selecting similar orbits. The three instruments on board the EXOSAT observatory were complementary, designed to give complete coverage over a wide energy band pass of 0.05-50 keY. An onboard processor could be programmed to give multiple data modes that could be optimized in response to science discoveries: These new capabilities resulted in many new discoveries including the first comprehensive study of AGN variability, new orbital periods in X-ray binaries and cataclysmic variables, new black holes, quasi-periodic oscillations from neutron stars and black holes and broad band X-ray spectroscopy. The EXOSAT team generated a well-organized database accessible worldwide over the nascent internet, allowing remote selection of data products, making samples and undertaking surveys from the data. The HEASARC was established by NASA at Goddard Space Flight Center in 1990 as the repository of NASA X-ray and Gamma-ray data. The proven EXOSAT database system became the core of the HEASARC infrastructure. The HEASARC pioneered many concepts now taken for granted including standardized formats using FITS files, restoring data from earlier missions, multi-mission analysis tools and a searchable archive over the world wide web.

White, Nicholas E.↗
DOE OSTI2024

Improving the estimate of higher-order moments from lidar observations near the top of the convective boundary layer

Abstract. Ground-based lidar data have proven extremely useful for profiling the convective boundary layer (CBL). Many groups have derived higher-order moments (e.g., variance, skewness, fluxes) from high-temporal-resolution lidar data using an autocovariance approach. However, these analyses are highly uncertain near the CBL top when the depth of the CBL (zi) is changing during the analysis period. This is because the autocovariance approach is usually applied to constant height levels and the character of the eddies is changing on either side of the changing CBL top. Here, a new approach is presented wherein the autocovariance analysis is performed on a normalized height grid, with a temporally smoothed zi. Output from a large eddy simulation model demonstrates that deriving higher-order moments from time series on a normalized height grid has better agreement with the slab-averaged quantities than the moments derived from the original height grid.

Rosenberger, Tessa E. (ORCID:0000000333205873)↗
NASA NTRSDate not supplied

NASA’s Data Preservation Strategies for Ensuring Ongoing Use and Access: Leveraging the Capabilities at the NASA DAACs

As NASA prepares for the follow on to EOS through the deployment of many new space and airborne-based assets envisioned under the ESO initiative, the need for ensuring user access to the very large and diverse archive of past missions remains a critical endeavor for current and future research. The ESDIS team has been working for many years on developing strategies and implementations that will make the linkage between the data held in our archives and the emerging missions possible. The focus on how best to preserve these past data has proven to be a collaborative effort, and one that may well benefit from the adoption of innovative technical tools that enable organizational efficiencies. This poster reveals both aspects of NASA’s EOS data repository strategy by discussing the collaborative community efforts of development of ISO 19165-2 – an international standard supported by a team from many countries around the world, the use by EOSDIS DAACs of a Preservation Content Specification, and ongoing work with mission teams to ensure mission data and all associated information are preserved and available for current and future use.

Francis Lindsay↗
NASA NTRS2009

Approach to Managing MeaSURES Data at the GSFC Earth Science Data and Information Services Center (GES DISC)

A major need stated by the NASA Earth science research strategy is to develop long-term, consistent, and calibrated data and products that are valid across multiple missions and satellite sensors. (NASA Solicitation for Making Earth System data records for Use in Research Environments (MEaSUREs) 2006-2010) Selected projects create long term records of a given parameter, called Earth Science Data Records (ESDRs), based on mature algorithms that bring together continuous multi-sensor data. ESDRs, associated algorithms, vetted by the appropriate community, are archived at a NASA affiliated data center for archive, stewardship, and distribution. See http://measures-projects.gsfc.nasa.gov/ for more details. This presentation describes the NASA GSFC Earth Science Data and Information Services Center (GES DISC) approach to managing the MEaSUREs ESDR datasets assigned to GES DISC. (Energy/water cycle related and atmospheric composition ESDRs) GES DISC will utilize its experience to integrate existing and proven reusable data management components to accommodate the new ESDRs. Components include a data archive system (S4PA), a data discovery and access system (Mirador), and various web services for data access. In addition, if determined to be useful to the user community, the Giovanni data exploration tool will be made available to ESDRs. The GES DISC data integration methodology to be used for the MEaSUREs datasets is presented. The goals of this presentation are to share an approach to ESDR integration, and initiate discussions amongst the data centers, data managers and data providers for the purpose of gaining efficiencies in data management for MEaSUREs projects.

Vollmer, Bruce↗