Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Reuse”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS1992

Compiler analysis for irregular problems in FORTRAN D

We developed a dataflow framework which provides a basis for rigorously defining strategies to make use of runtime preprocessing methods for distributed memory multiprocessors. In many programs, several loops access the same off-processor memory locations. Our runtime support gives us a mechanism for tracking and reusing copies of off-processor data. A key aspect of our compiler analysis strategy is to determine when it is safe to reuse copies of off-processor data. Another crucial function of the compiler analysis is to identify situations which allow runtime preprocessing overheads to be amortized. This dataflow analysis will make it possible to effectively use the results of interprocedural analysis in our efforts to reduce interprocessor communication and the need for runtime preprocessing.

Vonhanxleden, Reinhard↗
NASA NTRS2004

A Simple, Scalable, Script-based Science Processor

The production of Earth Science data from orbiting spacecraft is an activity that takes place 24 hours a day, 7 days a week. At the Goddard Earth Sciences Distributed Active Archive Center (GES DAAC), this results in as many as 16,000 program executions each day, far too many to be run by human operators. In fact, when the Moderate Resolution Imaging Spectroradiometer (MODIS) was launched aboard the Terra spacecraft in 1999, the automated commercial system for running science processing was able to manage no more than 4,000 executions per day. Consequently, the GES DAAC developed a lightweight system based on the popular Per1 scripting language, named the Simple, Scalable, Script-based Science Processor (S4P). S4P automates science processing, allowing operators to focus on the rare problems occurring from anomalies in data or algorithms. S4P has been reused in several systems ranging from routine processing of MODIS data to data mining and is publicly available from NASA.

Lynnes, Christopher↗
NASA NTRS2006

Efficient Agent-Based Cluster Ensembles

Numerous domains ranging from distributed data acquisition to knowledge reuse need to solve the cluster ensemble problem of combining multiple clusterings into a single unified clustering. Unfortunately current non-agent-based cluster combining methods do not work in a distributed environment, are not robust to corrupted clusterings and require centralized access to all original clusterings. Overcoming these issues will allow cluster ensembles to be used in fundamentally distributed and failure-prone domains such as data acquisition from satellite constellations, in addition to domains demanding confidentiality such as combining clusterings of user profiles. This paper proposes an efficient, distributed, agent-based clustering ensemble method that addresses these issues. In this approach each agent is assigned a small subset of the data and votes on which final cluster its data points should belong to. The final clustering is then evaluated by a global utility, computed in a distributed way. This clustering is also evaluated using an agent-specific utility that is shown to be easier for the agents to maximize. Results show that agents using the agent-specific utility can achieve better performance than traditional non-agent based methods and are effective even when up to 50% of the agents fail.

Agogino, Adrian↗
DOE OSTI2024

Wastewater reuse benefits for municipal complete retention lagoons: Life cycle assessment and dynamic modeling

Complete retention lagoons with wastewater reuse for agricultural purposes may offer sustainability advantages over alternative systems for small communities in semiarid regions. This study quantifies the environmental life cycle impact of adopting agriculture water reuse systems using case study data to estimate operating and building infrastructure impacts and spatial–temporal modeling to quantify resource trade-offs. Water reuse system benefits are highly dependent on supply–storage–demand dynamics. The relative size of irrigated agricultural land to the lagoon size was the most significant factor influencing site water application rates. The benefits are sensitive to changes in air emissions occurring from the agricultural land and further emphasize the importance of proper fertilizer management when adopting water reuse systems. Wastewater reuse from complete retention lagoons reduce life cycle GHG emissions, primarily through excavation reductions, offset fertilizer use, and especially from increased crop yields from wastewater reuse at previously rainfed sites.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Boosted decision tree reweighting of simulated neutrino interactions for O ( 1 ) GeV neutrino cross-section measurements

This paper illustrates a generic method for multidimensional reweighting of O ( 1 ) GeV neutrino interaction Monte Carlo samples. The reweighting is based on a boosted decision tree algorithm trained on high-dimensional space in detector final-state observables. This enables one generator’s events to be reweighted so that its reconstructed particle content and kinematics distributions, as well as detector efficiency, match those of a target model. The approach establishes an efficient way to reuse legacy Monte Carlo data, avoiding regeneration. As an example, we test its use in a measurement of transverse kinematic imbalance of the μ - and proton in charged-current quasielastic like ν μ events from the MINERvA experiment.

Lin, Z. [Rochester U.] (ORCID:0009000188903698)↗
DOE OSTI2022

Large Scale Caching and Streaming of Training Data for Online Deep Learning

The training of deep neural network models on large data remains a difficult problem, despite progress towards scalable techniques. In particular, there is a mismatch between the random but predetermined order in which AI flows select training samples and the streaming I/O patterns for which traditional HPC data storage (e.g., parallel file systems) are designed. In addition, as more data are obtained, it is feasible neither simply to train learning models incrementally, due to catastrophic forgetting (i.e., bias towards new samples), nor to train frequently from scratch, due to prohibitive time and/or resource constraints. In this paper, we study data management techniques that combine caching and streaming with rehearsal support in order to enable efficient access to training samples in both offline training and continual learning. We revisit state-of-art streaming approaches based on data pipelines that transparently handle prefetching, caching, shuffling, and data augmentation, and discuss the challenges and opportunities that arise when combining these methods with data-parallel training techniques. We also report on preliminary experiments that evaluate the I/O overheads involved in accessing the training samples from a parallel file system (PFS) under several concurrency scenarios, highlighting the impact of the PFS on the design of the data pipelines.

data pipelines↗
DOE OSTI2023

ESS-DIVE Unoccupied Aerial Systems (UAS) Reporting Format v1

Here we present documentation of the ESS-DIVE reporting format for Unoccupied Aerial System (UAS) data and metadata. This reporting format provides guidance to data contributors on how to store data to maximize their discoverability, facilitate their efficient reuse, and add value to individual datasets. For data users, the reporting format will better allow data repositories to optimize data search and extraction, and more readily integrate similar data into harmonized synthesis products. The reporting format provides templates and guidance for the reporting of metadata for UAS experimental campaigns, individual flights, platform and sensor description. To improve data access and discoverability, the reporting format proposes a data description scheme of Levels based on the degree of processing, where Level 0 includes raw data, through to Level 3 being derived data end products. A range of examples of data types for each Level are given, with suggested file naming schemes. The reporting format presented here is intended to form a foundation for future development that will accommodate new UAS technologies and approaches to data access and use in the future. The reporting format documentation is maintained and updated on the ESS-DIVE Community Space GitHub at https://github.com/ess-dive-community/essdive-uas. This data package is the first published version of this reporting format, and comprises a zip file of the complete content of https://github.com/ess-dive-community/essdive-uas v1.0. The zip contains the reporting format description, instructions and variable definitions in GitHub markdown language (*.md) and metadata templates in csv format. The reporting format is designed to be compatible with other ESS-DIVE formats, and it is specifically recommended that this reporting format be used in conjunction with the File-level metadata (FLMD) and comma separated values (csv) reporting formats for submission to the ESS-DIVE repository.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Gross and Net Soil Methane Flux and Ancillary Data, Edgewater, MD, USA, summer 2022

This data package contains measurements used to quantify methane cycling and environmental conditions in coastal forest soils during the 2022 growing season. It includes time‑series data of soil methane flux, soil respiration, soil temperature, and volumetric water content collected from soil monoliths transplanted along an inundation and salinity gradient. The package also provides one‑time measurements from a stable‑isotope pool‑dilution incubation, including gravimetric water content, methane headspace concentrations, and ¹³CH₄ enrichment over time. Data files are provided in comma‑separated values (CSV) format, with accompanying metadata and readme documentation in PDF and plain‑text formats. All files can be opened with standard software such as R, Python, or spreadsheet programs capable of handling CSV files. The metadata file describes variable definitions, units, processing steps, and the structure of each data table to support reuse and integration with other datasets.

13-C↗
NASA NTRS2005

An Ecological Forecasting Agent

The project goals are: Make data analysis faster and cheaper. Increase use of NASA data by removing barriers to data access. Cope with data heterogeneity. Support code reuse and rapid application development. Support multiple applications, users. Including fire and health domains. Improve QOS. Always provide an answer. Tell user how good it is, where it come from.

Golden, Keith↗
NASA NTRS2023

NLSP: NASA Life Sciences Portal

NASA’s Life Sciences Ports (NLSP) serves the scientific community by providing curated data from space life science experiment. The Human Research Program (HRP) with the help of NLSP is currently transforming their life sciences data archive systems and processes to improve compliance with the FAIR principles. Some of these improvements will at the same time support the twin pillars of Open Science: transparency of methods and reproducibility of results. This video is a high level overview of the NLSP for existing and new users.

Life Sciences data↗
NASA NTRS2005

Data Format Standardization of Space Weather Model Output at the Community Coordinated Modeling Center

The disparate nature of space weather model output provides many challenges with regards to the portability and reuse of not only the data itself, but also any tools that are developed for analysis and visualization. We are developing and implementing a comprehensive data format standardization methodology that allows heterogeneous model output data to be stored uniformly in any common science data format. We will discuss our approach to identifying core meta-data elements that can be used to supplement raw model output data, thus creating self-descriptive files. The meta-data should also contain information describing the simulation grid. This will ultimately assists in the development of efficient data access tools capable of extracting data at any given point and time. We will also discuss our experiences standardizing the output of two global magnetospheric models, and how we plan to apply similar procedures when standardizing the output of the solar, heliospheric, and ionospheric models that are also currently hosted at the Community Coordinated Modeling Center.

Maddox, M.↗
DOE OSTI2022

Why don't we share data and code? Perceived barriers and benefits to public archiving practices

The biological sciences community is increasingly recognizing the value of open, reproducible and transparent research practices for science and society at large. Despite this recognition, many researchers fail to share their data and code publicly. This pattern may arise from knowledge barriers about how to archive data and code, concerns about its reuse, and misaligned career incentives. Here, we define, categorize and discuss barriers to data and code sharing that are relevant to many research fields. We explore how real and perceived barriers might be overcome or reframed in the light of the benefits relative to costs. By elucidating these barriers and the contexts in which they arise, we can take steps to mitigate them and align our actions with the goals of open science, both as individual scientists and as a scientific community.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS2006

A-Train Data Depot: Integrating and Visualizing Atmospheric Measurements Along the A-Train Tracks

The succession of US and international satellites that follow each other, seconds to minutes apart, across the local afternoon equator crossing is called the ATrain. The A-Train consists of the following satellites, in order of equator crossing: OCO, EOS Aqua, CloudSat, CALIPSO, PARASOL, and EOS Aura. Flying in such formation increases the number of observations, validates observations, and enables coordination between science observations, resulting in a more complete virtual science platform (Kelly, 2000) The goal of this project is to create the first ever A-Train virtual data portal/center, the A-Train Data Depot, to process, archive, access, visualize, analyze and correlate distributed atmosphere measurements from various A-Train instruments along A-Train tracks. The A-Train Data Depot (ATDD) will enable the free movement of remotely located A-Train data so that they are combined to create a consolidated vertical view of the Earth s Atmosphere along the A-Train tracks. Once the infrastructure of the ATDD is in place, it will be easily evolved to serve data from all A-Train data measurements: one stop shopping. The innovative approach of analyzing and visualizing atmospheric profiles along the platforms track (i.e., time) will be accommodated by reusing the GSFC Atmospheric Composition Data and Information Services Center (ACDISC) visualization and analysis tool, GIOVANNI, existing data reduction tools, on-line archwing for fast data access, and Cooperative Institute for Research in the Atmosphere (CRA) data co-registration tools. Initial measurements utilized include CALIPSO lidar backscatter, CloudSat radar reflectivity, clear air relative humidity, water vapor and temperature from AIRS, and cloud properties and aerosols from both MODIS. This will be followed by associated measurements from MLS, OMI, HIRDLS, and TES. Given the independent nature of instrument/platform development, the ATDD project has been met with many interesting challenges that, once resolved, will provide a much greater understanding of the relative flight dynamics and data co-registration of the suite of A-Train instruments, thus greatly increasing the accuracy of A-Train data analysis. Some of these challenges will be discussed. The project s resulting visualizations and analysis illustrate the importance of managing data so that measurements from various missions can be combined to enhance the understanding of the atmosphere. A-Train data management coordination, as performed here, is extremely significant in facilitating the A-Train science of clouds, precipitation, aerosol and chemistry.

Kempler, Steven↗
NASA NTRS2006

A-Train Data Depot: Integrating, Visualizing, and Extracting Cloudsat, CALIPSO, MODIS, and AIRS Atmospheric Measurements Along the A-Train Tracks

The succession of US and international Earth observing satellites that follow each other, seconds to minutes apart, across the local afternoon equator crossing is called the ATrain. The A-Train consists of the following satellites, in order of equator crossing: OCO, EOS Aqua, CloudSat, CALIPSO, PARASOL, and EOS Aura. Flying in such formation increases the number of observations, validates observations, and enables coordination between science observations, resulting in a more complete virtual science platform (Kelly, 2000). The goal of this project is to create the first ever A-Train virtual data portal/center, the A-Train Data Depot (ATDD), to process, archive, access, visualize, analyze and correlate distributed atmosphere measurements from various A-Train instruments along A-Train tracks. The ATDD will enable the free movement of remotely located A-Train data so that they are combined to create a consolidated vertical view of the Earth's Atmosphere along the A-Train tracks. Once the infrastructure of the ATDD is in place, it will be easily evolved to serve data from all A-Train data measurements: one stop shopping. The innovative approach of analyzing and visualizing atmospheric profiles along the platforms track (i.e., time) will be accommodated by reusing the GSFC Atmospheric Composition Data and Information Services Center (ACDISC) visualization and analysis tool, GIOVANNI, existing data reduction tools, on-line archiving for fast data access, access to remote data without unnecessary data transfers, and data retrieval by users finding data desirable for further study. Initial measurements utilized include CALIPSO lidar backscatter, CloudSat radar reflectivity, clear air relative humidity, water vapor and temperature from AIRS, and cloud properties and aerosols from both MODIS. This will be foilowed by associated measurements from TVILS, =MI, HIRDLS, sad TES. Given the independent nature of instrumentlplatform development, the ATDD project has been met with many interesting challenges that, once resolved, will provide a much greater understanding of the relative flight dynamics and data co-registration of the suite of A-Train instruments, thus greatly increasing the accuracy of A-Train data analysis. Some of these challenges will be illustrated and discussed. The project's early visualizations and analysis efforts illustrate the importance of managing data so that measurements from various missions can be combined to enhance the understanding of the atmosphere. A-Train data management coordination, as performed here, is extremely significant in facilitating the A-Train science of clouds, precipitation, aerosol and chemistry.

Kempler, Steven↗
DOE OSTI2025

Opening doors to physical sample tracking and attribution in Earth and environmental sciences

Physical samples and their associated data and metadata underpin scientific discoveries across disciplines and can enable new science when appropriately archived. However, there are significant gaps in current practices and infrastructure that prevent accurate provenance tracking, reproducibility, and attribution. For most samples, descriptive metadata are often sparse, inaccessible, or absent. Samples and associated data and metadata may also be scattered across numerous physical collections, data repositories, laboratories, data files, and papers with no clear linkage or provenance tracking as new information is generated over time. The Earth Science Information Partners (ESIP) Physical Samples Curation Cluster has therefore developed guidance for scientific authors on ‘Publishing Open Research Using Physical Samples.’ This involved synthesizing existing practices, gathering community feedback, and assessing real-world examples. We identified improvements needed to enable authors to efficiently cite and link Earth science samples and related data, and track their use. Our goal is to help improve discoverability, interoperability, and reuse of physical samples, and associated data and metadata. Though primarily focused on the needs of Earth and environmental sciences, these guidelines are broadly applicable.

58 GEOSCIENCES↗
NASA NTRS1989

Application of reusable software components at the SEI

Robert Holibaugh of the Software Engineering Institute described a project which is studying the application of reusable software components. The primary goals are to gain practical experience with state-of-the-art reusable components, methods, and tools and to capture the lessons learned in the application of reuse technology. In addition the project will assess the impact of reuse on the software development process and products and will identify and validate the information that facilities software reuse during system development. The project includes two tasks - a reuse experiment and a redevelopment effort. The reuse experiment will define a life cycle and a methodology for reuse-based development, and define and implement a data collection mechanism for measuring the development. The redevelopment effort will construct a reuse test bed and will redevelop and realistically test subsystems from an embedded mission-critical real-time application. The reuse experiment will produce several products including a tested real-time application, reuse-based components and tools evaluation, a reuse-based development method, a framework for data collection, a framework for measuring productivity, and lessons learned data. Successful development with reusable components will require a rich set of components and an integrating methodology. The Tomahawk Land Attack Missile system is the application for the redevelopment effort.

Holibaugh, Robert↗
NASA NTRS2017

Upgrades to Common Data Acquisition System Software Development for NASA's Rocket Propulsion Test Facilities and Software Reuse

Approximately five years ago, the National Aeronautics and Space Administration (NASA) Stennis Space Center (SSC) resumed operation of its large rocket engine test facilities after thirty years of contractor control. During this period, contactors used their own proprietary Data Acquisition System (DAS) to record and process rocket propulsion test data. The transition from a contractor managed facility to a NASA managed facility posed a difficult challenge. In order to support the commercial space launch initiative, SSC needed to develop a software replacement for the contractor proprietary DAS. This replacement software would enable SSC to operate propulsion test facilities more cost effectively and to be more readily able to adapt software for reuse, while at the same time provide internal and external customers with reliable population test data. Therefore, SSC developed in-house, a non-proprietary software suite of applications to replace the previously used proprietary DAS. The requirements for the DAS suite included recording and processing propulsion test data. This capability eliminates the necessity for customers to provide a DAS or rely on a competitor's DAS. An additional benefit of owning the software suite included enabling the ability to add additional features and functionality at a lower cost. The Rocket Propulsion Test (RPT) Program Office reviewed consideration for funding this project with the caveat that development of the software included availability for use with minimal modifications to all SSC test facilities and RPT centers: Marshall Space Flight Center (MSFC), White Sands Test Facility (WSTF), and Glenn Research Center (GRC) Plum Brook Station. Based upon this guideline, SSC created the NASA Data Acquisition System (NDAS) software suite. The ability to use the software at multiple centers, even though each field center uses differing DAS hardware with different capabilities, drove a requirement that the software design be portable with minimal modifications to the software. Then, with software release requirements, evaluations, and approvals completed, the NDAS software suite could also become available to other government agencies, corporations, universities, and the general United States public.

Herbert, Phillip W., Sr.↗
DOE OSTI2023

Adaptive cache management based on programming model information

A processing system selectively allocates space to store a group of one or more cache lines at a cache level of a cache hierarchy having a plurality of cache levels based on memory access patterns of a software application executing at the processing system. The processing system generates bit vectors indicating which cache levels are to allocate space to store groups of one or more cache lines based on the memory access patterns, which are derived from data granularity and movement information. Based on the bit vectors, the processing system provides hints to the cache hierarchy indicating the lowest cache level that can exploit the reuse potential for a particular data.

Na, Weon Taek↗