Search NASA⌕ Search

SEARCH · Search NASA

Results for “Archive”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Integrating Characteristic Arctic Vegetation in a Land Surface Model Improves Representation of Carbon Dynamics Across a Tundra Landscape: Modeling Archive

This modeling archive is in support of the Next-Generation Ecosystem Experiments in the Arctic (NGEE Arctic) publication "Integrating Characteristic Arctic Vegetation in a Land Surface Model Improves Representation of Carbon Dynamics Across a Tundra Landscape", by Murphy et al. (2025). This archive contains model input files and outputs from landscape-scale simulations conducted using ELM, the land model component of the Department of Energy’s Energy Exascale Earth System Model (E3SM), at the Council NGEE Arctic field site (Council Road mile marker 71) on Alaska’s Seward Peninsula. Input data and model output from two sets of ELM simulations are provided. The first set of simulations were conducted with the two default ELM Arctic plant functional types (PFTs; broadleaf deciduous boreal shrub and a C3 grass) and the second set of simulations were conducted with a set of nine Arctic-specific PFTs including nonvascular mosses and lichens, graminoids, forbs, evergreen dwarf shrubs, three height classes of deciduous shrubs (dwarf, low, and low to tall), and deciduous alder shrubs (Sulman et al., 2021). Parameter names and major parameter changes in the Arctic-specific PFT configuration are described in Sulman et al. (2021) and archived in the Sulman et al. (2021) dataset (see below). Simulations were spatially explicit, covering an approximately 6.4X3.3 km domain at the Council site with a spatial resolution of 100 m for a total of 2,112 simulated grid cells under each ELM PFT configuration. The modeling archive contains meteorological forcing (seven *.nc files and one *.txt file), a domain definition file (one *.nc files), land surface configuration files (two *.nc files), parameter files (two *.nc files), annual ELM output files spanning 1980-2014 (68 *.nc files), and a User’s Guide (*pdf file). Additional information on the provided files is in the “Modeling Archive Contents” section of the User’s Guide. Model outputs are aggregated to the column scale (i.e. PFT-specific outputs are not provided here).

Murphy, Bailey [ORNL] (ORCID:0000000203995221)↗
DOE OSTI2021

Developing Bloom Filters for Web Archives’ Holdings (Final Project Report)

The main goal of the project was to develop a framework for web archives to create Bloom filters based on their holdings of archived web resources. A Bloom filter (BF) is a data structure that, in our scenario, contains hash values of all (or a subset of) URLs, of which an archive has one or more archival copies. Two main use cases fall in scope for this project and are supported by a BF implementation: 1) Sharing of an archive’s holdings (URLs) and 2) Querying the holdings of one or more archives. Since URL strings are hashed before ingested into the BF, the index of an archive is not shared in plain text when the BF is shared with trusted parties. On the other hand, a BF implementation does allow queries for a URL to confirm if an archive indeed has one or more archival copies of that URL. This collaborative project between Los Alamos National Laboratory (LANL) and the Croatian Web Archive (HAW), from the National and University Library in Zagreb (NSK), developed by NSK and University of Zagreb University Computing Center SRCE), aimed at developing software to create BFs, evaluate the scalability of the approach, pilot a search service based on BFs, and design a framework for archives to share their filters with trusted parties. In the remainder of this document, we will report on the work completed with respect to the individual deliverables, outline aspects of future work, and conclude with our recommendations for the use of BFs for the web archiving community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

ESS-DIVE guidelines for archiving terrestrial model data

This dataset contains supporting documents and images for ESS-DIVE terrestrial model data archiving guidelines.Terrestrial models are broadly defined as numerical models that couple both land dynamics and energy, water, carbon, or nutrient fluxes. We created these guidelines based on input from the U.S. Department of Energy’s Biological and Environmental Research land modeling community. The guidelines are intended to help modelers determine which components of their terrestrial model data associated with publication should be archived. Based on input from the land modeling community, the guidelines recommend archiving both model input and testing data, as well as code, script, and metadata. The guidelines also recommend archiving model data output, depending on the limitations set by data repositories. Lastly, we provide recommendations for bundling data files for publication as well as a discussion about tools that can facilitate model data archiving and reuse.This dataset is an archive of the associated GitHub repository for our model archiving guidelines (https://github.com/ess-dive-community/essdive-model-data-archiving-guidelines). The ‘README.pdf’ file gives a general introduction to the guidelines, and the ‘instructions.pdf’ file provides more detailed steps for following the guidelines. We also provide 2 figures in this data package: 1) a decision tree (model_data_guidelines_decision_tree.png) that can help users determine which components of their model data to archive. and 2) the ‘model_data_guidelines_flmd.png’ file depicts the different files that can be archived in addition to the model data itself. Lastly, we include 3 digitized tables from our associated manuscript and 3 CSV files with anonymized input from DOE scientists about the importance of different aspects of model data archiving from which we developed the guidelines.Dataset updates for v1.1.0: We updated this data package on 2021-11-22 in response to review comments on our related manuscript. In this update we removed one figure so that the model archiving guidelines are conveyed in text rather than an image. We updated the file-level metadata (FLMD) figure to be in accord with the most recent FLMD recommendations. We made minor edits to the README file to update the recommended citation and added two co-authors. We also added 6 new data files (3 are anonymized input from DOE scientists that helped to inform guidelines, and 3 are digitized tables from our manuscript.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Hashes are not suitable to verify fixity of the public archived web

Web archives, such as the Internet Archive, preserve the web and allow access to prior states of web pages. We implicitly trust their versions of archived pages, but as their role moves from preserving curios of the past to facilitating present day adjudication, we are concerned with verifying the fixity of archived web pages, or mementos, to ensure they have always remained unaltered. A widely used technique in digital preservation to verify the fixity of an archived resource is to periodically compute a cryptographic hash value on a resource and then compare it with a previous hash value. If the hash values generated on the same resource are identical, then the fixity of the resource is verified. We tested this process by conducting a study on 16,627 mementos from 17 public web archives. We replayed and downloaded the mementos 39 times using a headless browser over a period of 442 days and generated a hash for each memento after each download, resulting in 39 hashes per memento. The hash is calculated by including not only the content of the base HTML of a memento but also all embedded resources, such as images and style sheets. We expected to always observe the same hash for a memento regardless of the number of downloads. However, our results indicate that 88.45% of mementos produce more than one unique hash value, and about 16% (or one in six) of those mementos always produce different hash values. We identify and quantify the types of changes that cause the same memento to produce different hashes. These results point to the need for defining an archive-aware hashing function, as conventional hashing functions are not suitable for replayed archived web pages.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2022

The DSA Toolkit Shines Light Into Dark and Stormy Archives

Web archive collections are created with a particular purpose in mind. A curator selects seeds, or original resources, which are then captured by an archiving system and stored as archived web pages, or mementos. The systems that build web archive collections are often configured to revisit the same original resource multiple times. This is incredibly useful for understanding an unfolding news story or the evolution of an organization. Unfortunately, over time, some of these original resources can go off-topic and no longer suit the purpose for which the collection was originally created. They can go off-topic due to web site redesigns, changes in domain ownership, financial issues, hacking, technical problems, or because their content has moved on from the original topic. Even though they are off-topic, the archiving system will still capture them, thus it becomes imperative to anyone performing research on these collections to identify these off-topic mementos. Hence, we present the Off-Topic Memento Toolkit, which allows users to detect off-topic mementos within web archive collections. The mementos identified by this toolkit can then be separately removed from a collection or merely excluded from downstream analysis. The following similarity measures are available: byte count, word count, cosine similarity, Jaccard distance, Sørensen-Dice distance, Simhash using raw text content, Simhash using term frequency, and Latent Semantic Indexing via the gensim library. We document the implementation of each of these similarity measures. We possess a gold standard dataset generated by manual analysis, which contains both off-topic and on-topic mementos. Using this gold standard dataset, we establish a default threshold corresponding to the best F1 score for each measure. We also provide an overview of potential future directions that the toolkit may take.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2019

Profiling the Usage of an Extreme-Scale Archival Storage System

Profiling the archival storage system in scientific computing environments has received much less attention compared to the parallel file system, but is equally important since it stores the final data products safely, for a long duration. In this paper, we analyze eight years worth of data transfer logs for accessing the archival file system (HPSS) in the Oak Ridge Leadership Computing Facility (OLCF), which has been hosting the world's largest supercomputers and file systems. Our analysis encompasses about 135 million data transfer activities to the 80 PB High Performance Storage System (HPSS), between 2010 and 2017. We analyze the logs from several dimensions, including studying the workload characteristics (e.g., access patterns, frequency of accesses and temporal behavior), file system characteristics (e.g., directory depth, file system scaling trends, file types), and scientific user behavior (e.g., domain-specific usage and organization). Based on the analysis, we derive insights into the future evolution of the archive in terms of provisioning, desired features and functionality from the archive software, role and right sizing of the archive tiers, quota management, and the importance of smart and efficient metadata and storage management. We believe our study will prove useful for both operating current archival storage and better provisioning future systems.

Sim, Hyogi↗
DOE OSTI2022

Guidelines for Publicly Archiving Terrestrial Model Data to Enhance Usability, Intercomparison, and Synthesis

Scientific communities are increasingly publishing data to evaluate, accredit, and build on published research. However, guidelines for curating data for publication are sparse for model-related research, limiting the usability of archived simulation data. In particular, there are no established guidelines for archiving data related to terrestrial models that simulate land processes and their coupled interactions with climate. Terrestrial modelers have a unique set of challenges when publishing data due to the diversity of scientific domains, research questions, and the types and scales of simulations. Researchers in the U.S. Department of Energy’s (DOE) projects use a variety of multiscale models to advance robust predictions of terrestrial and subsurface ecosystem processes. Here, we synthesize archiving needs for data associated with different DOE models, and provide guidelines for publishing terrestrial model data components following FAIR (Findable, Accessible, Interoperable, Reusable) principles. The guidelines recommend archiving model inputs and testing data used in final simulation runs along with associated codes, workflow scripts, and metadata in public repositories. Researchers should consider archiving model outputs if they are within the storage limits of the repository. We also provide considerations for how to bundle files into different data publications with citable digital object identifiers. Finally, we identify repository features and tools that would enable storage and reuse of model data. Given the diversity of DOE terrestrial models, these guidelines are transferable to other model types and will enable efficient reuse of simulation data for purposes such as model intercomparisons, initialization, benchmarking, synthesis, and comparisons with field observations.

58 GEOSCIENCES↗
DOE OSTI2025

Digitizing and Enhancing Accessibility of the Fusion Safety Archives

This project focuses on the digitization and public accessibility to the Fusion Safety Archives at the Idaho National Laboratory. The first phase involves a thorough review of each document in the physical archives to determine its online availability. For documents that are available online, PDF copies and unique identifiers are collected for database integration. Documents not available online are delivered to Red Inc. for digitization. Additionally, defunct storage devices such as diskettes are sent to INL’s archival department for data retrieval where possible. The second phase of the project involves the creation of a comprehensive database to house the digital copies of the archives. The database will facilitate easy access and management of the digitized documents. Following the database creation, we plan to train a Retrieval-Augmented Generation (RAG) based AI on publicly available documents. The trained AI will be integrated into a front-facing application, allowing the public to easily access information from the Fusion Safety Archives. This project aims to preserve valuable historical data, improve accessibility, and promote transparency in fusion safety research.

70 - PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2018

Epi Archive: Automated Synthesis of Global Notifiable Disease Data

ObjectiveLANL has built software that automatically collects global notifiable disease data, synthesizes the data, and makes it available to humans and computers within the Biosurveillance Ecosystem (BSVE) as a novel data stream. These data have many applications including improving the prediction and early warning of disease events.IntroductionMost countries do not report national notifiable disease data in a machine-readable format. Data are often in the form of a file that contains text, tables and graphs summarizing weekly or monthly disease counts. This presents a problem when information is needed for more data intensive approaches to epidemiology, biosurveillance and public health.While most nations likely store incident data in a machine-readable format, governments are often hesitant to share data openly for a variety of reasons that include technical, political, economic, and motivational issues1.A survey conducted by LANL of notifiable disease data reporting in over fifty countries identified only a few websites that report data in a machine-readable format. The majority (>70%) produce reports as PDF files on a regular basis. The bulk of the PDF reports present data in a structured tabular format, while some report in natural language.The structure and format of PDF reports change often; this adds to the complexity of identifying and parsing the desired data. Not all websites publish in English, and it is common to find typos and clerical errors.LANL has developed a tool, Epi Archive, to collect global notifiable disease data automatically and continuously and make it uniform and readily accessible.MethodsWe conducted a survey of the national notifiable disease reporting systems notating how the data are reported and in what formats. We determined the minimal metadata that is required to contextualize incident counts properly, as well as optional metadata that is commonly found.The development of software to regularly ingest notifiable disease data and make it available involves three or four main steps: scraping, detecting, parsing and persisting.Scraping: we examine website design and determine reporting mechanisms for each country/website, as well as what varies across the reporting mechanisms. We then designed and wrote code to automate the downloading of the data for each country. We store all artifacts presented as files (PDF, XLSX, etc.) in their original form, along with appropriate metadata for parsing and data provenance.Detecting: This step is required when parsing structured non-machine-readable data such as tabular data in PDF files. We combined the Nurminen methodology of PDF table detection with in-house heuristics to find the desired data within PDF reports2.Parsing: We determined what to extract from each dataset and parsed these data into uniform data structures, correctly accommodating the variations in metadata (e.g., time interval definitions) and the various human languages.Persisting: We store the data in the Epi Archive database and make it available on the internet and through the BSVE. The data is persisted into a structured and normalized SQL database.ResultsThe Epi Archive tool currently contains national and/or subnational notifiable disease data from twenty nations. When a user accesses the Epi Archive site, they are prompted with four fields: country, subregion, disease of interest, and date duration. Upon form submission, a time series is generated from the users’ specifications. The generated graph can then be downloaded into a CSV file if a user is interested in performing personal analysis. Additionally, the data from Epi Archive can be reached through a REST API (Representational State Transfer Application Programming Interface).ConclusionsLANL, as part of a currently funded DTRA effort, is automatically and continually collecting global notifiable disease data. While 20 nations are in production, more are being brought online in the near future. These data are already being utilized and will have many applications including improving the prediction and early warning of disease events.References[1] van Panhuis WG, Paul P, Emerson C, et al. A systematic review of barriers to data sharing in public health. BMC Public Health. 2014. 14:1144. doi:10.1186/1471-2458-14-1144[2] Nurminen, Anssi. "Algorithmic extraction of data in tables in PDF documents." (2013).

Khalsa, Hari S.↗
DOE OSTI2021

Detecting and Analyzing Power System Disturbances in PMU Data with the Open-Source Archive Walker Tool

As the number of Phasor Measurement Units (PMUs) deployed in power systems increases, so do the archives of synchrophasor measurements stored by utilities. Significant value can be extracted from these archives by using them to support studies such as frequency response analysis, small-signal stability analysis, and power plant model validation. To better enable utilities to perform these studies, the open-source Archive Walker tool was developed. This tool examines PMU data for grid disturbances and periods of interest. Once identified, the data can be further analyzed inside Archive Walker or exported to specialty tools. In this paper, the various disturbance detectors available in Archive Walker are described, and its interactions with other openly-available tools are highlighted.

Follum, James D.↗
DOE OSTI2021

Nuclear Archival Electronic Database

The Nuclear Archival Electronic Database (NAED) supports a work scope that has been focused on providing a framework for and preserving key experimental programs and experiences which are critical to the licensing basis of currently operating nuclear reactors or which could be used in the safety basis for the next generation of reactors. It aims at preserving the results of experimental programs for the safety of light water cooled and advanced reactors by locating and documenting, to the extent possible, where the experimental test information for these programs has been archived. The NAED is a compilation of the following: (a) list of experiments or experiences categorized by coolant type, (b) results of an information search, and documentation, of the experiments and experiences that are already considered archived and (c) results of a deep dive and interview on a single experimental program or experience. The intent is to use the NAED as a technology source document by reactor vendors, regulators, and utility personnel.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Glacier ice archives nearly 15,000-year-old microbes and phages

Background Glacier ice archives information, including microbiology, that helps reveal paleoclimate histories and predict future climate change. Though glacier-ice microbes are studied using culture or amplicon approaches, more challenging metagenomic approaches, which provide access to functional, genome-resolved information and viruses, are under-utilized, partly due to low biomass and potential contamination. Results We expand existing clean sampling procedures using controlled artificial ice-core experiments and adapted previously established low-biomass metagenomic approaches to study glacier-ice viruses. Controlled sampling experiments drastically reduced mock contaminants including bacteria, viruses, and free DNA to background levels. Amplicon sequencing from eight depths of two Tibetan Plateau ice cores revealed common glacier-ice lineages including Janthinobacterium, Polaromonas, Herminiimonas, Flavobacterium, Sphingomonas, and Methylobacterium as the dominant genera, while microbial communities were significantly different between two ice cores, associating with different climate conditions during deposition. Separately, ~355- and ~14,400-year-old ice were subject to viral enrichment and low-input quantitative sequencing, yielding genomic sequences for 33 vOTUs. These were virtually all unique to this study, representing 28 novel genera and not a single species shared with 225 environmentally diverse viromes. Further, 42.4% of the vOTUs were identifiable temperate, which is significantly higher than that in gut, soil, and marine viromes, and indicates that temperate phages are possibly favored in glacier-ice environments before being frozen. In silico host predictions linked 18 vOTUs to co-occurring abundant bacteria (Methylobacterium, Sphingomonas, and Janthinobacterium), indicating that these phages infected ice-abundant bacterial groups before being archived. Functional genome annotation revealed four virus-encoded auxiliary metabolic genes, particularly two motility genes suggest viruses potentially facilitate nutrient acquisition for their hosts. Finally, given their possible importance to methane cycling in ice, we focused on Methylobacterium viruses by contextualizing our ice-observed viruses against 123 viromes and prophages extracted from 131 Methylobacterium genomes, revealing that the archived viruses might originate from soil or plants. Conclusions Together, these efforts further microbial and viral sampling procedures for glacier ice and provide a first window into viral communities and functions in ancient glacier environments. Such methods and datasets can potentially enable researchers to contextualize new discoveries and begin to incorporate glacier-ice microbes and their viruses relative to past and present climate change in geographically diverse regions globally.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Challenges and Opportunities in Turning Large U.S. Archives of Analog Seismograms into a Modern Usable Resource

Abstract Quantitative seismology is based firmly on the analysis of actual ground motions, and the transition to digital recording in the 1980s enabled sophisticated new capabilities to extract useful results from waveforms. With some effort, these tools can also be applied to analog records. Focusing on assets available within U.S. institutions, we review the necessary steps and the challenges in enabling “data rescue”—that is, preserving the scientific information latent in large analog seismogram archives and making it usable. They include: determining what assets are available (the analog seismogram archives held by various institutions, with associated metadata on instrument responses, station locations, and timing information); developing a consensus on the top level of a triage process (which analog records most definitely should be rescued?); deciding the level of quality needed in copying original seismograms to media suitable for digitizing; assessing the relative merits of scanning and digitizing; and, the need for a community service in distributing scans and digital records, as they accumulate. The necessary level of effort can benefit from practical experience. For example, specific studies have used digitized versions of analog recordings to model earthquake sources and assess seismic hazard. Other studies have used them to gain experience with nuclear explosion signals recorded at regional distances, noting that regional signals enable explosions to be monitored down to levels much lower than those attainable teleseismically. The opportunities presented by large archives of analog seismograms include the insights they present to current and future seismologists studying earthquakes and explosions, into the practical areas of assessing seismic hazard, monitoring for test ban compliance down to low explosion yields—and prompt characterization of actual explosions should they occur, as well the traditional academic pursuit of a better understanding of earthquake physics.

Geochemistry & Geophysics↗
DOE OSTI2018

Learning Memento archive routing with Character-based Artificial Neural Networks

This white paper describes a series of tests that were performed to determine if a neural network could learn patterns from a service that maintains a cache of routing decisions for the discovery of version information for discrete web-at-large URLs. Labeled training data was derived from a log file that records the by-archive availability of Memento availability for a given URL. Training data sets were generated from this log file on a by-archive basis (thus making it a binary classification problem). Each training data set consisted of equal numbers of hit and miss URLs, selected at random. The URLs were converted to a normalized numeric representation where each integer in a URL training vector represents a character in that URL. The corresponding label indicates whether or not Mementos were available for that URL in the selected archive. The training data matrix and the label vector became input to a neural network. A number of neural network architectures and network hyperparameters were explored, however the log entries themselves were used as-is, without any feature engineering, beyond the aforementioned normalization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Biological Mechanisms May Contribute to Soil Carbon Saturation Patterns: Modeling Archive

This Modeling Archive is in support of a TES-SFA publication “Biological Mechanisms May Contribute to Soil Carbon Saturation Patterns” (Craig et al., 2021). We ran and evaluated a multi-assumption soil organic carbon (SOC) model to investigate whether alternative assumptions regarding constraints on soil microbial biomass could lead to soil carbon saturation patterns. We developed this model in the Multi-Assumption Architecture and Testbed (MAAT, https://github.com/walkeranthonyp/MAAT, tag: v1.2.1_Craig2021; Walker et al. 2018). Using MAAT, we embedded three alternative hypotheses in a microbially explicit three-pool SOC model: 1) the efficiency of mineral-associated SOC formation decreases as mineral-associated SOC approaches a maximum value (“Mineral saturation”), 2) the microbial biomass turnover rate increases with increasing microbial biomass (“Density-dependent turnover”), and 3) community carbon use efficiency decreases as microbial biomass increases toward an upper limit (“Density-dependent growth”). We ran a factorial combination of these hypotheses resulting in eight models for three different classes of model (linear decay, Michaelis-Menten decay, or reverse Michaelis-Menten decay), resulting in 24 models, 12 of which are presented or discussed in the related publication. Models were parameterized using values from previous studies with similar models (Wang et al. 2013, Wieder et al. 2014, Li et al. 2014, Georgiou et al. 2017, Hassink and Whitmore 1997) and ran to an approximate steady state (200 years) at six (6) different C input rates corresponding to 0.5, 1, 2, 4, 7, and 10 times the default input value. Further model details are available in the related publication. This archive contains output from three MAAT simulations, and scripts to run these simulations and process and plot the data. Simulations are labeled “lin”, “MM_highKm”, and “RMM_highKm” reflecting factorial runs for linear, Michealis-Menten, and reverse Michaelis-Menten models, respectively. This archive contains: • 3 R scripts prepended with “init_MAAT_” to initialize runs (1 for each simulation), • 1 csv file containing years over which to run simulations (“met_year.csv”), • 1 bash script (.bs) to run MAAT, • 6 XML files that are output from MAAT describing a run (2 for each simulation), • 3 model output csv files prepended by “out_” (1 for each simulation), and • 1 analysis R script for reproducing figures 3 and 4 in Craig et al. 2021. See included user guide (Craig_2021_modeling_archive_20210315.pdf) for file organization details.

54 ENVIRONMENTAL SCIENCES↗