Search NASA⌕ Search

SEARCH · Search NASA

Results for “data integrity”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

Logic design for dynamic and interactive recovery.

Recovery in a fault-tolerant computer means the continuation of system operation with data integrity after an error occurs. This paper delineates two parallel concepts embodied in the hardware and software functions required for recovery; detection, diagnosis, and reconfiguration for hardware, data integrity, checkpointing, and restart for the software. The hardware relies on the recovery variable set, checking circuits, and diagnostics, and the software relies on the recovery information set, audit, and reconstruct routines, to characterize the system state and assist in recovery when required. Of particular utility is a handware unit, the recovery control unit, which serves as an interface between error detection and software recovery programs in the supervisor and provides dynamic interactive recovery.

Carter, W. C.↗

BASTet: Shareable and Reproducible Analysis and Visualization of Mass Spectrometry Imaging Data via OpenMSI

Mass spectrometry imaging (MSI) is a transformative imaging method that supports the untargeted, quantitative measurement of the chemical composition and spatial heterogeneity of complex samples with broad applications in life sciences, bioenergy, and health. While MSI data can be routinely collected, its broad application is currently limited by the lack of easily accessible analysis methods that can process data of the size, volume, diversity, and complexity generated by MSI experiments. The development and application of cutting-edge analytical methods is a core driver in MSI research for new scientific discoveries, medical diagnostics, and commercial-innovation. However, the lack of means to share, apply, and reproduce analyses hinders the broad application, validation, and use of novel MSI analysis methods. To address this central challenge, we introduce the Berkeley Analysis and Storage Toolkit (BASTet), a novel framework for shareable and reproducible data analysis that supports standardized data and analysis interfaces, integrated data storage, data provenance, workflow management, and a broad set of integrated tools. Based on BASTet, we describe the extension of the OpenMSI mass spectrometry imaging science gateway to enable web-based sharing, reuse, analysis, and visualization of data analyses and derived data products. Here, we demonstrate the application of BASTet and OpenMSI in practice to identify and compare characteristic substructures in the mouse brain based on their chemical composition measured via MSI.

97 MATHEMATICS AND COMPUTING↗

The PARADIGM Project: Case Study in Balancing Experiment Uncertainty with Design simplicity

Accurate nuclear data are required for simulations of many applications including nuclear criticality safety. Actinide nuclear data at intermediate energies (from 1 to 100s of keV) are imprecise and inaccurate, because of scarce differential data, and an insufficient theory approach to capture the structures expected in the data to yield evaluated nuclear data, and lack of integral data for proper validation. This is a known deficiency but has proved challenging to address. More specifically, only 5% of integral experiments in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) benchmark suite address intermediate energies (Fig. 1). Associated calculated effective multiplication factor, k eff , values for these experiments are far outside the experimental uncertainties and are 25× further from experiment than for fast energies. These differences could either stem from systematic biases in nuclear data, experiments or both. The goal of the PARADIGM (PARallel Approach of Differential and InteGral Measurements) project is to significantly reduce (by more than tens of percent) the uncertainties of intermediate energy actinide nuclear data. The PARADIGM project designed and intends to execute LANSCE (Los Alamos Neutron Science CEnter) and NCERC (National Criticality Experiments Research Center) intermediate experiments in parallel. They will specifically address a high priority nuclear data need—reducing bias and uncertainty in intermediate plutonium nuclear data. The two experiment will achieve that by informing each other and nuclear theory. By doing all these steps in parallel, the timeline to deliver improved nuclear data to users will significantly be reduced. This work will focus on the integral experiment final design and the balance of design and modeling simplicity while minimizing experiment uncertainty.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Anomaly Detection and Mitigation for Dynamic Frequency Regulation in Hydropower-Battery Systems

Hydropower operators and energy storage providers are increasingly interested in participating in frequency regulation services, driven by the incentives offered by independent system operators, such as the PJM Interconnection. This transition, however, unfolds against the backdrop of a modernizing and rapidly digitizing power grid, exposing the integrated legacy infrastructure to a multitude of cybersecurity threats. This work presents an approach for developing an anomaly detection and mitigation system to address cybersecurity challenges during the participation of a hydropower-integrated battery energy storage system (BESS) in a frequency regulation market. The applied anomaly detector utilizes machine learning algorithms to provide detailed classification of cyber-physical events. Later, the applied mitigation system triggers predefined corrective actions to minimize the impact of data integrity attacks on the regulation market and system stability. We evaluated the proposed approach on a hydropower-integrated BESS topology, specifically analyzing the slow regulation signal (Reg A) coming from the PJM market. Our simulation results demonstrate that the proposed approach performs well in detecting data integrity attacks within the allocated time frame and also minimizes the system's transient instability during the participation of hydropower and BESS in the regulation market.

battery energy storage system↗

FIRED (Fire Events Delineation): An Open, Flexible Algorithm and Database of US Fire Events Derived from the MODIS Burned Area Product (2001–2019)

Harnessing the fire data revolution, i.e., the abundance of information from satellites, government records, social media, and human health sources, now requires complex and challenging data integration approaches. Defining fire events is key to that effort. In order to understand the spatial and temporal characteristics of fire, or the classic fire regime concept, we need to critically define fire events from remote sensing data. Events, fundamentally a geographic concept with delineated spatial and temporal boundaries around a specific phenomenon that is homogenous in some property, are key to understanding fire regimes and more importantly how they are changing. Here, we describe Fire Events Delineation (FIRED), an event-delineation algorithm, that has been used to derive fire events (N = 51,871) from the MODIS MCD64 burned area product for the coterminous US (CONUS) from January 2001 to May 2019. The optimized spatial and temporal parameters to cluster burned area pixels into events were an 11-day window and a 5-pixel (2315 m) distance, when optimized against 13,741 wildfire perimeters in the CONUS from the Monitoring Trends in Burn Severity record. The linear relationship between the size of individual FIRED and Monitoring Trends in Burn Severity (MTBS) events for the CONUS was strong (R2 = 0.92 for all events). Importantly, this algorithm is open-source and flexible, allowing the end user to modify the spatio-temporal threshold or even the underlying algorithm approach as they see fit. We expect the optimized criteria to vary across regions, based on regional distributions of fire event size and rate of spread. We describe the derived metrics provided in a new national database and how they can be used to better understand US fire regimes. The open, flexible FIRED algorithm could be utilized to derive events in any satellite product. We hope that this open science effort will help catalyze a community-driven, data-integration effort (termed OneFire) to build a more complete picture of fire.

54 ENVIRONMENTAL SCIENCES↗

Container Security - part of the CORE system

A data integration system to support the US Customs and Border Protection Officers to supervise and make decisions for container inspections. CORE is designed to act as a framework to bridge the gaps between disparate data integration and delivery of disparate information visualization.

Gelston, Gariann↗

Case History : Using time-lapse vertical seismic profiling data to constrain velocity-saturation relations: the Frio brine pilot CO 2 injection: Time-lapse VSP to constrain VSRs

CO 2 sequestration projects benefit from quantitative assessment of saturation distribution and plume extent for field development and leakage prevention. In this work, we carry out quantitative analysis of time-lapse seismic by using rock physics and seismic modelling tools. We investigate the suitability of Gassmann's equation for a CO 2 sequestration project with 1600 tons of CO 2 injected into high-porosity, brine-saturated sandstone. We analyze the observed time delays and amplitude changes in a time-lapse vertical seismic profile dataset. Both reflected and transmitted waves are analyzed qualitatively and quantitatively. To interpret the changes obtained from the vertical seismic profile, we perform a 2.5D elastic, finite-difference modelling study. The results show a P-wave velocity reduction of 750 m/s in the proximity of the injection well evident by the first arrivals (travel-time delays and amplitude change) and reflected wave amplitude changes. These results do not match with our rock physics model using Gassmann's equation predictions even when taking uncertainty in CO 2 saturation and grain properties into account. We find that time-lapse vertical seismic profile data integrated with other information (e.g., core and well log) can be used to constrain the velocity–saturation relation and verify the applicability of theoretical models such as Gassmann's equation with considerable certainty. The study shows that possible nonelastic factors are in play after CO 2 injection (e.g., CO 2 –brine–rock interaction and pressure effect) as Gassmann's equation underestimated the velocity reduction in comparison with field data for all three sets of time-lapse vertical seismic profile attributes. Here, our work shows the importance of data integration to validate the applicability of theoretical models such as Gassmann's equation for quantitative analysis of time-lapse seismic data.

58 GEOSCIENCES↗

Anomaly Detection and Mitigation for Dynamic Frequency Regulation in Hydropower-Battery Systems: Preprint

Hydropower operators and energy storage providers are increasingly interested in participating in frequency regulation services, driven by the incentives offered by independent system operators, such as the Pennsylvania-New Jersey-Maryland Interconnection (PJM), in a competitive electricity market. This transition, however, unfolds against the backdrop of a modernizing and rapidly digitizing power grid, exposing the integrated legacy infrastructure and vulnerable communication networks to a multitude of cybersecurity threats. These evolving threats not only endanger grid operations but also have the potential to trigger cascading disruptions across the broader grid network and influence regulation markets. This work presents an approach for developing an anomaly detection and mitigation system to address cybersecurity challenges during the participation of a hydropower-integrated battery energy storage system (BESS) in a frequency regulation market. The applied anomaly detector utilizes machine learning algorithms to provide detailed classification of cyber-physical events and provide a comprehensive situation awareness to grid operators. Later, the applied mitigation system triggers predefined corrective actions to minimize the impact of data integrity attacks on the regulation market and system stability. We evaluated the proposed approach on a fully active BESS topology using the slow regulation signal (Reg A) coming from the PJM market. Our simulation results reveal that the proposed approach performs well in detecting data integrity attacks within the allocated time frame and also minimizes the system's instability and economic loss during the participation of hydropower and BESS in the regulation market.

battery energy storage system↗

INTEGRATION OF DATA ANALYTICS WITH SYSTEM HEALTH PROGRAMS

Industry equipment reliability and asset management programs are essential elements that help ensure the safe and economical operation of nuclear power plants. The effectiveness of these programs is addressed in several industry developed and regulatory programs. However, these programs have proven to be labor intensive and expensive. There is an opportunity to significantly enhance the collection, analysis, and use of this information to provide more cost-effective plant operation. Additionally, there is an acute industry need to leverage advanced technology to reduce costs and improve operational effectiveness. The goal of this paper is to provide effective and efficient analytical methods and tools to support risk-informed decisions for the equipment reliability and asset management programs at nuclear power plants. This is accomplished by creating a direct bridge between component health/lifecycle data and decision making (e.g., maintenance scheduling and project prioritization). Here we are supporting typical system engineer decisions regarding maintenance activity scheduling and component ageing management. This is performed in a risk-informed context where herein the term “risk” is broadly constructed to include both plant reliability and economics. This framework combines data analytics tools to analyze equipment reliability data with risk-informed methods designed to support system engineer decisions (e.g., maintenance and replacement schedules, optimal maintenance posture) in a customizable workflow. A challenge is that the structure of this workflow strongly depends on the decision that needs to be made, the type of data available, and the constraints that need to be considered. Current methods are designed to provide specific answers to specific problems; however, these methods might prove to be inadequate even when problem settings slightly change (e.g., different types of requirements, additional dependencies between system reliability and economics). We tackled this challenge by designing framework in a flexible and modular fashion such that the user can assemble and customize his/her own workflow that integrates SSC economic lifecycle models (e.g., maintenance and replacement costs), system reliability models, and optimization methods.

97 - MATHEMATICS AND COMPUTING↗

Performance Evaluation of Vertical Federated Machine Learning Against Adversarial Threats on Wide-Area Control System: Preprint

Federated machine learning (FL) is gaining significant popularity to develop cybersecurity solutions in power grids because of its advanced capability to support decentralized data handing at local devices, its privacy preservation, and its low-bandwidth requirement. However, the evolving adversarial machine learning (AML) threats raise significant concerns for the cybersecurity of FL architectures. The FL-based split neural network (SplitNN) achieves high performance through the decentralized training of local neural network models while preserving data privacy across multiple entities. In this paper, we propose a methodology for evaluating the performance of a vertical FLbased anomaly detector against different types of AML attacks, including denial-of-service attacks, adversarial data injection attacks, and replay attacks on the trained local models deployed in the grid network. For a case study, we consider the modified IEEE 13-bus system, and we develop SplitNN-based binary and multiclass classification models to detect, locate, and identify different types of data integrity attacks on the volt-watt control with two pooling layers: maximum pooling and AvgPool. Our experimental results, computed through performance metrics, reveal that the severity of these AML attacks varies with the integrated pooling mechanism, the type of classification model, and the nature of the cyberattack. Further, the AML attacks negatively impacted the prediction time per sample for the pretrained SplitNN during the online testing.

adversarial threats↗

Diagnostic assessment of reservoir response to fracturing: a case study from Hydraulic Fracturing Test Site (HFTS) in Midland Basin

Abstract This paper outlines a data collection and diagnostics case study involving multiple horizontal shale wells. We look at well production profiles using rate transient analysis, differences in near wellbore complexity, geologic variations within the area of interest, as well as compositional differences in the rocks based on cores obtained from within the stimulated reservoir. The Hydraulic Fracturing Test Site is a multi-well experiment involving stimulation of unconventional shale wells in the southeastern Midland portion of the Permian Basin. The targeted formations include both the upper as well as the middle Wolfcamp formations, also referred alternatively as Wolfcamp A and Wolfcamp B. Data integration and analysis shared in this paper help us understand the various geologic controls impacting well productivity, particularly the wide variance observed between the Wolfcamp A and Wolfcamp B formations. Rate transient analysis indicates similar system permeabilities for stimulated wells. However, we observe higher effective fracture half-lengths for upper Wolfcamp wells. Using observations from 3D seismic interpretations (such as pad scale faults) as well as petrophysical and image log data, we highlight the substantial differences in stimulation as we move along the well laterals from the heel toward the toe sections. These differences are further reconciled with observations from zones with high data density at the core locations through stimulated rock, as well as independent data such as microseismic emissions. At the test site, Wolfcamp A was found to be relatively quartz rich with significant heterogeneity whereas Wolfcamp B is richer in clay and organic content. This impacts the geomechanical characteristics of the rock mass with much higher natural fracture density in the shallower interval. Thus, the fracture growth is more uniform in the deeper interval and more heterogeneous with branching likely in upper interval. Increased complexity also leads to consistently better productivity from the wells in the shallower interval as demonstrated from RTA results. This case study is unique because it provides valuable insights from actual sampling of the stimulated zones in hydraulically fractured wells and helps understand impact of various factors that contribute toward variability in well production. The findings from this study provides insights into need for optimization of completion designs in the various Wolfcamp landing zones, such as optimization of cluster or fracture spacing in various Wolfcamp intervals. In addition, it provides a useful template for data collection and research direction in future field test sites of similar nature in unconventional reservoirs.

Energy & Fuels↗

Spatial data software integration - Merging CAD/CAM/mapping with GIS and image processing

The integration of CAD/CAM/mapping with image processing using geographic information systems (GISs) as the interface is examined. Particular emphasis is given to the development of software interfaces between JPL's Video Image Communication and Retrieval (VICAR)/Imaged Based Information System (IBIS) raster-based GIS and the CAD/CAM/mapping system. The design and functions of the VICAR and IBIS are described. Vector data capture and editing are studied. Various software programs for interfacing between the VICAR/IBIS and CAD/CAM/mapping are presented and analyzed.

Logan, Thomas L.↗

A-Train Data Depot: Integrating and Visualizing Atmospheric Measurements Along the A-Train Tracks

The succession of US and international satellites that follow each other, seconds to minutes apart, across the local afternoon equator crossing is called the ATrain. The A-Train consists of the following satellites, in order of equator crossing: OCO, EOS Aqua, CloudSat, CALIPSO, PARASOL, and EOS Aura. Flying in such formation increases the number of observations, validates observations, and enables coordination between science observations, resulting in a more complete virtual science platform (Kelly, 2000) The goal of this project is to create the first ever A-Train virtual data portal/center, the A-Train Data Depot, to process, archive, access, visualize, analyze and correlate distributed atmosphere measurements from various A-Train instruments along A-Train tracks. The A-Train Data Depot (ATDD) will enable the free movement of remotely located A-Train data so that they are combined to create a consolidated vertical view of the Earth s Atmosphere along the A-Train tracks. Once the infrastructure of the ATDD is in place, it will be easily evolved to serve data from all A-Train data measurements: one stop shopping. The innovative approach of analyzing and visualizing atmospheric profiles along the platforms track (i.e., time) will be accommodated by reusing the GSFC Atmospheric Composition Data and Information Services Center (ACDISC) visualization and analysis tool, GIOVANNI, existing data reduction tools, on-line archwing for fast data access, and Cooperative Institute for Research in the Atmosphere (CRA) data co-registration tools. Initial measurements utilized include CALIPSO lidar backscatter, CloudSat radar reflectivity, clear air relative humidity, water vapor and temperature from AIRS, and cloud properties and aerosols from both MODIS. This will be followed by associated measurements from MLS, OMI, HIRDLS, and TES. Given the independent nature of instrument/platform development, the ATDD project has been met with many interesting challenges that, once resolved, will provide a much greater understanding of the relative flight dynamics and data co-registration of the suite of A-Train instruments, thus greatly increasing the accuracy of A-Train data analysis. Some of these challenges will be discussed. The project s resulting visualizations and analysis illustrate the importance of managing data so that measurements from various missions can be combined to enhance the understanding of the atmosphere. A-Train data management coordination, as performed here, is extremely significant in facilitating the A-Train science of clouds, precipitation, aerosol and chemistry.

Kempler, Steven↗

A-Train Data Depot: Integrating, Visualizing, and Extracting Cloudsat, CALIPSO, MODIS, and AIRS Atmospheric Measurements Along the A-Train Tracks

The succession of US and international Earth observing satellites that follow each other, seconds to minutes apart, across the local afternoon equator crossing is called the ATrain. The A-Train consists of the following satellites, in order of equator crossing: OCO, EOS Aqua, CloudSat, CALIPSO, PARASOL, and EOS Aura. Flying in such formation increases the number of observations, validates observations, and enables coordination between science observations, resulting in a more complete virtual science platform (Kelly, 2000). The goal of this project is to create the first ever A-Train virtual data portal/center, the A-Train Data Depot (ATDD), to process, archive, access, visualize, analyze and correlate distributed atmosphere measurements from various A-Train instruments along A-Train tracks. The ATDD will enable the free movement of remotely located A-Train data so that they are combined to create a consolidated vertical view of the Earth's Atmosphere along the A-Train tracks. Once the infrastructure of the ATDD is in place, it will be easily evolved to serve data from all A-Train data measurements: one stop shopping. The innovative approach of analyzing and visualizing atmospheric profiles along the platforms track (i.e., time) will be accommodated by reusing the GSFC Atmospheric Composition Data and Information Services Center (ACDISC) visualization and analysis tool, GIOVANNI, existing data reduction tools, on-line archiving for fast data access, access to remote data without unnecessary data transfers, and data retrieval by users finding data desirable for further study. Initial measurements utilized include CALIPSO lidar backscatter, CloudSat radar reflectivity, clear air relative humidity, water vapor and temperature from AIRS, and cloud properties and aerosols from both MODIS. This will be foilowed by associated measurements from TVILS, =MI, HIRDLS, sad TES. Given the independent nature of instrumentlplatform development, the ATDD project has been met with many interesting challenges that, once resolved, will provide a much greater understanding of the relative flight dynamics and data co-registration of the suite of A-Train instruments, thus greatly increasing the accuracy of A-Train data analysis. Some of these challenges will be illustrated and discussed. The project's early visualizations and analysis efforts illustrate the importance of managing data so that measurements from various missions can be combined to enhance the understanding of the atmosphere. A-Train data management coordination, as performed here, is extremely significant in facilitating the A-Train science of clouds, precipitation, aerosol and chemistry.

Kempler, Steven↗

Optimization of Energy Flow through Synthetic Metabolic Modules and Regulatory Networks in a Model Photosynthetic Eukaryotic Microbe

Photosynthetic organisms have recently gained considerable attention for a role in development of renewable energy sources. Genome-enabled systems biology methods, coupled with functional and synthetic genomics, present opportunities to develop sustainable and economical applications such as fuel production within the next 10 to 15 years. However, optimization of light-driven metabolism for biomass or biofuel production will require a detailed systems biology understanding of photosynthetic processes and cellular metabolism. Genome-scale metabolic models (GEMs) are at the core of systems analysis of cellular processes and form a common organizational framework for analyses of data resulting from functional genomics experimental work and computational studies. Therefore, there is a clear demand for high quality photosynthetic model organisms and the appropriate computational tools that enable systems analysis of light-driven metabolism. Through research conducted we expanded the currently available repertoire of photosynthetic GEMs to include the commercially valuable model diatom Phaeoctylum tricornutum. Diatoms have a peculiar and distinct evolutionary footprint and represent a major eukaryotic lineage that is taxonomically and functionally distinct from green and red algae and vascular plants. Therefore, the true potential for light-driven metabolism aimed at biofuel production remains poorly understood at a systems level for a large subset of the global diversity of photosynthetic organisms. The metabolic capabilities of P. tricornutum were comparatively modeled with those from other photosynthetic groups in order to elucidate the occurrence of metabolic traits within and between phototrophs. Additionally, this research resulted in significant extension of the COnstraints Based Reconstruction and Analysis (COBRA) Toolbox to accommodate the crucial need for infrastructure required for ‘omics data integration and analysis in the context of genome-scale models. Therefore, the proposed research achieved two important goals. First, within the broad scope of photosynthetic organisms, we functionally compared and, as a result, identified cellular processes that require optimization in order to enable deployment as biofuel feedstock. Second, the proposed research resulted in development of key computational infrastructure, which can be further extended to other biological systems, that is currently lacking but necessary for multiple ‘omics data integration.

59 BASIC BIOLOGICAL SCIENCES↗

A Vision for Coupling Operation of US Fusion Facilities with HPC Systems and the Implications for Workflows and Data Management

The operation of large US Department of Energy (DOE) research facilities, like the DIII-D National Fusion Facility, results in the collection of complex multi-dimensional scientific datasets, both experimental and model-generated. In the future, it is envisioned that integrated data analysis coupled with large-scale high performance computing (HPC) simulations will be used to improve experimental planning and operation. Practically, massive data sets from these simulations provide the physics basis for generation of both reduced semi-analytic and machine-learning-based models. Storage of both HPC simulation datasets (generated from US DOE leadership computing facilities) and experimental datasets presents significant challenges. In this paper, we present a vision for a DOE-wide data management workflow that integrates US DOE fusion facilities with leadership computing facilities. Data persistence and long-term availability beyond the length of allocated projects is essential, particularly for verification and recalibration of artificial intelligence and machine learning (AI/ML) models. Because these data sets are often generated and shared among hundreds of users across multiple leadership computing facility centers, they would benefit from cross-platform accessibility, persistent identifiers (e.g. DOI, or digital object identifier), and provenance tracking. Here, the ability to handle different data access patterns suggests that a combination of low cost, high latency (e.g. for storing ML training sets) and high cost, low latency systems (e.g. for real-time, integrated machine control feedback) may be needed.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DMFS: A Data Migration File System for NetBSD

I have recently developed dmfs, a Data Migration File System, for NetBSD. This file system is based on the overlay file system, which is discussed in a separate paper, and provides kernel support for the data migration system being developed by my research group here at NASA/Ames. The file system utilizes an underlying file store to provide the file backing, and coordinates user and system access to the files. It stores its internal meta data in a flat file, which resides on a separate file system. Our data migration system provides archiving and file migration services. System utilities scan the dmfs file system for recently modified files, and archive them to two separate tape stores. Once a file has been doubly archived, files larger than a specified size will be truncated to that size, potentially freeing up large amounts of the underlying file store. Some sites will choose to retain none of the file (deleting its contents entirely from the file system) while others may choose to retain a portion, for instance a preamble describing the remainder of the file. The dmfs layer coordinates access to the file, retaining user-perceived access and modification times, file size, and restricting access to partially migrated files to the portion actually resident. When a user process attempts to read from the non-resident portion of a file, it is blocked and the dmfs layer sends a request to a system daemon to restore the file. As more of the file becomes resident, the user process is permitted to begin accessing the now-resident portions of the file. For simplicity, our data migration system divides a file into two portions, a resident portion followed by an optional non-resident portion. Also, a file is in one of three states: fully resident, fully resident and archived, and (partially) non-resident and archived. For a file which is only partially resident, any attempt to write or truncate the file, or to read a non-resident portion, will trigger a file restoration. Truncations and writes are blocked until the file is fully restored so that a restoration which only partially succeed does not leave the file in an indeterminate state with portions existing only on tape and other portions only in the disk file system. We chose layered file system technology as it permits us to focus on the data migration functionality, and permits end system administrators to choose the underlying file store technology. We chose the overlay layered file system instead of the null layer for two reasons: first to permit our layer to better preserve meta data integrity and second to prevent even root processes from accessing migrated files. This is achieved as the underlying file store becomes inaccessible once the dmfs layer is mounted. We are quite pleased with how the layered file system has turned out. Of the 45 vnode operations in NetBSD, 20 (forty-four percent) required no intervention by our file layer - they are passed directly to the underlying file store. Of the twenty five we do intercept, nine (such as vop_create()) are intercepted only to ensure meta data integrity. Most of the functionality was concentrated in five operations: vop_read, vop_write, vop_getattr, vop_setattr, and vop_fcntl. The first four are the core operations for controlling access to migrated files and preserving the user experience. vop_fcntl, a call generated for a certain class of fcntl codes, provides the command channel used by privileged user programs to communicate with the dmfs layer.

Studenmund, William↗