Search NASA⌕ Search

SEARCH · Search NASA

Results for “data replication”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Blinding scheme for the scale-dependence bias signature of local primordial non-Gaussianity for DESI 2024

The next generation of spectroscopic surveys is expected to achieve an unprecedented level of accuracy in the measurement of cosmological parameters. To avoid confirmation bias and thereby improve the reliability of these results, blinding procedures become a standard practice in the cosmological analyses of such surveys. Blinding is especially crucial when the impact of observational systematics is important relative to the cosmological signal, and a detection of that signal would have significant implications. This is the case for local primordial non-gaussianity, as probed by the scale-dependent bias of the galaxy power spectrum at large scales that are heavily sensitive to the dependence of the target selection on the imaging quality, known as imaging systematics. We propose a blinding method for the scale-dependent bias signature of local primordial non-gaussianity at the density field level which consists in generating a set of weights for the data that replicate the scale-dependent bias. The applied blinding is predictable, and can be straightforwardly combined with other catalog-level blinding procedures that have been designed for the baryon acoustic oscillation and redshift space distortion signals. The procedure is validated through simulations that replicate data from the first year of observation of the Dark Energy Spectroscopic Instrument, but may find applications to other upcoming spectroscopic surveys.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1990

Surrogate oracles, generalized dependency and simpler models

Software reliability models require the sequence of interfailure times from the debugging process as input. It was previously illustrated that using data from replicated debugging could greatly improve reliability predictions. However, inexpensive replication of the debugging process requires the existence of a cheap, fast error detector. Laboratory experiments can be designed around a gold version which is used as an oracle or around an n-version error detector. Unfortunately, software developers can not be expected to have an oracle or to bear the expense of n-versions. A generic technique is being investigated for approximating replicated data by using the partially debugged software as a difference detector. It is believed that the failure rate of each fault has significant dependence on the presence or absence of other faults. Thus, in order to discuss a failure rate for a known fault, the presence or absence of each of the other known faults needs to be specified. Also, in simpler models which use shorter input sequences without sacrificing accuracy are of interest. In fact, a possible gain in performance is conjectured. To investigate these propositions, NASA computers running LIC (RTI) versions are used to generate data. This data will be used to label the debugging graph associated with each version. These labeled graphs will be used to test the utility of a surrogate oracle, to analyze the dependent nature of fault failure rates and to explore the feasibility of reliability models which use the data of only the most recent failures.

Wilson, Larry↗
NASA NTRS2007

Simple, Script-Based Science Processing Archive

The Simple, Scalable, Script-based Science Processing (S4P) Archive (S4PA) is a disk-based archival system for remote sensing data. It is based on the data-driven framework of S4P and is used for data transfer, data preprocessing, metadata generation, data archive, and data distribution. New data are automatically detected by the system. S4P provides services such as data access control, data subscription, metadata publication, data replication, and data recovery. It comprises scripts that control the data flow. The system detects the availability of data on an FTP (file transfer protocol) server, initiates data transfer, preprocesses data if necessary, and archives it on readily available disk drives with FTP and HTTP (Hypertext Transfer Protocol) access, allowing instantaneous data access. There are options for plug-ins for data preprocessing before storage. Publication of metadata to external applications such as the Earth Observing System Clearinghouse (ECHO) is also supported. S4PA includes a graphical user interface for monitoring the system operation and a tool for deploying the system. To ensure reliability, S4P continuously checks stored data for integrity, Further reliability is provided by tape backups of disks made once a disk partition is full and closed. The system is designed for low maintenance, requiring minimal operator oversight.

Lynnes, Christopher↗
NASA NTRS1991

Performance analysis of static locking in replicated distributed database systems

Data replications and transaction deadlocks can severely affect the performance of distributed database systems. Many current evaluation techniques ignore these aspects, because it is difficult to evaluate through analysis and time consuming to evaluate through simulation. Here, a technique is discussed that combines simulation and analysis to closely illustrate the impact of deadlock and evaluate performance of replicated distributed databases with both shared and exclusive locks.

Kuang, Yinghong↗
NASA NTRS1991

Performance analysis of static locking in replicated distributed database systems

Data replication and transaction deadlocks can severely affect the performance of distributed database systems. Many current evaluation techniques ignore these aspects, because it is difficult to evaluate through analysis and time consuming to evaluate through simulation. A technique is used that combines simulation and analysis to closely illustrate the impact of deadlock and evaluate performance of replicated distributed database with both shared and exclusive locks.

Kuang, Yinghong↗
NASA NTRS1996

RAID Unbound: Storage Fault Tolerance in a Distributed Environment

Mirroring, data replication, backup, and more recently, redundant arrays of independent disks (RAID) are all technologies used to protect and ensure access to critical company data. A new set of problems has arisen as data becomes more and more geographically distributed. Each of the technologies listed above provides important benefits; but each has failed to adapt fully to the realities of distributed computing. The key to data high availability and protection is to take the technologies' strengths and 'virtualize' them across a distributed network. RAID and mirroring offer high data availability, which data replication and backup provide strong data protection. If we take these concepts at a very granular level (defining user, record, block, file, or directory types) and them liberate them from the physical subsystems with which they have traditionally been associated, we have the opportunity to create a highly scalable network wide storage fault tolerance. The network becomes the virtual storage space in which the traditional concepts of data high availability and protection are implemented without their corresponding physical constraints.

Ritchie, Brian↗
NASA NTRS1996

Design and Implementation of Replicated Object Layer

One of the widely used techniques for construction of fault tolerant applications is the replication of resources so that if one copy fails sufficient copies may still remain operational to allow the application to continue to function. This thesis involves the design and implementation of an object oriented framework for replicating data on multiple sites and across different platforms. Our approach, called the Replicated Object Layer (ROL) provides a mechanism for consistent replication of data over dynamic networks. ROL uses the Reliable Multicast Protocol (RMP) as a communication protocol that provides for reliable delivery, serialization and fault tolerance. Besides providing type registration, this layer facilitates distributed atomic transactions on replicated data. A novel algorithm called the RMP Commit Protocol, which commits transactions efficiently in reliable multicast environment is presented. ROL provides recovery procedures to ensure that site and communication failures do not corrupt persistent data, and male the system fault tolerant to network partitions. ROL will facilitate building distributed fault tolerant applications by performing the burdensome details of replica consistency operations, and making it completely transparent to the application.Replicated databases are a major class of applications which could be built on top of ROL.

Koka, Sudhir↗
NASA NTRS1993

Cirrus microphysics observations made during FIRE 2: Small particles, high concentrations, and probe comparisons

Aircraft observations of cirrus cloud microphysics were made near Coffeyville, Kansas during Nov. and Dec. 1991. Cloud microphysics measurements were made using both a PMS 2DC probe and an ice particle replicator, both were mounted on the UND Citation aircraft. Intercomparison is made of the size, area, and ice mass spectra determined from these probes. The PMS 2DC undercounts particles for D less than 70 microns and the replicator oversizes particles for D greater than 150 microns, at least when column rosettes are encountered. High concentrations of particles with D less than 50 microns are noted in selected portions of the 22 Nov. 91 replicator data set. Relations between the maximum dimension of a crystal and its shadow area (known as area dimensional relationships) are computed from the PMS data. Area and mass dimensional relationships are used to give a simple analytical expression for computing the wavelength dependent absorption coefficient averaged over a size bin. Calculations based upon the replicator data show that crystals with D less than 50 microns contribute significantly to the solar extinction and infrared absorption coefficients during some time intervals.

Arnott, W. P.↗
NASA NTRS1989

Exploiting replication in distributed systems

Techniques are examined for replicating data and execution in directly distributed systems: systems in which multiple processes interact directly with one another while continuously respecting constraints on their joint behavior. Directly distributed systems are often required to solve difficult problems, ranging from management of replicated data to dynamic reconfiguration in response to failures. It is shown that these problems reduce to more primitive, order-based consistency problems, which can be solved using primitives such as the reliable broadcast protocols. Moreover, given a system that implements reliable broadcast primitives, a flexible set of high-level tools can be provided for building a wide variety of directly distributed application programs.

Birman, Kenneth P.↗
DOE OSTI2013

OpenMSI: A High-Performance Web-Based Platform for Mass Spectrometry Imaging

Mass spectrometry imaging (MSI) enables researchers to directly probe endogenous molecules directly within the architecture of the biological matrix. Unfortunately, efficient access, management, and analysis of the data generated by MSI approaches remain major challenges to this rapidly developing field. Despite the availability of numerous dedicated file formats and software packages, it is a widely held viewpoint that the biggest challenge is simply opening, sharing, and analyzing a file without loss of information. Here we present OpenMSI, a software framework and platform that addresses these challenges via an advanced, high-performance, extensible file format and Web API for remote data access (http://openmsi.nersc.gov). The OpenMSI file format supports storage of raw MSI data, metadata, and derived analyses in a single, self-describing format based on HDF5 and is supported by a large range of analysis software (e.g., Matlab and R) and programming languages (e.g., C++, Fortran, and Python). Careful optimization of the storage layout of MSI data sets using chunking, compression, and data replication accelerates common, selective data access operations while minimizing data storage requirements and are critical enablers of rapid data I/O. The OpenMSI file format has shown to provide >2000-fold improvement for image access operations, enabling spectrum and image retrieval in less than 0.3 s across the Internet even for 50 GB MSI data sets. To make remote high-performance compute resources accessible for analysis and to facilitate data sharing and collaboration, we describe an easy-to-use yet powerful Web API, enabling fast and convenient access to MSI data, metadata, and derived analysis results stored remotely to facilitate high-performance data analysis and enable implementation of Web based data sharing, visualization, and analysis.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2016

SOL: An end-to-end solution for real-world remote monitoring systems

This paper introduces SOL, which is both an efficient data representation for sensor measurements and network statistics, and a complete low-power wireless sensor management system that builds around it. A SOL system consists of multiple low-power wireless mesh networks in which motes connected to sensors and actuators send data to a single server. It offers multi-tier data replication and the associated data recovery. SOL is used in 3 pilot deployments for micro-climate monitoring, building automation and agriculture applications. In conjunction with Metronome Systems' NeoMote and Manager devices, SOL is a turn-key ready-to-deploy end-to-end system.

47 OTHER INSTRUMENTATION↗
NASA NTRS1991

Measuring the effects of distributed database models on transaction availability measures

Data distribution, data replication, and system reliability are key factors in determining the availability measures for transactions in distributed database systems. In order to simplify the evaluation of these measures, database designers and researchers tend to make unrealistic assumptions about these factors. Here, the effect of such assumptions on the computational complexity and accuracy of such evaluations is investigated. A database system is represented with five parameters related to the above factors. Probabilistic analysis is employed to evaluate the availability of read-one and read-write transactions. Both the read-one/write-all and the majority-read/majority-write replication control policies are considered. It is concluded that transaction availability is more sensitive to variations in degrees of replication, less sensitive to data distribution, and insensitive to reliability variations in a heterogeneous system. The computational complexity of the evaluations is found to be mainly determined by the chosen distributed database model, while the accuracy of the results are not so much dependent on the models.

Mukkamala, Ravi↗
DOE OSTI2022

SCALE Calculations Replicating ARH-600 Data

This report documents Standardized Computer Analyses for Licensing Evaluation (SCALE) calculations replicating data from the ARH-600 criticality handbook. These calculations complement those reproduced with MCNP in PRC-NS-00009. Calculations were performed with SCALE version 6.1, using the v7-238 cross section library. The original ARH-600 data were calculated by the Atlantic Richfield Hanford (ARH) Company starting in the 1960s, and Monte Carlo N-Particle (MCNP) calculations were performed by the Hanford Plateau Remediation Company in 2010. This report is not intended to replace final analysis of fissile systems by qualified criticality personnel.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2019

dCache - Joining the noWORM Storage Club

For over a decade, dCache.ORG has provided robust software, called dCache, that is used at more than 80 universities and research institutes around the world, allowing these sites to provide reliable storage services for the WLCG experiments and many other scientific communities. The flexible architecture of dCache allows running it in a wide variety of configurations and platforms - from all-in-one Raspberry-Pi up to hundreds of nodes in multi-petabyte infrastructures. The life cycle of scientific data is well defined - collected, processed, archived and finally deleted, when it’s not needed anymore. Moreover, during all those stages the data is never modified: either the original data is used, or new derived data is produced. With this knowledge, dCache was designed to handle immutable files as efficiently as possible. Data replication, HSM connectivity and data-server independent operations are only possible due to the immutable nature of stored data. Nowadays many commercial vendors provide such write-once-read-many or WORM storage systems, as they become more and more demanded with grown demand of audio, photo and video content in the web. On the other hand by providing standard NFSv4.1 interface dCache is often used as a general-purpose file-system, especially by new communities, like photon scientists or microbiologists. Although many users are aware of data immutability, some applications and use cases still require in-place updates of stored files. To satisfy new requirements some fundamental changes have to be applied to dCache’s core design. However, new developments must not compromise any aspect of existing functionality. In this presentation we will show new developments in dCache to turn it into a regular file system. We will discuss the challenges to build a distributed storage system, ‘life’ with POSIX compliance, handling of multiple replicas and backward compatibility by providing WORM and noWORM capabilities within the same storage system.

Mkrtchyan, Tigran↗
NASA NTRS1978

Automatic flight performance of a transport airplane on complex microwave landing system paths

During this demonstration the microwave landing system was utilized to provide the terminal configured vehicle B-737 airplane with guidance for automatic control on complex, curved descending paths with precision turns into short final approaches terminating in landing and roll-out, even when subjected to strong and gusty tail- and cross-wind components and severe wind shear. The data collected from more than fifty approach flights during the demonstration provided an opportunity to analyze airplane flight performance on a statistical basis rather than on a single flight record basis as is customarily done with limited data replication. Mean and standard deviation data are presented for approach flight path tracking parameters. In addition, the adverse wind conditions encountered during these flights are described using three-dimensional wind vector characteristics computed from the extensive on-board sensor data.

Walsh, T. M.↗
DOE OSTI2015

Performance analysis of data intensive cloud systems based on data management and replication: a survey

As we delve deeper into the ‘Digital Age’, we witness an explosive growth in the volume, velocity, and variety of the data available on the Internet. For example, in 2012 about 2.5 quintillion bytes of data was created on a daily basis that originated from myriad of sources and applications including mobiledevices, sensors, individual archives, social networks, Internet of Things, enterprises, cameras, software logs, etc. Such ‘Data Explosions’ has led to one of the most challenging research issues of the current Information and Communication Technology era: how to optimally manage (e.g., store, replicated, filter, and the like) such large amount of data and identify new ways to analyze large amounts of data for unlocking information. It is clear that such large data streams cannot be managed by setting up on-premises enterprise database systems as it leads to a large up-front cost in buying and administering the hardware and software systems. Therefore, next generation data management systems must be deployed on cloud. The cloud computing paradigm provides scalable and elastic resources, such as data and services accessible over the Internet Every Cloud Service Provider must assure that data is efficiently processed and distributed in a way that does not compromise end-users’ Quality of Service (QoS) in terms of data availability, data search delay, data analysis delay, and the like. In the aforementioned perspective, data replication is used in the cloud for improving the performance (e.g., read and write delay) of applications that access data. Through replication a data intensive application or system can achieve high availability, better fault tolerance, and data recovery. In this paper, we survey data management and replication approaches (from 2007 to 2011) that are developed by both industrial and research communities. The focus of the survey is to discuss and characterize the existing approaches of data replication and management that tackle the resource usage and QoS provisioning with different levels of efficiencies. Moreover, the breakdown of both influential expressions (data replication and management) to provide different QoS attributes is deliberated. Furthermore, the performance advantages and disadvantages of data replication and management approaches in the cloud computing environments are analyzed. Open issues and future challenges related to data consistency, scalability, load balancing, processing and placement are also reported.

Malik, Saif Ur Rehman↗
DOE OSTI2019

Wireless Sensor Networks for Improved Snow Water Equivalent and Runoff Estimates

We leverage the frontiers of the Internet of Things technology in a recently developed end-to-end wireless sensor network (WSN) system that samples, collects, stores, and displays mountain hydrology measurements in near real-time. At the core of the system lies an ultra-low power, radio channel-hoping, and self-organizing mesh that allows for remote autonomous sampling of snow. Such properties, combined with a rugged weather-sealed design of the devices and multi-level data replication, provides reliable real-time data at spatial and temporal scales previously impractical to achieve in mountain environments. The system was deployed at three 1 km 2 sites across the North Fork of the Feather River basin with a cluster of 12 sensor nodes for each location. Measurements show that existing operational autonomous systems are non-representative spatially, with biases that can reach up to 50%. A comparison between a wet and dry year showed that snow depths exhibit strong multi-scale inter-year spatial stationarity with major rank conservation. Temporally dense analysis using elastic net regression shows that dominant features at the sub-km 2 scale are site-dependent and differ from the watershed scale. Newly introduced explanatory variables, based on the nearest neighbor with a Landsat assimilated historical product, consistently explained up to 90% of the variance in the watershed-scale SWE for both years. At two WSN sites, lagged cross-correlation of snowmelt with stream flow measurements showed a significant improvement of up to 100% compared with existing systems, suggesting that WSNs can be instrumental in improving runoff forecasting and water management.

47 OTHER INSTRUMENTATION↗