Search NASA⌕ Search

SEARCH · Search NASA

Results for “bulk data transfer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Improving the bulk data transfer experience

Scientific computations and collaborations increasingly rely on the network to provide high-speed data transfer, dissemination of results, access to instruments, support for computational steering, etc. The Energy Sciences Network is establishing a science data network to provide user driven bandwidth allocation. In a shared network environment, some reservations may not be granted due to the lack of available bandwidth on any single path. In many cases, the available bandwidth across multiple paths would be sufficient to grant the reservation. In this paper we investigate how to utilize the available bandwidth across multiple paths in the case of bulk data transfer.

99 GENERAL AND MISCELLANEOUS↗

UltraSciencenet: High- Performance Network Research Test-Bed

The high-performance networking requirements for next generation large-scale applications belong to two broad classes: (a) high bandwidths, typically multiples of 10Gbps, to support bulk data transfers, and (b) stable bandwidths, typically at much lower bandwidths, to support computational steering, remote visualization, and remote control of instrumentation. Current Internet technologies, however, are severely limited in meeting these demands because such bulk bandwidths are available only in the backbone, and stable control channels are hard to realize over shared connections. The UltraScience Net (USN) facilitates the development of such technologies by providing dynamic, cross-country dedicated 10Gbps channels for large data transfers, and 150 Mbps channels for interactive and control operations. Contributions of the USN project are two-fold: (a) Infrastructure Technologies for Network Experimental Facility: USN developed and/or demonstrated a number of infrastructure technologies needed for a national-scale network experimental facility. Compared to Internet, USN's data-plane is different in that it can be partitioned into isolated layer-1 or layer-2 connections, and its control-plane is different in the ability of users and applications to setup and tear down channels as needed. Its design required several new components including a Virtual Private Network infrastructure, a bandwidth and channel scheduler, and a dynamic signaling daemon. The control-plane employs a centralized scheduler to compute the channel allocations and a signaling daemon to generate configuration signals to switches. In a nutshell, USN demonstrated the ability to build and operate a stable national-scale switched network. (b) Structured Network Research Experiments: A number of network research experiments have been conducted on USN that cannot be easily supported over existing network facilities, including test-beds and production networks. It settled an open matter by demonstrating that the performance of switched connections and Multiple Protocol Label Switching tunnels over routed networks are comparable. Furthermore, such connections can be easily peered, and the performance of the resultant hybrid connections is still comparable to the constituent pure connections. USN experiments demonstrated that Infiniband transport can be effectively extended to wide-area connections of thousands of miles, which opens up new opportunities for efficient bulk data transport. USN provided dedicated connections to Cray X1 supercomputer and helped diagnose TCP performance problems which might have been otherwise incorrectly attributed to traffic on shared connections. USN contributed to the development of transport methods for dedicated connections to other traffic. Recently, experiments were conducted to assess the performance of application acceleration devices that employ flow optimization and data compression methods to improve TCP performance.

99 GENERAL AND MISCELLANEOUS↗

BigData Express: Toward Schedulable, Predictable, and High-Performance Data Transfer

Big Data has emerged as a driving force for scientific discoveries. Large scientific instruments (e.g., colliders, and telescopes) generate exponentially increasing volumes of data. To enable scientific discovery, science data must be collected, indexed, archived, shared, and analyzed, typically in a widely distributed, highly collaborative manner. Data transfer is now an essential function for science discoveries, particularly within big data environments. Although significant improvements have been made in the area of bulk data transfer, the currently available data transfer tools and services can not successfully address the high-performance and time-constraint challenges of data transfer required by extreme-scale science applications for the following reasons: disjoint end-to-end data transfer loops, cross-interference between data transfers, and existing data transfer tools and services are oblivious to user requirements (deadline and QoS requirements). Fermilab has been working on the BigData Express project to address these problems. BigData Express seeks to provide a schedulable, predictable, and high-performance data transfer service for big data science. The BigData Express software is being deployed and evaluated at multiple research institutions, which include UMD, StarLight, FNAL, KISTI, KSTAR, SURFnet, Ciena, and other sites. Meanwhile, the BigData Express research team is collaborating with the StarLight International/National Communications Exchange Facility to deploy BigData Express at various research platforms, including Pacific Research Platform, National Research Platform, and Global Research Platform. It is envisioned that we are working toward building a high-performance data transfer federation for big data science.

Lu, Qiming↗

FTP 100: An Ultra-High Speed Data Transfer Service Over Next Generation 100 Gigabit Per Second Network

Data-intensive applications, including high energy and nuclear physics, astrophysics, climate modeling, nano-scale materials science, genomics, and financing, are expected to generate exabytes of data over the coming years, which must be transferred, visualized, and analyzed by geographically distributed teams of users. High-performance network capabilities must be available to these users at the application level in a transparent, virtualized manner. Moreover, the application users must have the capability to move large datasets from local and remote locations across network environments to their home institutions. To solve these challenges, the main goal of our project is to design and evaluate high-performance data transfer software to support various data-intensive applications. First, we have designed a middleware software that provides access to Remote Direct Memory Access (RDMA) functionalities. This middleware integrates network access, memory management and multitasking in its core design. We address a number of issues related to its efficient implementation, for instance, explicit buffer management and memory registration, and parallelization of RDMA operations, which are vital to delivering the benefit of RDMA to the applications. Built on top of this middleware, an implementation and experimental evaluation of the RDMA-based FTP software, RFTP, is described and evaluated. This application has been implemented by our team to exploit the full capabilities of advanced RDMA mechanisms for ultra-high speed bulk data transfer applications on Energy Sciences Network (ESnet). Second, we designed our data transfer software to optimize TCP/IP based data transfer performance such that RFTP can be fully compatible with today’s Internet. Our kernel optimization techniques with Linux system calls sendfile and splice, can reduce data copy cost. In this report, we summarize the technical challenges of our project, the primary software design methods, the major project milestones achieved, as well as the testbed evaluation work and demonstrations during our project life time.

97 MATHEMATICS AND COMPUTING↗

Models of TCP in high-BDP environments and their experimental validation

In recent years, the computer networking community has seen a steady growth in bandwidth-delay products (BDPs). Several TCP variants were created to combat the shortcomings of legacy TCP when it comes to operation in high-BDP environments. These variants, among which are CUBIC, STCP, and H-TCP, have been extensively studied in some empirical contexts, and some analytical models exist for CUBIC and STCP. However, since these studies have been conducted, BDPs have risen even more, and new bulk data transfer tools have emerged that utilize multiple parallel TCP streams. In view of these new developments, it is imperative to revisit the question: Which congestion control algorithms are best adapted to current networking environments? In order to help resolve this question, we contribute the following: (i) using first principles, we develop a general throughput-prediction framework that takes into account buffer sizes and maximum window constraints; (ii) we validate the models using measurements and achieve low prediction errors; (iii) we note differences in TCP dynamics between two experimental configurations and find one of them to be significantly more deterministic than the other; we also find that CUBIC and H-TCP outperform STCP, especially when multiple streams are used; and (iv) we present preliminary results for modelling multiple TCP streams for CUBIC and STCP.

Vardoyan, G.↗

Efficient Bulk Data Replication for the Earth System Grid

The Earth System Grid (ESG) community faces the difficult challenge of managing the distribution of massive data sets to thousands of scientists around the world. To move data replicas efficiently, the ESG has developed a data transfer management tool called the Bulk Data Mover (BDM). We describe the performance results of the current system and plans towards extending the techniques developed so far for the up- coming project, in which the ESG will employ advanced networks to move multi-TB datasets with the ulti- mate goal of helping researchers understand climate change and its potential impacts on world ecology and society.

97 MATHEMATICS AND COMPUTING↗

Bulk Data Movement for Climate Dataset: Efficient Data Transfer Management with Dynamic Transfer Adjustment

Many scientific applications and experiments, such as high energy and nuclear physics, astrophysics, climate observation and modeling, combustion, nano-scale material sciences, and computational biology, generate extreme volumes of data with a large number of files. These data sources are distributed among national and international data repositories, and are shared by large numbers of geographically distributed scientists. A large portion of data is frequently accessed, and a large volume of data is moved from one place to another for analysis and storage. One challenging issue in such efforts is the limited network capacity for moving large datasets to explore and manage. The Bulk Data Mover (BDM), a data transfer management tool in the Earth System Grid (ESG) community, has been managing the massive dataset transfers efficiently with the pre-configured transfer properties in the environment where the network bandwidth is limited. Dynamic transfer adjustment was studied to enhance the BDM to handle significant end-to-end performance changes in the dynamic network environment as well as to control the data transfers for the desired transfer performance. We describe the results from the BDM transfer management for the climate datasets. We also describe the transfer estimation model and results from the dynamic transfer adjustment.

97 MATHEMATICS AND COMPUTING↗

Error Detection and Error Classification: Failure Awareness in Data Transfer Scheduling

Data transfer in distributed environment is prone to frequent failures resulting from back-end system level problems, like connectivity failure which is technically untraceable by users. Error messages are not logged efficiently, and sometimes are not relevant/useful from users point-of-view. Our study explores the possibility of an efficient error detection and reporting system for such environments. Prior knowledge about the environment and awareness of the actual reason behind a failure would enable higher level planners to make better and accurate decisions. It is necessary to have well defined error detection and error reporting methods to increase the usability and serviceability of existing data transfer protocols and data management systems. We investigate the applicability of early error detection and error classification techniques and propose an error reporting framework and a failure-aware data transfer life cycle to improve arrangement of data transfer operations and to enhance decision making of data transfer schedulers.

97 MATHEMATICS AND COMPUTING↗

Optimizing End-to-End Big Data Transfers over Terabits Network Infrastructure

While future terabit networks hold the promise of significantly improving big-data motion among geographically distributed data centers, significant challenges must be overcome even on today's 100 gigabit networks to realize end-to-end performance. Multiple bottlenecks exist along the end-to-end path from source to sink, for instance, the data storage infrastructure at both the source and sink and its interplay with the wide-area network are increasingly the bottleneck to achieving high performance. In this study, we identify the issues that lead to congestion on the path of an end-to-end data transfer in the terabit network environment, and we present a new bulk data movement framework for terabit networks, called LADS. LADS exploits the underlying storage layout at each endpoint to maximize throughput without negatively impacting the performance of shared storage resources for other users. LADS also uses the Common Communication Interface (CCI) in lieu of the sockets interface to benefit from hardware-level zero-copy, and operating system bypass capabilities when available. It can further improve data transfer performance under congestion on the end systems using buffering at the source using flash storage. With our evaluations, we show that LADS can avoid congested storage elements within the shared storage resource, improving input/output bandwidth, and data transfer rates across the high speed networks. We also investigate the performance degradation problems of LADS due to I/O contention on the parallel file system (PFS), when multiple LADS tools share the PFS. We design and evaluate a meta-scheduler to coordinate multiple I/O streams while sharing the PFS, to minimize the I/O contention on the PFS. Finally, with our evaluations, we observe that LADS with meta-scheduling can further improve the performance by up to 14 percent relative to LADS without meta-scheduling.

97 MATHEMATICS AND COMPUTING↗

LADS: Optimizing Data Transfers using Layout-Aware Data Scheduling

While future terabit networks hold the promise of signifi- cantly improving big-data motion among geographically distributed data centers, significant challenges must be overcome even on today s 100 gigabit networks to real- ize end-to-end performance. Multiple bottlenecks exist along the end-to-end path from source to sink. Data stor- age infrastructure at both the source and sink and its in- terplay with the wide-area network are increasingly the bottleneck to achieving high performance. In this paper, we identify the issues that lead to congestion on the path of an end-to-end data transfer in the terabit network en- vironment, and we present a new bulk data movement framework called LADS for terabit networks. LADS ex- ploits the underlying storage layout at each endpoint to maximize throughput without negatively impacting the performance of shared storage resources for other users. LADS also uses the Common Communication Interface (CCI) in lieu of the sockets interface to use zero-copy, OS-bypass hardware when available. It can further im- prove data transfer performance under congestion on the end systems using buffering at the source using flash storage. With our evaluations, we show that LADS can avoid congested storage elements within the shared stor- age resource, improving I/O bandwidth, and data transfer rates across the high speed networks.

Data Management↗

Fluid-Driven Mass Transfer During Retrograde Metamorphism and Exhumation of the UHP Western Gneiss Region Terrane, Norway

Dehydration reactions within subducted oceanic crust are important for fluid-mediated element transfer within the subducting plate and potentially to the mantle wedge. The effects of metamorphic reactions and fluid flow on element recycling that occur during retrogression and exhumation of subducted continental crust from mantle depths are poorly understood. We study two metabasite pods with fresh eclogite cores and retrogressed amphibolite-facies rims and surrounding host gneiss within the Western Gneiss Region (WGR), Norway, to better understand element mobility and mass transfer during exhumation of subducted continental crust. Bulk-rock data were collected from samples taken across the pod and into the host gneiss. Phengite breakdown in eclogite and epidote recrystallization in veins and/or gneiss within pod cores contributed large ion lithophile elements and REE to retrogressed eclogite closest to the pod cores. In gneiss hosting the pods, phengite and epidote breakdown provided fluid that mediated elemental transfer and redistribution to the pod rim or tail. Compared to the studied pod in the southern WGR, the pod in the northern WGR underwent higher P-T conditions, partial melting and higher strain rates. This resulted in the infiltration of external fluid farther into the pod interior from the rim and facilitated larger mass gain in trace elements in the amphibolite tail of the pod relative to fresh eclogite in the core. The results show clear evidence for retrogression dehydration reactions driving significant fluid-mediated element redistribution as observed on the outcrop scale during exhumation following ultrahigh-pressure metamorphism, which directly impacts element signatures within the exhuming crust.

58 GEOSCIENCES↗

Bulk Data Mover

Bulk Data Mover (BDM) is a high-level data transfer management tool. BDM handles the issue of large variance in file sizes and a big portion of small files by managing the file transfers with optimized transfer queue and concurrency management algorithms. For example, climate simulation data sets are characterized by large volume of files with extreme variance in file sizes. The BDN achieves high performance using a variety of techniques, including multi-threaded concurrent transfer connections, data channel caching, load balancing over multiple transfer servers, and storage i/o pre-fetching. Logging information from the BDM is collected and analyzed to study the effectiveness of the transfer management algorithms. The BDM can accept a request composed of multiple files or an entire directory. The request also contains the target site and directory where the replicated files will reside. If a directory is provided at the source, then the BDM will replicate the structure of the source directory at the target site. The BDM is capable of transferring multiple files concurrently as well as using parallel TCP streams. The optimal level of concurrency or parallel streams depends on the bandwidth capacity of the storage systems at both ends of the transfer as well as achievable bandwidth of the wide-area network. Hardware req.-PC, MAC, Multi-platform & Workstation; Software req.: Compile/version-Java 1.50_x or above; Type of files: source code, executable modules, installation instructions other, user guide; URL: http://sdm.lbl.gov/bdm/

Natarajan, Vijaya↗

Advance Resource Provisioning in Bulk Data Scheduling

Today?s scientific and business applications generate mas- sive data sets that need to be transferred to remote sites for sharing, processing, and long term storage. Because of increasing data volumes and enhancement in current net- work technology that provide on-demand high-speed data access between collaborating institutions, data handling and scheduling problems have reached a new scale. In this paper, we present a new data scheduling model with ad- vance resource provisioning, in which data movement operations are defined with earliest start and latest comple- tion times. We analyze time-dependent resource assign- ment problem, and propose a new methodology to improve the current systems by allowing researchers and higher-level meta-schedulers to use data-placement as-a-service, so they can plan ahead and submit transfer requests in advance. In general, scheduling with time and resource conflicts is NP-hard. We introduce an efficient algorithm to organize multiple requests on the fly, while satisfying users? time and resource constraints. We successfully tested our algorithm in a simple benchmark simulator that we have developed, and demonstrated its performance with initial test results.

97 MATHEMATICS AND COMPUTING↗

Integrated Process Testing of MSR Salt Spill Accidents

Part of the licensing process for new nuclear reactors requires vendors to assess the potential consequences of identified accident scenarios using accident progression modeling. The accident scenario that will likely be evaluated by all molten salt reactor (MSR) developers is a spill of radionuclide-bearing fuel salt onto the reactor containment floor (i.e., a salt spill accident). The development of accident progression models requires experimental data to inform which processes to incorporate, to enable the calculation of parameters to model these processes, and to validate the model predictions. The data should quantify the sensitivities of key processes (e.g., molten salt spreading, heat transfer, containment structure corrosion, radionuclide vaporization, and aerosol generation) towards the initial conditions of the spill, the ambient environment, and the features of the containment. In addition, results from integrated process tests that quantify coupled processes are required to validate systems-level models. This report documents results from integrated process tests conducted on simulated molten salt spill accidents. The generated experimental data simultaneously quantify the heat transfer behavior of the spilled salt, compositional changes to the bulk salt, and the release of surrogate fission products from the spilled salt as aerosol particles. All tests that were conducted used FLiNaK doped with surrogate fission products, and the variables that were evaluated included the initial salt temperature and the concentration of surrogate fission products present in the salt. The major accomplishments of this work include identifying surrogate fuel salt compositions that provide insight into the dispersal behavior of radionuclides of potential significance to the source term, employing previously developed methods and measurement techniques to simultaneously measure key processes, generating data on the coupled processes of molten salt heat transfer and surrogate fission product release as aerosol particles, demonstrating new test methods for real-time monitoring of the flow rate of the spill and aerosol size quantification in an argon atmosphere, and developing a mass transfer model for cesium and iodine release from molten FLiNaK to provide insight into aerosol formation by vapor condensation. The same methodology applied herein can be employed to study different salt compositions of interest to MSR developers, different environmental conditions, and other variables that are relevant to postulated accident scenarios. The insights gained from these integrated process tests conducted at a laboratory scale will be incorporated into future integral effects tests conducted at an engineering scale.

20 FOSSIL-FUELED POWER PLANTS↗

Functional Connectivity of Red Chlorophylls in Cyanobacterial Photosystem I Revealed by Fluence-Dependent Transient Absorption

External stressors modulate the oligomerization state of photosystem I (PSI) in cyanobacteria. The number of red chlorophylls (Chls), pigments lower in energy than the P700 reaction center, depends on the oligomerization state of PSI. Here, we use ultrafast transient absorption spectroscopy to interrogate the effective connectivity of the red Chls in excitonic energy pathways in trimeric PSI in native thylakoid membranes of the model cyanobacterium Synechocystis sp. PCC 6803, including emergent dynamics, as red Chls increase in number and proximity. Fluence-dependent dynamics indicate singlet–singlet annihilation within energetically connected red Chl sites in the PSI antenna but not within bulk Chl sites on the picosecond time scale. These data support picosecond energy transfer between energetically connected red Chl sites as the physical basis of singlet–singlet annihilation. The time scale of this energy transfer is faster than predicted by Förster resonance energy transfer calculations, raising questions about the physical mechanism of the process. Our results indicate distinct strategies to steer excitations through the PSI antenna; the red Chls present a shallow reservoir that direct excitations away from P 700 , extending the time to trapping by the reaction center.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

The ultrafast Einstein–de Haas effect

The Einstein-de Haas effect was originally observed in a landmark experiment demonstrating that the angular momentum associated with aligned electron spins in a ferromagnet can be converted to mechanical angular momentum by reversing the direction of magnetization using an external magnetic field. A related problem concerns the timescale of this angular momentum transfer. Experiments have established that intense photoexcitation in several metallic ferromagnets leads to a drop in magnetization on a timescale shorter than 100 femtoseconds—a phenomenon called ultrafast demagnetization. Although the microscopic mechanism for this process has been hotly debated, the key question of where the angular momentum goes on these femtosecond timescales remains unanswered. Here we use femtosecond time-resolved X-ray diffraction to show that most of the angular momentum lost from the spin system upon laser-induced demagnetization of ferromagnetic iron is transferred to the lattice on sub-picosecond timescales, launching a transverse strain wave that propagates from the surface into the bulk. By fitting a simple model of the X-ray data to simulations and optical data, we estimate that the angular momentum transfer occurs on a timescale of 200 femtoseconds and corresponds to 80 per cent of the angular momentum that is lost from the spin system. Finally, our results show that interaction with the lattice has an essential role in the process of ultrafast demagnetization in this system.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗