Search NASA⌕ Search

SEARCH · Search NASA

Results for “bulk data transfer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Improving the bulk data transfer experience

Scientific computations and collaborations increasingly rely on the network to provide high-speed data transfer, dissemination of results, access to instruments, support for computational steering, etc. The Energy Sciences Network is establishing a science data network to provide user driven bandwidth allocation. In a shared network environment, some reservations may not be granted due to the lack of available bandwidth on any single path. In many cases, the available bandwidth across multiple paths would be sufficient to grant the reservation. In this paper we investigate how to utilize the available bandwidth across multiple paths in the case of bulk data transfer.

99 GENERAL AND MISCELLANEOUS↗

UltraSciencenet: High- Performance Network Research Test-Bed

The high-performance networking requirements for next generation large-scale applications belong to two broad classes: (a) high bandwidths, typically multiples of 10Gbps, to support bulk data transfers, and (b) stable bandwidths, typically at much lower bandwidths, to support computational steering, remote visualization, and remote control of instrumentation. Current Internet technologies, however, are severely limited in meeting these demands because such bulk bandwidths are available only in the backbone, and stable control channels are hard to realize over shared connections. The UltraScience Net (USN) facilitates the development of such technologies by providing dynamic, cross-country dedicated 10Gbps channels for large data transfers, and 150 Mbps channels for interactive and control operations. Contributions of the USN project are two-fold: (a) Infrastructure Technologies for Network Experimental Facility: USN developed and/or demonstrated a number of infrastructure technologies needed for a national-scale network experimental facility. Compared to Internet, USN's data-plane is different in that it can be partitioned into isolated layer-1 or layer-2 connections, and its control-plane is different in the ability of users and applications to setup and tear down channels as needed. Its design required several new components including a Virtual Private Network infrastructure, a bandwidth and channel scheduler, and a dynamic signaling daemon. The control-plane employs a centralized scheduler to compute the channel allocations and a signaling daemon to generate configuration signals to switches. In a nutshell, USN demonstrated the ability to build and operate a stable national-scale switched network. (b) Structured Network Research Experiments: A number of network research experiments have been conducted on USN that cannot be easily supported over existing network facilities, including test-beds and production networks. It settled an open matter by demonstrating that the performance of switched connections and Multiple Protocol Label Switching tunnels over routed networks are comparable. Furthermore, such connections can be easily peered, and the performance of the resultant hybrid connections is still comparable to the constituent pure connections. USN experiments demonstrated that Infiniband transport can be effectively extended to wide-area connections of thousands of miles, which opens up new opportunities for efficient bulk data transport. USN provided dedicated connections to Cray X1 supercomputer and helped diagnose TCP performance problems which might have been otherwise incorrectly attributed to traffic on shared connections. USN contributed to the development of transport methods for dedicated connections to other traffic. Recently, experiments were conducted to assess the performance of application acceleration devices that employ flow optimization and data compression methods to improve TCP performance.

99 GENERAL AND MISCELLANEOUS↗

BigData Express: Toward Schedulable, Predictable, and High-Performance Data Transfer

Big Data has emerged as a driving force for scientific discoveries. Large scientific instruments (e.g., colliders, and telescopes) generate exponentially increasing volumes of data. To enable scientific discovery, science data must be collected, indexed, archived, shared, and analyzed, typically in a widely distributed, highly collaborative manner. Data transfer is now an essential function for science discoveries, particularly within big data environments. Although significant improvements have been made in the area of bulk data transfer, the currently available data transfer tools and services can not successfully address the high-performance and time-constraint challenges of data transfer required by extreme-scale science applications for the following reasons: disjoint end-to-end data transfer loops, cross-interference between data transfers, and existing data transfer tools and services are oblivious to user requirements (deadline and QoS requirements). Fermilab has been working on the BigData Express project to address these problems. BigData Express seeks to provide a schedulable, predictable, and high-performance data transfer service for big data science. The BigData Express software is being deployed and evaluated at multiple research institutions, which include UMD, StarLight, FNAL, KISTI, KSTAR, SURFnet, Ciena, and other sites. Meanwhile, the BigData Express research team is collaborating with the StarLight International/National Communications Exchange Facility to deploy BigData Express at various research platforms, including Pacific Research Platform, National Research Platform, and Global Research Platform. It is envisioned that we are working toward building a high-performance data transfer federation for big data science.

Lu, Qiming↗

FTP 100: An Ultra-High Speed Data Transfer Service Over Next Generation 100 Gigabit Per Second Network

Data-intensive applications, including high energy and nuclear physics, astrophysics, climate modeling, nano-scale materials science, genomics, and financing, are expected to generate exabytes of data over the coming years, which must be transferred, visualized, and analyzed by geographically distributed teams of users. High-performance network capabilities must be available to these users at the application level in a transparent, virtualized manner. Moreover, the application users must have the capability to move large datasets from local and remote locations across network environments to their home institutions. To solve these challenges, the main goal of our project is to design and evaluate high-performance data transfer software to support various data-intensive applications. First, we have designed a middleware software that provides access to Remote Direct Memory Access (RDMA) functionalities. This middleware integrates network access, memory management and multitasking in its core design. We address a number of issues related to its efficient implementation, for instance, explicit buffer management and memory registration, and parallelization of RDMA operations, which are vital to delivering the benefit of RDMA to the applications. Built on top of this middleware, an implementation and experimental evaluation of the RDMA-based FTP software, RFTP, is described and evaluated. This application has been implemented by our team to exploit the full capabilities of advanced RDMA mechanisms for ultra-high speed bulk data transfer applications on Energy Sciences Network (ESnet). Second, we designed our data transfer software to optimize TCP/IP based data transfer performance such that RFTP can be fully compatible with today’s Internet. Our kernel optimization techniques with Linux system calls sendfile and splice, can reduce data copy cost. In this report, we summarize the technical challenges of our project, the primary software design methods, the major project milestones achieved, as well as the testbed evaluation work and demonstrations during our project life time.

97 MATHEMATICS AND COMPUTING↗

Studying NASA's Transition to Ka-Band Communications for Low Earth Orbit

As the S-band spectrum becomes crowded, future space missions will need to consider moving command and telemetry services to Ka-band. NASA's Space Communications and Navigation (SCaN) Testbed provides a software-defined radio (SDR) platform that is capable of supporting investigation of this service transition. The testbed contains two S-band SDRs and one Ka-band SDR. Over the past year, SCaN Testbed has demonstrated Ka-band communications capabilities with NASAs Tracking and Data Relay Satellite System (TDRSS) using both open- and closed-loop antenna tracking profiles. A number of technical areas need to be addressed for successful transition to Ka-band. The smaller antenna beamwidth at Ka-band increases the criticality of antenna pointing, necessitating closed loop tracking algorithms and new techniques for received power estimation. Additionally, the antenna pointing routines require enhanced knowledge of spacecraft position and attitude for initial acquisition, versus an S-band antenna. Ka-band provides a number of technical advantages for bulk data transfer. Unlike at S-band, a larger bandwidth may be available for space missions, allowing increased data rates. The potential for high rate data transfer can also be extended for direct-to-ground links through use of variable or adaptive coding and modulation. Specific examples of Ka-band research from SCaN Testbeds first year of operation will be cited, such as communications link performance with TDRSS, and the effects of truss flexure on antenna pointing.

space communications↗

Studying NASA's Transition to Ka-Band Communications for Low Earth Orbit

As the S-band spectrum becomes crowded, future space missions will need to consider moving command and telemetry services to Ka-band. NASAs Space Communications and Navigation (SCaN) Testbed provides a software-defined radio (SDR) platform that is capable of supporting investigation of this service transition. The testbed contains two S-band SDRs and one Ka-band SDR. Over the past year, SCaN Testbed has demonstrated Ka-band communications capabilities with NASAs Tracking and Data Relay Satellite System (TDRSS) using both open- and closed-loop antenna tracking profiles. A number of technical areas need to be addressed for successful transition to Ka-band. The smaller antenna beamwidth at Ka-band increases the criticality of antenna pointing, necessitating closed loop tracking algorithms and new techniques for received power estimation. Additionally, the antenna pointing routines require enhanced knowledge of spacecraft position and attitude for initial acquisition, versus an S-band antenna. Ka-band provides a number of technical advantages for bulk data transfer. Unlike at S-band, a larger bandwidth may be available for space missions, allowing increased data rates. The potential for high rate data transfer can also be extended for direct-to-ground links through use of variable or adaptive coding and modulation. Specific examples of Ka-band research from SCaN Testbeds first year of operation will be cited, such as communications link performance with TDRSS, and the effects of truss flexure on antenna pointing.

International Space Station↗

Models of TCP in high-BDP environments and their experimental validation

In recent years, the computer networking community has seen a steady growth in bandwidth-delay products (BDPs). Several TCP variants were created to combat the shortcomings of legacy TCP when it comes to operation in high-BDP environments. These variants, among which are CUBIC, STCP, and H-TCP, have been extensively studied in some empirical contexts, and some analytical models exist for CUBIC and STCP. However, since these studies have been conducted, BDPs have risen even more, and new bulk data transfer tools have emerged that utilize multiple parallel TCP streams. In view of these new developments, it is imperative to revisit the question: Which congestion control algorithms are best adapted to current networking environments? In order to help resolve this question, we contribute the following: (i) using first principles, we develop a general throughput-prediction framework that takes into account buffer sizes and maximum window constraints; (ii) we validate the models using measurements and achieve low prediction errors; (iii) we note differences in TCP dynamics between two experimental configurations and find one of them to be significantly more deterministic than the other; we also find that CUBIC and H-TCP outperform STCP, especially when multiple streams are used; and (iv) we present preliminary results for modelling multiple TCP streams for CUBIC and STCP.

Vardoyan, G.↗

Efficient Bulk Data Replication for the Earth System Grid

The Earth System Grid (ESG) community faces the difficult challenge of managing the distribution of massive data sets to thousands of scientists around the world. To move data replicas efficiently, the ESG has developed a data transfer management tool called the Bulk Data Mover (BDM). We describe the performance results of the current system and plans towards extending the techniques developed so far for the up- coming project, in which the ESG will employ advanced networks to move multi-TB datasets with the ulti- mate goal of helping researchers understand climate change and its potential impacts on world ecology and society.

97 MATHEMATICS AND COMPUTING↗

Bulk Data Movement for Climate Dataset: Efficient Data Transfer Management with Dynamic Transfer Adjustment

Many scientific applications and experiments, such as high energy and nuclear physics, astrophysics, climate observation and modeling, combustion, nano-scale material sciences, and computational biology, generate extreme volumes of data with a large number of files. These data sources are distributed among national and international data repositories, and are shared by large numbers of geographically distributed scientists. A large portion of data is frequently accessed, and a large volume of data is moved from one place to another for analysis and storage. One challenging issue in such efforts is the limited network capacity for moving large datasets to explore and manage. The Bulk Data Mover (BDM), a data transfer management tool in the Earth System Grid (ESG) community, has been managing the massive dataset transfers efficiently with the pre-configured transfer properties in the environment where the network bandwidth is limited. Dynamic transfer adjustment was studied to enhance the BDM to handle significant end-to-end performance changes in the dynamic network environment as well as to control the data transfers for the desired transfer performance. We describe the results from the BDM transfer management for the climate datasets. We also describe the transfer estimation model and results from the dynamic transfer adjustment.

97 MATHEMATICS AND COMPUTING↗

Error Detection and Error Classification: Failure Awareness in Data Transfer Scheduling

Data transfer in distributed environment is prone to frequent failures resulting from back-end system level problems, like connectivity failure which is technically untraceable by users. Error messages are not logged efficiently, and sometimes are not relevant/useful from users point-of-view. Our study explores the possibility of an efficient error detection and reporting system for such environments. Prior knowledge about the environment and awareness of the actual reason behind a failure would enable higher level planners to make better and accurate decisions. It is necessary to have well defined error detection and error reporting methods to increase the usability and serviceability of existing data transfer protocols and data management systems. We investigate the applicability of early error detection and error classification techniques and propose an error reporting framework and a failure-aware data transfer life cycle to improve arrangement of data transfer operations and to enhance decision making of data transfer schedulers.

97 MATHEMATICS AND COMPUTING↗

CCD data acquisition systems at Lick and Keck Observatories

This paper will describe and compare two distinct but related CCD data acquisition systems (DAS) currently under development at Lick and Keck Observatories. Although these two systems have a number of major architectural differences, they share a considerable amount of common hardware and software. Both of these new systems build on a large body of proven software that is the foundation of the existing CCD DAS currently in use at Lick Observatory. Both will provide support for reading up to four on-chip amplifiers per CCD and/or reading out mosaics of CCD chips. In addition, they will provide the capability for interactive, real-time adjustment of CCD waveforms for engineering purposes. Each of these two systems is composed of three major subsystems: (1) an instrument computer and its software; (2) a data capture computer and its software; and (3) a CCD/dewar controller and its software. The instrument computer is a Unix workstation, and the functions it provides include user interfaces, the interactive real-time display of CCD images, and the recording of image and FITS header data to disk and/or tape. The data capture computer is responsible for the packaging and high-speed transfer of the CCD pixel data stream into a bulk RAM, and the subsequent transfer of this data to the instrument computer. The CCD/dewar controller generates the waveforms for clocking the CCD, digitizes the pixel data, and transmits it via high-speed link to the data capture computer. It is also responsible for monitoring and controlling the dewar temperature and cryogen levels. Given the number of different types of processors and high-speed data links employed in both systems, a major emphasis of this paper will be on the various forms of interprocessor communications utilized for data transfer and distributed process synchronization.

Kibrick, R. I.↗

The image-processing system for the Earth Resources Technology Satellite.

Description of the image-processing system for the Earth Resources Technology Satellite (ERTS), which will process video-tape recordings received from the satellite into imagery and digitized video data suitable for earth-resource study and analysis. The system is made up of three subsystems. The bulk processor produces 70-mm images and digitized records, corrected for systematic geometric and radiometric errors as well as for sensor-calibration and other errors. The precision processor modifies the bulk images to significantly increase geometric precision. The special processor transfers either bulk or precision data to computer-compatible tape.

Johnson, R. H.↗

Volume-energy parameters for heat transfer to supercritical fluids

Reduced Nusselt numbers of supercritical fluids from different sources were grouped by several volume-energy parameters. A modified bulk expansion parameter was introduced based on a comparative analysis of data scatter. Heat transfer experiments on liquefied methane were conducted under near-critical conditions in order to confirm the usefulness of the parameters. It was experimentally revealed that heat transfer characteristics of near-critical methane are similar to those of hydrogen. It was shown that the modified bulk expansion parameter and the Gibbs-energy parameter grouped the heat transfer data of hydrogen, oxygen and methane including the present data on near-critical methane. It was also indicated that the effects of surface roughness on heat transfer were very important in grouping the data of high Reynolds numbers.

A. Kumakawa↗

Optimizing End-to-End Big Data Transfers over Terabits Network Infrastructure

While future terabit networks hold the promise of significantly improving big-data motion among geographically distributed data centers, significant challenges must be overcome even on today's 100 gigabit networks to realize end-to-end performance. Multiple bottlenecks exist along the end-to-end path from source to sink, for instance, the data storage infrastructure at both the source and sink and its interplay with the wide-area network are increasingly the bottleneck to achieving high performance. In this study, we identify the issues that lead to congestion on the path of an end-to-end data transfer in the terabit network environment, and we present a new bulk data movement framework for terabit networks, called LADS. LADS exploits the underlying storage layout at each endpoint to maximize throughput without negatively impacting the performance of shared storage resources for other users. LADS also uses the Common Communication Interface (CCI) in lieu of the sockets interface to benefit from hardware-level zero-copy, and operating system bypass capabilities when available. It can further improve data transfer performance under congestion on the end systems using buffering at the source using flash storage. With our evaluations, we show that LADS can avoid congested storage elements within the shared storage resource, improving input/output bandwidth, and data transfer rates across the high speed networks. We also investigate the performance degradation problems of LADS due to I/O contention on the parallel file system (PFS), when multiple LADS tools share the PFS. We design and evaluate a meta-scheduler to coordinate multiple I/O streams while sharing the PFS, to minimize the I/O contention on the PFS. Finally, with our evaluations, we observe that LADS with meta-scheduling can further improve the performance by up to 14 percent relative to LADS without meta-scheduling.

97 MATHEMATICS AND COMPUTING↗

The performance of cable braids and terminations to lightning induced transients

The latest specification detailing the test waveforms for indirect lightning transients as applied to aircraft wiring systems specify very high voltages and currents. Although considerable data exists for measuring cable screen leakage using such methods as surface transfer impedance and bulk cable injection, there is little data on the likely core transient level that is likely to be induced from these threats. In particular, the new Waveform 5 at very high current levels (10 kA) is reputed to cause severe cable damage. A range of representative cables were made with various screen termination techniques and screening levels. These were tested first to determine their relative screening performance and then they were subjected to lightning transient testing to all the specified waveforms. Core voltages were measured for each test. Tests were also performed on bundles with fewer wires to determine the failure criteria with Waveform 5 and these tests also include flat conductor cables. The test showed that correctly terminated cable bundles performed well in all the tests and would provide a high level of protection to the electronic systems. The use of overbraides, provided the individual screens are well terminated, appears to be unnecessary.

Crofts, David↗

Experimental Investigation of Gas-Side Performance of a Compact Finned-Tube Heat Exchanger

Heat-transfer and pressure-drop data were obtained experimentally for the gas side of a liquid-metal to air, compact finned-tube heat exchanger. The heat exchanger was fabricated from 0.185-inch Inconel tubing in an inline array. The fins were made of 310 stainless-steel- clad copper with a total thickness of 0.010 inch, and the fin pitch was 15.3 fins per inch. The liquid used as the heating medium was sodium. The heat-exchanger inlet gas temperature was varied from 5100 to 1260 R by burning JP fuel for airflow rates of 0.4 to 10.5 pounds per second corresponding to an approximate Reynolds number range of 300 to 9000. The sodium inlet temperature was held at 1400 R with the exception of a few runs taken at 1700 and 1960 R. The maximum ratio of surface temperature to air bulk temperature was 1.45. Friction-factor data with heat transfer were best represented by a single line when the density and viscosity of Reynolds number were evaluated at the average film temperature. At the lower Reynolds numbers reported, the friction data with heat transfer plotted slightly above the friction data without heat transfer. The density of the friction factor was calculated at the average bulk temperature. Heat-transfer results of this investigation were correlated by evaluating the physical properties of air (specific heat, viscosity, and thermal conductivity) at the film temperature.

Gedeon, Louis↗