Search NASA⌕ Search

SEARCH · Search NASA

Results for “data transfer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

NUMA-Aware Thread Scheduling for Big Data Transfers over Terabits Network Infrastructure

The evergrowing trend of big data has led scientists to share and transfer the simulation and analytical data across the geodistributed research and computing facilities. However, the existing data transfer frameworks used for data sharing lack the capability to adopt the attributes of the underlying parallel file systems (PFS). LADS (Layout-Aware Data Scheduling) is an end-to-end data transfer tool optimized for terabit network using a layout-aware data scheduling via PFS. However, it does not consider the NUMA (Nonuniform Memory Access) architecture. In this paper, we propose a NUMA-aware thread and resource scheduling for optimized data transfer in terabit network. First, we propose distributed RMA buffers to reduce memory controller contention in CPU sockets and then schedule the threads based on CPU socket and NUMA nodes inside CPU socket to reduce memory access latency. We design and implement the proposed resource and thread scheduling in the existing LADS framework. Experimental results showed from 21.7% to 44% improvement with memory-level optimizations in the LADS framework as compared to the baseline without any optimization.

97 MATHEMATICS AND COMPUTING↗

Characterization of throughput on the AXI DMA bus for burst data transfer over Ethernet

cThe Xilinx AXI Direct Memory Access (AXI DMA) module is an efficient solution for medium-speed data transfer in Xilinx SoC FPGAs, supporting data rates greater than 1000 Gbps even in very suboptimal operating modes. It facilitates direct transfer of AXI stream data into processor memory without constant software intervention, which reduces overhead and ensures consistent data logging. By utilizing the FPGA's available memory, large circular buffers (1-5 GiB) are used to buffer data and accommodate network limitations, enabling high-rate data bursts. In this study, we measured the performance of AXI DMA under conditions simulating its lowest practical data transfer speeds. The Arbitrary Length Data Sender was used to transmit AXI stream packets at 32-bit width and 100 MHz frequency, a narrow width and slow speed. Results show that the AXI DMA can transfer up to 3192.76 Mbps with large packet sizes but experiences reduced performance for smaller packets, as low as 2.6 Mbps for 4-byte packets. For Ethernet-limited applications, packet sizes between 8,000 and 16,000 bytes provided optimal transfer speeds of 874 to 1600 Mbps. These findings suggest that the AXI DMA is not the limiting factor in systems where packet sizes exceed 8,000 bytes.

43 PARTICLE ACCELERATORS↗

The Practical Obstacles of Data Transfer: Why researchers still love scp

The importance of computing facilities is heralded every six months with the announcement of the new Top500 list, showcasing the world s fastest supercomputers. Unfortu- nately, with great computing capability does not come great long-term data storage capacity, which often means users must move their data to their local site archive, to remote sites where they may be doing future computation or anal- ysis, or back to their home institution, else face the dreaded data purge that most HPC centers employ to keep utiliza- tion of large parallel filesystems low to manage performance and capacity. At HPC centers, data transfer is crucial to the scientific workflow and will increase in importance as computing systems grow in size. The Energy Sciences Net- work (ESnet) recently launched its fifth generation network, a 100 Gbps high-performance, unclassified national network connecting more than 40 DOE research sites to support scientific research and collaboration. Despite the tenfold increase in bandwidth to DOE research sites amenable to multiple data transfer streams and high throughput, in prac- tice, researchers often under-utilize the network and resort to painfully-slow single stream transfer methods such as scp to avoid the complexity of using multiple stream tools such as GridFTP and bbcp, and contend with frustration from the lack of consistency of available tools between sites. In this study we survey and assess the data transfer methods pro- vided at several DOE supported computing facilities, includ- ing both leadership-computing facilities, connected through ESnet. We present observed transfer rates, suggested opti- mizations, and discuss the obstacles the tools must overcome to receive wide-spread adoption over scp.

Nam, Hai Ah↗

An Overview of High-performance Parallel Big Data transfers over multiple network channels with Transport Layer Security (TLS) and TLS plus Perfect Forward Secrecy (PFS)

This Technical Note provides an overview of high-performance parallel Big Data transfers with and without encryption for data in-transit over multiple network channels. It shows that with the parallel approach, it is feasible to carry out high-performance parallel "encrypted" Big Data transfers without serious impact to throughput. But other impacts, e.g. the energy-consumption part should be investigated. It also explains our rationales of using a statistics-based approach for gaining understanding from test results and for improving the system. The presentation is of high-level nature. Nevertheless, at the end we will pose some questions and identify potentially fruitful directions for future work.

97 MATHEMATICS AND COMPUTING↗

On Performance Prediction of Big Data Transfer in High-performance Networks

Big data generated by large-scale scientific and industrial applications need to be transferred between different geographical locations for remote storage, processing, and analysis. High-speed dedicated connections provisioned in High-performance Networks (HPNs) are increasingly utilized to carry out such big data transfer. HPN management highly relies on an important capability of performance (mainly throughput) prediction to reserve sufficient bandwidth and meanwhile avoid over-provisioning that may result in unnecessary resource waste. This capability is critical to improving the resource (mainly bandwidth) utilization of dedicated connections and meeting various user requests for data transfer. Conventional methods conduct performance prediction by fitting prior observed transfer history with predefined loss functions, without considering unobservable latent factors such as competing loads on end hosts. Such latent factors also have a significant impact on the application-level data transfer performance, which may result in an inaccurate prediction model. In this paper, we first investigate the impact of latent factors and propose a clustering-based method to eliminate their negative impact on performance prediction. We then develop a robust machine learning-based performance predictor by: i) incorporating the proposed latent factor elimination method into data preprocessing, and ii) adopting a customized domain guided loss function. Extensive experimental results show that our predictor achieves significantly higher prediction accuracy than several other state-of-the-art methods.

Liu, Wuji↗

A Framework for International Collaboration on ITER Using Large-Scale Data Transfer to Enable Near-Real-Time Analysis

The global nature of the ITER project along with its projected ~ petabyte per day data generation presents a unique challenge, but also an opportunity for the fusion community to rethink, optimize, and enhance our scientific discovery process. Recognizing this, collaborative research with computational scientists was undertaken over the past several years to create a framework for large-scale data movement across wide-area networks (WANs), to enable global near-real time analysis of fusion data. This would broaden the available computational resources for analysis/simulation, and increase the number of researchers actively participating in experiments. An official demonstration of this framework for fast, large data transfer and real-time analysis was carried out between the KSTAR tokamak in Daejeon, Korea and PPPL in Princeton, USA. Streaming large data transfer, with near real-time movie creation and analysis of the KSTAR Electron Cyclotron Emission imaging (ECEI) data, was performed using the I/O framework ADIOS, and comparisons made at PPPL with simulation results from the XGC1 code. These demonstrations were made possible utilizing an optimized network configuration at PPPL, which achieved over 8.8 Gbps (88% utilization) in throughput tests from NFRI to PPPL. This demonstration showed the feasibility for large-scale data analysis of KSTAR data, and provides a nascent framework to enable use of globally distributed computational and personnel resources in pursuit of scientific knowledge from the ITER experiment.

43 PARTICLE ACCELERATORS↗

Hardware Recommendations for Data Transfer Nodes

These recommendations are in regards to the LLNL Data Transfer Nodes (DTNs), which are used to transfer Climate Storage System (CSS) data between nodes of the ESGF grid. The hardware specifications listed are to be considered a base configuration for a node and can be enhanced in a variety of ways to suit a particular site.

54 ENVIRONMENTAL SCIENCES↗

FTP 100: An Ultra-High Speed Data Transfer Service Over Next Generation 100 Gigabit Per Second Network

Data-intensive applications, including high energy and nuclear physics, astrophysics, climate modeling, nano-scale materials science, genomics, and financing, are expected to generate exabytes of data over the coming years, which must be transferred, visualized, and analyzed by geographically distributed teams of users. High-performance network capabilities must be available to these users at the application level in a transparent, virtualized manner. Moreover, the application users must have the capability to move large datasets from local and remote locations across network environments to their home institutions. To solve these challenges, the main goal of our project is to design and evaluate high-performance data transfer software to support various data-intensive applications. First, we have designed a middleware software that provides access to Remote Direct Memory Access (RDMA) functionalities. This middleware integrates network access, memory management and multitasking in its core design. We address a number of issues related to its efficient implementation, for instance, explicit buffer management and memory registration, and parallelization of RDMA operations, which are vital to delivering the benefit of RDMA to the applications. Built on top of this middleware, an implementation and experimental evaluation of the RDMA-based FTP software, RFTP, is described and evaluated. This application has been implemented by our team to exploit the full capabilities of advanced RDMA mechanisms for ultra-high speed bulk data transfer applications on Energy Sciences Network (ESnet). Second, we designed our data transfer software to optimize TCP/IP based data transfer performance such that RFTP can be fully compatible with today’s Internet. Our kernel optimization techniques with Linux system calls sendfile and splice, can reduce data copy cost. In this report, we summarize the technical challenges of our project, the primary software design methods, the major project milestones achieved, as well as the testbed evaluation work and demonstrations during our project life time.

97 MATHEMATICS AND COMPUTING↗

End-System Aware Peak Link Utilization Transport for Big Data Transfer over Dedicated Connections

High-performance networks provisioning 10/40-Gbps dedicated channels have been rapidly developed andincreasingly deployed to support big data transfers on the order of terabytes or petabytes in large-scale network-intensiveapplications. However, end users have not seen corresponding increase in application throughput mainly because traditional end-to-end transport methods are not optimized for such connections. New congestion or flow control mechanisms are desirable to meet the challenges brought to transport protocol design by dedicated connections. The advent and proliferation of multi-core processors make it now possible to improve application throughput by providing multiple processing and network resources to a single data transfer. Based on the existing PLUT method, we proposed a new dynamic buffer management scheme and an automatic parallelism tuning mechanism to achieve the maximal bottleneck rate. The performance superiority of PLUT is justified by the experimental results collected over a local 10Gbps dedicated connection.

Rao, Nageswara↗

The effects of inlet turbulence and rotor/stator interactions on the aerodynamics and heat transfer of a large-scale rotating turbine model. Volume 2: Heat transfer data tabulation. 15 percent axial spacing

A combined experimental and analytical program was conducted to examine the effects of inlet turbulence on airfoil heat transfer. The experimental portion of the study was conducted in a large-scale (approx 5X engine), ambient temperature, rotating turbine model configured in both single stage and stage-and-a-half arrangements. Heat transfer measurements were obtained using low-conductivity airfoils with miniature thermcouples welded to a thin, electrically heated surface skin. Heat transfer data were acquired for various combinations of low or high inlet turbulence intensity, flow coefficient, first-stator/rotor axial spacing, Reynolds number and relative circumferential position of the first and second stators. Aerodynamic measurements obtained as part of the program include distributions of the mean and fluctuating velocities at the turbine inlet and, for each airfoil row, midspan airfoil surface pressures and circumferential distributions of the downstream steady state pressures and fluctuating velocities. Analytical results include airfoil heat transfer predictions produced using existing 2-D boundary layer computation schemes and an examination of solutions of the unsteady boundary layer equations. The results are reported in four separate volumes, of which this is Volume 2: Heat Transfer Data Tabulation; 15 Percent Axial Spacing.

Dring, R. P.↗

Bulk Data Movement for Climate Dataset: Efficient Data Transfer Management with Dynamic Transfer Adjustment

Many scientific applications and experiments, such as high energy and nuclear physics, astrophysics, climate observation and modeling, combustion, nano-scale material sciences, and computational biology, generate extreme volumes of data with a large number of files. These data sources are distributed among national and international data repositories, and are shared by large numbers of geographically distributed scientists. A large portion of data is frequently accessed, and a large volume of data is moved from one place to another for analysis and storage. One challenging issue in such efforts is the limited network capacity for moving large datasets to explore and manage. The Bulk Data Mover (BDM), a data transfer management tool in the Earth System Grid (ESG) community, has been managing the massive dataset transfers efficiently with the pre-configured transfer properties in the environment where the network bandwidth is limited. Dynamic transfer adjustment was studied to enhance the BDM to handle significant end-to-end performance changes in the dynamic network environment as well as to control the data transfers for the desired transfer performance. We describe the results from the BDM transfer management for the climate datasets. We also describe the transfer estimation model and results from the dynamic transfer adjustment.

97 MATHEMATICS AND COMPUTING↗

SU-E-T-138: Automated Chart Review Module Including Cross-Vendor Data Transfer Verification Developed for IHE-RO Initiative

Purpose: To follow the Integrating Healthcare Enterprise - Radiation Oncology (IHE-RO) initiative of proper cross-vendor technology integration, an automated chart checker (ACC) was developed. ACC compares extracted data from an approved patient plan in the Eclipse treatment planning system (TPS) against data existing in the Mosaiq treatment management system (TMS). ACC automatically analyzes these parameters using built-in quality checklists to provide further aid in chart review. Methods: Eclipse TPS data are obtained using Eclipse scripting API (ESAPI) while Mosaiq TMS data are obtained from a radiotherapy-treatment-planning (RTP) file. Using this information, ACC identifies TPS-TMS discrepancies in 18 primary beam parameters including MU, energy, jaw positions, gantry angle, table angle, accessories, and bolus for up to 31 beams. Next, approximately 40 items from traditional quality checklists are evaluated such as prescription consistency, DRR graticule placement, plan approval status, global max dose, and dose tracking coefficients. Parameters were artificially modified to determine if ACC would detect an error in data transfer and to test each component of quality checklists. Results: Using ESAPI scripting and RTP file-processing, ACC was able to properly aggregate data from TPS and TMS for up to 31 beams. Errors were artificially introduced into each plan parameter, and ACC was able to successfully detect all of them within seconds. Next, ACC was able to successfully detect mistakes in the chart by identifying deviations with its quality checklists, within seconds. Conclusion: ACC effectively addresses the potential issue of faulty cross-vendor data transfer, as described by IHE-RO. In addition, ACC was also able to detect deviations from its built-in quality checklists. ACC is already an invaluable tool for efficient and standardized chart review and will continue to improve as its incorporated checklists become more comprehensive.

60 APPLIED LIFE SCIENCES↗

Preliminary Analysis of STS-3 Entry Heat-Transfer Data for the Orbiter Windward Centerline

A preliminary analysis of heat transfer data on the space shuttle orbiter windward centerline for the STS-3 mission entry is presented. Temperature-time history plots for each measurement location and tabulated wall temperature and convective heating rate data at 21 selected trajectory points are included. The STS-3 flight data are also compared with predictions by two approximation methods for computing convective heat transfer rates in equilibrium air.

Throckmorton, D. A.↗

International data transfer for space very long baseline interferometry

Space very long baseline interferometry (SVLBI) experiments using a TDRSS satellite have successfully demonstrated the capability of using spacecraft to extend the effective baseline length of VLBI observations beyond the diameter of the Earth, thereby improving the resolution for imaging of active galactic nuclei at centimeter wavelengths. As a result, two spacecraft dedicated to SVLBI, VSOP (Japan) and RadioAstron (Russia), are scheduled to be launched into high Earth orbit in 1996 and 1997. The success of these missions depends on the cooperation of the international community in providing support from ground tracking stations, ground radio telescopes, and correlation facilities. The timely exchange and monitoring of data among the participants requires a well-designed and automated international data transfer system. In this paper, we will discuss the design requirements, data types and flows, and the operational responsibilities associated with the SVLBI data transfer system.

Wiercigroch, Alexandria B.↗

Heat-transfer data for hydrogen

Information is given regarding experimental heat-transfer data compiled for the turbulent flow of hydrogen within straight, electrically heated, round cross section tubes. Tube materials, test conditions, parameters studied, and generalized conclusions are presented.

Mc Carthy, J. R.↗

Analysis of entry aerodynamic heat-transfer data for the orbiter wing lower surface

Flight-derived aerodynamic heat-transfer data for the orbiter wing lower surface, from STS-2, -3, and -5, are presented and compared with both ground-based experimental results and state-of-the-art computational flowfield results for a nominal angle of attack of 40 degrees. The flight data clearly show the development of the interference heat-transfer region on the wing lower surface resulting from the downstream effects of the bow-shock/wing-shock interaction. The location of the interference heating region is well correlated with a region of minimum static enthalpy near the boundary-layer edge as predicted by a 3-dimensional, inviscid flowfield computation. The magnitude of the interference heat transfer is no greater than the undisturbed laminar heat transfer which occurs during the 'peak aerodynamic heating' portion of entry.

Throckmorton, D. A.↗