Search NASA⌕ Search

SEARCH · Search NASA

Results for “data transfer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Data communications in a parallel active messaging interface of a parallel computer

Data communications in a parallel active messaging interface (`PAMI`) of a parallel computer, the PAMI composed of data communications endpoints, each endpoint including a specification of data communications parameters for a thread of execution on a compute node, including specifications of a client, a context, and a task, endpoints coupled for data communications through the PAMI and through data communications resources, including receiving in an origin endpoint of the PAMI a SEND instruction, the SEND instruction specifying a transmission of transfer data from the origin endpoint to a first target endpoint; transmitting from the origin endpoint to the first target endpoint a Request-To-Send (`RTS`) message advising the first target endpoint of the location and size of the transfer data; assigning by the first target endpoint to each of a plurality of target endpoints separate portions of the transfer data; and receiving by the plurality of target endpoints the transfer data.

97 MATHEMATICS AND COMPUTING↗

MSFC institutional area network and ATM technology

The New Institutional Area Network (NEWIAN) at Marshall supports over 5000 end users with access to 26 file servers providing work presentation services. It is comprised of some 150 Ethernet LAN's interconnected by bridges/routers which are in turn connected to servers over two dual FDDI rings. The network supports various higher level protocols such as IP, IPX, AppleTalk (AT), and DECNet. At present IPX and AT protocols packets are routed, and IP protocol packets are bridged; however, work is in progress to route all IP packets. The impact of routing IP packets on network operation is examined. Broadband Integrated Services Data Network (BISDN), presently at various stages of development, is intended to provide voice, video, and data transfer services over a single network. BISDN will use asynchronous transfer mode (ATM) as a data transfer technique which provides for transmission, multiplexing, switching, and relaying of small size data units called cells. Limited ATM Wide Area Network (WAN) services are offered by Wiltel, AT&T, Sprint, and others. NASA is testing a pilot ATM WAN with a view to provide Program Support Communication Network services using ATM. ATM supports wide range of data rates and quality of service requirements. It is expected that ATM switches will penetrate campus networks as well. However, presently products in these areas are at various stages of development and standards are not yet complete. We examine development of ATM to help assess its role in the evolution of NEWIAN.

Amin, Ashok T.↗

The Modern Research Data Portal: A Design Pattern for Networked, Data-Intensive Science

Here we describe best practices for providing convenient, high-speed, secure access to large data via research data portals. We capture these best practices in a new design pattern, the Modern Research Data Portal, that disaggregates the traditional monolithic web-based data portal to achieve orders-of-magnitude increases in data transfer performance, support new deployment architectures that decouple control logic from data storage, and reduce development and operations costs. We introduce the design pattern; explain how it leverages high-performance Science DMZs and cloud-based data management services; review representative examples at research laboratories and universities, including both experimental facilities and supercomputer sites; describe how to leverage Python APIs for authentication, authorization, data transfer, and data sharing; and use coding examples to demonstrate how these APIs can be used to implement a range of research data portal capabilities. Sample code at a companion web site, https://docs.globus.org/mrdp, provides application skeletons that readers can adapt to realize their own research data portals.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

A comparison of WFTA and FFT programs

Bounds on the minimum number of data transfers (i.e., loads and stores) required by WFTA (Winograd Fourier Transform Algorithm) and FFT programs are presented. The analysis is applicable to those general-purpose computers with a small number of general processor registers (e.g., the IBM370, PDP-11, etc.). It is shown that the 1008-point WFTA requires about 21% more data transfers than the 1024-point radix-4 FFT; on the other hand, the 120-point WFTA has about 22% fewer data transfers than the 128-point radix-2 FFT. Finally, comparisons of the 'total' program execution times (multiplications, additions, and data transfers, but not indexing or permutations) are presented.

Nawab, H.↗

The NPOESS Preparatory Project (NPP) Science Data Segment (SDS) Data Depository and Distribution Element (SD3E) System Architecture

The National Polar-orbiting Operational Environmental Satellite System (NPOESS), the U.S. Government's future low-Earth orbiting satellite system, will monitor global weather and environmental conditions. Serving as a risk reduction for NPOESS, the NPOESS Preparatory Project (NPP) will provide remotely sensed atmospheric, land, ocean, ozone, and sounder data that will serve the meteorological and global climate change scientific communities. The National Aeronautics and Space Administration (NASA) NPP Science Data Segment's (SDS) primary role is to independently assess the quality of the NPP science and environmental data records for their ability to support climate research. The SDS is composed of nine elements; an input element that receives data from the operational agencies and acts as a buffer, a calibration analysis element, five elements devoted to measurement based quality assessment, an element used to test algorithmic improvements, and an element that provides overall science direction. Each element requires a set of sensor specific science data products for their evaluation. There are four NPP sensors that will be flown on the NPP observatory. They are the Visible Infrared Imagining Radiometer Suite (VIIRS), the Advanced Technology Microwave Sounder (ATMS), the Cross-Track Infrared Sounder (CrIS), and the Ozone Mapper/Profiler Suite (OMPS). It is estimated that these four sensors combined will make daily data requests for approximately six terabytes of NPP science products from the operational data providers. As a result, issues associated with duplicate data requests, data transfers of large volumes of diverse products, and data transfer failures raised concerns with respect to the network traffic and bandwidth consumption. Therefore, a central data broker system for receiving and buffering data requests and data products for the SDS was developed. The data element for this system is called the SDS Data Depository and Distribution Element (SD3E). It supports science mission data assessment by assuring the timely and validated acquisition and subsequent transfer of the NPP Science Mission data to the SDS Elements and NPP Science Team. The six science elements that interface with the SD3E span across the NASA Goddard Space Flight Center (GSFC), the NASA Jet Propulsion Laboratory (JPL), and the University of Wisconsin. As the primary communication vehicle for the science elements and science team, the SD3E has an interface to the operational data providers: National Environment Satellite, Data, and Information Service (NESDIS) Interface Data Processing System (IDPS) and the National Oceanic Atmospheric Administration's (NOAA) Comprehensive Large Array-data Stewardship system (CLASS) Archive Data System (ADS), that are responsible for product generation and archive and distribution respectively. The SD3E is designed to be a semi-customizable and semi-automated system. This system is designed to provide flexibility and ease of use for the science users in accessing the latest data products by creating a rolling data cache that temporarily stores the products locally before transferring the data to the SDS Measurement based elements for the land, ocean, atmosphere, sounder, and ozone. This paper describes the design and architecture of one of the nine SDS elements, the SD3E, and how this system has provided a mechanism for efficient data exchange, how it has helped in alleviating some of the network traffic and usage, and how it has contributed to reducing operational costs.

Ho, Evelyn L.↗

Data transmission by quantum matter wave modulation

Abstract Classical communication schemes exploiting wave modulation are the basis of our information era. Quantum information techniques with photons enable future secure data transfer in the dawn of decoding quantum computers. Here we demonstrate that also matter waves can be applied for secure data transfer. Our technique allows the transmission of a message by a quantum modulation of coherent electrons in a biprism interferometer. The data is encoded in the superposition state by a Wien filter introducing a longitudinal shift between separated matter wave packets. The transmission receiver is a delay line detector performing a dynamic contrast analysis of the fringe pattern. Our method relies on the Aharonov–Bohm effect but does not shift the phase. It is demonstrated that an eavesdropping attack will terminate the data transfer by disturbing the quantum state and introducing decoherence. Furthermore, we discuss the security limitations of the scheme due to the multi-particle aspect and propose the implementation of a key distribution protocol that can prevent active eavesdropping.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Delta: Data Reduction for Integrated Application Workflows

Integrated Application Workflows (IAWs) run multiple simulation workflow components concurrently on an HPC resource connecting these components using compute area resources and compensating for any performance or data processing rate mismatches. These IAWs require high frequency and high volume data transfers between compute nodes and staging area nodes during the lifetime of a large parallel computation. The available network band-width between the two areas may not be enough to efficiently support the data movement. As the processing power available to compute resources increases, the requirements for this data transfer will become more difficult to satisfy and perhaps will not be satisfiable at all since network capabilities are not expanding at a comparable rate. Furthermore, energy consumption in HPC environments is expected to grow by an order of magnitude as exascale systems become a reality. The energy cost of moving large amounts of data frequently will contribute to this issue. It is necessary to reduce the volume of data without reducing the quality of data when it is being processed and analyzed. Delta resolves the issue by addressing the lifetime data transfer operations. Delta removes subsequent identical copies of already transmitted data during transfers and restores those copies once the data has reached the destination. Delta is able to identify duplicated information and determine the most space efficient way to represent it. Initial tests show about 50% reduction in data movement while maintaining the same data quality and transmission frequency.

97 MATHEMATICS AND COMPUTING↗

Burner liner thermal/structural load modelling

The objective of this program is to develop a thermal data transfer computer program module for the burner liner thermal structural load modeling program. This will be accomplished by (1) reviewing existing methodologies for thermal data transfer and selecting three heat transfer codes for application in this program; (2) evaluating the selected codes to establish criteria for developing a computer program module to transfer thermal data from the heat transfer codes to selected stress analysis codes; (3) developing the automated thermal load transfer module; and (4) verifying and documenting the module. In aircraft turbine engine hot section components, cyclic thermal stresses are the most important damage mechanism. Consequently, accurate and reliable prediction of thermal loads is essential to improving durability. To achieve this goal, a considerable effort over the past 20 years has been devoted to the acquisition of engine temperature test data, as well as the development of accurate, reliable, and efficient computer codes for the prediction of steady state and transient temperatures and for the calculation of elastic and inelastic cyclic stresses and strains in hot section components. There is a need for continued development of these codes, because the availability of more accurate analysis techniques for complex configurations has enabled engine designers to use more sophisticated designs to achieve higher cycle efficiency and reduce weight.

Maffeo, R. J.↗

Burner liner thermal/structural load modelling

The objective of this program is to develop a thermal data transfer computer program module for the Burner Liner Thermal-Structural Load Modelling Program. This will be accomplished by reviewing existing methodologies for thermal data transfer and selecting three heat transfer codes for application in this program, evaluating the selected codes to establish criteria for developing a computer program module to transfer thermal data from the heat transfer codes to selected stress analysis codes, developing the automated thermal load transfer module, and verifying and documenting the module. The overall objectives of this thermal transfer module are that it handle independent mesh configurations, perform the transfer in an accurate and efficient fashion and that the total system be flexible for future improvements.

Maffeo, R. J.↗

Learning concave-convex profiles of data transport over dedicated connections

Dedicated data transport infrastructures are increasingly being deployed to support distributed big-data and high-performance computing scenarios. These infrastructures employ data transfer nodes that use sophisticated software stacks to support network transport among sites, which often house distributed file and storage systems. Throughput measurements collected over such infrastructures for a range of round trip times (RTTs) reflect the underlying complex end-to-end connections, and have revealed dichotomous throughput profiles as functions of RTT. In particular, concave regions of throughput profiles at lower RTTs indicate near-optimal performance, and convex regions at higher RTTs indicate bottlenecks due to factors such as buffer or credit limits. We present a machine learning method that explicitly infers these concave and convex regions and transitions between them using sigmoid functions. We also provide distribution-free confidence estimates for the generalization error of these concave-convex profile estimates. Throughput profiles for data transfers over 10 Gbps connections with 0–366 ms RTT provide important performance insights, including the near optimality of transfers performed with the XDD tool between XFS filesystems, and the performance limits of wide-area Lustre extensions using LNet routers. A direct application of generic machine learning packages does not adequately highlight these critical performance regions or provide as precise confidence estimates.

Rao, Nageswara↗

GPU Direct I/O with HDF5

Exascale HPC systems are being designed with accelerators, such as GPUs, to accelerate parts of applications. In machine learning workloads as well as large-scale simulations that use GPUs as accelerators, the CPU (or host) memory is currently used as a buffer for data transfers between GPU (or device) memory and the file system. If the CPU does not need to operate on the data, then this is sub-optimal because it wastes host memory by reserving space for duplicated data. Furthermore, this “bounce buffer” approach wastes CPU cycles spent on transferring data. A new technique, NVIDIA GPUDirect Storage (GDS), can eliminate the need to use the host memory as a bounce buffer. Thereby, it becomes possible to transfer data directly between the device memory and the file system. This direct data path shortens latency by omitting the extra copy and enables higher-bandwidth. To take full advantage of GDS in existing applications, it is necessary to provide support with existing I/O libraries, such as HDF5 and MPI-IO, which are heavily used in applications. In this paper, we describe our effort of integrating GDS with HDF5, the top I/O library at NERSC and at DOE leadership computing facilities. We design and implement this integration using a HDF5 Virtual File Driver (VFD). The GDS VFD provides a file system abstraction to the application that allows HDF5 applications to perform I/O without the need to move data between CPUs and GPUs explicitly. We compare performance of the HDF5 GDS VFD with explicit data movement approaches and demonstrate superior performance with the GDS method.

Ravi, J↗

Analysis of transmission error effects on the transfer of real-time simulation data

An analysis was made to determine the effect of transmission errors on the quality of data transferred from the Terminal Area Air Traffic Model to a remote site. Data formating schemes feasible within the operational constraints of the data link were proposed and their susceptibility to both random bit error and to noise burst were investigated. It was shown that satisfactory reliability is achieved by a scheme formating the simulation output into three data blocks which has the priority data triply redundant in the first block in addition to having a retransmission priority on that first block when it is received in error.

Credeur, L.↗

Realized ecological forecast through an interactive Ecological Platform for Assimilating Data (EcoPAD, v1.0) into models

Abstract. Predicting future changes in ecosystem services is not only highly desirable but is also becoming feasible as several forces (e.g., available big data, developed data assimilation (DA) techniques, and advanced cyber-infrastructure) are converging to transform ecological research into quantitative forecasting. To realize ecological forecasting, we have developed an Ecological Platform for Assimilating Data (EcoPAD, v1.0) into models. EcoPAD (v1.0) is a web-based software system that automates data transfer and processing from sensor networks to ecological forecasting through data management, model simulation, data assimilation, forecasting, and visualization. It facilitates interactive data–model integration from which the model is recursively improved through updated data while data are systematically refined under the guidance of model. EcoPAD (v1.0) relies on data from observations, process-oriented models, DA techniques, and the web-based workflow. We applied EcoPAD (v1.0) to the Spruce and Peatland Responses Under Climatic and Environmental change (SPRUCE) experiment in northern Minnesota. The EcoPAD-SPRUCE realizes fully automated data transfer, feeds meteorological data to drive model simulations, assimilates both manually measured and automated sensor data into the Terrestrial ECOsystem (TECO) model, and recursively forecasts the responses of various biophysical and biogeochemical processes to five temperature and two CO 2 treatments in near-real time (weekly). Forecasting with EcoPAD-SPRUCE has revealed that mismatches in forecasting carbon pool dynamics are more related to model (e.g., model structure, parameter, and initial value) than forcing variables, opposite to forecasting flux variables. EcoPAD-SPRUCE quantified acclimations of methane production in response to warming treatments through shifted posterior distributions of the CH 4 :CO 2 ratio and the temperature sensitivity ( Q 10 ) of methane production towards lower values. Different case studies indicated that realistic forecasting of carbon dynamics relies on appropriate model structure, correct parameterization, and accurate external forcing. Moreover, EcoPAD-SPRUCE stimulated active feedbacks between experimenters and modelers to identify model components to be improved and additional measurements to be taken. It has become an interactive model–experiment (ModEx) system and opens a novel avenue for interactive dialogue between modelers and experimenters. Altogether, EcoPAD (v1.0) acts to integrate multiple sources of information and knowledge to best inform ecological forecasting.

54 ENVIRONMENTAL SCIENCES↗

The PhEDEx next-gen website

PhEDEx is the data-transfer management solution written by CMS. It consists of agents running at each site, a website for presentation of information, and a web-based data-service for scripted access to information. The website allows users to monitor the progress of data-transfers, the status of site agents and links between sites, and the overall status and behaviour of everything about PhEDEx. It also allows users to make and approve requests for data-transfers and for deletion of data. It is the main point-of-entry for all users wishing to interact with PhEDEx. For several years, the website has consisted of a single perl program with about 10K SLOC. This program has limited capabilities for exploring the data, with only coarse filtering capabilities and no context-sensitive awareness. Graphical information is presented as static images, generated on the server, with no interactivity. It is also not well connected to the rest of the PhEDEx codebase, since much of it was written before the data-service was developed. All this makes it hard to maintain and extend. We are re-implementing the website to address these issues. The UI is being rewritten in Javascript, replacing most of the server-side code. We are using the YUI toolkit to provide advanced features and context-sensitive interaction, and will adopt a Javascript charting library for generating graphical representations client-side. This relieves the server of much of its load, and automatically improves server-side security. The Javascript components can be re-used in many ways, allowing custom pages to be developed for specific uses. In particular, standalone test-cases using small numbers of components make it easier to debug the Javascript than it is to debug a large server program. Information about PhEDEx is accessed through the PhEDEx data-service, since direct SQL is not available from the clients browser. This provides consistent semantics with other, externally written monitoring tools, which already use the data-service. It also reduces redundancy in the code, yielding a simpler, consolidated codebase. In this talk we describe our experience of re-factoring this monolithic server-side program into a lighter client-side framework. We describe some of the techniques that worked well for us, and some of the mistakes we made along the way. We present the current state of the project, and its future direction.

Egeland, R.↗

Tiling Framework for Heterogeneous Computing of Matrix based Tiled Algorithms

Tiling matrix operations can improve the load balancing and performance of applications on heterogeneous computing resources. Writing a tile-based algorithm for each operation with a traditional, hand-tuned tiling approach that uses for loops in C/C++ is cumbersome and error prone. Moreover, it must enable and support the heterogeneous memory management of data objects and also explore architecture-supported, native, tiled-data transfer APIs instead of copying the tiled data to continuous memory before the data transfer. The tiling framework provides a tiled data structure for heterogeneous memory mapping and parameterization to a heterogeneous task specification API. We have integrated our tiled framework into MatRIS (Math kernels library using IRIS). IRIS is a heterogeneous run-time framework with a heterogeneous programming model, memory model, and task execution model. Experiments reveal that the tiled framework for BLAS operations has improved the programmability of tiled BLAS and improved performance by ~20% when compared against the traditional method that copies the data to continuous memory locations for heterogeneous computing.

Miniskar, Narasinga Rao↗

FIFE-Jobsub: a grid submission system for intensity frontier experiments at Fermilab

The Fermilab Intensity Frontier Experiments use an integrated submission system known as FIFE-jobsub, part of the FIFE (Fabric for Frontier Experiments) initiative, to submit batch jobs to the Open Science Grid. FIFE-jobsub eases the burden on experimenters by integrating data transfer and site selection details in an easy to use and well-documented format. FIFE-jobsub automates tedious details of maintaining grid proxies for the lifetime of the grid job. Data transfer is handled using the Intensity Frontier Data Handling Client (IFDHC) [1] tool suite, which facilitates selecting the appropriate data transfer method from many possibilities while protecting shared resources from overload. Chaining of job dependencies into Directed Acyclic Graphs (Condor DAGS) is well supported and made easier through the use of input flags and parameters.

Box, Dennis↗

Radiation-Tolerant, SpaceWire-Compatible Switching Fabric

Current and future near-Earth and deep space exploration programs and space defense programs require the development of robust intra-spacecraft serial data transfer electronics that must be reconfigurable, fault-tolerant, and have the ability to operate effectively for long periods of time in harsh environmental conditions. Existing data transfer systems based on state-of-the-art serial data transfer protocols or passive backplanes are slow, power-hungry, and poorly reconfigurable. They provide limited expandability and poor tolerance to radiation effects and total ionizing dose (TID) in particular, which presents harmful threats to modern submicron electronics. This novel approach is based on a standard library of differential cells tolerant to TID, and patented, multi-level serial interface architecture that ensures the reliable operation of serial interconnects without application of a data-strobe or other encoding techniques. This proprietary, high-speed differential interface presents a lowpower solution fully compatible with the SpaceWire (SW) protocol. It replaces a dual data-strobe link with two identical independent data channels, thus improving the system s tolerance to harsh environments through additional double redundancy. Each channel incorporates an automatic line integrity control circuitry that delivers error signals in case of broken or shorted lines.

Katzman, Vladimir↗

End-to-end data movement using MPI-IO over routed terabits infrastructures. In: NDM '13 Proceedings of the Third International Workshop on Network-Aware Data Management, Article No. 9

Scientific discovery is nowadays driven by large-scale simulations running on massively parallel high-performance computing (HPC) systems. These applications each generate a large amount of data, which then needs to be post-processed for example for data mining or visualization. Unfortunately, the computing platform used for post processing might be different from the one on which the data is initially generated, introducing the challenge of moving large amount of data between computing platforms. This is especially challenging when these two platforms are geographically separated since the data needs to be moved between computing facilities. This is even more critical when scientists tightly couple their domain specific applications with a post processing application. The paper presents a solution for the data transfer between MPI applications using a dedicated wide area network (WAN) terabit infrastructure. The proposed solution is based on parallel access to data files and the Message Passing Interface (MPI) over the Common Communication Infrastructure (CCI) for the data transfer over a routed infrastructure. In the context of this research, the Energy Sciences Network (ESnet) of the U.S. Department of Energy (DOE) is targeted for the transfer of data between DOE national laboratories.

Vallée, Geoffroy↗