Search NASA⌕ Search

SEARCH · Search NASA

Results for “data transfer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

Friendly Extensible Transfer Tool Beta Version

Often data transfer software is designed to meet specific requirements or apply to specific environments. Frequently, this requires source code integration for added functionality. An extensible data transfer framework is needed to more easily incorporate new capabilities, in modular fashion. Using FrETT framework, functionality may be incorporated (in many cases without need of source code) to handle new platform capabilities: I/O methods (e.g., platform specific data access), network transport methods, data processing (e.g., data compression.).

Collins, William P.↗

Locality-aware and load-balanced static task scheduling for MapReduce

Task scheduling for MapReduce jobs has been an active area of research with the objective of decreasing the amount of data transferred during the shuffle phase via exploiting data locality. In the literature, generally only the scheduling of reduce tasks is considered with the assumption that scheduling of map tasks is already determined by the input data placement. However, in cloud or HPC deployments of MapReduce, the input data is located in a remote storage and scheduling map tasks gains importance. Here, we propose models for simultaneous scheduling of map and reduce tasks in order to improve data locality and balance the processors’ loads in both map and reduce phases. Our approach is based on graph and hypergraph models which correctly encode the interactions between map and reduce tasks. Partitions produced by these models are decoded to schedule map and reduce tasks. A two-constraint formulation utilized in these models enables balancing processors’ loads in both map and reduce phases. The partitioning objective in the hypergraph models correctly encapsulates the minimization of data transfer when a local combine step is performed prior to shuffle, whereas the partitioning objective in the graph models achieve the same feat when a local combine is not performed. Here, we show the validity of our scheduling on the MapReduce parallelizations of two important kernel operations – sparse matrix–vector multiplication (SpMV) and generalized sparse matrix–matrix multiplication (SpGEMM) – that are widely encountered in big data analytics and scientific computations. Compared to random scheduling, our models lead to tremendous savings in data transfer by reducing data traffic from several hundreds of megabytes to just a few megabytes in the shuffle phase and consequently leading up to 2.6x and 4.2x speedup for SpMV and SpGEMM, respectively.

97 MATHEMATICS AND COMPUTING↗

Transferability and data access issues

The evolution and current operational mode of the domestic information display system (DIDS) now known as the decision information display system are described. This minicomputer based system, developed from a NASA-generated image processing system, was designed to display federal statistical data for a variety of geographic areas in the form of choropleth maps. The application of DIDS in South Carolina is discussed as well as the progress made and issues that emerged in using the data base on a state and county level. The hardware base for the system, how this user friendly system works, and the possibility of transferring data to remote systems are examined.

Ferreros, A. V.↗

SDN-NGenIA, a Software Defined Next Generation Integrated Architecture for HEP and Data Intensive Science

This project aims to enable the LHC and other leading programs in high energy physics and other global science domains funded by the DOE to operate with a new level of efficiency and control, through the development of a Next Generation Integrated Architecture (NGenIA) based on intelligent software defined network (SDN)-driven network systems coupled to high throughput applications. While the initial focus is on the challenging LHC use case, the systems and products being developed are general, and apply to many fields of data intensive science ranging from astrophysical sky surveys to bioinformatics and earth observation, as well as other organizations facing the challenges of extracting knowledge from distributed multi-petabyte data stores. A central concept in this development program is a new paradigm ‘consistent network operations’ among widely distributed computing and storage facilities, where stable high throughput flows, at set rates, cross load-balanced network paths, up to flexible high water marks that are adjusted in real time to accommodate other network traffic. The large smooth flows are launched and managed by SDN services that act in concert with the experiments’ site-resident data distribution and management systems, to meet the expanding needs of the science programs. The technical goals include the construction of autonomous, intelligent site-resident services that interact dynamically with network-resident services, and with the science programs’ principal data distribution and management tools, to request or command network resources in support of high throughput petascale to exascale workflow Specific work items include: 1. Developing compact Data Transfer Nodes (DTNs) with auto-tuning functions that support data transfer rates in the 100 Gbps to the 1 Tbps range when used with high throughput data transfer applications 2. Deep site orchestration among virtualized clusters, storage subsystems and subnets to successfully coschedule CPU, storage and network resources 3. Science-program designed site architectures, operational modes, and policy and resource usage priorities, adjudicated across multiple network domains and virtual organizations, using the orchestration functions and methods 4. Seamlessly extending end-to-end operation across both extra-site and intra-site boundaries through the use of Open vSwitch (OVS), FireQoS + next generation Science DMZs 5. Novel methods of system integration that enable granular control of extreme scale long distance transfers through flow matching of scattered source-destination address pairs to multi-domain dynamic circuits 6. Funneling massive sets of streams to DTNs at the site edge hosting petascale buffer pools configured for flows of 100 Gbps and up, exploiting state of the art data transfers 7. Adaptive scheduling based on pervasive end-to-end monitoring, including DTN or compute-node resident agents providing comprehensive end-system profiling 8. Developing unsupervised and supervised machine learning and modeling methods to optimize the workflow involving terabyte to multi-petabyte datasets.

47 OTHER INSTRUMENTATION↗

The Modern Research Data Portal: a design pattern for networked, data-intensive science

We describe best practices for providing convenient, high-speed, secure access to large data via research data portals. Here, we capture these best practices in a new design pattern, the Modern Research Data Portal, that disaggregates the traditional monolithic web-based data portal to achieve orders-of-magnitude increases in data transfer performance, support new deployment architectures that decouple control logic from data storage, and reduce development and operations costs. We introduce the design pattern; explain how it leverages high-performance data enclaves and cloud-based data management services; review representative examples at research laboratories and universities, including both experimental facilities and supercomputer sites; describe how to leverage Python APIs for authentication, authorization, data transfer, and data sharing; and use coding examples to demonstrate how these APIs can be used to implement a range of research data portal capabilities. Sample code at a companion web site, https://docs.globus.org/mrdp , provides application skeletons that readers can adapt to realize their own research data portals.

97 MATHEMATICS AND COMPUTING↗

IRIS-DMEM: Efficient Memory Management for Heterogeneous Computing

This paper proposes an efficient data memory management approach for the Intelligent RuntIme System (IRIS) heterogeneous computing framework along with new data transfer policies. IRIS provides a task-based programming model for extreme heterogeneous computing (e.g., CPU, GPU, DSP, FPGA) with support for today's most important programming languages (e.g., OpenMP, OpenCL, CUDA, HIP, OpenACC). However, the IRIS framework either forces the programmer to introduce data transfer commands for each task or relies on suboptimal memory management for automatic and transparent data transfers. The work described here extends IRIS with novel heterogeneous memory handling and introduces novel data transfer policies by employing the Distributed data MEMory handler (DMEM) for efficient and optimal movement of data among the various computing resources. The proposed approach achieves performance gains of up to 7× for tiled LU factorization and tiled DGEMM (i.e., matrix multiplication) benchmarks. Moreover, this approach also reduces data transfers by up to 71% when compared to previous IRIS heterogeneous memory management handlers. This work compares the performance results of the IRIS framework's novel DMEM with the StarPU runtime and MAGMA math library for GPUs. Experiments show a performance gain of up to 1.95× over StarPU and 2.1× over MAGMA.

Miniskar, Narasinga Rao↗

2016 Site 300 Bat Monitoring (Final Report)

Garcia and Associates (GANDA) was contracted by the Lawrence Livermore National Laboratory (LLNL) to design and execute a long-term passive bat monitoring program for Site 300, southwest of Tracy, California. This effort included consulting and collaborating with LLNL staff for the acquisition of monitoring equipment, data collection, data storage, data transfer and data analysis.

54 ENVIRONMENTAL SCIENCES↗

CELFE: Coupled Eulerian-Lagrangian Finite Element program for high velocity impact. Part 2: Program user's manual

The CELFE computer program and user's manual, together with the execution of the CELFE/NASTRAN system, are described. The execution procedure and the transfer of data between the CELFE and NASTRAN programs are controlled through the use of DATA files in the Univac 1100 system. Five data files are used to control the runstream and data transfer, and three files are used to hold the programs. These files are contained on a single tape. Changes in NASTRAN routines required by the present analysis are also discussed in this report. All the program listings, except the last two files (where the absolute and relocatable elements are stored), are included in the appendixes.

Lee, C. H.↗

Finite element-finite difference thermal/structural analysis of large space truss structures

A technique of automated and efficient thermal-structural processing of truss structures that interfaces the finite element and finite difference method was developed. The thermal-structural analysis tasks include development of the thermal and structural math models, thermal analysis, development of an interface and data transfer between the models, and finally an evaluation of the thermal stresses and displacements in the structure. Consequently, the objective of the developed technique was to minimize the model development time, in order to assure an automatic transfer of data between the thermal and structural models as well as to minimize the computer resources needed for the analysis itself. The method and techniques described are illustrated on the thermal/structural analysis of the Space Station Freedom main truss.

Warren, Andrew H.↗

A-Train Data Depot: Integrating, Visualizing, and Extracting Cloudsat, CALIPSO, MODIS, and AIRS Atmospheric Measurements Along the A-Train Tracks

The succession of US and international Earth observing satellites that follow each other, seconds to minutes apart, across the local afternoon equator crossing is called the ATrain. The A-Train consists of the following satellites, in order of equator crossing: OCO, EOS Aqua, CloudSat, CALIPSO, PARASOL, and EOS Aura. Flying in such formation increases the number of observations, validates observations, and enables coordination between science observations, resulting in a more complete virtual science platform (Kelly, 2000). The goal of this project is to create the first ever A-Train virtual data portal/center, the A-Train Data Depot (ATDD), to process, archive, access, visualize, analyze and correlate distributed atmosphere measurements from various A-Train instruments along A-Train tracks. The ATDD will enable the free movement of remotely located A-Train data so that they are combined to create a consolidated vertical view of the Earth's Atmosphere along the A-Train tracks. Once the infrastructure of the ATDD is in place, it will be easily evolved to serve data from all A-Train data measurements: one stop shopping. The innovative approach of analyzing and visualizing atmospheric profiles along the platforms track (i.e., time) will be accommodated by reusing the GSFC Atmospheric Composition Data and Information Services Center (ACDISC) visualization and analysis tool, GIOVANNI, existing data reduction tools, on-line archiving for fast data access, access to remote data without unnecessary data transfers, and data retrieval by users finding data desirable for further study. Initial measurements utilized include CALIPSO lidar backscatter, CloudSat radar reflectivity, clear air relative humidity, water vapor and temperature from AIRS, and cloud properties and aerosols from both MODIS. This will be foilowed by associated measurements from TVILS, =MI, HIRDLS, sad TES. Given the independent nature of instrumentlplatform development, the ATDD project has been met with many interesting challenges that, once resolved, will provide a much greater understanding of the relative flight dynamics and data co-registration of the suite of A-Train instruments, thus greatly increasing the accuracy of A-Train data analysis. Some of these challenges will be illustrated and discussed. The project's early visualizations and analysis efforts illustrate the importance of managing data so that measurements from various missions can be combined to enhance the understanding of the atmosphere. A-Train data management coordination, as performed here, is extremely significant in facilitating the A-Train science of clouds, precipitation, aerosol and chemistry.

Kempler, Steven↗

Propellant Readiness Level: A Methodological Approach to Propellant Characterization

A methodological approach to defining propellant characterization is presented. The method is based on the well-established Technology Readiness Level nomenclature. This approach establishes the Propellant Readiness Level as a metric for ascertaining the readiness of a propellant or a propellant combination by evaluating the following set of propellant characteristics: thermodynamic data, toxicity, applications, combustion data, heat transfer data, material compatibility, analytical prediction modeling, injector/chamber geometry, pressurization, ignition, combustion stability, system storability, qualification testing, and flight capability. The methodology is meant to be applicable to all propellants or propellant combinations; liquid, solid, and gaseous propellants as well as monopropellants and propellant combinations are equally served. The functionality of the proposed approach is tested through the evaluation and comparison of an example set of hydrocarbon fuels.

Bossard, John A.↗

Performance of a Discrete Wavelet Transform for Compressing Plasma Count Data and its Application to the Fast Plasma Investigation on NASA's Magnetospheric Multiscale Mission

Plasma measurements in space are becoming increasingly faster, higher resolution, and distributed over multiple instruments. As raw data generation rates can exceed available data transfer bandwidth, data compression is becoming a critical design component. Data compression has been a staple of imaging instruments for years, but only recently have plasma measurement designers become interested in high performance data compression. Missions will often use a simple lossless compression technique yielding compression ratios of approximately 2:1, however future missions may require compression ratios upwards of 10:1. This study aims to explore how a Discrete Wavelet Transform combined with a Bit Plane Encoder (DWT/BPE), implemented via a CCSDS standard, can be used effectively to compress count information common to plasma measurements to high compression ratios while maintaining little or no compression error. The compression ASIC used for the Fast Plasma Investigation (FPI) on board the Magnetospheric Multiscale mission (MMS) is used for this study. Plasma count data from multiple sources is examined: resampled data from previous missions, randomly generated data from distribution functions, and simulations of expected regimes. These are run through the compression routines with various parameters to yield the greatest possible compression ratio while maintaining little or no error, the latter indicates that fully lossless compression is obtained. Finally, recommendations are made for future missions as to what can be achieved when compressing plasma count data and how best to do so.

Particles↗

Invited: uPSP Launch Vehicle Demonstration Test at NASA Ames Research Center

The Unsteady Pressure-Sensitive Paint (uPSP) Development Team outof NASA Ames Research Center (ARC) has spent the past five yearsimproving the systems and processes to advance the uPSP technology for production-level wind tunnel testing. Already considered turnkey for small-scale and research applications, development in acquisition, calibration, data transfer, and data processing were needed to be useful to customers testing at NASA wind tunnels. This development focused at ARC at the Unitary Plan Wind Tunnel (UPWT) 11-by 11-ft Transonic Wind Tunnel due to the large optical access of the test section and the NASA Advanced Supercomputer(NAS), also located at ARC. A Launch Vehicle Demonstration Test (LVDT) at the UPWT represents a milestone of this initial phase of development where several new improvements were demonstrated in a production wind tunnel environment for the first time. LVDT was conducted in April 2024 and used a 4% forebody Space Launch System (SLS) Block 1B model as the test article. Both a crew and cargo configuration were tested, with varying Mach numbers, pressures, model positions, and camera magnifications. This paper summarizes the details of the test and is part of a collection with four additional papers that provide greater detail on: high-speed lifetime methodology, spectral proper orthogonal decomposition analysis, quality of high-resolution data compared to Corcos model, and data quality, calibration, and uncertainty.

SLS↗

An Experimental Feasibility Study on Applying Neutron Tomography to Encapsulated Spent Nuclear Fuel - 20050

Visual inspection makes easier to ensure the integrity and safety of spent nuclear fuel (SNF) than any classical techniques. Various classical techniques have been applied but there are no reliable methods to qualitatively and quantitatively verify spent fuel in dry storage. Thus, the present authors have developed the prototype safeguards apparatus for dry storage employing the array of He-4 gas scintillation detectors (S670E, Arktis Radiation Detectors Ltd., Switzerland), newly designed to simultaneously measure thermal and fast neutrons without any moderators. The S670E detector has a cylindrical shape with a diameter of 52 mm and active length of 600 mm (total length: 875 mm). The detector is filled by He-4 gas with an approximate pressure of 180 bar for fast neutron detection, and its inner wall is coated by Li-6 for thermal neutron detection. The scintillation lights generated via Li-6 nuclear reaction and elastic scattering are collected by 24 SiPMs linearly paired at the center of the detector. The detector delivers a TTL (Transistor-Transistor Logic) output for pulse readout and UART (Universal Asynchronous Receiver Transmitter) for device control. In order to assess feasibility of the apparatus, an experimental system has been designed, built, and optimized via computational studies. Cf-252 neutron sources and linearly arrayed detectors, working as a single detector, were occupied for this study due to the difficulties in working with SNF. The laboratory scale cask (diameter: 0.67 m, height: 1.5 m), minimized by a factor of 10 compared to the actual thickness of a commercial TN-32 cask, was also manufactured. The detector array was designed to rotate the lab-scale cask and obtain 36 image profiles at every 10 degrees. All profiles were aligned in single frame image called a sinogram, and the cross-sectional image was then fabricated by the inverse radon transform algorithm. These experiments have been repeated with different configurations and numbers of sources. Some gamma-ray sources were also measured with neutron sources in order to distinguish between neutron and gamma-ray pulses. Basically, a He-4 detector is designed to run on Linux OS so it is difficult to directly apply to Windows-based equipment widely used in S. Korea. Therefore, a new data acquisition board working on Windows OS was designed and built. The board mainly consists of FPGA (Field Programmable Gate Array) and SoC (System on Chip) for TTL pulse readout, sorting measured data, and transferring data to a user interface. In conclusion, the tomographic system at lab-scale has shown considerable potential to detect a partial or gross defect of encapsulated assemblies in dry storage. Next steps of this study will be to 1) repeatedly carry out experiments to demonstrate scientific reliability and validity, and 2) numerically integrate signals with weight factors to enhance the image quality since the suggested system based on passive interrogation method requires longer measurement time. Finally, the system will apply to a commercial dry storage phased out soon in S. Korea. (authors)

07 ISOTOPE AND RADIATION SOURCES↗

Communication Bandwidth Considerations for Exploration Medical Care During Space Missions

Destinations beyond low Earth orbit, especially Mars, have several important constraints, including limited resupply, limited to no possibility of medical evacuation, and delayed communication with ground support teams. Therefore, medical care is driven towards greater autonomy and necessitates a medical system that supports this paradigm, including the potential for high medical data transfer rates in order to share medical information and coordinate care with the ground in an intermittent fashion as communication allows. The medical data transfer needs for a Martian exploration mission were estimated by defining two medical scenarios that would require high data rate communications between the spacecraft and Earth. One medical scenario involves a case of hydronephrosis (outflow obstruction of the kidney) that evolves into pyelonephritis (kidney infection), then urosepsis (systemic infection originating from the kidney), due to obstruction by a kidney stone. A second medical scenario involved the death of a crewmember’s child back on Earth that requires behavioral health care. For each of these scenarios, a data communications timeline was created following the medical care described by the scenario. From these timelines, total medical data transfers and burst transmission rates were estimated. Total data transferred from the vehicle-to-ground were estimated to be 94 gigabytes (GB) and 835 GB for the hydronephrosis and behavioral health scenarios, respectively. Data burst rates were estimated to be 7.7 megabytes per second (MB/s) and 15 MB/s for the hydronephrosis and behavioral health scenarios, respectively. Even though any crewed Mars mission should be capable of functioning autonomously, as long as the possibility of communication between Earth and Mars exists, Earth-based subject matter experts will be relied upon to augment mission medical capability. Therefore, setting an upper boundary limit for medical communication rates can help factor medical system needs into total vehicle communication requirements.

Krihak, Michael↗

Suppressing simulation bias in multi-modal data using transfer learning

Abstract Many problems in science and engineering require making predictions based on few observations. To build a robust predictive model, these sparse data may need to be augmented with simulated data, especially when the design space is multi-dimensional. Simulations, however, often suffer from an inherent bias. Estimation of this bias may be poorly constrained not only because of data sparsity, but also because traditional predictive models fit only one type of observed outputs, such as scalars or images, instead of all available output data modalities, which might have been acquired and simulated at great cost. To break this limitation and open up the path for multi-modal calibration, we propose to combine a novel, transfer learning technique for suppressing the bias with recent developments in deep learning, which allow building predictive models with multi-modal outputs. First, we train an initial neural network model on simulated data to learn important correlations between different output modalities and between simulation inputs and outputs. Then, the model is partially retrained, or transfer learned, to fit the experiments; a method that has never been implemented in this type of architecture. Using fewer than 10 inertial confinement fusion experiments for training, transfer learning systematically improves the simulation predictions while a simple output calibration, which we design as a baseline, makes the predictions worse. We also offer extensive cross-validation with real and carefully designed synthetic data. The method described in this paper can be applied to a wide range of problems that require transferring knowledge from simulations to the domain of experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

HEP Science Network Requirements--Final Report

The Energy Sciences Network (ESnet) is the primary provider of network connectivity for the US Department of Energy Office of Science, the single largest supporter of basic research in the physical sciences in the United States. In support of the Office of Science programs, ESnet regularly updates and refreshes its understanding of the networking requirements of the instruments, facilities, scientists, and science programs that it serves. This focus has helped ESnet to be a highly successful enabler of scientific discovery for over 20 years. In August 2009 ESnet and the Office of High Energy Physics (HEP), of the DOE Office of Science, organized a workshop to characterize the networking requirements of the programs funded by HEP. The International HEP community has been a leader in data intensive science from the beginning. HEP data sets have historically been the largest of all scientific data sets, and the communty of interest the most distributed. The HEP community was also the first to embrace Grid technologies. The requirements identified at the workshop are summarized below, and described in more detail in the case studies and the Findings section: (1) There will be more LHC Tier-3 sites than orginally thought, and likely more Tier-2 to Tier-2 traffic than was envisioned. It it not yet known what the impact of this will be on ESnet, but we will need to keep an eye on this traffic. (2) The LHC Tier-1 sites (BNL and FNAL) predict the need for 40-50 Gbps of data movement capacity in 2-5 years, and 100-200 Gbps in 5-10 years for HEP program related traffic. Other key HEP sites include LHC Tier-2 and Tier-3 sites, many of which are located at universities. To support the LHC, ESnet must continue its collaborations with university and international networks. (3) While in all cases the deployed 'raw' network bandwidth must exceed the user requirements in order to meet the data transfer and reliability requirements, network engineering for trans-Atlantic connectivity is more complex than network engineering for intra-US connectivity. This is because transoceanic circuits have lower reliability and longer repair times when compared with land-based circuits. Therefore, trans-Atlantic connectivity requires greater deployed bandwidth and diversity to ensure reliability and service continuity of the user-level required data transfer rates. (4) Trans-Atlantic traffic load and patterns must be monitored, and projections adjusted if necessary. There is currently a shutdown planned for the LHC in 2012 that may affect projections of trans-Atlantic bandwidth requirements. (5) There is a significant need for network tuning and troubleshooting during the establishment of new LHC Tier-2 and Tier-3 facilities. ESnet will work with the HEP community to help new sites effectively use the network. (6) SLAC is building the CCD camera for the LSST. This project will require significant bandwidth (up to 30Gbps) to NCSA over the next few years. (7) The accelerator modeling program at SLAC could require the movement of 1PB simulation data sets from the Leadership Computing Facilities at Argonne and Oak Ridge to SLAC. The data sets would need to be moved overnight, and moving 1PB in eight hours requires more than 300Gbps of throughput. This requirement is dependent on the deployment of analysis capabilities at SLAC, and is about five years away. (8) It is difficult to achieve high data transfer throughput to sites in China. Projects that need to transfer data in or out of China are encouraged to deploy test and measurement infrastructure (e.g. perfSONAR) and allow time for performance tuning.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Data communications in a parallel active messaging interface of a parallel computer

Data communications in a parallel active messaging interface (`PAMI`) of a parallel computer, the PAMI composed of data communications endpoints, each endpoint including a specification of data communications parameters for a thread of execution on a compute node, including specifications of a client, a context, and a task, endpoints coupled for data communications through the PAMI and through data communications resources, including receiving in an origin endpoint of the PAMI a SEND instruction, the SEND instruction specifying a transmission of transfer data from the origin endpoint to a first target endpoint; transmitting from the origin endpoint to the first target endpoint a Request-To-Send (`RTS`) message advising the first target endpoint of the location and size of the transfer data; assigning by the first target endpoint to each of a plurality of target endpoints separate portions of the transfer data; and receiving by the plurality of target endpoints the transfer data.

97 MATHEMATICS AND COMPUTING↗