Search NASA⌕ Search

SEARCH · Search NASA

Results for “File transfer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Oncentra brachytherapy planning system

In modern cancer management, treatment planning has progressed as a contemporary tool with all the advances in computing power in recent years. One of the advanced planning tools uses 3-dimensional (3D) data sets for accurate dose distributions in patient prescription. Among these planning processes, brachytherapy has been a very important part of a successful cancer management program, offering clinical benefits with specific or combined treatments with external beam therapy. In this chapter, we mainly discussed the Elekta Oncentra planning system, which is the main treatment planning tool for high-dose rate (HDR) modality in our facility and in many other facilities in the United States. HDR is a technically advanced form of brachytherapy; a high-intensity radiation source (3.6 mm in length) is delivered with step motor in submillimeter precision under computer guidance directly into the tumor areas while minimizing injury to surrounding normal healthy tissue. Oncentra planning is the key component to generate a deliverable brachytherapy procedure, which is executed on the microSelectron V3 remote afterloader treatment system. Creating a highly conformal plan can be a time-consuming task. The development of Oncentra software (version 4.5.3) offers a variety of useful tools that facilitate many of the clinical challenging tasks for planning, such as contouring and image reconstruction, as well as rapid planning calculations with dose and dose volume histogram analysis. Oncentra Brachy module creates workflow and optimizes the planning accuracy for wide varieties of clinical HDR treatments, such as skin, gynecologic (GYN), breast, prostate, and many other applications. The treatment file can also be transferred to the afterloader control station for speedy delivery. The design concept, calculation algorithms, and optimization modules presented some key characteristics to plan and treat the patients effectively and accurately. The dose distribution and accuracy of several clinical sample cases were discussed to illustrate the effectiveness and clinical efficacy. The American Association of Physicists in Medicine brachytherapy reports of TG-43 and TG-186 were also described and compared in evaluations of fundamental calculation methodologies.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2022

Understanding the Impact of Data Staging for Coupled Scientific Workflows

We report the rate of data generated by cutting-edge experimental science facilities and large-scale simulations enabled by current high-performance computing (HPC) systems has continued to grow at a far greater pace than the development of the network and storage capabilities on which these systems rely. To cope with this challenge, scientist are moving toward the creation of autonomous experiments and HPC simulations using machine learning. However, efficiently moving, storing, and processing large amounts of data away from the point of origin presents an incredible challenge. In-memory computing, in situ analysis, data staging, and data streaming are recognized viable alternatives to traditional file-based methods for transferring data between coupled workflows. However, the performance trade-offs and limitations for these methods are not fully understood when used in HPC applications. This article presents a comprehensive performance assessment of the current solutions for data staging when applied to applications that are not necessary I/O intensive which makes them not ideal candidates for these methods. Our study is based on experiments running at scale on Oak Ridge National Laboratory's Summit supercomputer using applications and simulations that cover typical computational motifs and patterns. We investigated the usability and cost/benefit trade-offs of staging algorithms for HPC applications under different scenarios and highlight opportunities for optimizing the dataflow between coupled simulation workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

CatCost™ Data Tools [SWR-20-59]

CatCost™ enables catalyst researchers to develop comprehensive estimates of the cost to produce pre-commercial catalysts early in the catalyst development process. It includes both web browser and Excel-based tools, each with distinct strengths. The web tool includes powerful visualization capabilities, while the Excel tool allows for high degrees of control and customization by power users. The CatCost™ Python Data Tools are designed to facilitate the transfer of data from the Excel tool to the web tool, so that users can take advantage of both the powerful visualization in the web tool and the higher degree of customization available in the excel tool. Rather than painstakingly copying entries from estimates, the materials, equipment, and spent catalyst libraries from Excel to a JSON format for the web tool, the CatCost™ Data Tools provide an intuitive applet which allows them to simply choose which library or estimate they wish to transfer, and then returns files that integrate smoothly with the web version of the tool.

Baddour, Frederick↗
DOE OSTI2019

NEAMS Workbench and Bison Fuel Performance Remote Application Configuration

The mission of the US Department of Energy (DOE) Nuclear Energy Advanced Modeling and Simulation (NEAMS) Program is to develop, apply, deploy, and support state-of-the-art predictive modeling and simulation tools for the design and analysis of current and future nuclear energy systems. This is accomplished using computing architectures that range from laptops to leadership class facilities. The NEAMS Workbench [1,2,3] initiative will facilitate the transition from conventional computer codes to high-fidelity ones by providing a common analysis environment and interface for model creation, review, execution, output review, and visualization for integrated codes. The BISON fuel performance code [4] is a finite element method (FEM)-based software being developed by multiple organizations led by the Idaho National Laboratory for the US Department of Energy Office of Nuclear Energy. BISON can simulate a variety of fuel and cladding compositions and geometries in varying dimensionalities, depending upon the problem type and complexity, including one-dimensional (1D), 1.5D, 2D axially symmetric (2D-rz), 2D radial-azimuthal (2D-rθ), or 3D models. The NEAMS Workbench allows execution of the BISON fuel performance code on local and remote compute resources. In this report, the detailed instructions of configuring the NEAMS Workbench for remote execution of the BISON fuel performance code on a portable batch system (PBS)-scheduled compute resource, as well the transfer of associated Bison files, is illustrated.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2024

Sandia Toolkit Manual (V.5.21.1)

This report provides documentation for the Sandia Toolkit (STK) modules. STK modules are intended to provide infrastructure that assists the development of computational engineering software such as finite-element analysis applications. STK includes modules for unstructured-mesh data structures, reading/writing mesh files, geometric proximity search, transfers, MPMD coupling support, and various other utilities. This document contains a chapter for each module, and each chapter contains overview descriptions and usage examples. Usage examples are primarily code listings which are generated from working test programs that are included in the STK code-base. A goal of this approach is to ensure that the usage examples will not fall out of date.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Profiling the Usage of an Extreme-Scale Archival Storage System

Profiling the archival storage system in scientific computing environments has received much less attention compared to the parallel file system, but is equally important since it stores the final data products safely, for a long duration. In this paper, we analyze eight years worth of data transfer logs for accessing the archival file system (HPSS) in the Oak Ridge Leadership Computing Facility (OLCF), which has been hosting the world's largest supercomputers and file systems. Our analysis encompasses about 135 million data transfer activities to the 80 PB High Performance Storage System (HPSS), between 2010 and 2017. We analyze the logs from several dimensions, including studying the workload characteristics (e.g., access patterns, frequency of accesses and temporal behavior), file system characteristics (e.g., directory depth, file system scaling trends, file types), and scientific user behavior (e.g., domain-specific usage and organization). Based on the analysis, we derive insights into the future evolution of the archive in terms of provisioning, desired features and functionality from the archive software, role and right sizing of the archive tiers, quota management, and the importance of smart and efficient metadata and storage management. We believe our study will prove useful for both operating current archival storage and better provisioning future systems.

Sim, Hyogi↗
DOE OSTI2022

Moving small files in a networked environment

Globally distributed computing infrastructures, such as clouds and supercomputers, are currently used to manage data that is generated with an unprecedented speed from a variety of resources. Coping with this trend, the volume of data exchanged across distant sites increases substantially. To accelerate data transfer, high-speed networks are provided to connect remote sites. Most existing data movement solutions are optimized for moving large files. However, it is still challenging to transfer a large number of small files across networks. This disadvantage not only lowers data transfer performance, but also decreases overall system utilization. Here, we identify that moving small files is mainly constrained by degraded file system throughput, not just network performance as might be suspected. We have built a data transfer pipeline model to analyze the impact of small network I/O and storage I/O on data movement. Extending one of the widely used open source data movement solutions, GridFTP, we demonstrate several appropriate engineering approaches that mitigate the bottleneck and increase data transfer efficiency. We show optimizations that improve data transfer performance more than 5 times. In comparison to existing solutions, our approaches can save a significant amount of system resources for moving lots of small files.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Fast Reactor Physics Model Verification Studies using ARC and PyARC Workflows

PyARC was recently developed at Argonne National Laboratory to automate many of the tasks required in the ARC (Argonne Reactor Computation) fast reactor simulation workflow, from input file generation, code execution, data transfer between ARC codes, and output postprocessing. PyARC will likely be the path forward to train new users of the ARC codes with the goal of wide adoption by the national laboratories, academia, and industry. In particular, for the ANL-JAEA collaboration under the Civil Nuclear Working Group (CNWG) project agreement NE-01, PyARC will be used to model the Joyo and EBR-II reactors for comparisons with measured data and calculated results from JAEA (Task 3: Fast Reactor Fuel and Core). As an additional avenue for verification and validation, this report investigates the use of PyARC towards a variety of existing ARC-based reactor models, in order to understand its efficacy in replicating the behavior of base ARC codes and better understand any limitations within modeling realistic fast reactor problems. To this end, PyARC was used to model the Joyo MKI, RBEC Benchmark-M, PRISM Mod-B, and EBR-II Run 138B cores, and its results were compared to those from existing ARC-based models. It was found that for hexagonal-based geometries PyARC was able to replicate the behavior of ARC codes to within 10 pcm for small reactor cores, and ~150pcm difference in eigenvalue for larger cores. These discrepancies are attributed primarily to differences in local mesh refinement options between ARC and PyARC, which currently cannot be resolved with PyARC’s latest version (1.6.0). In some of these cases, PyARC was used to model steady-state problems with initial core compositions originating from a prior REBUS depletion calculation. While PyARC was not designed to support such steady-state calculations, workarounds were applied to replicate the behavior of ARC-based calculations as closely as possible. Thus, these results demonstrate the wide extent to which they can be applied to fast reactor problems while still providing immense benefit to the user in terms of automating and standardizing common routines within the fast reactor analysis workflow. This study concluded that PyARC will be suitable for modeling the steady-state conditions of the EBR-II and Joyo fast reactors as part of the CNWG project agreement.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Usability and Optimization Improvements in MOOSE

The Multiphysics Object-Oriented Simulation Environment (MOOSE) framework is a foundational capability used by the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program to create over 15 different simulation tools for advanced nuclear reactors. Due to MOOSE’s broad use, improvements to the framework in support of modeling and simulation goals are critical to the program. Such improvements can take many forms, including optimization, improved user experience, streamlined application programming interfaces (APIs), parallelism, and new capabilities. The work described in this report was conducted in direct support of the simulation tools and has already been deployed. The capabilities were implemented in the same order as they are covered in this report: multiple nonlinear systems in the same input file, implementation of generic field transfers and other transfer system enhancements, support for stateful material property redistribution for adaptivity and distributed meshes, and dynamic linking and loading of individually compiled applications. These four additions are fundamental capabilities that will be leveraged by many NEAMS applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Continuing Efforts on Efficiency and Performance Improvement of mdtmFTP

mdtmFTP is a high-performance data transfer tool: • Pipelined I/O-centric design to streamline data transfer • Multicore-aware data transfer middleware (MDTM) optimizes use of underlying multicore system • Extremely efficient in transferring of Lots Of Small Files • Various optimization mechanisms including Asynchronous I/O and Batch processing. • Sponsored by DOE/SC/ASCR. http://mdtm.fnal.gov $\bullet$Continuing efforts on efficiency and Performance Improvement: • Refactoring event queue mechanism • I/O optimization • Scalability evaluation

Zhang, L.↗
DOE OSTI2018

ENRESA's Spent Nuclear Fuel and Special Waste Database - 18115

Empresa Nacional de Residuos Radiactivos S.A. (ENRESA, by its initials in Spanish) Spent Nuclear Fuel (SNF) and Special Waste (SW) database development was initiated in a common effort between ENRESA and UNESA (the Spanish Association of the Electricity Industry) as a consequence of new regulatory requirements in the frame of radioactive waste management plans. Although these requirements were only mandatory for operators of nuclear power plants, ENRESA deemed necessary to have a centralized database which collected the data from the plant operators. The most relevant features of this database are: - The information collected covers not only the general characteristics of the SNF and SW but also the operation details needed for their characterization. - An application, named Gestion de Elementos Combustibles y Residuos Especiales (GECYRE, by its initials in Spanish) has been implemented for easing the queries and the generation of predefined reports. - An agile data transfer scheme, based on XML files, is used for an effective communication between ENRESA and the operators. The database is currently in production phase since 2014. During this phase it has demonstrated its effectiveness as an information repository to address the SNF and SW interim storage projects in Spain foreseen in the near future. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2022

DYnamic and Asynchronous Data Streamliner

DYAD aims to help sharing data files between producer and consumer job elements, especially between co-scheduled jobs or within an ensemble. DYAD provides the service by two components: a FLUX module and a I/O wraper set. DYAD transparently synchronizes file I/O between producer and consumer, and transfers data from the producer location to the consumer location managed by the service. Users only need to use the file path that is under the directory managed by the service.

Ahn, DongH↗
DOE OSTI2019

High-performance End-to-End Integrity Verification on Big Data Transfer

The scale of scientific data generated by experimental facilities and simulations in high-performance computing facilities has been proliferating with the emergence of IoT-based big data. In many cases, this data must be transmitted rapidly and reliably to remote facilities for storage, analysis, or sharing, for the Internet of Things (IoT) applications. Simultaneously, IoT data can be verified using a checksum after the data has been written to the disk at the destination to ensure its integrity. However, this end-to-end integrity verification inevitably creates overheads (extra disk I/O and more computation). Thus, the overall data transfer time increases. In this article, we evaluate strategies to maximize the overlap between data transfer and checksum computation for astronomical observation data. Specifically, we examine file-level and block-level (with various block sizes) pipelining to overlap data transfer and checksum computation. We analyze these pipelining approaches in the context of GridFTP, a widely used protocol for scientific data transfers. Theoretical analysis and experiments are conducted to evaluate our methods. The results show that block-level pipelining is effective in maximizing the overlap mentioned above, and can improve the overall data transfer time with end-to-end integrity verification by up to 70% compared to the sequential execution of transfer and checksum, and by up to 60% compared to file-level pipelining.

data integrity↗
DOE OSTI2020

GPU Direct I/O with HDF5

Exascale HPC systems are being designed with accelerators, such as GPUs, to accelerate parts of applications. In machine learning workloads as well as large-scale simulations that use GPUs as accelerators, the CPU (or host) memory is currently used as a buffer for data transfers between GPU (or device) memory and the file system. If the CPU does not need to operate on the data, then this is sub-optimal because it wastes host memory by reserving space for duplicated data. Furthermore, this “bounce buffer” approach wastes CPU cycles spent on transferring data. A new technique, NVIDIA GPUDirect Storage (GDS), can eliminate the need to use the host memory as a bounce buffer. Thereby, it becomes possible to transfer data directly between the device memory and the file system. This direct data path shortens latency by omitting the extra copy and enables higher-bandwidth. To take full advantage of GDS in existing applications, it is necessary to provide support with existing I/O libraries, such as HDF5 and MPI-IO, which are heavily used in applications. In this paper, we describe our effort of integrating GDS with HDF5, the top I/O library at NERSC and at DOE leadership computing facilities. We design and implement this integration using a HDF5 Virtual File Driver (VFD). The GDS VFD provides a file system abstraction to the application that allows HDF5 applications to perform I/O without the need to move data between CPUs and GPUs explicitly. We compare performance of the HDF5 GDS VFD with explicit data movement approaches and demonstrate superior performance with the GDS method.

Ravi, J↗
DOE OSTI2024

NSTXU Diagnostic Disruption Dynamic Loading Represented by Response Spectra

This article presents the results of transient dynamic simulations of loads due to disruption eddy currents on the NSTXU vacuum vessel. Dynamic loading at diagnostic mounting locations is expressed as response spectra derived from the time history results of the dynamic structural simulations of a variety of disruption scenarios. The disruption simulations draw on a history of the project assessments of worst case disruptions for specific components. Major efforts to assess disruption loading have included the vacuum vessel which is the major structural support for the machine, as well as the passive plates (PPs), high harmonic fast wave (HHFW) antenna, and centerstack casing. Each one of these efforts included transient electromagnetic simulations producing time-dependent eddy current Lorentz loads (and in some cases halo loads) which then were applied to time-dependent structural dynamic analyses intended to obtain the proper dynamic amplification factors. In some instances, the EM model and structural model were identical allowing direct transfer of EM forces to the structural model. In other cases, the EM and structural model were not identical and the vector potential (VP) transfer method was used. The results files from these analyses were available (or re-run) to post process in ANSYS Classic time history postprocessor. In conclusion, the ANSYS command is used to create response spectra from time history data at desired points on the vessel.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2019

FTLADS: Object-Logging Based Fault-Tolerant Big Data Transfer System Using Layout Aware Data Scheduling

The layout-aware data scheduling (LADS) data movement framework optimizes congestion for end-to-end data transfers. During data transfer, LADS can avoid congested storage elements by exploiting the underlying storage layout at each endpoint. This improves the I/O bandwidth and hence the data transfer rate across high-speed networks. However, the absence of fault tolerance (FT) in LADS results in data retransmission overhead and may lead to possible data integrity issues upon faults. In this paper, we propose object-logging FT mechanisms to avoid transmitting the objects that are successfully written into the parallel file system (PFS) at the sink end. Depending on the number of log files created for the whole dataset, we have classified our FT mechanisms into three different categories: file logger, transaction logger, and universal logger. Also, to address the space overhead, we have proposed different methods of populating the log files with the information of the successfully transferred objects. We have evaluated the data transfer performance and recovery time overhead of the proposed object-logging-based FT mechanisms on the LADS data transfer framework. Our experimental results reflect that FT mechanisms exhibit negligible overhead (<; 1%) with respect to the data transfer time. Yet, the fault recovery time is 10% higher than the total data transfer time at any fault point.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Adaptive multi-level checkpointing

In some examples, with respect to adaptive multi-level checkpointing, a transfer parameter associated with transfer of checkpoint data from a node-local storage to a parallel file system may be ascertained for the checkpoint data stored in the node-local storage. The transfer parameter may be compared to a specified transfer parameter threshold. A determination may be made, based on the comparison of the transfer parameter to the specified transfer parameter threshold, as to whether to transfer the checkpoint data from the node-local storage to the parallel file system.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Magnetotelluric Data from Mountain Home, ID

This dataset includes magnetotelluric transfer functions in the form of EDI files for 16 stations collected by the USGS and 40 stations collected by Quantec Geoscience for Lawerence Berkeley National Lab around the Mountain Home area in Idaho. A 3D electrical resistivity model is included that images resistive and conductive bodies in the subsurface that maybe important for geothermal characterization. The model was created using ModEM using the high performance computer Yeti at the USGS.

15 GEOTHERMAL ENERGY↗