Search NASA⌕ Search

SEARCH · Search NASA

Results for “data transfer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Characterization of throughput on the AXI DMA bus for burst data transfer over Ethernet

cThe Xilinx AXI Direct Memory Access (AXI DMA) module is an efficient solution for medium-speed data transfer in Xilinx SoC FPGAs, supporting data rates greater than 1000 Gbps even in very suboptimal operating modes. It facilitates direct transfer of AXI stream data into processor memory without constant software intervention, which reduces overhead and ensures consistent data logging. By utilizing the FPGA's available memory, large circular buffers (1-5 GiB) are used to buffer data and accommodate network limitations, enabling high-rate data bursts. In this study, we measured the performance of AXI DMA under conditions simulating its lowest practical data transfer speeds. The Arbitrary Length Data Sender was used to transmit AXI stream packets at 32-bit width and 100 MHz frequency, a narrow width and slow speed. Results show that the AXI DMA can transfer up to 3192.76 Mbps with large packet sizes but experiences reduced performance for smaller packets, as low as 2.6 Mbps for 4-byte packets. For Ethernet-limited applications, packet sizes between 8,000 and 16,000 bytes provided optimal transfer speeds of 874 to 1600 Mbps. These findings suggest that the AXI DMA is not the limiting factor in systems where packet sizes exceed 8,000 bytes.

43 PARTICLE ACCELERATORS↗
DOE OSTI2025

Data Management in the Continuum: Cross-facility Object-based Data Transfers

Scientific workflows are evolving from relying on a monolithic storage subsystem at a single High-Performance Computing (HPC) facility to using geographically distributed file systems, repositories, and cloud storage. As a result, storing, accessing, transferring, and managing scientific data have become highly complex and prone to performance inefficiencies. This paper delves into these challenges by exploring an optimized end-to-end interface designed to seamlessly connect various local and remote storage systems, enabling efficient data movement of objects across HPC–Cloud and HPC–HPC environments. We showcase this capability through an object-focused data management runtime system, discuss the effects of relaxed consistency semantics in distributed object scenarios, and illustrate its application in an earthquake simulation workflow. Besides reducing the amount of data by selectively transferring regions of interest, our facility-local results achieved a speedup of 45 × over an optimized HDF5 usage and 15 × over the HDF5 with caching by using the new interface in PDC-XF.

Bez, Jean Luca↗
DOE OSTI2025

Piezo-Antenna: An Electromagnetic-Ultrasonic Data Transfer System

Electronic systems contained within sealed metallic enclosures, such as nuclear waste containers where wire penetration is not an option, require a through-wall data transfer technology to communicate. One such technology utilizes piezoelectric ultrasonic transducers to communicate through the barrier using elastic waves. Currently demonstrated ultrasonic through-wall data communication techniques have relied on having a direct interface with the ultrasonic transducers on either side. Here, this work studies the coupling of an antenna to such an ultrasonic communication system. A computationally efficient model of a piezobarrier ultrasonic communication system and antenna using a simplified antenna model and a transfer matrix piezobarrier model is utilized to perform a representative parameter space search. The system efficiency and bandwidth trends are shown for the system parameters as well as operating frequency. The tradeoff between efficiency and bandwidth is also presented. A system configuration within the parameter space is fabricated and demonstrated to achieve a data rate of up to 100 kb/s.

Antenna↗
DOE OSTI2026

Data for Photoenzymatic Stereoablative Enantioconvergence of γ-chiral Oximes via Hydrogen Atom Transfer

Producing enantioenriched molecules from racemic mixtures is essential for manufacturing. Traditional methods such as resolution, deracemization and enantioconvergent catalysis primarily involve separating or converting enantiomers without altering their structures, or functionalization of stereocentres at or proximal to functional groups. However, there are challenges in enantioselectively forging C–H bonds that are remote from functional groups via hydrogen atom transfer (HAT) with these methods. Here we introduce a strategy for the photoenzymatic stereoablative enantioconvergence of γ-chiral oximes using repurposed flavin-dependent ene-reductases. A photoinduced single-electron reduction of the γ-chiral oxime by an ene-reductase generates an iminyl radical, which then undergoes stereoablative 1,5-HAT at the γ-stereocentre. Subsequent chiral reconstruction through enzymatic HAT and spontaneous imine hydrolysis yields the γ-chiral ketone with high enantioselectivity. This work provides a robust method for remote stereoablative enantioconvergent HAT and broadens the synthetic utility of photobiocatalysis.

Bioproducts↗
DOE OSTI2024

Field Validation of MVA Technology for Offshore CCS: Novel Ultra-High-Resolution 3D Marine Seismic Technology (P-Cable) (Final Report)

The objectives of the proposed study were to deploy and validate a specific monitoring technology, high-resolution 3D marine seismic (HR3D), appropriate for large-demonstration and commercial-scale offshore CCS sites. The project accomplished successful acquisition two HR3D seismic surveys. The first HR3D dataset was over the offshore injection site of the Tomakomai, Japan integrated pilot CCS project, which at the time of survey acquisition was actively injecting CO 2 . The first survey also represented a successful international collaboration between the DOE NETL program and Japan’s national CCS program and was the first successful acquisition and use of HR3D over an active CO 2 injection site (Meckel, Feng et al. 2019). The Tomakomai HR3D survey successfully tested a novel 4-streamer HR3D system array in which, for the first time, no cross-cable (aka “P-Cable”) was utilized and only four GeoEel streamers were used instead of the standard 12-streamer configuration. Consequently, this was not, strictly speaking, a deployment of the “P-Cable” system of (Planke and Berndt 2004) but rather a modified version, thereof, and it is the first known demonstration of the modified system configuration. One very positive outcome from the Japanese collaboration earlier in the project was the ability to learn from the Japanese how they used tail buoys with GPS to determine the position of the seismic source and receivers in time and space. Based on that experience, GCCC designed and built six GPS receivers that could be used to position the streamer receivers and the seismic source via tail buoys. A fundamental advance that was made on the original design, was the ability to directly power the tail buoy GPS units and transfer data through the streamers (i.e., vs. the batteries used at Tomakomai). The bulkiness of the GPS batteries caused drag and episodic surging of the buoys, which affected data quality by lifting up the tail end of the streamers so the receivers were not at the same depth. The units were tested onshore for accuracy and functionality, and the design was subsequently and successfully tested in marine acquisition mode during the SLP survey acquisition. The marine acquisition test and survey satisfied Subtasks 2.2.2, Novel Positioning Technology Selection and Subtask 2.2.3, Novel Positioning Technology Deployment. Results of the novel positioning technology selection (Subtask 2.2.2) were considered successful and will be incorporated in future HR3D seismic acquisition projects to reduce costs, improve deployment safety at sea, and integrate both seismic and data recording via a single data transfer through the streamers to the recording system. The project also established a permitting process through NETL NEPA compliance, which included an Environmental Assessment in a marine setting and is required for conducting these types of surveys using Federal funding. The permitting process charted a “boilerplate,” which can allow future surveys related to other funded projects to move forward more expeditiously. Future improvements that could be considered are more robust seals on the GPS module and stronger materials (especially joints) on tail buoy fabrication. These would increase fixed costs, but would be advisable and probably more economic long-term if multiple HR3D surveys are planned. Project Accomplishments include: • Pre-survey Sensitivity Study • Marine geochemistry methods and data analysis • Successful HR3D seismic dataset acquired @ Tomakomai active CO 2 injection marine site • Developed advanced seismic processing techniques • No NRMS anomalies detected in overburden; Demonstration of containment • Repeatability study • Second survey collected @ San Luis Pass, TX • 4D application using positioning techniques developed in the project for monitoring were successful

3D seismic GPS positioning↗
DOE OSTI2025

Towards an IPv6-only WLCG: More successes in reducing IPv4

The Worldwide Large Hadron Collider Computing Grid (WLCG) community’s deployment of dual-stack IPv6/IPv4 on its worldwide storage infrastructure has been very successful. Dual-stack is not, however, a viable longterm solution; the HEPiX IPv6 Working Group has focused on studying where and why IPv4 is still being used, and how to flip such traffic to IPv6. The agreed end goal is to turn IPv4 off and run IPv6-only over the wide-area network to simplify both operations and security management.This paper reports our work since the CHEP2023 conference. Firstly, we present our campaign to deploy IPv6 on CPU services and Worker Nodes, with a deadline of end of June 2024. Then, the WLCG Data Challenge (DC24) performed in February 2024 was an excellent opportunity to observe the percentage of data transfers carried by IPv6. We observed the predominance of IPv6 in data transfers during DC24 and were able to understand yet more reasons for the use of IPv4 and areas for remedial action.The paper ends with the working group’s plans for moving WLCG to “IPv6- only”. One aspect of this is the possible automated use of IPv6-only clients configured with a customer-side translator, or CLAT, together with a deployment of NAT64 using what is often known as “IPv6-Mostly”, enabling IPv6-only sites to connect to non-WLCG IPv4-only services.

Attebury, Garhan [U. Nebraska, Lincoln]↗
DOE OSTI2024

Error-controlled Progressive Retrieval of Scientific Data under Derivable Quantities of Interest

The unprecedented amount of scientific data has introduced heavy pressure on the current data storage and transmission systems. Progressive compression has been proposed to mitigate this problem, which offers data access with on-demand precision. However, existing approaches only consider precision control on primary data, leaving uncertainties on the quantities of interest (QoIs) derived from it. In this work, we present a progressive data retrieval framework with guaranteed error control on derivable QoIs. Our contributions are three-fold. (1) We carefully derive the theories to strictly control QoI errors during progressive retrieval. Our theory is generic and can be applied to any QoIs that can be composited by the basis of derivable QoIs proved in the paper. (2) We design and develop a generic progressive retrieval framework based on the proposed theories, and optimize it by exploring feasible progressive representations. (3) We evaluate our framework using five real-world datasets with a diverse set of QoIs. Experiments demonstrate that our framework can faithfully respect any user-specified QoI error bounds in the evaluated applications. This leads to over 2.02× performance gain in data transfer tasks compared to transferring the primary data while guaranteeing a QoI error that is less than 1E-5.

Wu, Xuan↗
DOE OSTI2024

Streaming Large-Scale Microscopy Data to a Supercomputing Facility

Data management is a critical component of modern experimental workflows. As data generation rates increase, transferring data from acquisition servers to processing servers via conventional file-based methods is becoming increasingly impractical. The 4D Camera at the National Center for Electron Microscopy generates data at a nominal rate of 480 Gbit s -1 (87,000 frames s -1 ⁠), producing a 700 GB dataset in 15 s. To address the challenges associated with storing and processing such quantities of data, we developed a streaming workflow that utilizes a high-speed network to connect the 4D Camera’s data acquisition system to supercomputing nodes at the National Energy Research Scientific Computing Center, bypassing intermediate file storage entirely. In this work, we demonstrate the effectiveness of our streaming pipeline in a production setting through an hour-long experiment that generated over 10 TB of raw data, yielding high-quality datasets suitable for advanced analyses. Additionally, we compare the efficacy of this streaming workflow against the conventional file-transfer workflow by conducting a postmortem analysis on historical data from experiments performed by real users. Our findings show that the streaming workflow significantly improves data turnaround time, enables real-time decision-making, and minimizes the potential for human error by eliminating manual user interactions.

4D-STEM↗
DOE OSTI2025

Enhancing Cluster Identification in Atom Probe Tomography Data Using Transfer Learning

Atom Probe Tomography (APT) is a powerful technique for visualizing the atomic-scale distribution of solutes in materials, but quantitative cluster analysis of APT datasets remains a challenge due to the need for subjective parameter selection in clustering algorithms. While distance-based and density-based methods such as HDBSCAN are widely used, their performance is highly sensitive to user-defined parameters, which undermines reproducibility and accuracy. This study proposes an image-based, deep learning-aided workflow for automating parameter selection and cluster detection in APT data analysis. By projecting 3D APT point clouds onto 2D planes, we leverage pretrained convolutional neural networks (ConvNeXt-Tiny and ResNet-50) through transfer learning to predict the number of clusters present in synthetic datasets. The output is used to guide K-means clustering and estimate HDBSCAN parameters, specifically minimum cluster size and minimum sample points. This approach reduces reliance on manual parameter tuning, improving consistency and scalability. The methodology demonstrates the feasibility of using image-based deep learning for interpreting complex spatial patterns in APT data, enabling faster and more objective analysis. The complete workflow and code are made publicly available to support reproducibility and future research.

Density-based clustering↗
DOE OSTI2024

Enabling Low-Overhead HT-HPC Workflows at Extreme Scale using GNU Parallel

GNU Parallel is a versatile and powerful tool for process parallelization widely used in scientific computing. This paper demonstrates its effective application in high-performance computing (HPC) environments, particularly focusing on its scalability and efficiency in executing large-scale high-throughput high-performance computing (HT-HPC) workflows. Through real-world examples, we highlight GNU Parallel’s performance across various HPC workloads, including GPU computing, container-based workloads, and node-local NVMe storage. Our results on two leading supercomputers, OLCF’s Frontier and NERSC’s Perlmutter, showcase GNU Parallel’s rapid process dispatching ability and its capacity to maintain low overhead even at extreme scales. We explore GNU Parallel’s application in massive parallel file transfers using a scheduled Data Transfer Node (DTN) cluster, emphasizing its broad utility in diverse scientific workflows. Beyond its direct application as a viable workflow manager, GNU Parallel can be employed in conjunction with other workflow systems as a "last-mile" parallelizing driver and as a quick prototyping tool to design and extract parallel profiles from application executions. We then argue that the potential for GNU Parallel to transform workflow management at extreme scales is substantial, paving the way for more efficient and effective scientific discoveries.

Maheshwari, Ketan↗
DOE OSTI2025

Enhancing Cluster Identification in Atom Probe Tomography Data Using Transfer Learning

Atom probe tomography (APT) has enabled the direct visualization of solute clusters, providing valuable insights into material structures. This clustering is crucial for understanding the nanoscale composition and behavior of materials, which can significantly influence their mechanical and physical properties. However, the widely used clustering methods in the APT community face challenges such as subjective parametric selection and limited applicability, particularly in dealing with overlapping clusters, nested clusters, and artifacts across different scales, such as precipitates and dislocations. To address these challenges, we present a framework based on density-based cluster analysis that aims to be less dependent on user input, reproducible, and robust.

Density-based clustering↗
DOE OSTI2025

Portable and Adaptable Neutron Diagnostics for Advancing Fusion Energy Science Addendum

Activation detectors developed at LLNL for measuring real-time neutron fluence from fusion sources are used in the broader fusion community. The recommended fluence operating range of this diagnostic is 5x10 2 – 1x10 6 n/cm2. The upper limit on this fluence range is set by the dead time caused by data transfer between the detector and data acquisition computer. Delaying the start of counting is a possible strategy to operate these detectors in higher fluences.

42 ENGINEERING↗
DOE OSTI2025

DoCeph: DPU-Offloaded Messaging in Ceph for Reduced Host CPU Utilization

Ceph is a widely used distributed object store, but its messenger layer imposes substantial CPU overhead on the host. To address this limitation, we propose DoCeph, a DPU-offloaded storage architecture for Ceph that disaggregates the system by offloading the communication-intensive messaging component to the DPU while retaining the storage backend on the host. The DPU efficiently manages communication, using lightweight RPC for metadata operations and DMA for data transfer. Moreover, DoCeph introduces a pipelining technique that overlaps data transmission with buffer preparation, mitigating hardware-imposed transfer size limitations. We implemented DoCeph on a Ceph cluster with NVIDIA BlueField-3 DPUs. Evaluation results indicate that DoCeph cuts host CPU usage by up to 92% while sustaining stable throughput and providing larger performance benefits for object writes over 1 MB.

Park, Kuri [Sogang University]↗
DOE OSTI2026

FIRM: federated image reconstruction using multimodal tomographic data

Here, we propose a federated algorithm for reconstructing images using multimodal tomographic data sourced from dispersed locations, addressing the challenges of traditional unimodal approaches that are prone to noise and reduced image quality, as well as the limitations of centralized multimodal approaches that require extensive data transfer, leading to significant communication overhead, storage demands, and potential data privacy concerns. Our approach formulates a joint inverse optimization problem incorporating multimodality constraints and solves it in a federated framework through local gradient computations complemented by lightweight central operations, thereby ensuring data decentralization. Leveraging the connection between our federated algorithm and the quadratic penalty method, we introduce an adaptive step-size rule with guaranteed sublinear convergence. Numerical results demonstrate superior computational efficiency and improved image reconstruction quality compared to existing approaches.

federated algorithm↗
DOE OSTI2024

To Derive or Not to Derive: I/O Libraries Take Charge of Derived Quantities Computation

The ever-increasing volume of data produced by HPC simulations necessitates scalable methods for data exploration and knowledge extraction. Scientific data analysis often involves complex queries across distributed datasets, requiring manipulation of multiple primary variables and generating derived data that needs to be handled efficiently, creating challenges for applications that need to parse many large datasets. Relying on individual applications to handle all intermediate data generally leads to redundant computations across studies and unnecessary data transfers. In this paper, we investigate the performance of different approaches where applications define derived variables as quantities of interest (QoIs) and offload the computation and transfer of these QoIs to the I/O library. This significantly reduces redundancy and optimizes data movement across the distributed storage and processing infrastructure by allowing control over when and where derived variables are computed. We present a detailed analysis of the performance-storage trade-offs associated with different solutions and showcase results for our study on two large-scale datasets created from climate and combustion simulations.

Gainaru, Ana↗
DOE OSTI2025

University Data Management Pilot Utilizing the Nuclear Research Data System

Background In 2022, the Office of Science and Technology Policy (OSTP) issued a memo that significantly reshaped the landscape of access to federally funded research. The memo mandated that all taxpayer-funded research be made available to the public without delay upon publication, without an embargo period, superseding the 2013 OSTP public access policy. This public access policy promotes transparency and the democratization of knowledge, ensuring that the fruits of scientific endeavors funded by federal agencies could be immediately accessed and built upon by scientists, educators, students, and the public at large. To implement the requirements of the OSTP guidance and DOE Public Access Plan, the Office of Nuclear Energy (NE) has implemented public access plan guidance and has identified several areas where better data management practices would further expand public access to important nuclear energy related scientific data, reports, and other technical products. Significant NE supported efforts are already underway for data management and public access to important nuclear energy related data.1 2 To address gaps in data management practices, and improve retention and accessibility of data, NE is actively exploring enhanced data management options utilizing its high-performance computing resources administered by its Nuclear Scientific User Facility Program. A newly piloted system, the Nuclear Research Data System (NRDS) acts as a portal for data collection and dissemination. Nuclear Energy University Program Research and Development Portfolio According to Web of Science, NEUP has produced 2,345 journal publication that have been cited more than 61,000 times3 and countless conference proceedings. These publications are publicly available through OSTI.gov and in the open literature. Additional scientific and technical products including project milestones that are not publications and NEUP project final reports are vetted through OSTI.gov and released once reviewed and approved by DOE. Since 2009, NEUP has awarded close to 1,000 different R&D projects in technical areas across the NE research programs. As of June 2023, 512 NEUP reports are publicly available on OSTI. The underlying data for projects is still held at universities, and data transfer, co-location, and dissemination has not occurred in a systematic way. NEUP data is currently accessible through myriad university-based data repositories, or through direct requests to PIs. The program identified this patchwork of repositories, or often lack of publicly available data, as a significant barrier to an organized, accessible, and comprehensive solution to sharing data with the larger nuclear energy community. Approach The goal of this pilot project is to establish a pathway to a consolidated long-term repository for NEUP project data. To accomplish this goal, the pilot strives to accomplish the following objectives: Establish data collection standards, including a standard set of required supplementary information to contextualize and support raw data files. Work with the HPC group collect and upload information and to modify the NRDS system, as needed, to support a standardized approach. Resolve potential barriers to successful roll out of an expanded data collection strategy, including modifying data management plan guidelines and establishing a document and data release process that accounts for potential intellectual property and/or export control concerns. Results Overall, the pilot was successful in collecting 8,982 raw and processes data files, 220 reports, 56 calibration files, and 5,931 other supplementary documents. Supplementary documents included experimental plans, methods, journal publications and conference proceedings, milestone reports, and final reports. Figure 2 shows the number of data sets and supplementary project information provided by each project. Projects has significantly different input, depending on experimental data produced and completeness of the datasets provided.

Data collection↗
DOE OSTI2026

Federated learning for 2D synchrotron x-ray diffractometry: a cross-institutional approach for phase quantification of Ti–6Al–4V alloy

High-energy Two dimensional (2D) synchrotron x-ray diffractometry provides important insights into the atomistic structure and phase evolution of materials, yet traditional analysis methods remain complex, knowledge-intensive, and computationally demanding. Deep-learning models offer a powerful alternative for automating their analysis. Institutions that hold these datasets may be unwilling to share their data due to privacy and security policies, as well as the challenges associated with large-scale data transfer. As a result, models trained on local datasets often perform well only on their own data but exhibit bias and poor generalization across different instruments or facilities. To overcome these limitations, we explore federated learning (FL) for 2D synchrotron diffractograms, enabling collaborative model training without exchanging raw data. In this study, 2D synchrotron diffractograms of Ti–6Al–4V alloy collected from two independent facilities are used to train convolutional neural networks for predicting the β-phase volume fraction. Experimental results show that federated global models significantly outperform locally trained models in terms of generalization and achieve accuracy comparable to centralized trained models. These findings demonstrate the potential of FL to enable secure, cross-institutional collaboration and enhance the scalability of deep-learning-based materials characterization.

36 MATERIALS SCIENCE↗
DOE OSTI2026

A scalable framework for efficient coupling of thermal and microstructural simulations in additive manufacturing

Predicting microstructure evolution in metal additive manufacturing (AM) is important for process optimization, but spatiotemporal scale disparities between thermal transport and microstructure evolution create significant challenges for efficient data transfer between simulation codes. To address this, we present Stork, a scalable framework for coupling thermal and microstructural simulations. Stork uses a sparse data representation to identify and store active solidification sub-volumes, enabling highly parallel quad-linear interpolation from coarse thermal grids to fine microstructure grids without large intermediate storage. We demonstrate the framework by coupling the semi-analytic heat transfer code 3DThesis with the time-parallel cellular automata code Toucan. This approach achieves over two orders of magnitude reduction in data generation time and file size compared to prior workflows. Numerical studies show that quad-linear interpolation preserves grain morphology and crystallographic texture in laser powder bed fusion (LPBF) simulations for coarsening ratios up to 16. Overall, Stork provides a scalable pathway for high-throughput, component-scale AM simulations on modern high-performance computing systems.

36 MATERIALS SCIENCE↗