Search NASA⌕ Search

SEARCH · Search NASA

Results for “Network Performance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

SharP

The pre-exascale systems are expected to have a significant amount of hierarchical and heterogeneous on-node memory, and this trend of system architecture in extreme-scale systems is expected to continue into the exascale era. along with hierarchical-heterogeneous memory, the system typically has a high-performing network ad a compute accelerator. This system architecture is not only effective for running traditional High Performance Computing (HPC) applications (Big-Compute), but also for running data-intensive HPC applications and Big-Data applications. As a consequence, there is a growing desire to have a single system serve the needs of both Big-Compute and Big-Data applications. Though the system architecture supports the convergence of the Big-Compute and Big-Data, the programming models and software layer have yet to evolve to support either hierarchical-heterogeneous memory systems or the convergence. A programming abstraction to address this problem. The programming abstraction is implemented as a software library and runs on pre-exascale and exascale systems supporting current and emerging system architecture. Using distributed data-structures as a central concept, it provides (1) a simple, usable, and portable abstraction for hierarchical-heterogeneous memory and (2) a unified programming abstraction for Big-Compute and Big-Data applications.

Venkata, Manjunath Gorentla↗

Challenges for unsupervised anomaly detection in particle physics

Anomaly detection relies on designing a score to determine whether a particular event is uncharacteristic of a given background distribution. One way to define a score is to use autoencoders, which rely on the ability to reconstruct certain types of data (background) but not others (signals). In this paper, we study some challenges associated with variational autoencoders, such as the dependence on hyperparameters and the metric used, in the context of anomalous signal (top and W) jets in a QCD background. We find that the hyperparameter choices strongly affect the network performance and that the optimal parameters for one signal are non-optimal for another. In exploring the networks, we uncover a connection between the latent space of a variational autoencoder trained using mean-squared-error and the optimal transport distances within the dataset. We then show that optimal transport distances to representative events in the background dataset can be used directly for anomaly detection, with performance comparable to the autoencoders. Whether using autoencoders or optimal transport distances for anomaly detection, we find that the choices that best represent the background are not necessarily best for signal identification. These challenges with unsupervised anomaly detection bolster the case for additional exploration of semi-supervised or alternative approaches.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

The Portals 4.1 Network Programming Interface

This report presents a specification for the Portals 4 networ k programming interface. Portals 4 is intended to allow scalable, high-performance network communication betwee n nodes of a parallel computing system. Portals 4 is well suited to massively parallel processing and embedded syste ms. Portals 4 represents an adaption of the data movement layer developed for massively parallel processing platfor ms, such as the 4500-node Intel TeraFLOPS machine. Sandia's Cplant cluster project motivated the development of Version 3.0, which was later extended to Version 3.3 as part of the Cray Red Storm machine and XT line. Version 4 is tar geted to the next generation of machines employing advanced network interface architectures that support enh anced offload capabilities.

97 MATHEMATICS AND COMPUTING↗

FTP 100: An Ultra-High Speed Data Transfer Service Over Next Generation 100 Gigabit Per Second Network

Data-intensive applications, including high energy and nuclear physics, astrophysics, climate modeling, nano-scale materials science, genomics, and financing, are expected to generate exabytes of data over the coming years, which must be transferred, visualized, and analyzed by geographically distributed teams of users. High-performance network capabilities must be available to these users at the application level in a transparent, virtualized manner. Moreover, the application users must have the capability to move large datasets from local and remote locations across network environments to their home institutions. To solve these challenges, the main goal of our project is to design and evaluate high-performance data transfer software to support various data-intensive applications. First, we have designed a middleware software that provides access to Remote Direct Memory Access (RDMA) functionalities. This middleware integrates network access, memory management and multitasking in its core design. We address a number of issues related to its efficient implementation, for instance, explicit buffer management and memory registration, and parallelization of RDMA operations, which are vital to delivering the benefit of RDMA to the applications. Built on top of this middleware, an implementation and experimental evaluation of the RDMA-based FTP software, RFTP, is described and evaluated. This application has been implemented by our team to exploit the full capabilities of advanced RDMA mechanisms for ultra-high speed bulk data transfer applications on Energy Sciences Network (ESnet). Second, we designed our data transfer software to optimize TCP/IP based data transfer performance such that RFTP can be fully compatible with today’s Internet. Our kernel optimization techniques with Linux system calls sendfile and splice, can reduce data copy cost. In this report, we summarize the technical challenges of our project, the primary software design methods, the major project milestones achieved, as well as the testbed evaluation work and demonstrations during our project life time.

97 MATHEMATICS AND COMPUTING↗

Advancing Artificial Intelligence with Liquid Argon Neutrino Experiments (Technical Report)

The grant allowed two main contributions: 1) The development of a first successful demonstration of the employment of Optimal Transport in liquid argon time projection chamber neutrino detectors. Optimal Transport, used in other contexts and specifically with LHC calorimetric data, was adapted to address a key particle identification challenge in LArTPCs: the separation of pi0 backgrounds from single-electrons produced in charged-current electron neutrino interactions. The work, leveraging ML methods such as k-nearest-neighbor (kNN) and support-vector-machine (SVM), showed an increase in background rejection of a factor of two or more. Work is now ongoing to incorporate this development in physics analyses for LArTPC experiments and more broadly expand the use of OT in LArTPC detectors including DUNE. This work was done in collaboration with the phenomenology group led by Nathaniel Craig at UCSB. 2) The deployment of NuGraph2, a graph neural network developed for LArTPC reconstruction, in the MicroBooNE experiment. NuGraph2 uses novel graph-neural-network methods on the rather simple LArTPC inputs of reconstructed hits, greatly simplifying the workflow compared to the use of waveform or signal-deconvolved wire ROIs. The network performed particle classification and was shown to address many challenging problems in LArTPC imaging including track-shower separation and the identification of protons and charged pions from primary muons. Our group collaborated with Giuseppe Cerati (FNAL scientist) who is one of the core developers of NuGraph2 to integrate this tool in MicroBooNE’s analysis framework. This consisted in tow key contributions: a) Studying performance on real data, which came with several months of iterations because the MC-trained version of the network was found to show significant bias that our group investigated and addressed. b) Integrating the output hit labeling of NuGraph2 into the existing particle tracking and shower reconstruction code. As a result of this work led by our team NuGraph2 is now enabling a suite of new analyses which benefit from enhanced capabilities and thus broader physics reach. The grant supported primarily the salary of UCSB graduate student Chuyue “Michaelia” Fang as well as partial summer salary support for PI Caratelli. Some funds were used for travel by Michaelia to ML related schools and conferences.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Improving the NOvA 3-Flavour Neutrino Oscillation Analysis

NOvA is a long-baseline neutrino experiment studying neutrino oscillations, a quantum mechanical interference phenomena where the observed neutrino flavour differs from that measured earlier, stemming from neutrino mass and flavour states mixing. NOvA consists of two functionally identical tracking calorimeter detectors deployed in the Fermilab NuMI beam. Both detectors are placed 14.6 mrad off the beam axis to achieve a narrow energy peak at 1.8 GeV at an oscillation maximum. The NOvA 3-flavour oscillation analysis measures the neutrino oscillation parameters sin2θ23 and ∆m232 as well as sets limit to δCP , the octant of θ23 and the sign of ∆m32. The event selection for 3-flavour neutrino oscillation analysis ensures the maximum quantity of signal is made available and a minimal amount of background is present. The current disappearance analysis selection has an efficiency of 80% for selecting νμ CC events, and with some improvements additional events could be recovered into the analysis to improve the sensitivity to the aforementioned oscillation parameters. In an effort to recover the currently rejected signal event to the analysis, these events were trained in a classification neural network. The aim of the network was to divide the data into signal (νμ CC events) and background (NC and νe CC events). The highest performing network gave an additional figure of merit gain of 2.34 increasing the sensitivity by 4.3% in effective POT equal to 33 days of additional data taking. This was compared to changing the current particle identification event selection cuts, the best result out of the tested cut combinations gave an additional FOM of 3.36 equal to 3.7% increase in effective POT equivalent to 28 days of data taking.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Open MPI for Cray XE/XK Systems

Open MPI provides an implementation of the MPI standard supporting communications over a range of high-performance network interfaces. Recently, ORNL and LANL have collaborated on creating a port of Open MPI for Gemini, the network interface for Cray XE and XK systems. In this paper, we present our design and implementation of Open MPI's point-to-point and collective operations for Gemini, and techniques we employ to provide good scaling, and performance characteristics.

97 MATHEMATICS AND COMPUTING↗

Quantum adiabatic machine learning by zooming into a region of the energy surface

Recent work has shown that quantum annealing for machine learning, referred to as QAML, can perform comparably to state-of-the-art machine learning methods with a specific application to Higgs boson classification. Here, we propose QAML-Z, an algorithm that iteratively zooms in on a region of the energy surface by mapping the problem to a continuous space and sequentially applying quantum annealing to an augmented set of weak classifiers. Results on a programmable quantum annealer show that QAML-Z matches classical deep neural network performance at small training set sizes and reduces the performance margin between QAML and classical deep neural networks by almost 50% at large training set sizes, as measured by area under the receiver operating characteristic curve. The significant improvement of quantum annealing algorithms for machine learning and the use of a discrete quantum algorithm on a continuous optimization problem both opens a class of problems that can be solved by quantum annealers and suggests the approach in performance of near-term quantum machine learning towards classical benchmarks.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Idaho National Laboratory Quality Of Service Dataset

The code is designed to run tests to generate and collect data from a Wi-Fi network using OPENWRT or a simulated a 5G network using Open5gs and UERANSIM. The tests simulate the network performing downloads or uploads of various files with a varying number of concurrent users. The tests use tcpdump to collect the network traffic but only stores the summarized data. The summarized datasets will be included.

Krome, Cameron [Idaho National Laboratory (INL), I↗

Evolving Energy Efficient Convolutional Neural Networks

As deep neural networks have been deployed in more and more applications over the past half decade and are finding their way into an ever increasing number of operational systems, their energy consumption becomes a concern whether running in the datacenter or on edge devices. Hyperparameter optimization and automated network design for deep learning is a quickly growing field, but much of the focus has remained only on optimizing for the performance of the machine learning task. In this work, we demonstrate that the best performing networks created through this automated network design process have radically different computational characteristics (e.g. energy usage, model size, inference time), presenting the opportunity to utilize this optimization process to make deep learning networks more energy efficient and deployable to smaller devices. Optimizing for these computational characteristics is critical as the number of applications of deep learning continues to expand.

Young, Steven↗

Flexible Bandwidth Scheduling for Streaming Data Movement Over Dedicated Networks

A wide range of scientific disciplines are generating large amounts of data at a high speed, which must be transferred to remote sites for real-time processing. Reserving bandwidths over dedicated channels in high-performance networks (HPNs) within a specified time interval has proved to be an effective solution to such high-demanding data transfer. Given a bandwidth reservation request, if the desired bandwidth within the specified time interval cannot be satisfied, most of the existing scheduling algorithms simply reject the request, which would immediately terminate the application. One reasonable approach to mitigate this issue is to provide an alternative bandwidth reservation option to schedule the desired bandwidth within the time interval closest to the specified one. We propose a flexible bandwidth reservation algorithm that considers both the best and alternative bandwidth reservation options for a given request. Extensive simulations are conducted to show the superior performance of the proposed scheduling algorithm compared with a heuristic approach adapted from existing scheduling algorithms.

Zuo, Liudong↗

High-performance and scalable on-chip digital Fourier transform spectroscopy

Abstract On-chip spectrometers have the potential to offer dramatic size, weight, and power advantages over conventional benchtop instruments for many applications such as spectroscopic sensing, optical network performance monitoring, hyperspectral imaging, and radio-frequency spectrum analysis. Existing on-chip spectrometer designs, however, are limited in spectral channel count and signal-to-noise ratio. Here we demonstrate a transformative on-chip digital Fourier transform spectrometer that acquires high-resolution spectra via time-domain modulation of a reconfigurable Mach-Zehnder interferometer. The device, fabricated and packaged using industry-standard silicon photonics technology, claims the multiplex advantage to dramatically boost the signal-to-noise ratio and unprecedented scalability capable of addressing exponentially increasing numbers of spectral channels. We further explore and implement machine learning regularization techniques to spectrum reconstruction. Using an ‘elastic-D 1 ’ regularized regression method that we develop, we achieved significant noise suppression for both broad (>600 GHz) and narrow (<25 GHz) spectral features, as well as spectral resolution enhancement beyond the classical Rayleigh criterion.

47 OTHER INSTRUMENTATION↗

Assessing the Application of a Genomic Network Analysis in Population Ecology: Inferring Patterns of Dispersal and Geographic Structure in the Emerging Pathogen, Coccidioides

A challenge in population ecology studies is identifying how to best group individuals into populations, especially when individual origin is unknown. Machine learning has improved upon traditional methods of identifying population structure and is more efficient at handling large, complex datasets. We demonstrate the applicability of a machine learning method to identify hierarchical population structure in an emerging pathogen, Coccidioides spp., the causative agent of Valley fever. We compared the network clusters to structure identified by traditional tools as a validation of the network performance. We used publicly available whole-genome data for 48 C. immitis and 102 C. posadasii, resulting in 168,211 genome-wide SNPs among the two species. The network analysis grouped samples into populations comparable to the literature for these species but also identified fine-scale geographic structure and travel-associated cases not reported thus far. Exploring different resolutions in the network made it easy to identify unique genotypes specific to California and possibly Nevada, as well as Phoenix- and Tucson-acquired infections in non-endemic areas, regardless of reported travel history. The present study provides a promising example of how a ML-based network analysis can improve our ability to understand pathogen ecology, group cases into populations and infer travel-associated infections.

59 BASIC BIOLOGICAL SCIENCES↗

MiniMOD

SAND2025-03854O MiniMod is a user-friendly software tool designed to assess the performance of high-performance computing (HPC) systems. Researchers can use the program to test communication methods and computational tasks to understand how different setups can affect application efficiency. This software is particularly useful for optimizing network performance in scientific research, simulations, and data analysis. MiniMod‘s flexible design allows users to make informed decisions about their computing environments, which can enhance productivity and results in real-world applications. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Dosanjh, Matthew [Sandia National Lab. (SNL-CA), L↗

Parallel application-level behavioral attributes for performance and energy management of high-performance computing systems

Run time variability of parallel applications continues to present significant challenges to their performance and energy efficiency in high-performance computing (HPC) systems. When run times are extended and unpredictable, application developers perceive this as a degradation of system (or subsystem) performance. Extended run times directly contribute to proportionally higher energy consumption, potentially negating efforts by applications, or the HPC system, to optimize energy consumption using low-level control techniques, such as dynamic voltage and frequency scaling (DVFS). Therefore, successful systemic management of application run time performance can result in less wasted energy, or even energy savings. We have been studying run time variability in terms of communication time, from the perspective of the application, focusing on the interconnection network. More recently, our focus has shifted to developing a more complete understanding of the effects of HPC subsystem interactions on parallel applications. In this context, the set of executing applications on the HPC system is treated as a subsystem, along with more traditional subsystems like the communication subsystem, storage subsystem, etc. To gain insight into the run time variability problem, our earlier work developed a framework to emulate parallel applications (PACE) that stresses the communication subsystem. Evaluation of run time sensitivity to network performance of real applications is performed with a tool called PARSE, which uses PACE. In this paper, we propose a model defining application-level behavioral attributes, that collectively describes how applications behave in terms of their run time performance, as functions of their process distribution on the system (spacial locality), and subsystem interactions (communication subsystem degradation). These subsystem interactions are produced when multiple applications execute concurrently on the same HPC system. We also revisit our evaluation framework and tools to demonstrate the flexibility of our application characterization techniques, and the ease with which attributes can be quantified. The validity of the model is demonstrated using our tools with several parallel benchmarks and application fragments. Results suggest that it is possible to articulate application-level behavioral attributes as a tuple of numeric values that describe course-grained performance behavior.

Computer Science↗

Toward a persistent event-streaming system for high-performance computing applications

High-performance computing (HPC) applications have traditionally relied on parallel file systems and file transfer services to manage data movement and storage. Alternative approaches have been proposed that use direct communications between application components, trading persistence and fault tolerance for speed. Event-driven architectures, as popularized in enterprise contexts, present a compelling middle ground, avoiding the performance cost and API constraints of parallel file systems while retaining persistence and offering impedance matching between application components. However, adapting streaming frameworks to HPC workloads requires addressing challenges unique to HPC systems. This paper investigates the potential for a streaming framework designed for HPC infrastructures and use cases. We introduce Mofka, a persistent event-streaming framework designed specifically for HPC environments. Mofka combines the capabilities of a traditional streaming service with optimizations tailored to the HPC context, such as support for massively multicore nodes, efficient scaling for large producer-consumer workflows, RDMA-enabled high-performance network communications, specialized network fabrics with multiple links per node, and efficient handling of large scientific data payloads. Built using the Mochi suite of HPC data service components, Mofka provides a lightweight, modular, and high-performance solution for persistent streaming in HPC systems. We present the architecture of Mofka and evaluate its performance against Kafka and Redpanda using benchmarks on diverse platforms, including Argonne's Polaris and Oak Ridge's Frontier supercomputers, showing up to 8× improvement in throughput in some scenarios. We then demonstrate its utility in several real-world applications: a tomographic reconstruction pipeline, a workflow for the discovery of metal-organic frameworks for carbon capture, and the instrumentation of Dask workflows for provenance tracking and performance analysis.

HPC↗

End-System Aware Peak Link Utilization Transport for Big Data Transfer over Dedicated Connections

High-performance networks provisioning 10/40-Gbps dedicated channels have been rapidly developed andincreasingly deployed to support big data transfers on the order of terabytes or petabytes in large-scale network-intensiveapplications. However, end users have not seen corresponding increase in application throughput mainly because traditional end-to-end transport methods are not optimized for such connections. New congestion or flow control mechanisms are desirable to meet the challenges brought to transport protocol design by dedicated connections. The advent and proliferation of multi-core processors make it now possible to improve application throughput by providing multiple processing and network resources to a single data transfer. Based on the existing PLUT method, we proposed a new dynamic buffer management scheme and an automatic parallelism tuning mechanism to achieve the maximal bottleneck rate. The performance superiority of PLUT is justified by the experimental results collected over a local 10Gbps dedicated connection.

Rao, Nageswara↗

Status and trends in networking at LHC Tier1 facilities

The LHC is entering its fourth year of production operation. Most Tier1 facilities have been in operation for almost a decade, when development and ramp-up efforts are included. LHC's distributed computing model is based on the availability of high capacity, high performance network facilities for both the WAN and LAN data movement, particularly within the Tier1 centers. As a result, the Tier1 centers tend to be on the leading edge of data center networking technology. In this paper, we analyze past and current developments in Tier1 LAN networking, as well as extrapolating where we anticipate networking technology is heading. Our analysis will include examination into the following areas: Evolution of Tier1 centers to their current state Evolving data center networking models and how they apply to Tier1 centers Impact of emerging network technologies (e.g. 10GE-connected hosts, 40GE/100GE links, IPv6) on Tier1 centers Trends in WAN data movement and emergence of software-defined WAN network capabilities Network virtualization

Bobyshev, A.↗