Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Efficient Anomaly Detection Driven By Different Machine Learning Architectures And Models

The rapid growth and ubiquitous adoption of the internet and cyber-physical systems (CPS) have fundamentally transformed modern communication, work, and human-system interactions. While networks now form the backbone of critical digital ecosystems, enabling seamless data transmission across diverse, interconnected systems, this increased connectivity also expands the attack surface, making real-time detection of network intrusions and anomalies a pressing challenge. Detecting unusual activities within network infrastructure requires advanced data traffic analysis to differentiate between legitimate and malicious interactions. Traditional approaches to network anomaly detectionâ??such as rule-based and signature-based systemsâ??often depend on predefined patterns to identify known anomalies, limiting their effectiveness against emerging, stealthy, or previously unseen threats. These conventional methods suffer from high false alarm rates and fail to adapt to the ever-evolving nature of network traffic, particularly in large-scale, decentralized environments where data volume, velocity, and variety are constantly increasing. This dissertation presents artificial intelligence (AI)-driven approaches to anomaly detection that leverage graphics processing unit (GPU)-enabled high-performance computing (HPC) platforms for processing massive network traffic data and monitoring the components of cyber-physical systems (CPS) for potentially hazardous conditions. The research advances several key contributions: (1) Designing efficient machine learning techniques for CPS condition monitoring and anomaly detection; (2) enabling federated learning (FL) frameworks that enable distributed detection while preserving data privacy and system resilience; (3) exploring graph-based methodologies combining graph neural networks (GNN) and graph machine learning (ML) approaches for the Internet of Things (IoT) and automotive network security, and (4) performing distributed edge computing optimizations that integrate FL with scalable technologies for reduced communication overhead. Through extensive experiments, these methodologies demonstrate that complex anomaly detection and condition monitoring tasks can be achieved while balancing computational efficiency and detection accuracy through fine-grained network information processing. The frameworks developed in this research establish a robust foundation for network anomaly detection, providing scalable, adaptive, and privacy-preserving solutions for safeguarding CPS and IoT networks in an increasingly interconnected digital landscape. The practical implications of these research findings are significant, as they can inform the development of next-generation network security systems and contribute to the protection of critical infrastructure against sophisticated cyber attacks.

Marfo, William↗
DOE OSTI2025

Hybrid classical-quantum communication networks

Over the past several decades, the proliferation of global classical communication networks has transformed various facets of human society. Concurrently, quantum networking has emerged as a dynamic field of research, driven by its potential applications in distributed quantum computing, quantum sensor networks, and secure communications. This prompts a fundamental question: rather than constructing quantum networks from scratch, can we harness the widely available classical fiber-optic infrastructure to establish hybrid quantum–classical networks? This paper aims to provide a comprehensive review of ongoing research endeavors aimed at integrating quantum communication protocols, such as quantum key distribution, into existing lightwave networks. This approach offers the substantial advantage of reducing implementation costs by allowing classical and quantum communication protocols to share optical fibers, communication hardware, and other network control resources—arguably the most pragmatic solution in the near term. In the long run, classical communication will also reap the rewards of innovative quantum communication technologies, such as quantum memories and repeaters. Accordingly, our vision for the future of the Internet is that of heterogeneous communication networks thoughtfully designed for the seamless support of both classical and quantum communications.

Fiber-optic communication↗
DOE OSTI2024

Finding MIDDLE Ground: Scalable and Secure Distributed Learning

Edge computing methods allow devices to efficiently train a high-performing, robust, and personalized model for predictive tasks. However, these methods succumb to privacy and scalability concerns such as adversarial data recovery and expensive model communication. Furthermore, edge computing methods unrealistically assume that all devices train an identical model. In practice, edge devices have varying computational and memory constraints which may not allow certain devices to have the space or speed to train a specific model. To overcome these issues, we propose MIDDLE: a model independent distributed learning algorithm which allows heterogeneous edge devices to assist each other’s training while communicating only non-sensitive information. MIDDLE unlocks the ability for edge devices, regardless of computational or memory constraints, to assist each other even with completely different model architectures. Furthermore, MIDDLE does not require model or gradient communication which greatly reduces communication size and time. We prove that MIDDLE attains the optimal convergence rate O(1/sqrt(TM)) of stochastic gradient descent for convex and non-convex smooth optimization (for total iterations T and batch size M). Finally, our experimental results demonstrate that MIDDLE (even in non-IID data settings) attains robust and high-performing models without model or gradient communication.

Bornstein, Marc I.↗
DOE OSTI2024

WRF-Comfort: simulating microscale variability in outdoor heat stress at the city scale with a mesoscale model

Abstract. Urban overheating and its ongoing exacerbation due to global warming and urban development lead to increased exposure to urban heat and increased thermal discomfort and heat stress. To quantify thermal stress, specific indices have been proposed that depend on air temperature, mean radiant temperature (MRT), wind speed, and relative humidity. While temperature and humidity vary on scales of hundreds of meters, MRT and wind speed are strongly affected by individual buildings and trees and vary on the meter scale. Therefore, most numerical thermal comfort studies apply microscale models to limited spatial domains (commonly representing urban neighborhoods with building blocks) with resolutions on the order of 1 m and a few hours of simulation. This prevents the analysis of the impact of city-scale adaptation and/or mitigation strategies on thermal stress and comfort. To solve this problem, we develop a methodology to estimate thermal stress indicators and their subgrid variability in mesoscale models – here applied to the multilayer urban canopy parameterization BEP-BEM within the Weather Research and Forecasting (WRF) model. The new scheme (consisting of three main steps) can readily assess intra-neighborhood-scale heat stress distributions across whole cities and for timescales of minutes to years. The first key component of the approach is the estimation of MRT in several locations within streets for different street orientations. Second, mean wind speed and its subgrid variability are downscaled as a function of the local urban morphology based on relations derived from a set of microscale LES and RANS simulations across a wide range of realistic and idealized urban morphologies. Lastly, we compute the distributions of two thermal stress indices for each grid square, combining all the subgrid values of MRT, wind speed, air temperature, and absolute humidity. From these distributions, we quantify the high and low tails of the heat stress distribution in each grid square across the city, representing the thermal diversity experienced in street canyons. In this contribution, we present the core methodology as well as simulation results for Madrid (Spain), which illustrate strong differences between heat stress indices and common heat metrics like air or surface temperature both across the city and over the diurnal cycle.

Geology↗
DOE OSTI2025

Using Apptainer in a Pilot-based Distributed Workload

GlideinWMS is a pilot and pressure-based workload manager for distributed scientific computing. Many experiments like CMS and Fermilab’s Neutrino experiments use it to provision elastic clusters for their analysis and simulations, split into close to a million concurrent jobs. Most user jobs require containers, and the pilots use Apptainer to set up the desired platform. For the pilots that run as regular batch jobs, Apptainer is safer, lighter, and easier to use than other containerization solutions. Many images used by the pilots are expanded SIF images distributed via the CernVM-FS: this combination is very efficient. At Fermilab, for example, we store on GitHub Dockerfiles that mimic the platform in the worker nodes of local clusters. GitHub workflows build and push the images to Docker Hub, and a service periodically pulls and converts them to the expanded SIF images in the CernVM-FS, so the scientists can find a familiar environment everywhere. Apptainer has also been used to run services inside the pilot jobs, like benchmarks that characterize the worker node being used, or a Triton Inference Server that allows sharing a GPU with all the jobs that run in parallel on a node.

Mambelli, Marco [Fermilab] (ORCID:0000000294892681↗
DOE OSTI2024

D2NO: Efficient handling of heterogeneous input function spaces with distributed deep neural operators

Neural operators have been applied in various scientific fields, such as solving parametric partial differential equations, dynamical systems with control, and inverse problems. However, challenges arise when dealing with input functions that exhibit heterogeneous properties, requiring multiple sensors to handle functions with minimal regularity. To address this issue, discretization-invariant neural operators have been used, allowing the sampling of diverse input functions with different sensor locations. However, existing frameworks still require an equal number of sensors for all functions. We propose a novel distributed approach to further relax the discretization requirements and solve the heterogeneous dataset challenges. Our method involves partitioning the input function space and processing individual input functions using independent and separate neural networks. A centralized neural network is used to handle shared information across all output functions. This distributed methodology reduces the number of gradient descent back-propagation steps, improving efficiency while maintaining accuracy. Here, we demonstrate that the corresponding neural network is a universal approximator of continuous nonlinear operators and present three numerical examples to validate its performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Portable Software Environment for Ultrahigh-Resolution ELM Development on GPUs

This paper presents our endeavors in developing the large-scale, ultra-high-resolution E3SM Land Model (uELM), specifically designed for exascale computers furnished with accelerators such as Nvidia GPUs. The uELM is a sophisticated code that substantially relies on High-Performance Computing (HPC) environments, necessitating particular machine and software configurations. To facilitate community-based uELM developments employing GPUs, we have created a portable, standalone software environment preconfigured with uELM input datasets, simulation cases, and source code. This environment, utilizing Docker, encompasses all essential code, libraries, and system software for uELM development on GPUs. It also features a functional unit test framework and an offline model testbed for comprehensive numerical experiments. From a technical perspective, the paper discusses GPU-ready container generations, uELM code management, and input data distribution across computational platforms. Lastly, the paper demonstrates the use of environment for functional unit testing, end-to-end simulation on CPUs and GPUs, and collaborative code development.

E3SM Land Model↗
DOE OSTI2026

iDDS: intelligent distributed dispatch and scheduling for workflow orchestration

The intelligent distributed dispatch and scheduling (iDDS) service is a versatile workflow orchestration system designed for large-scale, distributed scientific computing. iDDS extends traditional workload and data management by integrating data-aware execution, conditional logic, and programmable workflows, enabling automation of complex and dynamic processing pipelines. Originally developed for the ATLAS experiment at the large hadron collider, iDDS has evolved into an experiment-agnostic platform that supports both template-driven workflows and a Function-as-a-Task model for Python-based orchestration. This paper presents the architecture and core components of iDDS, highlighting its scalability, modular message-driven design, and integration with systems such as PanDA and Rucio. We demonstrate its versatility through real-world use cases: fine-grained tape resource optimization for ATLAS, orchestration of large Directed Acyclic Graph (DAG) workflows for the Rubin Observatory, distributed hyperparameter optimization for machine learning applications, active learning for physics analyses, and AI-assisted detector design at the electron–ion collider. By unifying workload scheduling, data movement, and adaptive decision-making, iDDS reduces operational overhead and enables reproducible, high-throughput workflows across heterogeneous infrastructures. We conclude with current challenges and future directions, including interactive, cloud-native, and serverless workflow support.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Optimizing High-Throughput Inference on Graph Neural Networks at Shared Computing Facilities with the NVIDIA Triton Inference Server

Abstract With machine learning applications now spanning a variety of computational tasks, multi-user shared computing facilities are devoting a rapidly increasing proportion of their resources to such algorithms. Graph neural networks (GNNs), for example, have provided astounding improvements in extracting complex signatures from data and are now widely used in a variety of applications, such as particle jet classification in high energy physics (HEP). However, GNNs also come with an enormous computational penalty that requires the use of GPUs to maintain reasonable throughput. At shared computing facilities, such as those used by physicists at Fermi National Accelerator Laboratory (Fermilab), methodical resource allocation and high throughput at the many-user scale are key to ensuring that resources are being used as efficiently as possible. These facilities, however, primarily provide CPU-only nodes, which proves detrimental to time-to-insight and computational throughput for workflows that include machine learning inference. In this work, we describe how a shared computing facility can use the NVIDIA Triton Inference Server to optimize its resource allocation and computing structure, recovering high throughput while scaling out to multiple users by massively parallelizing their machine learning inference. To demonstrate the effectiveness of this system in a realistic multi-user environment, we use the Fermilab Elastic Analysis Facility augmented with the Triton Inference Server to provide scalable and high-throughput access to a HEP-specific GNN and report on the outcome.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

T-FSM: A Scalable Distributed Task-Based System for Frequent Subgraph Pattern Mining from a Big Graph

Finding frequent subgraph patterns in a big graph is an important problem with many applications such as classifying chemical compounds and building indexes to speed up graph queries. Since this problem is NP-hard, some recent parallel and distributed systems have been developed to accelerate the mining. However, they often have a huge memory cost, very long running time, suboptimal load balancing, poor scale-out capability, and possibly inaccurate results. In this article, we propose an efficient system called T-FSM for parallel mining of frequent subgraph patterns in a big graph. T-FSM supports a new anti-monotonic frequentness measure called Fraction-Score, which is more accurate than the widely used MNI measure. The execution engine of T-FSM supports both intra-machine parallelism and inter-machine parallelism. For intra-machine parallelism, T-FSM adopts a novel task-based execution model to ensure high multithreading concurrency, bounded memory consumption, and effective load balancing. For inter-machine parallelism, T-FSM ensures good scale-out performance with a lightweight pattern rebalancing approach that reduces workload skewness of pattern evaluations among machines. To avoid recomputing the contexts for migrated patterns, we design a novel context cache table to support concurrent and asynchronous requesting and caching of remote context data, which can timely evict and garbage collect used pattern contexts that are no longer needed to keep memory consumption bounded. Extensive experiments show that T-FSM is orders of magnitude faster than existing state-of-the-art parallel systems (more than 10×, 51×, 131×, 55× speedup over ScaleMine, DistGraph, Pangolin and Peregrine, respectively) and distributed systems (more than 42× and 88× over ScaleMine and DistGraph, respectively) for frequent subgraph pattern mining, and it scales out satisfactorily to 512 CPU cores on the Polaris supercomputer at Argonne National Laboratory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Clustering at Massive Scale

ClaMS provides hierarchical clustering technology for use on massive, high-dimensional datasets that require distributed memory for processing. The algorithm employed is inspired by the popular HDBSCAN algorithm but makes use of computational kernels better suited for distributed computing. ClaMS is built on scalable nearest neighbor graph construction, metric forest completion, and approximate minimum spanning tree techniques.

Stanley, ThomasA [Lawrence Livermore National Labo↗
DOE OSTI2025

Toucan: A performance portable, scalable implementation of the DECA algorithm

In the field of additive manufacturing (AM), cellular automata (CA) is extensively used to simulate microstructural evolution during solidification. However, while traditional CA approaches are relatively fast, they still require a substantial number of time steps, are limited to moderate volumes, and are relatively difficult to improve through parallelism due to the highly localized nature of the solidification front. Here, to address these issues of time to solution and load balancing, we introduce Toucan, a parallel, performance-portable, and scalable code written in C++ with the Kokkos library that leverages the discrete event inspired cellular automata (DECA) algorithm to perform parallel-in-time (PinT) grain growth simulations. Toucan effectively mitigates load balancing issues by distributing the computational workload more evenly across processors, enhancing scalability and efficiency. We conduct both strong and weak scaling studies on up to 64 GPUs on the Frontier supercomputer, demonstrating that Toucan significantly outperforms the current state-of-the-art, time-stepped CA code, ExaCA, on both single and multi-GPU simulations. Even in AM-specific weak scaling scenarios, Toucan maintains near-ideal scaling, in contrast to the linear increase observed with ExaCA due to the moving laser raster pattern. This study highlights Toucan’s potential to transform microstructural simulations in AM by radically improving both efficiency and scalability over existing methods.

36 MATERIALS SCIENCE↗
DOE OSTI2025

An implementation of a high-order generalized finite difference method for solving the time-harmonic cold plasma wave equation in toroidal geometry

A high-order physics-informed meshless finite difference numerical technique is introduced for solving the time-harmonic cold plasma wave equation in toroidal geometries, presenting a novel application of the generalized finite difference (GFD) method to plasma wave simulations. The algorithm employs an irregular distribution of computational points, with local point density informed by the shortest wavelength derived from the cold plasma dispersion relation. Numerical stability and robustness are addressed using regularization techniques. The algorithm, implemented for two spatial dimensions, solves for the wave electric field and is demonstrated to achieve convergence rates of $\mathcal{O}$($\mathcal{h}$ $\mathcal{P}$ )⁠. Verification tests reproduce plane wave solutions, and example simulations of ion cyclotron resonance heating and electron cyclotron resonance heating demonstrate its capability, approaching realistic tokamak plasma scenarios. This work contributes to laying a foundation for the GFD method to be used in more sophisticated, optimized, and physically realistic full-wave simulations in time-harmonic plasma wave research.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Excitation of whistler and slow-X waves by runaway electrons in a collisional plasma

Runaway electrons are known to provide robust ideal or collisionless kinetic drive for plasma wave instabilities in both the whistler and slow-X branches, via the anomalous Doppler-shifted cyclotron resonances. In a cold and dense post-thermal-quench plasma, collisional damping of the plasma waves can compete with the collisionless drive. Previous studies have found that, due to their higher wavelength and frequency, slow-X waves suffer stronger collisional damping than the whistlers, while the ideal growth rate of slow-X modes is higher. Here, we study runaway avalanche distributions that maintain the same eigen distribution and increase only in magnitude over time. The distributions are computed from the relativistic Fokker–Planck–Boltzmann solver, upon which a linear dispersion analysis is performed to search for the most unstable or least damped slow-X and whistler modes. Taking into account the effect of plasma density, plasma temperature, and effective charge number, we find that the slow-X modes tend to be excited before the whistlers in a runaway current ramp-up. Furthermore, even when the runaway current density is sufficiently high that both branches are excited, the most unstable slow-X mode has a much higher growth rate than the most unstable whistler mode. The qualitative and quantitative trends uncovered in the current study indicate that even though past experiments and modeling efforts have concentrated on whistler modes, there is a compelling case that slow-X modes should also be a key area of focus in the runaway self-mediation through wave instabilities.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Zero-added-loss entanglement multiplexing using time-bin spectral shearing

High-quality quantum communications that enable important capabilities, such as distributed quantum computing and sensing, will require quantum repeaters for providing high-quality entanglement. To realize high-rate heralded entanglement for quantum repeaters, Chen et al . [Phys. Rev. Appl. 19 , 054029 (2023)] proposed a scheme for heralded-multiplexed generation of quasideterministic entangled photon pairs, called zero-added-loss multiplexing (ZALM). Here, we propose a design of the ZALM source using time-bin entanglement and spectral shearing. Additionally, we provide an analysis of experimentally relevant spectral-shearing parameters to optimize the spectral multiplexing. Moreover, we experimentally verify the compatibility of time-bin pulses and spectral shearing, as supported by observation of no appreciable phase shift when the same shearing is applied to both time bins. These results expand the benefits of applying a ZALM source to time-bin entanglement use cases. Moreover, more fully demonstrating time-bin and spectral shearing compatibility clears a path toward a broader use of spectral shearing that provides a deterministic frequency shift of high utility.

entanglement production↗
DOE OSTI2025

Mitigation of birefringence in cavity-based quantum networks using frequency-encoded photons

Atom-cavity systems offer unique advantages for building large-scale distributed quantum computers by providing strong atom-photon coupling while allowing for high-fidelity local operations of atomic qubits. However, in prevalent schemes where the photonic state is encoded in polarization, cavity birefringence introduces an energy splitting of the cavity eigenmodes and alters the polarization states, thus limiting the fidelity of remote entanglement generation. To address this challenge, we propose a scheme that encodes the photonic qubit in the frequency degree-of-freedom. The scheme relies on resonant coupling of multiple transverse cavity modes to different atomic transitions that are well-separated in frequency. We numerically investigate the temporal properties of the photonic wavepacket, two-photon interference visibility, and atom-atom entanglement fidelity under various cavity polarization-mode splittings and find that our scheme is less affected by cavity birefringence. Finally, we propose practical implementations in two trapped ion systems, using the fine structure splitting in the metastable D state of 40 Ca + , and the hyperfine splitting in the ground state of 225 Ra + . Furthermore, our study presents an alternative approach for cavity-based quantum networks that is less sensitive to birefringent effects, and is applicable to a variety of atomic and solid-state emitter-cavity interfaces.

Cavity quantum electrodynamics↗
DOE OSTI2025

Systematic study of the validity of the eikonal model including uncertainties

Nuclear reactions at intermediate beam energies are often interpreted using the eikonal model. In the analysis of complex reaction probes, where few-body reaction methods are needed, the eikonal method may be used as an efficient way for describing the fragment-target reaction process. In this work, we perform a systematic study to test the validity of the eikonal approximation for nucleon-nucleus reactions. We also quantify uncertainties due to the nucleon optical potential on reaction observables. We inspect the validity of the eikonal model and its semiclassical correction by comparing it to exact solutions (obtained from solving the optical-model equation with a finite-differences method) for a wide range of reactions. We also study the effect of relativistic corrections, both kinematic and dynamic, by effectively incorporating the relativistic effects at intermediate energies. The uncertainties from a Bayesian global optical potential (KDUQ) are propagated to the observables of interest. Our study includes neutron and proton reactions on 27 Al , 40 Ca , 90 Zr , and 208 Pb , for a wide range of energies 𝐸 lab = 0–400 MeV. We calculate neutron-total cross sections (elastic and reactions) as well as proton-absorption cross sections as a function of beam energy, using the eikonal model, the eikonal model with a semiclassical correction, and the exact solution. Here, we also compute angular distributions for the methods above. Our results show that for the proton-absorption cross section, the eikonal model can be used down to around 60 MeV and the semiclassical correction extends its use to 30 MeV. However, the validity of the eikonal model for the neutron-total cross section only goes down to ≈120 MeV, a range extended to ≈ 50 MeV when using the semiclassical correction. We find the semiclassical correction to the eikonal model to be less effective in describing the angular distributions. The 1⁢𝜎 uncertainty intervals on the observables we studied is less than 5% for most of the energies considered, but increases rapidly for higher energies, namely energies outside the range of KDUQ (𝐸 lab > 200MeV).

Cluster models↗