Search NASA⌕ Search

SEARCH · Search NASA

Results for “large scale network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

An automated workflow that generates atom mappings for large‐scale metabolic models and its application to Arabidopsis thaliana

SUMMARY Quantification of reaction fluxes of metabolic networks can help us understand how the integration of different metabolic pathways determines cellular functions. Yet, intracellular fluxes cannot be measured directly but are estimated with metabolic flux analysis (MFA), which relies on the patterns of isotope labeling of metabolites in the network. The application of MFA also requires a stoichiometric model with atom mappings that are currently not available for the majority of large‐scale metabolic network models, particularly of plants. While automated approaches such as the Reaction Decoder Toolkit (RDT) can produce atom mappings for individual reactions, tracing the flow of individual atoms of the entire reactions across a metabolic model remains challenging. Here we establish an automated workflow to obtain reliable atom mappings for large‐scale metabolic models by refining the outcome of RDT, and apply the workflow to metabolic models of Arabidopsis thaliana . We demonstrate the accuracy of RDT through a comparative analysis with atom mappings from a large database of biochemical reactions, MetaCyc. We further show the utility of our automated workflow by simulating 15 N isotope enrichment and identifying nitrogen (N)‐containing metabolites which show enrichment patterns that are informative for flux estimation in future 15 N‐MFA studies of A. thaliana . The automated workflow established in this study can be readily expanded to other species for which metabolic models have been established and the resulting atom mappings will facilitate MFA and graph‐theoretic structural analyses with large‐scale metabolic networks.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Make the Fastest Faster: Importance Mask Synthesis for Interactive Volume Visualization using Reconstruction Neural Networks

Visualizing a large-scale volumetric dataset with high resolution is challenging due to the substantial computational time and space complexity. Recent deep learning-based image inpainting methods significantly improve rendering latency by reconstructing a high-resolution image for visualization in constant time on GPU from a partially rendered image where only a portion of pixels go through the expensive rendering pipeline. However, existing solutions need to render every pixel of either a predefined regular sampling pattern or an irregular sample pattern predicted from a low-resolution image rendering. Both methods require a significant amount of expensive pixel-level rendering. In this work, we provide Importance Mask Learning (IML) and Synthesis (IMS) networks, which are the first attempts to directly synthesize important regions of the regular sampling pattern from the user’s view parameters, to further minimize the number of pixels to render by jointly considering the dataset, user behavior, and the downstream reconstruction neural network. Our solution is a unified framework to handle various types of inpainting methods through the proposed differentiable compaction/decompaction layers. Experiments show our method can further improve the overall rendering latency of state-of-the-art volume visualization methods using reconstruction neural network for free when rendering scientific volumetric datasets. Our method can also directly optimize the off-the-shelf pre-trained reconstruction neural networks without elongated retraining.

Large-scale data↗
DOE OSTI2023

Scalable Graph Analytics and HPC Operational Enhancement: Parallel Computing and ML/DL Innovations

Parallel computing plays a pivotal role in the efficient processing of large-scale graphs. Complex network analysis stands as a capti- vating research frontier, holding promise across diverse scientific domains such as sociology, biology, online media, and recommenda- tion systems. In this era, Machine Learning (ML) and Deep Learning (DL) have emerged as indispensable tools, underpinning remarkable technological achievements. Within this dynamic landscape, my research revolves around advancing parallel algorithms tailored for large-scale graph operations. To achieve this, I harness the power of cutting-edge technologies including OpenMP, MPI, HIP, and CUDA, on the High-Performance Computing (HPC) platforms to unlock optimal performance. I also apply ML/DL techniques to HPC operational data, to streamline the monitoring and maintenance of supercomputers, alleviating the complexities associated with their upkeep and enhancing user support. My research echoes the syn- ergy between parallel computing, large-scale graph analysis, and ML/DL, improving computational efficiency and user experience.

Sattar, Naw Safrin↗
DOE OSTI2024

DyG-DPCD: A Distributed Parallel Community Detection Algorithm for Large-Scale Dynamic Graphs

Dynamic (Temporal) graphs capture the valuable evolution of real-world systems, from the continuously evolving patterns of social interactions and genetic pathways to the dynamic fluctuations of economic forces. Detecting communities for such evolving networks poses unique challenges. Detecting and analyzing the evolution of communities within dynamic graphs unlocks valuable insights into the underlying structural and temporal patterns of real-world systems. However, the sheer volume of modern graph data and the inherent complexity of the temporal dimension pose significant challenges to scalable community detection algorithms. Addressing this gap, our work explores the limited landscape of scalable distributed-memory parallel methods specifically designed for dynamic network community detection. We propose a novel parallel algorithm, DyG-DPCD (Dynamic Graph Distributed Parallel Community Detection), to detect communities in dynamic networks using the Message Passing Interface (MPI) framework. We present a vertex-centric approach, allowing us to detect communities through local optimization. Furthermore, we enhance our baseline algorithm by incorporating three heuristics, which improve the algorithm’s performance significantly while maintaining the quality of the solutions. We demonstrate the efficiency of our algorithm by experimenting on several real-world large-scale networks with hundreds of millions of edges spanning diverse domains. Notably, DyG-DPCD achieves speedups between 25× and 30× for large networks that we experimented on using NERSC compute nodes. In conclusion, our algorithm outperforms the STINGER parallel re-agglomeration algorithm by 30×.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Quantum teleportation coexisting with classical communications in optical fiber

The ability for quantum and conventional networks to operate in the same optical fibers would aid the deployment of quantum network technology on a large scale. Quantum teleportation is a fundamental operation in quantum networking, but has yet to be demonstrated in fibers populated with high-power conventional optical signals. Here we report, to the best of our knowledge, the first demonstration of quantum teleportation over fibers carrying conventional telecommunications traffic. Quantum state transfer is achieved over a 30.2-km fiber carrying 400-Gbps C-band classical traffic with a Bell state measurement performed at the fiber’s midpoint. To protect quantum fidelity from spontaneous Raman scattering noise, we use optimal O-band quantum channels, narrow spectro-temporal filtering, and multi-photon coincidence detection. Fidelity is shown to be well maintained with an elevated C-band launch power of 18.7 dBm for the single-channel 400-Gbps signal, which we project could support multiple classical channels totaling many terabits/s aggregate data rates. These results show the feasibility of advanced quantum and classical network applications operating within a unified fiber infrastructure.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Building and experimenting with an agent-based model to study the population-level impact of CommunityRx, a clinic-based community resource referral intervention

CommunityRx (CRx), an information technology intervention, provides patients with a personalized list of healthful community resources (HealtheRx). In repeated clinical studies, nearly half of those who received clinical “doses” of the HealtheRx shared their information with others (“social doses”). Clinical trial design cannot fully capture the impact of information diffusion, which can act as a force multiplier for the intervention. Furthermore, experimentation is needed to understand how intervention delivery can optimize social spread under varying circumstances. To study information diffusion from CRx under varying conditions, we built an agent-based model (ABM). This study describes the model building process and illustrates how an ABM provides insight about information diffusion through in silico experimentation. To build the ABM, we constructed a synthetic population (“agents”) using publicly-available data sources. Using clinical trial data, we developed empirically-informed processes simulating agent activities, resource knowledge evolution and information sharing. Using RepastHPC and chiSIM software, we replicated the intervention in silico, simulated information diffusion processes, and generated emergent information diffusion networks. The CRx ABM was calibrated using empirical data to replicate the CRx intervention in silico. We used the ABM to quantify information spread via social versus clinical dosing then conducted information diffusion experiments, comparing the social dosing effect of the intervention when delivered by physicians, nurses or clinical clerks. The synthetic population (N = 802,191) exhibited diverse behavioral characteristics, including activity and knowledge evolution patterns. In silico delivery of the intervention was replicated with high fidelity. Large-scale information diffusion networks emerged among agents exchanging resource information. Varying the propensity for information exchange resulted in networks with different topological characteristics. Community resource information spread via social dosing was nearly 4 fold that from clinical dosing alone and did not vary by delivery mode. This study, using CRx as an example, demonstrates the process of building and experimenting with an ABM to study information diffusion from, and the population-level impact of, a clinical information-based intervention. While the focus of the CRx ABM is to recreate the CRx intervention in silico, the general process of model building, and computational experimentation presented is generalizable to other large-scale ABMs of information diffusion.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Large-scale Gravitational Lens Modeling with Bayesian Neural Networks for Accurate and Precise Inference of the Hubble Constant

We investigate the use of approximate Bayesian neural networks (BNNs) in modeling hundreds of time delay gravitational lenses for Hubble constant (H 0 ) determination. Here, our BNN was trained on synthetic Hubble Space Telescope quality images of strongly lensed active galactic nuclei with lens galaxy light included. The BNN can accurately characterize the posterior probability density functions (PDFs) of model parameters governing the elliptical power-law mass profile in an external shear field. We then propagate the BNN-inferred posterior PDFs into an ensemble H 0 inference, using simulated time delay measurements from a plausible dedicated monitoring campaign. Assuming well-measured time delays and a reasonable set of priors on the environment of the lens, we achieve a median precision of 9.3% per lens in the inferred H 0 . A simple combination of a set of 200 test lenses results in a precision of 0.5 km s –1 Mpc –1 (0.7%), with no detectable bias in this H 0 recovery test. The computation time for the entire pipeline—including the generation of the training set, BNN training and H0 inference—translates to 9 minutes per lens on average for 200 lenses and converges to 6 minutes per lens as the sample size is increased. Being fully automated and efficient, our pipeline is a promising tool for exploring ensemble-level systematics in lens modeling for H 0 inference.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Systematic comparison of local approaches for isotopically nonstationary metabolic flux analysis

Quantification of reaction fluxes of metabolic networks can help us understand how the integration of different metabolic pathways determine cellular functions. Yet, intracellular fluxes cannot be measured directly but are estimated with metabolic flux analysis (MFA) that relies on the patterns of isotope labeling of metabolites in the network. For metabolic systems, typical for plants, where all potentially labeled atoms effectively have only one source atom pool, only isotopically nonstationary MFA can provide information about intracellular fluxes. There are several global approaches that implement MFA for an entire metabolic network and estimate, at once, a steady-state flux distribution for all reactions with identifiable fluxes in the network. In contrast, local approaches deal with estimation of fluxes for a subset of reactions, with smaller data demand for flux estimation. Here we present a systematic comparative review and benchmarking of the existing local approaches for isotopically nonstationary MFA. The comparison is conducted with respect to the required data and underlying computational problems solved on a synthetic network example. Furthermore, we benchmark the performance of these approaches in estimating fluxes for a subset of reactions using data obtained from the simulation of nitrogen fluxes in the Arabidopsis thaliana core metabolism. The findings pinpoint practical aspects that need to be considered when applying local approaches for flux estimation in large-scale plant metabolic networks.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Iterative self-organizing SCEne-LEvel sampling (ISOSCELES) for large-scale building extraction

Convolutional neural networks (CNN) provide state-of-the-art performance in many computer vision tasks, including those related to remote-sensing image analysis. Successfully training a CNN to generalize well to unseen data, however, requires training on samples that represent the full distribution of variation of both the target classes and their surrounding contexts. With remote sensing data, acquiring a sufficiently representative training set is a challenge due to both the inherent multi-modal variability of satellite or aerial imagery and the general high cost of labeling data. To address this challenge, we have developed ISOSCELES, an Iterative Self-Organizing SCEne LEvel Sampling method for hierarchical sampling of large image sets. Using affinity propagation, ISOSCELES automates the selection of highly representative training images. Compared to random sampling or using available reference data, the distribution of the training is principally data driven, reducing the chance of oversampling uninformative areas or undersampling informative ones. In comparison to manual sample selection by an analyst, ISOSCELES exploits descriptive features, spectral and/or textural, and eliminates human bias in sample selection. Using a hierarchical sampling approach, ISOSCELES can obtain a training set that reflects both between-scene variability, such as in viewing angle and time of day, and within-scene variability at the level of individual training samples. We verify the method by demonstrating its superiority to stratified random sampling in the challenging task of adapting a pre-trained model to a new image and spatial domain for country-scale building extraction. Using a pair of hand-labeled training sets comprising 1,987 sample image chips, a total of 496,000,000 individually labeled pixels, we show, across three distinct model architectures, an increase in accuracy, as measured by F1-score, of 2.2–4.2%.

42 ENGINEERING↗
DOE OSTI2026

Extreme-scale EV charging infrastructure planning for last-mile delivery using high-performance parallel computing

Here, this paper addresses stochastic charger location and allocation problems under queue congestion for last-mile delivery using electric vehicles (EVs). The objective is to decide where to open charging stations and how many chargers of each type to install, subject to budgetary and waiting-time constraints. We formulate the problem as a mixed-integer non-linear program, where each station-charger pair is modeled as a multiserver queue with stochastic arrivals and service times to capture the notion of waiting in fleet operations. The model is extremely large, with billions of variables and constraints for a typical metropolitan area; even loading the model in solver memory is difficult, let alone solving it. To address this challenge, we develop a Lagrangian-based dual decomposition framework that decomposes the problem by station and leverages parallelization on high-performance computing systems, where the subproblems are solved by using a cutting plane method and their solutions are collected at the master level. We also develop a three-step rounding heuristic to transform the fractional subproblem solutions into feasible integral solutions. Computational experiments on data from the Chicago metropolitan area with hundreds of thousands of households and thousands of candidate stations show that our approach produces high-quality solutions in cases where existing exact methods cannot even load the model in memory. We also analyze various policy scenarios, demonstrating that combining existing depots with newly built stations under multiagency collaboration substantially reduces costs and congestion. These findings offer a scalable and efficient framework for developing sustainable large-scale EV charging networks.

Capacity allocation↗
DOE OSTI2025

Graph-based Simulation Framework for Power Resilience Estimation and Enhancement

The increasing frequency of extreme weather events poses significant risks to power distribution systems, leading to widespread outages and severe economic and social consequences. This paper presents a novel simulation framework for assessing and enhancing the resilience of power distribution networks under such conditions. Resilience is estimated through Monte Carlo simulations, which simulate extreme weather scenarios and evaluate the impact on infrastructure fragility. Due to the proprietary nature of power network topology, a distribution network is synthesized using publicly available data. To generate the weather scenarios, an extreme weather generation method is developed. To enhance resilience, renewable resources such as solar panels and energy storage systems (batteries in this study) are incorporated. A customized Genetic Algorithm is proposed to determine the optimal locations and capacities for solar panels and battery installations, maximizing resilience while balancing cost constraints. Experiment results demonstrate that on a large-scale synthetic distribution network with more than 300,000 nodes and 300,000 edges, the proposed framework can efficiently evaluate the resilience, and enhance the resilience through the installations of distributed energy resources (DERs), providing utilities with valuable insights for community-level power system resilience estimation and enhancement.

Wang, Xuesong [Wayne State Univ., Detroit, MI (Uni↗
DOE OSTI2024

Neural network emulation of spontaneous fission

Large-scale computations of fission properties are an important ingredient for nuclear reaction network calculations simulating rapid neutron-capture process (the 𝑟 process) nucleosynthesis. Due to the large number of fissioning nuclei potentially contributing to the 𝑟 process, a microscopic description of fission based on nuclear density functional theory (DFT) is computationally challenging. Here, we explore the use of neural networks (NNs) to construct DFT emulators capable of predicting potential energy surfaces and collective inertia tensors across the whole nuclear chart, starting from a minimal set of DFT calculations. We use constrained Hartree-Fock-Bogoliubov (HFB) calculations to predict the potential energy and collective inertia tensor in the axial quadrupole and octupole collective coordinates, for a set of nuclei in the 𝑟-process region. We then employ NNs to emulate the HFB energy and collective inertia tensor across the considered region of the nuclear chart. Least-action pathways characterizing spontaneous fission half-lives and fragment yields are then obtained by means of the nudged elastic band method. The potential energy predicted by NNs agrees with the DFT value to within a root-mean-square error of 500 keV, and the collective inertia components agree to within an order of magnitude. These results are largely independent of the NN architecture. The exit points on the outer turning line are found to be well emulated. For the spontaneous fission half-lives the NN emulation provides values that are found to agree with the DFT predictions within a factor of 10 3 across more than 70 orders of magnitude. Neural networks are able to emulate the potential energy and collective inertia well enough to reasonably predict physical observables. Future directions of study, such as the inclusion of additional collective degrees of freedom and active learning, will improve the predictive power of microscopic theory and further enable large-scale fission studies.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelism

Here, we present scalable hybrid-parallel algorithms for training large-scale 3D convolutional neural networks. Deep learning-based emerging scientific workflows often require model training with large, high-dimensional samples, which can make training much more costly and even infeasible due to excessive memory usage. We solve these challenges by extensively applying hybrid parallelism throughout the end-to-end training pipeline, including both computations and I/O. Our hybrid-parallel algorithm extends the standard data parallelism with spatial parallelism, which partitions a single sample in the spatial domain, realizing strong scaling beyond the mini-batch dimension with a larger aggregated memory capacity. We evaluate our proposed training algorithms with two challenging 3D CNNs, CosmoFlow and 3D U-Net. Our comprehensive performance studies show that good weak and strong scaling can be achieved for both networks using up to 2K GPUs. More importantly, we enable training of CosmoFlow with much larger samples than previously possible, realizing an order-of-magnitude improvement in prediction accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A Julia Framework for Graph-Structured Nonlinear Optimization

Graph theory provides a convenient framework for modeling and solving structured optimization problems. Under this framework, the modeler can arrange/assemble the components of an optimization model (variables, constraints, objective functions, and data) within nodes and edges of a graph, and this representation can be used to visualize, manipulate, and solve the problem. In this work, we present a Julia framework for modeling and solving graph-structured nonlinear optimization problems. Our framework integrates the modeling package Plasmo.jl (which facilitates the construction and manipulation of graph models) and the nonlinear optimization solver MadNLP.jl (which provides capabilities for exploiting graph structures to accelerate solution). We illustrate with a simple example how model construction and manipulation can be performed in an intuitive manner using Plasmo.jl and how the model structure can be exploited by MadNLP.jl. We also demonstrate the scalability of the framework by targeting a large-scale, stochastic gas network problem that contains over 1.7 million variables.

Cole, David↗
DOE OSTI2022

Design of functional materials based on new principles of disorder. Final Report

Disorder is present to some extent in every material. The disorder can be sparse and localized – individual defects – or globally such as in a glass. Normally disorder is considered to be detrimental to function. However, this is not necessarily always the case. This project addressed the question: How does disorder provide new potentialities for the behavior of a material that would be absent in a crystalline material of the same composition? We have discovered a new principle for disordered matter: Independence of bond-level response. This work has introduced the possibility of creating new classes of mechanical meta-materials with applications everywhere from microscopic biological molecules (allostery) to large scale architectural disordered networks. During the course of this grant, we have subsequently elaborated on these novel ideas about bond-level independence of properties in disordered materials. In particular, this study has required that we understand the limits of disorder. That is, if crystals are perfect order, is there another diametrically opposed pole which is the most disordered state? We have shown that, in a well-defined way, jamming is one extreme disordered pole of rigid matter. Jamming is a far-from-equilibrium way of creating a rigid solid by rapidly quenching a random distribution of particles to a mechanically stable system at zero temperature. To understand the jammed state of matter, we have elucidated the low-frequency excitation of these systems. This has allowed a new way of understanding materials that cannot be easily classified as crystals or glasses. These ideas can be extended to address the mechanical properties of polycrystalline matter. Memory formation in matter can take inspiration from memory that occurs in a biological context. The issue of memory formation raises another set of questions at the heart of how the energy landscape is organized and how it can be manipulated to store information in a solid. This project has explored how to understand the degree to which multiple memories can be stored in a single system and has created a holistic understanding of how inputs can be used to create a desired response.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Efficient Scalable Contact Network Generation from Population Data

Modeling the contacts among a population is critical to understanding the dynamics of a disease outbreak. Contact networks, where nodes are individuals and edges are contacts among them, are used to represent these complex individual-level interactions. In this work, we are given the daily activity schedules of an urban population that represent the activity location and time of individuals in a population during a single twenty four hour period over multiple days. Using collocation to determine contact between individuals, our goal is to extract hourly contact networks from large-scale activity data. We improve upon the existing adjacency matrix-based method by implementing our custom sparse matrix multiplication algorithm. Starting with a Python implementation, we achieve a 1600x speed up in the computation with a fast custom designed sparse matrix multiplier algorithm implemented in the C++ language. This work is central to future parallel designs of the problem.

97 MATHEMATICS AND COMPUTING↗