Search NASA⌕ Search

SEARCH · Search NASA

Results for “Scalable Algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Position Papers for the ASCR Workshop on Cybersecurity and Privacy for Scientific Computing Ecosystems

At the request of the Department of Energy's (DOE) Office of Advanced Scientific Computing Research (ASCR), this program committee has been tasked with organizing a workshop to identify basic research needs in cybersecurity and privacy to better support DOE's science and energy mission. As part of the process, the program committee is soliciting community input in the form of position papers to help identify significant use cases, facility issues, and other barriers to enabling verifiably trustworthy computational science while preserving data confidentiality as appropriate for scientific workflows of interest to DOE. The program committee will review these position papers and based on the fit of their area of expertise and interest, selected contributors will have the opportunity to participate in the workshop currently planned as a virtual event November 3-5th, 2021. The thrust areas that will be explored by this workshop are the following: (1) Algorithms for secure, scalable, privacy-enhancing technologies and frameworks, including: Federated AI/ML, Differential privacy, Randomized algorithms, Adversarial modeling & simulation, Graph algorithms, and Formal methods; (2) Platforms to support the entire scientific-computing ecosystem, including edge computing for large-scale experiments, focusing on heterogeneous systems and distributed systems, including: Heterogeneous computing systems, Distributed computing systems, and Secure data architectures; and (3) Data workflows to allow agile use of data while preserving integrity and privacy, making the important properties verifiable either at runtime or post-computation, including: Integrity and provenance and Data management infrastructure. Topics that are out-of-scope for the workshop include discussing specific proposed solutions or areas that are clearly out of DOE's fundamental and applied-sciences mission scope, e.g., cryptography, enterprise security, and general-operations technology.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Solving the Bernstein-Vazirani problem using Majorana-based topological quantum algorithms

Executing quantum algorithms using Majorana zero modes—a major milestone for the field of topological quantum computing—requires a platform that can be scaled to large quantum registers, can be controlled in real time and space, and a braiding protocol that uses the unique properties of these exotic particles. Here, we demonstrate the first successful simulation of a Majorana-based, fault-tolerant quantum algorithm to solve the Bernstein-Vazirani problem in two-dimensional magnet-superconductor hybrid structures from initialization to read-out of the final many-body state. Utilizing the Majorana zero modes’ topological properties, we introduce an optimized braiding protocol for the algorithm and a scalable architecture for its implementation with an arbitrary number of qubits. We visualize the algorithm protocol in real time and space by computing the non-equilibrium density of states, which is proportional to the time-dependent differential conductance, and the non-equilibrium charge density, which assigns a unique signature to each final state of the algorithm.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Asynchronous and Load-Balanced Union-Find for Distributed and Parallel Scientific Data Visualization and Analysis

We present a novel distributed union-find algorithm that features asynchronous parallelism and k-d tree based load balancing for scalable visualization and analysis of scientific data. Applications of union-find include level set extraction and critical point tracking, but distributed union-find can suffer from high synchronization costs and imbalanced workloads across parallel processes. In this study, we prove that global synchronizations in existing distributed union-find can be eliminated without changing final results, allowing overlapped communications and computations for scalable processing. We also use a k-d tree decomposition to redistribute inputs, in order to improve workload balancing. We benchmark the scalability of our algorithm with up to 1,024 processes using both synthetic and application data. Here, we demonstrate the use of our algorithm in critical point tracking and super-level set extraction with high-speed imaging experiments and fusion plasma simulations, respectively.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A Stochastic Multi-Criteria Decision-Making Algorithm for Dynamic Load Prioritization in Grid-Interactive Efficient Buildings

Increasing deployment of advanced sensing, controls, and communication infrastructure enables buildings to provide services to the power grid, leading to the concept of grid-interactive efficient buildings. Since occupant activities and preferences primarily drive the availability and operational flexibility of building devices, there is a critical need to develop occupant-centric approaches that prioritize devices for providing grid services, while maintaining the desired end-use quality of service. In this paper, we present a decision-making framework that facilitates a building owner/operator to effectively prioritize loads for curtailment service under uncertainties, while minimizing any adverse impact on the occupants. The proposed framework uses a stochastic (Markov) model to represent the probabilistic behavior of device usage from power consumption data, and a load prioritization algorithm that dynamically ranks building loads using a stochastic multi-criteria decision-making algorithm. The proposed load prioritization framework is illustrated via numerical simulations in a residential building use-case, including plug-loads, air-conditioners, and plug-in electric vehicle chargers, in the context of load curtailment as a grid service. Suitable metrics are proposed to evaluate the closed-loop performance of the proposed prioritization algorithm under various scenarios and design choices. Scalability of the proposed algorithm is established via computational analysis, while time-series plots are used for intuitive explanation of the ranking choices.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Distributed-Memory Parallel JointNMF

Joint Nonnegative Matrix Factorization (JointNMF) is a hybrid method for mining information from datasets that contain both feature and connection information. We propose distributed-memory parallelizations of three algorithms for solving the JointNMF problem based on Alternating Nonnegative Least Squares, Projected Gradient Descent, and Projected Gauss-Newton. We extend well-known communication-avoiding algorithms using a single processor grid case to our coupled case on two processor grids. We demonstrate the scalability of the algorithms on up to 960 cores (40 nodes) with 60% parallel efficiency. The more sophisticated Alternating Nonnegative Least Squares (ANLS) and Gauss-Newton variants outperform the first-order gradient descent method in reducing the objective on large-scale problems. We perform a topic modelling task on a large corpus of academic papers that consists of over 37 million paper abstracts and nearly a billion citation relationships, demonstrating the utility and scalability of the methods.

Eswar, Srinivas↗
DOE OSTI2024

DyG-DPCD: A Distributed Parallel Community Detection Algorithm for Large-Scale Dynamic Graphs

Dynamic (Temporal) graphs capture the valuable evolution of real-world systems, from the continuously evolving patterns of social interactions and genetic pathways to the dynamic fluctuations of economic forces. Detecting communities for such evolving networks poses unique challenges. Detecting and analyzing the evolution of communities within dynamic graphs unlocks valuable insights into the underlying structural and temporal patterns of real-world systems. However, the sheer volume of modern graph data and the inherent complexity of the temporal dimension pose significant challenges to scalable community detection algorithms. Addressing this gap, our work explores the limited landscape of scalable distributed-memory parallel methods specifically designed for dynamic network community detection. We propose a novel parallel algorithm, DyG-DPCD (Dynamic Graph Distributed Parallel Community Detection), to detect communities in dynamic networks using the Message Passing Interface (MPI) framework. We present a vertex-centric approach, allowing us to detect communities through local optimization. Furthermore, we enhance our baseline algorithm by incorporating three heuristics, which improve the algorithm’s performance significantly while maintaining the quality of the solutions. We demonstrate the efficiency of our algorithm by experimenting on several real-world large-scale networks with hundreds of millions of edges spanning diverse domains. Notably, DyG-DPCD achieves speedups between 25× and 30× for large networks that we experimented on using NERSC compute nodes. In conclusion, our algorithm outperforms the STINGER parallel re-agglomeration algorithm by 30×.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Smart Monitoring and Diagnostic System (SMDS) for Packaged Air Conditioners and Heat Pumps for Small/Medium Commercial Buildings: Preparation for Commercialization: CRADA 478 [Abstract only]

This project will enable Pacific Northwest National Laboratory (PNNL) and industry partner, mCloud Technologies, to work collaboratively to ready the Smart Monitoring and Diagnostic System (SMDS) for commercial deployment and validate its performance under real-world conditions at field sites. This project will specifically focus on 1) implementing the SMDS algorithms in a scalable cloud-based software architecture, 2) designing new, innovative, complimentary commercial services based on the SMDS, 3) enhancing the SMDS energy and cost impact algorithms to reduce uncertainty in estimates, 4) determining the lower limits on SMDS performance degradation detection, 5) validating algorithm performance and default values of adjustable thresholds with existing data from controlled physical testing and from customer packaged air conditioners and heat pumps (commonly referred to as rooftop units or RTUs), 6) field testing to validate the system on multiple customer buildings in diverse environments, and 7) expanding field deployment to a larger set of mCloud’s customer buildings. Project results by validating, enhancing, and quantifying the performance of the SMDS will position mCloud, and potential future licensees, to implement the SMDS in commercial offerings that encourage and enable use of condition-based and predictive maintenance, leading to significant reductions in energy use and greenhouse gas emissions associated with space conditioning by RTUs. Furthermore, these enhancements will increase the value of the SMDS for users and increase the potential market for its use and impacts.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Distributed-Memory Parallel Symmetric Nonnegative Matrix Factorization

We develop the first distributed-memory parallel implementation of Symmetric Nonnegative Matrix Factorization (SymNMF), a key data analytics kernel for clustering and dimensionality reduction. Our implementation includes two different algorithms for SymNMF, which give comparable results in terms of time and accuracy. The first algorithm is a parallelization of an existing sequential approach that uses solvers for non symmetric NMF. The second algorithm is a novel approach based on the Gauss-Newton method. It exploits second-order information without incurring large computational and memory costs. We evaluate the scalability of our algorithms on the Summit system at Oak Ridge National Laboratory, scaling up to 128 nodes (4,096 cores) with 70% efficiency. Additionally, we demonstrate our software on an image segmentation task.

Eswar, Srinivas↗
DOE OSTI2021

A FETI approach to domain decomposition for meshfree discretizations of nonlocal problems

We propose a domain decomposition method for the efficient simulation of nonlocal problems. Our approach is based on a multi-domain formulation of a nonlocal diffusion problem where the subdomains share “nonlocal” interfaces of the size of the nonlocal horizon. This system of nonlocal equations is first rewritten in terms of minimization of a nonlocal energy, then discretized with a meshfree approximation and finally solved via a Lagrange multiplier approach in a way that resembles the finite element tearing and interconnect method. Specifically, we propose a distributed projected gradient algorithm for the solution of the Lagrange multiplier system, whose unknowns determine the nonlocal interface conditions between subdomains. Several two-dimensional numerical tests on problems as large as 191 million unknowns illustrate the strong and the weak scalability of our algorithm, which outperforms the standard approach to the distributed numerical solution of the problem. Finally, this work is the first rigorous numerical study in a two-dimensional multi-domain setting for nonlocal operators with finite horizon and, as such, it is a fundamental step towards increasing the use of nonlocal models in large scale simulations.

42 ENGINEERING↗
DOE OSTI2021

Network-Level Optimization for Unbalanced Power Distribution System: Approximation and Relaxation

The nonlinear programming (NLP) problem to solve distribution-level optimal power flow (D-OPF) poses convergence issues and does not scale well for unbalanced distribution systems. The existing scalable D-OPF algorithms either use approximations that are not valid for an unbalanced power distribution system, or apply relaxation techniques to the nonlinear power flow equations that do not guarantee a feasible power flow solution. In this paper, we propose scalable D-OPF algorithms that simultaneously achieve optimal and feasible solutions by solving multiple iterations of approximate, or relaxed, D-OPF subproblems of low complexity. The first algorithm is based on a successive linear approximation of the nonlinear power flow equations around the current operating point, where the D-OPF solution is obtained by solving multiple iterations of a linear programming (LP) problem. The second algorithm is based on the relaxation of the nonlinear power flow equations as conic constraints together with directional constraints, which achieves optimal and feasible solutions over multiple iterations of a second-order cone programming (SOCP) problem. Finally, it is demonstrated that the proposed algorithms are able to reach an optimal and feasible solution while significantly reducing the computation time as compared to an equivalent NLPD-OPF model for the same distribution system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Space–time reduced order model for large-scale linear dynamical systems with application to Boltzmann transport problems

A classical reduced order model for dynamical problems involves spatial reduction of the problem size. However, temporal reduction accompanied by the spatial reduction can further reduce the problem size without losing much accuracy, which results in a considerably more speed-up than the spatial reduction only. Recently, a novel space–time reduced order model for dynamical problems has been developed [17], where the space–time reduced order model shows an order of a hundred speed-up with a relative error of 10 –4 for small academic problems. However, in order for the method to be applicable to a large-scale problem, an efficient space–time reduced basis construction algorithm needs to be developed. Here we present the incremental space–time reduced basis construction algorithm. The incremental algorithm is fully parallel and scalable. Additionally, the block structure in the space–time reduced basis is exploited, which enables the avoidance of constructing the reduced space–time basis. These novel techniques are applied to a large-scale particle transport simulation with million and billion degrees of freedom. The numerical example shows that the algorithm is scalable and practical. Also, it achieves a tremendous speed-up, maintaining a good accuracy. Finally, error bounds for space-only and space–time reduced order models are derived.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

An Empirical Validation of a Constrained Bin Packing Algorithm for a Home Energy Management System

The increasing number of intelligent electrical appliances and home energy management systems provide a big opportunity for demand response services from residential and small commercial buildings to the grid. Simultaneously, direct control of individual devices by utilities can cause communication bottlenecks, as well as coordination and privacy concerns. These challenges can be addressed by combining the constituent devices into a single house battery equivalent for the purposes of demand response, using Minkowski sum and a 2d bin packing problem. However, the well-studied traditional problems have not been tested in a real house, as implementation carries significant challenges of its own. We deploy the packing problem on residential devices in a controllable house. We report the barriers we found, such as charge forecast and scalability of the algorithm, and discuss our solutions. The study serves as an intermediate step between existing theoretical research and possible future steps, such as prototype deployments of systems that provide residential demand response.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Evaluating a quantum-classical quantum Monte Carlo algorithm with Matchgate shadows

Solving the electronic structure problem of molecules and solids to high accuracy is a major challenge in quantum chemistry and condensed matter physics. The rapid emergence and development of quantum computers offer a promising route to systematically tackle this problem. Recent work by [Huggins et al ., Nature (London) 603 , 416 (2022)] proposed a hybrid quantum-classical quantum Monte Carlo (QC-QMC) algorithm using Clifford shadows to determine the ground state of a Fermionic Hamiltonian. This approach displayed inherent noise resilience and the potential for improved accuracy compared to its purely classical counterpart. Nevertheless, the use of Clifford shadows introduces an exponentially scaling postprocessing cost. In this work, we investigate an improved QC-QMC scheme utilizing the recently developed Matchgate shadows technique [Commun. Math. Phys. 404 , 629 (2023)], which removes the aforementioned exponential bottleneck. We observe from experiments on quantum hardware that the use of Matchgate shadows in QC-QMC is inherently noise robust. We show that this noise resilience has a more subtle origin than in the case of Clifford shadows. Nevertheless, we find that classical postprocessing, while asymptotically efficient, requires hours of runtime on thousands of classical CPUs for even the smallest chemical systems, presenting a major challenge to the scalability of the algorithm.

Monte Carlo methods↗
DOE OSTI2025

FIRE: A Failure-Adaptive RL Framework for Edge Computing Migrations

In edge computing, users' service profiles are migrated between edge servers due to user mobility. Reinforcement Learning (RL) frameworks have been proposed to do so, often trained on simulated data. However, existing RL frameworks overlook occasional server failures, which although rare, impact latency-sensitive applications like AR/VR and real- time obstacle detection. These rare failures, being not adequately represented in historical training data, pose a challenge for data-driven RL algorithms. We introduce FIRE, a framework that adapts to rare events by training a RL policy in an edge computing digital twin environment. We propose FIRE-ImRE, an importance sampling-based Q-learning algorithm, which samples rare events proportionally to their impact on the value function. FIRE considers delay, migration, failure, and backup placement costs across individual and shared service profiles. We prove FIRE-ImRE's boundedness and convergence to optimality. Next, we introduce novel deep Q-learning (FIRE-ImDQL) and actor critic (FIRE-ImACRE) versions of our algorithm to enhance scalability. Here, we extend our framework to accommodate users with varying risk tolerances of rare failure events. Through trace-driven experiments, we show that FIRE reduces edge computing costs compared to vanilla RL and the greedy baseline in the event of failures.

Edge computing↗
DOE OSTI2025

Efficient Neural Network Approaches for Conditional Optimal Transport with Applications in Bayesian Inference

In this work, we present two neural network approaches that approximate the solutions of static and dynamic conditional optimal transport (COT) problems. Both approaches enable conditional sampling and conditional density estimation, which are core tasks in Bayesian inference—particularly in the simulation-based (“likelihood-free”) setting. Our methods represent the target conditional distribution as a transformation of a tractable reference distribution. Obtaining such a transformation, chosen here to be an approximation of the COT map, is computationally challenging even in moderate dimensions. To improve scalability, our numerical algorithms use neural networks to parameterize candidate maps and further exploit the structure of the COT problem. Our static approach approximates the map as the gradient of a partially input convex neural network. It uses a novel numerical implementation to increase computational efficiency compared to state-of-the-art alternatives. Our dynamic approach approximates the conditional optimal transport via the flow map of a regularized neural ODE; compared to the static approach, it is slower to train but offers more modeling choices and can lead to faster sampling. We demonstrate both algorithms numerically, comparing them with competing state-of-the-art approaches, using benchmark datasets and simulation-based Bayesian inverse problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

NAPAbench 2: A network synthesis algorithm for generating realistic protein-protein interaction (PPI) network families

Comparative network analysis provides effective computational means for gaining novel insights into the structural and functional compositions of biological networks. In recent years, various methods have been developed for biological network alignment, whose main goal is to identify important similarities and critical differences between networks in terms of their topology and composition. A major impediment to advancing network alignment techniques has been the lack of gold-standard benchmarks that can be used for accurate and comprehensive performance assessment of such algorithms. The original NAPAbench (network alignment performance assessment benchmark) was developed to address this problem, and it has been widely utilized by many researchers for the development, evaluation, and comparison of novel network alignment techniques. In this work, we introduce NAPAbench 2—a major update of the original NAPAbench that was introduced in 2012. NAPAbench 2 includes a completely redesigned network synthesis algorithm that can generate protein-protein interaction (PPI) network families whose characteristics closely match those of the latest real PPI networks. Furthermore, the network synthesis algorithm comes with an intuitive GUI that allows users to easily generate PPI network families with an arbitrary number of networks of any size, according to a flexible user-defined phylogeny. In addition, NAPAbench 2 provides updated benchmark datasets—created using the redesigned network synthesis algorithm—which can be used for comprehensive performance assessment of network alignment algorithms and their scalability.

42 ENGINEERING↗
DOE OSTI2022

Intelligent System Partitioning for Agent-Based Security Constrained Optimal Power Flow

This project developed scalable, computationally efficient algorithms to solve realistic large-scale power system optimization problems as part of a larger series of competitions run by ARPA-E. These problems are important because the secure and reliable operation of the power grid, especially under increased uncertainty and variability, is growing increasingly challenging. The economic feasibility of the proposed methods developed by our team is quite low, considering it’s a purely software-based solution to operate power grids more efficiently. The technical effectiveness, as evidenced by our performance in the competition, balances heuristics and approximations to provide a tradeoff between speed and accuracy.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Fast and robust strategies for large-scale mixed-integer SCOPF

This project develops scalable, computationally efficient algorithms to solve realistic large-scale power system optimization problems, including systems with more than 8,000 buses, as part of a larger series of competitions run by ARPA-E. These problems are critical because the secure and reliable operation of the power grid is becoming increasingly challenging, especially under conditions of increased uncertainty and variability. The economic feasibility of our methods is high, given that they are purely software-based solutions designed to operate power grids more efficiently. The technical effectiveness balances heuristics and approximations to provide a trade-off between speed and accuracy.

24 POWER TRANSMISSION AND DISTRIBUTION↗