Search NASA⌕ Search

SEARCH · Search NASA

Results for “algorithms scalability”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56
All sourcesNASA NTRSDOE OSTI
NASA NTRS2005

A Testbed Demonstration of an Intelligent Archive in a Knowledge Building System

The last decade's influx of raw data and derived geophysical parameters from several Earth observing satellites to NASA data centers has created a data-rich environment for Earth science research and applications. While advances in hardware and information management have made it possible to archive petabytes of data and distribute terabytes of data daily to a broad community of users, further progress is necessary in the transformation of data into information, and information into knowledge that can be used in particular applications in order to realize the full potential of these valuable datasets. In examining what is needed to enable this progress in the data provider environment that exists today and is expected to evolve in the next several years, we arrived at the concept of an Intelligent Archive in context of a Knowledge Building System (IA/KBS). Our prior work and associated papers investigated usage scenarios, required capabilities, system architecture, data volume issues, and supporting technologies. We identified six key capabilities of an IA/KBS: Virtual Product Generation, Significant Event Detection, Automated Data Quality Assessment, Large-Scale Data Mining, Dynamic Feedback Loop, and Data Discovery and Efficient Requesting. Among these capabilities, large-scale data mining is perceived by many in the community to be an area of technical risk. One of the main reasons for this is that standard data mining research and algorithms operate on datasets that are several orders of magnitude smaller than the actual sizes of datasets maintained by realistic earth science data archives. Therefore, we defined a test-bed activity to implement a large-scale data mining algorithm in a pseudo-operational scale environment and to examine any issues involved. The application chosen for applying the data mining algorithm is wildfire prediction over the continental U.S. This paper reports a number of observations based on our experience with this test-bed. While proof-of-concept for data mining scalability and utility has been a major goal for the research reported here, it was not the only one. The other five capabilities of an WKBS named above have been considered as well, and an assessment of the implications of our experience for these other areas will also be presented. The lessons learned through the testbed effort and presented in this paper will benefit technologists, scientists, and system operators as they consider introducing IA/KBS capabilities into production systems.

Ramapriyan, Hampapuram↗
DOE OSTI2019

Graph Analytics and Optimization Methods for Insights from the Uber Movement Data

In this work we leverage the Uber movement dataset for the Los Angeles (LA) area where partial TAZ to TAZ (Traffic Analysis Zone) trip time data is available. We first create a TAZ-TAZ network based on nearest neighbors and propose a model that allows us to complete the $(O-D)$ (Origin-Destination) travel time matrix, using optimization methods such as non-negative least squares. We apply these algorithms for several communities in the TAZ-TAZ network and present insights in the form of completed $(O-D)$ matrices and associated temporal trends. We qualify the error performance and scalability of our flows. We conclude by pointing out the directions in our ongoing work to improve the quality and scale of travel time estimation.

Visweswara Sathanur, Arun↗
DOE OSTI2020

Engineering microscale systems for fully autonomous intracellular neural interfaces

Conventional electrodes and associated positioning systems for intracellular recording from single neurons in vitro and in vivo are large and bulky, which has largely limited their scalability. Further, acquiring successful intracellular recordings is very tedious, requiring a high degree of skill not readily achieved in a typical laboratory. We report here a robotic, MEMS-based intracellular recording system to overcome the above limitations associated with form-factor, scalability and highly skilled and tedious manual operations required for intracellular recordings. This system combines three distinct technologies: 1) novel microscale, glass-polysilicon penetrating electrode for intracellular recording, 2) electrothermal microactuators for precise microscale movement of each electrode and 3) closed-loop control algorithm for autonomous positioning of electrode inside single neurons. Here, we demonstrate the novel, fully integrated system of glass-polysilicon microelectrode, microscale actuators and controller for autonomous intracellular recordings from single neurons in the abdominal ganglion of Aplysia Californica (n = 5 cells). Consistent resting potentials (< –35 mV) and action potentials (> 60 mV) were recorded after each successful penetration attempt with the controller and microactuated glass-polysilicon microelectrodes. The success rate of penetration and quality of intracellular recordings achieved using electrothermal microactuators were comparable to that of conventional positioning systems. The MEMS-based system offers significant advantages: 1) reduction in overall size for potential use in behaving animals, 2) scalable approach to potentially realize multi-channel recordings and 3) a viable method to fully automate measurement of intracellular recordings. Furthermore, this system will be evaluated in vivo in future rodent studies.

42 ENGINEERING↗
DOE OSTI2019

A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements

Multiphase flow can be difficult to simulate with high accuracy due to the wide range of scales associated with various multiphase phenomena. These scales may range from the size of individual particles to the entire domain of interest. Traditionally, large scale systems can only be simulated using averaging approaches that filter out the locations of individual particles. In this work, the Euler–Lagrange method is used to simulate large-scale dense particle systems in which each individual particle is tracked. In order to accomplish this, the highly scalable spectral element code nek5000 has been extended to handle the multiple levels of multiphase coupling in these systems. These levels include what has been called one-, two-, and four-way coupling. Here, each level has been separated to detail the computational impact of each stage. A binned ghost particle algorithm has also been developed to efficiently handle the challenges of two- and four-way coupling in a parallel processing context. The algorithms and their implementations are then shown to scale to 65,536 Message Passing Interface (MPI) ranks in both the strong and weak limits. After this, validation is performed through simulation of a small-scale fluidized bed. Lastly, a large-scale fluidized bed is simulated with 65,536 MPI ranks and is able to capture the unique physics of the onset of fluidization.

Computer Science↗
DOE OSTI2011

Cheetah: A Framework for Scalable Hierarchical Collective Operations

Collective communication operations, used by many scientific applications, tend to limit overall parallel application performance and scalability. Computer systems are becoming more heterogeneous with increasing node and core-per-node counts. Also, a growing number of data-access mechanisms, of varying characteristics, are supported within a single computer system. We describe a new hierarchical collective communication framework that takes advantage of hardware-specific data-access mechanisms. It is flexible, with run-time hierarchy specification, and sharing of collective communication primitives between collective algorithms. Data buffers are shared between levels in the hierarchy reducing collective communication management overhead. We have implemented several versions of the Message Passing Interface (MPI) collective operations, MPI Barrier() and MPI Bcast(), and run experiments using up to 49, 152 processes on a Cray XT5, and a small InfiniBand based cluster. At 49, 152 processes our barrier implementation outperforms the optimized native implementation by 75%. 32 Byte and one Mega-Byte broadcasts outperform it by 62% and 11%, respectively, with better scalability characteristics. Improvements relative to the default Open MPI implementation are much larger.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2011

Investigating the effectiveness of many-core network processors for high performance cyber protection systems. Part I, FY2011.

This report documents our first year efforts to address the use of many-core processors for high performance cyber protection. As the demands grow for higher bandwidth (beyond 1 Gbits/sec) on network connections, the need to provide faster and more efficient solution to cyber security grows. Fortunately, in recent years, the development of many-core network processors have seen increased interest. Prior working experiences with many-core processors have led us to investigate its effectiveness for cyber protection tools, with particular emphasis on high performance firewalls. Although advanced algorithms for smarter cyber protection of high-speed network traffic are being developed, these advanced analysis techniques require significantly more computational capabilities than static techniques. Moreover, many locations where cyber protections are deployed have limited power, space and cooling resources. This makes the use of traditionally large computing systems impractical for the front-end systems that process large network streams; hence, the drive for this study which could potentially yield a highly reconfigurable and rapidly scalable solution.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

Noise-Resilient Quantum Machine Learning for Stability Assessment of Power Systems

Transient stability assessment (TSA) is a cornerstone for resilient operations of todays interconnected power grids. This paper is a confluence of quantum computing, data science and machine learning to potentially address the power system TSA issue. Here, we devise a quantum TSA (QTSA) method to enable scalable and efficient data-driven transient stability prediction for bulk power systems, which is the first attempt to tackle the TSA issue with quantum computing. Our contributions are three-fold: 1) A high expressibility, low-depth (HELD) quantum circuit is designed for accurate and noise-resilient TSA; 2) A quantum natural gradient descent algorithm is developed for efficient HELD circuit training; 3) A systematical analysis on QTSAs performance under various quantum factors is per-formed. QTSA underpins a foundation of quantum-enabled and data-driven power grid stability analytics. It renders the intractable TSA straightforward and effortless in the Hilbert space, and therefore provides stability information for power system operations. Extensive experiments on quantum simulators and real quantum computers verify the accuracy, noise-resilience, scalability and universality of QTSA.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2023

A Novel Resilience-Oriented Cellular Grid Formation Approach for Distribution Systems with Behind-the-Meter Distributed Energy Resources

This paper presents a resilience-oriented cellular grid formation approach to achieve scalable and reconfigurable community microgrid operations for distribution systems with behind-the-meter distributed energy resources. A set of interconnected solar photovoltaics, energy storage systems, and load is termed as a cell, implying a subset of the grid that can operate independently using its own resources. Cells are identified such that each cell inherently has sufficient energy resources to black start and can provide a certain level of backup power for its load under the loss of utility power supply. The proposed cell formation approach builds on a unique self-organizing map-based method (SomRes) to quantify a system's resilience. Using SomRes and a non-dominated sorting-based genetic algorithm (NSGA-II), a fast and efficient cell formation algorithm is developed to identify cells in a distribution system that are resilient against extreme events. The efficacy of the proposed approach is demonstrated on a numerical model of a real distribution feeder in Colorado, United States.

distributed energy resource↗
DOE OSTI2015

Forces and stress in second order Møller-Plesset perturbation theory for condensed phase systems within the resolution-of-identity Gaussian and plane waves approach

The forces acting on the atoms as well as the stress tensor are crucial ingredients for calculating the structural and dynamical properties of systems in the condensed phase. Here, these derivatives of the total energy are evaluated for the second-order Møller-Plesset perturbation energy (MP2) in the framework of the resolution of identity Gaussian and plane waves method, in a way that is fully consistent with how the total energy is computed. This consistency is non-trivial, given the different ways employed to compute Coulomb, exchange, and canonical four center integrals, and allows, for example, for energy conserving dynamics in various ensembles. Based on this formalism, a massively parallel algorithm has been developed for finite and extended system. The designed parallel algorithm displays, with respect to the system size, cubic, quartic, and quintic requirements, respectively, for the memory, communication, and computation. All these requirements are reduced with an increasing number of processes, and the measured performance shows excellent parallel scalability and efficiency up to thousands of nodes. Additionally, the computationally more demanding quintic scaling steps can be accelerated by employing graphics processing units (GPU’s) showing, for large systems, a gain of almost a factor two compared to the standard central processing unit-only case. In this way, the evaluation of the derivatives of the RI-MP2 energy can be performed within a few minutes for systems containing hundreds of atoms and thousands of basis functions. With good time to solution, the implementation thus opens the possibility to perform molecular dynamics (MD) simulations in various ensembles (microcanonical ensemble and isobaric-isothermal ensemble) at the MP2 level of theory. Geometry optimization, full cell relaxation, and energy conserving MD simulations have been performed for a variety of molecular crystals including NH{sub 3}, CO{sub 2}, formic acid, and benzene.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Hierarchical Flexibility Offering Strategy for Integrated Hybrid Resources in Real-time Energy Markets

This paper proposes a hierarchical model for determining the energy flexibility offering strategy of integrated hybrid resources (IHRs) in power distribution systems to participate in real-time energy markets. The proposed model utilizes the scalability, fast response time, and uncertainty observation of deep reinforcement learning (DRL) to overcome the scalability issue of operating numerous flexible resources and deliverability of energy flexibility to the real-time markets in the presence of the network constraints. To that end, the power distribution system is divided into multiple IHRs, where different types of flexible loads, energy storage systems, and solar plants with controllable inverters are operated through local IHR controllers, trained by deep deterministic policy gradient (DDPG) algorithm. Active power request and reactive power capacity of IHRs are then transmitted to a central flexibility controller, where a quadratic optimization model ensures the deliverability of the energy flexibility to the real-time energy market by satisfying the distribution network constraints. The proposed model is implemented on the 123-bus test power distribution system, demonstrating the capability of DRL-based hierarchical model for scalable operation of IHRs in order to offer deliverable energy flexibility to the real-time energy market.

Majidi, Majid↗
DOE OSTI2010

Design and performance of a scalable, parallel statistics toolkit.

Most statistical software packages implement a broad range of techniques but do so in an ad hoc fashion, leaving users who do not have a broad knowledge of statistics at a disadvantage since they may not understand all the implications of a given analysis or how to test the validity of results. These packages are also largely serial in nature, or target multicore architectures instead of distributed-memory systems, or provide only a small number of statistics in parallel. This paper surveys a collection of parallel implementations of statistics algorithm developed as part of a common framework over the last 3 years. The framework strategically groups modeling techniques with associated verification and validation techniques to make the underlying assumptions of the statistics more clear. Furthermore it employs a design pattern specifically targeted for distributed-memory parallelism, where architectural advances in large-scale high-performance computing have been focused. Moment-based statistics (which include descriptive, correlative, and multicorrelative statistics, principal component analysis (PCA), and k-means statistics) scale nearly linearly with the data set size and number of processes. Entropy-based statistics (which include order and contingency statistics) do not scale well when the data in question is continuous or quasi-diffuse but do scale well when the data is discrete and compact. We confirm and extend our earlier results by now establishing near-optimal scalability with up to 10,000 processes.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2018

DeepHyper: Asynchronous Hyperparameter Search for Deep Neural Networks

Hyperparameters employed by deep learning (DL) methods play a substantial role in the performance and reliability of these methods in practice. Unfortunately, finding performance-optimizing hyperparameter settings is a notoriously difficult task. Hyperparameter search methods typically have limited production-strength implementations or do not target scalability within a highly parallel machine, portability across different machines, experimental comparison between different methods, and tighter integration with workflow systems. In this paper, we present DeepHyper, a Python package that provides a common interface for the implementation and study of scalable hyperparameter search methods. It adopts the Balsam workflow system to hide the complexities of running large numbers of hyperparameter configurations in parallel on high-performance computing (HPC) systems. We implement and study asynchronous model-based search methods that consist of sampling a small number of input hyperparameter configurations and progressively fitting surrogate models over the input-output space until exhausting a user-defined budget of evaluations. We evaluate the efficacy of these methods relative to approaches such as random search, genetic algorithms, Bayesian optimization, and hyperband on DL benchmarks on CPU- and GPU-based HPC systems.

Balaprakash, Prasanna↗
DOE OSTI2016

Solvers for $\mathcal{O} (N)$ Electronic Structure in the Strong Scaling Limit

Here we present a hybrid OpenMP/Charm\tt++ framework for solving the $\mathcal{O} (N)$ self-consistent-field eigenvalue problem with parallelism in the strong scaling regime, $P\gg{N}$, where $P$ is the number of cores, and $N$ is a measure of system size, i.e., the number of matrix rows/columns, basis functions, atoms, molecules, etc. This result is achieved with a nested approach to spectral projection and the sparse approximate matrix multiply [Bock and Challacombe, SIAM J. Sci. Comput., 35 (2013), pp. C72--C98], and involves a recursive, task-parallel algorithm, often employed by generalized $N$-Body solvers, to occlusion and culling of negligible products in the case of matrices with decay. Lastly, employing classic technologies associated with generalized $N$-Body solvers, including overdecomposition, recursive task parallelism, orderings that preserve locality, and persistence-based load balancing, we obtain scaling beyond hundreds of cores per molecule for small water clusters ([H${}_2$O]${}_N$, $N \in \{ 30, 90, 150 \}$, $P/N \approx \{ 819, 273, 164 \}$) and find support for an increasingly strong scalability with increasing system size $N$.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Corrective Control to Handle Forecast Uncertainty: A Chance Constrained Optimal Power Flow

Higher shares of electricity generation from renewable energy sources and market liberalization is increasing uncertainty in power systems operation. At the same time, operation is becoming more flexible with improved control systems and new technology such as phase shifting transformers (PSTs) and high voltage direct current connections (HVDC). Previous studies have shown that the use of corrective control in response to outages contributes to a reduction in operating cost, while maintaining N-1 security. In this work, we propose a method to extend the use of corrective control of PSTs and HVDCs to react to uncertainty. We characterize the uncertainty as continuous random variables, and define the corrective control actions through affine control policies. This allows us to efficiently model control reactions to a large number of uncertainty sources. The control policies are then included in a chance constrained optimal power flow formulation, which guarantees that the system constraints are enforced with a desired probability. Lastly, by applying an analytical reformulation of the chance constraints, we obtain a second-order cone problem for which we develop an efficient solution algorithm. In a case study for the IEEE 118 bus system, we show that corrective control for uncertainty leads to a decrease in operational cost, while maintaining system security. Further, we demonstrate the scalability of the method by solving the problem for the IEEE 300 bus and the Polish system test cases.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Software Control Program For Transportable Microgrid State-of-charge Balancing And Frequency Stability Controls

A deterministic state-of-charge (SOC) balancing approach software control code is introduced as an integral secondary management to primary control layer of an islanded small microgrid or nanogrid system made up of multiple grid-forming inverter/battery/solar combination systems, where each set of batteries with each inverter are on independent DC buses (i.e. non-paralleled on the DC sides). A DERMS-level control approach, algorithm and automation controller program was developed to improve coordination and enable microgrid asset compliance and SOC balancing, enabling provision of a system-level power stability support architecture, load support, and asset scalability. The architecture is configured to treat each unit or micro/nano-grid as a node in a microgrid network, allowing for autonomous DERMS control regarding load and SOC balancing and power stability. As the network grows with the addition of units, greater coordination efforts may be required. The ideal small network microgrid ranges from 2-10 inverter/battery units before additional control parameters must be considered in the existing architecture. The control approach focuses on a deterministic state-of-charge analysis as the primary level control process followed by a secondary control loop using a forced frequency-watt droop strategy to conform off-the-shelf components into behaving under a leader-follower configuration. Adopting this control scheme has been shown to allow for a balanced, unit-coordinated microgrid network, enabling stable power flow. The deterministic state-of-charge approach is introduced as an integral primary control layer of an islanded small network microgrid. A standard strategy for SOC balancing is implementing a battery management system (BMS) to control SOC on the DC side. An alternative approach is to determine how to coordinate sending and receiving power on the AC side with multiple units. The latter approach assesses all the integrated units in the microgrid network. Once the individual units are identified, further system data is required to calculate each unit's total kWh, provided information about its capability to supply or consume kWh and availability. The secondary control layer in the multi-layered small network microgrid methodology uses the primary layer’s decision to initiate frequency setpoint changes, initializing the SOC balancing. The secondary control layer considers numerous system-dependent variables to enable a charging and discharging profile based on adjustable frequency setpoints. The combined architecture will result in stable, coordinated power flow enhancing an AC microgrid's functionalities.

Myers, KurtS [Idaho National Laboratory (INL), Ida↗
DOE OSTI2011

Multigrid Smoothers for Ultra-Parallel Computing: Additional Theory and Discussion

This paper investigates the properties of smoothers in the context of algebraic multigrid (AMG) running on parallel computers with potentially millions of processors. The development of multigrid smoothers in this case is challenging, because some of the best relaxation schemes, such as the Gauss-Seidel (GS) algorithm, are inherently sequential. Based on the sharp two-grid multigrid theory from we characterize the smoothing properties of a number of practical candidates for parallel smoothers, including several C-F, polynomial, and hybrid schemes. We show, in particular, that the popular hybrid GS algorithm has multigrid smoothing properties which are independent of the number of processors in many practical applications, provided that the problem size per processor is large enough. This is encouraging news for the scalability of AMG on ultra-parallel computers. We also introduce the more robust `1 smoothers, which are always convergent and have already proven essential for the parallel solution of some electromagnetic problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Scalable Computation of 2D-Minkowski Sum of Arbitrary Non-Convex Domains: Modeling Flexibility in Energy Resources

The flexibility of active ($p$) and reactive power ($q$) consumption in distributed energy resources (DERs) can be represented as a (potentially non-convex) set of points in the $p$-$q$ plane. Modeling of the aggregated flexibility in a heterogeneous ensemble of DERs as a Minkowski sum (M-sum) is computationally intractable even for moderately sized populations. In this article, we propose a scalable method of computing the M-sum of the flexibility domains of a heterogeneous ensemble of DERs, which are allowed to be non-convex, non-compact. In particular, the proposed algorithm computes a guaranteed superset of the true M-sum, with desired accuracy. The worst-case complexity of the algorithm is computed. Special cases are considered, and it is shown that under certain scenarios, it is possible to achieve a complexity that is linear with the size of the ensemble. Numerical examples are provided by computing the aggregated flexibility of different mix of DERs under varying scenarios.

Kundu, Soumya↗
NASA NTRS2021

Towards the Development of a Multi-Agent Cognitive Networking System for the Lunar Environment

This paper details the development of a multi-agent cognitive system intended to optimize networking performance in the lunar environment. NASA’s current concept of the future of lunar communication, LunaNet [1], outlines a complex network of networks. Challenges such as scalability, interoperability and reliability must first be addressed to successfully fulfill this vision. Machine intelligence can greatly reduce the reliance on human operators and enable efficient operations for tasks such as scheduling and network management. The application of machine learning, artificial intelligence, and other automated decision-making techniques can be used to allow network nodes to intelligently sense and adapt to changes in the environment such as link disruptions, new nodes joining the network, and support for a diverse range of protocols. Cognitive networking seeks to evolve these technologies into an autonomous system with improved science data return, reliability, and scalability. In this paper, we study three main areas a means to further develop cognitive networking capabilities: networking and flight software development, analysis of wireless data for modeling and simulation, and development of algorithms for a multi-agent system.

Cognitive Networking↗