Search NASA⌕ Search

SEARCH · Search NASA

Results for “Load Balancing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
DOE OSTI2014

MPI-Bash: Parallel scripting right from the Bourne-Again Shell (Bash)

Parallel, scientific applications running on massively parallel supercomputers commonly produce large numbers of large data files. While parallel filesystems improve the performance of file generation, postprocessing activities such as archiving and compressing the data are often run sequentially (and therefore slowly), squandering the supercomputer's vast performance. Consequently, data that take hours to generate may take days to postprocess. Writing and debugging parallel postprocessing programs is often too time-consuming to be worthwhile, especially for one-off postprocessing operations. What the community needs is a simple mechanism for cobbling together scripts that distribute postprocessing work across a supercomputer's many nodes. MPI-Bash attempts to supply that mechanism by augmenting the GNU project's command shell, Bash, with support for the Message Passing Interface (MPI) and, if available, the Libcircle library for work distribution and load balancing. MPI-Bash makes it easy to parallelize bash scripts that run a set of Linux commands independently over a large number of input files. Because MPI-Bash includes various MPI functions for data transfer and synchronization, it is not limited to embarrassingly parallel workloads but can incorporate phased operations (i.e., all workers must finish operation X before any worker is allowed to begin operation Y).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

OLiMPS. OpenFlow Link-layer MultiPath Switching

The OLiMPS project’s goal was the development of an OpenFlow controller application allowing load balancing over multiple switched paths across a complex network topology. The second goal was to integrate the controller with Dynamic Circuit Network systems such as ESnet’s OSCARS. Both goals were achieved successfully, as laid out in this report.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

FOX: A Fault Oblivious Extreme-Scale Execution Environment

The FOX project, funded under the ASCR X-stack I program, developed systems software and runtime libraries for a new approach to the data and work distribution for massively parallel, fault oblivious application execution. Our work was motivated by the premise that exascale computing systems will provide a thousand-fold increase in parallelism and a proportional increase in failure rate relative to today’s machines. To deliver the capability of exascale hardware, the systems software must provide the infrastructure to support existing applications while simultaneously enabling efficient execution of new programming models that naturally express dynamic, adaptive, irregular computation; coupled simulations; and massive data analysis in a highly unreliable hardware environment with billions of threads of execution. Our OS research has prototyped new methods to provide efficient resource sharing, synchronization, and protection in a many-core compute node. We have experimented with alternative task/dataflow programming models and shown scalability in some cases to hundreds of thousands of cores. Much of our software is in active development through open source projects. Concepts from FOX are being pursued in next generation exascale operating systems. Our OS work focused on adaptive, application tailored OS services optimized for multi → many core processors. We developed a new operating system NIX that supports role-based allocation of cores to processes which was released to open source. We contributed to the IBM FusedOS project, which promoted the concept of latency-optimized and throughput-optimized cores. We built a task queue library based on distributed, fault tolerant key-value store and identified scaling issues. A second fault tolerant task parallel library was developed, based on the Linda tuple space model, that used low level interconnect primitives for optimized communication. We designed fault tolerance mechanisms for task parallel computations employing work stealing for load balancing that scaled to the largest existing supercomputers. Finally, we implemented the Elastic Building Blocks runtime, a library to manage object-oriented distributed software components. To support the research, we won two INCITE awards for time on Intrepid (BG/P) and Mira (BG/Q). Much of our work has had impact in the OS and runtime community through the ASCR Exascale OS/R workshop and report, leading to the research agenda of the Exascale OS/R program. Our project was, however, also affected by attrition of multiple PIs. While the PIs continued to participate and offer guidance as time permitted, losing these key individuals was unfortunate both for the project and for the DOE HPC community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Combinatorial Algorithms to Enable Computational Science and Engineering: Work from the CSCAPES Institute

This final progress report summarizes the work accomplished at the Combinatorial Scientific Computing and Petascale Simulations Institute. We developed Zoltan, a parallel mesh partitioning library that made use of accurate hypergraph models to provide load balancing in mesh-based computations. We developed several graph coloring algorithms for computing Jacobian and Hessian matrices and organized them into a software package called ColPack. We developed parallel algorithms for graph coloring and graph matching problems, and also designed multi-scale graph algorithms. Three PhD students graduated, six more are continuing their PhD studies, and four postdoctoral scholars were advised. Six of these students and Fellows have joined DOE Labs (Sandia, Berkeley), as staff scientists or as postdoctoral scientists. We also organized the SIAM Workshop on Combinatorial Scientific Computing (CSC) in 2007, 2009, and 2011 to continue to foster the CSC community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Scalable Parallel Methods for Analyzing Metagenomics Data at Extreme Scale

The field of bioinformatics and computational biology is currently experiencing a data revolution. The exciting prospect of making fundamental biological discoveries is fueling the rapid development and deployment of numerous cost-effective, high-throughput next-generation sequencing technologies. The result is that the DNA and protein sequence repositories are being bombarded with new sequence information. Databases are continuing to report a Moore’s law-like growth trajectory in their database sizes, roughly doubling every 18 months. In what seems to be a paradigm-shift, individual projects are now capable of generating billions of raw sequence data that need to be analyzed in the presence of already annotated sequence information. While it is clear that data-driven methods, such as sequencing homology detection, are becoming the mainstay in the field of computational life sciences, the algorithmic advancements essential for implementing complex data analytics at scale have mostly lagged behind. Sequence homology detection is central to a number of bioinformatics applications including genome sequencing and protein family characterization. Given millions of sequences, the goal is to identify all pairs of sequences that are highly similar (or “homologous”) on the basis of alignment criteria. While there are optimal alignment algorithms to compute pairwise homology, their deployment for large-scale is currently not feasible; instead, heuristic methods are used at the expense of quality. In this dissertation, we present the design and evaluation of a parallel implementation for conducting optimal homology detection on distributed memory supercomputers. Our approach uses a combination of techniques from asynchronous load balancing (viz. work stealing, dynamic task counters), data replication, and exact-matching filters to achieve homology detection at scale. Results for a collection of 2.56M sequences show parallel efficiencies of ~75-100% on up to 8K cores, representing a time-to-solution of 33 seconds. We extend this work with a detailed analysis of single-node sequence alignment performance using the latest CPU vector instruction set extensions. Preliminary results reveal that current sequence alignment algorithms are unable to fully utilize widening vector registers.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2015

Review and Status of Wind Integration and Transmission in the United States. Key Issues and Lessons Learned

The objective in electric power system operation is to use generation and transmission resources within organizational constraints and operational rules and regulations to reliably and costeffectively balance load and generation. To meet this objective, system operational practices have been created to accommodate the innate variability and uncertainty that comes from a variety of sources, such as uncertainty of demand forecasts, whether a specific generating unit will be available when called upon, the variability of demand from many different types of customers, and others. As more wind power is connected to the power system, operating experiences acquired during the past several years have generally confirmed the findings of wind integration studies: wind energy increases the level of variability and uncertainty that a system operator must manage.

17 WIND ENERGY↗
DOE OSTI2015

The Economic Benefits Of Multipurpose Reservoirs In The United States- Federal Hydropower Fleet

The United States is home to over 80,000 dams, of which approximately 3% are equipped with hydroelectric generating capabilities. When a dam serves as a hydropower facility, it provides a variety of energy services that range from clean, reliable power generation to load balancing that supports grid stability. In most cases, the benefits of dams and their associated reservoirs go far beyond supporting the nation s energy demand. As evidenced by the substantial presence of non-powered dams with the ability to store water in large capacities, the primary purpose of a dam may not be hydropower, but rather one of many other purposes. A dam and reservoir may support navigation, recreation, flood control, irrigation, and water supply, with each multipurpose benefit providing significant social and economic impacts on a local, regional, and national level. When hydropower is one of the services provided by a multipurpose reservoir, it is then part of an integrated system of competing uses. Operating rules, management practices, consumer demands, and environmental constraints must all be balanced to meet the multipurpose project s objectives. When federal dams are built, they are authorized by Congress to serve one or more functions. Legislation such as the Water Resources Development Act regulates the operation of the facility in order to coordinate the authorized uses and ensure the dam s intended objectives are being met. While multipurpose reservoirs account for billions of dollars in contributions to National Economic Development (NED) every year, no attempt has been made to evaluate their benefits on a national scale. This study is an on-going work conducted by Oak Ridge National Laboratory in an effort to estimate the economic benefits of multipurpose hydropower reservoirs in the United States. Given the important role that federal hydropower plays in the U.S., the first focus of this research will target the three main federal hydropower owners Tennessee Valley Authority, U.S. Army Corps of Engineers, and U.S. Bureau of Reclamation. Together these three agencies own and operate 157 powered dams which account for almost half of the total installed hydropower capacity in the U.S. Future work will include engaging publicly-owned utilities and the private sector in order to quantify the benefits of all multipurpose hydropower reservoirs in the U.S.

13 HYDRO ENERGY↗
DOE OSTI2016

Parallel Execution of Functional Mock-up Units in Buildings Modeling

A Functional Mock-up Interface (FMI) defines a standardized interface to be used in computer simulations to develop complex cyber-physical systems. FMI implementation by a software modeling tool enables the creation of a simulation model that can be interconnected, or the creation of a software library called a Functional Mock-up Unit (FMU). This report describes an FMU wrapper implementation that imports FMUs into a C++ environment and uses an Euler solver that executes FMUs in parallel using Open Multi-Processing (OpenMP). The purpose of this report is to elucidate the runtime performance of the solver when a multi-component system is imported as a single FMU (for the whole system) or as multiple FMUs (for different groups of components as sub-systems). This performance comparison is conducted using two test cases: (1) a simple, multi-tank problem; and (2) a more realistic use case based on the Modelica Buildings Library. In both test cases, the performance gains are promising when each FMU consists of a large number of states and state events that are wrapped in a single FMU. Load balancing is demonstrated to be a critical factor in speeding up parallel execution of multiple FMUs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

SIERRA Multimechanics Module: Aria User Manual – Version 4.40

Aria is a Galerkin finite element based program for solving coupled-physics problems described by systems of PDEs and is capable of solving nonlinear, implicit, transient and direct-to-steady state problems in two and three dimensions on parallel architectures. The suite of physics currently supported by Aria includes thermal energy transport, species transport, and electrostatics as well as generalized scalar, vector and tensor transport equations. Additionally, Aria includes support for manufacturing process flows via the incompressible Navier-Stokes equations specialized to a low Reynolds number (Re < 1) regime. Enhanced modeling support of manufacturing processing is made possible through use of either arbitrary Lagrangian- Eulerian (ALE) and level set based free and moving boundary tracking in conjunction with quasi-static nonlinear elastic solid mechanics for mesh control. Coupled physics problems are solved in several ways including fully-coupled Newton’s method with analytic or numerical sensitivities, fully-coupled Newton- Krylov methods and a loosely-coupled nonlinear iteration about subsets of the system that are solved using combinations of the aforementioned methods. Error estimation, uniform and dynamic ℎ-adaptivity and dynamic load balancing are some of Aria’s more advanced capabilities. Aria is based upon the Sierra Framework.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2014

Stochastic Particle Real Time Analyzer (SPARTA) Validation and Verification Suite

This report presents the test cases used to verify, validate and demonstrate the features and capabilities of the first release of the 3D Direct Simulation Monte Carlo (DSMC) code SPARTA (Stochastic Real Time Particle Analyzer). The test cases included in this report exercise the most critical capabilities of the code like the accurate representation of physical phenomena (molecular advection and collisions, energy conservation, etc.) and implementation of numerical methods (grid adaptation, load balancing, etc.). Several test cases of simple flow examples are shown to demonstrate that the code can reproduce phenomena predicted by analytical solutions and theory. A number of additional test cases are presented to illustrate the ability of SPARTA to model flow around complicated shapes. In these cases, the results are compared to other well-established codes or theoretical predictions. This compilation of test cases is not exhaustive, and it is anticipated that more cases will be added in the future.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

Stochastic Particle Real Time Analyzer (SPARTA) Validation and Verification Suite

This report presents the test cases used to verify, validate and demonstrate the features and capabilities of the first release of the 3D Direct Simulation Monte Carlo (DSMC) code SPARTA (Stochastic Real Time Particle Analyzer). The test cases included in this report exercise the most critical capabilities of the code like the accurate representation of physical phenomena (molecular advection and collisions, energy conservation, etc.) and implementation of numerical methods (grid adaptation, load balancing, etc.). Several test cases of simple flow examples are shown to demonstrate that the code can reproduce phenomena predicted by analytical solutions and theory. A number of additional test cases are presented to illustrate the ability of SPARTA to model flow around complicated shapes. In these cases, the results are compared to other well-established codes or theoretical predictions. This compilation of test cases is not exhaustive, and it is anticipated that more cases will be added in the future.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Investigating the Role of Biogeochemical Processes in the Northern High Latitudes on Global Climate Feedbacks Using an Efficient Scalable Earth System Model

The overall objectives of this DOE funded project is to combine scientific and computational challenges in climate modeling by expanding our understanding of the biogeophysical-biogeochemical processes and their interactions in the northern high latitudes (NHLs) using an earth system modeling (ESM) approach, and by adopting an adaptive parallel runtime system in an ESM to achieve efficient and scalable climate simulations through improved load balancing algorithms.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

Investigating Power System Primary and Secondary Reserve Interaction under High Wind Power Penetration

Power system frequency needs to be maintained close to its nominal value at all times to successfully balance load and generation and maintain system reliability. Adequate primary frequency response and secondary frequency response are the primary forces to correct an energy imbalance at the second-to-minute level. As wind energy becomes a larger portion of the world's energy portfolio, there is an increased need for wind to provide frequency response. This paper addresses one of the major concerns about using wind for frequency regulation: the unknown factor of the interaction between primary and secondary reserves. The lack of a commercially available tool to model this has limited the energy industry's understanding of when the depletion of primary reserves will impact the performance of secondary response or vice versa. This paper investigates the issue by developing a multi-area frequency response integration tool with combined primary and secondary capabilities. The simulation is conducted in close coordination with economical energy scheduling scenarios to ensure credible simulation results.

17 WIND ENERGY↗
DOE OSTI2015

Simulation of Internal Combustion Engines with High-Performance Computing Tools

Traditional Lagrangian spray modeling approaches for internal combustion engines are highly griddependent due to insufficient resolution in the near nozzle region. This is primarily because of inherent restrictions of volume fraction with the Lagrangian assumption together with high computational costs associated with small grid sizes. A state-of-the-art grid-convergent spray modeling approach was developed and implemented by Senecal et al. (ASME-ICEF2012-92043) in the CONVERGE software. The key features of the methodology include Adaptive Mesh Refinement (AMR), advanced liquid-gas momentum coupling, and improved distribution of the liquid phase, which enables use of cell sizes smaller than the nozzle diameter. This modeling approach was rigorously validated against nonevaporating, evaporating, and reacting data from the literature. The current numerical study focusses on further demonstration of the grid-convergent modeling approach for simulating a single-cylinder Cat® compression ignition engine. The simulated injector is characterized with a nominal nozzle exit diameter of 259 μm. Simulations using various minimum grid sizes (ranging from 125 μm to 1000 μm) are compared for engine performance and emissions parameters of interest such as pressure, heat release rate, ignition delay, NOx, HC, and soot emissions. The peak cell-count for the highest resolution simulation was on the order of 34 million. These computationally expensive simulations were facilitated at a high-performance computing facility at Argonne National Laboratory. METIS load-balancing algorithm was developed and implemented in Converge code for the simulations. Scaling studies were also performed. The validity of previously recommended grid settings (ASME-ICEF2012-92043) for accuracy/runtime trade-off is further assessed. Efficacy of a simplified combustion model is also compared against a detailed chemical kinetics based combustion model.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2017

Computing Properties of Hadrons, Nuclei and Nuclear Matter from Quantum Chromodynamics

This project was part of a coordinated software development effort which the nuclear physics lattice QCD community pursues in order to ensure that lattice calculations can make optimal use of present, and forthcoming leadership-class and dedicated hardware, including those of the national laboratories, and prepares for the exploitation of future computational resources in the exascale era. The UW team improved and extended software libraries used in lattice QCD calculations related to multi-nucleon systems, enhanced production running codes related to load balancing multi-nucleon production on large-scale computing platforms, and developed SQLite (addressable database) interfaces to efficiently archive and analyze multi-nucleon data and developed a Mathematica interface for the SQLite databases.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2017

Packet Capture Solutions: PcapDB Benchmark for High-Bandwidth Capture, Storage, and Searching

PcapDB stands alone when looking at the overall field of competitors, from the cost-effective COTS hardware, to the efficient utilization of disk space that enables a longer packet history. A scalable, 100GbE-enabled system that indexes every packet and indexes flow data without complicated load-balancing requirements. The Transport Layer search and indexing approach led to patent-pending flow indexing technology, providing a specialized database system specifically optimized around providing fast flow searches. While there are a plethora of options in network packet capture, there are very few that are able to effectively manage capture rates of more than 10 Gb/s, distributed capture and querying, and a responsive user interface. By far, the primary competitor in the market place is Endace and DeepSee; in addition to meeting the technical requirements we set out in this document, they provide technical support and a fully 'appliance like' system. In terms of cost, however, our experience has been that the yearly maintenance charges alone outstrip the entire hardware cost of solutions like PcapDB. Investment in cyber security research and development is a large part of what has enabled us to build the base of knowlegable workers needed to defend government resources in the rapidly evolving cyber security landscape. We believe projects like Bro, WireCap, and Farm do more than just fill temporary gaps in our capabilities. They give allow us to build the firm foundation needed to tackle the next generation of cyber challenges. PcapDB was built with loftier ambitions than simply solving the packet capture of a single lab site, but instead to provide a robust, scaleable packet capture solution to the DOE complex and beyond.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2017

SIERRA Multimechanics Module: Aria User Manual (V.4.44)

Aria is a Galerkin finite element based program for solving coupled-physics problems described by systems of PDEs and is capable of solving nonlinear, implicit, transient and direct-to-steady state problems in two and three dimensions on parallel architectures. The suite of physics currently supported by Aria includes thermal energy transport, species transport, and electrostatics as well as generalized scalar, vector and tensor transport equations. Additionally, Aria includes support for manufacturing process flows via the incompressible Navier-Stokes equations specialized to a low Reynolds number ( %3C 1 ) regime. Enhanced modeling support of manufacturing processing is made possible through use of either arbitrary Lagrangian- Eulerian (ALE) and level set based free and moving boundary tracking in conjunction with quasi-static nonlinear elastic solid mechanics for mesh control. Coupled physics problems are solved in several ways including fully-coupled Newton's method with analytic or numerical sensitivities, fully-coupled Newton- Krylov methods and a loosely-coupled nonlinear iteration about subsets of the system that are solved using combinations of the aforementioned methods. Error estimation, uniform and dynamic h -adaptivity and dynamic load balancing are some of Aria's more advanced capabilities. Aria is based upon the Sierra Framework.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

SIERRA Multimechanics Module: Aria User Manual Version 4.46.

Aria is a Galerkin fnite element based program for solving coupled-physics problems described by systems of PDEs and is capable of solving nonlinear, implicit, transient and direct-to-steady state problems in two and three dimensions on parallel architectures. The suite of physics currently supported by Aria includes thermal energy transport, species transport, and electrostatics as well as generalized scalar, vector and tensor transport equations. Additionally, Aria includes support for manufacturing process fows via the incompressible Navier-Stokes equations specialized to a low Reynolds number ( %3C 1 ) regime. Enhanced modeling support of manufacturing processing is made possible through use of either arbitrary Lagrangian- Eulerian (ALE) and level set based free and moving boundary tracking in conjunction with quasi-static nonlinear elastic solid mechanics for mesh control. Coupled physics problems are solved in several ways including fully-coupled Newton's method with analytic or numerical sensitivities, fully-coupled Newton- Krylov methods and a loosely-coupled nonlinear iteration about subsets of the system that are solved using combinations of the aforementioned methods. Error estimation, uniform and dynamic h -adaptivity and dynamic load balancing are some of Aria's more advanced capabilities. Aria is based upon the Sierra Framework.

97 MATHEMATICS AND COMPUTING↗