Search NASA⌕ Search

SEARCH · Search NASA

Results for “algorithms scalability”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Pele: An Exascale-Ready Suite of Combustion Codes

High fidelity simulations of realistic combustion devices are extremely demanding computationally because of the requirements to capture complex fuel chemical decomposition, its intricate interactions with turbulent, often multiphase, flows, and the wide separation of space and time scales between the thin flame and the device boundaries. Software required to carry out such computations tends to be extremely complex, particularly when designed to exploit hardware accelerators, and can be difficult to port and maintain. We present Pele, a performance portable suite of tools for the simulation of combustion systems, including codes to evolve reactive multiphase configurations in the low Mach number and compressible flow regimes, along with a set of inter-compatible post processing and in situ analysis tools. The Pele suite of tools is built on top of the AMReX framework for block-structured adaptive mesh refinement, which provides efficient data structures and algorithms that enable the development of a wide variety of efficient mesh and particle based PDE integration schemes. A hierarchical MPI+X parallelism scheme supports CPU-only and accelerated architectures, where X can be OpenMP, CUDA, and HIP based approaches for intra-node computational work distribution. The algorithms and data structures underlying the Pele simulation and analysis tools are highly scalable and performant across a wide variety of high-performance computing platforms, including DOEs newest exascale-class machines, Frontier and Aurora. The simulation and analysis tools are fully documented and freely distributed as open source via GitHub. We present key algorithmic and software challenges, solution strategies, performance and resulting set of capabilities.

AMReX↗
DOE OSTI2015

Open Research Challenges with Big Data - A Data-Scientist s Perspective

In this paper, we discuss data-driven discovery challenges of the Big Data era. We observe that recent innovations in being able to collect, access, organize, integrate, and query massive amounts of data from a wide variety of data sources have brought statistical data mining and machine learning under more scrutiny and evaluation for gleaning insights from the data than ever before. In that context, we pose and debate the question - Are data mining algorithms scaling with the ability to store and compute? If yes, how? If not, why not? We survey recent developments in the state-of-the-art to discuss emerging and outstanding challenges in the design and implementation of machine learning algorithms at scale. We leverage experience from real-world Big Data knowledge discovery projects across domains of national security, healthcare and manufacturing to suggest our efforts be focused along the following axes: (i) the data science challenge - designing scalable and flexible computational architectures for machine learning (beyond just data-retrieval); (ii) the science of data challenge the ability to understand characteristics of data before applying machine learning algorithms and tools; and (iii) the scalable predictive functions challenge the ability to construct, learn and infer with increasing sample size, dimensionality, and categories of labels. We conclude with a discussion of opportunities and directions for future research.

Big data↗
DOE OSTI2016

The GPU-enabled divide-expand-consolidate RI-MP2 method (DEC-RI-MP2)

We report porting of the Divide-Expand-Consolidate Resolution of the Identity second-order Møller–Plesset perturbation (DEC-RI-MP2) method to the graphic processing units (GPUs) using OpenACC compiler directives. It is shown that the OpenACC compiler directives implementation efficiently accelerates the rate-determining step of the DEC-RI-MP2 method with minor implementation effort. Moreover, the GPU acceleration results in a better load balance and thus in an overall scaling improvement of the DEC algorithm. The resulting cross-platform hybrid MPI/OpenMP/OpenACC implementation has scalable and portable performance on heterogeneous HPC architectures. The GPU-enabled code was benchmarked using a reduced version of the S12L test set of Stefan Grimme (Grimme, Chem. Eur. J. 2012, 18, 9955) consisting of supramolecular complexes up to 158 atoms and 4292 contracted basis functions (cc-pVTZ). The test set results demonstrate the general applicability of the DEC-RI-MP2 method showing results consistent with the DEC-RI-MP2 introductory paper (Baudin et al., J. Chem. Phys. 2016, 144, 054102) on molecules of complicated electronic structures. © 2016 Wiley Periodicals, Inc.

Chemistry↗
DOE OSTI2020

Atlas of Transcription Factor Binding Sites from ENCODE DNase Hypersensitivity Data across 27 Tissue Types

Characterizing the tissue-specific binding sites of transcription factors (TFs) is essential to reconstruct gene regulatory networks and predict functions for non-coding genetic variation. DNase-seq footprinting enables the prediction of genome-wide binding sites for hundreds of TFs simultaneously. Despite the public availability of high-quality DNase-seq data from hundreds of samples, a comprehensive, up-to-date resource for the locations of genomic footprints is lacking. Here, we develop a scalable footprinting workflow using two state-of-the-art algorithms: Wellington and HINT. We apply our workflow to detect footprints in 192 ENCODE DNase-seq experiments and predict the genomic occupancy of 1,515 human TFs in 27 human tissues. We validate that these footprints overlap true-positive TF binding sites from ChIP-seq. We demonstrate that the locations, depth, and tissue specificity of footprints predict effects of genetic variants on gene expression and capture a substantial proportion of genetic risk for complex traits.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2019

Enhancing Network Visibility and Security through Tensor Analysis

The increasing size, variety, rate of growth and change, and complexity of network data has warranted advanced network analysis and services. Tools that provide automated analysis through traditional or advanced signature-based systems or machine learning classifiers suffer from practical difficulties. These tools fail to provide comprehensive and contextual insights into the network when put to practical use in operational cyber security. In this paper, we present an effective tool for network security and traffic analysis that uses high-performance data analytics based on a class of unsupervised learning algorithms called tensor decompositions. The tool aims to provide a scalable analysis of the network traffic data and also reduce the cognitive load of network analysts and be network-expert-friendly by presenting clear and actionable insights into the network. With this being said, In this paper, we demonstrate the successful use of the tool in two completely diverse operational cyber security environments, namely, (1) security operations center (SOC) for the SCinet network at the SuperComputing (SC) Conference in 2016 and 2017 and (2) Reservoir Labs’ Local Area Network (LAN). In each of these environments, we produce actionable results for cyber security specialists including (but not limited to) (1) finding malicious network traffic involving internal and external attackers using port scans, SSH brute forcing, and NTP amplification attacks, (2) uncovering obfuscated network threats such as data exfiltration using DNS port and using ICMP traffic, and (3) finding network misconfiguration and performance degradation patterns.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Accelerated kinetic model for global macro stability studies of high-beta fusion reactors

The field reversed configuration (FRC), such as studied in the C-2W experiment at TAE Technologies, is an attractive candidate for realizing a nuclear fusion reactor. In an FRC, kinetic ion effects play the majority role in macroscopic stability, which allows global stability studies to make use of fluid-kinetic hybrid (also referred to as Ohm's law) models wherein ions are treated kinetically while electrons are treated as a fluid. The development and validation of such a hybrid particle-in-cell algorithm in the Exascale Computing Project code WarpX are reported here. Implementation of this model in the WarpX framework benefits from the numerical efficiency of WarpX as well as its scalability on large HPC systems and portability to different architectures. Performance benchmarks of the new algorithm for large, 3-dimensional, full device simulations from the Perlmutter supercomputer are presented. Results of a series of FRC simulations are discussed in which the impact of two-fluid effects on the tilt-mode growth rate was studied. It was observed that, in agreement with previous Hall-MHD studies, two-fluid effects have a stabilizing impact on the tilt mode.

Physics↗
DOE OSTI2012

Massively parallel X-ray scattering simulations

Although present X-ray scattering techniques can provide tremendous information on the nano-structural properties of materials that are valuable in the design and fabrication of energy-relevant nano-devices, a primary challenge remains in the analyses of such data. In this paper we describe a high-performance, flexible, and scalable Grazing Incidence Small Angle X-ray Scattering simulation algorithm and codes that we have developed on multi-core/CPU and many-core/GPU clusters. We discuss in detail our implementation, optimization and performance on these platforms. Our results show speedups of ~125x on a Fermi-GPU and ~20x on a Cray-XE6 24-core node, compared to a sequential CPU code, with near linear scaling on multi-node clusters. To our knowledge, this is the first GISAXS simulation code that is flexible to compute scattered light intensities in all spatial directions allowing full reconstruction of GISAXS patterns for any complex structures and with highresolutions while reducing simulation times from months to minutes.

Sarje, Abhinav↗
DOE OSTI2024

Matrix-Free High-Performance Saddle-Point Solvers for High-Order Problems in \(\boldsymbol{H}(\operatorname{\textbf{div}})\)

Here, this work describes the development of matrix-free GPU-accelerated solvers for high-order finite element problems in H(div). The solvers are applicable to grad-div and Darcy problems in saddle-point formulation, and have applications in radiation diffusion and porous media flow problems, among others. Using the interpolation–histopolation basis, efficient matrix-free preconditioners can be constructed for the (1, 1)-block and Schur complement of the block system. With these approximations, block-preconditioned MINRES converges in a number of iterations that is independent of the mesh size and polynomial degree. The approximate Schur complement takes the form of an M-matrix graph Laplacian and therefore can be well-preconditioned by highly scalable algebraic multigrid methods. High-performance GPU-accelerated algorithms for all components of the solution algorithm are developed, discussed, and benchmarked. Numerical results are presented on a number of challenging test cases, including the “crooked pipe” grad-div problem, the SPE10 reservoir modeling benchmark problem, and a nonlinear radiation diffusion test case.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Umpire

The Umpire library consists of a collection of software components that provide memory and resource management capabilities that can be used by diverse high performance computing (HPC) applications. The current set of capabilities that Umpire provides Includes: • Flexible allocators to provision memory In the range of available memory resources on a target system. • An extensible allocation strategy model, allowing custom algorithms to be used to allocate memory resources in a scalable and high­performance fashion. • Memory-location aware operations that can be used to migrate and modify memory using specialized functions dependent on source and target data locations • A set of Introspection capabilities to allow the user to query the state of the different allocators and memory allocations.

ECP↗
DOE OSTI2018

Deep data analysis via physically constrained linear unmixing: universal framework, domain examples, and a community-wide platform

Abstract Many spectral responses in materials science, physics, and chemistry experiments can be characterized as resulting from the superposition of a number of more basic individual spectra. In this context, unmixing is defined as the problem of determining the individual spectra, given measurements of multiple spectra that are spatially resolved across samples, as well as the determination of the corresponding abundance maps indicating the local weighting of each individual spectrum. Matrix factorization is a popular linear unmixing technique that considers that the mixture model between the individual spectra and the spatial maps is linear. Here, we present a tutorial paper targeted at domain scientists to introduce linear unmixing techniques, to facilitate greater understanding of spectroscopic imaging data. We detail a matrix factorization framework that can incorporate different domain information through various parameters of the matrix factorization method. We demonstrate many domain-specific examples to explain the expressivity of the matrix factorization framework and show how the appropriate use of domain-specific constraints such as non-negativity and sum-to-one abundance result in physically meaningful spectral decompositions that are more readily interpretable. Our aim is not only to explain the off-the-shelf available tools, but to add additional constraints when ready-made algorithms are unavailable for the task. All examples use the scalable open source implementation from https://github.com/ramkikannan/nmflibrary that can run from small laptops to supercomputers, creating a user-wide platform for rapid dissemination and adoption across scientific disciplines.

47 OTHER INSTRUMENTATION↗
DOE OSTI2018

Forming IDEAS: Interactive Data Exploration & Analysis System

Modern cyber security operations collect an enormous amount of logging and alerting data. While analysts have the ability to query and compute simple statistics and plots from their data, current analytical tools are too simple to admit deep understanding. To detect advanced and novel attacks, analysts turn to manual investigations. While commonplace, current investigations are time-consuming, intuition-based, and proving insufficient. Our hypothesis is that arming the analyst with easy-to-use data science tools will increase their work efficiency, provide them with the ability to resolve hypotheses with scientific inquiry of their data, and support their decisions with evidence over intuition. To this end, we present our work to build IDEAS (Interactive Data Exploration and Analysis System). We present three real-world use-cases that drive the system design from the algorithmic capabilities to the user interface. Finally, a modular and scalable software architecture is discussed along with plans for our pilot deployment with a security operation command.

Bridges, Robert↗
DOE OSTI2010

Scaling Linear Algebra Kernels using Remote Memory Access

This paper describes the scalability of linear algebra kernels based on remote memory access approach. The current approach differs from the other linear algebra algorithms by the explicit use of shared memory and remote memory access (RMA) communication rather than message passing. It is suitable for clusters and scalable shared memory systems. The experimental results on large scale systems (Linux-Infiniband cluster, Cray XT) demonstrate consistent performance advantages over ScaLAPACK suite, the leading implementation of parallel linear algebra algorithms used today. For example, on a Cray XT4 for a matrix size of 102400, our RMA-based matrix multiplication achieved over 55 teraflops while ScaLAPACK’s pdgemm measured close to 42 teraflops on 10000 processes.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

Hierarchical Model-Free Transactive Control of Building Loads to Support Grid Services

Residential buildings consume 4.4 quads of electricity annually, approximately 37% of the total electricity consumption in the United States. This represents a vast resource that can be used for demand management and other ancillary services. This project aims to develop a robust, scalable hierarchical transactional control mechanism incorporating elements of model-free control (MFC) and game theory to harness buildings to provide ancillary services to the grid. This approach is being taken to address the challenges of incorporating traditional transactional control schemes into existing buildings. The challenges include small individual building sizes requiring aggregation of many buildings, unpredictable energy usage that makes model identification difficult, and satisfying the sensitive occupant comfort constraints. In the proposed approach, by separating the control mechanism into two layers above and below the load aggregator, MFC can be used below the aggregator to modulate flexible building loads in response to pricing signals with guaranteed performance. This allows the burden of identifying an accurate model of the system to be shifted to the above-aggregator layer, where fluctuations in individual building usage have less impact on predicted building system behavior. Game theory concepts can then be used to determine pricing curves and control signals among regional aggregators. Managing this control in a game-theoretic approach will allow us to build in financial incentives that increase customer engagement. Additionally, the usage of MFC necessitates less burdensome computational and communication requirements, thus, it is easily deployable on small, embedded devices. In a broader sense, developing a strategy capable of effectively incorporating residential and small commercial buildings will allow greater throughput of existing and emerging grid services in addition to future transactive energy grid management methods. Using MFC within a hierarchical control architecture will allow the shifting of existing forecasting challenges to an aggregate level, where dynamics are slower and more predictable. This will enable a smooth interface between the grid services requests of utilities and the reliable control required by participating buildings. MFC, which supports distributed control architecture, permits a scalable solution that can be deployed to neighborhood-size systems as well as individual buildings. This project focuses on three objectives: (1) developing the mathematical framework, algorithm toolkit, and software toolset of the two-layer transactive control testbed; (2) developing a scalable solution for application over many residential and small-size commercial buildings with sparse distributed communication; and (3) field testing and implementation on hardware of the control strategies developed in the previous two objectives. The research and development activities are focused and designed to be impactful within the relevant 2025 targets timeframe. An open-source control framework for exploiting variability and dispatchability of building loads will be delivered as the outcome of the project. This capability enables greater participation of loads in electricity markets and ancillary services that are both useful for the utility and financially beneficial for building owners.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2018

Scalable Cloning on Large-Scale GPU Platforms with Application to Time-Stepped Simulations on Grids

Cloning is a technique to efficiently simulate a tree of multiple what-if scenarios that are unraveled during the course of a base simulation. However, cloned execution is highly challenging to realize on large, distributed memory computing platforms, due to the dynamic nature of the computational load across clones, and due to the complex dependencies spanning the clone tree. In this paper, we present the conceptual simulation framework, algorithmic foundations, and runtime interface of CloneX, a new system we designed for scalable simulation cloning. It efficiently and dynamically creates whole logical copies of a dynamic tree of simulations across a large parallel system without full physical duplication of computation and memory. The performance of a prototype implementation executed on up to 1,024 graphical processing units of a supercomputing system has been evaluated with three benchmarks—heat diffusion, forest fire, and disease propagation models—delivering a speed up of over two orders of magnitude compared to replicated runs. Finally, the results demonstrate a significantly faster and scalable way to execute many what-if scenario ensembles of large simulations via cloning using the CloneX interface.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Distributed Outage Detection in Power Distribution Networks

Real time topology knowledge is essential for situational awareness of power distribution networks. Line outages change the topology of a distribution network. Hence, outage detection is an important task. Most of the existing outage detection algorithms are centralized, in which sensors communicate their data to a control center which performs outage detection using the received data. However, with the increasing size of the distribution network and with different areas of the network being monitored by different operators, communication is a bottleneck and scalability is a major concern. To address these issues, we propose a novel outage detection algorithm using a divide and conquer approach. First, we divide a distribution network into sub-networks, such that outage detection can be run in parallel in each sub-network independently ensuring scalability to large networks. Further, to reduce the latency, bandwidth and attenuation challenges associated with communications in a large network, we divide each sub-network into multiple control areas which communicate only with their neighbors. We employ a distributed iterative load estimation across the control areas of each sub-network and then use the load estimate for local outage detection in each control area. Here, the performance of our algorithm is evaluated for multiple feeder models and compared against traditional centralized outage detection algorithms.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2015

Fast and Accurate Support Vector Machines on Large Scale Systems

Support Vector Machines (SVM) is a supervised Machine Learning and Data Mining (MLDM) algorithm, which has become ubiquitous largely due to its high accuracy and obliviousness to dimensionality. The objective of SVM is to find an optimal boundary --- also known as hyperplane --- which separates the samples (examples in a dataset) of different classes by a maximum margin. Usually, very few samples contribute to the definition of the boundary. However, existing parallel algorithms use the entire dataset for finding the boundary, which is sub-optimal for performance reasons. In this paper, we propose a novel distributed memory algorithm to eliminate the samples which do not contribute to the boundary definition in SVM. We propose several heuristics, which range from early (aggressive) to late (conservative) elimination of the samples, such that the overall time for generating the boundary is reduced considerably. In a few cases, a sample may be eliminated (shrunk) pre-emptively --- potentially resulting in an incorrect boundary. We propose a scalable approach to synchronize the necessary data structures such that the proposed algorithm maintains its accuracy. We consider the necessary trade-offs of single/multiple synchronization using in-depth time-space complexity analysis. We implement the proposed algorithm using MPI and compare it with libsvm--- de facto sequential SVM software --- which we enhance with OpenMP for multi-core/many-core parallelism. Our proposed approach shows excellent efficiency using up to 4096 processes on several large datasets such as UCI HIGGS Boson dataset and Offending URL dataset.

Vishnu, Abhinav↗
DOE OSTI2023

Superconducting Hyperdimensional Associative Memory Circuit for Scalable Machine Learning

Here we propose a generalized architecture for the first rapid-single-flux-quantum (RSFQ) associative memory circuit. The circuit employs hyperdimensional computing (HDC), a machine learning (ML) paradigm utilizing vectors with dimensionality in the thousands to represent information. HDC designs have small memory footprints, simple computations, and simple training algorithms compared to superconducting neural network accelerators (SNNAs), making them a better option for scalable SFQ machine learning (ML) solutions. The proposed superconducting HDC (SHDC) circuit uses entirely on-chip RSFQ memory which is tightly integrated with logic, operates at 33.3 GHz, is applicable to general ML tasks, and is manufacturable at practically useful scales given current SFQ fabrication limits. Tailored to a language recognition task, SHDC consists of ~ 2-20 M Josephson junctions (JJs) and consumes up to three times less power than an analogous CMOS HDC circuit while achieving 78-84% higher throughput. SHDC is capable of outperforming the state of the art RSFQ SNNA, SuperNPU, by 48-99% for all benchmark NN architectures tested while occupying up to 90% less area and consuming up to nine times less power. To the best of the authors' knowledge, SHDC is currently the only superconducting ML approach feasible at practically useful scales for real-world ML tasks and capable of online learning.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2024

How to Build a Quantum Supercomputer: Scaling from Hundreds to Millions of Qubits

In the span of four decades, quantum computation has evolved from an intellectual curiosity to a potentially realizable technology. Today, small-scale demonstrations have become possible for quantum algorithmic primitives on hundreds of physical qubits and proof-of-principle error-correction on a single logical qubit. Nevertheless, despite significant progress and excitement, the path toward a full-stack scalable technology is largely unknown. There are significant outstanding quantum hardware, fabrication, software architecture, and algorithmic challenges that are either unresolved or overlooked. These issues could seriously undermine the arrival of utility-scale quantum computers for the foreseeable future. Here, we provide a comprehensive review of these scaling challenges. We show how the road to scaling could be paved by adopting existing semiconductor technology to build much higher-quality qubits, employing system engineering approaches, and performing distributed quantum computation within heterogeneous high-performance computing infrastructures. These opportunities for research and development could unlock certain promising applications, in particular, efficient quantum simulation/learning of quantum data generated by natural or engineered quantum systems. To estimate the true cost of such promises, we provide a detailed resource and sensitivity analysis for classically hard quantum chemistry calculations on surface-code error-corrected quantum computers given current, target, and desired hardware specifications based on superconducting qubits, accounting for a realistic distribution of errors. Furthermore, we argue that, to tackle industry-scale classical optimization and machine learning problems in a cost-effective manner, heterogeneous quantum-probabilistic computing with custom-designed accelerators should be considered as a complementary path toward scalability.

Mohseni, Masoud↗