SEARCH · Search NASA

Results for “Parallel application frameworks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Bayesian Optimization for Anything (BOA): An open-source framework for accessible, user-friendly Bayesian optimization

We introduce Bayesian Optimization for Anything (BOA), a high-level Bayesian Optimization (BO) framework and model wrapping toolkit, which presents a novel approach to simplifying BO, with the goal of making it more accessible and user-friendly, particularly for those with limited expertise in the field. BOA addresses common barriers in implementing BO, focusing on ease of use, reducing the need for deep domain knowledge, and cutting down on extensive coding requirements. A notable feature of BOA is its language-agnostic architecture, which facilitates broader application in various fields and to a wider audience. We showcase BOA's application through three examples: a high-dimensional optimization with parameters of the SWAT+ watershed model, a highly parallelized optimization of this intrinsically non-parallel model, and a multi-objective optimization of the FETCH Tree-Crown Hydrodynamics model. Furthermore, these test cases illustrate BOA's effectiveness in addressing complex optimization challenges in diverse scenarios.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2018

Distributed caching for processing raw arrays

As applications continue to generate multi-dimensional data at exponentially increasing rates, fast analytics to extract meaningful results is becoming extremely important. The database community has developed array databases that alleviate this problem through a series of techniques. In-situ mechanisms provide direct access to raw data in the original format---without loading and partitioning. Parallel processing scales to the largest datasets. In-memory caching reduces latency when the same data are accessed across a workload of queries. However, we are not aware of any work on distributed caching of multi-dimensional raw arrays. In this paper, we introduce a distributed framework for cost-based caching of multi-dimensional arrays in native format. Given a set of files that contain portions of an array and an online query workload, the framework computes an effective caching plan in two stages. First, the plan identifies the cells to be cached locally from each of the input files by continuously refining an evolving R-tree index. In the second stage, an optimal assignment of cells to nodes that collocates dependent cells in order to minimize the overall data transfer is determined. We design cache eviction and placement heuristic algorithms that consider the historical query workload. A thorough experimental evaluation over two real datasets in three file formats confirms the superiority -- by as much as two orders of magnitude -- of the proposed framework over existing techniques in terms of cache overhead and workload execution time.

Zhao, Weijie↗

DOE OSTI2025

ACHILLES-GENIE Interface for Neutrino Simulations, ADRIANO2 Tile Prototype for High-Granularity Dual-Readout Calorimetry

ICARUS (Imaging Cosmic And Rare Underground Signals) is a liquid argon time projection chamber (LArTPC) detector that pursues the sterile neutrino, which relies on accurate simulations of neutrino-argon interactions. REDTOP (Rare Eta Decays To Observe new Physics) is a proposed low-energy, high-intensity meson factory designed to explore rare $\eta$/$\eta'$ meson decays and probe physics beyond the Standard Model. As a next-generation experiment, this requires both accurate simulations and innovative detector technologies. This project contributes to both ICARUS, from a simulation perspective, and REDTOP, from both a simulation and detection perspective, through the event generation of lepton-nucleon interactions and the physical enhancement of the calorimeter technology within the REDTOP detector. We developed an interface between ACHILLES (A CHIcago Land Lepton Event Simulator), a theory-driven lepton-level event generator, and GENIE, a robust event generator framework used for neutrino physics. By incorporating the precise theoretical cross-section calculations of ACHILLES into the experimental realism of GENIE, the interface allows for improved accuracy of neutrino-nucleon simulations, which can be adapted for the proton beam specifications of the REDTOP meson factory as well as for the ICARUS experiment. In parallel, we developed an improved prototype for the ADRIANO2 (A Dual Readout Integrally Active Non-segmented Option) dual-readout calorimeter tiles for the REDTOP detector. To improve the efficiency of the lead-glass tiles trapping Cherenkov light for energy reconstruction and particle identification, we optimized the application of a highly reflective coating. Through viscosity and thickness control, masking, and a custom spray technique, we refined the coating process to reduce surface defects and improve light yield. Together, these efforts strengthen the ICARUS neutrino program and REDTOP's capability of detecting rare decay events.

Visser, Erin [Michigan State U.]↗

DOE OSTI2023

CompLaB v1.0: a scalable pore-scale model for flow, biogeochemistry, microbial metabolism, and biofilm dynamics

Abstract. Microbial activity and chemical reactions in porous media depend on the local conditions at the pore scale and can involve complex feedback with fluid flow and mass transport. We present a modeling framework that quantitatively accounts for the interactions between the bio(geo)chemical and physical processes and that can integrate genome-scale microbial metabolic information into a dynamically changing, spatially explicit representation of environmental conditions. The model couples a lattice Boltzmann implementation of Navier–Stokes (flow) and advection–diffusion-reaction (mass conservation) equations. Reaction formulations can include both kinetic rate expressions and flux balance analysis, thereby integrating reactive transport modeling and systems biology. We also show that the use of surrogate models such as neural network representations of in silico cell models can speed up computations significantly, facilitating applications to complex environmental systems. Parallelization enables simulations that resolve heterogeneity at multiple scales, and a cellular automaton module provides additional capabilities to simulate biofilm dynamics. The code thus constitutes a platform suitable for a range of environmental, engineering and – potentially – medical applications, in particular ones that involve the simulation of microbial dynamics.

58 GEOSCIENCES↗

DOE OSTI2025

Enhancing ICARUS and REDTOP Software and Hardware: Event Generator Interface Development and Calorimeter Tile Prototype

Visser, Erin [Michigan State U.] (ORCID:0009000184↗

DOE OSTI2026

Benchmarking of massively parallel phase-field codes for directional solidification

We present a detailed benchmark comparing two state-of-the-art phase-field implementations for simulating alloy solidification under experimentally relevant conditions. The study investigates the directional solidification of Al-3wt%Cu under high-velocity solidification conditions and SCN-0.46wt% camphor under microgravity conditions from National Aeronautics and Space Administration (NASA) DECLIC-DSI-R experiments. Both codes, one employing finite-difference discretization with uniform mesh and GPU-acceleration (GPU-PF) and the other one employing finite-element discretization with adaptive-mesh and CPU-parallelization (PRISMS-PF), solve the same quantitative phase-field formulation that incorporates an anti-trapping current for the solidification of dilute alloys. We evaluate the predictions of each code for dendritic morphology, primary spacing, and tip dynamics in both 2D and 3D, as well as their numerical convergence and computational performance. While existing benchmark problems have primarily focused on simplified or small-scale simulations, they do not reflect the computational and modeling challenges posed by employing experimentally relevant time and length scales. Our results provide a practical framework for assessing phase-field code performance as well as validating and facilitating their application in integrated computational materials engineering (ICME) workflows that require integration with realistic experimental data.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Neuromorphic Graph Algorithms

Graph algorithms enable myriad large-scale applications including cybersecurity, social network analysis, resource allocation, and routing. The scalability of current graph algorithm implementations on conventional computing architectures are hampered by the demise of Moore’s law. We present a theoretical framework for designing and assessing the performance of graph algorithms executing in networks of spiking artificial neurons. Although spiking neural networks (SNNs) are capable of general-purpose computation, few algorithmic results with rigorous asymptotic performance analysis are known. SNNs are exceptionally well-motivated practically, as neuromorphic computing systems with 100 million spiking neurons are available, and systems with a billion neurons are anticipated in the next few years. Beyond massive parallelism and scalability, neuromorphic computing systems offer energy consumption orders of magnitude lower than conventional high-performance computing systems. We employ our framework to design and analyze new spiking algorithms for shortest path and dynamic programming problems. Our neuromorphic algorithms are message-passing algorithms relying critically on data movement for computation. For fair and rigorous comparison with conventional algorithms and architectures, which is challenging but paramount, we develop new models of data-movement in conventional computing architectures. This allows us to prove polynomial-factor advantages, even when we assume a SNN consisting of a simple grid-like network of neurons. To the best of our knowledge, this is one of the first examples of a rigorous asymptotic computational advantage for neuromorphic computing.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

Integration Of PanDA Workload Management System With Supercomputers for ATLAS and Data Intensive Science

The.LHC, operating at CERN, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe. ATLAS, one of the largest collaborations ever assembled in the sciences, is at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, the ATLAS experiment is relying on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. While PanDA currently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, LHC data taking runs require more resources than grid can possibly provide. To alleviate these challenges, LHC experiments are engaged in an ambitious program to expand the current computing model to include additional resources such as the opportunistic use of supercomputers. We will describe a project aimed at integration of PanDA WMS with supercomputers in United States, in particular with Titan supercomputer at Oak Ridge Leadership Computing Facility. Current approach utilizes modified PanDA pilot framework for job submission to the supercomputers batch queues and local data management, with light-weight MPI wrappers to run single threaded workloads in parallel on LCFs multi-core worker nodes. This implementation was tested with a variety of Monte-Carlo workloads on several supercomputing platforms for ALICE and ATLAS experiments and it is in full pro duction for the ATLAS since September 2015. We will present our current accomplishments with running PanDA at supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications, such as bioinformatics and astro-particle physics.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Models

Recent progress in scientific machine learning (SciML) has opened up the possibility of training novel neural network architectures that solve complex partial differential equations (PDEs). Several (nearly data free) approaches have been recently reported that successfully solve PDEs, with examples including deep feed forward networks, generative networks, and deep encoder-decoder networks. However, practical adoption of these approaches is limited by the difficulty in training these models, especially to make predictions at large output resolutions (≥1024×1024). Here we report on a software framework for data parallel distributed deep learning that resolves the twin challenges of training these large SciML models - training in reasonable time as well as distributing the storage requirements. Our framework provides several out of the box functionality including (a) loss integrity independent of number of processes, (b) synchronized batch normalization, and (c) distributed higher-order optimization methods. We show excellent scalability of this framework on both cloud as well as HPC clusters, and report on the interplay between bandwidth, network topology and bare metal vs cloud. We deploy this approach to train generative models of sizes hitherto not possible, showing that neural PDE solvers can be viably trained for practical applications. We also demonstrate that distributed higher-order optimization methods are 2-3× faster than stochastic gradient-based methods and provide minimal convergence drift with higher batch-size.

PDEs↗

DOE OSTI2021

A scalable exponential-DG approach for nonlinear conservation laws: With application to Burger and Euler equations

In this work, we propose an Exponential DG framework for partial differential equations. We decompose 7 governing equations into linear and nonlinear parts to which we apply the discontinuous Galerkin 8 (DG) spatial discretization. In particular, we construct the linear part using Jacobian that effectively 9 capture stiff characteristics in the system. The former is integrated analytically, whereas the latter 10 is approximated. This approach i) is stable with a large Courant number (Cr > 1); ii) supports 11 high-order solutions both in time and space; iii) is computationally favorable compared to IMEX 12 DG methods with no preconditioner; iv) becomes comparable to explicit RKDG methods on uniform 13 mesh and beneficial on non-uniform grid for Euler equations; v) is scalable in a modern massively 14 parallel computing architecture due to its explicit nature of exponential time integrators and com15 pact communication stencil of DG method. Numerical results demonstrate the performance of our 16 proposed methods through various examples. We also discuss the stability and convergence analysis 17 for our exponential DG scheme in the context of Burgers equation.

42 ENGINEERING↗

DOE OSTI2018

Coping with silent and fail-stop errors at scale by combining replication and checkpointing

This paper provides a model and an analytical study of replication as a technique to cope with silent errors, as well as a mixture of both silent and fail-stop errors on large-scale platforms. Compared with fail-stop errors that are immediately detected when they occur, silent errors require a detection mechanism. To detect silent errors, many application-specific techniques are available, either based on algorithms (e.g., ABFT), invariant preservation or data analytics, but replication remains the most transparent and least intrusive technique. We explore the right level (duplication, triplication or more) of replication for two frameworks: (i) when the platform is subject to only silent errors, and (ii) when the platform is subject to both silent and fail-stop errors. A higher level of replication is more expensive in terms of resource usage but enables to tolerate more errors and to even correct some errors, hence there is a trade-off to be found. Replication is combined with checkpointing and comes with two flavors: process replication and group replication. Process replication applies to message-passing applications with communicating processes. Each process is replicated, and the platform is composed of process pairs, or triplets. Group replication applies to black-box applications, whose parallel execution is replicated several times. The platform is partitioned into two halves (or three thirds). In both scenarios, results are compared before each checkpoint, which is taken only when both results (duplication) or two out of three results (triplication) coincide. Otherwise, one or more silent errors have been detected, and the application rolls back to the last checkpoint, as well as when fail-stop errors have struck. We provide a detailed analytical study for all of these scenarios, with formulas to decide, for each scenario, the optimal parameters as a function of the error rate, checkpoint cost, and platform size. We also report a set of extensive simulation results that nicely corroborates the analytical model.

Checkpointing↗

DOE OSTI2026

Draft ASME Code Case to qualify L-PBF 316H material for Section III, Division 5 applications

This report documents the AMMT program’s development and submission of a draft ASME Code Case to qualify Laser Powder Bed Fusion (L PBF) Type 316H stainless steel for Section III, Divi-sion 5 Class A and SM high temperature nuclear applications. It summarizes the technical basis, the comprehensive high temperature mechanical test database assembled between 2023–2026, and the proposed code language and qualification framework submitted to ASME. The work was co-ordinated across multiple national laboratories and leverages prior ASME efforts to integrate additive manufacturing into the Boiler & Pressure Vessel Code. The body of the report describes the experimental database and analysis supporting the Code Case: tensile, creep, fatigue, creep fatigue, and thermal aging tests collected from multiple additive manufacturing sites, machine types, and powder lots, with material processed by a solution anneal heat treatment. The dataset — including both full size and subsized specimens and tests oriented parallel and perpendicular to build direction — shows limited tensile anisotropy, tensile properties comparable to wrought 316H, creep strength within the scatter of wrought material, but markedly reduced creep ductility above about 650 °C associated with rapid σ phase formation in L PBF microstructures. The draft Code Case itself prescribes a staged qualification model (manufacturing process qualification, component qualification, and per build witness testing), treats L PBF components as equivalent to Type 316 weld metal for design and inspection, and requires mechanical, chemical, and metallographic controls tied to ASTM/ISO 52946. Key acceptance criteria include tensile tests within a 90% prediction interval of the AMMT dataset, a creep fatigue screening test adapted from ASME Section III, Division 5, Subsection HB, HBB 2800 but with the cycle acceptance reduced to 100 for L PBF material, and double volumetric inspection of production components. The report concludes that the present data support treating L PBF 316H as analogous to conventional fusion weld metal for Division 5 design and inspection, while highlighting important caveats: the σ phase driven loss of creep ductility above ~650 °C, preliminary indications of enhanced creep fatigue sensitivity in some lots, and remaining gaps in long term aging and additional cyclic testing. Recommended next actions include completing outstanding cyclic and long duration creep/aging tests on the solution annealed condition, supporting inclusion of the 316H chemistry and heat treatment in ASTM/ISO 52946, and continuing engagement with ASME and NRC during balloting and review to enable industry adoption.

Messner, Mark C. (ORCID:0000000200404385)↗

NASA NTRS1994

An HP Adaptive Discontinuous Galerkin Method for Hyperbolic Conservation Laws

This dissertation addresses various issues for model classes of hyperbolic conservation laws. The basic approach developed in this work employs a new family of adaptive, hp-version, finite element methods based on a special discontinuous Galerkin formulation for hyperbolic problems. The discontinuous Galerkin formulation admits high-order local approximations on domains of quite general geometry, while providing a natural framework for finite element approximations and for theoretical developments. The use of hp-versions of the finite element method makes possible exponentially convergent schemes with very high accuracies in certain cases; the use of adaptive hp-schemes allows h-refinement in regions of low regularity and p-enrichment to deliver high accuracy, while keeping problem sizes manageable and dramatically smaller than many conventional approaches. The use of discontinuous Galerkin methods is uncommon in applications, but the methods rest on a reasonable mathematical basis for low-order cases and has local approximation features that can be exploited to produce very efficient schemes, especially in a parallel, multiprocessor environment. The place of this work is to first and primarily focus on a model class of linear hyperbolic conservation laws for which concrete mathematical results, methodologies, error estimates, convergence criteria, and parallel adaptive strategies can be developed, and to then briefly explore some extensions to more general cases. Next, we provide preliminaries to the study and a review of some aspects of the theory of hyperbolic conservation laws. We also provide a review of relevant literature on this subject and on the numerical analysis of these types of problems.

Bey, Kim S.↗

DOE OSTI2017

A performance spectrum for parallel computational frameworks that solve PDEs

Summary Important computational physics problems are often large‐scale in nature, and it is highly desirable to have robust and high performing computational frameworks that can quickly address these problems. However, it is no trivial task to determine whether a computational framework is performing efficiently or is scalable. The aim of this paper is to present various strategies for better understanding the performance of any parallel computational frameworks for solving PDEs. Important performance issues that negatively impact time‐to‐solution are discussed, and we propose a performance spectrum analysis that can enhance one's understanding of critical aforementioned performance issues. As proof of concept, we examine commonly used finite element simulation packages and software and apply the performance spectrum to quickly analyze the performance and scalability across various hardware platforms, software implementations, and numerical discretizations. It is shown that the proposed performance spectrum is a versatile performance model that is not only extendable to more complex PDEs such as hydrostatic ice sheet flow equations but also useful for understanding hardware performance in a massively parallel computing environment. Potential applications and future extensions of this work are also discussed.

Chang, J.↗

DOE OSTI2024

Part-scale microstructure prediction for laser powder bed fusion Ti-6Al-4V using a hybrid mechanistic and machine learning model

Laser powder bed fusion (LPBF) Ti-6Al-4V is widely studied for use in structural applications in aerospace and medical industries, but mechanical anisotropy and microstructural inhomogeneity prohibits its wider adoption. Although successful microstructure prediction models have been developed, a remaining challenge is their limited integration across length/time scales and validation by experimental studies. Here, this work proposes a physics-augmented machine learning surrogate model to unite predictions of LPBF temperature, β phase morphology and texture, and α/α’ formation into a single framework that is calibrated and validated with experiments. First, a phase field (PF) model of the martensitic β→α’ transformation is developed and calibrated using data from in-situ synchrotron cyclic heating/cooling studies quantifying the variation of α phase fraction with time. In parallel, an established finite difference-Monte Carlo (FDMC) model predicts the part-scale temperature profile and β grain formation during solidification. A dataset is developed using LPBF cyclic temperature descriptors from the FDMC model as inputs and corresponding α/α’ phase fraction and width from the PF model as outputs. Five machine learning (ML) regression models are tested and optimized, having mean absolute error in testing ≤ 4 %, and the k-nearest neighbors (KNN) model is selected as the best performing. The KNN model is called at the nodal level during post-processing of the FDMC model to replace and downscale the response of the PF model. The combined agility and accuracy of the hybrid FDMC-ML model enables part-scale microstructure predictions that can be further used for property predictions to accelerate AM process optimization.

36 MATERIALS SCIENCE↗

DOE OSTI2017

Deep Packet/Flow Analysis using GPUs

Deep packet inspection (DPI) faces severe performance challenges in high-speed networks (40/100 GE) as it requires a large amount of raw computing power and high I/O throughputs. Recently, researchers have tentatively used GPUs to address the above issues and boost the performance of DPI. Typically, DPI applications involve highly complex operations in both per-packet and per-flow data level, often in real-time. The parallel architecture of GPUs fits exceptionally well for per-packet network traffic processing. However, for stateful network protocols such as TCP, their data stream need to be reconstructed in a per-flow level to deliver a consistent content analysis. Since the flow-centric operations are naturally antiparallel and often require large memory space for buffering out-of-sequence packets, they can be problematic for GPUs, whose memory is normally limited to several gigabytes. In this work, we present a highly efficient GPU-based deep packet/flow analysis framework. The proposed design includes a purely GPU-implemented flow tracking and TCP stream reassembly. Instead of buffering and waiting for TCP packets to become in sequence, our framework process the packets in batch and uses a deterministic finite automaton (DFA) with prefix-/suffix- tree method to detect patterns across out-of-sequence packets that happen to be located in different batches. In conclusion, evaluation shows that our code can reassemble and forward tens of millions of packets per second and conduct a stateful signature-based deep packet inspection at 55 Gbit/s using an NVIDIA K40 GPU.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Generalizable coordination of large multiscale workflows: challenges and learnings at scale

The advancement of machine learning techniques and the heterogeneous architectures of most current supercomputers are propelling the demand for large multiscale simulations that can automatically and autonomously couple diverse components and map them to relevant resources to solve complex problems at multiple scales. Nevertheless, despite the recent progress in workflow technologies, current capabilities are limited to coupling two scales. In the first-ever demonstration of using three scales of resolution, we present a scalable and generalizable framework that couples pairs of models using machine learning and in situ feedback. We expand upon the massively parallel Multiscale Machine-Learned Modeling Infrastructure (MuMMI), a recent, award-winning workflow, and generalize the framework beyond its original design. We discuss the challenges and learnings in executing a massive multiscale simulation campaign that utilized over 600,000 node hours on Summit and achieved more than 98% GPU occupancy for more than 83% of the time. We present innovations to enable several orders of magnitude scaling, including simultaneously coordinating 24,000 jobs, and managing several TBs of new data per day and over a billion files in total. Finally, we describe the generalizability of our framework and, with an upcoming open-source release, discuss how the presented framework may be used for new applications.

Bhatia, Harsh↗

DOE OSTI2020

Automated pipeline framework for processing of large-scale building energy time series data

Commercial buildings account for one third of the total electricity consumption in the United States and a significant amount of this energy is wasted. Therefore, there is a need for “virtual” energy audits, to identify energy inefficiencies and their associated savings opportunities using methods that can be non-intrusive and automated for application to large populations of buildings. Here we demonstrate virtual energy audits applied to large populations of buildings’ time-series smart-meter data using a systematic approach and a fully automated Building Energy Analytics (BEA) Pipeline that unifies, cleans, stores and analyzes building energy datasets in a non-relational data warehouse for efficient insights and results. This BEA pipeline is based on a custom compute job scheduler for a high performance computing cluster to enable parallel processing of Slurm jobs. Within the analytics pipeline, we introduced a data qualification tool that enhances data quality by fixing common errors, while also detecting abnormalities in a building’s daily operation using hierarchical clustering. We analyze the HVAC scheduling of a population of 816 buildings, using this analytics pipeline, as part of a cross-sectional study. With our approach, this sample of 816 buildings is improved in data quality and is efficiently analyzed in 34 minutes, which is 85 times faster than the time taken by a sequential processing. The analytical results for the HVAC operational hours of these buildings show that among 10 building use types, food sales buildings with 17.75 hours of daily HVAC cooling operation are decent targets for HVAC savings. Overall, this analytics pipeline enables the identification of statistically significant results from population based studies of large numbers of building energy time-series datasets with robust results. These types of BEA studies can explore numerous factors impacting building energy efficiency and virtual building energy audits. This approach enables a new generation of data-driven buildings energy analysis at scale.

36 MATERIALS SCIENCE↗