Search NASA⌕ Search

SEARCH · Search NASA

Results for “scalable computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,441 records · Page 80
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Deploying and Operating CephFS for Scientific Applications at Fermilab

Fermilab has been running a Ceph cluster in production for several years to support high-throughput scientific computing. Our primary use case is CephFS, which serves interactive data analysis workloads, with growing interest in using RGW for scalable object storage of scientific datasets. In this talk, we'll share lessons learned from successfully deploying and maintaining our Ceph cluster with cephadm, including challenges faced, performance tuning, and operational practices. We'll also present custom tools we've developed to streamline monitoring and management and discuss how Ceph fits into our broader storage architecture for large-scale scientific research.

Peisker, Alison [Fermilab]↗
DOE OSTI2024

A robust approach to Gaussian process implementation

Abstract. Gaussian process (GP) regression is a flexible modeling technique used to predict outputs and to capture uncertainty in the predictions. However, the GP regression process becomes computationally intensive when the training spatial dataset has a large number of observations. To address this challenge, we introduce a scalable GP algorithm, termed MuyGPs, which incorporates nearest-neighbor and leave-one-out cross-validation during training. This approach enables the evaluation of large spatial datasets with state-of-the-art accuracy and speed in certain spatial problems. Despite these advantages, conventional quadratic loss functions used in the MuyGPs optimization, such as root mean squared error (RMSE), are highly influenced by outliers. We explore the behavior of MuyGPs in cases involving outlying observations and, subsequently, develop a robust approach to handle and mitigate their impact. Specifically, we introduce a novel leave-one-out loss function based on the pseudo-Huber function (LOOPH) that effectively accounts for outliers in large spatial datasets within the MuyGPs framework. Our simulation study shows that the LOOPH loss method maintains accuracy despite outlying observations, establishing MuyGPs as a powerful tool for mitigating unusual observation impacts in the large data regime. In the analysis of US ozone data, MuyGPs provides accurate predictions and uncertainty quantification, demonstrating its utility in managing data anomalies. Through these efforts, we advance the understanding of GP regression in spatial contexts.

Mukangango, Juliette↗
NASA NTRS1988

A case for Redundant Arrays of Inexpensive Disks (RAID)

Increasing performance of CPUs and memories will be squandered if not matched by a similar performance increase in I/O. While the capacity of Single Large Expensive Disks (SLED) has grown rapidly, the performance improvement of SLED has been modest. Redundant Arrays of Inexpensive Disks (RAID), based on the magnetic disk technology developed for personal computers, offers an attractive alternative to SLED, promising improvements of an order of magnitude in performance, reliability, power consumption, and scalability. This paper introduces five levels of RAIDs, giving their relative cost/performance, and compares RAID to an IBM 3380 and a Fujitsu Super Eagle.

Patterson, David A.↗
NASA NTRS2000

Implementation of BT, SP, LU, and FT of NAS Parallel Benchmarks in Java

A number of Java features make it an attractive but a debatable choice for High Performance Computing. We have implemented benchmarks working on single structured grid BT,SP,LU and FT in Java. The performance and scalability of the Java code shows that a significant improvement in Java compiler technology and in Java thread implementation are necessary for Java to compete with Fortran in HPC applications.

Schultz, Matthew↗
NASA NTRS2000

SUPREM-DSMC: A New Scalable, Parallel, Reacting, Multidimensional Direct Simulation Monte Carlo Flow Code

An AFRL/NRL team has recently been selected to develop a scalable, parallel, reacting, multidimensional (SUPREM) Direct Simulation Monte Carlo (DSMC) code for the DoD user community under the High Performance Computing Modernization Office (HPCMO) Common High Performance Computing Software Support Initiative (CHSSI). This paper will introduce the JANNAF Exhaust Plume community to this three-year development effort and present the overall goals, schedule, and current status of this new code.

Campbell, David↗
DOE OSTI2021

Scalable Approaches to Selecting Key Entities in Large Networked Infrastructure Systems

This work aims at bringing advances in discrete optimization algorithms to solving practical engineering problems at scale. Often times, in many engineering design problems, there is a need to select a small set of influential or representative elements from a large ground set of entities in an optimal fashion. Submodular optimization provides for a formal way to solve such problems. Common examples with infrastructure systems involve sensor placement and identification of key entities with certain objectives. However, scaling these approaches to large infrastructure systems can be challenging because of the high computational complexity of the overall framework that include the optimization algorithms as well as high-complexity compute-oracles that provide the necessary objective function values. In this work, we explore a well-studied and widely-applicable paradigm, namely leader-selection in a multi-agent networked setting in the context of scalable methodologies. We demonstrate novel frameworks that utilize variations of accelerated submodular optimization algorithms along with linear-algebraic methods that can help accelerate the oracle computations. We further explore this combination in conjunction with graph partitioning paradigms to take advantage of the accelerated algorithms in a distributed setting. Finally we demonstrate the key findings on a practical problem in an operational setting. For this, we leverage an example road network with approximately 18k nodes and 27k edges in a traffic control application, where we seek a limited number of k=200 key intersections. This problem can be solved in a serial setting in just under 5 hours providing more than 2 orders of magnitude speed-up over methods that do not consider acceleration techniques.

Visweswara Sathanur, Arun↗
DOE OSTI2017

An improved hyperbolic embedding algorithm

Because hyperbolic space has properties that make it amenable to graph representations, there is significant interest in scalable hyperbolic-space embedding methods. These embeddings enable constant-time approximation of shortest-path distances, and so are significantly more efficient than full shortest-path computations. In this article, we improve on existing landmark-based hyperbolic embedding algorithms for large-scale graphs. Whereas previous methods compute the embedding by using the derivative-free Nelder–Mead simplex optimization method, our approach uses the limited-memory BFGS (LBFGS) method, which is quasi-Newton optimization, with analytic gradients. Our method is not only significantly faster but also produces higher-quality embeddings. Moreover, we are able to include the hyperbolic curvature as a variable in the optimization. We compare our hyperbolic embedding method implementation in Python (called Hypy) against the best publicly available software, Rigel. Our method is an order of magnitude faster and shows significant improvements in the accuracy of the shortest-path distance calculations. Furthermore, tests are performed on a variety of real-world networks, and we show the scalability of our method by embedding a graph with 1.8 billion edges and 65 million nodes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Chapter 13. Exploring Use of the Reserved Core

In this chapter, we illustrate benefits of thinking in terms of thread management techniques when using a centralized scheduler model along with interoperability of MPI and PThread. This is facilitated through an exploration of thread placement strategies for an algorithm modeling radiative heat transfer with special attention to the 61st core. This algorithm plays a key role within the Uintah Computational Framework (UCF) and current efforts taking place at the University of Utah to model next-generation, large-scale clean coal boilers. In such simulations, this algorithm models the dominant form of heat transfer and consumes a large portion of compute time. Exemplified by a real-world example, this chapter presents our early efforts in porting a key portion of a scalability-centric codebase to the Intel Xeon Phi coprocessor. Specifically, this chapter presents results from our experiments profiling the native execution of a reverse Monte-Carlo ray tracing-based radiation model on a single coprocessor. These results demonstrate that our fastest run configurations utilized the 61st core and that performance was not profoundly impacted when explicitly oversubscribing the coprocessor operating system thread. Additionally, this chapter presents a portion of radiation model source code, a MIC-centric UCF cross-compilation example, and less conventional thread management technique for developers utilizing the PThreads threading model.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

CoSim: A Simulator for Co-Scheduling of Batch and On-Demand Jobs in HPC Datacenters

The increasing scale and complexity of scientific applications are rapidly transforming the ecosystem of tools, methods, and workflows adopted by the high-performance computing (HPC) community. Big data analytics and deep learning are gaining traction as essential components in this ecosystem in a variety of scenarios, such as, steering of experimental instruments , acceleration of high-fidelity simulations through surrogate computations, and guided ensemble searches. In this context, the batch job model traditionally adopted by the supercomputing in-frastructures needs to be complemented with support to schedule opportunistic on-demand analytics jobs, leading to the problem of efficient preemption of batch jobs with minimum loss of progress. In this paper, we design and implement a simulator, CoSim, that enables on-the-fly analysis of the trade-offs arising between delaying the start of opportunistic on-demand jobs, which leads to longer analytics latency, and loss of progress due to preemption of batch jobs, which is necessary to make room for on-demand jobs. To this end, we propose an algorithm based on dynamic programming with predictable performance and scalability that enables supercomputing infrastructure schedulers to analyze the aforementioned trade-off and take decisions in near real-time. Compared with other state-of-art approaches using traces of the Theta pre-Exascale machine, our approach is capable of finding the optimal solution, while achieving high performance and scalability.

batch job preemption↗
DOE OSTI2015

Genetic algorithm based task reordering to improve the performance of batch scheduled massively parallel scientific applications

The growth in size of networked high performance computers along with novel accelerator-based node architectures has further emphasized the importance of communication efficiency in high performance computing. The world's largest high performance computers are usually operated as shared user facilities due to the costs of acquisition and operation. Applications are scheduled for execution in a shared environment and are placed on nodes that are not necessarily contiguous on the interconnect. Furthermore, the placement of tasks on the nodes allocated by the scheduler is sub-optimal, leading to performance loss and variability. Here, we investigate the impact of task placement on the performance of two massively parallel application codes on the Titan supercomputer, a turbulent combustion flow solver (S3D) and a molecular dynamics code (LAMMPS). Benchmark studies show a significant deviation from ideal weak scaling and variability in performance. The inter-task communication distance was determined to be one of the significant contributors to the performance degradation and variability. A genetic algorithm-based parallel optimization technique was used to optimize the task ordering. This technique provides an improved placement of the tasks on the nodes, taking into account the application's communication topology and the system interconnect topology. As a result, application benchmarks after task reordering through genetic algorithm show a significant improvement in performance and reduction in variability, therefore enabling the applications to achieve better time to solution and scalability on Titan during production.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Optimal design of acoustic metamaterial cloaks under uncertainty

In this work, we consider the problem of optimal design of an acoustic cloak under uncertainty and develop scalable approximation and optimization methods to solve this problem. The design variable is taken as an infinite-dimensional spatially-varying field that represents the material property, while an additive infinite-dimensional random field represents the variability of the material property or the manufacturing error. Discretization of this optimal design problem results in high-dimensional design variables and uncertain parameters. To solve this problem, we develop a computational approach based on a Taylor approximation and an approximate Newton method for optimization, which is based on a Hessian derived at the mean of the random field. We show our approach is scalable with respect to the dimension of both the design variables and uncertain parameters, in the sense that the necessary number of acoustic wave propagations is essentially independent of these dimensions, for numerical experiments with up to one million design variables and half a million uncertain parameters. Additionally, we demonstrate that, using our computational approach, an optimal design of the acoustic cloak that is robust to material uncertainty is achieved in a tractable manner. The optimal design under uncertainty problem is posed and solved for the classical circular obstacle surrounded by a ring-shaped cloaking region, subjected to both a single-direction single-frequency incident wave and multiple-direction multiple-frequency incident waves. Finally, we apply the method to a deterministic large-scale optimal cloaking problem with complex geometry, to demonstrate that the approximate Newton method’s Hessian computation is viable for large, complex problems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

UAS Edge Computing of Energy Infrastructure Damage Assessment

Energy infrastructure assessments are needed within 72 hours of natural disasters, and previous data collection methods have proven too slow. We demonstrate a scalable end-to-end solution using a prototype unmanned aerial system that performs on-the-edge detection, classification (i.e., damaged or undamaged), and geo-location of utility poles. The prototype is suitable for disaster response because it requires no local communication infrastructure and is capable of autonomous missions. Collections before, during, and after Hurricane Ida in 2021 were used to test the system. The system delivered an F1 score of 0.65 operating with a 2.7 s/frame processing speed with the YOLOv5 large model and an F1 score of 0.55 with a 0.48 s/frame with the YOLOv5 small model. Geo-location uncertainty in the bottom half of the frame was ~8 m, mostly driven by error in camera pointing measurement. With additional training data to improve performance and detect additional types of features, a fleet of similar drones could autonomously collect actionable post-disaster data.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2016

Logic circuit prototypes for three-terminal magnetic tunnel junctions with mobile domain walls

Spintronic computing promises superior energy efficiency and nonvolatility compared to conventional field-effect transistor logic. But, it has proven difficult to realize spintronic circuits with a versatile, scalable device design that is adaptable to emerging material physics. Here we present prototypes of a logic device that encode information in the position of a magnetic domain wall in a ferromagnetic wire. We show that a single three-terminal device can perform inverter and buffer operations. We demonstrate one device can drive two subsequent gates and logic propagation in a circuit of three inverters. This prototype demonstration shows that magnetic domain wall logic devices have the necessary characteristics for future computing, including nonlinearity, gain, cascadability, and room temperature operation.

42 ENGINEERING↗
DOE OSTI2019

Scalable Topological Data Analysis and Visualization for Evaluating Data-Driven Models in Scientific Applications

With the rapid adoption of machine learning techniques for large-scale applications in science and engineering comes the convergence of two grand challenges in visualization. First, the utilization of black box models (e.g., deep neural networks) calls for advanced techniques in exploring and interpreting model behaviors. Second, the rapid growth in computing has produced enormous datasets that require techniques that can handle millions or more samples. Although some solutions to these interpretability challenges have been proposed, they typically do not scale beyond thousands of samples, nor do they provide the high-level intuition scientists are looking for. Here, we present the first scalable solution to explore and analyze high-dimensional functions often encountered in the scientific data analysis pipeline. By combining a new streaming neighborhood graph construction, the corresponding topology computation, and a novel data aggregation scheme, namely topology aware datacubes, we enable interactive exploration of both the topological and the geometric aspect of high-dimensional data. Following two use cases from high-energy-density (HED) physics and computational biology, we demonstrate how these capabilities have led to crucial new insights in both applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Parallel-in-Time Multigrid Methods for Hyperbolic Problems, with a Focus on the Shallow Water Equations

The coming massive parallelism of exascale computing presents a pressing challenge for the many DOE simulations of time-dependent partial differential equations, which typically use traditional sequential time stepping methods. Since this traditional approach is inherently serial, it presents a sequential bottleneck when moving to exascale computing, because future performance gains will come through greater concurrency, not faster clock speeds. Thus, the goal of this work is to research parallelism in time, i.e., methods that compute multiple time values simultaneously, not sequentially. The focus will be on hyperbolic problems of programmatic interest to DOE, with the goal of enabling scalable simulations of time-dependent hyperbolic problems on future architectures. The difficulty lies in the fact that hyperbolic problems are well-known to be difficult for parallel-in-time methods, with the most common parallel-in-time method, Parareal, diverging in many cases. Here, the chosen methodology for scalably solving these hyperbolic space-time equations is multigrid-reductionin-time (MGRIT), because multigrid (when it works) is a powerful, optimal, and scalable solver for discretized PDEs. To further research in this area, this project will explore a model hyperbolic problem (the shallow water equations) in the context of recent advances (e.g., by Wingate and Haut) in constructing improved coarse time-grid time-propagators by using the slow asymptotic structure of the equations. In particular, we will investigate if such coarse time-propagators offer significant advantages in an MGRIT setting, and explore any broader insights gained into parallel-in-time for hyperbolic problems.

97 MATHEMATICS AND COMPUTING↗