Search NASA⌕ Search

SEARCH · Search NASA

Results for “high performance and distributed computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

PROTEUS: Machine Learning Driven Resilience for Extreme-scale Systems

The objective of this project is to design, develop, and evaluate scalable software to enhance resilience, data checkpointing, program restart, and analysis. The proposed tasks are to 1) develop scalable machine learning techniques to learn temporal change patterns in a scalable and in-situ manner, and to minimize data movement and maximize learning locally closest to data; 2) design a concise data representation and indexing mechanism to capture the distribution of changes in data that can guarantee point-wise user-defined tolerable errors while reducing the data storage requirements by an order of magnitude or more; 3) develop data reduction techniques as library modules; 4) exploit local SSD for minimizing data movement in storage hierarchy; 5) develop anomaly detection algorithms that can predict corruptions based on learning of emerging patterns; 6) develop software libraries to be incorporated within widely used data formats and APIs; and 7) evaluate the proposed software using DOE scientific applications. The outcomes of the proposed work are to satisfy many synergistic data reduction and resilience requirements for large-scale data intensive applications executed on extreme-scale computing systems. The developed mechanism for error-bound data approximation is directly applicable to existing scientific applications. Through machine learning from historical events and change distribution, this work will enable anomaly detection for DOE computer facility.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2013

Mr. Scan: extreme scale density-based clustering using a tree-based network of GPGPU nodes, In: SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

Density-based clustering algorithms are a widely-used class of data mining techniques that can find irregularly shaped clusters and cluster data without prior knowledge of the number of clusters it contains. DBSCAN is the most well-known density-based clustering algorithm. We introduce our version of DBSCAN, called Mr. Scan, which uses a hybrid parallel implementation that combines the MRNet tree-based distribution network with GPGPU-equipped nodes. Mr. Scan avoids the problems of existing implementations by effectively partitioning the point space and by optimizing DBSCAN's computation over dense data regions. We tested Mr. Scan on both a geolocated Twitter dataset and image data obtained from the Sloan Digital Sky Survey. At its largest scale, Mr. Scan clustered 6.5 billion points from the Twitter dataset on 8,192 GPU nodes on Cray Titan in 17.3 minutes. All other parallel DBSCAN implementations have only demonstrated the ability to cluster up to 100 million points.

Computer Science↗
DOE OSTI2016

An Efficient Multicore Implementation of a Novel HSS-Structured Multifrontal Solver Using Randomized Sampling

Here, we present a sparse linear system solver that is based on a multifrontal variant of Gaussian elimination and exploits low-rank approximation of the resulting dense frontal matrices. We use hierarchically semiseparable (HSS) matrices, which have low-rank off-diagonal blocks, to approximate the frontal matrices. For HSS matrix construction, a randomized sampling algorithm is used together with interpolative decompositions. The combination of the randomized compression with a fast ULV HSS factoriz ation leads to a solver with lower computational complexity than the standard multifrontal method for many applications, resulting in speedups up to 7 fold for problems in our test suite. The implementation targets many-core systems by using task parallelism with dynamic runtime scheduling. Numerical experiments show performance improvements over state-of-the-art sparse direct solvers. The implementation achieves high performance and good scalability on a range of modern shared memory parallel systems, including the Intel Xeon Phi (MIC). The code is part of a software package called STRUMPACK - STRUctured Matrices PACKage, which also has a distributed memory component for dense rank-structured matrices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Meso-scale microstructural agglomerate quantification in boron carbide using X-ray microcomputed tomography

Highlights: • First known quantification of agglomerate defect distributions comprising of carbonaceous and aluminum nitride phases B{sub 4}C. • In-depth statistical analyses of the size, orientation, and morphology of the defects. • The determination of directional dependence of the hot-pressing direction on the agglomerate distributions. - Abstract: Failure in brittle materials is governed, in part, by the specific microstructure of that material, with heterogeneous microstructures generally leading to reduced mechanical properties and performance. In ceramics used for armor, an increase in compressive strength can be an indicator of improved ballistic performance. In hot-pressed boron carbide (B{sub 4}C) ceramic plates, failure has been attributed to carbonaceous flakes observed on the fracture surfaces of failed specimens. However, this current work shows agglomerates, in addition to the carbonaceous flakes, are present in this boron carbide. The density and distribution of these agglomerates are believed to influence the ballistic performance. Therefore, it is of interest to improve the quantification of meso-scale microstructural agglomerates so that strategies can be developed to mitigate them. This work is focused on the microstructural characterization of dumbbell-shaped compression specimens that were machined from a single tile of pressure-aided densification (PAD) B{sub 4}C, using high-resolution micro-computed tomography (microCT). Large clusters of non-B{sub 4}C agglomerates, up to 125 μm in diameter, were observed in addition to the typical smaller carbonaceous flakes and aluminum-based phases. Agglomerate size distribution, orientation, morphology, and spacing were quantified both parallel and perpendicular to the pressing direction. This defect quantification provides valuable information for a better understanding of the material microstructure which is a critical step in improving armor materials for the future.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Towards QCD-assisted hydrodynamics for heavy-ion collision phenomenology

Heavy-ion collisions are well described by a dynamical evolution with a long hydrodynamical phase. In this phase the properties of the strongly coupled quark-gluon plasma are reflected in the equation of state (EoS) and the transport coefficients, most prominently by the shear and bulk viscosity over entropy density ratios η/s( T ) and ς/s( T ), respectively. While the EoS is by now known to a high accuracy, the transport coefficients and in particular their temperature and density dependence are not well known from first-principle computations yet, as well as the possible influence they can have once used in hydrodynamical simulations. In this work, the most recent QCD-based parameters are provided as input to the MUSIC framework. A ratio η/s( T ) computed with a QCD based approach is used for the first time [1, 2]. The IP-Glasma model is used to describe the initial energy density distribution, and UrQMD for the dilute hadronic phase. Simulations are performed for Pb–Pb collisions at √ s NN = 2.76 TeV, for different centrality intervals. The resulting kinematic distributions of the particles produced in the collisions are compared to data from the LHC, for several experimental observables. Furthermore, the high precision of the experimental results and the broad variety of observables considered allow to critically verify the quality of the description based on first-principle input to the hydrodynamic evolution.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Learning earthquake ground motions via conditional generative modeling

Predicting high-fidelity ground motions for future earthquakes is crucial for seismic hazard assessment and infrastructure resilience. Conventional empirical simulations suffer from sparse sensor distribution and geographically localized earthquake locations, while physics-based methods are computationally intensive and require accurate representations of Earth structures and earthquake sources. We propose an artificial intelligence (AI) spectrogram generator, Conditional Generative Modeling for Ground Motion (CGM-GM). CGM-GM leverages earthquake magnitudes and geographic coordinates of earthquakes and sensors as inputs, when postprocessed with phase information, capturing spatially continuous Fourier amplitude spectra (FAS) as well as properties such as P and S arrivals, and waveform durations, without explicit physics constraints. This is achieved through a probabilistic autoencoder that extracts latent distributions in the time-frequency domain and variational sequential models for prior and posterior distributions. We evaluate the performance of CGM-GM using small-magnitude earthquake records from the San Francisco Bay Area, a region with high seismic risks. Here, we report that CGM-GM demonstrates potential for complementing physics-based simulations and non-ergodic empirical ground motion models, as well as shows promise in seismology and beyond.

geophysics↗
DOE OSTI2023

Engineering Functionalized 2D Metal-Organic Frameworks Nanosheets with Fast Li + Conduction for Advanced Solid Li Batteries

Solid-state batteries can ensure high energy density and safety in lithium metal batteries, while polymer electrolytes are plagued by slow ion kinetics and low selective transport of Li + . Metal-organic frameworks (MOFs) are proposed as emerging fillers for solid-state poly(ethylene oxide)(PEO) electrolytes, however, developing functionalized MOFs and understanding their roles on ion transfer has proven challenging. Herein, combining computational and experimental results, the functional group regulation in MOFs can effectively change surficial charge distribution and limit anion movement is revealed, providing a potential solution to these issues. Specifically, functionalized 2D MOF sheets are designed through molecular engineering to construct high-performance composite electrolytes, where the electron-donating effect of substituents in 2D-MOFs effectively limits the movement of ClO 4 - and promotes mechanical properties and ion migration numbers (0.36 up to 0.64) of PEO. As a result, Li/Li cells with composite electrolyte exhibit superior cyclability for 1000 h at a current density of 0.2 mA cm -2 . Meanwhile, the solid LiFePO 4 /Li battery delivers highly reversible capacities of 148.8 mAh g -1 after 200 cycles. In conclusion, these findings highlight a new approach for anion confinement through the use of functional group electronic effects, leading to enhanced ionic conductivity, and a feasible direction for high-performance solid-state batteries.

25 ENERGY STORAGE↗
DOE OSTI2025

HPC Campaign Management: Remote data access with user-defined error bound using ADIOS and ZFP

Remote access to large-scale scientific datasets, like those generated by combustion simulations or other high-performance computing (HPC) applications, presents a significant challenge. Downloading entire datasets is often impractical due to their size and the bandwidth limitations of typical networks. To address this challenge, we propose a novel approach that enables efficient remote access to large datasets distributed across multiple facilities. Our method enables technologies to download only the data values of a select variable, in a select region of interest, to a user-defined accuracy. For this purpose, we extended the ADIOS IO library to provide read functions with user-defined accuracy, a remote data server that understands multidimensional selections of specific variables, steps and accuracy from an ADIOS dataset, and which uses lossy compression on the remote site to reduce the data to be transferred back to the client. In addition, our extension of the ADIOS library collects metadata from multiple datasets in small files called Campaign Archives, which can be shared among project participants on any HPC, cloud or laptop, and which can easily facilitate the discovery of content and pointers to the data location as well as remote access to the data by local tools as if data was local. This feature called Campaign Management, enables a group of scientists to manage related datasets stored in multiple files, across multiple facilities as if it was in a single file/database. We demonstrate the effectiveness of our approach using a 1.5 TB dataset from the S3D combustion simulation on Frontier at the Oak Ridge Leadership Facility. Even a single variable from this dataset, at 64 GB, is too large to be processed on a standard laptop. We show two different reading patterns for 2D plots and 3D visualization, with careful settings that a scientist studying combustion data would do and show that running the same Python scripts on Frontier directly takes comparable time than running them on the local laptop with remote access to the data on Frontier.

Podhorszki, Norbert [ORNL] (ORCID:000000019647542X↗
DOE OSTI2025

An Image-Plane Approach to Gravitational Lens Modeling of Interferometric Data

Strong gravitational lensing acts as a cosmic telescope, enabling the study of the high-redshift universe. Astronomical interferometers, such as the Atacama Large Millimeter/submillimeter Array (ALMA), have provided high-resolution images of strongly lensed sources at millimeter and submillimeter wavelengths. To model the mass and light distributions of lensing and source galaxies from strongly lensed images, strong lens modeling for interferometric observations is conventionally performed in the visibility space, which is computationally expensive. In this paper, we implement an image-plane lens modeling methodology for interferometric dirty images by accounting for noise correlations. We show that the image-plane likelihood function produces accurate model values when tested on simulated ALMA observations with an ensemble of noise realizations. We also apply our technique to ALMA observations of two sources selected from the South Pole Telescope survey, comparing our results with previous visibility-based models. Our model results are consistent with previous models for both parametric and pixelated source-plane reconstructions. We implement this methodology for interferometric lens modeling in the open-source software package lenstronomy.

Zhang, Nan [Illinois U., Urbana (main)] (ORCID:000↗
DOE OSTI2016

An asynchronous traversal engine for graph-based rich metadata management

Rich metadata in high-performance computing (HPC) systems contains extended information about users, jobs, data files, and their relationships. Property graphs are a promising data model to represent heterogeneous rich metadata flexibly. Specifically, a property graph can use vertices to represent different entities and edges to record the relationships between vertices with unique annotations. The high-volume HPC use case, with millions of entities and relationships, naturally requires an out-of-core distributed property graph database, which must support live updates (to ingest production information in real time), low-latency point queries (for frequent metadata operations such as permission checking), and large-scale traversals (for provenance data mining). Among these needs, large-scale property graph traversals are particularly challenging for distributed graph storage systems. Most existing graph systems implement a "level synchronous" breadth-first search algorithm that relies on global synchronization in each traversal step. This performs well in many problem domains; but a rich metadata management system is characterized by imbalanced graphs, long traversal lengths, and concurrent workloads, each of which has the potential to introduce or exacerbate stragglers (i.e., abnormally slow steps or servers in a graph traversal) that lead to low overall throughput for synchronous traversal algorithms. Previous research indicated that the straggler problem can be mitigated by using asynchronous traversal algorithms, and many graph-processing frameworks have successfully demonstrated this approach. Such systems require the graph to be loaded into a separate batch-processing framework instead of being iteratively accessed, however. In this work, we investigate a general asynchronous graph traversal engine that can operate atop a rich metadata graph in its native format. We outline a traversal-aware query language and key optimizations (traversal-affiliate caching and execution merging) necessary for efficient performance. We further explore the effect of different graph partitioning strategies on the traversal performance for both synchronous and asynchronous traversal engines. Our experiments show that the asynchronous graph traversal engine is more efficient than its synchronous counterpart in the case of HPC rich metadata processing, where more servers are involved and larger traversals are needed. Furthermore, the asynchronous traversal engine is more adaptive to different graph partitioning strategies.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2023

Scalable self attraction and loading calculations for unstructured ocean tide models

Self attraction and earth-loading effects are important for accurately modeling global tides. A common approach of handling this forcing is to expand mass anomalies into spherical harmonics, which are scaled by load Love numbers to account for elastic earth deformation. We investigate two different approaches to perform these calculations for ocean models that employ unstructured meshes and distributed memory parallelization. The first approach leverages a highly efficient spherical harmonics library, but requires all-to-one and one-to-all communications and interpolation operations between the unstructured and a structured mesh. This approach is compared to a parallel algorithm that computes the spherical harmonic transformations directly on the unstructured mesh with an all-reduce communication. Here, our results show that although the unstructured mesh calculations are more expensive, the scalability of the unstructured mesh approach allows for more efficient spherical harmonics transforms for high-resolution meshes and large processor counts. This methodology enables the efficient inclusion of tidal dynamics large-scale Earth system model simulations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Scalable Volume Visualization for Big Scientific Data Modeled by Functional Approximation

Considering the challenges posed by the space and time complexities in handling extensive scientific volumetric data, various data representations have been developed for the analysis of large-scale scientific data. Multivariate functional approximation (MFA) is an innovative data model designed to tackle substantial challenges in scientific data analysis. It computes values and derivatives with high-order accuracy throughout the spatial domain, mitigating artifacts associated with zero- or first-order interpolation. However, the slow query time through MFA makes it less suitable for interactively visualizing a large MFA model. In this work, we develop the first scalable interactive volume visualization pipeline, MFA-DVV, for the MFA model encoded from large-scale datasets. Our method achieves low input latency through distributed architecture, and its performance can be further enhanced by utilizing a compressed MFA model while still maintaining a high-quality rendering result for scientific datasets. We conduct comprehensive experiments to show that MFA-DVV can decrease the input latency and achieve superior visualization results for big scientific data compared with existing approaches.

big scientific dataset↗
DOE OSTI2022

Deconvoluting the benefits of porosity distribution in layered electrodes on the electrochemical performance of Li-ion batteries

Thick Lithium Ion Battery (LIB) electrodes suffer from poor rate capability and high ionic impedance due to their thickness and mesostructure. Therefore, optimizing thick electrode architectures becomes crucial. In this work, we report a systematic assessment of the ionic resistance in heterogeneous porous electrodes through the combination of computational simulations using a 4D-resolved model and experimental measurements. The first part of the study is devoted to a general assessment of Electrochemical Impedance Spectroscopy (EIS) spectra, mapping the impact of ionic and electronic resistances on the overall impedances of uncalendered and calendered LiNi 1/3 Mn 1/3 Co 1/3 O 2 , LiFePO 4 and graphite electrodes. In the second part, in silico-generated electrodes with different porosities are used in computational EIS simulations to analyze the impact of the electrode porosity on the ionic impedance. As expected, the results show that a lower porosity leads to a higher ionic impedance because of a higher electrode tortuosity factor. Furthermore, in silico-generated electrodes with different porosities were stacked and assembled to create heterogeneities of porosity along the thickness, and used in computational EIS and galvanostatic discharge simulations. The computational results show that the porosity heterogeneity along the electrode thickness has a significant effect on the ionic impedance and capacity of the electrode. The electrode architecture with progressively decreasing porosity from separator to current collector shows the highest performance, a trend validated by our in house experimental EIS and galvanostatic discharge also reported in this manuscript. Overall, we conclude that the ionic resistance in a thick electrode can be effectively reduced through proper tuning of the porosity heterogeneity. The proposed heterogeneous electrode architectures presented here could enormously help building efficient thick electrodes for LIBs.

25 ENERGY STORAGE↗
DOE OSTI2021

Neoclassical toroidal viscosity torque prediction via deep learning

GPECnet is a densely connected neural network that has been trained on GPEC data, to predict the plasma stability, neoclassical toroidal viscosity (NTV) torque, and optimized 3D coil current distributions for desired NTV torque profiles. Using NTV torque, driven by non-axisymmetric field perturbations in a tokamak, can be vital in optimizing pedestal performance by controlling the rotation profile in both the core, to ensure tearing stability, and the edge, to avoid edge localized modes (ELMs). The generalized perturbed equilibrium code (GPEC) software package can be used to calculate the plasma stability to 3D perturbations and the NTV torque profile generated by applied 3D magnetic fields. These calculations, however, involve complex integrations over space and energy distributions, which takes time to compute. Initially, GPECnet has been trained solely on data representative of the quiescent H-mode (QH) scenario, in which neutral beams are often balanced and toroidal rotation is low across the plasma profile. Lastly, this work provides the foundation for active control of the rotation shear using a combination of beams and 3D fields for robust and high performance QH mode operation.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Q-Cluster: Quantum Error Mitigation Through Noise-Aware Unsupervised Learning

Quantum error mitigation (QEM) is critical in reducing the impact of noise in the pre-fault-tolerant era, and is expected to complement error correction in fault-tolerant quantum computing (FTQC). In this work, we propose a novel QEM approach, Q-Cluster, that uses unsupervised learning (clustering) to reshape the measured bit-string distribution. Our approach starts with a simplified bit-flip noise model. It first performs clustering on noisy measurement results, i.e., bit-strings, based on the Hamming distance. The centroid of each cluster is calculated using a qubit-wise majority vote. Next, the noisy distribution is adjusted with the clustering outcomes and the bitflip error rates using Bayesian inference. Our simulation results show that Q-Cluster can mitigate high noise rates (up to 40% per qubit) with the simple bit-flip noise model. However, real quantum computers do not fit such a simple noise model. To address the problem, we (a) apply Pauli twirling to tailor the complex noise channels to Pauli errors, and (b) employ a machine learning model, ExtraTrees regressor, to estimate an effective bit-flip error rate using a feature vector consisting of machine calibration data (gate & measurement error rates), circuit features (number of qubits, numbers of different types of gates, etc.) and the shape of the noisy distribution (entropy). Our experimental results show that our proposed Q-Cluster scheme improves the fidelity by a factor of 1.46x, on average, compared to the unmitigated output distribution, for a set of low-entropy benchmarks on five different IBM quantum machines. Our approach outperforms the state-of-art QEM approaches RZNE [28], M3 [24], Hammer [35], and QBEEP [33] by 1.26x,1.29x,1.47x, and 2.65 x, respectively.

42 ENGINEERING↗
DOE OSTI2020

Striped Data Analysis Framework

A columnar data representation is known to be an efficient way for data storage, specifically in cases when the analysis is often done based only on a small fragment of the available data structures. A data representation like Apache Parquet is a step forward from a columnar representation, which splits data horizontally to allow for easy parallelization of data analysis. Based on the general idea of columnar data storage, working on the [LDRD Project], we have developed a striped data representation, which, we believe, is better suited to the needs of High Energy Physics data analysis. A traditional columnar approach allows for efficient data analysis of complex structures. While keeping all the benefits of columnar data representations, the striped mechanism goes further by enabling easy parallelization of computations without requiring special hardware. We will present an implementation and some performance characteristics of such a data representation mechanism using a distributed no-SQL database or a local file system, unified under the same API and data representation model. The representation is efficient and at the same time simple so that it allows for a common data model and APIs for wide range of underlying storage mechanisms such as distributed no-SQL databases and local file systems. Striped storage adopts Numpy arrays as its basic data representation format, which makes it easy and efficient to use in Python applications. The Striped Data Server is a web service, which allows to hide the server implementation details from the end user, easily exposes data to WAN users, and allows to utilize well known and developed data caching solutions to further increase data access efficiency. We are considering the Striped Data Server as the core of an enterprise scale data analysis platform for High Energy Physics and similar areas of data processing. We have been testing this architecture with a 2TB dataset from a CMS dark matter search and plan to expand it to multiple 100 TB or even PB scale. We will present the striped format, Striped Data Server architecture and performance test results.

Gutsche, Oliver↗
DOE OSTI2018

Striped Data Server for Scalable Parallel Data Analysis

A columnar data representation is known to be an efficient way for data storage, specifically in cases when the analysis is often done based only on a small fragment of the available data structures. A data representation like Apache Parquet is a step forward from a columnar representation, which splits data horizontally to allow for easy parallelization of data analysis. Based on the general idea of columnar data storage, working on the [LDRD Project], we have developed a striped data representation, which, we believe, is better suited to the needs of High Energy Physics data analysis. A traditional columnar approach allows for efficient data analysis of complex structures. While keeping all the benefits of columnar data representations, the striped mechanism goes further by enabling easy parallelization of computations without requiring special hardware. We will present an implementation and some performance characteristics of such a data representation mechanism using a distributed no-SQL database or a local file system, unified under the same API and data representation model. The representation is efficient and at the same time simple so that it allows for a common data model and APIs for wide range of underlying storage mechanisms such as distributed no-SQL databases and local file systems. Striped storage adopts Numpy arrays as its basic data representation format, which makes it easy and efficient to use in Python applications. The Striped Data Server is a web service, which allows to hide the server implementation details from the end user, easily exposes data to WAN users, and allows to utilize well known and developed data caching solutions to further increase data access efficiency. We are considering the Striped Data Server as the core of an enterprise scale data analysis platform for High Energy Physics and similar areas of data processing. As a result, we have been testing this architecture with a 2TB dataset from a CMS dark matter search and plan to expand it to multiple 100 TB or even PB scale. We will present the striped format, Striped Data Server architecture and performance test results.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2017

NPTFit: A Code Package for Non-Poissonian Template Fitting

We present NPTFit, an open-source code package, written in Python and Cython, for performing non-Poissonian template fits (NPTFs). The NPTF is a recently developed statistical procedure for characterizing the contribution of unresolved point sources (PSs) to astrophysical data sets. The NPTF was first applied to Fermi gamma-ray data to provide evidence that the excess of ∼GeV gamma-rays observed in the inner regions of the Milky Way likely arises from a population of sub-threshold point sources, and the NPTF has since found additional applications studying sub-threshold extragalactic sources at high Galactic latitudes. The NPTF generalizes traditional astrophysical template fits to allow for the ability to search for populations of unresolved PSs that may follow a given spatial distribution. NPTFit builds upon the framework of the fluctuation analyses developed in X-ray astronomy, thus it likely has applications beyond those demonstrated with gamma-ray data. The NPTFit package utilizes novel computational methods to perform the NPTF efficiently. The code is available at http://github.com/bsafdi/NPTFit and up-to-date and extensive documentation may be found at http://nptfit.readthedocs.io.

79 ASTRONOMY AND ASTROPHYSICS↗