Search NASA⌕ Search

SEARCH · Search NASA

Results for “high performance and distributed computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Spack-Based Packaging and Development for HEP

The art event-based analysis framework is used by multiple Intensity Frontier and other experiments and projects in High Energy Physics (HEP). A system of tools and scripts based around a Fermilab-originated package management system called UNIX Product Support (UPS) has evolved to provide development, packaging, release management, and distribution of the art suite, related experimental software and external dependencies in relocatable binary form. The existing system has limitations in environments such as macOS with System Integrity Protection (SIP) enabled and the various High Performance Computing (HPC) systems that the field of HEP is increasingly looking to leverage. This has led to a search for other ways of providing experiments with a packaging and build system that meets their needs. We describe our efforts to develop a packaging and release management system based on the Spack HPC package management tool to replace the existing UPS-based system. We additionally describe a companion system: SpackDev and cetmodules, allowing simultaneous development of multiple HEP software packages in a consistent environment. A successful implementation of the intended system would have applicability across HEP rather than merely to the users of the art framework.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2017

MPI-FAUN: An MPI-Based Framework for Alternating-Updating Nonnegative Matrix Factorization

Non-negative matrix factorization (NMF) is the problem of determining two non-negative low rank factors W and H, for the given input matrix A, such that A≈WH. NMF is a useful tool for many applications in different domains such as topic modeling in text mining, background separation in video analysis, and community detection in social networks. Despite its popularity in the data mining community, there is a lack of efficient parallel algorithms to solve the problem for big data sets. The main contribution of this work is a new, high-performance parallel computational framework for a broad class of NMF algorithms that iteratively solves alternating non-negative least squares (NLS) subproblems for W and H. It maintains the data and factor matrices in memory (distributed across processors), uses MPI for interprocessor communication, and, in the dense case, provably minimizes communication costs (under mild assumptions). The framework is flexible and able to leverage a variety of NMF and NLS algorithms, including Multiplicative Update, Hierarchical Alternating Least Squares, and Block Principal Pivoting. Our implementation allows us to benchmark and compare different algorithms on massive dense and sparse data matrices of size that spans from few hundreds of millions to billions. We demonstrate the scalability of our algorithm and compare it with baseline implementations, showing significant performance improvements. The code and the datasets used for conducting the experiments are available online.

2D↗
DOE OSTI2021

Machine learning-based analysis of COVID-19 pandemic impact on US research networks

Here in this study we explore how fallout from the changing public health policy around COVID-19 has changed how researchers access and process their science experiments. Using a combination of techniques from statistical analysis and machine learning, we conduct a retrospective analysis of historical network data for a period around the stay-at-home orders that took place in March 2020. Our analysis takes data from the entire ESnet infrastructure to explore DOE high-performance computing (HPC) resources at OLCF, ALCF, and NERSC, as well as User sites such as PNNL and JLAB. We look at detecting and quantifying changes in site activity using a combination of t-Distributed Stochastic Neighbor Embedding (t-SNE) and decision tree analysis. Our findings bring insights into the working patterns and impact on data volume movements, particularly during late-night hours and weekends.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Modeling primary atomization of liquid fuels using a multiphase DNS/LES Approach

We report on the development of a model framework to simulate spray flames from direct injection of liquid fuel into an automotive cylinder engine. The approach to this challenging problem was twofold. On one hand, the interface-capturing multiphase computer code CLSVOF was used to resolve the rapidly evolving, topologically convoluted interfaces that separate the liquid fuel from the gas at injection: the main challenges to address were the treatment of the high-pressure flow inside the injector, which required the inclusion of compressibility effects; and the computational framework necessary to achieve a Direct Numerical Simulation (DNS) level of accuracy. On the other hand, the scales of turbulent fuel mixing and combustion in the cylinder engine were addressed by the high-performance computer code RAPTOR within the Large Eddy Simulation (LES) framework. To couple the two computational methods, a novel methodology was developed to de- scribe the dense spray dynamics in Raptor from the assigned spray size distribution and dispersion angle derived from CLSVOF. This new, independent Eulerian Multi-Fluid (EMF) spray module was developed based on the kinetic description of a system of droplets as a pressure-less gas; as we will show, it was demonstrated to efficiently render the near-nozzle coupling in mass, momentum, and energy with the carrier gas phase.

02 PETROLEUM↗
NASA NTRS1991

Development of an UltraNet Based Distributed Visualization Application

The example application is a distributed visualization involving a supercomputer and a graphics workstation. The visualization computation is performed on a Connection Machine, end the results are rendered using a Silicon Graphics Workstations The UltraNet network installed at NAB allows high-bandwidth communication between the computers. Ideally, taking advantage of the UltraNet is no more complex than developing TCP/IP and Unix BSD socket-type applications on a single machine. In practice, there are several problems in developing an Application using the UltraNet. This paper identifies potential problems and discusses techniques for overcoming them. Performance of UltraNet communication is measured and found to be 10 MB/sec for SGI VGX workstations.

Krystynak, John↗
NASA NTRS1989

Comparison of measured induced velocities with results from a closed-form finite state wake model in forward flight

An unsteady, three-dimensional induced-flow model is applied to compute the induced-flow distribution of a rotor in forward flight, and numerical results are compared against LDV measurements for both time-averaged and unsteady induced flow at the disk. The former flow shows good agreement with measured data, except just behind the pylon at the lowest advanced ratio and near the blade tips for rectangular blades at high advance ratios. The method performs as well or better than other codes that have been applied to these data, takes less computing time, and is better suited to aeroelastic analysis. Results with only four harmonics and 15 state variables converge to all fundamental characteristics of the time-averaged flow.

Peters, David A.↗
DOE OSTI2022

Power and Communications Hardware-in-the-Loop CPS Architecture and Platform for DER Monitoring and Control Applications: Preprint

The rapid growth of distributed energy resources (DERs) has prompted increasing interest in the monitoring and control of DERs through hybrid smart grid communications. The deployment of communications and computation has transformed the traditional physical power grid into a smart cyber-physical system (CPS). To fully understand the interdependency between physical grid and cyber netowrks, this study designed a power and communications hardware-in-the-loop (PCommHIL) CPS architecture, which enables the flexible verification of DER monitoring and control with hybrid communications architectures and Internet protocols. Design, development and case study of a PCommHIL testbed for the DER coordination are discussed in detail, and the proposed platform integrates DER devices, Advanced Metering Infrastructures (AMIs), and a suite of hybrid communications networks for distribution automation applications. Case study on DER situational awareness and Volt-Var control validates the efficacy of this proposed PCommHIL platform with hybrid communications designs. Results show that the HAN communication technologies play a critical role in hybrid designs and it is the bottleneck for DER applications. High performance communication technologies are highly recommended to be applied in the HAN for enhanced monitoring and real-time control of DERs.

AMIs↗
DOE OSTI2024

CP violation searches and distributed computing development with the Belle II Experiment at the University of Mississippi and Brookhaven National Laboratory (Final Report)

This report reflects the work performed at the University of Mississippi under the support of DOE EPSCoR grant DE‐SC0021274 during the period of September 2020 through August 2024, including a one-year, no-cost extension. A summary of the research outcomes is given, with reference to the project goals as stated in the proposal. This successful project supported the mission of the DOE High Energy Physics program by leveraging the complimentary expertise of researchers at the University of Mississippi and Brookhaven National Lab to search for CP violation in charmed baryon decays using data from the Belle II experiment and to provide vital support for Belle II distributed computing.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1988

Consistent Data Distribution Over Optical Links

Fiber optics combined with IDE's provide consistent data communication between fault-tolerant computers. Data-transmission-checking system designed to provide consistent and reliable data communications for fault-tolerant and highly reliable computers. New technique performs variant of algorithm for fault-tolerant computers and uses fiber optics and independent decision elements (IDE's) to require fewer processors and fewer transmissions of messages. Enables fault-tolerant computers operating at different levels of redundancy to communicate with each other over triply redundant bus. Level of redundancy limited only by maximum number of wavelengths active on bus.

Palumbo, Daniel L.↗
DOE OSTI2016

CFD Study of Full-Scale Aerobic Bioreactors: Evaluation of Dynamic O2 Distribution, Gas-Liquid Mass Transfer and Reaction

If advanced biofuels are to measurably displace fossil fuels in the near term, they will have to operate at levels of scale, efficiency, and margin unprecedented in the current biotech industry. For aerobically-grown products in particular, scale-up is complex and the practical size, cost, and operability of extremely large reactors is not well understood. Put simply, the problem of how to attain fuel-class production scales comes down to cost-effective delivery of oxygen at high mass transfer rates and low capital and operating costs. To that end, very large reactor vessels (>500 m3) are proposed in order to achieve favorable economies of scale. Additionally, techno-economic evaluation indicates that bubble-column reactors are more cost-effective than stirred-tank reactors in many low-viscosity cultures. In order to advance the design of extremely large aerobic bioreactors, we have performed computational fluid dynamics (CFD) simulations of bubble-column reactors. A multiphase Euler-Euler model is used to explicitly account for the spatial distribution of air (i.e., gas bubbles) in the reactor. Expanding on the existing bioreactor CFD literature (typically focused on the hydrodynamics of bubbly flows), our simulations include interphase mass transfer of oxygen and a simple phenomenological reaction representing the uptake and consumption of dissolved oxygen by submerged cells. The simulations reproduce the expected flow profiles, with net upward flow in the center of column and downward flow near the wall. At high simulated oxygen uptake rates (OUR), oxygen-depleted regions can be observed in the reactor. By increasing the gas flow to enhance mixing and eliminate depleted areas, a maximum oxygen transfer (OTR) rate is obtained as a function of superficial velocity. These insights regarding minimum superficial velocity and maximum reactor size are incorporated into NREL's larger techno-economic models to supplement standard reactor design equations.

09 BIOMASS FUELS↗
DOE OSTI2019

High Order Anchoring and Reinitialization of Level Set Function for Simulating Interface Motion

A second order interface anchoring method has been developed and used with fast sweeping algorithm for reinitialization of a level set function. The algebraic anchoring formulation ensures that the location of the actual interface is preserved, leading to better mass conservation property. It also provides high order accurate algebraic constraint for solving the Eikonal equation on a finite difference grid. Geometric properties of the interface such as surface normal and curvature are subsequently computed from the reinitialized distance function. Various analytical functions for modeling distortion in level set field are considered and accuracy of reinitialization is evaluated using first and second order anchoring schemes. It is also shown that accurate computation of interface curvature requires a high order anchor in addition to a high order fast sweeping method. Mass conservation property of reinitialization is also analyzed by considering test problems from literature including the classic Rider–Kothe single vortex problem. The formulation is suitable for efficient parallelization for both distributed memory and on-node shared memory parallel systems. In conclusion, scalability and performance of the reinitialization scheme on multiple architectures are demonstrated.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Linear Solver for Electromagnetic Simulation of General Distribution Feeders

High-fidelity electromagnetic transient (EMT) modeling is required for accurate simulation and analysis of power system dynamics in modern distribution feeders. However, the high-fidelity of EMT models often leads to significant computational challenges, particularly in terms of computational resources and simulation time. This paper investigates the development and application of a detailed EMT model for general distribution feeders, with a focus on improving computational efficiency. A direct linear solver is proposed for a bordered block diagonal (BBD) matrix structure commonly encountered in a EMT model of distribution feeders. The solver integrates the Schur complement method with the block tridiagonal matrix algorithm to enhance the computational performance. The proposed solver is validated using the primary feeder of the IEEE 342-node test system, demonstrating its accuracy and efficiency in EMT simulations. Furthermore, the solver’s performance is benchmarked against MATLAB’s built-in linear solvers, showing significant improvements in computation time while maintaining high fidelity and accuracy in simulation results.

Choi, Jongchan [ORNL] (ORCID:000000025952455X)↗
DOE OSTI2024

AGS-GNN: Attribute-guided Sampling for Graph Neural Networks

We propose AGS-GNN, a novel attribute-guided sampling algorithm for Graph Neural Networks (GNNs) that exploits node features and connectivity structure of a graph while simultaneously adapting for both homophily and heterophily in graphs. (In homophilic graphs vertices of the same class are more likely to be connected, and vertices of different classes tend to be linked in heterophilic graphs.) While GNNs have been successfully applied to homophilic graphs, their application to heterophilic graphs remains challenging. The best-performing GNNs for heterophilic graphs do not fit the sampling paradigm, suffer high computational costs, and are not inductive. We employ samplers based on feature-similarity and feature-diversity to select subsets of neighbors for a node, and adaptively capture information from homophilic and heterophilic neighborhoods using dual channels. Currently, AGS-GNN is the only algorithm that we know of that explicitly controls homophily in the sampled subgraph through similar and diverse neighborhood samples. For diverse neighborhood sampling, we employ submodularity, which was not used in this context prior to our work. The sampling distribution is pre-computed and highly parallel, achieving the desired scalability. Using an extensive dataset consisting of 35 small (<=100K nodes) and large (>100K nodes) homophilic and heterophilic graphs, we demonstrate the superiority of AGS-GNN compare to the current approaches in the literature. AGS-GNN achieves comparable test accuracy to the best-performing heterophilic GNNs, even outperforming methods using the entire graph for node classification. AGS-GNN also converges faster compared to methods that sample neighborhoods randomly, and can be incorporated into existing GNN models that employ node or graph sampling.

artificial intelligence↗
DOE OSTI2018

Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) Winter Intensive Operational Period: Aerodynamic Particle Sizer Field Campaign Report

An aerodynamic particle sizer spectrometer (APS, model 3321 by TSI Inc., Shoreview, Minnesota) was deployed at the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) user facility Eastern North Atlantic (ENA) observatory during the second (winter) intensive operational period (IOP) of the Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) field campaign. Data were collected from January 9 through February 26, 2018, covering the second deployment of the ARM Aerial Facility (AAF) aircraft within ACE-ENA. The standard ENA instrument suite does not provide measurements of aerosol particle size distribution for diameters greater than one micron. Properties of coarse-mode (super-micron) aerosols are rarely measured, although super-micron particles can play an important role in the aerosol life cycle, affecting aerosol optical properties and cloud nuclei activity (e.g., Clarke et al. 2003). The APS was deployed to fill this gap in particle size measurements: it is capable of measuring size-resolved concentration of the aerosols in the 0.5-20 μm range (aerodynamic diameter). The APS was installed in the aerosol trailer to sample through the main stack. Special attention was given to selection of the instrument location (just below the main stack) to minimize particle losses in the connecting lines. Most atmospheric aerosols are hygroscopic, so the measurement of the aerosol size distributions should either be performed in a controlled humidity environment or accompanied by concurrent measurement of the temperature and humidity of the sample air. The latter strategy was used during this deployment: a high-precision humidity/temperature transmitter (model EE08 by E+E Elektonik GMBH, Engerwitzdorf, Germany) was mounted in the APS sampling line immediately upstream of the APS inlet. A dedicated laptop computer was used for data collection and communications. The APS was controlled with vendor-supplied software (Aerosol Instrument Manager, ver. 10.1, TSI); aerosol size distribution data were collected over a period of 55 seconds and converted to one-minute averages. Temperature and humidity analog signals were acquired via analog-to-digital converter (ADC) module (model WTAIN-M by Weeder Technologies, Fort Walton Beach, Florida) using vendor-supplied software (ModCom) with 1- sec resolution. In post-processing, the APS data were exported from TSI proprietary format into “comma-separated value” American Standard Code for Information Interchange (ASCII) files and matched with appropriately averaged temperature and humidity data. Two significant periods of missing data were caused by local power outages. Vendor-supplied software was not designed for unattended operation: it needs to be manually restarted including sensor configuration setup.

ACE-ENA, Azores, ultra-high-sensitivity aerosol sp↗
DOE OSTI2022

Proximity Portability and in Transit , M-to-N Data Partitioning and Movement in SENSEI [Book Chapter]

In high-performance parallel in situ processing, the term in transit processing refers to those configurations where data must move from a producer to a consumer that runs on separate resources. In the context of parallel and distributed computing on an HPC platform one of the central challenges is to determine a mapping of data from producer ranks to consumer ranks. This problem is complicated by the heterogeneity that arises in producer-consumer pairs, such as when producer and consumer codes have different levels of concurrency, different scaling characteristics, or different data models. The resulting mapping and movement of data from M producer to N consumer ranks can have a significant impact on aggregate application performance, particularly when the data consumer requires only a subset of the overall data for its task. This chapter focuses on the design considerations that underlie SENSEI’s implementation to this challenging problem. These design considerations extend the core SENSEI architecture and include ideas like the need to accommodate flexibility in the choice of different partitioning methods, the ability for a data consumer to request and receive only the subset of data needed for its particular operation, and the ability to leverage any of several different data transport tools. The idea of proximity portability, being able to use different data transport methods as part of an in transit workflow, is illustrated through the use of three different transport layers where switching from one transport tool to another is accomplished with only a configuration file change. Here, the chapter also includes a performance analysis summary showing the performance gains that are possible in terms of multiple metrics, such as memory footprint, time to solution, and amount of data moved, when using optimized partitioners in an in transit setting, gains that are made possible by the implementation shaped by specific design considerations.

Bethel, E. Wes↗
DOE OSTI2021

PETSc Users Manual (Revision 3.15)

This manual describes the use of the Portable, Extensible Toolkit for Scientific Computation (PETSc) for the numerical solution of partial differential equations and related problems on high-performance computers. PETSc is a suite of data structures and routines that provide the building blocks for the implementation of large-scale application codes on parallel (and serial) computers. PETSc uses the MPI standard for all distributed memory communication.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Userspace Squash Filesystem for Launching Linux Containers on HPC Systems [Thesis]

The demand for user defined software stacks (UDSS) has been increasing in the high-performance computing (HPC) community. Container technology has become popular due to the flexibility and isolation it provides to HPC users. Container images must be available to all nodes involved for use in HPC and can be distributed to compute nodes in a variety of ways. A common method for container image distribution is to simply copy the container image to memory on each compute node, which can be time consuming at scale, and uses valuable memory on each node. The kernel mounted squash filesystem (squashfs) has proven fast and efficient for this task but requires root-level access. This paper will show a user space mounted squashfs is an efficient and secure solution for container image distribution.

97 MATHEMATICS AND COMPUTING↗