Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel parameters optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Coping with silent and fail-stop errors at scale by combining replication and checkpointing

This paper provides a model and an analytical study of replication as a technique to cope with silent errors, as well as a mixture of both silent and fail-stop errors on large-scale platforms. Compared with fail-stop errors that are immediately detected when they occur, silent errors require a detection mechanism. To detect silent errors, many application-specific techniques are available, either based on algorithms (e.g., ABFT), invariant preservation or data analytics, but replication remains the most transparent and least intrusive technique. We explore the right level (duplication, triplication or more) of replication for two frameworks: (i) when the platform is subject to only silent errors, and (ii) when the platform is subject to both silent and fail-stop errors. A higher level of replication is more expensive in terms of resource usage but enables to tolerate more errors and to even correct some errors, hence there is a trade-off to be found. Replication is combined with checkpointing and comes with two flavors: process replication and group replication. Process replication applies to message-passing applications with communicating processes. Each process is replicated, and the platform is composed of process pairs, or triplets. Group replication applies to black-box applications, whose parallel execution is replicated several times. The platform is partitioned into two halves (or three thirds). In both scenarios, results are compared before each checkpoint, which is taken only when both results (duplication) or two out of three results (triplication) coincide. Otherwise, one or more silent errors have been detected, and the application rolls back to the last checkpoint, as well as when fail-stop errors have struck. We provide a detailed analytical study for all of these scenarios, with formulas to decide, for each scenario, the optimal parameters as a function of the error rate, checkpoint cost, and platform size. We also report a set of extensive simulation results that nicely corroborates the analytical model.

Checkpointing↗
NASA NTRS1994

A Numerical Comparison of Barrier and Modified Barrier Methods for Large-Scale Bound-Constrained Optimization

When a classical barrier method is applied to the solution of a nonlinear programming problem with inequality constraints, the Hessian matrix of the barrier function becomes increasingly ill-conditioned as the solution is approached. As a result, it may be desirable to consider alternative numerical algorithms. We compare the performance of two methods motivated by barrier functions. The first is a stabilized form of the classical barrier method, where a numerically stable approximation to the Newton direction is used when the barrier parameter is small. The second is a modified barrier method where a barrier function is applied to a shifted form of the problem, and the resulting barrier terms are scaled by estimates of the optimal Lagrange multipliers. The condition number of the Hessian matrix of the resulting modified barrier function remains bounded as the solution to the constrained optimization problem is approached. Both of these techniques can be used in the context of a truncated-Newton method, and hence can be applied to large problems, as well as on parallel computers. In this paper, both techniques are applied to problems with bound constraints and we compare their practical behavior.

Nash, Stephen G.↗
DOE OSTI2025

Two-level overlapping additive Schwarz preconditioner for training scientific machine learning applications

In this work we introduce a novel two-level overlapping additive Schwarz preconditioner for accelerating the training of scientific machine learning applications. The design of the proposed preconditioner is motivated by the nonlinear two-level overlapping additive Schwarz preconditioner. The neural network parameters are decomposed into groups (subdomains) with overlapping regions. In addition, the network’s feed-forward structure is indirectly imposed through a novel subdomain-wise synchronization strategy and a coarse-level training step. Through a series of numerical experiments, which consider physicsinformed neural networks and operator learning approaches, we demonstrate that the proposed two-level preconditioner significantly speeds up the convergence of the standard (LBFGS) optimizer while also yielding more accurate machine learning models. Moreover, the devised preconditioner is designed to take advantage of model-parallel computations, which can further reduce the training time.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Optimizing Distributed Training on Frontier for Large Language Models

Large language models (LLMs) have demonstrated remarkable success as foundational models, benefiting various downstream applications through fine-tuning. Loss scaling studies have demonstrated the superior performance of larger LLMs compared to their smaller counterparts. Nevertheless, training LLMs with billions of parameters poses significant challenges and requires considerable computational resources. For example, training a one trillion parameter GPT-style model on 20 trillion tokens requires a staggering 120 million exaflops. This research explores efficient distributed training strategies to extract this computation from Frontier, the world's first exascale supercomputer. We enable and investigate various model and data parallel training techniques, such as tensor parallelism, pipeline parallelism, and sharded data parallelism, to facilitate training a trillion-parameter model on Frontier. We empirically assess these techniques and their associated parameters to determine their impact on memory footprint, communication latency, and GPU's computational efficiency. We analyze the complex interplay among these techniques and find a strategy to combine them to achieve high throughput through hyperparameter tuning. We have identified efficient strategies for training large LLMs of varying sizes through empirical analysis and hyperparameter tuning. For 22 Billion, 175 Billion, and 1 Trillion parameters, we achieved GPU throughputs of 38.38%, 36.14%, and 31.96%, respectively. For the training of the 175 Billion parameter model and the 1 Trillion parameter model, we achieved 100% weak scaling efficiency on 1024 and 3072 Mi250X GPUs, respectively. We also achieved strong scaling efficiencies of 89% and 87% for these two models. We trained these models only tens of iterations instead of training till completion.

Yin, Junqi↗
DOE OSTI2022

Optimization of quasi-symmetric stellarators with self-consistent bootstrap current and energetic particle confinement

Quasi-symmetry can greatly improve the confinement of energetic particles and thermal plasma in a stellarator. The magnetic field of a quasi-symmetric stellarator at high plasma pressure is significantly affected by the bootstrap current, but the computational cost of accurate stellarator bootstrap calculations has precluded use inside optimization. Here, a new efficient method is demonstrated for optimization of quasi-symmetric stellarator configurations such that the bootstrap current profile is consistent with the geometry. The approach is based on the fact that all neoclassical phenomena in quasi-symmetry are isomorphic to those in axisymmetry. Therefore, accurate formulas for the bootstrap current in tokamaks, which can be evaluated rapidly, can be applied also in stellarators. The deviation between this predicted parallel current and the actual parallel current in the magnetohydrodynamic equilibrium is penalized in the objective function, and the current profile of the equilibrium is included in the parameter space. Quasi-symmetric configurations with significant pressure are thereby obtained with self-consistent bootstrap current and excellent confinement. In a comparison of fusion-produced alpha particle confinement across many stellarators, the new configurations have significantly lower alpha energy losses than many previous designs.

Landreman, M. (ORCID:000000027233577X)↗
DOE OSTI2024

Crystal structure of perfluorononanoic acid, C 9 HF 17 O 2

The crystal structure of perfluorononanoic acid (PFNA) was solved via parallel tempering using synchrotron powder diffraction data obtained from the Brockhouse X-ray Diffraction and Scattering (BXDS) Wiggler Lower Energy (WLE) beamline at the Canadian Light Source. PFNA crystallizes in monoclinic space group P2 1 /c (#14) with lattice parameters a = 26.172(1) Å, b = 5.6345(2) Å, c = 10.9501(4) Å, and β = 98.752(2)°. The crystal structure is composed of dimers, with pairs of PFNA molecules connected by hydrogen bonds via the carboxylic acid functional groups. The Rietveld-refined structure was compared to a density functional theory-optimized structure, and the root-mean-square Cartesian difference was larger than normally observed for correct powder structures. The powder data likely exhibited evidence of disorder which was not successfully modeled.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2019

Avoiding Communication in Primal and Dual Block Coordinate Descent Methods

Primal and dual block coordinate descent methods are iterative methods for solving regularized and unregularized optimization problems. Distributed-memory parallel implementations of these methods have become popular in analyzing large machine learning datasets. However, existing implementations communicate at every iteration, which, on modern data center and supercomputing architectures, often dominates the cost of floating-point computation. Recent results on communication-avoiding Krylov subspace methods suggest that large speedups are possible by re-organizing iterative algorithms to avoid communication. We show how applying similar algorithmic transformations can lead to primal and dual block coordinate descent methods that only communicate every $s$ iterations---where $s$ is a tuning parameter---instead of every iteration for the regularized least-squares problem. We show that the communication-avoiding variants reduce the number of synchronizations by a factor of $s$ on distributed-memory parallel machines without altering the convergence rate and attain strong scaling speedups of up to $6.1\times$ over the “standard algorithm" on a Cray XC30 supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Versatile soil gas concentration and isotope monitoring: optimization and integration of novel soil gas probes with online trace gas detection

Abstract. Gas concentrations and isotopic signatures can unveil microbial metabolisms and their responses to environmental changes in soil. Currently, few methods measure in situ soil trace gases such as the products of nitrogen and carbon cycling or volatile organic compounds (VOCs) that constrain microbial biochemical processes like nitrification, methanogenesis, respiration, and microbial communication. Versatile trace gas sampling systems that integrate soil probes with sensitive trace gas analyzers could fill this gap with in situ soil gas measurements that resolve spatial (centimeters) and temporal (minutes) patterns. We developed a system that integrates new porous and hydrophobic sintered polytetrafluoroethylene (sPTFE) diffusive soil gas probes that non-disruptively collect soil gas samples with a transfer system to direct gas from multiple probes to one or more central gas analyzer(s) such as laser and mass spectrometers. Here, we demonstrate the feasibility and versatility of this automated multiprobe system for soil gas measurements of isotopic ratios of nitrous oxide (δ18O, δ15N, and the 15N site preference of N2O), methane, carbon dioxide (δ13C), and VOCs. First, we used an inert silica matrix to challenge probe measurements under controlled gas conditions. By changing and controlling system flow parameters, including the probe flow rate, we optimized recovery of representative soil gas samples while reducing sampling artifacts on subsurface concentrations. Second, we used this system to provide a real-time window into the impact of environmental manipulation of irrigation and soil redox conditions on in situ N2O and VOC concentrations. Moreover, to reveal the dynamics in the stable isotope ratios of N2O (i.e., 14N14N16O, 14N15N16O, 15N14N16O, and 14N14N18O), we developed a new high-precision laser spectrometer with a reduced sample volume demand. Our integrated system – a tunable infrared laser direct absorption spectrometry (TILDAS) in parallel with Vocus proton transfer reaction mass spectrometry (PTR-MS), in line with sPTFE soil gas probes – successfully quantified isotopic signatures for N2O, CO2, and VOCs in real time as responses to changes in the dry–wetting cycle and redox conditions. Broadening the collection of trace gases that can be monitored in the subsurface is critical for monitoring biogeochemical cycles, ecosystem health, and management practices at scales relevant to the soil system.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2015

Vibroacoustic Tailoring of a Rod-Stiffened Composite Fuselage Panel with Multidisciplinary Considerations

An efficient multi-objective design tailoring procedure seeking to improve the vibroacoustic performance of a fuselage panel while maintaining or reducing weight is presented. The structure considered is the pultruded rod stitched efficient unitized structure, a highly integrated composite structure concept designed for a noncylindrical, next-generation flight vehicle fuselage. Modifications to a baseline design are evaluated within a six-parameter design space including spacing, flange width, and web height for both frame and stringer substructure components. The change in sound power radiation attributed to a design change is predicted using finite-element models sized and meshed for analyses in the 500 Hz, 1 kHz, and 2 kHz octave bands. Three design studies are carried out in parallel while considering a diffuse acoustic field excitation and two types of turbulent boundary-layer excitation. Kriging surrogate models are used to reduce the computational costs of resolving the vibroacoustic and weight objective Pareto fronts. The resulting Pareto optimal designs are then evaluated under a static pressurization ultimate load to assess structural strength and stability. Results suggest that choosing alternative configurations within the considered design space can reduce weight and improve vibroacoustic performance without compromising strength and stability of the structure under the static load condition considered, but the tradeoffs are significantly influenced by the spatial characteristics of the assumed excitation field.

Allen, Albert R.↗
DOE OSTI2022

Prediction of DIII-D Pedestal Structure from Externally Controllable Parameters

The sharp increase of pressure at the edge of a high confinement mode (H-mode) plasma, the pedestal, strongly impacts overall plasma performance. Predicting the pedestal is a necessity to control and optimize tokamak operations. An experimental data-driven machine learning (ML) approach is presented that predicts the pedestal heights and widths of electron density (ne) and electron temperature (Te) profiles as well as the separatrix ne from externally controllable parameters such as the plasma shape, heating method and power, and gas puff rate and integrated gas puff. The OMFIT framework was used with DIII-D data to efficiently, robustly, and automatically build a database of pedestal parameters to train machine learning models. Database creation was enabled by the search engine tool for DIII-D data, TokSearch, which parallelizes data fetching, enabling fast searches through basic signals of thousands of DIII-D shots and selection of relevant time intervals. Principal Component Analysis (PCA) separated the database into three clusters that represent classes of plasma shapes that are regularly used in DIII-D. The most important parameters for setting the pedestal structure were plasma current (Ip), toroidal magnetic field (Bφ), neutral beam heating power (PNBI) and shaping quantities. The Deep Jointly Informed Neural Networks (DJINN) algorithm was applied to identify suitable neural network (NN) architectures that appropriately capture the features of the pedestal database. Separate NNs were implemented for each pedestal parameter, and ensembling methods were used to improve the prediction accuracy and allowed estimation of the prediction uncertainty. The pedestal predictions of the test dataset lie within the measurement uncertainties of the pedestal parameters. The NN outperformed simple Linear Regression (LR) analysis, indicating non-linear dependencies in the pedestal structure. The presented achievements illustrate a promising path for future research, using feature extraction to infer experimental trends and thereby improve pedestal models as well as deploying NN for a fast pedestal prediction in DIII-D scenario development.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1987

Development of a stereo analysis algorithm for generating topographic maps using interactive techniques of the MPP

A local area matching algorithm was developed on the Massively Parallel Processor (MPP). It is an iterative technique that first matches coarse or low resolution areas and at each iteration performs matches of higher resolution. Results so far show that when good matches are possible in the two images, the MPP algorithm matches corresponding areas as well as a human observer. To aid in developing this algorithm, a control or shell program was developed for the MPP that allows interactive experimentation with various parameters and procedures to be used in the matching process. (This would not be possible without the high speed of the MPP). With the system, optimal techniques can be developed for different types of matching problems.

Strong, James P.↗
NASA NTRS1988

Applications of concurrent neuromorphic algorithms for autonomous robots

This article provides an overview of studies at the Oak Ridge National Laboratory (ORNL) of neural networks running on parallel machines applied to the problems of autonomous robotics. The first section provides the motivation for our work in autonomous robotics and introduces the computational hardware in use. Section 2 presents two theorems concerning the storage capacity and stability of neural networks. Section 3 presents a novel load-balancing algorithm implemented with a neural network. Section 4 introduces the robotics test bed now in place. Section 5 concerns navigation issues in the test-bed system. Finally, Section 6 presents a frequency-coded network model and shows how Darwinian techniques are applied to issues of parameter optimization and on-line design.

Barhen, J.↗
NASA NTRS1993

Extended ecosystem signatures with application to Eos synergism requirements

The primary objective is to define the advantages of synergistically combining optical and microwave remote sensing measurements for the determination of biophysical properties important in ecosystem modeling. This objective was approached in a stepwise fashion starting with ground-based observations of controlled agricultural and orchard canopies and progressing to airborne observations of more natural forest ecosystems. This observational program is complemented by a parallel effort to model the visible reflectance and microwave scattering properties of composite vegetation canopies. The goals of the modeling studies are to verify our basic understanding of the sensor-scene interaction physics and to provide the basis for development of inverse models optimized for retrieval of key biophysical properties. These retrieval algorithms can then be used to simulate the expected performance of various aspects of Eos including the need for simultaneous SAR and HIRIS observations or justification for other (non-synchronous) relative timing constraints and the frequency, polarization, and angle of incidence requirements for accurate biophysical parameter extractions. This program completed a very successful series of truck-mounted experiments, made remarkable progress in development and validation of optical reflectance and microwave scattering models for vegetation, extended the scattering models to accommodate discontinuous and periodic canopies, developed inversion approaches for surface and canopy properties, and disseminated these results widely through symposia and journal publications. In addition, the third generation of the computer code for the microwave scattering models was provided to a number of other US, Canadian, Australian, and European investigators who are currently presenting and publishing results using the MIMICS research code.

Ulaby, Fawwaz T.↗
DOE OSTI2022

m-CUBES An efficient and portable implementation of multi-dimensional integration for gpus

The task of multi-dimensional numerical integration is frequently encountered in physics and other scientific fields, e.g., in modeling the effects of systematic uncertainties in physical systems and in Bayesian parameter estimation. Multi-dimensional integration is often time-prohibitive on CPUs. Efficient implementation on many-core architectures is challenging as the workload across the integration space cannot be predicted a priori. We propose m-Cubes, a novel implementation of the well-known Vegas algorithm for execution on GPUs. Vegas transforms integration variables followed by calculation of a Monte Carlo integral estimate using adaptive partitioning of the resulting space. m-Cubes improves performance on GPUs by maintaining relatively uniform workload across the processors. As a result, our optimized Cuda implementation for Nvidia GPUs outperforms parallelization approaches proposed in past literature. We further demonstrate the efficiency of m-Cubes by evaluating a six-dimensional integral from a cosmology application, achieving significant speedup and greater precision than the CUBA library's CPU implementation of VEGAS. We also evaluate m-Cubes on a standard integrand test suite. m-Cubes outperforms the serial implementations of the Cuba and GSL libraries by orders of magnitude speedup while maintaining comparable accuracy. Our approach yields a speedup of at least 10 when compared against publicly available Monte Carlo based GPU implementations. In summary, m-Cubes can solve integrals that are prohibitively expensive using standard libraries and custom implementations. A modern C++ interface header-only implementation makes m-Cubes portable, allowing its utilization in complicated pipelines with easy to define stateful integrals. Compatibility with non-Nvidia GPUs is achieved with our initial implementation of m-Cubes using the Kokkos framework.

Sakiotis, Ioannis↗
DOE OSTI2022

Toward fully automated UED operation using two-stage machine learning model

To demonstrate the feasibility of automating UED operation and diagnosing the machine performance in real time, a two-stage machine learning (ML) model based on self-consistent start-to-end simulations has been implemented. This model will not only provide the machine parameters with adequate precision, toward the full automation of the UED instrument, but also make real-time electron beam information available as single-shot nondestructive diagnostics. Furthermore, based on a deep understanding of the root connection between the electron beam properties and the features of Bragg-diffraction patterns, we have applied the hidden symmetry as model constraints, successfully improving the accuracy of energy spread prediction by a factor of five and making the beam divergence prediction two times faster. The capability enabled by the global optimization via ML provides us with better opportunities for discoveries using near-parallel, bright, and ultrafast electron beams for single-shot imaging. It also enables directly visualizing the dynamics of defects and nanostructured materials, which is impossible using present electron-beam technologies.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Prediction of DIII-D Pedestal Structure From Externally Controllable Parameters

The sharp increase of pressure at the edge of a high confinement mode (H-mode) plasma, the pedestal, strongly impacts overall plasma performance. Predicting the pedestal is a necessity to control and optimize tokamak operations. Here, an experimental data-driven machine learning (ML) approach is presented that predicts the pedestal heights and widths of electron density (n e ) and electron temperature (T e ) profiles as well as the separatrix ne from externally controllable parameters such as the plasma shape, heating method and power, and gas puff rate and integrated gas puff. The OMFIT framework was used with DIII-D data to efficiently, robustly, and automatically build a database of pedestal parameters to train machine learning models. Database creation was enabled by the search engine tool for DIII-D data, TokSearch, which parallelizes data fetching, enabling fast searches through basic signals of thousands of DIII-D shots and selection of relevant time intervals. Principal Component Analysis (PCA) separated the database into three clusters that represent classes of plasma shapes that are regularly used in DIII-D. The most important parameters for setting the pedestal structure were plasma current (I p ), toroidal magnetic field (B Φ ), neutral beam heating power (P NBI ) and shaping quantities. The Deep Jointly Informed Neural Networks (DJINN) algorithm was applied to identify suitable neural network (NN) architectures that appropriately capture the features of the pedestal database. Separate NNs were implemented for each pedestal parameter, and ensembling methods were used to improve the prediction accuracy and allowed estimation of the prediction uncertainty. The pedestal predictions of the test dataset lie within the measurement uncertainties of the pedestal parameters. The NN outperformed simple Linear Regression (LR) analysis, indicating non-linear dependencies in the pedestal structure. The presented achievements illustrate a promising path for future research, using feature extraction to infer experimental trends and thereby improve pedestal models as well as deploying NN for a fast pedestal prediction in DIII-D scenario development.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Instability Issue of Paralleled Dies in an SiC Power Module in Solid-State Circuit Breaker Applications

Paralleled dies in a power module could have instability issues during high current switching transients. Here, the instability is caused by the differential-mode oscillation among paralleled MOSFETs. Conventional analyses of paralleled MOSFETs’ stability are normally limited to a single operating point, which ignores the influences of the switching trajectory and nonlinear device parameters on stability. This article reveals that the switching trajectory can significantly influence parallel stability. The analysis is improved by solving eigenvalues of state-space modeling system matrices of all operating points that the switching trajectory goes through considering nonlinear device parameters. Higher voltage and current stresses result in greater real parts of complex eigenvalues, which explains why the paralleled MOSFETs are more unstable with higher voltage and current stresses. To improve stability in solid-state circuit breaker applications, we propose a method to manipulate the switching trajectory to avoid the unstable region where the conventional hard switching trajectory normally goes through. Experimental results show that the turn- off current capability can be increased from ~five times of rated current with the gate oscillation using the conventional turn- off trajectory to ~ten times of rated current without the gate oscillation using the optimal turn- off trajectory.

42 ENGINEERING↗
DOE OSTI2017

Effect of 3D magnetic perturbations on divertor conditions and detachment in tokamak and stellarator

Enhanced perpendicular heat and momentum transport induces parallel pressure loss leading to divertor detachment, which can be produced by the increase of density in 2D tokamaks. However, in the 3D configurations such as tokamaks with 3D fields and stellarators, the fraction of perpendicular transport can be higher even in a lower density regime, which could lead to the early transition to detachment without passing through the high-recycling regime. 3D fields applied to the limiter tokamak plasmas produce edge stochastic layers close to the last closed flux surface (LCFS), which can allow for enhanced perpendicular transport and indeed the absence of high recycling regime and early detachment have been observed in TEXTOR and Tore Supra. However, in the X-point divertor tokamaks with the applied 3D fields, the parallel transport is still dominant and the detachment facilitation has not been observed yet. Rather, 3D fields affected detachment adversely under certain conditions, either by preventing detachment onset as seen in DIII-D or by re-attaching the existing detached plasma as shown in NSTX. The possible way for strong 3D effects to induce access to the early detachment in divertor tokamaks appears to be via significant perpendicular loss of parallel momentum by frictional force for the counter-streaming flows between neighboring flow channels in the divertor. In principle, the adjacent lobes in the 3D divertor tokamak may generate the counter-streaming flow channels. However, an EMC3-EIRENE simulation for ITER H-mode plasmas demonstrated that screened RMP leads to significantly reduced counter-flows near the divertor target, therefore the momentum loss effect leading to detachment facilitation is expected to be small. This is consistent with the observation in LHD, which showed screening (amplification) of RMP fields in the attachment (stable detachment) case. In conclusion, work for optimal parameter window for best divertor operation scenario is needed particularly for the 3D divertor tokamak configuration.

3D effects on divertor↗