Search NASA⌕ Search

SEARCH · Search NASA

Results for “penalty parameter”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Inverse determination of the penalty parameter in penalized weighted least-squares algorithm for noise reduction of low-dose CBCT

Purpose: A statistical projection restoration algorithm based on the penalized weighted least-squares (PWLS) criterion can substantially improve the image quality of low-dose CBCT images. The performance of PWLS is largely dependent on the choice of the penalty parameter. Previously, the penalty parameter was chosen empirically by trial and error. In this work, the authors developed an inverse technique to calculate the penalty parameter in PWLS for noise suppression of low-dose CBCT in image guided radiotherapy (IGRT). Methods: In IGRT, a daily CBCT is acquired for the same patient during a treatment course. In this work, the authors acquired the CBCT with a high-mAs protocol for the first session and then a lower mAs protocol for the subsequent sessions. The high-mAs projections served as the goal (ideal) toward, which the low-mAs projections were to be smoothed by minimizing the PWLS objective function. The penalty parameter was determined through an inverse calculation of the derivative of the objective function incorporating both the high and low-mAs projections. Then the parameter obtained can be used for PWLS to smooth the noise in low-dose projections. CBCT projections for a CatPhan 600 and an anthropomorphic head phantom, as well as for a brain patient, were used to evaluate the performance of the proposed technique. Results: The penalty parameter in PWLS was obtained for each CBCT projection using the proposed strategy. The noise in the low-dose CBCT images reconstructed from the smoothed projections was greatly suppressed. Image quality in PWLS-processed low-dose CBCT was comparable to its corresponding high-dose CBCT. Conclusions: A technique was proposed to estimate the penalty parameter for PWLS algorithm. It provides an objective and efficient way to obtain the penalty parameter for image restoration algorithms that require predefined smoothing parameters.

61 RADIATION PROTECTION AND DOSIMETRY↗

Robust and Simple ADMM Penalty Parameter Selection

We present a new method for online selection of the penalty parameter for the alternating direction method of multipliers (ADMM) algorithm. ADMM is a widely used method for solving a range of optimization problems, including those that arise in signal and image processing. In its standard form, ADMM includes a scalar hyperparameter, known as the penalty parameter, which usually has to be tuned to achieve satisfactory empirical convergence. In this work, we develop a framework for analyzing the ADMM algorithm applied to a quadratic problem as an affine fixed point iteration. Using this framework, we develop a new method for automatically tuning the penalty parameter by detecting when it has become too large or small. We analyze this and several other methods with respect to their theoretical properties, i.e., robustness to problem transformations, and empirical performance on several optimization problems. Our proposed algorithm is based on a theoretical framework with clear, explicit assumptions and approximations, is theoretically covariant/invariant to problem transformations, is simple to implement, and exhibits competitive empirical performance.

42 ENGINEERING↗

Boundary-consistent B-spline filtering schemes and application to high-fidelity simulations of turbulence

A filtering operation, based on B-spline discretizations, is introduced to target weakly growing mesh-scale oscillations that can arise in high-fidelity turbulence simulations. This is a spectral regularization that can be described using the singular values of a banded matrix operator, with the filtering strength set by a scalar- or vector-valued penalty parameter. The penalty parameter can be specified though it can also be advantageously selected to minimize the generalized cross validation (GCV) measure of distance between the pre- and post-filtered solutions. Efficient algorithms are developed to compute both the scalar and vector penalty parameters. The B-spline filter has a sharper localization to high-wavenumber than compact or explicit filters of the same stencil width and is demonstrated for solutions of the Burgers' equation, decaying Burgers' turbulence, and compressible Navier–Stokes turbulent channel flow. Furthermore, these simulations confirm the scheme's numerical stability and ability to narrowly target the high wavenumber components of numerical solutions. An advantage over finite-difference filters is that these B-spline filters are stable on bounded domains and even preserve formal order of accuracy.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A provably stable numerical method for the anisotropic diffusion equation in confined magnetic fields

We present a novel numerical method for solving the anisotropic diffusion equation in magnetic fields confined to a periodic box which is accurate and provably stable. We derive energy estimates of the solution of the continuous initial boundary value problem. A discrete formulation is presented using operator splitting in time with the summation by parts finite difference approximation of spatial derivatives for the perpendicular diffusion operator. Weak penalty procedures are derived for implementing both boundary conditions and parallel diffusion operator obtained by field line tracing. We prove that the fully-discrete approximation is unconditionally stable. Discrete energy estimates are shown to match the continuous energy estimate given the correct choice of penalty parameters. A nonlinear penalty parameter is shown to provide an effective method for tuning the parallel diffusion penalty and significantly minimises rounding errors. Several numerical experiments, using manufactured solutions, the “NIMROD benchmark” problem and a single island problem, are presented to verify numerical accuracy, convergence, and asymptotic preserving properties of the method. Finally, we present a magnetic field with chaotic regions and islands and show the contours of the anisotropic diffusion equation reproduce key features in the field.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

An adaptive sampling augmented Lagrangian method for stochastic optimization with deterministic constraints

The primary goal of this paper is to provide an efficient solution algorithm based on the augmented Lagrangian framework for optimization problems with a stochastic objective function and deterministic constraints. Our main contribution is combining the augmented Lagrangian framework with adaptive sampling, resulting in an efficient optimization methodology validated with practical examples. To achieve the presented efficiency, here we consider inexact solutions for the augmented Lagrangian subproblems, and through an adaptive sampling mechanism, we control the variance in the gradient estimates. Furthermore, we analyze the theoretical performance of the proposed scheme by showing equivalence to a gradient descent algorithm on a Moreau envelope function, and we prove sublinear convergence for convex objectives and linear convergence for strongly convex objectives with affine equality constraints. The worst-case sample complexity of the resulting algorithm, for an arbitrary choice of penalty parameter in the augmented Lagrangian function, is $\mathscr{O}$(ϵ -3-δ ) , where ϵ > 0 is the expected error of the solution and δ > 0 is a user-defined parameter. If the penalty parameter is chosen to be $\mathscr{O}$(ϵ -1 ), we demonstrate that the result can be improved to $\mathscr{O}$(ϵ -2 ) , which is competitive with the other methods employed in the literature. Moreover, if the objective function is strongly convex with affine equality constraints, we obtain $\mathscr{O}$(ϵ -1 log(1/ϵ)) complexity. Finally, we empirically verify the performance of our adaptive sampling augmented Lagrangian framework in machine learning optimization and engineering design problems, including topology optimization of a heat sink with environmental uncertainty.

97 MATHEMATICS AND COMPUTING↗

An Adaptive Multiparameter Penalty Selection Method for Multiconstraint and Multiblock ADMM

This work presents a new method for online selection of multiple penalty parameters for the alternating direction method of multipliers (ADMM) algorithm applied to optimization problems with multiple constraints or functions with block matrix components. ADMM is widely used for solving constrained optimization problems in a variety of fields, including signal and image processing. Implementations of ADMM often utilize a single hyperparameter, referred to as the penalty parameter, which needs to be tuned to control the rate of convergence. However, in problems with multiple constraints, ADMM may demonstrate slow convergence regardless of penalty parameter selection due to scale differences between constraints. Accounting for scale differences between constraints to improve convergence in these cases requires introducing a penalty parameter for each constraint. The proposed method is able to adaptively account for differences in scale between constraints, providing robustness with respect to problem transformations and initial selection of penalty parameters. It is also simple to understand and implement. Our numerical experiments demonstrate that the proposed method performs favorably compared to a variety of existing penalty parameter selection methods.

97 MATHEMATICS AND COMPUTING↗

Implementation of Hybrid V-Cycle Multilevel Methods for Mixed Finite Element Systems with Penalty

The goal of this paper is the implementation of hybrid V-cycle hierarchical multilevel methods for the indefinite discrete systems which arise when a mixed finite element approximation is used to solve elliptic boundary value problems. By introducing a penalty parameter, the perturbed indefinite system can be reduced to a symmetric positive definite system containing the small penalty parameter for the velocity unknown alone. We stabilize the hierarchical spatial decomposition approach proposed by Cai, Goldstein, and Pasciak for the reduced system. We demonstrate that the relative condition number of the preconditioner is bounded uniformly with respect to the penalty parameter, the number of levels and possible jumps of the coefficients as long as they occur only across the edges of the coarsest elements.

Lai, Chen-Yao G.↗

A Reinforcement Learning Approach to Parameter Selection for Distributed Optimal Power Flow

With the increasing penetration of distributed energy resources, distributed optimization algorithms have attracted significant attention for power systems applications due to their potential for superior scalability, privacy, and robustness to a single point-of-failure. The Alternating Direction Method of Multipliers (ADMM) is a popular distributed optimization algorithm; however, its convergence performance is highly dependent on the selection of penalty parameters, which are usually chosen heuristically. In this work, we use reinforcement learning (RL) to develop an adaptive penalty parameter selection policy for alternating current optimal power flow (ACOPF) problem solved via ADMM with the goal of minimizing the number of iterations until convergence. We train our RL policy using deep Q-learning and show that this policy can result in significantly accelerated convergence (up to a 59% reduction in the number of iterations compared to existing, curvatureinformed penalty parameter selection methods). Furthermore, we show that our RL policy demonstrates promise for generalizability, performing well under unseen loading schemes as well as under unseen losses of lines and generators (up to a 50% reduction in iterations). This work thus provides a proof-of-concept for using RL for parameter selection in ADMM for power systems applications.

alternating current optimal power flow↗

Penalty-Based Finite Element Interface Technology for Analysis of Homogeneous and Composite Structures

An effective and robust interface element technology able to connect independently modeled finite element subdomains has been developed. This method is based on the use of penalty constraints and allows coupling of finite element models whose nodes do not coincide along their common interface. Additionally, the present formulation leads to a computational approach that is very efficient and completely compatible with existing commercial software. A significant effort has been directed toward identifying those model characteristics (element geometric properties, material properties, and loads) that most strongly affect the required penalty parameter, and subsequently to developing simple 'formulae' for automatically calculating the proper penalty parameter for each interface constraint. This task is especially critical in composite materials and structures, where adjacent sub-regions may be composed of significantly different materials or laminates. This approach has been validated by investigating a variety of two-dimensional problems, including composite laminates.

Averill, Ronald C.↗

Noise Reduction in Low-Dose X-Ray Fluoroscopy for Image-Guided Radiation Therapy

Purpose: To improve the quality of low-dose X-ray fluoroscopic images using statistics-based restoration algorithm so that the patient fluoroscopy can be performed with reduced radiation dose. Method and Materials: Noise in the low-dose fluoroscopy was suppressed by temporal and spatial filtering. The temporal correlation among neighboring frames was considered by the Karhunen-Loeve (KL) transform (i.e., principal component analysis). After the KL transform, the selected neighboring frames of fluoroscopy were decomposed to uncorrelated and ordered principal components. For each KL component, a penalized weighted least-squares (PWLS) objective function was constructed to restore the ideal image. The penalty was chosen as anisotropic quadratic, and the penalty parameter in each KL component was inversely proportional to its corresponding eigenvalue. Smaller KL eigenvalue is associated with the KL component of lower signal-to-noise ratio (SNR), and a larger penalty parameter should be used for such KL component. The low-dose fluoroscopic images were acquired using a Varian Acuity simulator. A quality assurance phantom and an anthropomorphic chest phantom were used to evaluate the presented algorithm. Results: In the images restored by the proposed KL domain PWLS algorithm, noise is greatly suppressed, whereas fine structures are well preserved. Average improvement rate of SNR is 75% among selected regions of interest. Comparison studies with traditional techniques, such as the mean and median filters, show that the proposed algorithm is advantageous in terms of structure preservation. Conclusions: The proposed noise reduction algorithm can significantly improve the quality of low-dose X-ray fluoroscopic image and allows for dose reduction in X-ray fluoroscopy.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Nonlinear convergence in contact mechanics: Immersed boundary finite volume

In this report we present an immersed boundary finite volume (IBM) method for simulating quasistatic contact mechanics of linearly elastic domains at small strains. In IBM, all external boundaries and internal contacts of an object are represented by embedded surfaces inside a Cartesian mesh, which need not conform to the grid lines. The contact constraints consist of the non-penetrability condition and Coulomb’s friction law, which are discretized using special interpolation stencils and enforced via penalty parameters. The resulting nonlinear system depends on displacement unknowns only. To solve it, we use the Newton method but find that it diverges frequently. To understand the divergence pattern, we analyze a simplified 2-cell problem and show that the global convergence of Newton cannot be ensured for any choice of penalty parameters. We thus propose a modified Newton solver, which guarantees convergence for the 2-cell problem and is numerically verified to converge for all the challenging simulations considered herein. While both 1 st - and 2 nd -order variants of IBM, in displacement unknowns, are proposed, the modified Newton solver applies only to the 1 st -order variant.

42 ENGINEERING↗

A duality-based coupling of Cosserat crystal plasticity and phase field theories for modeling grain refinement

High-rate deformation processes of metals entail intense grain refinement and special attention needs to be paid to capture the evolution of microstructure. In this article, a new formulation for coupling Cosserat crystal plasticity and phase field is developed. A common approach is to penalize kinematic incompatibility between lattice orientation and displacement-based elastic rotation. However, this can lead to significant solution sensitivity to the penalty parameter, resulting in low accuracy and convergence rates. To address these issues, a duality-based formulation is developed which directly imposes the rotational kinematic compatibility. A weak inf-sup-based skew-symmetric stress projection is introduced to suppress instabilities present in the dual formulation. An additional least squares stabilization is introduced to suppress the spurious lattice rotation with a suitable parameter range derived analytically and validated numerically. The required high-order continuity is attained by the reproducing kernel approximation. It is observed that equal order displacement-rotation-phase field approximations are stable, which allows efficient employment of the same set of shape functions for all independent variables. The proposed formulation is shown to yield superior accuracy and convergence with marginal parameter sensitivity compared to the penalty-based approach and successfully captures the dominant rotational recrystallization mechanism including block dislocation structures and grain boundary migration.

42 ENGINEERING↗

Hybridized Methods for Quantum Simulation in the Interaction Picture

Conventional methods of quantum simulation involve trade-offs that limit their applicability to specific contexts where their use is optimal. In particular, the interaction picture simulation has been found to provide substantial asymptotic advantages for some Hamiltonians, but incurs prohibitive constant factors and is incompatible with methods like qubitization. We provide a framework that allows different simulation methods to be hybridized and thereby improve performance for interaction picture simulations over known algorithms. These approaches show asymptotic improvements over the individual methods that comprise them and further make interaction picture simulation methods practical in the near term. Physical applications of these hybridized methods yield a gate complexity scaling as log 2 ⁡ Λ in the electric cutoff Λ for the Schwinger Model and independent of the electron density for collective neutrino oscillations, outperforming the scaling for all current algorithms with these parameters. For the general problem of Hamiltonian simulation subject to dynamical constraints, these methods yield a query complexity independent of the penalty parameter λ used to impose an energy cost on time-evolution into an unphysical subspace.

97 MATHEMATICS AND COMPUTING↗

Modified SUMT for structural synthesis

The concepts of the singular perturbation theory are employed to modify the SUMT (Sequential Unconstrained Minimization Technique) algorithm of Fiacco-McCormick (1968) in order to make it more robust and reliable. The algorithm, which uses a sequence of penalty parameters to convert a constrained problem into a sequence of unconstrained problems, suffers from the need to minimize an ill-conditioned penalty function. By using the modified SUMT algorithm on two different structural optimization problems, it is shown that the singular perturbation SUMT easily converges to accurate solutions.

Kamat, M. P.↗

A Smoothed Augmented Lagrangian Framework for Convex Optimization with Nonsmooth Constraints

Augmented Lagrangian (AL) methods have proven remarkably useful in solving optimization problems with complicated constraints. The last decade has seen the development of overall complexity guarantees for inexact AL variants. Yet, a crucial gap persists in addressing nonsmooth convex constraints. To this end, we present a smoothed augmented Lagrangian (AL) framework where nonsmooth terms are progressively smoothed with a smoothing parameter $\eta _k$ . The resulting AL subproblems are $\eta _k$ -smooth, allowing for leveraging accelerated schemes. By a careful selection of the inexactness level $\epsilon _k$ (for inexact subproblem resolution), the penalty parameter $\rho _k$ , and smoothing parameter $\eta _k$ at epoch k, we derive rate and complexity guarantees of $\tilde{\mathcal {O}}(1/{\varepsilon }^{3/2})$ and $\tilde{\mathcal {O}}(1/{\varepsilon })$ in convex and strongly convex regimes for computing an ${\varepsilon }$ -optimal solution, when $\rho _k$ increases at a geometric rate, a significant improvement over the best available guarantees for AL schemes for convex programs with nonsmooth constraints. Analogous guarantees are developed for settings with $\rho _k = \rho$ as well as $\eta _k = \eta$ . Preliminary numerics on a fused Lasso problem display promise.

augmented Lagrangian↗

Optimization of laminated stacking sequence for buckling load maximization by genetic algorithm

The use of a genetic algorithm to optimize the stacking sequence of a composite laminate for buckling load maximization is studied. Various genetic parameters including the population size, the probability of mutation, and the probability of crossover are optimized by numerical experiments. A new genetic operator - permutation - is proposed and shown to be effective in reducing the cost of the genetic search. Results are obtained for a graphite-epoxy plate, first when only the buckling load is considered, and then when constraints on ply contiguity and strain failure are added. The influence on the genetic search of the penalty parameter enforcing the contiguity constraint is studied. The advantage of the genetic algorithm in producing several near-optimal designs is discussed.

Le Riche, Rodolphe↗

An Augmented Lagrangian Method for a Class of Inverse Quadratic Programming Problems

We consider an inverse quadratic programming (QP) problem in which the parameters in the objective function of a given QP problem are adjusted as little as possible so that a known feasible solution becomes the optimal one. We formulate this problem as a minimization problem with a positive semidefinite cone constraint and its dual is a linearly constrained semismoothly differentiable (SC{sup 1}) convex programming problem with fewer variables than the original one. We demonstrate the global convergence of the augmented Lagrangian method for the dual problem and prove that the convergence rate of primal iterates, generated by the augmented Lagrange method, is proportional to 1/r, and the rate of multiplier iterates is proportional to 1/{radical}r, where r is the penalty parameter in the augmented Lagrangian. As the objective function of the dual problem is a SC{sup 1} function involving the projection operator onto the cone of symmetrically semi-definite matrices, the analysis requires extensive tools such as the singular value decomposition of matrices, an implicit function theorem for semismooth functions, and properties of the projection operator in the symmetric-matrix space. Furthermore, the semismooth Newton method with Armijo line search is applied to solve the subproblems in the augmented Lagrange approach, which is proven to have global convergence and local quadratic rate. Finally numerical results, implemented by the augmented Lagrangian method, are reported.

97 MATHEMATICS AND COMPUTING↗

Scalable Preconditioning of Block-Structured Linear Algebra Systems using ADMM

In this paper, we study the solution of block-structured linear algebra systems arising in optimization by using iterative solution techniques. These systems are the core computational bottleneck of many problems of interest such as parameter estimation, optimal control, network optimization, and stochastic programming. Our approach uses a Krylov solver (GMRES) that is preconditioned with an alternating method of multipliers (ADMM). We show that this ADMM-GMRES approach overcomes well-known scalability issues of Schur complement decomposition in problems that exhibit a high degree of coupling. The effectiveness of the approach is demonstrated using linear systems that arise in stochastic optimal power flow problems and that contain up to 2 million total variables and 4,000 coupling variables. We find that ADMM-GMRES is nearly an order of magnitude faster than Schur complement decomposition. Moreover, we demonstrate that the approach is robust to the selection of the augmented Lagrangian penalty parameter, which is a key advantage over the direct use of ADMM.

97 MATHEMATICS AND COMPUTING↗