Search NASA⌕ Search

SEARCH · Search NASA

Results for “penalty parameter”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Inverse determination of the penalty parameter in penalized weighted least-squares algorithm for noise reduction of low-dose CBCT

Purpose: A statistical projection restoration algorithm based on the penalized weighted least-squares (PWLS) criterion can substantially improve the image quality of low-dose CBCT images. The performance of PWLS is largely dependent on the choice of the penalty parameter. Previously, the penalty parameter was chosen empirically by trial and error. In this work, the authors developed an inverse technique to calculate the penalty parameter in PWLS for noise suppression of low-dose CBCT in image guided radiotherapy (IGRT). Methods: In IGRT, a daily CBCT is acquired for the same patient during a treatment course. In this work, the authors acquired the CBCT with a high-mAs protocol for the first session and then a lower mAs protocol for the subsequent sessions. The high-mAs projections served as the goal (ideal) toward, which the low-mAs projections were to be smoothed by minimizing the PWLS objective function. The penalty parameter was determined through an inverse calculation of the derivative of the objective function incorporating both the high and low-mAs projections. Then the parameter obtained can be used for PWLS to smooth the noise in low-dose projections. CBCT projections for a CatPhan 600 and an anthropomorphic head phantom, as well as for a brain patient, were used to evaluate the performance of the proposed technique. Results: The penalty parameter in PWLS was obtained for each CBCT projection using the proposed strategy. The noise in the low-dose CBCT images reconstructed from the smoothed projections was greatly suppressed. Image quality in PWLS-processed low-dose CBCT was comparable to its corresponding high-dose CBCT. Conclusions: A technique was proposed to estimate the penalty parameter for PWLS algorithm. It provides an objective and efficient way to obtain the penalty parameter for image restoration algorithms that require predefined smoothing parameters.

61 RADIATION PROTECTION AND DOSIMETRY↗

Robust and Simple ADMM Penalty Parameter Selection

We present a new method for online selection of the penalty parameter for the alternating direction method of multipliers (ADMM) algorithm. ADMM is a widely used method for solving a range of optimization problems, including those that arise in signal and image processing. In its standard form, ADMM includes a scalar hyperparameter, known as the penalty parameter, which usually has to be tuned to achieve satisfactory empirical convergence. In this work, we develop a framework for analyzing the ADMM algorithm applied to a quadratic problem as an affine fixed point iteration. Using this framework, we develop a new method for automatically tuning the penalty parameter by detecting when it has become too large or small. We analyze this and several other methods with respect to their theoretical properties, i.e., robustness to problem transformations, and empirical performance on several optimization problems. Our proposed algorithm is based on a theoretical framework with clear, explicit assumptions and approximations, is theoretically covariant/invariant to problem transformations, is simple to implement, and exhibits competitive empirical performance.

42 ENGINEERING↗

Boundary-consistent B-spline filtering schemes and application to high-fidelity simulations of turbulence

A filtering operation, based on B-spline discretizations, is introduced to target weakly growing mesh-scale oscillations that can arise in high-fidelity turbulence simulations. This is a spectral regularization that can be described using the singular values of a banded matrix operator, with the filtering strength set by a scalar- or vector-valued penalty parameter. The penalty parameter can be specified though it can also be advantageously selected to minimize the generalized cross validation (GCV) measure of distance between the pre- and post-filtered solutions. Efficient algorithms are developed to compute both the scalar and vector penalty parameters. The B-spline filter has a sharper localization to high-wavenumber than compact or explicit filters of the same stencil width and is demonstrated for solutions of the Burgers' equation, decaying Burgers' turbulence, and compressible Navier–Stokes turbulent channel flow. Furthermore, these simulations confirm the scheme's numerical stability and ability to narrowly target the high wavenumber components of numerical solutions. An advantage over finite-difference filters is that these B-spline filters are stable on bounded domains and even preserve formal order of accuracy.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A provably stable numerical method for the anisotropic diffusion equation in confined magnetic fields

We present a novel numerical method for solving the anisotropic diffusion equation in magnetic fields confined to a periodic box which is accurate and provably stable. We derive energy estimates of the solution of the continuous initial boundary value problem. A discrete formulation is presented using operator splitting in time with the summation by parts finite difference approximation of spatial derivatives for the perpendicular diffusion operator. Weak penalty procedures are derived for implementing both boundary conditions and parallel diffusion operator obtained by field line tracing. We prove that the fully-discrete approximation is unconditionally stable. Discrete energy estimates are shown to match the continuous energy estimate given the correct choice of penalty parameters. A nonlinear penalty parameter is shown to provide an effective method for tuning the parallel diffusion penalty and significantly minimises rounding errors. Several numerical experiments, using manufactured solutions, the “NIMROD benchmark” problem and a single island problem, are presented to verify numerical accuracy, convergence, and asymptotic preserving properties of the method. Finally, we present a magnetic field with chaotic regions and islands and show the contours of the anisotropic diffusion equation reproduce key features in the field.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

An adaptive sampling augmented Lagrangian method for stochastic optimization with deterministic constraints

The primary goal of this paper is to provide an efficient solution algorithm based on the augmented Lagrangian framework for optimization problems with a stochastic objective function and deterministic constraints. Our main contribution is combining the augmented Lagrangian framework with adaptive sampling, resulting in an efficient optimization methodology validated with practical examples. To achieve the presented efficiency, here we consider inexact solutions for the augmented Lagrangian subproblems, and through an adaptive sampling mechanism, we control the variance in the gradient estimates. Furthermore, we analyze the theoretical performance of the proposed scheme by showing equivalence to a gradient descent algorithm on a Moreau envelope function, and we prove sublinear convergence for convex objectives and linear convergence for strongly convex objectives with affine equality constraints. The worst-case sample complexity of the resulting algorithm, for an arbitrary choice of penalty parameter in the augmented Lagrangian function, is $\mathscr{O}$(ϵ -3-δ ) , where ϵ > 0 is the expected error of the solution and δ > 0 is a user-defined parameter. If the penalty parameter is chosen to be $\mathscr{O}$(ϵ -1 ), we demonstrate that the result can be improved to $\mathscr{O}$(ϵ -2 ) , which is competitive with the other methods employed in the literature. Moreover, if the objective function is strongly convex with affine equality constraints, we obtain $\mathscr{O}$(ϵ -1 log(1/ϵ)) complexity. Finally, we empirically verify the performance of our adaptive sampling augmented Lagrangian framework in machine learning optimization and engineering design problems, including topology optimization of a heat sink with environmental uncertainty.

97 MATHEMATICS AND COMPUTING↗

An Adaptive Multiparameter Penalty Selection Method for Multiconstraint and Multiblock ADMM

This work presents a new method for online selection of multiple penalty parameters for the alternating direction method of multipliers (ADMM) algorithm applied to optimization problems with multiple constraints or functions with block matrix components. ADMM is widely used for solving constrained optimization problems in a variety of fields, including signal and image processing. Implementations of ADMM often utilize a single hyperparameter, referred to as the penalty parameter, which needs to be tuned to control the rate of convergence. However, in problems with multiple constraints, ADMM may demonstrate slow convergence regardless of penalty parameter selection due to scale differences between constraints. Accounting for scale differences between constraints to improve convergence in these cases requires introducing a penalty parameter for each constraint. The proposed method is able to adaptively account for differences in scale between constraints, providing robustness with respect to problem transformations and initial selection of penalty parameters. It is also simple to understand and implement. Our numerical experiments demonstrate that the proposed method performs favorably compared to a variety of existing penalty parameter selection methods.

97 MATHEMATICS AND COMPUTING↗

A Reinforcement Learning Approach to Parameter Selection for Distributed Optimal Power Flow

With the increasing penetration of distributed energy resources, distributed optimization algorithms have attracted significant attention for power systems applications due to their potential for superior scalability, privacy, and robustness to a single point-of-failure. The Alternating Direction Method of Multipliers (ADMM) is a popular distributed optimization algorithm; however, its convergence performance is highly dependent on the selection of penalty parameters, which are usually chosen heuristically. In this work, we use reinforcement learning (RL) to develop an adaptive penalty parameter selection policy for alternating current optimal power flow (ACOPF) problem solved via ADMM with the goal of minimizing the number of iterations until convergence. We train our RL policy using deep Q-learning and show that this policy can result in significantly accelerated convergence (up to a 59% reduction in the number of iterations compared to existing, curvatureinformed penalty parameter selection methods). Furthermore, we show that our RL policy demonstrates promise for generalizability, performing well under unseen loading schemes as well as under unseen losses of lines and generators (up to a 50% reduction in iterations). This work thus provides a proof-of-concept for using RL for parameter selection in ADMM for power systems applications.

alternating current optimal power flow↗

Noise Reduction in Low-Dose X-Ray Fluoroscopy for Image-Guided Radiation Therapy

Purpose: To improve the quality of low-dose X-ray fluoroscopic images using statistics-based restoration algorithm so that the patient fluoroscopy can be performed with reduced radiation dose. Method and Materials: Noise in the low-dose fluoroscopy was suppressed by temporal and spatial filtering. The temporal correlation among neighboring frames was considered by the Karhunen-Loeve (KL) transform (i.e., principal component analysis). After the KL transform, the selected neighboring frames of fluoroscopy were decomposed to uncorrelated and ordered principal components. For each KL component, a penalized weighted least-squares (PWLS) objective function was constructed to restore the ideal image. The penalty was chosen as anisotropic quadratic, and the penalty parameter in each KL component was inversely proportional to its corresponding eigenvalue. Smaller KL eigenvalue is associated with the KL component of lower signal-to-noise ratio (SNR), and a larger penalty parameter should be used for such KL component. The low-dose fluoroscopic images were acquired using a Varian Acuity simulator. A quality assurance phantom and an anthropomorphic chest phantom were used to evaluate the presented algorithm. Results: In the images restored by the proposed KL domain PWLS algorithm, noise is greatly suppressed, whereas fine structures are well preserved. Average improvement rate of SNR is 75% among selected regions of interest. Comparison studies with traditional techniques, such as the mean and median filters, show that the proposed algorithm is advantageous in terms of structure preservation. Conclusions: The proposed noise reduction algorithm can significantly improve the quality of low-dose X-ray fluoroscopic image and allows for dose reduction in X-ray fluoroscopy.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Nonlinear convergence in contact mechanics: Immersed boundary finite volume

In this report we present an immersed boundary finite volume (IBM) method for simulating quasistatic contact mechanics of linearly elastic domains at small strains. In IBM, all external boundaries and internal contacts of an object are represented by embedded surfaces inside a Cartesian mesh, which need not conform to the grid lines. The contact constraints consist of the non-penetrability condition and Coulomb’s friction law, which are discretized using special interpolation stencils and enforced via penalty parameters. The resulting nonlinear system depends on displacement unknowns only. To solve it, we use the Newton method but find that it diverges frequently. To understand the divergence pattern, we analyze a simplified 2-cell problem and show that the global convergence of Newton cannot be ensured for any choice of penalty parameters. We thus propose a modified Newton solver, which guarantees convergence for the 2-cell problem and is numerically verified to converge for all the challenging simulations considered herein. While both 1 st - and 2 nd -order variants of IBM, in displacement unknowns, are proposed, the modified Newton solver applies only to the 1 st -order variant.

42 ENGINEERING↗

A duality-based coupling of Cosserat crystal plasticity and phase field theories for modeling grain refinement

High-rate deformation processes of metals entail intense grain refinement and special attention needs to be paid to capture the evolution of microstructure. In this article, a new formulation for coupling Cosserat crystal plasticity and phase field is developed. A common approach is to penalize kinematic incompatibility between lattice orientation and displacement-based elastic rotation. However, this can lead to significant solution sensitivity to the penalty parameter, resulting in low accuracy and convergence rates. To address these issues, a duality-based formulation is developed which directly imposes the rotational kinematic compatibility. A weak inf-sup-based skew-symmetric stress projection is introduced to suppress instabilities present in the dual formulation. An additional least squares stabilization is introduced to suppress the spurious lattice rotation with a suitable parameter range derived analytically and validated numerically. The required high-order continuity is attained by the reproducing kernel approximation. It is observed that equal order displacement-rotation-phase field approximations are stable, which allows efficient employment of the same set of shape functions for all independent variables. The proposed formulation is shown to yield superior accuracy and convergence with marginal parameter sensitivity compared to the penalty-based approach and successfully captures the dominant rotational recrystallization mechanism including block dislocation structures and grain boundary migration.

42 ENGINEERING↗

Hybridized Methods for Quantum Simulation in the Interaction Picture

Conventional methods of quantum simulation involve trade-offs that limit their applicability to specific contexts where their use is optimal. In particular, the interaction picture simulation has been found to provide substantial asymptotic advantages for some Hamiltonians, but incurs prohibitive constant factors and is incompatible with methods like qubitization. We provide a framework that allows different simulation methods to be hybridized and thereby improve performance for interaction picture simulations over known algorithms. These approaches show asymptotic improvements over the individual methods that comprise them and further make interaction picture simulation methods practical in the near term. Physical applications of these hybridized methods yield a gate complexity scaling as log 2 ⁡ Λ in the electric cutoff Λ for the Schwinger Model and independent of the electron density for collective neutrino oscillations, outperforming the scaling for all current algorithms with these parameters. For the general problem of Hamiltonian simulation subject to dynamical constraints, these methods yield a query complexity independent of the penalty parameter λ used to impose an energy cost on time-evolution into an unphysical subspace.

97 MATHEMATICS AND COMPUTING↗

A Smoothed Augmented Lagrangian Framework for Convex Optimization with Nonsmooth Constraints

Augmented Lagrangian (AL) methods have proven remarkably useful in solving optimization problems with complicated constraints. The last decade has seen the development of overall complexity guarantees for inexact AL variants. Yet, a crucial gap persists in addressing nonsmooth convex constraints. To this end, we present a smoothed augmented Lagrangian (AL) framework where nonsmooth terms are progressively smoothed with a smoothing parameter $\eta _k$ . The resulting AL subproblems are $\eta _k$ -smooth, allowing for leveraging accelerated schemes. By a careful selection of the inexactness level $\epsilon _k$ (for inexact subproblem resolution), the penalty parameter $\rho _k$ , and smoothing parameter $\eta _k$ at epoch k, we derive rate and complexity guarantees of $\tilde{\mathcal {O}}(1/{\varepsilon }^{3/2})$ and $\tilde{\mathcal {O}}(1/{\varepsilon })$ in convex and strongly convex regimes for computing an ${\varepsilon }$ -optimal solution, when $\rho _k$ increases at a geometric rate, a significant improvement over the best available guarantees for AL schemes for convex programs with nonsmooth constraints. Analogous guarantees are developed for settings with $\rho _k = \rho$ as well as $\eta _k = \eta$ . Preliminary numerics on a fused Lasso problem display promise.

augmented Lagrangian↗

An Augmented Lagrangian Method for a Class of Inverse Quadratic Programming Problems

We consider an inverse quadratic programming (QP) problem in which the parameters in the objective function of a given QP problem are adjusted as little as possible so that a known feasible solution becomes the optimal one. We formulate this problem as a minimization problem with a positive semidefinite cone constraint and its dual is a linearly constrained semismoothly differentiable (SC{sup 1}) convex programming problem with fewer variables than the original one. We demonstrate the global convergence of the augmented Lagrangian method for the dual problem and prove that the convergence rate of primal iterates, generated by the augmented Lagrange method, is proportional to 1/r, and the rate of multiplier iterates is proportional to 1/{radical}r, where r is the penalty parameter in the augmented Lagrangian. As the objective function of the dual problem is a SC{sup 1} function involving the projection operator onto the cone of symmetrically semi-definite matrices, the analysis requires extensive tools such as the singular value decomposition of matrices, an implicit function theorem for semismooth functions, and properties of the projection operator in the symmetric-matrix space. Furthermore, the semismooth Newton method with Armijo line search is applied to solve the subproblems in the augmented Lagrange approach, which is proven to have global convergence and local quadratic rate. Finally numerical results, implemented by the augmented Lagrangian method, are reported.

97 MATHEMATICS AND COMPUTING↗

Scalable Preconditioning of Block-Structured Linear Algebra Systems using ADMM

In this paper, we study the solution of block-structured linear algebra systems arising in optimization by using iterative solution techniques. These systems are the core computational bottleneck of many problems of interest such as parameter estimation, optimal control, network optimization, and stochastic programming. Our approach uses a Krylov solver (GMRES) that is preconditioned with an alternating method of multipliers (ADMM). We show that this ADMM-GMRES approach overcomes well-known scalability issues of Schur complement decomposition in problems that exhibit a high degree of coupling. The effectiveness of the approach is demonstrated using linear systems that arise in stochastic optimal power flow problems and that contain up to 2 million total variables and 4,000 coupling variables. We find that ADMM-GMRES is nearly an order of magnitude faster than Schur complement decomposition. Moreover, we demonstrate that the approach is robust to the selection of the augmented Lagrangian penalty parameter, which is a key advantage over the direct use of ADMM.

97 MATHEMATICS AND COMPUTING↗

Goal-oriented optimal design of experiments for large-scale Bayesian linear inverse problems

We develop a framework for goal-oriented optimal design of experiments (GOODE) for large-scale Bayesian linear inverse problems governed by PDEs. This framework differs from classical Bayesian optimal design of experiments (ODE) in the following sense: we seek experimental designs that minimize the posterior uncertainty in the experiment end-goal, e.g. a quantity of interest (QoI), rather than the estimated parameter itself. This is suitable for scenarios in which the solution of an inverse problem is an intermediate step and the estimated parameter is then used to compute a QoI. In such problems, a GOODE approach has two benefits: the designs can avoid wastage of experimental resources by a targeted collection of data, and the resulting design criteria are computationally easier to evaluate due to the often low-dimensionality of the QoIs. We present two modified design criteria, A-GOODE and D-GOODE, which are natural analogues of classical Bayesian A- and D-optimal criteria. We analyze the connections to other ODE criteria, and provide interpretations for the GOODE criteria by using tools from information theory. Then, we develop an efficient gradient-based optimization framework for solving the GOODE optimization problems. Additionally, we present comprehensive numerical experiments testing the various aspects of the presented approach. The driving application is the optimal placement of sensors to identify the source of contaminants in a diffusion and transport problem. As a result, we enforce sparsity of the sensor placements using an L1-norm penalty approach, and propose a practical strategy for specifying the associated penalty parameter.

97 MATHEMATICS AND COMPUTING↗

Enhancing the cooling performance of thermocouples: a power-constrained topology optimization procedure

Abstract Heat pumping through thermoelectric devices has many advantages over traditional cooling. However, their current efficiency is a limiting factor in their implementation. In this paper, we approach the non-convex topology optimization of thermoelectrical elements for cooling applications through the method of moving asymptotes (MMA) to improve their cooling capabilities per watt usage. The optimization problem is defined for a given power budget, aiming for the minimum temperature with a known heat pumping need. The introduction of power as a constraint justifies the introduction of the voltage gradient across the thermocouple as a design variable to maintain the thermoelectrical device in its optimum power-to-heat extraction ratio. To better understand the convergence of this non-convex problem, we present a two-variable analytical thermoelectric optimization model. This example provides information on how to select the penalty parameters used to scale the three material coefficients involved in the problem to obtain lower objective values and better convergence using MMA. The analytical model shows the non-convexity of the problem and provides the recommendation to use penalization coefficients of the form $$p_k=p_{\sigma }>p_{\alpha }=1$$ p k = p σ > p α = 1 for the thermal conductivity, electrical conductivity, and Seebeck coefficients. We tested these penalization coefficients through optimizations of a model based on the 1MC10-031 commercial thermoelectric-cooler (TEC) using the finite element method (FEM). These penalization coefficients provided local minima without the need for volume constraints. With this procedure, we found designs that provided temperatures close to 10 degrees lower using 60% less semiconductor material volume compared to the initial design.

Gutiérrez, G. Reales↗

An infeasible-start framework for convex quadratic optimization, with application to constraint-reduced interior-point and other methods

A framework is proposed for solving general convex quadratic programs (CQPs) from an infeasible starting point by invoking an existing feasible-start algorithm tailored for inequality-constrained CQPs. The central tool is an exact penalty function scheme equipped with a penalty-parameter updating rule. The feasible-start algorithm merely has to satisfy certain general requirements, and so is the updating rule. Under mild assumptions, the framework is proved to converge on CQPs with both inequality and equality constraints and, at a negligible additional cost per iteration, produces an infeasibility certificate, together with a feasible point for an (approximately) ℓ 1 -least relaxed feasible problem, when the given problem does not have a feasible solution. The framework is applied to a feasible-start constraint-reduced interior-point algorithm previously proved to be highly performant on problems with many more inequality constraints than variables (“imbalanced”). Numerical comparison with popular codes (OSQP, qpOASES, MOSEK) is reported on both randomly generated problems and support-vector machine classifier training problems. The results show that the former typically outperforms the latter on imbalanced problems. Finally, application of the proposed infeasible-start framework to other feasible-start algorithms is briefly considered, and is tested on a simplex iteration.

97 MATHEMATICS AND COMPUTING↗

$L^1$ penalization of volumetric dose objectives in optimal control of PDEs

This work is concerned with a class of PDE-constrained optimization problems that are motivated by an application in radiotherapy treatment planning. Here the primary design objective is to minimize the volume where a functional of the state violates a prescribed level, but prescribing these levels in the form of pointwise state constraints leads to infeasible problems. We therefore propose an alternative approach based on L 1 penalization of the violation that is also applicable when state constraints are infeasible. We establish well-posedness of the corresponding optimal control problem, derive first-order optimality conditions, discuss convergence of minimizers as the penalty parameter tends to infinity, and present a semismooth Newton method for their efficient numerical solution. Finally, the performance of this method for a model problem is illustrated and contrasted with an alternative approach based on (regularized) state constraints.

97 MATHEMATICS AND COMPUTING↗