Search NASA⌕ Search

SEARCH · Search NASA

Results for “Augmented Lagrangian method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

An adaptive sampling augmented Lagrangian method for stochastic optimization with deterministic constraints

The primary goal of this paper is to provide an efficient solution algorithm based on the augmented Lagrangian framework for optimization problems with a stochastic objective function and deterministic constraints. Our main contribution is combining the augmented Lagrangian framework with adaptive sampling, resulting in an efficient optimization methodology validated with practical examples. To achieve the presented efficiency, here we consider inexact solutions for the augmented Lagrangian subproblems, and through an adaptive sampling mechanism, we control the variance in the gradient estimates. Furthermore, we analyze the theoretical performance of the proposed scheme by showing equivalence to a gradient descent algorithm on a Moreau envelope function, and we prove sublinear convergence for convex objectives and linear convergence for strongly convex objectives with affine equality constraints. The worst-case sample complexity of the resulting algorithm, for an arbitrary choice of penalty parameter in the augmented Lagrangian function, is $\mathscr{O}$(ϵ -3-δ ) , where ϵ > 0 is the expected error of the solution and δ > 0 is a user-defined parameter. If the penalty parameter is chosen to be $\mathscr{O}$(ϵ -1 ), we demonstrate that the result can be improved to $\mathscr{O}$(ϵ -2 ) , which is competitive with the other methods employed in the literature. Moreover, if the objective function is strongly convex with affine equality constraints, we obtain $\mathscr{O}$(ϵ -1 log(1/ϵ)) complexity. Finally, we empirically verify the performance of our adaptive sampling augmented Lagrangian framework in machine learning optimization and engineering design problems, including topology optimization of a heat sink with environmental uncertainty.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Load Shedding for Voltage Regulation With Probabilistic Agent Compliance

With the increased observability and controllability of distribution systems, the share of behind-the-meter systems is trending upwards rapidly. As a consequence, the impact of human behaviors on system performance can no longer be ignored and should be reflected in the energy management system models. In this paper, we discuss the problem of distribution system voltage control by active power curtailment where the agent compliance of the load curtailment signal is probabilistic. We discuss the modeling of the optimal voltage control problem with probabilistic agent compliance as a chance-constrained optimization problem, its tractable safe approximation using convex restriction, and a scenario-based mixed-integer reformulation as well as the associated solution method based on augmented Lagrangian method. The numerical simulation on IEEE test system validates the effectiveness of the proposed approach in obtaining high-quality feasible load curtailment signal with low computational cost, which makes it a viable tool for real time decision making.

augmented Lagrangian method↗
DOE OSTI2023

Quantum Annealing with Inequality Constraints: The Set Cover Problem

Abstract Quantum annealing is a promising method for solving hard optimization problems by transforming them into quadratic unconstrained binary optimization (QUBO) problems. However, when constraints are involved, particularly multiple inequality constraints, incorporating them into the objective function poses challenges. In this paper, the authors present two novel approaches for solving problems with multiple inequality constraints on a quantum annealer and apply them to the set cover problem (SCP). The first approach uses the augmented Lagrangian method to represent the constraints, while the second approach employs a higher‐order binary optimization (HUBO) formulation. The experiments show that both approaches outperform the standard approach for solving the SCP on the D‐Wave Advantage quantum annealer. The HUBO formulation performs slightly better than the augmented Lagrangian method in solving the SCP, but its scalability in terms of embeddability in the quantum chip is worse. The results demonstrate that the proposed augmented Lagrangian and HUBO methods can successfully implement a large number of inequality constraints, making them applicable to a broad range of constrained problems beyond the SCP.

Djidjev, Hristo N.↗
DOE OSTI2022

Posterior Regularized Bayesian Neural Network incorporating soft and hard knowledge constraints

Neural Networks (NNs) have been widely used in supervised learning due to their ability to model complex nonlinear patterns, often presented in high-dimensional data such as images and text. However, traditional NNs often lack the ability for uncertainty quantification. Bayesian NNs (BNNS) could help measure the uncertainty by considering the distributions of the NN model parameters. Besides, domain knowledge is commonly available and could improve the performance of BNNs if it can be appropriately incorporated. In this work, we propose a novel Posterior-Regularized Bayesian Neural Network (PR-BNN) model by incorporating different types of knowledge constraints, such as the soft and hard constraints, as a posterior regularization term. Furthermore, we propose to combine the augmented Lagrangian method and the existing BNN solvers for efficient inference. Furthermore, the experiments in simulation and two case studies about aviation landing prediction and solar energy output prediction have shown the knowledge constraints and the performance improvement of the proposed model over traditional BNNs without the constraints.

14 SOLAR ENERGY↗
DOE OSTI2026

Using Filter Methods to Guide Convergence for ADMM, with Applications to Nonnegative Matrix Factorization Problems

Nonconvex, nonlinear optimization problems arise naturally in parameter fitting and machine learning. While augmented Lagrangian methods have demonstrated robust convergence for classes of these problems, their convergence for block updates has been relatively unexplored outside of the context of the alternating direction method of multipliers (ADMM). ADMM has seen extensive use in these applications, but may exhibit uncertain convergence behavior in many practical nonconvex settings, and struggles with general nonlinear constraints. In contrast, filter methods have proved effective in enforcing convergence for sequential quadratic programming methods and interior point methods with feasibility criteria. We develop an ADMM-filter method for highly nonlinear and nonconvex problems. Here, we show convergence under mild assumptions for several types of coordinate descent schemes, and demonstrate our algorithm on nonnegative matrix factorization and completion problems in imaging and chemical spectrum analysis.

Nonconvex optimization↗
DOE OSTI2024

An Empirical Quantile Estimation Approach for Chance-Constrained Nonlinear Optimization Problems

We investigate an empirical quantile estimation approach to solve chance-constrained nonlinear optimization problems. Our approach is based on the reformulation of the chance constraint as an equivalent quantile constraint to provide stronger signals on the gradient. In this approach, the value of the quantile function is estimated empirically from samples drawn from the random parameters, and the gradient of the quantile function is estimated via a finite-difference approximation on top of the quantile-function-value estimation. We establish a convergence theory of this approach within the framework of an augmented Lagrangian method for solving general nonlinear constrained optimization problems. The foundation of the convergence analysis is a concentration property of the empirical quantile process, and the analysis is divided based on whether or not the quantile function is differentiable. In contrast to the sampling-and-smoothing approach used in the literature, the method developed in this paper does not involve any smoothing function and hence the quantile-function gradient approximation is easier to implement and there are less accuracy-control parameters to tune. Furthermore, we demonstrate the effectiveness of this approach and compare it with a smoothing method for the quantile-gradient estimation. Numerical investigation shows that the two approaches are competitive for certain problem instances.

Applied Probability↗
DOE OSTI2025

Extended FFT-based micromechanical formulation to consider general non-periodic boundary conditions

Here, this paper presents a new approach for applying non-periodic boundary conditions in the context of FFT-based methods to solve micromechanical problems in heterogeneous solids. The domain of the original problem is extended to satisfy the periodicity requirements at the boundary of the extended domain. The velocity constraint on the boundary of the original domain is replaced by a corresponding constraint on the velocity gradient in the extended volume, and a two-level augmented Lagrangian method is used to enforce the constraint. The proposed method is implemented as an extension of the large-strain elasto-viscoplastic FFT-based (LS-EVPFFT) model of Zecevic et al. (2022). The proposed method is verified in the cases of fully imposed velocity boundary conditions and mixed velocity/traction-free boundary conditions. The accuracy and convergence of the method are studied next, followed by applications to bending and indentation of polycrystals that illustrate the extended capabilities of the proposed formulation.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Stress‐constrained topology optimization of structures subjected to nonproportional loading

Abstract This work considers the topology optimization of hyperelastic structures for maximum stiffness (minimum compliance) subject to constraints on their volume and maximum stress. In contrast to almost all previous works, we subject the structures to nonproportional loading, wherein the maximum stress does not necessarily occur at the final load step. As such, the stress is constrained at each load step. The augmented Lagrangian method is used to formulate the optimization problem with its many constraints. In numerical examples, we investigate different load trajectories for the same terminal load and compare the optimized designs and their performances. The results show the importance of considering the entire load trajectory as the load history significantly influences the optimized designs.

42 ENGINEERING↗
DOE OSTI2024

Pressure stability in explicitly coupled simulations of poromechanics with application to CO 2 sequestration

We study in detail the pressure stabilizing effects of the non-iterated fixed-stress splitting in poromechanical problems which are nearly undrained and incompressible. When applied in conjunction with a spatial discretization which does not satisfy the discrete inf–sup condition, namely a mixed piecewise linear–piecewise constant spatial discretization, the explicit fixed-stress scheme can have a pressure stabilizing effect in transient problems. This effect disappears, however, upon time step refinement or the attainment of steady state. The interpretation of the scheme as an Augmented Lagrangian method similar to Uzawa iteration for incompressible flow helps explain these results. Moreover, due to the slowly evolving solution within undrained seal regions, we show that the explicit fixed-stress scheme requires very large time steps to reveal its pressure stabilizing effect in examples of geologic CO 2 sequestration. We note that large time steps can result in large errors in drained regions, such as the aquifer or reservoir regions of these examples, and can prevent convergence of nonlinear solvers in the case of multiphase flows, which can make the explicit scheme an unreliable source of pressure stabilization. We conclude by demonstrating that pressure jump stabilization is as effective in the explicit fixed-stress setting as in the fully implicit setting for undrained problems, while maintaining the stability and convergence of the fixed-stress split for drained problems.

58 GEOSCIENCES↗
DOE OSTI2023

Nonlinear, real-time optimization for actuator management in tokamaks

Experiments in DIII-D have been carried out to test a novel actuator management approach in tokamaks. Here, the actuator management scheme is posed as a nonlinear-optimization problem in which the actuator commands are calculated in real time according to the changing control priorities, plasma state, and actuator availability. Such optimization problem is solved using the augmented Lagrangian method, combined with a gradient projection method and a conjugate-gradient iteration algorithm. The algorithmic approach followed in this work does not depend on the particular control objectives or actuators considered, which facilitates its integration with other independently-designed control components within a plasma-control system. In addition, the actuator-management algorithm is able to handle the optimization problem in a computationally efficient manner, making it suitable for real-time implementations. Initial DIII-D results in the steady-state high-q min scenario have demonstrated the capabilities of the actuator manager to perform both simultaneous multiple mission and repurposing sharing, which will be required in ITER.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Near-ideal relaxed MHD in slab geometry

We investigate the solutions of the relaxed magnetohydrodynamic (MHD) model (RxMHD) of R. Dewar and Z. Qu. This model generalizes Taylor relaxation by including the ideal Ohm's law constraint using an augmented Lagrangian method, providing a pathway to extend the multi-region relaxed MHD (MRxMHD) model. We present the first numerical solution of the RxMHD model by Dewar and Qu, demonstrating that it is mathematically well-defined and computationally feasible for constructing MHD equilibria in slab geometry. We also show that a cross-field flow can exist without enforcing an arbitrary constraint on the angular momentum, as is done in the case of MRxMHD with flow. Our results also demonstrate the self-organization of fully relaxed regions during the optimization, which was an important motivation behind developing this model.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Posterior Regularized Bayesian Neural Network

Traditional NNs often lack the ability for uncertainty quantification. Bayesian NNs(BNNs) could help measure the confidence level by using distributions in NNs modeling. Besides, knowledge is commonly available and could improve the performance of BNNs if it can be properly incorporated. In this work, we propose a novel Posterior-Regularized BNN(PR-BNN) model by incorporating soft and hard constraints as a posterior regularization term. We also propose an augmented Lagrangian method and stochastic optimization algorithm for efficient updating via Monte Carlo sampling. The simulations and case studies for solar PV plants have shown the performance improvement of the proposed model over traditional BNNs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Posterior Regularized Bayesian Neural Network

Traditional NNs often lack the ability for uncertainty quantification. Bayesian NNs(BNNs) could help measure the confidence level by using distributions in NNs modeling. Besides, knowledge is commonly available and could improve the performance of BNNs if it can be properly incorporated. In this work, we propose a novel Posterior-Regularized BNN(PR-BNN) model by incorporating soft and hard constraints as a posterior regularization term. We also propose an augmented Lagrangian method and stochastic optimization algorithm for efficient updating via Monte Carlo sampling. The simulations and case studies in solar energy prediction have shown the performance improvement of the proposed model over traditional BNNs.

14 SOLAR ENERGY↗
DOE OSTI2023

ALESQP: An Augmented Lagrangian Equality-Constrained SQP Method for Optimization with General Constraints

Here we present a new algorithm for infinite-dimensional optimization with general constraints, called ALESQP. In short, ALESQP is an augmented Lagrangian method that penalizes inequality constraints and solves equality-constrained nonlinear optimization subproblems at every iteration. The subproblems are solved using a matrix-free trust-region sequential quadratic programming (SQP) method that takes advantage of iterative, i.e., inexact linear solvers, and is suitable for large-scale applications. A key feature of ALESQP is a constraint decomposition strategy that allows it to exploit problem-specific variable scalings and inner products. We analyze convergence of ALESQP under different assumptions. We show that strong accumulation points are stationary. Consequently, in finite dimensions ALESQP converges to a stationary point. In infinite dimensions we establish that weak accumulation points are feasible in many practical situations. Under additional assumptions we show that weak accumulation points are stationary. We present several infinite-dimensional examples where ALESQP shows remarkable discretization-independent performance in all of its iterative components, requiring a modest number of iterations to meet constraint tolerances at the level of machine precision. Also, we demonstrate a fully matrix-free solution of an infinite-dimensional problem with nonlinear inequality constraints.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A Smoothed Augmented Lagrangian Framework for Convex Optimization with Nonsmooth Constraints

Augmented Lagrangian (AL) methods have proven remarkably useful in solving optimization problems with complicated constraints. The last decade has seen the development of overall complexity guarantees for inexact AL variants. Yet, a crucial gap persists in addressing nonsmooth convex constraints. To this end, we present a smoothed augmented Lagrangian (AL) framework where nonsmooth terms are progressively smoothed with a smoothing parameter $\eta _k$ . The resulting AL subproblems are $\eta _k$ -smooth, allowing for leveraging accelerated schemes. By a careful selection of the inexactness level $\epsilon _k$ (for inexact subproblem resolution), the penalty parameter $\rho _k$ , and smoothing parameter $\eta _k$ at epoch k, we derive rate and complexity guarantees of $\tilde{\mathcal {O}}(1/{\varepsilon }^{3/2})$ and $\tilde{\mathcal {O}}(1/{\varepsilon })$ in convex and strongly convex regimes for computing an ${\varepsilon }$ -optimal solution, when $\rho _k$ increases at a geometric rate, a significant improvement over the best available guarantees for AL schemes for convex programs with nonsmooth constraints. Analogous guarantees are developed for settings with $\rho _k = \rho$ as well as $\eta _k = \eta$ . Preliminary numerics on a fused Lasso problem display promise.

augmented Lagrangian↗
DOE OSTI2026

A robust framework for frictional fault contact in geological formations using a stabilized augmented Lagrangian approach

Numerical simulations are essential to evaluate the performance and safety of engineered subsurface systems such as geological carbon storage sites, enhanced geothermal fields, and oil and gas reservoirs. A key challenge lies in accurately modeling the frictional contact behavior along fault surfaces. This problem involves inequality constraints that arise from the physics of frictional slip, requiring specialized numerical methods to handle the resulting highly nonlinear and path-dependent behavior. Here, in this work, we address this challenge using an Augmented Lagrangian Method (ALM) implemented via the Uzawa algorithm. The formulation employs mixed finite element spaces, combining low-order piecewise linear displacements within the 3D domain cells with piecewise constant tractions defined on the fault surfaces. Furthermore, to ensure stability and satisfy the inf-sup condition, the discrete displacement space is enriched with face bubble functions on both sides of the contact interfaces. This approach offers several advantages over other stabilization techniques that rely on additional terms, and it integrates naturally in the Uzawa framework.

58 GEOSCIENCES↗
DOE OSTI2026

Hourglass control in staggered-grid hydrodynamics using virtual element stabilization techniques

Numerical simulations using the staggered-grid hydrodynamics (SGH) discretization suffer from hourglass instabilities. In this work, we develop a stabilization method to suppress the hourglass instabilities using techniques from the virtual element method (VEM). The stiffness matrix of the VEM consists of two terms: the consistency matrix which is rank deficient and the stability matrix. Here, we first show that in two dimensions and on general polygons, the stiffness matrix of the SGH is identical to the consistency matrix of the linear VEM for both the diffusion equation and the linear elasticity equation. These analyses explain the origin of the hourglass instabilities of the SGH discretization method, and establish a theoretical foundation for our proposed stabilization method by augmenting the stiffness matrix of the SGH discretization using the VEM stability matrix. Then, we present numerical examples using Lagrangian SGH simulations. The numerical experiments demonstrate that the proposed VEM stabilization method is effective at eliminating hourglass modes in the SGH discretization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

An adaptive stochastic sequential quadratic programming with differentiable exact augmented lagrangians

In this study, we consider solving nonlinear optimization problems with a stochastic objective and deterministic equality constraints. We assume for the objective that its evaluation, gradient, and Hessian are inaccessible, while one can compute their stochastic estimates by, for example, subsampling. We propose a stochastic algorithm based on sequential quadratic programming (SQP) that uses a differentiable exact augmented Lagrangian as the merit function. To motivate our algorithm design, we first revisit and simplify an old SQP method Lucidi developed for solving deterministic problems, which serves as the skeleton of our stochastic algorithm. Based on the simplified deterministic algorithm, we then propose a non-adaptive SQP for dealing with stochastic objective, where the gradient and Hessian are replaced by stochastic estimates but the stepsizes are deterministic and prespecified. Finally, we incorporate a recent stochastic line search procedure Paquette and Scheinberg into the non-adaptive stochastic SQP to adaptively select the random stepsizes, which leads to an adaptive stochastic SQP. The global "almost sure" convergence for both non-adaptive and adaptive SQP methods is established. Numerical experiments on nonlinear problems in CUTEst test set demonstrate the superiority of the adaptive algorithm.

97 MATHEMATICS AND COMPUTING↗