Search NASA⌕ Search

SEARCH · Search NASA

Results for “Quadratic convergence”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Sequential Linearization Method for Bound-Constrained Mathematical Programs with Complementarity Constraints

Here, we propose an algorithm for solving bound-constrained mathematical programs with complementarity constraints on the variables. Each iteration of the algorithm involves solving a linear program with complementarity constraints in order to obtain an estimate of the active set. The algorithm enforces descent on the objective function to promote global convergence to B-stationary points. We provide a convergence analysis and preliminary numerical results on a range of test problems. We also study the effect of fixing the active constraints in a bound-constrained quadratic program that can be solved on each iteration in order to obtain fast convergence.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Metric Learning to Accelerate Convergence of Operator Splitting Methods

Recent developments in machine learning have led to promising advances in accelerating the solution of constrained optimization problems. Increasing demand for real-time decision-making capabilities in applications such as artificial intelligence and optimal control has led to a variety of proposed strategies for learning to produce fast solutions to optimization problems. For example, recent works have shown that it is possible to accelerate the convergence of optimization algorithms by learning to select their parameters, such as gradient descent stepsizes. This work proposes a new approach, in which the underlying metric spaces of proximal operator splitting algorithms are learned to maximize convergence rate. While prior works in optimization theory have derived optimal metrics in simple cases, no such result exists for many practical problem forms including general Quadratic Programming (QP). This paper shows how differentiable optimization can enable the end-to-end learning of proximal metrics, enhancing the convergence of proximal algorithms for QP problems beyond what is possible based on known theory. Additionally, the results illustrate a strong connection between the learned proximal metrics and active constraints at the optima, leading to an interpretation in which the predicted proximal metrics can be viewed as a form of active set prediction.

King, Ethan [BATTELLE (PACIFIC NW LAB)]↗
DOE OSTI2025

Local practically safe extremum seeking with assignable rate of attractivity to the safe set

We present Assignably Safe Extremum Seeking (ASfES), an algorithm designed to minimize a measured, static objective function while maintaining a measured, static metric of safety (a control barrier function or CBF) to be positive in a practical sense. We ensure that for trajectories with safe initial conditions, the violation of safety can be made arbitrarily small through appropriately chosen design constants. We also guarantee an assignable “attractivity” rate: from unsafe initial conditions, the trajectories approach the safe set, in the sense of the measured CBF, at a rate no slower than a user-assigned rate. Similarly, from safe initial conditions, the trajectories approach the unsafe set, in the sense of the CBF, no faster than the assigned attractivity rate. The feature of assignable attractivity is not present in the semiglobal version of safe extremum seeking, where the semiglobality of convergence is achieved by slowing the adaptation. We also demonstrate local convergence of the parameter to a neighborhood of the minimum of a quadratic objective function constrained to the safe set with a linear CBF. The ASfES algorithm and analysis are multivariable, but we also extend the algorithm to a Newton-Based ASfES scheme which we show is only useful in the scalar case. The proven properties of the designs are illustrated through simulation examples.

42 ENGINEERING↗
DOE OSTI2020

Multiresolution GPC-Structured Control of a Single-Loop Cold-Flow Chemical Looping Testbed

Chemical looping is a near-zero emission process for generating power from coal. It is based on a multi-phase gas-solid flow and has extremely challenging nonlinear, multi-scale dynamics with jumps, producing large dynamic model uncertainty, which renders traditional robust control techniques, such as linear parameter varying H ∞ design, largely inapplicable. This process complexity is addressed in the present work through the temporal and the spatiotemporal multiresolution modeling along with the corresponding model-based control laws. Namely, the nonlinear autoregressive with exogenous input model structure, nonlinear in the wavelet basis, but linear in parameters, is used to identify the dominant temporal chemical looping process dynamics. The control inputs and the wavelet model parameters are calculated by optimizing a quadratic cost function using a gradient descent method. The respective identification and tracking error convergence of the proposed self-tuning identification and control schemes, the latter using the unconstrained generalized predictive control structure, is separately ascertained through the Lyapunov stability theorem. The rate constraint on the control signal in the temporal control law is then imposed and the control topology is augmented by an additional control loop with self-tuning deadbeat controller which uses the spatiotemporal wavelet riser dynamics representation. The novelty of this work is three-fold: (1) developing the self-tuning controller design methodology that consists in embedding the real-time tunable temporal highly nonlinear, but linearly parametrizable, multiresolution system representations into the classical rate-constrained generalized predictive quadratic optimal control structure, (2) augmenting the temporal multiresolution loop by a more complex spatiotemporal multiresolution self-tuning deadbeat control loop, and (3) demonstrating the effectiveness of the proposed methodology in producing fast recursive real-time algorithms for controlling highly uncertain nonlinear multiscale processes. The latter is shown through the data from the implemented temporal and augmented spatiotemporal solutions of a difficult chemical looping cold flow tracking control problem.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

A fast and accurate domain decomposition nonlinear manifold reduced order model

Here, this paper integrates nonlinear-manifold reduced order models (NM-ROMs) with domain decomposition (DD). NM ROMs approximate the full order model (FOM) state in a nonlinear-manifold by training a shallow, sparse autoencoder using FOM snapshot data. These NM-ROMs can be advantageous over linear-subspace ROMs (LS-ROMs) for problems with slowly decaying Kolmogorov n-width. However, the number of NM-ROM parameters that need to be trained scales with the size of the FOM. Moreover, for “extreme-scale” problems, the storage of high-dimensional FOM snapshots alone can make ROM training expensive. To alleviate the training cost, this paper applies DD to the FOM, computes NM-ROMs on each subdomain, and couples them to obtain a global NM-ROM. This approach has several advantages: Subdomain NM-ROMs can be trained in parallel, involve fewer parameters to be trained than global NM-ROMs, require smaller subdomain FOM dimensional training data, and can be tailored to subdomain specific features of the FOM. The shallow, sparse architecture of the autoencoder used in each subdomain NM-ROM allows application of hyper-reduction (HR), reducing the complexity caused by nonlinearity and yielding computational speedup of the NM-ROM. This paper provides the first application of NM-ROM (with HR) to a DD problem. In particular, this paper details an algebraic DD reformulation of the FOM, training a NM-ROM with HR for each sub domain, and a sequential quadratic programming (SQP) solver to evaluate the coupled global NM-ROM. Theoretical convergence results for the SQP method and a priori and a posteriori error estimates for the DD NM-ROM with HR are provided. The proposed DD NM-ROM with HR approach is numerically compared to a DD LS-ROM with HR on the 2D steady-state Burgers’ equation, showing an order of magnitude improvement in accuracy of the proposed DD NM-ROM over the DD LS-ROM.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Time-Varying Feedback Optimization for Quadratic Programs with Heterogeneous Gradient Step Sizes

Online feedback-based optimization has become a promising framework for real-time optimization and control of complex engineering systems. This tutorial paper surveys the recent advances in the field as well as provides novel convergence results for primal-dual online algorithms with heterogeneous step sizes for different elements of the gradient. The analysis is performed for quadratic programs and the approach is illustrated on applications for adaptive step-size and model-free online algorithms, in the context of optimal control of modern power systems.

gradient methods↗
DOE OSTI2026

FIRM: federated image reconstruction using multimodal tomographic data

Here, we propose a federated algorithm for reconstructing images using multimodal tomographic data sourced from dispersed locations, addressing the challenges of traditional unimodal approaches that are prone to noise and reduced image quality, as well as the limitations of centralized multimodal approaches that require extensive data transfer, leading to significant communication overhead, storage demands, and potential data privacy concerns. Our approach formulates a joint inverse optimization problem incorporating multimodality constraints and solves it in a federated framework through local gradient computations complemented by lightweight central operations, thereby ensuring data decentralization. Leveraging the connection between our federated algorithm and the quadratic penalty method, we introduce an adaptive step-size rule with guaranteed sublinear convergence. Numerical results demonstrate superior computational efficiency and improved image reconstruction quality compared to existing approaches.

federated algorithm↗
DOE OSTI2025

Application-specific machine-learned interatomic potentials: exploring the trade-off between DFT convergence, MLIP expressivity, and computational cost

Machine-learned interatomic potentials (MLIPs) are revolutionizing computational materials science and chemistry by offering an efficient alternative to ab initio molecular dynamics (MD) simulations. However, fitting high-quality MLIPs remains a challenging, time-consuming, and computationally intensive task where numerous trade-offs have to be considered, e.g., How much and what kind of atomic configurations should be included in the training set? Which level of ab initio convergence should be used to generate the training set? Which loss function should be used for fitting the MLIP? Which machine learning architecture should be used to train the MLIP? The answers to these questions significantly impact both the computational cost of MLIP training and the accuracy and computational cost of subsequent MLIP MD simulations. In this study, we use a configurationally diverse beryllium dataset and quadratic spectral neighbor analysis potential. We demonstrate that joint optimization of energy versus force weights, training set selection strategies, and convergence settings of the ab initio reference simulations, as well as model complexity can lead to a significant reduction in the overall computational cost associated with training and evaluating MLIPs. This opens the door to computationally efficient generation of high-quality MLIPs for a range of applications which demand different accuracy versus training and evaluation cost trade-offs.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Robust and Simple ADMM Penalty Parameter Selection

We present a new method for online selection of the penalty parameter for the alternating direction method of multipliers (ADMM) algorithm. ADMM is a widely used method for solving a range of optimization problems, including those that arise in signal and image processing. In its standard form, ADMM includes a scalar hyperparameter, known as the penalty parameter, which usually has to be tuned to achieve satisfactory empirical convergence. In this work, we develop a framework for analyzing the ADMM algorithm applied to a quadratic problem as an affine fixed point iteration. Using this framework, we develop a new method for automatically tuning the penalty parameter by detecting when it has become too large or small. We analyze this and several other methods with respect to their theoretical properties, i.e., robustness to problem transformations, and empirical performance on several optimization problems. Our proposed algorithm is based on a theoretical framework with clear, explicit assumptions and approximations, is theoretically covariant/invariant to problem transformations, is simple to implement, and exhibits competitive empirical performance.

42 ENGINEERING↗
DOE OSTI2024

E-PVT: enhanced position-velocity-time scheduler for computer-controlled optical finishing with comprehensive considerations of dynamics constraints, continuity and efficiency

Deterministic computer-controlled optical finishing is an essential approach for achieving high-quality optical surfaces. Its determinism and convergence rely heavily on precise and smooth motion control to guide the machine tool over an optical surface to correct residual errors. One widely supported and smooth motion control model is position-velocity-time (PVT), which employs piecewise cubic polynomials to describe positions. Our prior research introduced a PVT-based velocity scheduling method, demonstrating sub-nanometer level convergence in ion beam figuring (IBF) processes. However, three challenges remained. Firstly, this method relies on quadratic programming, resulting in computational intensiveness for dense tool paths. Secondly, the dynamics constraints and velocity and acceleration continuities are not comprehensively considered, limiting the full potential of PVT-based control. Thirdly, no compensation mechanism existed when dynamics constraints are exceeded. In this study, in response to these challenges, we proposed the Enhanced PVT (E-PVT) method, reducing the time complexity from O ( n 3 ) to O ( n ) while fully addressing dynamics constraints and continuities. A novel compensation method utilizing particle swarm optimization was proposed to address situations where dynamics constraints might be exceeded while maintaining the overall processing efficiency. Validation through simulation and experimentation confirmed the improved performance of E-PVT.

36 MATERIALS SCIENCE↗
DOE OSTI2023

How to detect lensing rotation

Gravitational lensing rotation of images is predicted to be negligible at linear order in density perturbations, but can be produced by the post-Born lens-lens coupling at second order. This rotation is somewhat enhanced for Cosmic Microwave Background (CMB) lensing due to the large source path length, but remains small and very challenging to detect directly by CMB lensing reconstruction alone. We show the rotation may be detectable at high significance as a cross-correlation signal between the curl reconstructed with Simons Observatory (SO) or CMB-S4 data, and a template constructed from quadratic combinations of large-scale structure (LSS) tracers. Equivalently, the lensing rotation-tracer-tracer bispectrum can also be detected, where LSS tracers considered include the CMB lensing convergence, galaxy density, and the Cosmic Infrared Background (CIB), or optimal combinations thereof. We forecast that an optimal combination of these tracers can probe post-Born rotation at the level of 5.7σ–6.1σ with SO and 13.6σ–14.7σ for CMB-S4, depending on whether standard quadratic estimators or maximum a posteriori iterative methods are deployed. We also show possible improvement up to 21.3σ using a CMB-S4 deep patch observation with polarization-only iterative lensing reconstruction. However, these cross-correlation signals have non-zero bias because the rotation template is quadratic in the tracers, and exists even if the lensing is rotation free. We estimate this bias analytically, and test it using simple null-hypothesis simulations to confirm that the bias remains subdominant to the rotation signal of interest. Detection and then measurement of the lensing rotation cross-spectrum is therefore a realistic target for future observations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Accelerating gradient descent and Adam via fractional gradients

Here we propose a class of novel fractional-order optimization algorithms. We define a fractional-order gradient via the Caputo fractional derivatives that generalizes integer-order gradient. We refer it to as the Caputo fractional-based gradient, and develop an efficient implementation to compute it. A general class of fractional-order optimization methods is then obtained by replacing integer-order gradients with the Caputo fractional-based gradients. To give concrete algorithms, we consider gradient descent (GD) and Adam, and extend them to the Caputo fractional GD (CfGD) and the Caputo fractional Adam (CfAdam). We demonstrate the superiority of CfGD and CfAdam on several large scale optimization problems that arise from scientific machine learning applications, such as ill-conditioned least squares problem on real-world data and the training of neural networks involving non-convex objective functions. Numerical examples show that both CfGD and CfAdam result in acceleration over GD and Adam, respectively. We also derive error bounds of CfGD for quadratic functions, which further indicate that CfGD could mitigate the dependence on the condition number in the rate of convergence and results in significant acceleration over GD.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A nonsmooth nonconvex optimization algorithm for two-stage optimization problems

An optimization algorithm for a group of nonsmooth nonconvex problems inspired by two-stage stochastic programming problems is proposed. The main challenges for these problems include (1) the problems lack the popular lower-type properties such as prox-regularity assumed in many nonsmooth nonconvex optimization algorithms, (2) the objective can not be analytically expressed and (3) the evaluation of function values and subgradients are computationally expensive. To address these challenges, this report first examines the properties that exist in many two-stage problems, specifically upper-C 2 objectives. Then, we show that quadratic penalty method for securityconstrained alternating current optimal power flow (SCACOPF) contingency problems can make the contingency solution functions upper-C 2 . Based on these observations, a simplified bundle algorithm that bears similarity to sequential quadratic programming (SQP) method is proposed. It is more efficient in implementation and computation compared to conventional bundle methods. Global convergence analysis of the algorithm is presented under novel and reasonable assumptions. The proposed algorithm therefore fills the gap of theoretical convergence for smoothed SCACOPF problems. The inconsistency that might arise in our treatment of the constraints are addressed through a penalty algorithm whose convergence analysis is also provided. Finally, theoretical capabilities and numerical performance of the algorithm are demonstrated through numerical examples.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A deterministic gradient-based approach to avoid saddle points

Abstract Loss functions with a large number of saddle points are one of the major obstacles for training modern machine learning (ML) models efficiently. First-order methods such as gradient descent (GD) are usually the methods of choice for training ML models. However, these methods converge to saddle points for certain choices of initial guesses. In this paper, we propose a modification of the recently proposed Laplacian smoothing gradient descent (LSGD) [Osher et al., arXiv:1806.06317 ], called modified LSGD (mLSGD), and demonstrate its potential to avoid saddle points without sacrificing the convergence rate. Our analysis is based on the attraction region, formed by all starting points for which the considered numerical scheme converges to a saddle point. We investigate the attraction region’s dimension both analytically and numerically. For a canonical class of quadratic functions, we show that the dimension of the attraction region for mLSGD is $\lfloor (n-1)/2\rfloor$ , and hence it is significantly smaller than that of GD whose dimension is $n-1$ .

Mathematics↗
DOE OSTI2023

A Sequential Quadratic Programming Algorithm for Nonsmooth Problems with Upper- \({\boldsymbol{\mathcal{C}^2}}\) Objective

An optimization algorithm for nonsmooth nonconvex constrained optimization problems with upper- \({\boldsymbol{\mathcal{C}^2}}\) objective functions is proposed and analyzed. Upper- \({\boldsymbol{\mathcal{C}^2}}\) is a weakly concave property that exists in difference of convex (DC) functions and arises naturally in many applications, particularly certain classes of solutions to parametric optimization problems e.g., recourse of stochastic programming and projection onto closed sets. The algorithm can be viewed as an extension of sequential quadratic programming (SQP) to nonsmooth problems with upper- \({\boldsymbol{\mathcal{C}^2}}\) objectives or a simplified bundle method. It is globally convergent with bounded algorithm parameters that are updated with a trust-region criterion. The algorithm handles general smooth constraints through linearization and uses a line search to ensure progress. The potential inconsistencies from the linearization of the constraints are addressed through a penalty method. In conclusion, the capabilities of the algorithm are demonstrated by solving both simple upper- \({\boldsymbol{\mathcal{C}^2}}\) problems and a real-world optimal power flow problem used in current power grid industry practices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

On the convergence of an IEQ-based first-order semi-discrete scheme for the Beris-Edwards system

We present a convergence analysis of an unconditionally energy-stable first-order semi-discrete numerical scheme designed for a hydrodynamic Q-tensor model, the so-called Beris-Edwards system, based on the Invariant Energy Quadratization Method (IEQ). The model consists of the Navier–Stokes equations for the fluid flow, coupled to the Q-tensor gradient flow describing the liquid crystal molecule alignment. By using the Invariant Energy Quadratization Method, we obtain a linearly implicit scheme, accelerating the computational speed. However, this introduces an auxiliary variable to replace the bulk potential energy and it is a priori unclear whether the reformulated system is equivalent to the Beris-Edward system. In this work, we prove stability properties of the scheme and show its convergence to a weak solution of the coupled liquid crystal system. We also demonstrate the equivalence of the reformulated and original systems in the weak sense.

Weber, Franziska↗
DOE OSTI2025

A unified funnel restoration SQP algorithm

We consider nonlinearly constrained optimization problems and discuss a generic double-loop framework consisting of basic algorithmic ingredients that unifies a broad range of nonlinear optimization solvers. This framework has been implemented in the open-source solver Uno, a Swiss Army knife-like C++ optimization framework that unifies many nonlinearly constrained nonconvex optimization solvers. We illustrate the framework with a sequential quadratic programming (SQP) algorithm that maintains an acceptable upper bound on the constraint violation, called a funnel, that is monotonically decreased to control the feasibility of the iterates. Infeasible quadratic subproblems are handled by a feasibility restoration strategy. Globalization is controlled by a line search or a trust-region method. We prove global convergence of the trust-region funnel SQP method, building on known results from filter methods. We implement the algorithm in Uno, and we provide extensive test results for the trust-region line-search funnel SQP on small CUTEst instances.

Kiessling, David [Katholieke Univ. Leuven, Heverle↗