Search NASA⌕ Search

SEARCH · Search NASA

Results for “zeroth-order methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Adaptive sampling quasi-Newton methods for zeroth-order stochastic optimization

Here, we consider unconstrained stochastic optimization problems with no available gradient information. Such problems arise in settings from derivative-free simulation optimization to reinforcement learning. We propose an adaptive sampling quasi-Newton method where we estimate the gradients using finite differences of stochastic function evaluations within a common random number framework. We develop modified versions of a norm test and an inner product quasi-Newton test to control the sample sizes used in the stochastic approximations and provide global convergence results to the neighborhood of a locally optimal solution. We present numerical experiments on simulation optimization problems to illustrate the performance of the proposed algorithm. When compared with classical zeroth-order stochastic gradient methods, we observe that our strategies of adapting the sample sizes significantly improve performance in terms of the number of stochastic function evaluations required.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Stochastic Trust-Region Algorithm in Random Subspaces with Convergence and Expected Complexity Analyses

Here, this work proposes a framework for large-scale stochastic derivative-free optimization (DFO) by introducing STARS, a trust-region method based on iterative minimization in random subspaces. This framework is both an algorithmic and theoretical extension of a random subspace derivative-free optimization (RSDFO) framework, and an algorithm for stochastic optimization with random models (STORM). Moreover, like RSDFO, STARS achieves scalability by minimizing interpolation models that approximate the objective in low-dimensional affine subspaces, thus significantly reducing per-iteration costs in terms of function evaluations and yielding strong performance on largescale stochastic DFO problems. The user-determined dimension of these subspaces, when the latter are defined, for example, by the columns of so-called Johnson-Lindenstrauss transforms, turns out to be independent of the dimension of the problem. For convergence purposes, inspired by the analyses of RSDFO and STORM, both a particular quality of the subspace and the accuracies of random function estimates and models are required to hold with sufficiently high, but fixed, probabilities. Using martingale theory under the latter assumptions, an almost sure global convergence of STARS to a first-order stationary point is shown, and the expected number of iterations required to reach a desired first-order accuracy is proved to be similar to that of STORM and other stochastic DFO algorithms, up to constants.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Randomized Federated Learning Methods for Nonsmooth, Nonconvex, and Hierarchical Optimization (Final Technical Report)

This final technical report summarizes the outcomes of a DOE-funded project on federated scientific machine learning (FL) under nonsmooth, nonconvex, and hierarchical optimization settings. The project develops new mathematical models, algorithms, and theoretical guarantees for decentralized stochastic, bilevel, and minimax optimization problems arising in DOE mission-relevant applications. A unified framework of randomized and zeroth-order federated optimization methods is introduced, providing provable convergence, communication efficiency, and sample-complexity guarantees. The report documents algorithmic design, theoretical analysis, and empirical validation of the proposed federated learning methods. The project also contributes to workforce development through graduate training and dissemination of results via publications and seminars.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Derivative-free optimization methods

In many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide derivative information. Such settings necessitate the use of methods for derivative-free, or zeroth-order, optimization. We provide a review and perspectives on developments in these methods, with an emphasis on highlighting recent developments and on unifying treatment of such problems in the non-linear optimization and machine learning literature. We categorize methods based on assumed properties of the black-box functions, as well as features of the methods. We first overview the primary setting of deterministic methods applied to unconstrained, non-convex optimization problems where the objective function is defined by a deterministic black-box oracle. We then discuss developments in randomized methods, methods that assume some additional structure about the objective (including convexity, separability and general non-smooth compositions), methods for problems where the output of the black-box oracle is stochastic, and methods for handling different types of constraints.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A zeroth-order active-space frozen-orbital embedding scheme for multireference calculations

Multireference computations of large-scale chemical systems are typically limited by the computational cost of quantum chemistry methods. In this work, we develop a zeroth-order active space embedding theory [ASET(0)], a simple and automatic approach for embedding any multireference dynamical correlation method based on a frozen-orbital treatment of the environment. ASET(0) is combined with the second-order multireference driven similarity renormalization group and tested on several benchmark problems, including the excitation energy of 1-octene and bond-breaking in ethane and pentyldiazene. Finally, we apply ASET(0) to study the singlet–triplet gap of p-benzyne and 9,10-anthracyne diradicals adsorbed on a NaCl surface. Our results show that despite its simplicity, ASET(0) is a powerful and sufficiently accurate embedding scheme applicable when the coupling between the fragment and the environment is in the weak to medium regime.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent

Despite the great achievements of the modern deep neural networks (DNNs), the vulnerability/robustness of state-of-the-art DNNs raises security concerns in many application domains requiring high reliability. Various adversarial attacks are proposed to sabotage the learning performance of DNN models. Among those, the black-box adversarial attack methods have received special attentions owing to their practicality and simplicity. Black-box attacks usually prefer less queries in order to maintain stealthy and low costs. However, most of the current black-box attack methods adopt the first-order gradient descent method, which may come with certain deficiencies such as relatively slow convergence and high sensitivity to hyper-parameter settings. In this paper, we propose a zeroth-order natural gradient descent (ZO-NGD) method to design the adversarial attacks, which incorporates the zeroth-order gradient estimation technique catering to the black-box attack scenario and the second-order natural gradient descent to achieve higher query efficiency. The empirical evaluations on image classification datasets demonstrate that ZO-NGD can obtain significantly lower model query complexities compared with state-of-the-art attack methods.

Zhao, Pu↗
DOE OSTI2022

Increased phase precision of spatial light modulators using irrational slopes: application to attosecond metrology

The ability of spatial light modulators (SLMs) to modify the amplitude and phase of light has proved them invaluable to the optics and photonics community. In many applications, the bit-depth of SLMs is a major limiting factor dictated by a digital processor. As a result, there is usually a compromise between refresh speed and bit-depth. Here, we present a method to increase the effective bit-depth of SLMs, which utilizes a linear slope, as is commonly applied to deal with the zeroth-order effect. This technique was tested using two interferometric transient absorption spectroscopy setups. Through the high harmonic generation in gases producing a train of attosecond pulses and harmonics from solids in the ultraviolet, two pulses are generated that interfere in the far field providing a measurement of the optical phase. An increase in the precision far beyond the limit dictated by the digital processor in the bit-depth was found.

Harrison, Geoffrey R. (ORCID:0000000152141866)↗
DOE OSTI2020

Optimal control of coupled quantum systems based on the first-order Magnus expansion: Application to multiple dipole-dipole-coupled molecular rotors

This paper presents a method for performing approximate optimal control simulations for quantum systems with multiple coupled degrees of freedom. In this work, the time evolution is simulated using the first-order Magnus expansion in the interaction picture, where the couplings between different degrees of freedom are treated as the perturbation. A numerical implementation procedure is presented that leverages upon pairwise couplings and the separability of the zeroth-order time evolution operator to achieve a reduced computational cost, which is analyzed with respect to the number of degrees of freedom. The formulation is compatible with gradient-free methods to optimize the control field, and a stochastic hill climbing algorithm is adopted for this purpose. As illustrations, optimal control simulations are performed for systems of two and three dipole-dipole-coupled molecular rotors under the influence of a control field. For the two-rotor system, the field is optimized to achieve either orientation or entanglement objectives. For the three-rotor system, the field is optimized either to orient all three rotors in the same direction or to orient one rotor in a particular direction while the other two rotors point in the opposite direction.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Ab initio study and assignment of electronic states in molecular RaCl

Radium compounds have attracted recently considerable attention due to both development of experimental techniques for high-precision laser spectroscopy of molecules with short-lived nuclei and amenability of certain radium compounds for direct cooling with lasers. Currently, radium monofluoride (RaF) is one of the most studied molecules among the radium compounds, both theoretically and recently also experimentally. Complementary studies of further diatomic radium derivatives are highly desired to assess the influence of chemical substitution on diverse molecular parameters, especially on those connected with laser cooling, such as vibronic transition probabilities, and those related to violations of fundamental symmetries. Here in this article high-precision ab initio studies of electronic and vibronic levels of diatomic radium monochloride (RaCl) are presented. Recently developed approaches for treating electronic correlation with Fock-space coupled cluster methods are applied for this purpose. Theoretical results are compared to an early experimental investigation by Lagerqvist and used to partially reassign the experimentally observed transitions and molecular electronic levels of RaCl. Effective constants of $\mathscr{P}$-odd hyperfine interaction W a and $\mathscr{P, T}$-odd scalar-pseudoscalar nucleus-electron interaction Ws in the ground electronic state of RaCl are estimated within the framework of a quasirelativistic Zeroth-Order Regular Approximation approach and compared to parameters in RaF and RaOH.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Solution Irregularity Remediation for Spatial Discretization Error Estimation for S N Transport Solutions

The discrete ordinates linear Boltzmann transport equation is typically solved in its spatially discretized form, incurring spatial discretization error. Quantification of this error for purposes such as adaptive mesh refinement or error analysis requires an a posteriori estimator, which utilizes the numerical solution to the spatially discretized equation to compute an estimate. Because the quality of the numerical solution informs the error estimate, irregularities, present in the true solution for any realistic problem configuration, tend to cause the largest deviation in the error estimate vis-a-vis the true error. In this paper, an analytical partial singular characteristic tracking (pSCT) procedure for reducing the estimator’s error is implemented within our novel residual source estimator for a zeroth-order discontinuous Galerkin scheme, at the additional cost of a single inner iteration. Here, a metric-based evaluation of the pSCT scheme versus the standard residual source estimator is performed over the parameter range of a Method of Manufactured Solutions test suite. The pSCT scheme generates near-ideal accuracy in the estimate in problems where the dominant source of the estimator’s error is the solution irregularity, namely, problems where the true solution is discontinuous and problems where the true solution’s first derivative is discontinuous and the scattering ratio is low. In problems where the scattering ratio is high and the true solution is discontinuous in the first derivative, the error in the scattering source, which is not converged by the pSCT scheme, is greater than the error incurred due to the irregularity. Ultimately, a pSCT scheme is judged to be useful for error estimation in problems where the computational cost of the scheme is justified. In the presence of many irregularities, such a scheme may be intractable for general use, but in benchmarks, as an analytical tool, or in problems that have nondissipative discontinuities, the scheme may prove invaluable.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

N 5 -Scaling Excited-State-Specific Perturbation Theory

In this paper, we show that by working in a basis similar to that of the natural transition orbitals and using a modified zeroth-order Hamiltonian, the cost of a recently introduced perturbative correction to excited-state mean field theory can be reduced from seventh to fifth order in the system size. The (occupied) 2 (virtual) 3 asymptotic scaling matches that of ground-state second-order Møller-Plesset theory but with a significantly higher prefactor because the bottleneck is iterative: it appears in the Krylov-subspace-based solution of the linear equation that yields the first-order wave function. Here, we discuss the details of the modified zeroth-order Hamiltonian we use to reduce the cost and the automatic code generation process we used to derive and verify the cost scaling of the different terms. Overall, we find that our modifications have little impact on the method's accuracy, which remains competitive with singles and doubles equation-of-motion coupled cluster.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2018

The Dynamics of a Field in a Cavity with a Slowly Oscillating Metal Mirror

We have examined analytically the behavior of an electromagnetic field in a cavity of one of two metal mirrors that is stationary, while the other mirror oscillates in the vicinity of its equilibrium position. The case in which the period of the mirror oscillations substantially exceeds the characteristic roundtrip time of the electromagnetic radiation in the cavity has been considered. To solve the problem, we have applied the method of two time scales. The method makes it possible to solve the problem by using the expansion with respect to a small parameter, for which the ratio of the cavity roundtrip time to the mirror-oscillation period is used. The solution to the problem in the zeroth-order approximation with respect to the small parameter has been obtained and examined. This solution has been shown to be correct up to the mirror-oscillation amplitude being on the order of the average cavity length. We have showed that this scheme can be used to gain and generate electromagnetic radiation. We have compared our results obtained for the oscillating mirror with the results of an exact solution of the problem when the mirror moves uniformly and rectilinearly. We have showed that, in these two cases, a change in the total field energy in the cavity is inversely proportional to the current value of the cavity length.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Perfectly Matched Layers and Characteristic Boundaries in Lattice Boltzmann: Accuracy vs Cost

Artificial boundary conditions (BCs) play a ubiquitous role in numerical simulations of transport phenomena in several diverse fields, such as fluid dynamics, electromagnetism, acoustics, geophysics, and many more. They are essential for accurately capturing the behavior of physical systems whenever the simulation domain is truncated for computational efficiency purposes. Ideally, an artificial BC would allow relevant information to enter or leave the computational domain without introducing artifacts or unphysical effects. Boundary conditions designed to control spurious wave reflections are referred to as nonreflective boundary conditions (NRBCs). Another approach is given by the perfectly matched layers (PMLs), in which the computational domain is extended with multiple dampening layers, where outgoing waves are absorbed exponentially in time. Here, in this work, the definition of PML is revised in the context of the lattice Boltzmann method. The impact of adopting different types of BCs at the edge of the dampening zone is evaluated and compared, in terms of both accuracy and computational costs. It is shown that for sufficiently large buffer zones, PMLs allow stable and accurate simulations even when using a simple zeroth-order extrapolation BC. Moreover, employing PMLs in combination with NRBCs potentially offers significant gains in accuracy at a modest computational overhead, provided the parameters of the BC are properly tuned to match the properties of the underlying fluid flow.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Moment-Fourier approach to ion parallel fluid closures and transport for a toroidally confined plasma

A general method of solving the drift kinetic equation is developed for an axisymmetric magnetic field. Expanding a distribution function in general moments, a set of ordinary differential equations is obtained. Successively expanding the moments and magnetic-field involved quantities in Fourier series, a set of linear algebraic equations is obtained. The set of full (Maxwellian and non-Maxwellian) moment equations is solved to express the first-order density, temperature, and flow velocity in terms of radial gradients of the zeroth-order pressure and temperature. Closure relations that connect parallel heat flux density and viscosity to the radial gradients and parallel gradients of temperature and flow velocity are also obtained by solving the non-Maxwellian moment equations. The closure relations combined with the linearized fluid equations reproduce the same solution obtained directly from the full moment equations. Furthermore, the method can be generalized to derive closures and transport for an electron-ion plasma and a multi-ion plasma in a general magnetic field.

neoclassical transport↗
DOE OSTI2025

Derivative-free stochastic optimization via adaptive sampling strategies

In this paper, we present a novel derivative-free framework for solving unconstrained stochastic optimization problems. Many problems in fields ranging from simulation optimization to reinforcement learning to quantum computing involve settings where only stochastic function values are obtained via a zeroth-order oracle, which has no available gradient information and necessitates the usage of derivative-free optimization methodologies. Our approach includes estimating gradients using stochastic function evaluations and integrating adaptive sampling techniques to control the accuracy in these stochastic approximations. Our framework encapsulates several gradient estimation techniques, including standard finite-difference, Gaussian smoothing, sphere smoothing, randomized coordinate finite-difference, and randomized subspace finite-difference methods. We provide theoretical convergence guarantees for our framework and analyze the worst-case iteration and sample complexities associated with each gradient estimation method. Finally, we demonstrate the empirical performance of the methods on logistic regression and nonlinear least squares problems.

Adaptive sampling↗
DOE OSTI2025

Multiscale astrobiology with the Vera C. Rubin Observatory Legacy Survey of Space and Time

The Vera C. Rubin Observatory’s Legacy Survey of Space and Time (LSST) presents an unprecedented opportunity to advance the search for astrobiologically relevant data across diverse astronomical environments. Through its extensive imaging capabilities, LSST will enhance our inventory of Solar System objects and assess their potential astrobiological conditions. Beyond the Solar System, LSST will survey billions of stars, contributing to the detection of exoplanets, characterizing planetary atmospheres at a zeroth-order level, identifying transient phenomena, and exploring conditions within habitable zones. In this paper, we highlight LSST’s unique contribution to astrobiology, complementing other missions and expanding the search for life beyond Earth in the coming decades.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Second harmonic Bessel-Gauss beam shaping with elliptic axicon aberrations

The second harmonic (SH) of an axicon generated Bessel-Gauss beam is created through the nonlinear interaction of photons with a crystal, resulting in the energy doubling of the output photons. In this work, we show experimentally that in addition to frequency doubling, the SH of Bessel-Gauss beams under asymmetric aberrations from an elliptic axicon exhibit intriguing beam formation. Particularly, the central region of the SH beam profile is composed of two central spots of various geometries surrounded by nested ellipses; one of which is the configuration of two central gamma dots with similar radius knotted by nested ellipses for a zeroth-order Bessel-Gauss pump. These SH beams consistently maintain their spatial profile throughout propagation, reminiscing the behavior of screw dislocations in wave patterns. Our numerical simulations produce beam dynamics consistent with that of experiments and further implicate the remarkable interweaving of bright spots with dark vortices. This is especially noticeable when the beams dynamically oscillate along the optical axis, resulting in the genesis of spatially polarized beams with a knotted framework. The insights gained from our study establish a novel paradigm for exploring interactions of Bessel-like beams with vortex dynamics. This, in turn, has the potential to spark innovations in optical applications, fostering new methodologies to harness and manipulate complex light structures. Our experimental findings could spur new methods of generating logical states of light and new opportunities for material processing control. Published by the American Physical Society 2025

Wang, Tianhong (ORCID:0000000169104201)↗