Search NASA⌕ Search

SEARCH · Search NASA

Results for “generalized gradient approximations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Approximating Nash Equilibrium in Day-ahead Electricity Market Bidding with Multi-agent Deep Reinforcement Learning

In this paper, a day-ahead electricity market bidding problem with multiple strategic generation company (GEN-CO) bidders is studied. The problem is formulated as a Markov game model, where GENCO bidders interact with each other todevelop their optimal day-ahead bidding strategies. Considering unobservable information in the problem, a model-free and data-driven approach, known as multi-agent deep deterministic policy gradient (MADDPG), is applied for approximating the Nash equilibrium (NE) in the above Markov game. The MADDPG algorithm has the advantage of generalization due to the automatic feature extraction ability of the deep neural networks. The algorithm is tested on an IEEE 30-bus system with three competitive GENCO bidders in both an uncongested caseand a congested case. Comparisons with a truthful bidding strategy and state-of-the-art deep reinforcement learning methods including deep Q network and deep deterministic policy gradient (DDPG) demonstrate that the applied MADDPG algorithm can find a superior bidding strategy for all the market participants with increased profit gains. In addition, the comparison with a conventional model-based method shows that the MADDPG algorithm has higher computational efficiency, which is feasible for real-world applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

L-BFGS Class Implementation in C++

This report presents a header-only C++ class implementation of the Limited-memory BroydenFletcher-Goldfarb-Shanno (L-BFGS) algorithm. The L-BFGS method is a general purpose quasi-Netwon optimization method that builds an approximation of the descent direction from consecutive iterate and gradient vectors. The limited-memory aspect stems from the replacement of the N × N approximation matrix of the original BFGS method with M vectors of length N. An example usage of the class is included along with the reference source code.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Identifying Controlling Variables for Mercury Vapors in Alpha-4 at Y-12: Two Year Data Collection Update

Multiple sensor packages were deployed at Alpha-4 by SRNL, in collaboration with United Cleanup Oak Ridge LLC (UCOR), to monitor mercury vapor concentrations and meteorological parameters. These sensors collected data, inside and outside of the legacy-use facility, for approximately two years. Though data gaps still exist, particularly in colder months, several controlling variables were identified that govern mercury vapor concentrations within Alpha-4. Temperature, barometric pressure gradients, humidity, and wind speed have been identified as controlling variables. A strong positive correlation was seen between mercury vapor concentrations and temperature which generally followed diurnal fluctuations. Temperatures below approximately 10 degrees Celsius did not show any spikes above the PEL, indicating more work can be performed at any time during the winter months – more data should be collected to confirm consistency in this finding. Additionally, spikes in mercury vapor concentrations above that of the permissible exposure limit (PEL; 100 µg/m 3 ) occurred primarily in late afternoon or evening/overnight hours (between 3 PM and 6 AM), which suggests D&D operations might be best scheduled during morning or daytime hours prior to the late afternoon. However, a limited number of spikes did occur outside of the identified window, although this may be attributed to disturbances in air flow and mercury vapor release from work activities performed inside of the Alpha-4 building. The analysis conducted allows for a strong predictive capability for estimating mercury vapor concentrations based upon accurate meteorological parameters. Still, additional data collection, particularly in the winter months, could help to strengthen the predictive power and validate the identified data trends. Further, increased temporal resolution could also help to better characterize the incipient stages of the increases and decreases in the mercury vapor concentration. Continued monitoring support by SRNL at Y-12 is underway at Alpha-4 to further close remaining data gaps and support deactivation and decommissioning work. Within a collaborative effort with UCOR, the SRNL team is collecting mercury vapor data to study the efficacy of a novel mercury suppressant, FerroBlack® which was recently deployed at Alpha-4. In addition, modifications to the current monitoring setup to increase measurement resolution is also being investigated.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Leptodermic corrections to the TOV equations and nuclear astrophysics within the effective surface approximation

The macroscopic model for a neutron star (NS) as a liquid drop at the equilibrium is used to extend the Tolman-Oppenheimer-Volkoff (TOV) equations taking into account the gradient terms responsible for the system surface. The parameters of the Schwarzschild metric in the spherical case are found with these surface corrections to the known leading (zero) order of the leptodermic approximation a/R << 1, where a is the NS effective-surface (ES) thickness, and R is the effective NS radius. The energy density $\mathscr{E}$ is considered in a general form including the functions of the particle number density and of its gradient terms. The macroscopic gravitational component $Φ$(ρ) of the energy density is taken into account in the simplest form as expansion in powers of $ρ$ – $\overline{ρ}$, where $\overline{ρ}$ is the saturation density, up to second order, in terms of its contributions to the separation particle energy and incompressibility. Density distributions ρ across the NS ES in the normal direction to the ES, which are derived in the simple analytical form at the same leading approximation, was used for the derivation of the modified TOV (MTOV) equations by accounting for their NS surface corrections. As a result, the MTOV equations are analytically solved at first order and the results are compared with the standard TOV approach of the zero order.

Magner, A. G. [Institute for Nuclear Research, Kyi↗
DOE OSTI2024

An Empirical Quantile Estimation Approach for Chance-Constrained Nonlinear Optimization Problems

We investigate an empirical quantile estimation approach to solve chance-constrained nonlinear optimization problems. Our approach is based on the reformulation of the chance constraint as an equivalent quantile constraint to provide stronger signals on the gradient. In this approach, the value of the quantile function is estimated empirically from samples drawn from the random parameters, and the gradient of the quantile function is estimated via a finite-difference approximation on top of the quantile-function-value estimation. We establish a convergence theory of this approach within the framework of an augmented Lagrangian method for solving general nonlinear constrained optimization problems. The foundation of the convergence analysis is a concentration property of the empirical quantile process, and the analysis is divided based on whether or not the quantile function is differentiable. In contrast to the sampling-and-smoothing approach used in the literature, the method developed in this paper does not involve any smoothing function and hence the quantile-function gradient approximation is easier to implement and there are less accuracy-control parameters to tune. Furthermore, we demonstrate the effectiveness of this approach and compare it with a smoothing method for the quantile-gradient estimation. Numerical investigation shows that the two approaches are competitive for certain problem instances.

Applied Probability↗
DOE OSTI2023

High absorption photovoltaic material and methods of making the same

A high absorption photovoltaic material and method of making the material for use in a solar cell are disclosed. The photovoltaic material includes a surface modified with a layer of repeating photonic crystal structures. The photonic crystal structures are approximately inverse conically shaped and have a curved sidewall that has an approximately Gaussian shape. The photonic crystal structures generally have a high vertical depth and sidewall angle. The structures also have a gradient refractive index profile and exhibit the parallel-to-interface refraction light trapping effect. An anti-reflective coating is disposed over the photonic crystal structure layer. The photovoltaic material exhibits near unity light absorption over a broad range of visible and near infrared wavelengths and incidence angles, even at reduced thicknesses. The photovoltaic structures are formed via a combined photolithography and reactive-ion etching method at low power with a gas mixture having a high ratio of an etchant component to a passivation component.

Kuang, Ping↗
DOE OSTI2024

Symmetry-Based Structured Matrices for Efficient Approximately Equivariant Networks

There has been much recent interest in designing symmetry-aware neural networks (NNs) exhibiting relaxed equivariance. Such NNs aim to interpolate between being exactly equivariant and being fully flexible, affording consistent performance benefits. In a separate line of work, certain structured parameter matrices -- those with displacement structure, characterized by low displacement rank (LDR) -- have been used to design small-footprint NNs. Displacement structure enables fast function and gradient evaluation, but permits accurate approximations via compression primarily to classical convolutional neural networks (CNNs). In this work, we propose a general framework -- based on a novel construction of symmetry-based structured matrices -- to build approximately equivariant NNs with significantly reduced parameter counts. Our framework integrates the two aforementioned lines of work via the use of so-called Group Matrices (GMs), a forgotten precursor to the modern notion of regular representations of finite groups. GMs allow the design of structured matrices -- resembling LDR matrices -- which generalize the linear operations of a classical CNN from cyclic groups to general finite groups and their homogeneous spaces. We show that GMs can be employed to extend all the elementary operations of CNNs to general discrete groups. Further, the theory of structured matrices based on GMs provides a generalization of LDR theory focussed on matrices with cyclic structure, providing a tool for implementing approximate equivariance for discrete groups. We test GM-based architectures on a variety of tasks in the presence of relaxed symmetry. We report that our framework consistently performs competitively compared to approximately equivariant NNs, and other structured matrix-based compression frameworks, sometimes with a one or two orders of magnitude lower parameter count.

Samudre, Ashwin↗
DOE OSTI2021

Plateau Phenomenon in Gradient Descent Training of RELU Networks: Explanation, Quantification, and Avoidance

The ability of neural networks to provide ‘best in class’ approximation across a wide range of applications is well-documented. Nevertheless, the powerful expressivity of neural networks comes to naught if one is unable to effectively train (choose) the parameters defining the network. In general, neural networks are trained by gradient descent type optimization methods,a stochastic variant thereof. In practice, such methods result in the loss function decreases rapidly at the beginning of training but then, after a relatively small number of steps, significantly slow down. The loss may even appear to stagnate over the period of a large number of epochs, only to then suddenly start to decrease fast again for no apparent reason. This so-called plateau phenomenon manifests itself in many learning tasks. The present work aims to identify and quantify the root causes of plateau phenomenon.analysis is carried out in the setting of univariate ReLU networks. No assumptions are made on the number of neurons relative to the number of training data, and our results hold for both the lazy and adaptive regimes. Here, the main findings are: plateaux correspond to periods during which activation patterns remain constant, where activation pattern refers to the number of data points that activate a given neuron; quantification of convergence of the gradient flow dynamics; and, characterization stationary points in terms solutions of local least squares regression lines over subsets of the training data. Based on these conclusions, we propose a new iterative training method, the Active Neuron Least Squares (ANLS), characterised by the explicit adjustment of the activation pattern at each step, which is designed to enable a quick exit from a plateau. Illustrative numerical examples are included throughout.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Online Convex Optimization of Programmable Quantum Computers to Simulate Time-Varying Quantum Channels

Simulating quantum channels is a fundamental primitive in quantum computing, since quantum channels define general (trace-preserving) quantum operations. An arbitrary quantum channel cannot be exactly simulated using a finite-dimensional programmable quantum processor, making it important to develop optimal approximate simulation techniques. In this paper, we study the challenging setting in which the channel to be simulated varies adversarially with time. We propose the use of matrix exponentiated gradient descent (MEGD), an online convex optimization method, and analytically show that it achieves a sublinear regret in time. Through experiments, we validate the main results for time-varying dephasing channels using a programmable generalized teleportation processor.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Drag force to all orders in gradients

We study the energy loss of a heavy quark slowly moving through an evolving strongly coupled plasma. We use the linearized fluid/gravity correspondence to describe small perturbations of the medium flow with general spacetime dependence. This all order linearized hydrodynamics results in a drag force exerted on a heavy quark even when it is at rest with the fluid element. We show how the general contribution to the drag force can be derived order by order in the medium velocity gradients and provide explicit results valid up to the third order. We then obtain an approximate semi-analytic result for the drag force to all orders in the gradient expansion but linearized in the medium velocity. Thus, the effects of a class of hydrodynamic gradients on the drag force are re-summed, giving further insight into the dissipative properties of strongly coupled plasmas. The all order result allows us to study the drag force in the non-hydrodynamic regime of linear medium perturbations that vary rapidly in space and time.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Deep probabilistic direction prediction in 3D with applications to directional dark matter detectors

Abstract We present the first method to probabilistically predict 3D direction in a deep neural network model. The probabilistic predictions are modeled as a heteroscedastic von Mises-Fisher distribution on the sphere S 2 , giving a simple way to quantify aleatoric uncertainty. This approach generalizes the cosine distance loss which is a special case of our loss function when the uncertainty is assumed to be uniform across samples. We develop approximations required to make the likelihood function and gradient calculations stable. The method is applied to the task of predicting the 3D directions of electrons, the most complex signal in a class of experimental particle physics detectors designed to demonstrate the particle nature of dark matter and study solar neutrinos. Using simulated Monte Carlo data, the initial direction of recoiling electrons is inferred from their tortuous trajectories, as captured by the 3D detectors. For 40 keV electrons in a 70% He 30% CO 2 gas mixture at STP, the new approach achieves a mean cosine distance of 0.104 (26 ∘ ) compared to 0.556 (64 ∘ ) achieved by a non-machine learning algorithm. We show that the model is well-calibrated and accuracy can be increased further by removing samples with high predicted uncertainty. This advancement in probabilistic 3D directional learning could increase the sensitivity of directional dark matter detectors.

Computer Science↗
DOE OSTI2024

On the Departure from Monin–Obukhov Surface Similarity and Transition to the Convective Mixed Layer

Large-eddy simulations are used to evaluate mean profile similarity in the convective boundary layer (CBL). Particular care is taken regarding the grid sensitivity of the profiles and the mitigation of inertial oscillations in the simulation spin-up. The nondimensional gradients Φ for wind speed and air temperature generally align with Monin–Obukhov similarity across cases but have a steeper slope than predicted within each profile. The same trend has been noted in several other recent studies. The Businger-Dyer relations are modified here with an exponential cutoff term to account for the decay in Φ to first-order approximation, yielding improved similarity from approximately 0.05z i to above 0.3z i , where z i is the CBL depth. The necessity for the exponential correction is attributed to an extended transition from surface scaling to zero gradient in the mixed layer, where the departure from Monin–Obukhov similarity may be negligible at the surface but becomes substantial well below the conventional surface layer height of 0.1 z i .

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Bayesian sparse learning with preconditioned stochastic gradient MCMC and its applications

Deep neural networks have been successfully employed in an extensive variety of research areas, including solving partial differential equations. Despite its significant success, there are some challenges in effectively training DNN, such as avoiding overfitting in over-parameterized DNNs and accelerating the optimization in DNNs with pathological curvature. Here, we propose a Bayesian type sparse deep learning algorithm. The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network. The hierarchical Bayesian mixture will be trained using an adaptive empirical method. That is, one will alternatively sample from the posterior using preconditioned stochastic gradient Langevin Dynamics (PSGLD), and optimize the latent variables via stochastic approximation. The sparsity of the network is achieved while optimizing the hyperparameters with adaptive searching and penalizing. A popular SG-MCMC approach is Stochastic gradient Langevin dynamics (SGLD). However, considering the complex geometry in the model parameter space in nonconvex learning, updating parameters using a universal step size in each component as in SGLD may cause slow mixing. To address this issue, we apply a computationally manageable preconditioner in the updating rule, which provides a step-size parameter to adapt to local geometric properties. Moreover, by smoothly optimizing the hyperparameter in the preconditioning matrix, our proposed algorithm ensures a decreasing bias, which is introduced by ignoring the correction term in the preconditioned SGLD. According to the existing theoretical framework, we show that the proposed algorithm can asymptotically converge to the correct distribution with a controllable bias under mild conditions. Numerical tests are performed on both synthetic regression problems and learning solutions of elliptic PDE, which demonstrate the accuracy and efficiency of the present work.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems

Here, we propose new symplectic networks (SympNets) for identifying Hamiltonian systems from data based on a composition of linear, activation and gradient modules. In particular, we define two classes of SympNets: the LA-SympNets composed of linear and activation modules, and the G-SympNets composed of gradient modules. Correspondingly, we prove two new universal approximation theorems that demonstrate that SympNets can approximate arbitrary symplectic maps based on appropriate activation functions. We then perform several experiments including the pendulum, double pendulum and three-body problems to investigate the expressivity and the generalization ability of SympNets. The simulation results show that even very small size SympNets can generalize well, and are able to handle both separable and non-separable Hamiltonian systems with data points resulting from short or long time steps. In all the test cases, SympNets outperform the baseline models, and are much faster in training and prediction. We also develop an extended version of SympNets to learn the dynamics from irregularly sampled data. This extended version of SympNets can be thought of as a universal model representing the solution to an arbitrary Hamiltonian system.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

A Novel Technique for Distributed Measuring of Strain in the Vadose Zone

Characterizing water content and pressure changes in the vadose zone is important to understanding a variety of geologic processes, ranging from permeability, to evapotranspiration, and aquifer recharge. Changes in water content, or pressure, cause strain in the solid porous medium of the vadose zone, and it is possible that measuring those strains could be used a characterization tool. The Coherence-length-gated Microwave Photonics Interferometry (CMPI) technique measures strain at high resolution along many intervals defined by pairs of reflectors distributed along an optical fiber. This technique has recently been developed at Clemson University, and it has the spatial, and temporal resolution to characterize the strains that are expected to occur with hydrologic changes in the vadose zone. However, the technique has never been used to measure strain in porous media, so its capabilities remain uncertain. The objective of this thesis is to evaluate the ability of using CMPI to measure strain changes in the vadose zone. The research approach consists of conducting laboratory tests using a column filled with sand that was subjected to changes in water content and pressure. An optical fiber with CMPI reflectors was deployed in a high surface area ribbon and used to measure strain along the axis of the column. The column was made from 8-inch, Schedule 40 PVC pipe (75 cm tall, 20 cm inner diameter) and filled with medium-grained sand (K = 2.5x10-6 m/s, porosity = 0.28, Coefficient of Uniformity = 2, van Genuchten (n = 4.34 and  = 0.00039 m-1), Young’s Modulus = 18 – 47 MPa). Five pressure and four temperature sensors attached to probes inserted into the wall. The optical fiber includes five reflectors spaced 10 cm apart on the inside and outside of the column along the vertical axis. Strain measurements are made between pairs of reflectors. The reflectors are created in 250-micron-diameter acrylate-coated single mode Corning SMF28e+ optical fiber using a femtosecond laser. The optical fiber was laminated between two pieces of polyester film creating a large surface area to transfer strain from the porous media to the optical fiber. The experiments were conducted by filling the column from the bottom or infiltrating water from the top. The water was allowed to equilibrate to room temperature prior to each test in order to limit thermoelastic strain. Five injection tests with a rate of 250 ml/min and three infiltration tests at varying rates were conducted, and the results show patterns of strain and pressure are generally similar. Hydrologic conditions define three zones based on the pressure magnitude and distribution. 1.) Ambient Zone where the pressure heads are quasi-static and the pressure gradient is roughly unity (head gradient of zero). This is the uppermost zone and is characterized by negative pressures. 2.) Transition Zone where the pressures increase from ambient to zero, are changing relatively rapidly and the pressure gradient is relatively steep (pressure head gradients of 2). The Transition zone is 10 to 15 cm thick. 3.) Positive Pressure Zone where the pressure is positive, the rate of change is slower than in the transition zone and the gradient is flatter (pressure head gradient 1.1 to 1.2). This is the lowest zone in the column. Injection of water causes the pressure to increase and the three zones to move upward. The Transition zone moves at a rate of approximately 0.0002 m/s +/- .00005, according to analyses of pressure profiles. This is the same as the average velocity of the water calculated as volumetric flux/effective porosity =5.4x10-5 m/s/0.28. Strain signals in the range of 10s  were observed with a noise level of generally less than 0.1  (signal to noise ratio of greater than 100) during injection and drainage. The spatial and temporal distributions of strain caused by injection depend on the location of the moving pressure zones. The locations of the different pressure zones were determined from pressure profiles at different times and these data were transferred to strain time series. This showed that strain in the Ambient pressure zone is either unchanged or slightly compressive, whereas strain in Transition zone is tensile and roughly proportional to the pressure change, and strain ranges from tensile to compressive in the Positive Pressure zone. In general, the strain is variable at the top of the Positive Pressure zone, but it appears to be consistently tensile lower in the zone. The spatial distribution of strain causes three distinct stages in the strain times series measured between pairs of CMPI reflectors. The strain is unchanged or slightly compressive during Stage 1, it increases (positive strain is tensile) approximately linearly at a rate of 0.02 to 0.03 μ/s during the Stage 2, and then flattens out in Stage 3. The strain increases again during Stage 4 of the time series. The pressure also changes in stages. It is unchanged during Stage 1 and then increases relatively rapidly at a rate of between 4 and 10 Pa/s during Stage 2 and continues to increase during Stages 3 and 4, but at a rate slightly slower than during Stage 2 (from 2.5 to 3.5 Pa/s). The rates of pressure and strain change are consistent with basic analyses. The rate of pressurization is similar to the calculated velocity of flow in the saturated zone, and this is similar to the measured velocity of the strain increase. The ratio of the pressurization rate and strain rate during Stage 2 (2.5 (Pa/s) /0.03 (μ/s)) is 80 MPa, which is approximately equal to the upper range expected considering the uniaxial compression and the Young’s Modulus of the sand. An interesting effect occurs when the upper surface of the sand becomes saturated. Significant compression (several 10s of ) occurs as the pressure and saturation increase at the upper surface. Compression occurs throughout the column, but the effect is greatest at the top of the column. The rate of compression is fastest slightly before ponding occurs, but it slows markedly and nearly stops when water starts to accumulate at the surface (ponding). This effect reverses when the surface of the sand is drained, resulting in tension throughout the column. The compression caused by this effect can be as large or larger than the tensile strain that accumulated during filling. This effect was unexpected because increasing pressure is normally associated with tensile strain. Nevertheless, this effect was observed consistently in all tests where the pressure changes at the upper surface, including tests where water was injected from below or infiltrated from above. This effect behaves as if the pore pressure at the upper surface of the sand exerts a normal force on the boundary (increasing pore pressure exerts a downward compression on the boundary). The results of these laboratory experiments indicate that CMPI can measure strain caused by fluid pressure changes in the vadose zone with a signal to noise ratio of 100 or more. Injection and drainage cause a strain signal that is complex, but repeatable. The magnitudes of the strain signal are consistent with magnitudes that are expected based on poroelastic calculations using independently measured properties of the sand. These results indicate that the CMPI technique with an optical fiber laminated in a polyester ribbon generates data that represent the strain distribution during hydrologic processes of imbibition and drainage in variably saturated sand. This suggests that distributed strain measurements using CMPI could be a viable approach for evaluating processes in the vadose zone, laying the groundwork for future field implementation.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Overall chilled water system energy consumption modeling and optimization

The emergence of increasingly affordable variable-speed drive technology has changed the approach used to control chilled water systems equipped with these drives. The purpose of this research was to develop an integrated chilled water modeling technique that can determine the optimal system setpoints and estimate the energy saving potential of chiller system. The chiller system equipped with Variable Frequency Drives (VFDs) on cooling tower fans and condenser water pumps. To accomplish the objective, physical component models of the centrifugal chiller, cooling tower and condenser water pump were established with the goal of incorporating the system’s condenser water flow rate and cooling tower fan speeds as optimization variables. Furthermore, a cooling load prediction algorithm was developed using a multiple non-linear regression model to approximate the building’s cooling load subject to a range of environmental conditions. The inputs and outputs of the individual component models were linked to estimate how adjusting the cooling tower fan and condenser water pump speed would influence the system’s comprehensive performance. Here, the overall system model was then optimized using a generalized reduced gradient optimization algorithm to determine the potential energy savings through speed control with VFDs and to ascertain a control logic strategy for the building automation system to operate the heating and cooling system. A case-study was performed on a single chiller system at a museum and the model was calibrated according to logged data collected over four months. Results showed that for the system analyzed, the energy saving of optimizing the cooling tower fan system was found to be 12–15%, while the energy saving potential of optimizing the condenser water pump with the cooling tower fan was negligible. Additionally, comparing different cooling tower fan control strategies showed that a wet-bulb approach-based cooling tower control strategy was shown to have the highest correlation to the optimized fan speed with an R 2 of 0.924.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Probabilistic partition of unity networks for high–dimensional regression problems

We explore the probabilistic partition of unity network (PPOU-Net) model in the context of high-dimensional regression problems and propose a general framework focusing on adaptive dimensionality reduction. With the proposed framework, the target function is approximated by a mixture of experts model on a low-dimensional manifold, where each cluster is associated with a fixed-degree polynomial. We present a training strategy that leverages the expectation maximization (EM) algorithm. During the training, we alternate between (i) applying gradient descent to update the DNN coefficients; and (ii) using closed-form formulae derived from the EM algorithm to update the mixture of experts model parameters. Under the probabilistic formulation, step (ii) admits the form of embarrassingly paralleliazable weighted least-squares solves. The PPOU-Nets consistently outperform the baseline fully-connected neural networks of comparable sizes in numerical experiments of various data dimensions. Here, we also explore the proposed model in applications of quantum computing, where the PPOU-Nets act as surrogate models for cost landscapes associated with variational quantum circuits.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Nonlinear Matrix Approximation with Radial Basis Function Components

We introduce and investigate matrix approximation by decomposition into a sum of radial basis function (RBF) components. An RBF component is a generalization of the outer product between a pair of vectors, where an RBF function replaces the scalar multiplication between individual vector elements. Even though the RBF functions are positive definite, the summation across components is not restricted to convex combinations and allows us to compute the decomposition for any real matrix that is not necessarily symmetric or positive definite. We formulate the problem of seeking such a decomposition as an optimization problem with a nonlinear and non-convex loss function. Several modern versions of the gradient descent method, including their scalable stochastic counterparts, are used to solve this problem. We provide extensive empirical evidence of the effectiveness of the RBF decomposition and that of the gradient-based fitting algorithm. While being conceptually motivated by singular value decomposition (SVD), our proposed nonlinear counterpart outperforms SVD by drastically reducing the memory required to approximate a data matrix with the same L2 error for a wide range of matrix types. For example, it leads to 2 to 6 times memory save for Gaussian noise, graph adjacency matrices, and kernel matrices. Moreover, this proximity-based decomposition can offer additional interpretability in applications that involve, e.g., capturing the inner low-dimensional structure of the data, retaining graph connectivity structure, and preserving the acutance of images.

Rebrova, Elizaveta↗