Search NASA⌕ Search

SEARCH · Search NASA

Results for “least squares problem”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Some new preconditioned generalized AOR methods for solving weighted linear least squares problems

In this paper, we propose some new preconditioned GAOR methods for solving weighted linear least squares problems and discuss their comparison results. Comparison results show that the convergence rates of the new preconditioned GAOR methods are better than those of the preconditioned GAOR methods presented by Shen et al. (Appl Math Mech Engl Ed 33(3):375–384, 2012) and Wang et al. (J Appl Math, doi: 10.1155/2012/563586 , 2012) whenever these methods are convergent. Finally, numerical experiments are provided to confirm the theoretical results obtained in this paper.

97 MATHEMATICS AND COMPUTING↗

Fast Combinatorial Algorithm for the Solution of Linearly Constrained Least Squares Problems

A fast combinatorial algorithm can significantly reduce the computational burden when solving general equality and inequality constrained least squares problems with large numbers of observation vectors. The combinatorial algorithm provides a mathematically rigorous solution and operates at great speed by reorganizing the calculations to take advantage of the combinatorial nature of the problems to be solved. The combinatorial algorithm exploits the structure that exists in large-scale problems in order to minimize the number of arithmetic operations required to obtain a solution.

Van Benthem, Mark H↗

Robust parallel iterative solvers for linear and least-squares problems (Final Technical Report)

The primary goal of this project is to study and develop robust iterative methods for solving linear systems of equations and least squares systems. The focus of the Minnesota team is on algorithms development, robustness issues, and on tests and validation of the methods on realistic problems. 1. The project begun with an investigation on how to practically update a preconditioner obtained from an ILU-type factorization, when the coefficient matrix changes. 2. We investigated strategies to improve robustness in parallel preconditioners in a specific case of a PDE with discontinuous coefficients. 3. We explored ways to adapt standard preconditioners for solving linear systems arising from the Helmholtz equation. These are often difficult linear systems to solve by iterative methods. 4. We have also worked on purely theoretical issues related to the analysis of Krylov subspace methods for linear systems. 5. We developed an effective strategy for performing ILU factorizations for the case when the matrix is highly indefinite. The strategy uses shifting in some optimal way. The method was extended to the solution of Helmholtz equations by using complex shifts, yielding very good results in many cases. 6. We addressed the difficult problem of preconditioning sparse systems of equations on GPUs. 7. A by-product of the above work is a software package consisting of an iterative solver library for GPUs based on CUDA. This was made publicly available. It was the first such library that offers complete iterative solvers for GPUs. 8. We considered another form of ILU which blends coarsening techniques from Multigrid with algebraic multilevel methods. 9. We have released a new version on our parallel solver - called pARMS [new version is version 3]. As part of this we have tested the code in complex settings - including the solution of Maxwell and Helmholtz equations and for a problem of crystal growth.10. As an application of polynomial preconditioning we considered the problem of evaluating f(A)v which arises in statistical sampling. 11. As an application to the methods we developed, we tackled the problem of computing the diagonal of the inverse of a matrix. This arises in statistical applications as well as in many applications in physics. We explored probing methods as well as domain-decomposition type methods. 12. A collaboration with researchers from Toulouse, France, considered the important problem of computing the Schur complement in a domain-decomposition approach. 13. We explored new ways of preconditioning linear systems, based on low-rank approximations.

97 MATHEMATICS AND COMPUTING↗

Regularized Differentiation for Bioburden Density Estimation in Planetary Protection

In this paper, we propose and investigate the performance of two novel shrinkage estimators for bioburden density estimation in planetary protection. The estimators are based on the regularized differentiation of a cumulative count of colony forming units collected throughout the data collecting session or the life cycle of the entire mission. The regularized differentiation recasts the problem of bioburden density estimation as a linear least squares problem. The least squares problem is then solved through regularization techniques, such as truncated singular value decomposition and penalized least squares. The regularization is necessary to avoid noise amplification during the differentiation of noisy data. The two regularization estimators are compared with four other commonly used estimators to simultaneously evaluate the means of multivariable independent Poisson distributions: the maximum likelihood, noninformative Bayes estimator with Jeffreys prior, Empirical Bayes using conjugate gamma-Poisson model with gamma parameters selected by method of moments, and the Clevenson-Zidek estimator. It is shown through computer-simulated data that the regularized differentiation based on ridge regression has the smallest mean-squared error among all estimators. The analysis of shrinkage mechanism implemented by regularized differentiation is performed, and it is shown that the regularized differentiation amounts to performing a weighted averaging of all the samples. The weights are determined by the regularization parameter automatically selected by the L-curve technique. Since the method of least squares makes no distributional assumptions about the data, it presents an attractive technique for bioburden density estimation when there are concerns about the misspecification of the distributional model. The paper concludes with the analysis of the bioburden data collected during InSight mission and directions for future work.

97 - MATHEMATICS AND COMPUTING↗

Randomized Algorithms for Symmetric Nonnegative Matrix Factorization

Symmetric Nonnegative Matrix Factorization (SymNMF) is a technique in data analysis and machine learning that approximates a matrix with a product of a nonnegative, low-rank matrix and it transpose. To design faster and more scalable algorithms for SymNMF we develop two randomized algorithms for its computation. The first method uses randomized matrix sketching to compute an initial low-rank approximation to the input matrix and proceeds to uses this as a low-rank input to rapidly compute a SymNMF. The second methods uses randomized leverage score sampling to approximately solve constrained least squares problems. Many successful methods for SymNMF rely on (approximately) solving sequences of constrained least squares problems. Here, we prove theoretically that leverage score sampling can approximately solve constrained least squares problems to e-accuracy. Finally we demonstrate both methods work in practice by applying them to graph clustering tasks on large real world data sets. These experiments show that our methods approximately maintain solution quality and achieve significant speed ups for both large dense and large sparse problems.

97 MATHEMATICS AND COMPUTING↗

Fast Combinatorial Nonnegative Least Squares v 1.0

Software that employs an algorithm that rapidly and rigorously solves nonnegative least squares problems with many right hand sides. This is the latest internal version of MATLAB pseudocode that was described in a published in M. H. Van Benthem and M. R. Keenan, "Fast algorithm for the solution of largescale non-negativity-constrained least squares problems," J. Chemom. 18(10), 441-450 (2004). Example code of this algorithm was included in the publication. This algorithm was also patented in M. H. Van Benthem and M. R. Keenan, "Fast combinatorial algorithm for the solution of linearly constrained least squares problems,"USPTO, Ed., Sandia Corporation, USA (2008). SAND2019-10098 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525.

Van Benthem, Mark↗

Operator inference for non-intrusive model reduction of systems with non-polynomial nonlinear terms

Here in this work we present a non-intrusive model reduction method to learn low-dimensional models of dynamical systems with non-polynomial nonlinear terms that are spatially local and that are given in analytic form. In contrast to state-of-the-art model reduction methods that are intrusive and thus require full knowledge of the governing equations and the operators of a full model of the discretized dynamical system, the proposed approach requires only the non-polynomial terms in analytic form and learns the rest of the dynamics from snapshots computed with a potentially black-box full-model solver. The proposed method learns operators for the linear and polynomially nonlinear dynamics via a least-squares problem, where the given non-polynomial terms are incorporated on the right-hand side. The least-squares problem is linear and thus can be solved efficiently in practice. The proposed method is demonstrated on three problems governed by partial differential equations, namely the diffusion–reaction Chafee–Infante model, a tubular reactor model for reactive flows, and a batch-chromatography model that describes a chemical separation process. The numerical results provide evidence that the proposed approach learns reduced models that achieve comparable accuracy as models constructed with state-of-the-art intrusive model reduction methods that require full knowledge of the governing equations.

42 ENGINEERING↗

Scalable tensor factorizations with missing data.

The problem of missing data is ubiquitous in domains such as biomedical signal processing, network traffic analysis, bibliometrics, social network analysis, chemometrics, computer vision, and communication networks|all domains in which data collection is subject to occasional errors. Moreover, these data sets can be quite large and have more than two axes of variation, e.g., sender, receiver, time. Many applications in those domains aim to capture the underlying latent structure of the data; in other words, they need to factorize data sets with missing entries. If we cannot address the problem of missing data, many important data sets will be discarded or improperly analyzed. Therefore, we need a robust and scalable approach for factorizing multi-way arrays (i.e., tensors) in the presence of missing data. We focus on one of the most well-known tensor factorizations, CANDECOMP/PARAFAC (CP), and formulate the CP model as a weighted least squares problem that models only the known entries. We develop an algorithm called CP-WOPT (CP Weighted OPTimization) using a first-order optimization approach to solve the weighted least squares problem. Based on extensive numerical experiments, our algorithm is shown to successfully factor tensors with noise and up to 70% missing data. Moreover, our approach is significantly faster than the leading alternative and scales to larger problems. To show the real-world usefulness of CP-WOPT, we illustrate its applicability on a novel EEG (electroencephalogram) application where missing data is frequently encountered due to disconnections of electrodes.

99 GENERAL AND MISCELLANEOUS↗

Scalable tensor factorizations with incomplete data.

The problem of incomplete data - i.e., data with missing or unknown values - in multi-way arrays is ubiquitous in biomedical signal processing, network traffic analysis, bibliometrics, social network analysis, chemometrics, computer vision, communication networks, etc. We consider the problem of how to factorize data sets with missing values with the goal of capturing the underlying latent structure of the data and possibly reconstructing missing values (i.e., tensor completion). We focus on one of the most well-known tensor factorizations that captures multi-linear structure, CANDECOMP/PARAFAC (CP). In the presence of missing data, CP can be formulated as a weighted least squares problem that models only the known entries. We develop an algorithm called CP-WOPT (CP Weighted OPTimization) that uses a first-order optimization approach to solve the weighted least squares problem. Based on extensive numerical experiments, our algorithm is shown to successfully factorize tensors with noise and up to 99% missing data. A unique aspect of our approach is that it scales to sparse large-scale data, e.g., 1000 x 1000 x 1000 with five million known entries (0.5% dense). We further demonstrate the usefulness of CP-WOPT on two real-world applications: a novel EEG (electroencephalogram) application where missing data is frequently encountered due to disconnections of electrodes and the problem of modeling computer network traffic where data may be absent due to the expense of the data collection process.

99 GENERAL AND MISCELLANEOUS↗

Preconditioning for Hyper-reduction in Reduced Order Models

Many projection-based reduced order models (pROM) that utilize the governing equation and data to accelerate physical simulations can be applied to nonlinear dynamical systems. To avoid full order model (FOM) scale update for each time step, hyperreduction techniques are developed to sample high dimensional nonlinear terms. Our study aims to investigate if preconditioning the least-squares problem used for the nonlinear approximation can improve the robustness of the condition number of the problem while achieving high accuracy. In our study, we use the row-normalization matrix motivated by the Christoffel function as the preconditioner and solve the corresponding weighted least-squares problem. Numerical results for Lagrangian hydrodynamics examples are analyzed to explore how the preconditioner works compared to existing hyper-reduction techniques.

97 MATHEMATICS AND COMPUTING↗

LOCO with Constraints and Improved Fitting Technique

LOCO has been a powerful beam-based diagnostics and optics control method for storage rings and synchrotrons worldwide ever since it was established at NSLS by J. Safranek. This method measures the orbit response matrix and optionally the dispersion function of the machine. The data are then fitted to a lattice model by adjusting parameters such as quadrupole and skew quadrupole strengths in the model, BPM gains and rolls, corrector gains and rolls of the measurement system. Any abnormality of the machine that affects the machine optics can then be identified. The resulting lattice model is equivalent to the real machine lattice as seen by the BPMs. Since there are usually two or more BPMs per betatron period in modern circular accelerators, the model is often a very accurate representation of the real machine. According to the fitting result, one can correct the machine lattice to the design lattice by changing the quadrupole and skew quadrupole strengths. LOCO is so important that it is routinely performed at many electron storage rings to guarantee machine performance, especially after the Matlab-based LOCO code became available. However, for some machines, LOCO is not easy to carry out. In some cases, LOCO fitting converges to an unrealistic solution with large changes to the quadrupole strengths {Delta}K. The quadrupole gradient changes can be so large that the resulting lattice model fails to find a closed orbit and subsequent iterations become impossible. In cases when LOCO converges, the solution can have {Delta}K that is larger than realistic and often along with a spurious zigzag pattern between adjacent quadrupoles. This degeneracy behavior of LOCO is due to the correlation between the fitting parameters - usually between neighboring quadrupoles. The fitting scheme is therefore less restrictive over certain patterns of changes to these quadrupoles with which the correlated quadrupoles fight each other and the net effect is very inefficient {chi}{sup 2} reduction, i.e., small {chi}{sup 2} reduction with large changes of {Delta}K. Under effects of random noise, the fitting solution tends to crawl toward these patterns and ends up with unrealistically large {Delta}K. Such a solution is not very useful in optics correction because after the solution is dialed in, the quadrupoles will not respond as predicted by the lattice model due to magnet hysteresis. We will show that adding constraints to the fitting parameters is an effective way to combat this problem of LOCO. In fact, it improves optics calibration precision even for machines that don't show severe degeneracy behavior. LOCO fitting is essentially to solve a nonlinear least square problem with an iterative approach. The linear least square technique is applied in each iteration to move the solution toward the minimum. This approach is commonly referred to as the Gauss-Newton method. By using singular value decomposition (SVD) to invert the Jacobian matrix, this method has generally been very successful for LOCO. However, this method is based on a linear expansion of the residual vector over the fitting parameters which is valid only when the starting solution is sufficiently close to the real minimum. The fitting algorithm can have difficulties to converge when the initial guess is too far off. For example, it's possible for the {chi}{sup 2} merit function to increase after an iteration instead of decrease. This situation can be improved by using more robust nonlinear least square fitting algorithms, such as the Levenberg-Marquardt method. We will discuss the degeneracy problem in section 2 and then show how the constrained fitting can help in section 3. The application of Levenberg-Marquadt method to LOCO is shown in section 4. A summary is given in section 5.

43 PARTICLE ACCELERATORS↗

Time and Frequency Domain Methods for Basis Selection in Random Linear Dynamical Systems

Polynomial chaos methods have been extensively used to analyze systems in uncertainty quantification. Furthermore, several approaches exist to determine a low-dimensional approximation (or sparse approximation) for some quantity of interest in a model, where just a few orthogonal basis polynomials are required. In this work, we consider linear dynamical systems consisting of ordinary differential equations with random variables. The aim of this paper is to explore methods for producing low-dimensional approximations of the quantity of interest further. We investigate two numerical techniques to compute a low-dimensional representation, which both fit the approximation to a set of samples in the time domain. On the one hand, a frequency domain analysis of a stochastic Galerkin system yields the selection of the basis polynomials. It follows a linear least squares problem. On the other hand, a sparse minimization yields the choice of the basis polynomials by information from the time domain only. An orthogonal matching pursuit produces an approximate solution of the minimization problem. Finally, we compare the two approaches using a test example from a mechanical application.

97 MATHEMATICS AND COMPUTING↗

A line search algorithm for wind field adjustment with incomplete data and RBF approximation

The problem of concern in this work is the construction of free divergence fields given scattered horizontal components. As customary, the problem is formulated as a PDE constrained least squares problem. The novelty of our approach is to construct the so-called adjusted field, as the unique solution along an appropriately chosen descent direction. The latter is obtained by the adjoint equation technique. It is shown that the classical adjusted field of Sasaki’s is a particular case. On choosing descent directions, the underlying mass consistent model leads to the solution of an elliptic problem which is solved by means of a radial basis functions method. Finally, some numerical results for wind field adjustment are presented.

97 MATHEMATICS AND COMPUTING↗

A moving discontinuous Galerkin finite element method with interface condition enforcement for compressible flows

A variation of moving discontinuous Galerkin finite element method with interface condition enforcement (MDG-ICE) is developed for solving the compressible Euler equations. The MDG-ICE method, originating from the work of Corrigan et al. [1], [2], [3], [4], is based on the space-time DG formulation, where both flow field and grid geometry are considered as independent variables and the conservation laws are enforced both on discrete elements and element interfaces. The element conservation laws are solved in the standard discontinuous solution space to determine conservative quantities, while the interface conservation is enforced using a variational formulation in a continuous space to determine discrete grid geometry. The resulting over-determined system of nonlinear equations arising from the MDG-ICE formulation can then be solved in a least-squares sense, leading to an unconstrained nonlinear least-squares problem that is regularized and solved by Levenberg-Marquardt method. A number of numerical experiments for both 1D unsteady and 2D steady state compressible flow problems are conducted to assess the accuracy and robustness of the MDG-ICE method. Numerical results obtained indicate that the MDG-ICE method is able to implicitly detect and track all types of discontinuities via interface conservation enforcement and satisfy the conservation law on both elements and interfaces via grid movement and grid management, demonstrating that an exponential rate of convergence for Sod and Lax-Harden shock tube problems can be achieved and highly accurate solutions without overheating to both double-rarefaction wave and Noh problems can be obtained.

97 MATHEMATICS AND COMPUTING↗

State and Parameter Estimation for Natural Gas Pipeline Networks Using Transient State Data

Here, we formulate two estimation problems for pipeline systems in which measurements of the compressible gas flowing through a network of pipes are affected by time-varying injections, withdrawals, and compression. We consider a state estimation problem that is then extended to a joint state and parameter estimation problem that can be used for data assimilation. In both formulations, the flow dynamics are described on each pipe by space- and time-dependent densities and mass flux which evolve according to a system of coupled partial differential equations, in which momentum dissipation is modeled using the Darcy-Wiesbach friction approximation. These dynamics are first spatially discretized to obtain a system of nonlinear ordinary differential equations on which state and parameter estimation formulations are given as nonlinear least squares problems. A rapid, scalable computational method for performing a nonlinear least squares estimation is developed. Extensive simulations and computational experiments on multiple pipeline test networks demonstrate the effectiveness of the formulations in obtaining state and parameter estimates in the presence of measurement and process noise.

03 NATURAL GAS↗

A new accelerated algorithm for ill-conditioned ridge regression problems

In this work, we study the primal and dual formulations of the regularized least squares problem, in special norm L{sub 2}, named ridge regression. We state the Gradient method for the corresponding primal-dual problem through a fixed point approach. For this formulation, we present an original convergence analysis involving the spectral radius of a suitable iteration matrix. The main contribution of this work is to accelerate the studied method by means of a memory-like strategy. Some connections with the spectral properties of the associated iteration matrices were established to prove the convergence of the accelerated method. Preliminary experiments showed the good performance of the proposed algorithm when applied to solve ill-conditioned problems, providing better results than the conjugate gradient method for this class of problems.

97 MATHEMATICS AND COMPUTING↗