Block Gram-Schmidt algorithms and their stability properties
Not Available
Carson, Erin (ORCID:0000000194697467)↗
SEARCH · Search NASA
Results for “Gram-Schmidt”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Gram-Schmidt algorithms for covariance propagation
This paper addresses the time propagation of triangular covariance factors. Attention is focused on the square-root free factorization, P = UDU/T/, where U is unit upper triangular and D is diagonal. An efficient and reliable algorithm for U-D propagation is derived which employs Gram-Schmidt orthogonalization. Partitioning the state vector to distinguish bias and colored process noise parameters increases mapping efficiency. Cost comparisons of the U-D, Schmidt square-root covariance and conventional covariance propagation methods are made using weighted arithmetic operation counts. The U-D time update is shown to be less costly than the Schmidt method; and, except in unusual circumstances, it is within 20% of the cost of conventional propagation.
Thornton, C. L.↗
Gram-Schmidt algorithms for covariance propagation
This paper addresses the time propagation of triangular covariance factors. Attention is focused on the square-root free factorization, P = UD(transpose of U), where U is unit upper triangular and D is diagonal. An efficient and reliable algorithm for U-D propagation is derived which employs Gram-Schmidt orthogonalization. Partitioning the state vector to distinguish bias and coloured process noise parameters increase mapping efficiency. Cost comparisons of the U-D, Schmidt square-root covariance and conventional covariance propagation methods are made using weighted arithmetic operation counts. The U-D time update is shown to be less costly than the Schmidt method; and, except in unusual circumstances, it is within 20% of the cost of conventional propagation.
Thornton, C. L.↗
Variational calculations of subbands in a quantum well with uniform electric field - Gram-Schmidt orthogonalization approach
Variational calculations of subband eigenstates in an infinite quantum well with an applied electric field using Gram-Schmidt orthogonalized trial wave functions are presented. The results agree very well with the exact numerical solutions even up to 1200 kV/cm. It is also shown that, for increasing electric fields, the energy of the ground state decreases, while that of higher subband states increases slightly up to 1000 kV/cm and then decreases for a well size of 100 A.
Ahn, Doyeol↗
Low-synch Gram–Schmidt with delayed reorthogonalization for Krylov solvers
The parallel strong-scaling of iterative methods is often determined by the number of global reductions at each iteration. Low-synch Gram-Schmidt algorithms are applied here to the Arnoldi algorithm to reduce the number of global reductions and therefore to improve the parallel strong-scaling of iterative solvers for nonsymmetric matrices such as the GMRES and the Krylov-Schur iterative methods. In the Arnoldi context, the factorization is "left-looking" and processes one column at a time. Among the methods for generating an orthogonal basis for the Arnoldi algorithm, the classical Gram-Schmidt algorithm, with reorthogonalization (CGS2) requires three global reductions per iteration. A new variant of CGS2 that requires only one reduction per iteration is presented and applied to the Arnoldi algorithm. Delayed CGS2 (DCGS2) employs the minimum number of global reductions per iteration (one) for a one-column at-a-time algorithm. The main idea behind the new algorithm is to group global reductions by rearranging the order of operations. DCGS2 must be carefully integrated into an Arnoldi expansion or a GMRES solver. Numerical stability experiments assess robustness for Krylov-Schur eigenvalue computations. Performance experiments on the ORNL Summit supercomputer then establish the superiority of DCGS2 over CGS2.
97 MATHEMATICS AND COMPUTING↗
Iterated Gauss-Seidel GMRES
The GMRES algorithm of Saad and Schultz [SIAM J. Sci. Stat. Comput., 7 (1986), pp. 856-869] is an iterative method for approximately solving linear systems Ax = b, with initial guess x0 and residual r0 = b Ax0. The algorithm employs the Arnoldi process to generate the Krylov basis vectors (the columns of Vk ). It is well known that this process can be viewed as a QR factorization of the matrix Bk = [r0, AVk] at each iteration. Despite an O (..epsilon..)..kappa.. (Bk ) loss of orthogonality, for unit roundoff ..epsilon..and condition number ..kappa.. , the modified Gram-Schmidt formulation was shown to be backward stable in the seminal paper by Paige et al. [SIAM J. Matrix Anal.Appl., 28 (2006), pp. 264-284]. We present an iterated Gauss-Seidel formulation of the GMRES algorithm (IGS-GMRES) based on the ideas of Ruhe [Linear Algebra Appl., 52 (1983), pp. 591-601] and Swirydowicz et al. [Numer. Linear Algebra Appl., 28 (2020), pp. 1-20]. IGS-GMRES maintains orthogonality to the level O (..epsilon..)..kappa.. (Bk ) or O (..epsilon..), depending on the choice of one or two iterations; for two Gauss-Seidel iterations, the computed Krylov basis vectors remain orthogonal to working accuracy and the smallest singular value of Vk remains close to one. The resulting GMRES method is thus backward stable. We show that IGS-GMRES can be implemented with only a single synchronization point per iteration, making it relevant to large-scale parallel computing environments. We also demonstrate that, unlike MGS-GMRES, in IGS-GMRES the relative Arnoldi residual corresponding to the computed approximate solution no longer stagnates above machine precision even for highly nonnormal systems.
Arnoldi-QR↗
Automatic Determination of the Weak-Beam Condition in Dark Field X-ray Microscopy
Mechanical properties in crystals are strongly correlated to the arrangement of 1D line defects, termed dislocations. Recently, Dark field X-ray Microscopy (DFXM) has emerged as a new tool to image and interpret dislocations within crystals using multidimensional scans. However, the methods required to reconstruct meaningful dislocation information from high-dimensional DFXM scans are still nascent and require significant manual oversight (i.e. supervision). In this work, we present a new relatively unsupervised method that extracts dislocation-specific information (features) from a 3D dataset (x, y, Φ) using Gram-Schmidt orthogonalization to represent the large dataset as an array of 3-component feature vectors for each position, corresponding to the weak-beam conditions and the strong-beam condition. Finally, this method offers key opportunities to significantly reduce dataset size while preserving only the crystallographic information that is important for data reconstruction.
36 MATERIALS SCIENCE↗
A dictionary learning algorithm for compression and reconstruction of streaming data in preset order
There has been an emerging interest in developing and applying dictionary learning (DL) to process massive datasets in the last decade. Many of these efforts, however, focus on employing DL to compress and extract a set of important features from data, while considering restoring the original data from this set a secondary goal. On the other hand, although several methods are able to process streaming data by updating the dictionary incrementally as new snapshots pass by, most of those algorithms are designed for the setting where the snapshots are randomly drawn from a probability distribution. In this paper, we present a new DL approach to compress and denoise massive dataset in real time, in which the data are streamed through in a preset order (instances are videos and temporal experimental data), so at any time, we can only observe a biased sample set of the whole data. Here, our approach incrementally builds up the dictionary in a relatively simple manner: if the new snapshot is adequately explained by the current dictionary, we perform a sparse coding to find its sparse representation; otherwise, we add the new snapshot to the dictionary, with a Gram-Schmidt process to maintain the orthogonality. To compress and denoise noisy datasets, we apply the denoising to the snapshot directly before sparse coding, which deviates from traditional dictionary learning approach that achieves denoising via sparse coding. Compared to full-batch matrix decomposition methods, where the whole data is kept in memory, and other mini-batch approaches, where unbiased sampling is often assumed, our approach has minimal requirement in data sampling and storage: i) each snapshot is only seen once then discarded, and ii) the snapshots are drawn in a preset order, so can be highly biased. Through experiments on climate simulations and scanning transmission electron microscopy (STEM) data, we demonstrate that the proposed approach performs competitively to those methods in data reconstruction and denoising.
97 MATHEMATICS AND COMPUTING↗
On the orthonormalization of sparse vectors.
Optimum order for column orthonormalization of sparse matrices, applying results to Gram-Schmidt and Householder methods
Tewarson, R. P.↗
HADY-I, a FORTRAN program for the compressible stability analysis of three-dimensional boundary layers
A computer program HADY-I for calculating the linear incompressible or compressible stability characteristics of the laminar boundary layer on swept and tapered wings is described. The eigenvalue problem and its adjoint arising from the linearized disturbance equations with the appropriate boundary conditions are solved numerically using a combination of Newton-Raphson interative scheme and a variable step size integrator based on the Runge-Kutta-Fehlburh fifth-order formulas. The integrator is used in conjunction with a modified Gram-Schmidt orthonormalization procedure. The computer program HADY-I calculates the growth rates of crossflow or streamwise Tollmien-Schlichting instabilities. It also calculates the group velocities of these disturbances. It is restricted to parallel stability calculations, where the boundary layer (meanflow) is assumed to be parallel. The meanflow solution is an input to the program.
El-Hady, N. M.↗
Nonlinear optimization with linear constraints using a projection method
Nonlinear optimization problems that are encountered in science and industry are examined. A method of projecting the gradient vector onto a set of linear contraints is developed, and a program that uses this method is presented. The algorithm that generates this projection matrix is based on the Gram-Schmidt method and overcomes some of the objections to the Rosen projection method.
Fox, T.↗
Solving large-scale dynamic systems using band Lanczos method in Rockwell NASTRAN on CRAY X-MP
The improved cost effectiveness using better models, more accurate and faster algorithms and large scale computing offers more representative dynamic analyses. The band Lanczos eigen-solution method was implemented in Rockwell's version of 1984 COSMIC-released NASTRAN finite element structural analysis computer program to effectively solve for structural vibration modes including those of large complex systems exceeding 10,000 degrees of freedom. The Lanczos vectors were re-orthogonalized locally using the Lanczos Method and globally using the modified Gram-Schmidt method for sweeping rigid-body modes and previously generated modes and Lanczos vectors. The truncated band matrix was solved for vibration frequencies and mode shapes using Givens rotations. Numerical examples are included to demonstrate the cost effectiveness and accuracy of the method as implemented in ROCKWELL NASTRAN. The CRAY version is based on RPK's COSMIC/NASTRAN. The band Lanczos method was more reliable and accurate and converged faster than the single vector Lanczos Method. The band Lanczos method was comparable to the subspace iteration method which was a block version of the inverse power method. However, the subspace matrix tended to be fully populated in the case of subspace iteration and not as sparse as a band matrix.
Gupta, V. K.↗
Recursive form of the eigensystem realization algorithm for system identification
An algorithm is developed for recursively calculating the minimum realization of a linear system from sampled impulse response data. The Gram-Schmidt orthonormalization technique is used to generate an orthonormal basis for factorization of the data matrix. The system matrix thus identified is in upper Hessenberg form, which has advantages for the identification of modal parameters including damping coefficients, frequencies, mode shapes, and modal participation factors. It also has the property that once an element of the system matrix is computed, it is never altered as the dimension of the model is increased in the recursive process. Numerical examples are presented for comparison of the recursive and nonrecursive forms of the eigensystem realization algorithm.
Longman, Richard W.↗
On recursive least-squares filtering algorithms and implementations
In many real-time signal processing applications, fast and numerically stable algorithms for solving least-squares problems are necessary and important. In particular, under non-stationary conditions, these algorithms must be able to adapt themselves to reflect the changes in the system and take appropriate adjustments to achieve optimum performances. Among existing algorithms, the QR-decomposition (QRD)-based recursive least-squares (RLS) methods have been shown to be useful and effective for adaptive signal processing. In order to increase the speed of processing and achieve high throughput rate, many algorithms are being vectorized and/or pipelined to facilitate high degrees of parallelism. A time-recursive formulation of RLS filtering employing block QRD will be considered first. Several methods, including a new non-continuous windowing scheme based on selectively rejecting contaminated data, were investigated for adaptive processing. Based on systolic triarrays, many other forms of systolic arrays are shown to be capable of implementing different algorithms. Various updating and downdating systolic algorithms and architectures for RLS filtering are examined and compared in details, which include Householder reflector, Gram-Schmidt procedure, and Givens rotation. A unified approach encompassing existing square-root-free algorithms is also proposed. For the sinusoidal spectrum estimation problem, a judicious method of separating the noise from the signal is of great interest. Various truncated QR methods are proposed for this purpose and compared to the truncated SVD method. Computer simulations provided for detailed comparisons show the effectiveness of these methods. This thesis deals with fundamental issues of numerical stability, computational efficiency, adaptivity, and VLSI implementation for the RLS filtering problems. In all, various new and modified algorithms and architectures are proposed and analyzed; the significance of any of the new method depends crucially on specific application.
Hsieh, Shih-Fu↗
Stiffness-generated rigid-body mode shapes for Lanczos eigensolution with SUPORT DOF by way of a MSC/NASTRAN DMAP alter
When using all MSC/NASTRAN eigensolution methods except Lanczos, the analyst can replace the coupled system rigid-body modes calculated within DMAP module READ with mass orthogonalized and normalized rigid-body modes generated from the system stiffness. This option is invoked by defining MSC/NASTRAN r-set degrees of freedom via the SUPORT bulk data card. The newly calculated modes are required if the rigid-body modes calculated by the eigensolver are not 'clean' due to numerical roundoffs in the solution. When performing transient structural dynamic load analysis, the numerical roundoffs can result in inaccurate rigid-body accelerations which affect steady-state responses. Unfortunately, when using the Lanczos method and defining r-set degrees of freedom, the rigid-body modes calculated within DMAP module REIGL are retained. To overcome this limitation and to allow MSC/NASTRAN to handle SUPORT degrees of freedom identically for all eigensolvers, a DMAP Alter has been written which replaces Lanczos-calculated rigid-body modes with stiffness-generated rigid-body modes. The newly generated rigid-body modes are normalized with respect to the system mass and orthogonalized using the Gram-Schmidt technique. This algorithm has been implemented as an enhancement to an existing coupled loads methodology.
Abdallah, Ayman A.↗
Method of Conjugate Radii for Solving Linear and Nonlinear Systems
This paper describes a method to solve a system of N linear equations in N steps. A quadratic form is developed involving the sum of the squares of the residuals of the equations. Equating the quadratic form to a constant yields a surface which is an ellipsoid. For different constants, a family of similar ellipsoids can be generated. Starting at an arbitrary point an orthogonal basis is constructed and the center of the family of similar ellipsoids is found in this basis by a sequence of projections. The coordinates of the center in this basis are the solution of linear system of equations. A quadratic form in N variables requires N projections. That is, the current method is an exact method. It is shown that the sequence of projections is equivalent to a special case of the Gram-Schmidt orthogonalization process. The current method enjoys an advantage not shared by the classic Method of Conjugate Gradients. The current method can be extended to nonlinear systems without modification. For nonlinear equations the Method of Conjugate Gradients has to be augmented with a line-search procedure. Results for linear and nonlinear problems are presented.
Nachtsheim, Philip R.↗
Zernike-like Orthogonal Basis Functions for Wavefront Characterization over Sampled, Irregular Apertures
For optical systems with circular apertures, wavefronts are often analyzed using Zernike polynomials, and individual Zernike functions are associated with familiar optical aberrations. For systems with noncircular apertures, or in practical situations in which wavefronts are measured at a limited number of points in the aperture, the Zernike polynomials are no longer an orthogonal basis for the measured data. Although there are an endless number of ways to create a basis for such measured data, a "Zernike-like" basis is useful to connect with our experience with the usual optical aberrations. In this paper, the steps required to identify a Zernike-like basis for wavefronts over sampled, irregular apertures are presented, based on the Gram-Schmidt orthogonalization technique. The benefits of analyzing optical wavefronts using an orthogonal basis specific to an optical system's aperture shape and wavefront sampling, instead of using the traditional Zernike polynomials, are detailed in two examples, from image-based wavefront sensing on a segmented-aperture telescope (the James Webb Space Telescope Testbed Telescope at Ball Aerospace) and from interferometer characterization for surface metrology of a hexagonal mirror segment.
Aronstein, David L.↗
Updates to Multi-Dimensional Flux Reconstruction for Hypersonic Simulations on Tetrahedral Grids
The quality of simulated hypersonic stagnation region heating with tetrahedral meshes is investigated by using an updated three-dimensional, upwind reconstruction algorithm for the inviscid flux vector. An earlier implementation of this algorithm provided improved symmetry characteristics on tetrahedral grids compared to conventional reconstruction methods. The original formulation however displayed quantitative differences in heating and shear that were as large as 25% compared to a benchmark, structured-grid solution. The primary cause of this discrepancy is found to be an inherent inconsistency in the formulation of the flux limiter. The inconsistency is removed by employing a Green-Gauss formulation of primitive gradients at nodes to replace the previous Gram-Schmidt algorithm. Current results are now in good agreement with benchmark solutions for two challenge problems: (1) hypersonic flow over a three-dimensional cylindrical section with special attention to the uniformity of the solution in the spanwise direction and (2) hypersonic flow over a three-dimensional sphere. The tetrahedral cells used in the simulation are derived from a structured grid where cell faces are bisected across the diagonal resulting in a consistent pattern of diagonals running in a biased direction across the otherwise symmetric domain. This grid is known to accentuate problems in both shock capturing and stagnation region heating encountered with conventional, quasi-one-dimensional inviscid flux reconstruction algorithms. Therefore the test problems provide a sensitive indicator for algorithmic effects on heating. Additional simulations on a sharp, double cone and the shuttle orbiter are then presented to demonstrate the capabilities of the new algorithm on more geometrically complex flows with tetrahedral grids. These results provide the first indication that pure tetrahedral elements utilizing the updated, three-dimensional, upwind reconstruction algorithm may be used for the simulation of heating and shear in hypersonic flows in upwind, finite volume formulations.
Gnoffo, Peter A.↗