Search NASA⌕ Search

SEARCH · Search NASA

Results for “Tensor rank determination”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

General-Purpose Bayesian Tensor Learning With Automatic Rank Determination and Uncertainty Quantification

A major challenge in many machine learning tasks is that the model expressive power depends on model size. Low-rank tensor methods are an efficient tool for handling the curse of dimensionality in many large-scale machine learning models. The major challenges in training a tensor learning model include how to process the high-volume data, how to determine the tensor rank automatically, and how to estimate the uncertainty of the results. While existing tensor learning focuses on a specific task, this paper proposes a generic Bayesian framework that can be employed to solve a broad class of tensor learning problems such as tensor completion, tensor regression, and tensorized neural networks. We develop a low-rank tensor prior for automatic rank determination in nonlinear problems. Our method is implemented with both stochastic gradient Hamiltonian Monte Carlo (SGHMC) and Stein Variational Gradient Descent (SVGD). We compare the automatic rank determination and uncertainty quantification of these two solvers. We demonstrate that our proposed method can determine the tensor rank automatically and can quantify the uncertainty of the obtained results. We validate our framework on tensor completion tasks and tensorized neural network training tasks.

Bayesian inference↗

Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination

Post-training model compression can reduce the inference costs of deep neural networks, but uncompressed training still consumes enormous hardware resources and energy. To enable low-energy training on edge devices, it is highly desirable to directly train a compact neural network from scratch with a low memory cost. Low-rank tensor decomposition is an effective approach to reduce the memory and computing costs of large neural networks. However, directly training low-rank tensorized neural networks is a very challenging task because it is hard to determine a proper tensor rank a priori, and the tensor rank controls both model complexity and accuracy. Here, this paper presents a novel end-to-end framework for low-rank tensorized training. We first develop a Bayesian model that supports various low-rank tensor formats (e.g., CANDECOMP/PARAFAC, Tucker, tensor-train, and tensor-train matrix) and reduces neural network parameters with automatic rank determination during training. Then we develop a customized Bayesian solver to train large-scale tensorized neural networks. Our training methods shows orders-of-magnitude parameter reduction and little accuracy loss (or even better accuracy) in the experiments. On a very large deep learning recommendation system with over 4.2 ×10 9 model parameters, our method can reduce the parameter number to 1.6 ×10 5 automatically in the training process (i.e., by 2.6 ×10 4 times) while achieving almost the same accuracy. Code is available at https://github.com/colehawkins/bayesian-tensor-rank-determination.

compact neural networks↗

Bayesian tensorized neural networks with automatic rank selection

Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank. In order to address this challenge, this paper proposes a low-rank Bayesian tensorized neural network. Our Bayesian method performs automatic model compression via an adaptive tensor rank determination. We also present approaches for posterior density calculation and maximum a posteriori (MAP) estimation for the end-to-end training of our tensorized neural network. Here, we provide experimental validation on a two-layer fully connected neural network, a 6-layer CNN and a 110-layer residual neural network where our work produces 7.4x to 137x more compact neural networks directly from the training while achieving high prediction accuracy.

Low-rank tensor↗

Circular Photocurrent in Weyl Semimetals with Mirror Symmetry

We have considered theoretically Weyl’s semimetals, for which the point symmetry group contains reflection planes and which have equivalent valleys with opposite chiralities. These materials include the most often studied compounds, viz., monopnictides TaAs, NbAs, TaP, NbP of transition metals, as well as Bi{sub 1 ‒} {sub x}Sb{sub x} alloys. Circular photogalvanic current inverting its direction upon the sign reversal of the circular polarization has been calculated for light absorption under direct optical transitions near Weyl points. The total contribution of all valleys to the photocurrent differs from zero beyond the limits of Weyl’s model when the spin-independent tilt linear in wavevector k or spin-dependent terms cubic in k are taken into account additionally in the effective electron Hamiltonian. When the tilt of the energy dispersion curve in a Weyl semimetal of symmetry C{sub 4v} is considered, the photogalvanic current can be expressed in terms of the components of rank-two symmetric tensor determining the energy spectrum of carriers near a Weyl’s node. At low temperatures, this contribution to the photocurrent is excited in a certain limited frequency interval Δ. The photocurrent associated with cubic corrections in the optical absorption region is proportional to the square of light frequency and is generated both within window Δ and beyond its limits.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Complete algebraic reduction of one-loop tensor Feynman integrals

We set up a new, flexible approach for the tensor reduction of one-loop Feynman integrals. The 5-point tensor integrals up to rank R=5 are expressed by 4-point tensor integrals of rank R-1, such that the appearance of the inverse 5-point Gram determinant is avoided. The 4-point tensor coefficients are represented in terms of 4-point integrals, defined in d dimensions, 4-2{epsilon}{<=}d{<=}4-2{epsilon}+2(R-1), with higher powers of the propagators. They can be further reduced to expressions which stay free of the inverse 4-point Gram determinants but contain higher-dimensional 4-point integrals with only the first power of scalar propagators, plus 3-point tensor coefficients. A direct evaluation of the higher-dimensional 4-point functions would avoid the appearance of inverse powers of the Gram determinants completely. The simplest approach, however, is to apply here dimensional recurrence relations in order to reduce them to the familiar 2- to 4-point functions in generic dimension d=4-2{epsilon}, introducing thereby coefficients with inverse 4-point Gram determinants up to power R for tensors of rank R. For small or vanishing Gram determinants--where this reduction is not applicable--we use analytic expansions in positive powers of the Gram determinants. Improving the convergence of the expansions substantially with Pade approximants we close up to the evaluation of the 4-point tensor coefficients for larger Gram determinants. Finally, some relations are discussed which may be useful for analytic simplifications of Feynman diagrams.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Complete reduction of one-loop tensor 5- and 6-point integrals

We perform a complete analytical reduction of general one-loop Feynman integrals with five and six external legs for tensors up to rank R=3 and 4, respectively. An elegant formalism with extensive use of signed minors is developed for the cancellation of inverse Gram determinants. The 6-point tensor functions of rank R are expressed in terms of 5-point tensor functions of rank R-1, and the latter are reduced to scalar four-, three-, and two-point functions. The resulting compact formulas allow both for a study of analytical properties and for efficient numerical programming. They are implemented in Fortran and Mathematica.

97 MATHEMATICS AND COMPUTING↗

Dirac matrices for Chern-Simons gravity

A genuine gauge theory for the Poincare, de Sitter or anti-de Sitter algebras can be constructed in (2n- 1)-dimensional spacetime by means of the Chern-Simons form, yielding a gravitational theory that differs from General Relativity but shares many of its properties, such as second order field equations for the metric. The particular form of the Lagrangian is determined by a rank n, symmetric tensor invariant under the relevant algebra. In practice, the calculation of this invariant tensor can be reduced to the computation of the trace of the symmetrized product of n Dirac Gamma matrices {Gamma}{sub ab} in 2n-dimensional spacetime. While straightforward in principle, this calculation can become extremely cumbersome in practice. For large enough n, existing computer algebra packages take an inordinate long time to produce the answer or plainly fail having used up all available memory. In this talk we show that the general formula for the trace of the symmetrized product of 2n Gamma matrices {Gamma}{sub ab} can be written as a certain sum over the integer partitions s of n, with every term being multiplied by a numerical cofficient {alpha}{sub s}. We then give a general algorithm that computes the {alpha}-coefficients as the solution of a linear system of equations generated by evaluating the general formula for different sets of tensors B{sup ab} with random numerical entries. A recurrence relation between different coefficients is shown to hold and is used in a second, 'minimal' algorithm to greatly speed up the computations. Runtime of the minimal algorithm stays below 1 min on a typical desktop computer for up to n = 25, which easily covers all foreseeable applications of the trace formula.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Piezo-optic tensor of crystals from quantum-mechanical calculations

An automated computational strategy is devised for the ab initio determination of the full fourth-rank piezo-optic tensor of crystals belonging to any space group of symmetry. Elastic stiffness and compliance constants are obtained as numerical first derivatives of analytical energy gradients with respect to the strain and photo-elastic constants as numerical derivatives of analytical dielectric tensor components, which are in turn computed through a Coupled-Perturbed-Hartree-Fock/Kohn-Sham approach, with respect to the strain. Both point and translation symmetries are exploited at all steps of the calculation, within the framework of periodic boundary conditions. The scheme is applied to the determination of the full set of ten symmetry-independent piezo-optic constants of calcium tungstate CaWO{sub 4}, which have recently been experimentally reconstructed. Present calculations unambiguously determine the absolute sign (positive) of the π{sub 61} constant, confirm the reliability of 6 out of 10 experimentally determined constants and provide new, more accurate values for the remaining 4 constants.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Compressed sparse tensor based quadrature for vibrational quantum mechanics integrals

A new method for fast evaluation of high dimensional integrals arising in quantum mechanics is proposed. Here, the method is based on sparse approximation of a high dimensional function followed by a low-rank compression. In the first step, we interpret the high dimensional integrand as a tensor in a suitable tensor product space and determine its entries by a compressed sensing based algorithm using only a few function evaluations. Secondly, we implement a rank reduction strategy to compress this tensor in a suitable low-rank tensor format using standard tensor compression tools. This allows representing a high dimensional integrand function as a small sum of products of low dimensional functions. Finally, a low dimensional Gauss–Hermite quadrature rule is used to integrate this low-rank representation, thus alleviating the curse of dimensionality. Finally, numerical tests on synthetic functions, as well as on energy correction integrals for water and formaldehyde molecules demonstrate the efficiency of this method using very few function evaluations as compared to other integration strategies.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Extended Lagrangian Born–Oppenheimer molecular dynamics using a Krylov subspace approximation

It is shown how the electronic equations of motion in extended Lagrangian Born–Oppenheimer molecular dynamics simulations can be integrated using low-rank approximations of the inverse Jacobian kernel. This kernel determines the metric tensor in the harmonic oscillator extension of the Lagrangian that drives the evolution of the electronic degrees of freedom. The proposed kernel approximation is derived from a pseudoinverse of a low-rank estimate of the Jacobian, which is expressed in terms of a generalized set of directional derivatives with directions that are given from a Krylov subspace approximation. The approach allows a tunable and adaptive approximation that can take advantage of efficient preconditioning techniques. The proposed kernel approximation for the integration of the electronic equations of motion makes it possible to apply extended Lagrangian first-principles molecular dynamics simulations to a broader range of problems, including reactive chemical systems with numerically sensitive and unsteady charge solutions. This can be achieved without requiring exact full calculations of the inverse Jacobian kernel in each time step or relying on iterative non-linear self-consistent field optimization of the electronic ground state prior to the force evaluations as in regular direct Born–Oppenheimer molecular dynamics. We note the low-rank approximation of the Jacobian is directly related to Broyden’s class of quasi-Newton algorithms and Jacobian-free Newton–Krylov methods and provides a complementary formulation for the solution of nonlinear systems of equations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Sparse low rank approximation of potential energy surfaces with applications in estimation of anharmonic zero point energies and frequencies

We introduce a method that exploits sparse representation of potential energy surfaces (PES) on a polynomial basis set selected by compressed sensing . The method is useful for studies involving large numbers of PES evaluations, such as the search for local minima, transition states, or integration. We apply this method for estimating zero point energies and frequencies of molecules using a three step approach. In the first step, we interpret the PES as a sparse tensor on polynomial basis and determine its entries by a compressed sensing based algorithm using only a few PES evaluations. Then, we establish a rank reduction strategy to compress this tensor in a suitable low-rank canonical tensor format using standard tensor compression tools. This allows representing a high dimensional PES as a small sum of products of one dimensional functions. Finally, a low dimensional Gauss–Hermite quadrature rule is used to integrate the product of sparse canonical low-rank representation of PES and Green’s function in the second-order diagrammatic vibrational many-body Green’s function theory (XVH2) for estimation of zero-point energies and frequencies. Numerical tests on molecules considered in this work indicate a more efficient scaling of computational cost with molecular size as compared to other methods.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Mechanisms of sequential particle transfer and characteristics of light neutron-excess and oriented nuclei

The procedure for evaluating the second-order corrections to the matrix elements of the reaction A(x, y)B, which are obtained using the method of distorted waves with a finite radius of intercluster interaction (DWBAFR), is developed. It is based on the assumption of a virtual cluster structure of light nuclei and uses integral equations for a four-body problem in the Alt-Grassberger-Sandhas formalism. These corrections are related with the mechanisms of sequential particles transfer. The latter are represented by the quadrangle diagrams. Their matrix elements are summed up coherently with those given by the pole and triangle diagrams which were calculated by using DWBAFR. The computer code QUADRO is written for the numerical implementation of the method proposed. The statistical tensors of nucleus B formed in the reaction A(x, y)B at incident particle energies of about 10 MeV/nucleon in the center of mass frame are determined. Specific calculations allowed for description of both the experimental cross sections (0-rank statistical tensors) of various reactions (including those where nucleus B has some excess neutrons) and polarized characteristics of nucleus B* (in the case of the latter produced in the exited state). A two-neutron periphery of nuclei {sup 6}He, {sup 10}Be, {sup 12}B (both in dineutron and cigarlike configurations) is restored by analyzing the differential cross sections of elastic alpha-6He-scattering and {sup 9}Be(d, p){sup 10}Be and {sup 10}B(t, p){sup 12}B reactions. It is shown that the structure of neutron peripheries is fundamentally different for these nuclei and its feature depends on the way those neutron-excess nuclei are formed: in {sup 6}He both configurations contribute to a two-neutron halo, while in {sup 10}Be there is a barely noticeable one-neutron halo, and in {sup 12}B there is a “dineutron skin”. Orientation characteristics of nuclei B* are calculated. Their comparison with experimental data made it possible to draw important conclusions about a contribution to the statistical tensors of nucleus B* coming from the two-step mechanisms and its impact on the properties of oriented light nuclei, including their polarization. Finally, a simplified method for calculating the matrix elements of mechanisms, which take into account sequential particle transfer, is proposed. It is demonstrated to be correct by evaluating a contribution of the corresponding corrections to the total amplitude of the reaction.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

A framework for moment equations for magnetized plasmas

Mathematical formalism to solve a system of general moment equations [J.-Y. Ji and E. D. Held, Phys. Plasmas 13, 102103 (2006); 16, 102108 (2009)] for magnetized plasmas is presented. A series of ordered moment equations are written using a perturbative expansion based on large cyclotron frequency. For the most general solution, formulas for homogeneous and particular solutions are obtained. These formulas generalize the CGL [G. F. Chew et al., Proc. R. Soc. London, Ser. A 236, 112 (1956)] and non-CGL [C. T. Hsu et al., Phys. Fluids 29, 1480 (1986)] tensors, respectively, from rank-2 to arbitrary rank. The parallel moment equations to determine parallel moments in the homogeneous solution are derived. The formalism can be applied to plasmas of general collisionality and magnetic geometry with accurate collision operators provided.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Compact video content representation for video coding using low multi-linear tensor rank approximation with dynamic core tensor order

Tensor is a multi-way representation of an entity that is widely used in many applications of mathematics and physics. The wide range of tensor algebra operations attract the researchers of graphical and vision field to identify unique methods for various applications such as segmentation, compression, indexing and so on. In this paper, low multi-linear rank approximation of a tensor is used for decomposition so as to perform compression on a video that is represented as a 4D tensor. Thus, the encoder has blocks for (1) multilinear rank approximation method so as to aid decomposition, (2) sparsity removal of residual data to reduce the required memory storage to a small level, (3) quantization on the sparse less residual data and (4) LZ77 dictionary coding for the video. The decoder uses a tensor reconstruction block along with residual error correction block that reduces loss/error in the reconstructed video. The size of core tensor is an important factor in rank approximation. It is identified at run time depending on the video contents. This size is determined adaptively using Tikhonov’s regularization method. The best value of core tensor size is identified with the corner of L-curve which preserves the video contents from heavy loss/error. Experimental results of the proposed work on comparison with H.264/AVC compression method indicate that the proposed method achieves up to 10% improvement in compression ratio with the least amount of loss and a maximum signal to noise ratio of 0.14. The proposed system gives better compression ratio than H.265 for videos with more number of frames without compromising quality.

97 MATHEMATICS AND COMPUTING↗

Motion of small bodies in classical field theory

I show how prior work with R. Wald on geodesic motion in general relativity can be generalized to classical field theories of a metric and other tensor fields on four-dimensional spacetime that (1) are second-order and (2) follow from a diffeomorphism-covariant Lagrangian. The approach is to consider a one-parameter-family of solutions to the field equations satisfying certain assumptions designed to reflect the existence of a body whose size, mass, and various charges are simultaneously scaled to zero. (That such solutions exist places a further restriction on the class of theories to which our results apply.) Assumptions are made only on the spacetime region outside of the body, so that the results apply independent of the body's composition (and, e.g., black holes are allowed). The worldline 'left behind' by the shrinking, disappearing body is interpreted as its lowest-order motion. An equation for this worldline follows from the 'Bianchi identity' for the theory, without use of any properties of the field equations beyond their being second-order. The form of the force law for a theory therefore depends only on the ranks of its various tensor fields; the detailed properties of the field equations are relevant only for determining the charges for a particular body (which are the ''monopoles'' of its exterior fields in a suitable limiting sense). I explicitly derive the force law (and mass-evolution law) in the case of scalar and vector fields, and give the recipe in the higher-rank case. Note that the vector force law is quite complicated, simplifying to the Lorentz force law only in the presence of the Maxwell gauge symmetry. Example applications of the results are the motion of 'chameleon' bodies beyond the Newtonian limit, and the motion of bodies in (classical) non-Abelian gauge theory. I also make some comments on the role that scaling plays in the appearance of universality in the motion of bodies.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Density-Matrix Based Extended Lagrangian Born–Oppenheimer Molecular Dynamics

Extended Lagrangian Born–Oppenheimer molecular dynamics [ Phys. Rev. Lett. 2008, 100, 123004] is presented for Hartree–Fock theory, where the extended electronic degrees of freedom are represented by a density matrix, including fractional occupation numbers at elevated electronic temperatures. In contrast to regular direct Born–Oppenheimer molecular dynamics simulations, no iterative self-consistent field optimization is required prior to the force evaluations. To sample regions of the potential energy landscape where the gap is small or vanishing, which leads to particular convergence problems in regular direct Born–Oppenheimer molecular dynamics simulations, an adaptive integration scheme for the extended electronic degrees of freedom is presented. The integration scheme is based on a tunable, low-rank approximation of a fourth-order kernel, K, that determines the metric tensor, T ≡ K T K, used in the extended harmonic oscillator of the Lagrangian that generates the dynamics of the electronic degrees of freedom. Here, the formulation and algorithms provide a general guide to implement extended Lagrangian Born–Oppenheimer molecular dynamics for quantum chemistry, density functional theory, and semiempirical methods using a density matrix formalism.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Self-diffusion in the non-Newtonian regime of shearing liquid crystal model systems based on the Gay-Berne potential

The self-diffusion coefficients of nematic phases of various model systems consisting of regular convex calamitic and discotic ellipsoids and non-convex bodies such as bent-core molecules and soft ellipsoid strings have been obtained as functions of the shear rate in a shear flow. Then the self-diffusion coefficient is a second rank tensor with three different diagonal components and two off-diagonal components. These coefficients were found to be determined by a combination of two mechanisms, which previously have been found to govern the self-diffusion of shearing isotropic liquids, namely, (i) shear alignment enhancing the diffusion in the direction parallel to the streamlines and hindering the diffusion in the perpendicular directions and (ii) the distortion of the shell structure in the liquid whereby a molecule more readily can escape from a surrounding shell of nearest neighbors, so that the mobility increases in every direction. Thus, the diffusion parallel to the streamlines always increases with the shear rate since these mechanisms cooperate in this direction. In the perpendicular directions, these mechanisms counteract each other so that the behaviour becomes less regular. In the case of the nematic phases of the calamitic and discotic ellipsoids and of the bent core molecules, mechanism (ii) prevails so that the diffusion coefficients increase. However, the diffusion coefficients of the soft ellipsoid strings decrease in the direction of the velocity gradient because the broadsides of these molecules are oriented perpendicularly to this direction due the shear alignment (i). The cross coupling coefficient relating a gradient of tracer particles in the direction of the velocity gradient and their flow in the direction of the streamlines is negative and rather large, whereas the other coupling coefficient relating a gradient in the direction of the streamlines and a flow in the direction of the velocity gradient is very small.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Anisotropic Cahn-Hilliard free energy and interfacial energies for binary alloys with pairwise interactions

The original Cahn-Hilliard derivation of the contribution of compositional inhomogeneity to the free energy of a binary alloy with pairwise interactions is extended to include higher-order inhomogeneity terms. For alloys on a cubic lattice, the coefficient of the first inhomogeneity is a second-rank tensor and reduces to a scalar, but it is shown that the second order and the third order inhomogeneity terms are weighted by fourth-rank and sixth-rank tensors, thus resulting in anisotropic contributions. Furthermore, each interaction shell generates a unique set of inhomogeneity coefficients that is determined by the set of vectors connecting an atom to its neighbors on that shell. These coefficients are calculated for fcc and bcc alloys with interactions up fourth nearest neighbors. Phase field simulations based on these extended Cahn-Hilliard free energies are performed to measure interface free energies along specific crystallographic directions as a function of temperature, and to obtain the equilibrium shape of precipitates. Interface free energies, and the resulting anisotropies, are compared to those obtained by discrete models and Monte Carlo simulations.

36 MATERIALS SCIENCE↗