Efficient Floating-Point Arithmetic on Fault-Tolerant Quantum Computers
Explore the source record for details and available documents.
O., Oluwadara↗
SEARCH · Search NASA
Results for “Arithmetic”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Increasing Accuracy of Iterative Refinement in Limited Floating-Point Arithmetic on Half-Precision Accelerators.
Abstract not provided.
Yamazaki, Ichitaro↗
Dynamic variable precision computation
A conversion unit converts operands from a conventional number system that represents each binary number in the operands as one bit to redundant number system (RNS) operands that represent each binary number as a plurality of bits. An arithmetic logic unit performs an arithmetic operation on the RNS operands in a direction from a most significant bit (MSB) to a least significant bit (LSB). The arithmetic logic unit stops performing the arithmetic operation prior to performing the arithmetic operation on a target binary number indicated by a dynamic precision associated with the RNS operands. In some cases, a power supply provides power to bit slices in the arithmetic logic unit and a clock signal generator provides clock signals to the bit slices. Gate logic is configured to gate the power or the clock signals provided to a subset of the bit slices.
97 MATHEMATICS AND COMPUTING↗
Dynamic variable precision computation
A conversion unit converts operands from a conventional number system that represents each binary number in the operands as one bit to redundant number system (RNS) operands that represent each binary number as a plurality of bits. An arithmetic logic unit performs an arithmetic operation on the RNS operands in a direction from a most significant bit (MSB) to a least significant bit (LSB). The arithmetic logic unit stops performing the arithmetic operation prior to performing the arithmetic operation on a target binary number indicated by a dynamic precision associated with the RNS operands. In some cases, a power supply provides power to bit slices in the arithmetic logic unit and a clock signal generator provides clock signals to the bit slices. Gate logic is configured to gate the power or the clock signals provided to a subset of the bit slices.
97 MATHEMATICS AND COMPUTING↗
Fine-Grained Exploitation of Mixed Precision for Faster CNN Training
As deep convolutional neural networks (CNNs) have become increasingly popular and successful at an ever-widening number of machine learning tasks specialized hardware has become increasingly available for training and deploying them. NVIDIA's recent Volta architecture includes tensor cores which perform a fused operation reduced and mixed precision (16-bit multiply, 32-bit accumulate). Recent research indicates that, typically, very little is lost (in terms of training accuracy) when half precision is used in place of single precision, and performance gains can be made by doing arithmetic in reduced precision. In this work we demonstrate that making layer-by-layer choices as to the arithmetic/data precision can lead to further performance improvement. In our study of 25,200 CNNs we demonstrate an average speedup (over purely half precision) of 1.27x and speedups as high as 3.64x by appropriately combining single and half precision arithmetic and data types on a layer-by-layer basis.
Johnston, Travis↗
TabVar: Tabulated Variables
TabVar: A Python library for manipulating datasets in the form of tabulated variables. Tables in tabvar contain many columns representing independent variables, but exactly one distinguished column for the dependent variable. Having a single distinguished column allows a natural lifting of arithmetic operators to tables, much (and in fact fully generalizing) multidimensional array arithmetic. The convenient syntax of whole-table arithmetic, along with the usual operations of filtering and aggregation, and all in the setting of python's interactive REPL allows for rapid exploration of datasets.
Bachan, John↗
Technology Performance Level (TPL) Scoring Tool
Three different ways of combining scores are used in the revised formulation. These are arithmetic mean, geometric mean and multiplication with normalisation. Arithmetic mean is used when combining scores that measure similar attributes, e.g. used for combining costs. The arithmetic mean has the property that it is similar to a logical OR, e.g. when combining costs it does not matter what the individual costs are only what the combined cost is. Geometric mean and Multiplication are used when combining scores that measure disparate attributes. Multiplication is similar to a logical AND, it is used to combine ‘must haves.’ As a result, this method is more punitive than the geometric mean; to get a good score in the combined result it is necessary to have a good score in ALL of the inputs. e.g. the different types of survivability are ‘must haves.’ On balance, the revised TPL is probably less punitive than the previous spreadsheet, multiplication is used sparingly as a method of combining scores. This is in line with the feedback of the Wave Energy Prize judges.
97 MATHEMATICS AND COMPUTING↗
Technical report: Design of the accessor
As we have seen the arithmetic power of hardware architectures growing much faster than the memory bandwidth, there is an urgent need to rethink the way we operate, communicate, and store data. A promising strategy for overcoming the “memory wall” is the decoupling of memory precision from arithmetic precision as proposed in Anzt et al. Such a decoupling of memory precision and arithmetic precision requires an “accessor” that handles the conversion between formats (resp. compression of the data) on the fly in registers. Such an accessor has to be general in the sense that it can handle a wide range of floating point formats and compression strategies, and still allows for further extension. At the same time, is important that the use of the accessor is not a burden for the developer. Therefore, it is important to abstract all implementation details from the user, hide the underlying storage format from the user, and make the accessor cross-platform portable. In this document we present the design of an accessor that aims at fulfilling all these requirements.
97 MATHEMATICS AND COMPUTING↗
Technical Report: Accessor Realization
As we have seen the arithmetic power of hardware architectures growing much faster than the memory bandwidth, there is an urgent need to rethink the way we operate, communicate, and store data. A promising strategy for overcoming the “memory wall” is the decoupling of memory precision from arithmetic precision as proposed in Anzt et al.. Such a decoupling of memory precision and arithmetic precision requires a “memory accessor” that handles the conversion between formats (resp. compression of the data) on the fly in registers. For such a memory accessor to be useful in practice, it has to combine multiple characteristics.
97 MATHEMATICS AND COMPUTING↗
The Implications of 3D Thermal Structure on 1D Atmospheric Retrieval
Using the atmospheric structure from a 3D global radiation-hydrodynamic simulation of HD 189733b and the open-source Bayesian Atmospheric Radiative Transfer (BART) code, we investigate the difference between the secondary-eclipse temperature structure produced with a 3D simulation and the best-fit 1D retrieved model. Synthetic data are generated by integrating the 3D models over the Spitzer , the Hubble Space Telescope ( HST ), and the James Web Space Telescope ( JWST ) bandpasses, covering the wavelength range between 1 and 11 μ m where most spectroscopically active species have pronounced features. Using the data from different observing instruments, we present detailed comparisons between the temperature–pressure profiles recovered by BART and those from the 3D simulations. We calculate several averages of the 3D thermal structure and explore which particular thermal profile matches the retrieved temperature structure. We implement two temperature parameterizations that are commonly used in retrieval to investigate different thermal profile shapes. To assess which part of the thermal structure is best constrained by the data, we generate contribution functions for our theoretical model and each of our retrieved models. Our conclusions are strongly affected by the spectral resolution of the instruments included, their wavelength coverage, and the number of data points combined. We also see some limitations in each of the temperature parametrizations, as they are not able to fully match the complex curvatures that are usually produced in hydrodynamic simulations. The results show that our 1D retrieval is recovering a temperature and pressure profile that most closely matches the arithmetic average of the 3D thermal structure. When we use a higher resolution, more data points, and a parametrized temperature profile that allows more flexibility in the middle part of the atmosphere, we find a better match between the retrieved temperature and pressure profile and the arithmetic average. The Spitzer and HST simulated observations sample deep parts of the planetary atmosphere and provide fewer constraints on the temperature and pressure profile, while the JWST observations sample the middle part of the atmosphere, providing a good match with the middle and most complex part of the arithmetic average of the 3D temperature structure.
79 ASTRONOMY AND ASTROPHYSICS↗
Evaluating marginal likelihood with thermodynamic integration method and comparison with several other numerical methods
Evaluating marginal likelihood is the most critical and computationally expensive task, when conducting Bayesian model averaging to quantify parametric and model uncertainties. The evaluation is commonly done by using Laplace approximations to evaluate semianalytical expressions of the marginal likelihood or by using Monte Carlo (MC) methods to evaluate arithmetic or harmonic mean of a joint likelihood function. This study introduces a new MC method, i.e., thermodynamic integration, which has not been attempted in environmental modeling. Instead of using samples only from prior parameter space (as in arithmetic mean evaluation) or posterior parameter space (as in harmonic mean evaluation), the thermodynamic integration method uses samples generated gradually from the prior to posterior parameter space. This is done through a path sampling that conducts Markov chain Monte Carlo simulation with different power coefficient values applied to the joint likelihood function. The thermodynamic integration method is evaluated using three analytical functions by comparing the method with two variants of the Laplace approximation method and three MC methods, including the nested sampling method that is recently introduced into environmental modeling. The thermodynamic integration method outperforms the other methods in terms of their accuracy, convergence, and consistency. The thermodynamic integration method is also applied to a synthetic case of groundwater modeling with four alternative models. The application shows that model probabilities obtained using the thermodynamic integration method improves predictive performance of Bayesian model averaging. As a result, the thermodynamic integration method is mathematically rigorous, and its MC implementation is computationally general for a wide range of environmental problems.
97 MATHEMATICS AND COMPUTING↗
Mixed-precision iterative refinement using tensor cores on GPUs to accelerate solution of linear systems
Double-precision floating-point arithmetic (FP64) has been the de facto standard for engineering and scientific simulations for several decades. Problem complexity and the sheer volume of data coming from various instruments and sensors motivate researchers to mix and match various approaches to optimize compute resources, including different levels of floating-point precision. In recent years, machine learning has motivated hardware support for half-precision floating-point arithmetic. A primary challenge in high-performance computing is to leverage reduced-precision and mixed-precision hardware. We show how the FP16/FP32 Tensor Cores on NVIDIA GPUs can be exploited to accelerate the solution of linear systems of equations Ax = b without sacrificing numerical stability. The techniques we employ include multiprecision LU factorization, the preconditioned generalized minimal residual algorithm (GMRES), and scaling and auto-adaptive rounding to avoid overflow. We also show how to efficiently handle systems with multiple right-hand sides. On the NVIDIA Quadro GV100 (Volta) GPU, we achieve a 4×-5× performance increase and 5× better energy efficiency versus the standard FP64 implementation while maintaining an FP64 level of numerical stability.
GMRES↗
An Analytical Time–Domain Expression for the Net Ripple Produced by Parallel Interleaved Converters
We apply modular arithmetic and Fourier series to analyze the superposition of N interleaved triangular waveforms with identical amplitudes and duty-ratios. Here, interleaving refers to the condition when a collection of periodic waveforms with identical periods are each uniformly phase-shifted across one period. The main result is a time-domain expression which provides an exact representation of the summed and interleaved triangular waveforms, where the peak amplitude and parameters of the time-periodic component are all specified in closed-form. Analysis is general and can be used to study various applications in multi-converter systems. This model is unique not only in that it reveals a simple and intuitive expression for the net ripple, but its derivation via modular arithmetic and Fourier series is distinct from prior approaches. The analytical framework is experimentally validated with a system of three parallel converters under time-varying operating conditions.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Improved Nested Sampling and Surrogate-Enabled Comparison With Other Marginal Likelihood Estimators
Estimating marginal likelihood is of central importance to Bayesian model selection and/or model averaging. The nested sampling method has been recently used together with the Metropolis-Hasting (M-H) sampling algorithm for estimating marginal likelihood. This study develops a new implementation of nested sampling by using the DiffeRential Evolution Adaptive Metropolis (DREAMzs) sampling algorithm. The two implementations of nested sampling are evaluated for four models of a synthetic groundwater flow modeling. The DREAMzs-based nested sampling outperforms the M-H-based nested sampling in terms of their accuracy, convergence, efficiency, and stability, which is attributed to the fact that DREAMzs is more robust than M-H for parameter sampling. The nested sampling method is also compared with four other methods (arithmetic mean, harmonic mean, stabilized harmonic mean, and thermodynamic integration) commonly used for estimating marginal likelihood and posterior probability of the four groundwater models. The comparative study requires hundreds of millions of model executions, which would not be possible without using surrogate models to replace the original models. Using the arithmetic mean estimator as the reference, the comparison reveals that thermodynamic integration outperforms nested sampling in terms of accuracy and stability, whereas nested sampling is more computationally efficient to reach to convergence. Here, the harmonic mean and stabilized harmonic mean methods give the worst marginal likelihood estimation. Accurate marginal likelihood estimation is important for accurate estimation of posterior model probability and better predictive performance of Bayesian model averaging.
54 ENVIRONMENTAL SCIENCES↗
Multistage mixed precision iterative refinement
Abstract Low precision arithmetic, in particular half precision (16‐bit) floating point arithmetic, is now available in commercial hardware. Using lower precision can offer significant savings in computation and communication costs with proportional savings in energy. Motivated by this, there has been a renewed interest in mixed precision iterative refinement schemes for solving linear systems , and new variants of GMRES‐based iterative refinement have been developed. Each particular variant with a given combination of precisions leads to different condition number‐based constraints for convergence of the backward and forward errors, and each has different performance costs. The constraints for convergence given in the literature are, as an artifact of the analyses, often overly strict in practice, and thus could lead a user to select a more expensive variant when a less expensive one would have sufficed. In this work, we develop a multistage mixed precision iterative refinement solver which aims to combine existing mixed precision approaches to balance performance and accuracy and improve usability. For a user‐specified initial combination of precisions, the algorithm begins with the least expensive approach and convergence is monitored via inexpensive computations with quantities produced during the iteration. If slow convergence or divergence is detected using particular stopping criteria, the algorithm switches to use a more expensive, but more reliable variant. A novel aspect of our approach is that, unlike existing implementations, our algorithm first attempts to use “stronger” GMRES‐based solvers for the solution update before resorting to increasing the precision(s). In some scenarios, this can avoid the need to refactorize the matrix in higher precision. We perform extensive numerical experiments on a variety of random dense problems and problems from real applications which confirm the benefits of the multistage approach.
Oktay, Eda↗
Numerically stable, scalable formulas for parallel and online computation of higher-order multivariate central moments with arbitrary weights
Formulas for incremental or parallel computation of second order central moments have long been known, and recent extensions of these formulas to univariate and multivariate moments of arbitrary order have been developed. Such formulas are of key importance in scenarios where incremental results are required and in parallel and distributed systems where communication costs are high. We survey these recent results, and improve them with arbitrary-order, numerically stable one-pass formulas which we further extend with weighted and compound variants. We also develop a generalized correction factor for standard two-pass algorithms that enables the maintenance of accuracy over nearly the full representable range of the input, avoiding the need for extended-precision arithmetic. We then empirically examine algorithm correctness for pairwise update formulas up to order four as well as condition number and relative error bounds for eight different central moment formulas, each up to degree six, to address the trade-offs between numerical accuracy and speed of the various algorithms. Finally, we demonstrate the use of the most elaborate among the above mentioned formulas, with the utilization of the compound moments for a practical large-scale scientific application.
97 MATHEMATICS AND COMPUTING↗
Accelerating nuclear configuration interaction calculations through a preconditioned block iterative eigensolver
We describe a number of recently developed techniques for improving the performance of large-scalenuclear configuration interaction calculations on high performance parallel computers. We show thebenefit of using a preconditioned block iterative method to replace the Lanczos algorithm that hastraditionally been used to perform this type of computation. The rapid convergence of the block iterativemethod is achieved by a proper choice of starting guesses of the eigenvectors and the construction of aneffective preconditioner. These acceleration techniques take advantage of special structure of the nuclearconfiguration interaction problem which we discuss in detail. The use of a block method also allows usto improve the concurrency of the computation, and take advantage of the memory hierarchy of modernmicroprocessors to increase the arithmetic intensity of the computation relative to data movement. Wealso discuss the implementation details that are critical to achieving high performance on massivelyparallel multi-core supercomputers, and demonstrate that the new block iterative solver is two to threetimes faster than the Lanczos based algorithm for problems of moderate sizes on a Cray XC30 system.
Shao, Meiyue↗
Higher-order particle representation for particle-in-cell simulations
In this paper we present an alternative approach to the representation of simulation particles for unstructured electrostatic and electromagnetic PIC simulations. In our modified PIC algorithm we represent particles as having a smooth shape function limited by some specified finite radius, r 0 . A unique feature of our approach is the representation of this shape by surrounding simulation particles with a set of virtual particles with delta shape, with fixed offsets and weights derived from Gaussian quadrature rules and the value of r 0 . As the virtual particles are purely computational, they provide the additional benefit of increasing the arithmetic intensity of traditionally memory bound particle kernels. The modified algorithm is implemented within Sandia National Laboratories' unstructured EMPIRE-PIC code, for electrostatic and electromagnetic simulations, using periodic boundary conditions. We show results for a representative set of benchmark problems, including electron orbit, a transverse electromagnetic wave propagating through a plasma, numerical heating, and a plasma slab expansion. In this work, good error reduction across all of the chosen problems is achieved as the particles are made progressively smoother, with the optimal particle radius appearing to be problem-dependent.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗