Analysis of floating-point round-off error in linear algebra routines for graph clustering
Explore the source record for details and available documents.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Automated techniques for rigorous floating-point round-off error analysis are a prerequisite to placing important activities in HPC such as precision allocation, verification and code optimization on a formal footing. Yet existing techniques cannot provide tight bounds for expressions beyond a few dozen operators; barely enough for HPC. In this work, we offer an approach embedded in a new tool called SATIRE that scales error analysis by four orders of magnitude compared to today’s best-of-class tools. We explain how three key ideas underlying SATIRE help it attain such scale: path strength reduction, bound optimization and abstraction. SATIRE provides tight bounds and rigorous guarantees on significantly larger expressions with well over a hundred thousand operators, covering important examples including FFT, matrix multiplication and PDE stencils.
Verification, validation and uncertainty quantification (VVUQ) have become a common practice in thermal-hydraulics analysis. In general, these activities deal with propagations of uncertainties in computer code simulations, e.g., through system analysis codes. However, most existing such activities in thermal-hydraulics analysis have been primarily focused on input and model uncertainties, while numerical errors were largely overlooked. Numerical errors can appear in many different forms, e.g., round-off error, statistical sampling error, and discretization error. In thermal-hydraulics analysis, especially two-phase flow simulations commonly encountered in reactor safety analysis, the lack of consideration of discretization error is mainly due to the difficulty in estimating them. Accurate estimations of discretization error require continuous mesh refinement and/or implementation of high-order numerical schemes in system analysis code, both of which are difficult to achieve in existing codes. In this work, we will build a computer code that incorporates both first-order and second-order numerical methods to solve the two-phase flow problems. The first-order method resembles the one used in many existing system analysis codes; and the second-order method works as the reference to estimate numerical errors. Numerical verification of spatial discretization schemes will be presented in the form of mesh convergence study. It will also be demonstrated via case studies that, in practical scenarios, discretization errors can be as large as, or even larger than, model uncertainties.
In recent years, verification, validation and uncertainty quantification (VVUQ) have become a common practice in thermal-hydraulics analysis. In general, these activities deal with propagations of uncertainties in computer code simulations, e.g., through system analysis codes. However, most existing such activities in thermal-hydraulics analysis have been primarily focused on model uncertainties, while numerical uncertainties were largely overlooked. Numerical uncertainties (or numerical errors) can appear in many different forms, e.g., round-off error, statistical sampling error, and discretization error. As pointed out by Oberkampf and Roy, discretization error is usually the largest however the most difficult type of error to estimate reliably. In thermal-hydraulics analysis, especially two-phase flow simulations commonly encountered in reactor safety analysis, the lack of consideration on discretization error is mainly due to the difficulty in estimating them. Accurate estimations on discretization error require continuous mesh refinement and/or implementation of high-order numerical schemes in system analysis code, both of which are unfortunately difficult to achieve for existing codes. At first, it should be emphasized that mesh refinement (or mesh sensitivity study) is a well-understood concept in the thermal-hydraulics field. However, it is not commonly practiced for many reasons, e.g., lack of understanding on numerical errors by code users, consideration on computational cost, deteriorated numerical stability performance with refined meshes, and etc. Secondly, high-order (e.g., second-order) numerical schemes are simply unavailable in almost any existing system analysis code. Implementation of high-order numerical schemes in these codes is difficult, if not impossible, given the complexity of these codes. On the other hand, people tend to argue that numerical uncertainties (or discretization error as we have emphasized) are not as important as the commonly large uncertainties in two-phase flow models, e.g., closure correlations. As a result, nodalization is sometimes played unintentionally, or maybe intentionally, to match experimental data, which is referred as 'user effects' or 'compensating errors' in thermal-hydraulics community. Such a practice is highly criticized by Levy. In this work, we will at first build a computer code that incorporates both first-order and second-order numerical methods to solve the two-phase flow problems. The first-order method resembles the one used in many existing system analysis codes; and the second-order method works as the reference to estimate numerical errors. Numerical verifications on spatial discretization schemes will be presented in the form of mesh convergence study. It will also be demonstrated that, in practical scenarios, discretization errors can be as large as, or even larger than, model uncertainties via case studies.
Compression of floating-point data will play an important role in high-performance computing as data bandwidth and storage become dominant costs. Lossy compression of floating-point data is powerful, but theoretical results are needed to bound its errors when used to store look-up tables, simulation results, or even the solution state during the computation. In this paper, we analyze the round-off error introduced by ZFP, a lossy compression algorithm. Here, the stopping criteria for ZFP depends on the compression mode specified by the user: fixed rate, fixed accuracy, or fixed precision [P. Lindstrom, ZFP 0.5.3 Documentation, 2018]. While most of our discussion is focused on the fixed precision mode of ZFP, we establish a bound on the error introduced by all three compression modes. In order to tightly capture the error, first we introduce a vector space that allows us to work with binary representations of components. Under this vector space, we define operators that implement each step of the ZFP compression and decompression to establish a bound on the error caused by ZFP. To conclude, numerical tests are provided to demonstrate the accuracy of the established bounds.
Efficient analytical sensitivity computations are essential elements of gradient-based optimization schemes; however, they can be difficult to implement. This implementation issue is often resolved by adopting the semi-analytical method which exhibits the efficiency of the analytical methods and the ease of implementation of the finite difference method. However, care must be taken as semi-analytical sensitivities may exhibit errors due to truncation and round-off. Additional errors are introduced if the convergence tolerance of the primal analysis is not sufficiently small. This paper gives a general overview and some new developments of the analytical and semi-analytical sensitivity analyses for nonlinear steady-state, transient, and dynamic problems. We discuss the restrictive assumptions, accuracy, and consistency of these methods. Both adjoint and direct differentiation methods are studied. Numerical examples are provided.
We present a new magnetohydrodynamic (MHD) code for the simulation of wave propagation in the solar atmosphere, under the effects of electrical resistivity—but not dominant—and heat transference in a uniform 3D grid. The code is based on the finite-volume method combined with the HLLE and HLLC approximate Riemann solvers, which use different slope limiters like MINMOD, MC, and WENO5. In order to control the growth of the divergence of the magnetic field, due to numerical errors, we apply the Flux Constrained Transport method, which is described in detail to understand how the resistive terms are included in the algorithm. In our results, it is verified that this method preserves the divergence of the magnetic fields within the machine round-off error (∼1×10{sup −12}). For the validation of the accuracy and efficiency of the schemes implemented in the code, we present some numerical tests in 1D and 2D for the ideal MHD. Later, we show one test for the resistivity in a magnetic reconnection process and one for the thermal conduction, where the temperature is advected by the magnetic field lines. Moreover, we display two numerical problems associated with the MHD wave propagation. The first one corresponds to a 3D evolution of a vertical velocity pulse at the photosphere–transition–corona region, while the second one consists of a 2D simulation of a transverse velocity pulse in a coronal loop.
In this paper we consider the variants of Gram–Schmidt such as Classical Gram–Schmidt and Modified Gram–Schmidt algorithms. It is shown that for problems of dimension more than two the round-off error of operation q{sub 1}{sup T}q{sub 2} has more propagation in both of algorithms. To cure this difficulty we will present an algorithm, namely Optimized Modified Gram–Schmidt algorithm. Numerical examples indicate the accuracy of this algorithm. We show that this method can improve the loss of orthogonality of the orthogonalization in some ill-conditioned cases.
Coupling peridynamics based computational tools with those using classical continuum mechanics can be very beneficial, because it can provide a means to generate a computational method that combines the efficiency of classical continuum mechanics with the capability to simulate crack propagation, typical of peridynamics. This paper presents an overlooked issue in this type of coupled computational methods: the lack of overall equilibrium. This can be the case even if the coupling strategy satisfies the usual numerical tests involving rigid body motions as well as uniform and linear strain distributions. We focus our investigation on the lack of overall equilibrium in an approach to couple peridynamics and classical continuum mechanics recently proposed by the authors. In our examples, the magnitude of the out-of-balance forces is a fraction of a percent of the applied forces, but it cannot be assumed to be a numerical round-off error. We show analytically and numerically that the main reason for the existence of out-of-balance forces is a lack of balance between the local and nonlocal tractions at the coupling interface. This usually results from the presence of high-order derivatives of displacements in the coupling zone.
We present a robust algorithm for the computation of electromagnetic fields radiated by point sources (Hertzian dipoles) in cylindrically stratified media where each layer may exhibit material properties (permittivity, permeability, and conductivity) with uniaxial anisotropy. Analytical expressions are obtained based on the spectral representation of the tensor Green's function based on cylindrical Bessel and Hankel eigenfunctions, and extended for layered uniaxial media. Due to the poor scaling of these eigenfunctions for extreme arguments and/or orders, direct numerical evaluation of such expressions can produce numerical instability, i.e., underflow, overflow, and/or round-off errors under finite precision arithmetic. To circumvent these problems, we develop a numerically stable formulation through suitable rescaling of various expressions involved in the computational chain, to yield a robust algorithm for all parameter ranges. Numerical results are presented to illustrate the robustness of the formulation including cases of practical interest.
A fast dynamic aperture (DA) optimization method for storage rings has been developed through the use of reversal integration. Even if dynamical systems have an exact reversal symmetry, a numerical forward integration differs from its reversal. For a chaotic trajectory, cumulative round-off errors are scaled, which results in an exponential growth on the difference. The exponential effect is a generic chaos indicator which represents the sensitivity of the chaotic motion to its initial condition. The chaos indicator of the charged particle motion can be obtained by comparing the forward integrations of particle trajectories with corresponding reversals, a.k.a. “backward integrations.” The indicator is observable even through short-term particle tracking simulations. Therefore, adopting it as an objective function could speed up optimization. Finally, the DA of the National Synchrotron Light Source II storage ring, and another test diffraction-limited light source ring, were optimized using this method for the purpose of demonstration.
Recent studies have shown that reducing the precision of floating-point calculations in an atmospheric model can improve the models' computational performance without affecting model fidelity, but code changes are needed to accommodate lower precision, and a small number of calculations might need to retain higher precision to prevent undue round-off error. For complex and computationally expensive systems like the Energy Exascale Earth System Model (E3SM), a method is needed to objectively assess the quality of the lower-precision simulations and to quickly identify problematic code pieces This paper demonstrates that solution correctness can be evaluated through a simple and quantitative error metric based on time step convergence. The proposed test method can unambiguously detect the accumulation of precision error and objectively assess its impact on solution accuracy. Since fast physical processes are known to affect key features of the multi-year mean climate in an atmospheric model, we show that the convergence test applied to short simulations can provide useful information about the impact of reduced precision on a model’s long-term behavior. In contrast, the traditional way of climate model evaluation requires multi-year simulations and involves inspecting many physical quantities for statistical significance in the presence of natural variability. The simplicity, computational efficiency, and objectivity of the proposed method makes it very attractive for high-resolution model development. The method is also expected to be applicable to other models that numerically solve time evolution equations.
The stochastic protein kinetic equations can be stiff for certain parameters, which makes their numerical simulation rely on very small time step sizes, resulting in large computational cost and accumulated round-off errors. For such situation, we provide a method of reducing stiffness of the stochastic protein kinetic equation by means of a kind of variable transformation. Theoretical and numerical analysis show effectiveness of this method. Its generalization to a more general class of stochastic differential equation models is also discussed.
In this work we carried out the disappearance analysis of muon neutrinos produced in the Fermilab Booster Neutrino Beam, using the data released to the public by the collaborations of the MiniBooNE and SciBooNE experiments. The calculations were made with programs in C and C++, implementing the ROOT libraries. From the analysis, using both the classical Pearson method and the Feldman and Cousins frequentist corrections, we obtained the 90\% C.L. limit for the oscillation parameters sin 2 2θ and Δm 2 in the region 0.1 ≤ Δm 2 ≤ 10 eV 2 using a two neutrino model. The result presented in this work is consistent with the official one, with small deviations ascribed to round-off errors in the format of the used data, as well as statistical fluctuations in the generation of fake experiments used in the Feldman and Cousins method. As the official one, our result is consistent with the null oscillation hypothesis. This work was carried out independently to the MiniBooNE and SciBooNE collaborations and its results are not official.
Over the past several years, the graphics processing unit (GPU) technology has rapidly gained ground in scientific computing due to its outstanding performance and programmability. GPU implementation of Monte Carlo radiation transport for dose calculations has been reported by many investigators. The majority of these studies adopted single-precision floating point format because of the higher peak floating point operations per second (FLOPS) the GPUs can deliver than double-precision. It has been known that calculation using single-precision is more prone to numerical round-off errors, especially when a single tally data is accumulated 'atomically' and repeatedly by thousands of GPU threads. To mitigate this problem, the least intrusive solution in theory is to replace the single-precision atomic-add tally function with a double-precision version. However, the complexity lies in the fact that some GPUs (Nvidia GPUs prior to the Pascal generation; all current AMD GPUs) do not readily offer such double-precision function at hardware level, and that software emulation is too slow to use if not optimized properly. This paper discusses several atomic-add tally methods with reduced numerical errors used throughout ARCHER development. The original software-based compare-and-swap method (CAS) was shown to be inefficient due to high intra-warp thread contention, whereas the improved software-based warp-aggregated method (WAG) and Kahan summation method (KAS) eliminated the thread contention and performed very well on Kepler and Maxwell GPUs, being more than 13 times faster than CAS in our tests. The hardware-based (HB) double-precision atomic-add feature available on Pascal and Volta GPUs exceeded software emulation and offered the best performance universally, so did WIB, a combination of WAG and HB methods. It was also shown that in a single-precision dose engine, KAS managed to maintain high reliability when the single-precision atomic-add tally underestimated the dose by 25%. (authors)
Abstract Reproducibility of results is a cornerstone of the scientific method. Scientific computing encounters two challenges when aiming for this goal. Firstly, reproducibility should not depend on details of the runtime environment, such as the compiler version or computing environment, so results are verifiable by third-parties. Secondly, different versions of software code executed in the same runtime environment should produceconsistent numerical results for physical quantities. In this manuscript, we test the feasibility of reproducing scientific results obtained using theIllinoisGRMHDcode that is part of an open-source community software for simulation in relativistic astrophysics, theEinstein Toolkit. We verify that numerical results of simulating a single isolated neutron star withIllinoisGRMHDcan be reproduced, and compare them to results reported by the code authors in 2015. We use two different supercomputers: Expanse at SDSC, and Stampede2 at TACC. By compiling the source code archived along with the paper on both Expanse and Stampede2, we find thatIllinoisGRMHDreproduces results published in its announcement paper up to errors comparable to round-off level changes in initial data parameters. We also verify that a current version ofIllinoisGRMHDreproduces these results once we account for bug fixes which have occurred since the original publication.
We present FPDetect, a low-overhead approach for detecting logical errors and soft errors affecting stencil computations without generating false positives. We develop an offline analysis that tightly estimates the number of floating-point bits preserved across stencil applications. This estimate rigorously bounds the values expected in the data space of the computation. Violations of this bound can be attributed with certainty to errors. FPDetect helps synthesize error detectors customized for user-specified levels of accuracy and coverage. FPDetect also enables overhead reduction techniques based on deploying these detectors coarsely in space and time. Experimental evaluations demonstrate the practicality of our approach.
ABSTRACT Since its formulation by Sir Isaac Newton, the problem of solving the equations of motion for three bodies under their own gravitational force has remained practically unsolved. Currently, the solution for a given initialization can only be found by performing laborious iterative calculations that have unpredictable and potentially infinite computational cost, due to the system’s chaotic nature. We show that an ensemble of converged solutions for the planar chaotic three-body problem obtained using an arbitrarily precise numerical integrator can be used to train a deep artificial neural network (ANN) that, over a bounded time interval, provides accurate solutions at a fixed computational cost and up to 100 million times faster than the numerical integrator. In addition, we demonstrate the importance of training an ANN using converged solutions from an arbitrary precise integrator, relative to solutions computed by a conventional fixed precision integrator, which can introduce errors in the training data, due to numerical round-off and time discretization, that are learned by the ANN. Our results provide evidence that, for computationally challenging regions of phase space, a trained ANN can replace existing numerical solvers, enabling fast and scalable simulations of many-body systems to shed light on outstanding phenomena such as the formation of black hole binary systems or the origin of the core collapse in dense star clusters.