NASA NTRS · 19950065240
Reducing Truncation Error In Integer Processing
Abstract
Improved method of rounding off (truncation of least-significant bits) in integer processing of data devised. Provides for reduction, to extremely low value, of numerical bias otherwise generated by accumulation of truncation errors from many arithmetic operations. Devised for use in integer signal processing, in which rescaling and truncation usually performed to reduce number of bits, which typically builds up in sequence of operations. Essence of method to alternate direction of roundoff (plus, then minus) on alternate occurrences of truncated values contributing to bias.
Keep this discovery
Explore connections, maps & timelines
Thomas, J. Brooks, Berner, Jeffrey B., Graham, J. Scott. 1995-02-01. Reducing Truncation Error In Integer Processing. https://ntrs.nasa.gov/citations/19950065240
Cite the original work for its findings. Save a collection to share your selection of sources.