Handbook of Floating-Point Arithmetic

Transcription

1 Jean-Michel Muller Nicolas Brisebarre Florent de Dinechin Claude-Pierre Jeannerod Vincent Lerevre Guillaume Melquiond Nathalie Revol Damien Stehle Serge Tones Handbook of Floating-Point Arithmetic Birkhäuser Boston Basel Berlin

2 Preface List of Figures List of Tables xv xvii xxi I Introduction, Basic Definitions, and Standards 1 1 Introduction Some History Desirable Properties Some Strange Behaviors Some famous bugs Difficult problems 8 2 Definitions and Basic Notions Floating-Point Numbers Rounding Rounding modes Useful properties Relative error due to rounding Exceptions Lost or Preserved Properties of the Arithmetic on the Real Numbers Note on the Choice of the Radix Representation errors A case for radix Tools for Manipulating Floating-Point Errors The ulp function Errors in ulps and relative errors An example: iterated products Unit roundoff Note on Radix Conversion 40 v

3 vi Contents Conditions on the formats Conversion algorithms 2.8 The Fused Multiply-Add (FMA) Instruction 2.9 Interval Arithmetic Intervalls with floating-point bounds Optimized rounding Floating-Point Formats and Environment The IEEE Standard Formats specified by IEEE Little-endian, big-endian Rounding modes specified by IEEE Operations specified by IEEE Exceptions specified by IEEE Special values The IEEE Standard Constraints internal to a format Various formats and the constraints between them Conversions between floating-point numbers and decimal strings Rounding Operations Comparisons Exceptions The Need for a Revision A typical problem: "double rounding" Various ambiguities The New IEEE Standard Formats specified by the revised standard Binary interchange format encodings Decimal interchange format encodings Larger formats Extended and extendable precisions Attributes Operations specified by the standard Comparisons Conversions Default exception handling Recommended transcendental functions Floating-Point Hardware in Current Processors The common hardware denominator Fused multiply-add Extended precision Rounding and precision control 105

4 vii SIMD instructions Floating-point on x86 processors: SSE2 versus x Decimal arithmetic 3.6 Floating-Point Hardware in Recent Graphics Processing Units 3.7 Relations with Programming Languages The Language Independent Arithmetic (LIA) standard Programming languages 3.8 Checking the Environment MACHAR Paranoia UCBTest TestFloat IeeeCC Miscellaneous 116 II Cleverly Using Floating -Point Arithmetic Basic Properties and Algorithms Testing the Computational Environment Computing the radix Computing the precision Exact Operations Exact addition Exact multiplications and divisions Accurate Computations of Sums of Two Numbers The Fast2Sum algorithm The 2Sum algorithm If we do not use rounding to nearest Computation of Products Veltkamp splitting Dekker's multiplication algorithm Complex numbers Various error bounds Error bound for complex multiplication Complex division Complex square root The Fused Multiply-Add Instruction The 2Mu1tFMA Algorithm Computation of Residuals of Division and Square Root Newton Raphson-Based Division with an FMA Variants of the Newton Raphson iteration 155

5 viii Contents Using the Newton-Raphson iteration for correctly rounded division 160 Newton-Raphson-Based Square Root with an FMA The basic iterations Using the Newton-Raphson iteration for correctly rounded square roots 168 Multiplication by an Arbitrary-Precision Constant Checking for a given constant C if Algorithm 5.2 will always work 172 Evaluation of the Error of an FMA 175 Evaluation of Integer Powers Enhanced Floating-Point Sums, Dot Products, and Polynomial Values Preliminaries Floating-point arithmetic models Notation for error analysis and classical error estimates Properties for deriving running error bounds Computing Validated Running Error Bounds Computing Sums More Accurately Reordering the operands, and a bit more Compensated sums Implementing a "long accumulator" On the sum of three floating-point numbers Compensated Dot Products Compensated Polynomial Evaluation Languages and Compilers A Play with Many Actors Floating-point evaluation in programming languages Processors, compilers, and operating systems In the hands of the programmer Floating Point in the C Language Standard C99 headers and IEEE support Types Expression evaluation Code transformations Enabling unsafe optimizations Summary: a few horror stories Floating-Point Arithmetic in the C++ Language Semantics Numeric limits Overloaded functions FORTRAN Floating Point in a Nutshell 223

6 ix Philosophy IEEE 754 support in FORTRAN Java Floating Point in a Nutshell Philosophy Types and classes Infinities, NaNs, and signed zeros Missing features Reproducibility The BigDecimal package Conclusion 234 III Implementing Floating-Point Operators Algorithms for the Five Basic Operations Overview of Basic Operation Implementation Implementing IEEE Rounding Rounding a nonzero finite value with unbounded exponent range Overflow Underflow and subnormal results The inexact excep tion Rounding for actual operations Floating-Point Addition and Subtraction Decimal addition Decimal addition using binary encoding Subnormal inputs and outputs in binary addition Floating-Point Multiplication Normal case Handling subnormal numbers in binary multiplication Decimal specifics Floating-Point Fused Multiply-Add Case analysis for normal inputs Handling subnormal inputs Handling decimal cohorts Overview of a binary FMA implementation Floating-Point Division Overview and special cases Computing the significand quotient Managing subnormal numbers The inexact exception Decimal specifics Floating-Point Square Root Overview and special cases 265

7 8.7.2 Computing the significand square root Managing subnormal numbers The inexact exception Decimal specifics Hardware Implementation of Floating-Point Arithmetic 9.1 Introduction and Context Processor internal formats Hardware handling of subnormal numbers Full-custom VLSI versus reconfigurable circuits Hardware decimal arithmetic Pipelining The Primitives and Their Cost Integer adders Digit-by-integer multiplication in hardware Using nonstandard representations of numbers Binary integer multiplication Decimal integer multiplication Shifters Leading-zero courtters Tables and table-based methods for fixed-point function approximation Binary Floating-Point Addition Overview A first dual-path architecture Leading-zero anticipation Probing further on floating-point adders Binary Floating-Point Multiplication Basic architecture FPGA implementation VLSI implementation optimized for delay Managing subnormals Binary Fused Multiply-Add Classic architecture To probe further Division Digit-recurrence division Decimal division Conclusion: Beyond the FPU Optimization in context of standard operators Operation with a constant operand Block floating point Specific architectures for accumulation Coarser-grain operators 317

8 xi 9.8 Probing Further Software Implementation of Floating-Point Arithmetic Implementation Context Standard encoding of binary floating-point data Available integer operators First examples Design choices and optimizations Binary Floating-Point Addition Handling special values Computing the sign of the result Swapping the operands and computing the alignment shift Getting the correctly rounded result Binary Floating-Point Multiplication Handling special values Sign and exponent computation Overflow detection Getting the correctly rounded result Binary Floating-Point Division Handling special values Sign and exponent computation Overflow detection Getting the correctly rounded result Binary Floating-Point Square Root Handling special values Exponent computation Getting the correctly rounded result 365 IV Elementary Functions Evaluating Floating-Point Elementary Functions Basic Range Reduction Algorithms Cody and Waite's reduction algorithm Payne and Hanek's algorithm Bounding the Relative Error of Range Reduction More Sophisticated Range Reduction Algorithms An example of range reduction for the exponential function An example of range reduction for the logarithm Polynomial or Rational Approximations L2 case L", or minimax case 390

9 xii Contents "Truncated" approximations Evaluating Polynomials Correct Rounding of Elementary Functions to binary The Table Maker's Dilemma and Ziv's onion peeling strategy When the TMD is solved Rounding test Accurate second step Error analysis and the accuracy/performance tradeoff Computing Error Bounds The point with efficient code Example: a "double-double" polynomial evaluation Solving the Table Maker's Dilemma Introduction The Table Maker's Dilemma Brief history of the TMD Organization of the chapter Preliminary Remarks on the Table Maker's Dilemma Statistical arguments: what can be expected in practice In some domains, there is no need to find worst cases Deducing the worst cases from other functions or domains The Table Maker's Dilemma for Algebraic Functions Algebraic and transcendental numbers and functions The elementary case of quotients Around Liouville's theorem Generating bad rounding cases for the square root using Hensel 2-adic lifting Solving the Table Maker's Dilemma for Arbitrary Functions Lindemann's theorem: application to some transcendental functions A theorem of Nesterenko and Waldschmidt A first method: tabulated differences From the TMD to the distance between a grid and a segment Linear approximation: Lefevre's algorithm The SLZ algorithm Periodic functions on large arguments Some Results Worst cases for the exponential, logarithmic, trigonometric, and hyperbolic functions A special case: integer powers Current Limits and Perspectives 458

10 xiii V Extensions Formalisms for Certifying Floating-Point Algorithms Formalizing Floating-Point Arithmetic Defining floating-point numbers Simplifying the definition Defining rounding operators Extending the set of numbers Formalisms for Certifying Algorithms by Hand Hardware units Low-level algorithms Advanced algorithms Automating Proofs Computing on bounds Counting digits Manipulating expressions Handling the relative error Using Gappa Toy implementation of sine Integer division on Itanium Extending the Precision Double-Words, Triple-Words Double-word arithmetic Static triple-word arithmetic Quad-word arithmetic Floating-Point Expansions Floating-Point Numbers with Batched Additional Exponent Large Precision Relying on Processor Integers Using arbitrary-precision integer arithmetic for arbitrary-precision floating-point arithmetic A brief introduction to arbitrary-precision integer arithmetic 513 VI Perspectives and Appendix Conclusion and Perspectives Appendix: Number Theory Tools for Floating-Point Arithmetic Continued Fractions The LLL Algorithm 524 Bibliography 529 Index 567