CS321. Introduction to Numerical Methods

Transcription

1 CS3 Introduction to Numerical Methods Lecture Number Representations and Errors Professor Jun Zhang Department of Computer Science University of Kentucky Lexington, KY August 7, 05

2 Number in Different Bases Humans use number base for reasons of the number of fingers Computers use number base for reasons of on-off switch A number based on 8 can be used to facilitate conversion between the base and the base Base Expansion: The general formula for a base integer is a n a n a a a 0 a 0 0 a a a n n a n n

3 Binary Numbers 3

4 Number System 4

5 Definitions There are some definitions for representing a real number in base and in base 5

6 Nonintegers and Fractions An example of a fractional number The general formula for a decimal fraction 3 0. bbb3 b b b3 Decimal fractional numbers can be repeating or nonrepeating (rational or irrational numbers) ( a a a a. b b b ) n n 0 n k ak k 0 k b k 3 k 6

7 Number Conversion 7

8 Base β Numbers Convert an integer number from base 8 system to base system (763 General formula ) (3 8(6 8 3 ( ( ) 6 8 8(6 (8(7))) 8(7 ))) 7 8 (3994 ) Convert a fractional number between 0 and (0.763 ) ( a n a n k ak β k 0 k n aa0. bb b3 ) β b k β k 3 3 Convert integer and fractional parts separately 8

9 From system α to system β with α < β Conversion between Systems express (N) α in nested form using powers of α replace each digit by corresponding base β number carry out indicated arithmetic in base β Two examples were given in the previous page Note that one needs to convert the integer and the fractional parts separately The method is in theory can be used to convert a number between any two bases. But the third step is not easy for humans if the base is not 9

10 Division Algorithm Use remainder-quotient-split method to convert an integer, best if α > β ( N ) α ( c c c 0 m m β ( c c c 0 β ( c ) β β ( c m ) )). Divide the representation by β;. Take the remainder as the next digit in the base β representation; 3. Repeat the procedure on the quotient. This procedure is easier to carry out by hand Note that the first digit obtained is the one closest to the radix point

11 Repeated Division To convert a base integer number to a base β< number, we can use repeated division algorithm / 8 5 / 8 So we have / It can be further converted to a binary number using the binary-octal table (in a later slide) ( 576.) / 8 ( 576.) remainder remainder remainder remainder (500.) 8 Note that the first digit you obtain is the first digit next to the radix point (500.) 8 ( ) 0 0 5

12 Multiplication Algorithm Use integer-fraction-split process for converting a fractional part x k c k - β k (0. c cc3 ) β.) multiply the (fractional) number by β;.) take the integer part as the first (next) digit; 3.) repeat the process on the remaining fractional part. Again, the first digit obtained is the one closest to the radix point Terminating fractional number may become non-terminating in different base systems, and vice versa

13 Integer-Fraction-Split To convert a fractional base number to a base β< number, we can use the integer-fractional-split algorithm So we have ( 0.37 ) (0.0...) 3

14 Use Intermediate Base We can convert a base number to a base 8 (octal) number, then to base (binary) number, and vice versa Binary-octal table Groups of three binary digits can be translated to an octal number using the binary-octal table ( ) (65.347) 8 (?) (57.3) (?) 8 (?) 4

15 Base 6 Numbers Certain computers with word length being multiples of 4 use hexadecimal system, need A, B, C, D, E, and F to represent,,, 3, 4, and 5. Binary-hexadecimal table ( ) ( ) (BC.DF)6 5

16 Definitions There are some definitions for representing a real number in base and in base 6

17 Write a number in the form of Normalized Scientific Notation A normalized floating-point number has the form 3 where d 0,n is an integer x ±. d d d 0 3 n In a simple notation n x ± r r < r is called normalized mantissa and n is the exponent. Also for binary representation m ± q q < x 7

18 Machine Numbers The real numbers that can be represented exactly in a computer are called the machine numbers for this computer Most real numbers are not machine numbers If a computer has word length of the form 0.d d d 3 d 4, then 0. is a machine number, but 0. is not Machine numbers are machine dependent. The use of normalized floating-point numbers creates a phenomenon of hole at zero, a bunch of numbers close to 0 are not representable This is mainly caused by the under-flow problem, i.e., small numbers close to zero will be treated as zero in a computer. (0.00 cannot be stored in the above mentioned computer, why?) 8

19 A 3-bit Machine Single precision floating-point numbers with 3-bit word length How to store a floating-point number with 3 bits? Sign of q needs bit Integer m needs 8 bits Number q needs 3 bits m ± q Single precision IEEE standard floating-point format s c 7 ( ) (. f ) 9

20 3-bit Representation The exponent number c is actually stored, so m c 7, this is an excess- 7 code (make sure only positive numbers are stored, 0< c <55) With normalized representation, the first bit is always and needs not be stored. The mantissa actually contains 4 binary digits with a hidden bit With a mantissa of 3 bits, a machine can have about six significant decimal digits of accuracy, since 3 7. The smallest positive number є such that ε is called the machine epsilon or unit roundoff error For single precision, ε. 7 For double precision, ε. 6 0

21 3-bit Representation IEEE 3-bit single precision

22 64-bit Representation IEEE 3-bit single precision and 64-bit double precision

23 The exponent c is stored, with More About the Exponent 0 < < 55 ( ) The actual range of exponent is is reserved for the special number ±0, and 55is for ± This strategy avoids the need to handle sign for the exponent The largest number can be stored 3.4 The smallest number can be stored. 3

24 How to represent a real number x? Representation Procedure.) if x is zero, stored it by a full word of zero bits, (a possible sign bit).) for nonzero x, first consider sign bit and then consider x 3.) convert both integer and fractional parts of x from decimal to octal, then to binary 4.) one-plus normalize ( x ) by shifting the binary point 5.) find the 4 bit -plus normalized mantissa 6.) find the exponent of by setting it equal to c-7 and determine c 7.) denote the 3-bit representation as 8 hexadecimal digits 4

25 A 3-bit single-precision pattern Summary and Examples can be interpreted as the real number Examples b bb3 b9bb b3 Find the 3-bit representation of Integer part (5.) (64.) 8 ( 0.) Fractional part (.34375) (.7) 8 (.00 ) b ( b b b ) ( ) (. b b b 3 ) 5

26 Examples cont. ( ) ( 0.00) ( ) 5 The exponent is (5), we need to write it as c so c 3 The stored exponent is ( 3) The representation of is ( ) ( ) (C50F000) 6 What number has the representation (45DE4000) 6? (04) 8 ( 000 0) 6

27 Errors in Representing Numbers Non-machine numbers are represented by a nearest machine number in computer Additional digits will be truncated Correct rounding and roundoff error For a 3-bit single-precision machine with 3 bits for mantissa, the relative error in correct rounding is 3 The unit roundoff error is ε 3 For a computer with a 4 binary digit word length, the number 0. will be stored as 0. for rounding off, and 0. for correct rounding 7

28 Arithmetic Operations Most computers use double-length arithmetic operations. Numbers are extended to double length, arithmetic operations are performed, and the result is rounded to a single length number On a machine with 4 digit word length, this number needs to be rounded to 0.34 resulting in an error

29 Notation fl(x) Use fl(x) to denote the floating-point machine number corresponding to a real number x fl(x) x( δ), For the 3-bit machine, we have δ ε It is easy to see that if є < -3, then fl( є) Note that the numbers are smaller than ε. It has more than 3 zeros. 9

30 Inverse-Error Analysis More generally known as backward error analysis Denote as one of the basic arithmetic operations, then fl(x y) (x y)( δ), Two interpretations ( δ 4 ) fl(z) (x y)( δ) perturbation of sum fl(z) x( δ) y( δ) sum of perturbations Direct-error analysis and reverse-error analysis Forward-error analysis and backward-error analysis 30

31 Loss of Significance Subtraction of two nearly equal numbers may result in loss of significant digits on a finite precision machine Cure: reprogram or use higher precision arithmetic, may be costly Use a 4-word length computer: Loss of Precision Theorem Let x and y be normalized floating-point machine numbers with x > y > 0. If p q y/ x for some positive integers p and q, then at most p and at least q significant digits are lost in the subtraction x - y 3

32 Loss of Significance Since y<x, the computer has to shift y before carrying out the subtraction so Look at the mantissa Hence, to normalize the representation of x-y, a shift of at least q bits to the left is necessary. Thus at least q (spurious) zeros are supplied on the right-hand side of the mantissa ) ) ( ) ( p q n m n m r x y r r s r s r > < ( n n m s y ) (, where,, s r s y r x m n n n m s r y x ) ( 3

33 x and y y , x An Example from Book which is between and At least and at most binary digits are lost when computing x - y Exactly how many digits lost depends on a computer x and y Suppose a machine has 5 decimal digits of accuracy We have x and ~ y The machine computes ~ , ~ ~ 3 x y

34 An Example Cont. Exact computation The relative error is x y ~ ~ ( x y) ( x y) x y This relative error is considered to be large since the machine has 5 decimal digits of accuracy Note that the double length computing operations are carried out after the numbers are stored If we use a machine with at least 8 decimal digits of accuracy, we can have the exact value as

35 Avoiding Loss of Significance Analyze possible loss of significance, reschedule computation to avoid subtraction of two nearly equal numbers, modify algorithm Example. Evaluating f ( x) x at x Using 5-decimal-digit arithmetic for x -3, we have f(x) 0 Rationalizing the numerator, we have x f ( x) x 0 computing ( 3 6 ) 0., and f ( x)

36 More Examples Evaluating f ( x) x sin x at x 0 Using Taylor series for sin x sin x x 3 x 3! 5 x 5! 7 x 7! Then f ( x) Compute x 0. with four-decimal-digit arithmetic, sin(0.) So x -sin x But x 3 /3! This strategy is not good for large x. For x π, x sin π π, but x 3 /3! Range reduction for periodic functions 3 x 3! 5 x 5! 7 x 7! ( 53.4) sin( π ) sin( 3.47) sin 36