HUFFMAN CODING AND HUFFMAN TREE

Size: px

Start display at page:

Download "HUFFMAN CODING AND HUFFMAN TREE"

Edwin Hodges
9 years ago
Views:

1 oding: HUFFMN OING N HUFFMN TR Reducing strings over arbitrary alphabet Σ o to strings over a fixed alphabet Σ c to standardize machine operations ( Σ c < Σ o ). inary representation of both operands and operators in machine instructions in computers. It must be possible to uniquely decode a code-string (string over Σ c )toasource-string (string over Σ o ). Not all code-string need to correspond to a source-string. oth the coding and decoding should be efficient. Word: finite non-empty string over analphabet (Σ o or Σ c ). Simple oding Mechanism: code(a i )=anon-empty string over Σ c,for a i Σ o. code(a 1 a 2 a n )=code(a 1 ).code(a 2 ) code(a n ). xample. Σ 0 ={,,,,}and Σ c ={0, 1}. Prefixproperty code( ) = ; yes easy to decode code() =code( ) =001; no not always possible to uniquely decode prefix-free code yes not prefix-free code no

Not all code-string need to correspond to a source-string. oth the coding and decoding should be efficient. Word: finite non-empty string over analphabet (Σ o or Σ c ).

2 4.2 PRFIX-FR O efinition: Nocode(a i )isaprefix of another code(a j ). In particular, code(a i ) code(a j )for a i a j. inary-tree representation of prefix-free binary code: 0 =label(left branch) and 1 = label(right branch) dvantage: and have shorter code-word; each non-terminal node has 2 children. One can decode the symbols from left to right, i.e., as they are received. sufficient condition for left-to-right unique decoding is the prefix property. Question:? How can we keep prefix-free property and assign shorter codes to some of the symbols {,,, }?? What should we do if the symbols in Σ o occur with probabilities p() = 0.1 = p(), p() = 0.3, p() = p() =?

0 1 0 1 0 1 1 001 000 0 1 0 1 dvantage: 011 10 0 110 0 1 0 1 01 10 11 0 1 000 001 and have shorter code-word; each non-terminal node has 2 children.

3 4.3 HUFFMN-TR inary tree with each non-terminal node having 2 children. Gives optimal (min average code-length) prefix-free binary code to each a i Σ o for a given probabilities p(a i )>0. Huffman s lgorithm: 1. reate aterminal node for each a i Σ o,with probability p(a i ) and let S =the set of terminal nodes. 2. Select nodes x and y in S with the two smallest probabilities. 3. Replace x and y in S by a node with probability p(x) + p(y). lso, create a node in the tree which is the parent of x and y. 4. Repeat (2)-(3) untill S =1. xample. Σ 0 ={,,, } and p() =0.1 = p(), p() =0.3, p() = p()=. The nodes in S are shown shaded fter redrawing the tree

reate aterminal node for each a i Σ o,with probability p(a i ) and let S =the set of terminal nodes. 2. Select nodes x and y in S with the two smallest probabilities. 3.

4 4.4 NUMR OF INRY TRS 0-2 inary Tree: ach non-terminal node has 2 children. #(inary trees with N nodes) = 1 2N + 1 #(0-2 inary trees with N terminal nodes) =#(inary trees with N 1nodes) = 2N + 1 N. 1 2N 1 2N 1 N 1 2N2. xample. inary trees with N 1 = 3 nodes correspond to 0-2 binary trees with N =4terminal nodes. Merits of Huffman s lgorithm: Itfinds the optimal coding in O(N. log N) time. It does so without having to search through possible 0-2 binary trees. 1 2N 1 2N 1 N 1

N. 1 2N 1 2N 1 N 1 2N2. xample. inary trees with N 1 = 3 nodes correspond to 0-2 binary trees with N =4terminal nodes.

5 4.5 Main Operations: T-STRUTUR FOR IMPLMNTING HUFFMN S LGORITHM hoosing the two nodes with minimum associated probabilities (and creating a parent node, etc). Use heap data-structure for this part. This is done N 1 times; total work for this part is O(N. log N). ddition of each parent node and connecting with the children takes a constant time per node. total of N 1parent nodes are added, and total time for this O(N). omplexity: O(N. log N).

This is done N 1 times; total work for this part is O(N. log N).

6 4.6 XRIS 1. onsider the codes shows below (a) rrange the codes in a binary tree form, with 0 = label(leftbranch) and 1 = label(rightbranch). (b oes these codes have the prefix-property? How do you decode the string ? (c) Modify the above code (keeping the prefix property) so that the new code will have less average length no matter what the probabilities of the symbols are. Show the binary tree for the new code. (d) What are the two key properties of the new binary tree (hint: compare with your answer for part (a))? (e) Give a suitable probability for the symbols such that prob( ) < prob() < prob() < prob() < prob() and the new code in part (c) is optimal (minimum aver. length) for those probabilities. 2. Show the successive heaps in creating the Huffman-Tree for the probabilities p( ) = 0.1 = p(), p() = 0.3, p() = 0.14, p() =0.12, and p(f) = Give some probabilities for the items in Σ o ={,,, F} that give the largest possible value for optimal average code length. 4. rgue that for an optimal Huffman-tree, any subtree is optimal (w.r.t to the relative probabilities of its terminal nodes), and also the tree obtained by removing all children and other descendants of a node x gives atree which is optimal w.r.t to p(x) and the probabilities of its other terminal nodes.

(c) Modify the above code (keeping the prefix property) so that the new code will have less average length no matter what the probabilities of the symbols are. Show the binary tree for the new code.

Class Notes CS 3137. 1 Creating and Using a Huffman Code. Ref: Weiss, page 433

Class Notes CS 3137. 1 Creating and Using a Huffman Code. Ref: Weiss, page 433 Class Notes CS 3137 1 Creating and Using a Huffman Code. Ref: Weiss, page 433 1. FIXED LENGTH CODES: Codes are used to transmit characters over data links. You are probably aware of the ASCII code, a fixed-length