Speed Up Your Flow Table in a Nutshell

Transcription

1 SCALABLE, DETERMINISTIC AND UPDATABLE FLOW PROCESSING FRAMEWORK TO ACCELERATE SOFTWARE DEFINED NETWORKING By HAI SUN A dissertation submitted in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY WASHINGTON STATE UNIVERSITY School of Electrical Engineering and Computer Science MAY 2015

2 To the Faculty of Washington State University: The members of the Committee appointed to examine the dissertation of HAI SUN find it satisfactory and recommend that it be accepted. Min Sik Kim, Ph.D., Chair Carl H. Hauser, Ph.D. David E. Bakken, Ph.D. Ananth Kalyanaraman, Ph.D. ii

3 ACKNOWLEDGEMENTS First of all I would like to acknowledge my research adviser Dr. Min Sik Kim, who guided me to accomplish my graduation. His great patience and diligent mentoring throughout my PhD program encouraged and inspired me along the forward road. Only with the opportunity and resources he provided as well as his insightful opinions in research, I could accomplish the dissertation. I would like to thank Yan Sun for his tremendous help in every aspect of my research. His incisive intuition and insights in the networking area were really inspiring. I highly appreciate his time and discussion during the past years. I acknowledge the other two Network Research Lab group members, Vic and Haiqin for their individual help to my graduate work. It has been an honor to have Dr. Carl H. Hauser, Dr. David E. Bakken and Dr. Ananth Kalyanaraman in my Ph.D. research committee. I appreciate their time and effort before and during my dissertation work. Eventually I would like to offer my most heartfelt appreciations to my parents and my wife Yanrong for their love throughout my life. Especially I am grateful to their inspiration, unconditional support and encouragement during my dissertation work. I would like to thank my aunt and uncle for their care and help, and thank dear Kelsey and Jerry for every happy moment. iii

4 SCALABLE, DETERMINISTIC AND UPDATABLE FLOW PROCESSING FRAMEWORK TO ACCELERATE SOFTWARE DEFINED NETWORKING Abstract by Hai Sun, Ph.D. Washington State University May 2015 Chair: Min Sik Kim To accomplish high-performance and granular traffic control across a great number of network devices, flow processing in an OpenFlow switch is of great importance for any Software Defined Networking (SDN) architecture. The fundamental challenge is to classify incoming packets efficiently according to a pipeline of flow tables characterized by numerous fields and match types. So far current packet classification algorithms is not able to process a flow table with an arbitrary number of fields and any match type due to several crucial drawbacks, especially scalability, incremental update and nondeterminism. To address the new challenge we propose an innovative decomposition algorithm. Meanwhile we propose three different schemes to process flow tables with only exact match, prefix match and classic quintuple match. They are (1) a highly deterministic hashing scheme featured by efficient collision resolution, (2) a hierarchical hashing scheme featured by taking advantage of bitmap and hashing techniques, and (3) a divide-and-conquer scheme based on Ternary Content Addressable Memory (TCAM) hardware by addressing the range expansion problem. Together with the proposed decomposition algorithm, these four approaches cover all categories of match types and comprise a high-performance table lookup accelerator to boost flow processing in an OpenFlow switch. iv

5 As far as we know, our proposed decomposition scheme is the first efficient approach to addressing multidimensional match problem in an OpenFlow flow table with an arbitrary number of fields and any match type. And our proposed high-performance table lookup accelerator is the first comprehensive proposal to process the entire OpenFlow table pipeline by covering all categories of match types and any number of fields. The accelerator is available to be implemented and deployed in either software or hardware devices with respect to a variety of performance and cost circumstances. v

6 TABLE OF CONTENTS Page ABSTRACT iv LIST OF TABLES x LIST OF FIGURES xi CHAPTER 1 Introduction Highly Deterministic Hashing Scheme Using Bitmap Filter Background Related Work DO hashing and BH hashing DO hashing BH hashing Hierarchy Design Load Analysis Dimension Assignment Query Simulation Re-balance Influence Update Efficiency Memory Evaluation and Filter Comparison Summary Hierarchical Hashing Scheme to Accelerate Longest Prefix Matching vi

7 3.1 Introduction to IP address lookup Background Scheme Details Observations and Ideas Two-Layer Hierarchy Preprocessing, Lookup and Update Preprocessing Algorithm Lookup Algorithm Update Algorithm Performance Evaluation Theoretical Analysis Evaluation using Real Routing Tables Discussion Summary TCAM-Based Classification Using Divide-and-Conquer for Range Expansion Introduction to Packet Classification Software-based Algorithms TCAM-based Algorithms Background DCS Design Problem Statement and Overview Start Point: Worst-Case Scenario Divide and Conquer DCS Application Architecture vii

8 4.3.5 Recursive DCS Evaluation Theoretical Analysis Evaluation using Synthetic Packet Classifiers Summary OpenFlow Accelerator: A Decomposition-based Hashing Approach for Flow Processing Background Related Work Multidimensional Flow Processing Problem Statement Overview Preprocessing Query Update Evaluation Theoretical Comparison Implementation and Measurement Summary Flow Processing Accelerator Framework Overview Performance Evaluation and Analysis Search Time Memory Consumption Scalability viii

9 6.2.4 Updatability Determinism Conclusion BIBLIOGRAPHY ix

10 LIST OF TABLES 1.1 OpenFlow Required Field Table table Hierarchy Comparison Results An IPv4 Routing Table Search Time per IPv4 Packet (µs) A Simple Packet Classifier Comparison of Key Metrics for Range Encoding Schemes Range Statistics in Packet Classifiers Mask Contribution for Packet Classifiers A sample flow table Theoretical Comparison x

11 LIST OF FIGURES 1.1 Flow Table Pipeline High-Performance Flow Processing Framework Three Bucket States Insert and Delete in a DO Table Bitmap Format, Insert and Delete in the BH Table Hash Table Hierarchy Query Example Rebalancing Consequence Searching Direction A Simple Example The Bitmap for /24 Entry Hash Table Access Model Memory Access Comparison between BSOL and our scheme WAMA comparison Prefix Distribution for IPv4 routing tables Prefix Distribution for IPv6 routing tables IPv4 Hash Entry Distribution IPv6 Hash Entry Distribution Memory Usage (MB) Range Expansion Example Using PE Range Expansion Example Using DCS xi

12 4.3 Worst-case Range Expansion Scenario DCS Range Partition and Encoding DCS Range Encoding and Key Encoding Example DCS Block Division on Two Ranges Recursive DCS Range Encoding Recursive DCS Example fac ES Comparison on Source Range fac ES Comparison on Destination Range fac ES Comparison on Either Range full-fac ES Comparison on both ranges fac ES Comparison on Destination Range with RDCS Flow Table Pipeline Preprocess Query Processing Protocol Field Processing Prefix-Match Fields Rule Hash Table Construction Query Example Delete Example Search Time (µs) in LP M Fj,HT Fi and RHT WAMA comparison Memory Usage (MB) for 1k classifiers Memory Usage (MB) for 5k classifiers xii

13 5.13 P L Update frequencies Search Time (µs) in LP M Fj,HT Fi and RHT with more fields Preprocessing Flow Tables Query in the Framework Main Memory Access Memory Usage (MB) xiii

14 CHAPTER ONE INTRODUCTION As the number of users on computing networks grows and the number of applications available to them explodes, it results in more individual network conversations (flows) on the network at any time [4]. A flow is a stream of relevant packets that meet a collective matching criteria and share the same characteristics. A collection of flows is usually grouped into a flow table. Flow processing is the ability for a communications application to maintain active state (session) on each individual network conversation when all packets belonging to the state traverse the device for the entire duration. The state allows each packet to be treated the same way as defined in a flow table. Compared to packet processing which is regarded as stateless due to a basis of packet-by-packet processing without regard to flow or state information, flow processing requires the creation of sessions for individual flows. There is an increasing interest in designing high-performance network algorithms, frameworks or devices to perform flow processing. Applications such as stateful access control, deep inspection and flow-based load balancing require efficient flow processing. Fundamental network operations inherent to such applications include packet classification in flow-level based on packet classifiers, flow state management for stateful analysis, and per-flow packet order-preserving for traditional switch architecture. Meanwhile maintaining the network state on all flows passing through a system is of great significance for Software Defined Networking (SDN). The dissertation aims to accelerate flow processing in OpenFlow switches on which SDN architecture is established. OpenFlow uses the concept of flows to identify network traffic based on predefined match rules that can be statically or dynamically programmed by the SDN control software [18]. In 1

15 Openflow Switch Pipeline Ingress port Packet In Action Set = 0 Table 0 Packet + Ingress port + metadata Action Set Table 1 Table n Packet Action Set Execute Action Set Figure 1.1: Flow Table Pipeline the SDN architecture, the control and data planes are decoupled, network intelligence and state are logically centralized, and the underlying network infrastructure is abstracted from the applications [18]. As a consequence enterprises and carriers benefit from unprecedented programmability, automation, and network control. Highly scalable, flexible networks can thus be built to readily adapt to ever-changing business needs. OpenFlow is the first standard interface designed specifically for SDN, providing high-performance and granular traffic control across a variety of network devices. The core component in an OpenFlow switch lies upon its flow tables. Each entry (flow) of a table is associated with an action to guide the switch how to process it. All the flow tables are grouped together to form a pipeline shown in Figure 1.1. Figure 1.1 shows the flow table pipeline in an OpenFlow switch. A packet goes throughout a variety of flow tables, matches the highest-priority rule in each table and results in a corresponding action. Eventually a set of actions aggregated from individual actions is taken for the packet. According to [1], switch designers are free to implement the internals in any way convenient, provided that correct match and instruction semantics are preserved. Meanwhile the pipeline exposed by an OpenFlow switch makes it possible to adapt a variety of flow classification mechanisms, software or hardware, for different flow tables according to practical performance requirements. Besides the traditional layer-3 and layer-4 packet header fields, much more header fields 2

16 Table 1.1: OpenFlow Required Field Table Required Field Description IN PORT ETH DST ETH SRC ETH TYPE IP PROTO IPV4 SRC IPV4 DST IPV6 SRC IPV6 DST TCP SRC TCP DST UDP SRC UDP DST Ingress port Ethernet destination address Ethernet source address Ethernet type of the OpenFlow packet payload IPv4 or IPv6 protocol number IPv4 source address IPv4 destination address IPv6 source address IPv6 destination address TCP source port TCP destination port UDP source port UDP destination port are defined in OpenFlow specification [1]. Table 1.1 refers to Table-11 in [1] and presents 13 required flow table fields. Several fields such as IN PORT or IP PROTO only contain either a specific value or an arbitrary bitmask. A handful of other fields such as IPV4 SRC or IPV6 DST may contain subnet mask or arbitrary bitmask. Totally 42 fields are defined in [1] and in the future OpenFlow specification more new fields may be defined. In practice many kinds of lookup tables widely used in popular networking applications may be and are actually used straightforwardly as flow tables, e.g. IP routing tables, packet classifiers such as firewall filters and Access Control Lists (ACLs), etc. Thus far OpenFlow specification involves three match types: exact match, prefix match and wildcards. For example IP addresses using full-length mask (128-bit for IPv6 and 32-bit for IPv4) are exact-match fields. IP addresses using partial-length mask (e.g. /64 mask bits for IPv6 and /24 3

17 mask bits for IPv4) are prefix-match fields. Usually wildcards are also called ANY or don t care, e.g. in the field of IPv4 or IPv6 protocol number (type). It is highly possible that ranges will be added to represent some field values in the future OpenFlow specification as range match is widely used in numerous high-speed networking scenarios, e.g. packet classification. Usually a wildcard field value can be expanded into a collection of prefixes or exact values and hence we only need to consider the other three match types in practical problems. Given that an OpenFlow switch may contain a number of flow tables and any flow table may be any of the match type or a combination of match types, we need a variety of algorithms. For exact match, an efficient hashing scheme is a practical solution to store exact values; for prefix match, a high-performance IP address lookup (IP forwarding) algorithm is a good choice to conduct longest prefix match; if the set of match fields is the classic quintuple criteria, packet classification is the best approach to perform flow processing. Although exact match, prefix match and classic quintuple classification have been intensively studied as traditional research problems to accommodate high-speed networking applications, we still need to select appropriate algorithms in terms of performance and implementation cost. For multidimensional match characterized by an arbitrary number of fields and any match type, it is tricky to seek a universal solution. Current packet classification mechanisms fail in three drawbacks: scalability, incremental update and nondeterminism. 1. Scaling to the number of fields. Although some packet classification algorithms scale well in IP length (32-bit IPv4 and 128-bit IPv6) and flow table size, they do not scale in the number of fields. For example, trie or decision-tree based packet classification algorithms such as HyperCuts [45] or HiCuts [20] are inefficient because long tree depth due to increasing cutting dimensions leads to more costly memory accesses and degrades query performance. So are TCAM-based solutions as TCAM (Ternary Content Addressable Memory) entry width 4

18 is extremely restricted owing to expensive production cost and hence fail to scale to the number of fields. Only decomposition mechanisms such as Recursive Flow Classification (RFC) [19] or Bit Vector (BV) [16] have such scalability. 2. Incremental Update. A sophisticated controller dynamically adds and removes entries from flow tables in an OpenFlow switch on behalf of multiple independent experiments conducted by researchers with different accounts and permission [36]. As a consequence incremental update is compulsory for any practical flow processing solution. Unfortunately current decomposition algorithms such as RFC are incapable of incremental update. 3. Determinism. Deterministic query is highly desired for any high-speed flow processing solution to be implemented and deployed in advanced, special-purpose, high-performance hardware devices such as Network Processing Unit (NPU). However ordinary decomposition algorithms such as RFC are heuristic-based and lack necessary determinism. Especially when implemented in NPU in which multiple threads are to be coordinated to accelerate processing, the slowest thread determines the overall speed for a stack of classification tasks and non-deterministic property in such decomposition algorithms brings worse lookup performance. Due to these drawbacks, undoubtedly processing a flow table with an arbitrary number of fields and any match type poses an entirely new challenge to current packet classification mechanisms because the new problem only recently emerges as a consequence of development of new SDN and OpenFlow technologies. To address the new problem, we propose an innovative decomposition algorithm to achieve scalability, incremental update and determinism. Meanwhile the variety of match types within flow tables motivates us to design three other algorithms to handle exact-match, prefix-match and classic quintuple match. Altogether these four 5

19 flow table pipeline Hashing Engine IP lookup engine TCAM PC engine Universal-Processing engine Table of exact values Table of prefixes Table of classic 5-tuple entries Table with multiple dimensions Table entry Table entry Table entry Table entry entry action address mask action src adr dst adr src port dst port type action any any action Figure 1.2: High-Performance Flow Processing Framework algorithms provide necessary balance between performance and implementation cost. A highperformance flow processing accelerator (framework) is thus proposed to process the pipeline of flow tables. The framework highlighted in Figure 1.2 is composed of four components (engines or modules): a hashing engine for exact-match only tables, an IP lookup engine for prefix-match only tables, a TCAM-based engine for classic quintuple tables, and a universal-processing engine for flow tables featured by multidimensional match with an arbitrary number of fields and any match type. As shown in Figure 1.2 each framework component adopts a specific approach to processing a collection of flow tables. 1. Hashing engine using a highly deterministic hashing scheme [54] for exact-match only tables. We propose two novel collision resolution mechanisms, Double Out hashing and Bidirectional Hop Hashing and establish a multiple-segment hash construction to facilitate deterministic queries. Collision is restricted to only one segment and the length of a probe sequence in each segment is minimized to 1. In addition an important category of false pos- 6

20 itive is reduced to 0 due to exact bitmap filters. The use of a unique set of hash functions for filters and hash tables avoids unnecessary computation. The hashing scheme continues our previous work by utilizing hashing and bitmap for not only IP address lookup but also general-purpose high-speed packet processing. 2. IP lookup engine using a hierarchical hashing scheme [52] for prefix-match tables. By taking advantage of bitmap and hashing techniques effectively used in Tree Bitmap (TBMP) algorithm [14] and Binary hash Searching On prefix Length (BSOL) algorithm [66], our proposed hierarchical hashing scheme significantly improve IP lookup efficiency by remarkably reducing the number of memory accesses, consuming less memory and enabling fast update. 3. TCAM-based engine characterized by a Divide-and-Conquer Scheme (DCS) [53] for classic quintuple tables. The scheme addresses the range expansion problem which sharply degrades TCAM storage performance. It allows fast preprocessing, constant time searching, and dynamic incremental update. Classic quintuple ACL classifiers can be straightforwardly used as flow tables in an Open- Flow switch. TCAM-based solutions benefit from deterministic, constant-time performance. Meanwhile only TCAM-based solution satisfies the high-speed packet-by-packet processing requirement in particular scenarios. [36] introduces such a scenario where a high-performance packet classification engine takes effect when the engine must be placed outside from an OpenFlow switch. From a broader perspective any hardware solution for flow processing is desirable if production cost is bearable. 4. Universal-processing engine using a decomposition approach [55] for flow tables featured by multidimensional match with an arbitrary number of fields and any match type. The ap- 7

21 proach performs individual search on each field and conducts a single query in a single hash table. We observe that single field searching is well studied and an efficient LPM method contributes to fast query in individual prefix-match fields without respect to the number of fields in a flow table. Furthermore we address the challenge of efficiently aggregating and combining the results of the single field searches. Our approach scales well to the field number and maintains low latency and incremental update. Meanwhile these components are not independent. The hashing engine and IP lookup engine can be used in the universal-processing engine. Except TCAM-based engine, others can be implemented in either software or hardware, especially in NPU. In the next chapters we introduce each algorithm corresponding to each component in an individual chapter and lay out the framework with the overall performance evaluation and analysis. Eventually a conclusion is made in the last chapter. 8

22 CHAPTER TWO HIGHLY DETERMINISTIC HASHING SCHEME USING BITMAP FILTER 2.1 Background Hash tables are widely used in various network applications for high-speed packet processing due to O(1) primitive operations of query, insert and delete [9, 47]. However collisions occur frequently when the table occupancy (load) increases. Newly inserted elements that collide with existing elements are inserted into other buckets thereby leading to an increase in the length of the probe sequence or collision chain, which is followed during the query [31]. Elements at the tail of a long sequence require considerable more probing time than the elements close to the head. Owing to non-determinism caused by long probe sequences the query performance in real-time networking applications is vulnerable to adversarial traffic. In modern hardware devices, multiple threads are coordinated to accelerate hash operations and synchronization is compulsory to keep processing order. While synchronization ensures that requests are tackled in order they arrived, it also reduces overall performance to that of the slowest thread determines. As the number of non-deterministic threads increases, the slowest thread tends to turn much slower and the average query performance degrades tremendously. Among a variety of hash collision resolution approaches, multiple-segment hashing balances the bucket load by reducing the maximum number of keys in a bucket among all buckets. To avoid probing multiple buckets, on-chip summaries such as Bloom filters link keys in multiple buckets. Several multiple-segment hashing systems proposed in a string of papers [23, 30, 31, 47] remarkably improve deterministic hash operations and achieve good average performance. For instance Peacock hashing [31] limits the length of collision chains in the table segments to a small 9

23 constant. Peacock hashing and approaches [30, 47] adopt probabilistic, on-chip filters to reduce off-chip memory accesses and enable deterministic hashing performance. Nevertheless we uncover critical drawbacks in these systems. Firstly collisions are still allowed in each segment; next the length of the probe sequence is not minimized; and probabilistic Bloom filters only create approximate summary and cannot eliminate an important category of false positive which invites costly off-chip memory accesses in a great number of high-speed networking applications; finally Bloom filters usually employ a distinct set of hash functions from those for table segments. The hash computation overhead for filter query is worthless to hash table access but never trivial when under heavy network traffic. Bloom filters used in previous multiple-segment hashing systems do not discriminate between two categories of false positive. The first category occurs when any request must match some element in one segment but the match result is wrongly reported in the other segment(s). In other words only when the element is exactly matched in the unique, correct segment, the first category of false positive never happens. The second category occurs when a request should not match any element but the matching result is wrongly reported in one or more segments. The first category is of great significance to high-speed network applications which do not need to handle the second category. For example, IP address lookup forwards packets using routing tables. In a routing table a rule with lowest priority usually matches any packet which does not match others. For another instance, rule sets used in packet classification algorithms usually contain a rule which matches any packet. Or there is always a miss-flow entry in an OpenFlow flow table with the same functionality. A Bloom filter can only reduce but never eliminate the first category. In a high-speed network device which processes millions of packets every second, tens of thousands of packets are mismatched even with a 1% false positive in a multiple-segment hashing system using Bloom filters, leading to costly off-chip memory access. The situation becomes worse as the volume of 10

24 network traffic increases. In the dissertation we establish a multiple-segment hashing system using Double Out hashing (DO hashing) and Bidirectional Hop hashing (BH hashing). Our hashing construction, called the hierarchy, consists of the number of n hash segments (tables) ordered by insert sequence. Each hash table acts as a collision buffer for the tables with higher order and uses a distinct hash function to reduce collision probability. Collision is only allowed in the lowest-order table adopting BH hashing. Other tables use DO hashing to avoid collision with the aid of on-chip bitmap filters. Our scheme outperforms previous multiple-segment hashing schemes by: promoting deterministic query performance to facilitate network processing by restricting collision to only one hash table and minimizing the probe sequence to only 1 in each hash table; eliminating a significant category of false positive using innovative on-chip bitmap filters; avoiding unnecessary hash function computation for on-chip filter queries. 2.2 Related Work A number of related work discussed various perspectives to improve hash performance for high-speed network applications. The importance of determinism for hardware performance was studied in [12, 31]. Demetriades et al. [12] resolves collision and guarantees deterministic IP lookup by dynamically migrating IP prefixes. Researchers [47, 59] attempt to ensure that a single bucket reference is used for each table lookup with high probability. Simple hash functions [28] are suggested for essentially the same performance as a truly random hash function which causes heavy computation overhead. The Hash Index Table [24] proposes a novel on-chip filter. Huang et al. [23] 11

25 reduce on-chip memory consumption by storing hashed bucket addresses into an intermediate index table but incur space overhead and delays due to indirect access. Our scheme is influenced by these work and will show determinism can be improved by better resolving collisions in each hash table. Our scheme uses simple hash functions and on-chip bitmap filters for exact summary and quick query. Some researchers [9,12,25,27] argue multiple-segment hashing is among the best solutions to resolve collision. Our primary starting points are Fast Hash Table (FHT) [47] which summarizes the locations of items in a hash table that uses multiple hash functions, and Peacock hashing [31] that greatly reduces on-chip memory consumption compared to FHT while keeping determinism by limiting the length of the probe sequence in each table to a small constant. Both achieve a deterministic performance by providing multiple table locations before inserting any new element. To avoid multiple off-chip memory accesses they use probabilistic Bloom filters and hence enable deterministic hashing performance. We will show our scheme further improves determinism and eliminates drawbacks of Bloom filters. 2.3 DO hashing and BH hashing DO hashing We invent DO hashing for highly deterministic query. A Double Out hash table (DO table) T i (1 i n 1) is an array of buckets. Given an element E its index idx in T i is computed using h i (E), T i s associated hash function. A bucket contains at most one element. T i is associated with three auxiliary structures: a main bitmap filter B i, a collision table ColT i and a collision bitmap filter ColB i. B i maintains an exact summary of T i buckets and each B i bit is associated with a specific T i bucket. It is set if the bucket contains an element, or clear otherwise. ColT i keeps 12

26 I D Collision State 3 D I State 2 Occupied I D State 1 Empty B i bit clear and ColB i bit set B i bit set and ColB i bit clear Bit clear in both B i and ColB i Figure 2.1: Three Bucket States references to all the collided elements in T i. A reference (j, index) to E indicates E s residing table T j and corresponding index. A ColT i bucket is associated with a specific T i bucket and keeps a linked list of references to collided elements in the T i bucket. ColB i maintains an exact summary of ColT i buckets and each ColB i bit is associated with a specific ColT i bucket. It is set if the list in the bucket is not empty, or clear otherwise. A DO table T i and its auxiliary structures have the same dimension. Furthermore a T i bucket s index is the same position to access its associated B i bit, ColB i bit, and ColT i bucket. As the only hash function for T i and its auxiliary structures, h i is used for index computation. A bucket state reveals how primitive operations are performed in a DO table. Figure 2.1 demonstrates the state transfer among State 1 (Empty), State 2 (Occupied), and State 3 (Collision). A circle symbolizes a state and a directed arrow expresses a single primitive operation, e.g. D for a delete and I for an insert attempt. A bucket state is marked by a B i bit and a ColB i bit collectively, e.g. State 2 by B i bit set and ColB i bit clear. State 1 stands for an empty bucket. When an element is inserted, State 1 is transferred to State 2. State 2 implies that an element resides in the bucket. After deleting it State 2 is transferred to State 1. When another element is indexed to a bucket in State 2, the collision occurs and State 2 is transferred to State 3. The existing element is deleted and both elements try the next lower-order 13

27 hash table. A State 3 bucket remains empty and rejects the insert attempt of any new element which has to try the next table. Thus no state transfer occurs in a State 3 bucket for any insert attempt. A delete in a State 3 bucket causes two outcomes. The first outcome does not incur state transfer but the second one does. We will illustrate them later. A DO table disallows collision and restricts the length of a probe sequence to only 1. The consequence of inserting an element E to T j relies on current bucket state. State 1: no collision and inserted. State 2: encounter collision. Remove the existing element and try both elements in T j+1. State 3: still collision and try E in T j+1. Algorithm 1 states the insert procedure. Re-balancing will be discussed in the delete procedure. All collided elements which fail to be inserted to the DO table T n 1 will try T n. The insert procedure reveals that an element must reside in only one hash table. Once E is inserted into T i, its reference is added into each ColT j (1 j i 1) and the ColT j bucket s index is computed using h j (E). Figure 2.2 exhibits a simple insert and delete example using two DO tables consisting of 8 and 6 buckets respectively. In Figure 2.2(a) T 1 contains E 1 in b 3 (index as subscript). T 2 is empty and not shown. New element E 2 collides with E 1 in b 3. Figure 2.2(b) demonstrates the collision outcome. ColT 2 and ColB 2 are not shown. Suppose E 1 and E 2 are inserted into different buckets b 5 and b 0 in T 2. So list L 3 of ColT 1 contains their references, (2, 5) and (2, 0) respectively. B 1 s corresponding bit is clear and ColB 1 s bit in the same position is set while B 2 s corresponding bits are set. Elements in T 1 can be deleted without involving any collision table. However if an element E is deleted from T j (1 < j n), its references are removed from corresponding lists of collision 14

28 Algorithm 1 Insert Algorithm for DO Table FuncDoubleOutInsert(E,i) {i: table number of T i } int idx = h i (E); {calculate index} Bucket b = T i.getbucket(idx); if b in State 1 then T i.insert(idx, E); for all j from 1 to i 1 do int newidx = h j (E); {recalculate index} ColT j.insert(newidx,i,idx); {insert reference} else if b in State 2 then E c = T i.remove(idx); {E c : existing element} FuncDoubleOutInsert(E c, i + 1); {try E c in T i+1 } FuncDoubleOutInsert(E, i + 1); {try E in T i+1 } else FuncDoubleOutInsert(E, i + 1); {b in State 3} B 1 E ColB T 1 b 0 b 1 b 2 E 1 b 4 b 5 b 6 b 7 ColT 1 b 0 b 1 b 2 b 3 b 4 b 5 b 6 b 7 B 1 (a) try E 2 in T 1 and it collides with E 1 in b 3 2, 5 3, ColB T 1 b 0 b 1 b 2 b 3 b 4 b 5 b 6 b 7 ColT b 1 0 b 1 b 2 L 3 b 4 b 5 b 6 b 7 B 2 T E 2 b 1 b 2 b 3 b 4 E 1 (b) insert E1 into b5 of T2, E2 in b6 of T2 2, 0 B ColB T 1 b 0 b 1 b 2 b 3 b 4 b 5 b 6 b 7 ColT 1 b 0 b 1 b 2 L 3 b 4 b 5 b 6 b 7 B 2 T E 2 b 1 b 2 b 3 b 4 b 5 (c) after deleting E1, L3 of ColT1 contains only E2 reference B ColB T 1 b 0 b 1 b 2 E 2 b 4 b 5 b 6 b 7 ColT 1 b 0 b 1 b 2 b 3 b 4 b 5 b 6 b 7 (d) move E 2 to b 3 of T 1 Figure 2.2: Insert and Delete in a DO Table 15

29 tables from ColT 1 to ColT j 1 and each list resides in the bucket with index of h j (E). As we discussed in Figure 2.1, two outcomes happen after a delete in a State 3 bucket. After removing E s reference if any list contains at least two references, this is the first outcome and no state transfer occurs in any bucket. However if any list contains only one reference, the second outcome happens as shown in Figure 2.2(c) and (d). Return to the example and now delete E 1. Figure 2.2(c) displays its reference in L 3 of ColT 1 is also removed. Since L 3 of ColT 1 contains only one reference of E 2, no collision exists in b 3 of T 1 any more and under this circumstance E 2 should be moved back. Figure 2.2(d) shows the result of moving E 2 from b 0 of T 2 to b 3 of T 1. Its reference is removed from ColT 1. T 2 is empty and not shown. B 1 s associated bit with b 3 of T 1 is set and ColB 1 s associated bit with L 3 of ColT 1 is cleared. The delete in one bucket, e.g. b 5 of T 2, causes the state transfer at another bucket, e.g. b 3 of T 1, from State 3 to State 2 in Figure 2.1. Algorithm 2 states the delete procedure. Given an element E to be deleted, its current residing table T i and its bucket index b, the delete procedure proceeds as illustrated in Figure 2.2(c) and (d) if the collision list with the only reference after the deletion is detected. E s reference in all its previous collision tables should be removed. Once it is removed, the detection of one-reference list in current collision table is performed. If current list contains only one reference, that indicates no collision for this reference. New insertion should be conducted. Therefore a simple deletion may lead to a great number of chaining reaction for insertion and deletion. It is not reasonable for frequent update. Re-balancing is defined as a process to achieve load balance among hash tables by moving elements from low-order tables to high-order tables. Re-balancing takes place whenever a delete operation gives rise to a one-reference collision list which leads to more deletions and insertions. Through re-balancing an element is moved to the first hash table where it has no collision, e.g. moving E 2 from T 2 to T 1 in Figure 2.2(d). Re-balancing also takes effect when one-reference 16

30 Algorithm 2 Delete Algorithm for DO Table FuncDoubleOutDelete(E,i,b) {E: element to be deleted; i: E in T i ; b: bucket index} T i.remove(b);{delete this element} B i.clearbit(b);{clear corresponding B i bit} for all j from 1 to i 1 do int nb = h j (E); {calculate new index} ColT j.delete(nb,i,b); {delete E s reference} if ColT j [b].size = 1 then int ni = gett No(ColT j [b](0)); {the only element in T ni } int nindex = getindex(colt j [b](0)); {this element s index in T ni } ne = getelem(ni, nindex); {retrieve this element} ColT j.delete(j,ni,nindex); {delete this reference} FuncDoubleOutInsert(nE, ni);{insert ne in T ni } list is generated by discarding one element due to an insert in a State 2 bucket. Re-balancing is of extraordinary significance to maintain global load balance in the hash tables. Without it State 3 buckets in high-order tables cannot preserve new elements, leading to ever-increasing load in low-order tables and eventually high discard rate in T n BH hashing We design BH hashing for T n (the BH table) which allows one collision for each bucket. The first collided element in a bucket may be inserted in a backup bucket through bidirectional searching in a neighborhood area. The backup bucket is called next-hop and current bucket is called previous-hop. Their relation is expressed using a bitmap. An element is discarded if it is indexed to a bucket with next-hop or the bidirectional searching returns a non-empty bucket. The discard rate is the percentage of elements dropped in T n. If T n contains more next-hops the discard rate increases. If an element E in a bucket is deleted, the element in the bucket s next-hop needs to be moved to the bucket. This substantially improves T n load and diminishes the discard rate. In addition E s references in all collision tables are removed. 17

31 A binary bitmap is associated with a bucket to encode collision information. We choose the value of k, a tunable parameter, to vary the bidirectional searching distance. Larger k values increase the opportunity for an element to find its next-hop in the bidirectional searching and hence reduce the discard rate in general. Figure 2.3(a) depicts a bitmap format with 2k+3 bits. From left the first bit indicates whether the bucket is empty. The second bit indicates whether the bucket has a next-hop. If it is set, the k-bit next hop block (index from 3 to k+2) encodes the relative distance to the next-hop in binary. The third bit indicates whether the bucket has prev-hop. If it is set, the k-bit prev-hop block (index from k+3 to 2k+2) encodes the relative distance to prev-hop. In practice the second bit and k-bit next-hop block (or the third bit and k-bit prev-hop block) act as the next pointer (or the previous pointer) in a double linked list. The neighborhood of a bucket b p covers 2 k buckets, forwardly from b p+1 to b p+2 k 1 and backwardly from b p 1 to b p 2 k 1. The bidirectional searching starts forwardly and then backwardly. As soon as an empty bucket is found, the bidirectional searching terminates and returns the bucket. If no empty bucket is found, discard the element. Figure 2.3 displays an example of insert and delete in the BH table. Assume k = 3. T n contains four elements. b 3 (index as subscript) contains E 1 and has no next hop. The first bit of b 3 bitmap is set. b 4, b 5 and b 6 contain E 2, E 3 and E 4 respectively and their bitmaps are not shown. In Figure 2.3(b) E 5 is indexed to b 3 and collides with E 1. The bidirectional searching begins from b 4 and returns b 7 to which E 2 is inserted since buckets from b 4 to b 6 are not empty. Figure 2.3(c) describes the connection between b 3 and b 7. The second bit in b 3 bitmap is set for next-hop and k-bit next-hop block encodes the distance 4 (011 in binary). Likewise the first and third bits in b 7 bitmap are set for non-empty and prev-hop indication, and k-bit prev-hop block encodes the same distance. In Figure 2.3(d) E 1 is deleted. E 5 in b 3 s next hop b 7 is moved back to b 3. So b 7 is empty and its bitmap is not shown. Eventually only the first bit in b 3 bitmap is set to represent E 5. 18

32 Empty flag Next flag Prev flag k-bit next hop block k-bit prev hop block bitmap k+1 k+2 k+3 k+4 2k+1 2k+2 (a) bitmap with 2k+3 bits for a bucket E 5 insert b 3 bitmap T n b 1 b 2 E 1 E 2 E 3 E 4 b 7 b 8 b m-1 b m (b) try E 5 in b 3 and collide with E b 3 bitmap b 7 bitmap T n b 1 b 2 E 1 E 2 E 3 E 4 E 5 b 8 b m-1 b m (c) insert E 5 in b 7 and reset bitmap of b 3 ; E 1 to be deleted b 3 bitmap move back b 1 b 2 E 5 E 2 E 3 E 4 b 7 b 8 b m-1 b m (d) delete E 1 and move back E 5 to b 3 Figure 2.3: Bitmap Format, Insert and Delete in the BH Table By choosing a reasonable k value, a cache line can contain an element and its next-hop. In consequence only one memory access is sufficient to query an element. This eradicates the drawback of cuckoo hashing [38] or linear hashing, i.e. the need to access sequences of unrelated locations on different cache lines. 2.4 Hierarchy Design From what we have discussed above, multiple-segment hashing balances the bucket load by reducing the maximum number of keys in a bucket among all buckets. The use of on-chip summaries further reduces the number of probes in multiple buckets. Several multiple-segment hashing systems such as Peacock hashing and FHT have fairly good performance. Thus our proposed scheme is based on similar hierarchical structure with multiple hash segments. In our scheme DO hashing and BH hashing work collaboratively. All the hash tables except the last one employ 19

33 High B 1 T b 0 b 1 b 2 b 3 b 4 b 5 b 6 ColB 1 ColT b 0 b 1 b 2 b 3 b 4 b 5 b 6 B ColB Insert order T 2 B n-1 b 0 b 1 b 2 b 3 b ColT 2 ColB n-1 b 0 b 1 b 2 b 3 b T n-1 b 0 b 1 ColT n-1 b 0 b 1 Low T n b 0 b 1 b 2 Figure 2.4: Hash Table Hierarchy DO hashing for the benefit of fast query using on-chip filters. Collision is disallowed in each DO table, and the length of a probe sequence is limited to only 1 accordingly. The BH table adopts BH hashing, stores the collided elements in all the DO tables, and allows one collision for each bucket. Only one memory access is needed in the BH table to look up two associated elements in a cache line. Figure 2.4 depicts the hierarchy framework. The vertical block highlights n number of hash tables. The insert order is the table sequence to insert an element and T 1 is always the first choice to try an element E. If E collides in T 1, try T 2. Continue to try E in each lower-order table along the insert order until it is either inserted into a table without collision or discarded if it fails to be inserted in T n. Thus each hash table except T 1 acts as a collision buffer for higher-order tables. The horizontal block encircles T 1 and its three auxiliary structures, also in every DO table. Our scheme follows two fundamental principles. 1. Keep as few elements as possible in T n. 2. Insert an element to the first hash table with no collision. 20

34 Principle 1 enables DO tables to contain the majority of elements for the sake of fast query using on-chip filters. According to Principle 2 an element is inserted to T j (j > 1) only if it collides in all the higher-order tables, i.e. from T 1 to T j 1. A delete of an element E from T j (j > 1) leads to removal of E s all references in the collision tables from ColT 1 to ColT j 1. Principle 2 results in element movement due to re-balancing under one-reference list circumstance. In terms of the framework and principles we need to determine three essential aspects in the hierarchy. Firstly is there an optimal load for a DO table and what is relation between load factor and discard rate in the BH table? Secondly what is an ideal number of hash tables in the hierarchy and how to assign table dimension given an input set? Finally how to query an element using filters and tables? Our purpose is to achieve balance between memory consumption, discard rate and query efficiency. In the remaining dissertation c i denote the dimension of a hash table. m and m i represent the numbers of elements in all hash tables and T i (the i th table) respectively Load Analysis An element is discarded if indexed to a T n bucket with next-hop or the bidirectional searching returns a non-empty bucket. Therefore larger T n load gives rise to more next-hops and a larger discard rate. We discover that at 38% load the discard rate is about 1% and all the discarded elements are only attribute to next-hop collision. The bidirectional searching always returning empty buckets even with a fairly small k value, e.g. 2. So 3 w=1 Formally S w represents the number of buckets in a DO table T i under State w (1 w 3). S k = c i wherein c i is T i dimension. Suppose an element has an equal probability to be indexed in any bucket. Equation (2.1) counts the number of State 2 buckets in T i where r i is the 21