TimeMemory TradeOffs: False Alarm Detection Using Checkpoints


 Nickolas Gallagher
 1 years ago
 Views:
Transcription
1 TimeMemory TradeOs: False Alarm Detection Using Checkpoints Gildas Avoine 1, Pascal Junod 2, and Philippe Oechslin 1,3 1 EPFL, Lausanne, Switzerland 2 Nagravision SA (Kudelski Group), Switzerland 3 Objecti Sécurité, Gland, Switzerland Abstract. Since the original publication o Martin Hellman s cryptanalytic timememory tradeo, a ew improvements on the method have been suggested. In all these variants, the cryptanalysis time decreases with the square o the available memory. However, a large amount o work is wasted during the cryptanalysis process due to socalled alse alarms. In this paper we present a method o detection o alse alarms which signiicantly reduces the cryptanalysis time while using a minute amount o memory. Our method, based on checkpoints, reduces the time by much more than the square o the additional memory used, e.g., an increase o 0.89% o memory yields a 10.99% increase in perormance. Beyond this practical improvement, checkpoints constitute a novel approach which has not yet been exploited and may lead to other interesting results. In this paper, we also present theoretical analysis o timememory tradeos, and give a complete characterization o the variant based on rainbow tables. Key words: timememory tradeo, cryptanalysis, precomputation 1 Introduction Many cryptanalytic problems can be solved in theory using an exhaustive search in the key space, but are still hard to solve in practice because each new instance o the problem requires to restart the process rom scratch. The basic idea o a timememory tradeo is to carry out an exhaustive search once or all such that ollowing instances o the problem become easier to solve. Thus, i there are N possible solutions to a given problem, a timememory tradeo can solve it with T units o time and M units o memory. In the methods we are looking at T is proportional to N 2 /M 2 and a typical setting is T = M = N 2/3. The cryptanalytic timememory tradeo has been introduced in 1980 by Hellman [8] and applied to DES. Given a plaintext P and a ciphertext C, the problem consists in recovering the key K such that C = S K (P ) where S is an encryption unction assumed to ollow the behavior o a random unction. Encrypting P under all possible keys and storing each corresponding ciphertext
2 allows or immediate cryptanalysis but needs N elements o memory. The idea o a tradeo is to use chains o keys. It is achieved thanks to a reduction unction R which generates a key rom a ciphertext. Using S and R, chains o alternating ciphertexts and keys can thus be generated. The key point is that only the irst and the last element o each chain are stored. In order to retrieve K, a chain is generated rom C. I at some point it yields a stored end o chain, then the entire chain is regenerated rom its starting point. However, inding a matching end o chain does not necessarily imply that the key will be ound in the regenerated chain. There exist situations where the chain that has been generated rom C merges with a chain that is stored in the memory which does not contains K. This situation is called a alse alarm. Matsumoto, with Kusuda [10] in 1996 and with Kim [9] in 1999, gave a more precise analysis o the parameters o the tradeo. In 1991, Fiat and Naor [6, 7] showed that there exist cryptographically sound oneway unctions that cannot be inverted with such a tradeo. Since the original work o Hellman, several improvements have been proposed. In 1982, Rivest [5] suggested an optimization based on distinguished points (DP) which greatly reduces the amount o lookup operations which are needed to detect a matching end point in the table. Distinguished points are keys (or ciphertexts) that satisy a given criterion, e.g., the last n bits are all zero. In this variant, chains are not generated with a given length but they stop at the irst occurrence o a distinguished point. This greatly simpliies the cryptanalysis. Indeed, instead o looking up in the table each time a key is generated on the chain rom C, keys are generated until a distinguished point is ound and only then a lookup is carried out in the table. I the average length o the chains is t, this optimization reduces the amount o lookups by a actor t. Because merging chains signiicantly degrades the eiciency o the tradeo, Borst, Preneel, and Vandewalle [4] suggested in 1998 to clean the tables by discarding the merging and cycling chains. This new kind o tables, called perect table, substantially decreases the required memory. Later, Standaert, Rouvroy, Quisquater, and Legat [14] dealt with a more realistic analysis o distinguished points and also proposed an FPGA implementation applied to DES with 40bit keys. Distinguished points can also be used to detect collisions when a unction is iterated, as proposed by Quisquater and Delescaille [13], and van Oorschot and Wiener [15]. In 2003, Oechslin [12] introduced the tradeo based on rainbow tables and demonstrated the eiciency o his technique by recovering Windows passwords. A rainbow table uses a dierent reduction unction or each column o the table. Thus two dierent chains can merge only i they have the same key at the same position o the chain. This makes it possible to generate much larger tables. Actually, a rainbow table acts almost as i each column o the table was a separate single classic 4 table. Indeed, collisions within a classic table (or a column o a rainbow table) lead to merges whereas collisions between dierent classic tables (or dierent columns o a rainbow table) do not lead to a merge. This analogy can be used to demonstrate that a rainbow table o mt chains o length t has 4 By classic we mean the tables as described in the original Hellman paper.
3 the same success rate as t single classic tables o m chains o length t. As the tradeo based on distinguished point, rainbow tables reduce the amount o lookups by a actor o t, compared to the classic tradeo. Up until now, tradeo techniques based on rainbow tables are the most eicient ones. Recently, an FPGA implementation o rainbow tables has been proposed by Mentens, Batina, Preneel, and Verbauwhede [11] in order to retrieve Unix passwords. Whether it is the classic Hellman tradeo, the distinguished points or the rainbow tables, they all suer rom a signiicant quantity o alse alarms. Contrarily to what is claimed in the original Hellman paper, alse alarms may increase the time complexity o the cryptanalysis by more than 50%. We will explain this point below. In this paper, we propose a technique whose goal is to reduce the time spent to detect alse alarms. It works with the classic tradeo, with distinguished points, and with rainbow tables. Such an improvement is especially pertinent in practical cryptanalysis, where timememory tradeos are generally used to avoid to repeat an exhaustive search many times. For example, when several passwords must be cracked [12], each o them should not take more than a ew seconds. In [1], the rainbow tables are used to speed up the search process in a special database. In such a commercial application, time is money, and thereore any improvement o timememory tradeo also. In Section 2, we give a rough idea o our technique based on checkpoints. We provide in Section 3 a detailed and ormal analysis o the rainbow tables. These new results allow to ormally compute the probability o success, the computation time, and the optimal size o the tables. Based on this analysis we can describe and evaluate our checkpoint technique in detail. We illustrate our method by cracking Windows passwords based on DES. In Section 4, we show how a tradeo can be characterized in general. This leads to the comparison o the three existing variants o tradeo. Finally, we give in Section 5 several implementation tips which signiicantly improve the tradeo in practice. 2 Checkpoint Primer 2.1 False Alarms When the precalculation phase is achieved, a table containing m starting points S 1,..., S m and m end points E 1,..., E m is stored in memory. This table can be regenerated by iterating the unction, deined by (K) := R(S K (P )), on the starting points. Given a row j, let X j,i+1 := (X j,i ) be the ith iteration o on S j and E j := X j,t. We have: S 1 = X 1,1 X1,2 X1,3... X1,t = E 1 S 2 = X 2,1 X2,2 X2,3... X2,t = E 2.. S m = X m,1 Xm,2 Xm,3... Xm,t = E m
4 In order to increase the probability o success, i.e., the probability that K appears in the stored values, several tables with dierent reduction unctions are generated. Given a ciphertext C = S K (P ), the online phase o the cryptanalysis works as ollows: R is applied on C in order to obtain a key Y 1, and then the unction is iterated on Y 1 until matching any E j. Let s be the length o the generated chain rom Y 1 : C R Y 1 Y2... Ys Then the chain ending with E j is regenerated rom S j until yielding the expected key K. Unortunately K is not in the explored chain in most o the cases. Such a case occurs when R collides: the chain generated rom Y 1 merged with the chain regenerated rom S j ater the column where Y 1 is. That is a alse alarm, which requires (t s) encryptions to be detected. Hellman [8] points out that the expected computation due to alse alarms increases the expected computation by at most 50 percent. This reasoning relies on the act that, or any i, i (Y 1 ) is computed by iterating i times. However i (Y 1 ) should be computed rom Y i because i (Y 1 ) = (Y i ). In this case, the computation time required to reach a chain s end is signiicantly reduced on average while the computation time required to rule out alse alarms stays the same. Thereore, alse alarms can increase by more than 50 percent the expected computation. For example, ormulas given in Section 3 allow to determine the computation wasted during the recovering o Windows passwords [12]: alse alarms increase by 125% the expected computation. 2.2 Ruling Out False Alarms Using Checkpoints Our idea consists in deining a set o positions α i in the chains to be checkpoints. We calculate the value o a given unction G or each checkpoint o each chain j and store these G(X j,αi ) with the end o each chain X j,t. During the online phase, when we generate Y 1, Y 2,..., Y s, we also calculate the values or G at each checkpoint, yielding the values G(Y αi +s t). I Y s matches the end o a chain that we have stored, we compare the values o G or each checkpoint that the chain Y has gone through with the values stored in the table. I they dier at least or one checkpoint we know that this is a alse alarm. I they are identical, we cannot determine i a alse alarm will occur without regenerating the chain. In order to be eicient, G should be easily computable and the storage o its output should require ew bits. Below, we consider the unction G such that G(X) simply outputs the less signiicant bit o X. Thus we have: Pr{G(X j,α ) G(Y α+s t ) X j,α Y α+s t } = 1 ( 1 1 ) K 2. The case X j,α Y α+s t occurs when the merge appears ater the column α (Fig 1). The case X j,α = Y α+s t occurs when either K appears in the regenerated chain or the merge occurs beore the column α (Fig. 2).
5 S j checkpoint X j,α E j S j checkpoint X j,α E j Y s Y α+s t Y s Y α+s t Y 1 Y 1 Fig. 1. False alarm detected with probability 1/2 Fig. 2. False alarm not detected In the next section we will analyze the perormances o perect rainbow tables in detail. Then, we will introduce the checkpoint concept in rainbow tables and analyze both theoretical and practical results. 3 Perect Rainbow Tables and Checkpoints 3.1 Perect Tables The key to an eicient tradeo is to ensure that the available memory is used most eiciently. Thus we want to avoid the use o memory to store chains that contain elements which are already part o other chains. To do so, we irst generate more chains than we actually need. Then we search or merges and remove chains until there are no merges. The resulting tables are called perect tables. They have been introduced by [4] and analyzed by [14]. Creating perect rainbow and DP tables is easy since merging chains can be recognized by their identical end points. Since end points need to be sorted to acilitate the lookups, identiying the merges comes or ree. Classic chains do not have this advantage. Every single element o every classic chain that is generated has to be looked up in all elements o all chains o the same table. This requires mtl lookups in total where l is the number o stored tables. A more eicient method o generating perect classic tables is described in [2] Perect classic and DP tables are made o unique elements. In perect rainbow tables, no element appears twice in any given column, but it may appear more than once across dierent columns. This is consistent with the view that each column o a rainbow table acts like a single classic table. In all variants o the tradeo, there is a limit to the size o the perect tables that can be generated. The bruteorce way o inding the maximum number o chains o given length t that will not merge is to generate a chain rom each o the N possible keys and remove the merges. In the ollowing sections, we will consider perect tables only. 3.2 Optimal Coniguration From [12], we know that the success rate o a single unperect rainbow table is 1 t ( i=1 1 m i ) N where mi is the number o dierent keys in column i. With
6 perect rainbow tables, we have m i = m or all i s.t. 1 i t. The success rate o a single perect rainbow table is thereore ( P rainbow = 1 1 m ) t. (1) N The astest cryptanalysis time is reached by using the largest possible perect tables. This reduces the amount o duplicate inormation stored in the table and reduces the number o tables that have to be searched. For a given chain length t, the maximum number m max (t) o rainbow chains that can be generated without merges is obtained (see [12]) by calculating the number o independent elements at column t i we start with N elements in the irst column. Thus we have m max (t) = m t ( where m 1 = N and m n+1 = N 1 e mn N For non small t we can ind a closed orm or m max (see [2]): m max (t) 2N t + 2. ) where 0 < n < t. From (1), we deduce the probability o success o a single perect rainbow table having m max chains: ( Prainbow max = 1 1 m max N ) t 1 e t mmax N 1 e 2 86%. Interestingly, or any N and or t not small, this probability tends toward a constant value. Thus the smallest number o tables needed or a tradeo only depends on the desired success rate P. This makes the selection o optimal parameters very easy (see [2] or more details): ln(1 P ) l =, m = M ln(1 P ), and t = 2 l ln(1 M ln )l N M ln(1 P ). 3.3 Perormance o the TradeO Having deined the optimal coniguration o the tradeo, we now calculate the exact amount o work required during the online phase. The simplicity o rainbow tables makes it possible to include the work due to alse alarms both or the average and the worst case. Cryptanalysis with a set o rainbow tables is done by searching or the key in the last column o each table and then searching sequentially through previous columns o all tables. There are thus a maximum o lt searches. We calculate the expectation o the cryptanalysis eort by calculating the probability o success and the amount o work or each search k. When searching a key at position c o a table, the amount o work to generate a chain that goes to the end o the table is t c. The additional amount o work due to a possible alse alarm is c
7 since the chain has to be regenerated rom the start to c in order to rule out the alse alarm. The probability o success in the search k is given below: p k = m ( 1 m ) k 1. (2) N N We now compute the probability o a alse alarm during the search k. When we generate a chain rom a given ciphertext and lookup the end o the chain in the table, we can either not ind a matching end, ind the end o the correct chain or ind an end that leads to a alse alarm. Thus we can write that the probability o a alse alarm is equal to one minus the probability o actually inding the key minus the probability o inding no end point. The probability o not inding an end point is the probability that all points that we generate are not part o the chains that lead into the end points. At column i, these are the m i chains that we used to build the table. The probability o a alse alarm at search k (i.e., in column c = t k l ) is thus the ollowing: q c = 1 m i=t N ( 1 m ) i N where c = t k l, mt = m, and m i 1 = N ln(1 m i N ). When the tables have exactly the maximum number o chains m max we ind a short closed orm or q c (see [2] or more details): q c = 1 m N The average cryptanalysis time is thus: T = k=lt i=c p k (W (t c 1) + Q(c)) l + (1 k=1 c=t k l (3) c(c + 1) (t + 1)(t + 2). (4) i=x i=t where W (x) = i and Q(x) = q i i. i=1 m N )lt (W (t) + Q(1)) l (5) The second term o (5) is the work that is being carried out every time no key is ound in the table while the irst term corresponds to the work that is being carried out during the search k. W represents the work needed to generate a chain until matching a end point. Q represents the work to rule out a alse alarm. We can rewrite (5) as ollows: T = = k=lt p k k=1 c=t k l k=lt p k k=1 c=t k l ( i=t c 1 i=1 i=x ) ( i=t i + q i i l + (1 m i=t ) i=t N )lt i + q i i l i=c ( (t c)(t c 1) 2 i=1 i=1 ) ( i=t + q i i l + (1 mn )lt i=c t(t 1) 2 ) i=t + q i i l i=1
8 We have run a ew experiments to illustrate T. The results are given in Table 1. Table 1. Calculated and measured perormance o rainbow tables N = , t = 10000, m = , l = 4 theory measured over 1000 experiments encryptions (average) encryptions (worst case) number o alse alarms (average) number o alse alarms (worst case) Checkpoints in Rainbow Tables From results o Section 3.3, we establish below the gain brought by the checkpoints. We irstly consider only one checkpoint α. Let Y 1... Y s be a chain generated rom a given ciphertext C. From (3), we know that the probability that Y 1... Y s merges with a stored chain is q t s. The expected work due to a alse alarm is thereore q t s (t s). We now compute the probability that the checkpoint detects the alse alarm. I the merge occurs beore the checkpoint (Fig. 2) then the alse alarm cannot be detected. I the chain is long enough, i.e., α + s > t, the merge occurs ater the checkpoint (Fig. 1) with probability q α. In this case, the alse alarm is detected with probability Pr{G(X j,α ) G(Y α+s t ) X j,α Y α+s t }. We deine g α (s) as ollows: 0 i there is no checkpoint in column α, g α (s) = 0 i (α + s) t, i.e. the generated chain does not reach column α, Pr{G(X j,α ) G(Y α+s t ) X j,α Y α+s t } otherwise. i=t We can now rewrite Q(x) = i (q i q α g α (t i)). i=x We applied our checkpoint technique with N = , t = 10000, m = , l = 4 and G as deined in Section 2.2. Both theoretical and experimental results are plotted on Fig. 3. We can generalize to t checkpoints. We can rewrite Q(x) as ollows: i=t Q(x) = i q i q i g i (t i) i=x j=t j=i+1 ( q j g j (t j) k))) (1 g k (t. k=j 1 k=i
9 theory 0.08 gain experiment position o the checkpoint Fig. 3. Theoretical and experimental gain when one checkpoint is used We now deine memory cost and time gain. Let M, T, N and M, T, N be the parameters o two tradeos respectively. We deine σ M and σ T as ollows: M = σ M M and T = σ T T. The memory cost o the second tradeo over the irst one is straightorwardly deined by (σ M 1) = M /M 1 and the time gain is (1 σ T ) = 1 T /T. When a tradeo stores more chains, it implies a memory cost. Given that T N 2 /M 2 the time gain is: (1 T T ) = 1 1 σm 2. Instead o storing additional chains, the memory cost can be used to store checkpoints. Thus, given a memory cost, we can compare the time gains when the additional memory is used to store chains and when it is used to store checkpoints. Numerical results are given in Table 2. The numerical results are amazing. An additional 0.89% o memory saves about 10.99% o cryptanalysis time. This is six times more than the 1.76% o gain that would be obtained by using the same amount o memory to store additional chains. Our checkpoints thus perorm much better than the basic tradeo. As we add more and more checkpoints, the gain per checkpoint decreases. In our example it is well worth to use 6 bits o checkpoint values (5.35% o additional memory) per chain to obtain a gain o 32.04%. The 0.89% o memory per checkpoint are calculated by assuming that the start and the end o the chains are stored in 56 bits each, as our example uses DES keys. As we explain in Section 5 the amount o bits used to store chain can be optimized and reduced to 49 bits in our example. In this case a bit o checkpoint data adds 2% o memory and it is still well worth using three checkpoints o one bit each to save 23% o work.
10 Table 2. Cost and gain o using checkpoint in password cracking, with N = , t = 10000, m = , and l = 4 Number o checkpoints Cost (memory) 0.89% 1.78% 2.67% 3.57% 4.46% 5.35% Gain (time) storing chains 1.76% 3.47% 5.14% 6.77% 8.36% 9.91% Gain (time) storing checkpoints 10.99% 18.03% 23.01% 26.76% 29.70% 32.04% Optimal checkpoints ± 5 ± 5 ± 5 ± 5 ± 50 ± Characterization and Comparison o TradeOs In this section we give a generic way o characterizing the dierent variants o the tradeo. We calculate the characteristic o rainbow tables exactly and compare it to measured characteristics o other variants. 4.1 TimeMemory Graphs Knowing how to calculate the success rate and the number o operations needed to invert a unction, we can now set out to plot the timememory graphs. In order to do so, we ix a given success rate and or each memory size we ind the table coniguration that yields the astest tradeo and plot the time that it takes. The graphs show that cryptanalysis time decreases with the square o the memory size, independently o the success rate. We can thus write the timememory relation as T = N 2 γ(p ) (6) M 2 where γ(p ) is a actor that depends only on the success probability. It is interesting to note that or P = 86% which is the the maximum success probability o a single rainbow table, the actor is equal to 1. In that case we ind the typical tradeo which was already described by Hellman, namely that M = T = N 2 3. Note that this simple expression o the tradeo perormance was not possible or the previous variants. In those cases, calculations were always based on nonperect tables, on the worst case (the key is not ound in any table) and ignoring the amount o work due to alse alarms. Optimizations have been proposed with these limitations, but to our knowledge the actual average amount o work, including alse alarms has never been used to ind optimal parameter. Our simple ormula allows or a very simple calculation o the optimal parameters when any two o the success rate, the inversion time or the memory are given.
11 % 20% 50% 86% 90% 99% 99.9% T/N e05 1e06 1e07 1e08 1e06 1e M/N Fig. 4. TimeMemory graphs or rainbow tables, with various success rates.for P rainbow = 86% the graph ollows exactly T = N 2 /M The TimeMemory Characteristic The previous section conirms that rainbow tables ollow the same T N 2 /M 2 relation as other variants o the tradeo. Still, they seem to perorm better. We thus need a criterion to compare the tradeos. We propose to use γ(p ) as the tradeo characteristic. The evolution o γ over a range o P shows how a variant is better than another. Figure 5 shows a plot o γ(p ) or rainbow tables: In the ollowing sections, we compare the perormance o rainbow tables with the perormance o classic tables and DP tables. DP tables are much harder to analyze because o the variable length o the chains. We will thus concentrate on classic tables irst. 4.3 Classic and DP Tables The tradeo using classic or DP tables can also be characterized using the γ actor. Indeed both tradeos ollow the T N 2 /M 2 relation in a large part o the parameter space up to a actor which depends o the success rate and the type o tradeo. We have irst devised a strategy to generate the largest possible perect tables or each variant o the tradeo and have then used as many tables as necessary to reach a given success rate. The details o this work and the resulting timememory graphs are available in [2]. In Figure 6 we show the evolution o the tradeo characteristic o classic tables and o DP tables. The experiments and analysis show that rainbow tables outperorm classic tables and DP tables or success rates above 80%. Below this limit, perect classic tables are slightly better than perect rainbow tables in terms o hash operations needed or cryptanalysis. However, the price o using classic tables is that they need t times more table lookups. Since these do not come or ree in most architectures (content addressable memory could be an exception), rainbow tables seem to be the best option in any case.
12 0 12 rainbow 14 rainbow classic dp (p) 6 (p) P 1P Fig. 5. The TimeMemory characteristic o rainbow tables.steps happen every time an additional table has to be used to achieve the given success probability. Fig. 6. The TimeMemory characteristics o rainbow, classic and DP tables compared. 5 Implementation Tips For the sake o completeness we want to add some short remarks on the optimized implementation o the tradeos. Indeed, an optimized implementation can yield perormance gains almost as important as the algorithmic optimizations. We limit our tips to the implementation o rainbow tables. 5.1 Storing the Chain End Points The number o operations o the tradeo decreases with the square o the available memory. Since available memory is measured in bytes and not in number o chains, it is important to choose an eicient ormat or storing the chains. A irst issue is whether to use inputs or outputs o the unction to be inverted (keys or ciphertexts) as beginning and end o chains. In practice the keys are usually smaller than the ciphertexts. It is thus more eicient to store keys (the key at the end o the chain has no real unction but the extra reduction needed to generate it rom the last ciphertext is well worth the saved memory). A second and more important issue is that we can take advantage o the way the tables are organized. Indeed a table consists o pairs o beginnings and ends o chains. To acilitate lookups the chains are sorted by increasing values o the chain ends. Since the ends are sorted, successive ends oten have an identical preix. As suggested in [3] we can thus remove a certain length o preix and replace it by an index table that indicates where every preix starts in the table. In our Windows password example, there are about 2 37 keys o 56 bits. Instead o storing the 56 bits, we store a 37 bit index. From this index we take 21 bits as preix and store only the last 16 bits in memory. We also store a table with 2 21 entries that point to the corresponding suixes or each possible preix.
13 5.2 Storing the Chain Starting Points The set o keys used or generating all the chains is usually smaller that the total set o keys. Since rainbow tables allow us to choose the starting points, we can use keys with increasing value o their index. In our example we used about 300 million starting points. This value can be expressed in 29 bits, so we only need to store the 29 lower bits o the index. The total amount o memory needed to store a chain is thus bits or the start and the end. The table that relates the preixes to the suixes incurs about 3.5 bits per chain. Altogether we thus need 49 bits per chain. A simple implementation that stores the ull 56 bits o the start and end chain would need 2.25 times more memory and be 5 times slower. 5.3 Storing the Checkpoints For reasons o eiciency o memory access it may in some implementations be more eicient to store the start and the end o a chain (that is, its suix) in multiples o 8 bits. I the size o some parameters does not exactly match the size o the memory units, the spare bits can be used to store checkpoints or ree. In our case, the 29 bits o the chain start are stored in a 32 bit word, leaving 3 bits available or checkpoints. 6 Conclusion We have introduced a new optimization or cryptanalytic timememory tradeos which perorms much better than the usual T N 2 /M 2. Our method works by reducing the work due to alse alarms. Since this work is only a part o the total work our method can not reduce the work indeinitely. Besides having better perormance, checkpoints can be generated almost or ree while generating the tradeo tables. There is thus no indication or not using checkpoints and we conjecture that they will be used in many uture implementations o the tradeo. Also, checkpoints are a new concept in timememory tradeos and they may lead to urther optimizations and applications. In order to analyze the gain due to checkpoints we have presented a complete analysis o the rainbow tables. Using this analysis we are able to predict the gain that can be achieved with checkpoints. Finally we have also presented a simple way o comparing the existing variants o the tradeo with a socalled tradeo characteristic. We have calculated this characteristic or rainbow tables and measured it or the other variants. The results show that rainbow tables outperorm the other variants in all cases except when table lookups are ree and the success probability is below 80%. The act that the cryptanalysis time decreases with the square o the number o elements stored in memory indicates that it is very important to reduce the memory usage. This is why we have shared our tips on how this can be achieved in practice.
14 Reerences 1. Gildas Avoine, Etienne Dysli, and Philippe Oechslin. Reducing time complexity in RFID systems. In Selected Areas in Cryptography SAC 2005, Lecture Notes in Computer Science, Kingston, Canada, August SpringerVerlag. 2. Gildas Avoine, Pascal Junod, and Philippe Oechslin. Timememory tradeos: False alarms detection using checkpoints. Technical Report LASECREPORT , Swiss Federal Institute o Technology (EPFL), Security and Cryptography Laboratory (LASEC), Lausanne, Switzerland, September Alex Biryukov, Adi Shamir, and David Wagner. Real time cryptanalysis o A5/1 on a PC. In Fast Sotware Encryption FSE 00, volume 1978 o Lecture Notes in Computer Science, pages 1 18, New York, USA, April SpringerVerlag. 4. Johan Borst, Bart Preneel, and Joos Vandewalle. On the timememory tradeo between exhaustive key search and table precomputation. In Symposium on Inormation Theory in the Benelux, pages , Veldhoven, The Netherlands, May Dorothy Denning. Cryptography and Data Security, page 100. AddisonWesley, Boston, Massachusetts, USA, June Amos Fiat and Moni Naor. Rigorous time/space tradeos or inverting unctions. In ACM Symposium on Theory o Computing STOC 91, pages , New Orleans, Louisiana, USA, May ACM, ACM Press. 7. Amos Fiat and Moni Naor. Rigorous time/space tradeos or inverting unctions. SIAM Journal on Computing, 29(3): , December Martin Hellman. A cryptanalytic timememory trade o. IEEE Transactions on Inormation Theory, IT26(4): , July Iljun Kim and Tsutomu Matsumoto. Achieving higher success probability in timememory tradeo cryptanalysis without increasing memory size. IEICE Transactions on Communications/Electronics/Inormation and Systems, E82A(1):123, January Koji Kusuda and Tsutomu Matsumoto. Optimization o timememory tradeo cryptanalysis and its application to DES, FEAL32, and Skipjack. IEICE Transactions on Fundamentals, E79A(1):35 48, January Nele Mentens, Lejla Batina, Bart Preneel, and Ingrid Verbauwhede. Cracking Unix passwords using FPGA platorms. SHARCS  Special Purpose Hardware or Attacking Cryptographic Systems, February Philippe Oechslin. Making a aster cryptanalytic timememory tradeo. In Advances in Cryptology CRYPTO 03, volume 2729 o Lecture Notes in Computer Science, pages , Santa Barbara, Caliornia, USA, August IACR, SpringerVerlag. 13. JeanJacques Quisquater and JeanPaul Delescaille. How easy is collision search? Application to DES (extended summary). In Advances in Cryptology EURO CRYPT 89, volume 434 o Lecture Notes in Computer Science, pages , Houthalen, Belgium, April IACR, SpringerVerlag. 14. FrançoisXavier Standaert, Gael Rouvroy, JeanJacques Quisquater, and Jean Didier Legat. A timememory tradeo using distinguished points: New analysis & FPGA results. In Workshop on Cryptographic Hardware and Embedded Systems CHES 2002, volume 2523 o Lecture Notes in Computer Science, pages , Redwood Shores, Caliornia, USA, August SpringerVerlag. 15. Michael Wiener and Paul van Oorschot. Parallel collision search with cryptanalytic applications. Journal o Cryptology, 12(1):1 28, March 1999.
Cuckoo Filter: Practically Better Than Bloom
Cuckoo Filter: Practically Better Than Bloom Bin Fan, David G. Andersen, Michael Kaminsky, Michael D. Mitzenmacher Carnegie Mellon University, Intel Labs, Harvard University {binfan,dga}@cs.cmu.edu, michael.e.kaminsky@intel.com,
More informationA Tutorial on Physical Security and SideChannel Attacks
A Tutorial on Physical Security and SideChannel Attacks François Koeune 12 and FrançoisXavier Standaert 1 1 UCL Crypto Group Place du Levant, 3. 1348 LouvainlaNeuve, Belgium fstandae@dice.ucl.ac.be
More informationWhat s the Difference? Efficient Set Reconciliation without Prior Context
hat s the Difference? Efficient Set Reconciliation without Prior Context David Eppstein Michael T. Goodrich Frank Uyeda George arghese,3 U.C. Irvine U.C. San Diego 3 ahoo! Research ABSTRACT e describe
More informationLogical Cryptanalysis as a SAT Problem
Journal of Automated Reasoning 24: 165 203, 2000. 2000 Kluwer Academic Publishers. Printed in the Netherlands. 165 Logical Cryptanalysis as a SAT Problem Encoding and Analysis of the U.S. Data Encryption
More informationClean Answers over Dirty Databases: A Probabilistic Approach
Clean Answers over Dirty Databases: A Probabilistic Approach Periklis Andritsos University of Trento periklis@dit.unitn.it Ariel Fuxman University of Toronto afuxman@cs.toronto.edu Renée J. Miller University
More informationMining Data Streams. Chapter 4. 4.1 The Stream Data Model
Chapter 4 Mining Data Streams Most of the algorithms described in this book assume that we are mining a database. That is, all our data is available when and if we want it. In this chapter, we shall make
More informationOPRE 6201 : 2. Simplex Method
OPRE 6201 : 2. Simplex Method 1 The Graphical Method: An Example Consider the following linear program: Max 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2
More informationLoad Balancing and Rebalancing on Web Based Environment. Yu Zhang
Load Balancing and Rebalancing on Web Based Environment Yu Zhang This report is submitted as partial fulfilment of the requirements for the Honours Programme of the School of Computer Science and Software
More informationApproximately Detecting Duplicates for Streaming Data using Stable Bloom Filters
Approximately Detecting Duplicates for Streaming Data using Stable Bloom Filters Fan Deng University of Alberta fandeng@cs.ualberta.ca Davood Rafiei University of Alberta drafiei@cs.ualberta.ca ABSTRACT
More informationAnalysis of Nonfortuitous Predictive States of the RC4 Keystream Generator
Analysis of Nonfortuitous Predictive States of the RC4 Keystream Generator Souradyuti Paul and Bart Preneel Katholieke Universiteit Leuven, Dept. ESAT/COSIC, Kasteelpark Arenberg 10, B 3001 LeuvenHeverlee,
More informationSide channels in cloud services, the case of deduplication in cloud storage
Side channels in cloud services, the case of deduplication in cloud storage Danny Harnik IBM Haifa Research Lab dannyh@il.ibm.com Benny Pinkas Bar Ilan University benny@pinkas.net Alexandra ShulmanPeleg
More informationGeneralized compact knapsacks, cyclic lattices, and efficient oneway functions
Generalized compact knapsacks, cyclic lattices, and efficient oneway functions Daniele Micciancio University of California, San Diego 9500 Gilman Drive La Jolla, CA 920930404, USA daniele@cs.ucsd.edu
More informationOn Collisions for MD5
Eindhoven University of Technology Department of Mathematics and Computing Science MASTER S THESIS On Collisions for MD5 By M.M.J. Stevens Supervisor: Prof. dr. ir. H.C.A. van Tilborg Advisors: Dr. B.M.M.
More informationHow to Use Expert Advice
NICOLÒ CESABIANCHI Università di Milano, Milan, Italy YOAV FREUND AT&T Labs, Florham Park, New Jersey DAVID HAUSSLER AND DAVID P. HELMBOLD University of California, Santa Cruz, Santa Cruz, California
More informationEfficient ExternalMemory Bisimulation on DAGs
Efficient ExternalMemory Bisimulation on DAGs Jelle Hellings Hasselt University and transnational University of Limburg Belgium jelle.hellings@uhasselt.be George H.L. Fletcher Eindhoven University of
More informationPacket Classification for Core Routers: Is there an alternative to CAMs?
Packet Classification for Core Routers: Is there an alternative to CAMs? Florin Baboescu, Sumeet Singh, George Varghese Abstract A classifier consists of a set of rules for classifying packets based on
More informationThe Predecessor Attack: An Analysis of a Threat to Anonymous Communications Systems
The Predecessor Attack: An Analysis of a Threat to Anonymous Communications Systems MATTHEW K. WRIGHT, MICAH ADLER, and BRIAN NEIL LEVINE University of Massachusetts Amherst and CLAY SHIELDS Georgetown
More informationDesign and Evaluation of a New Approach to RAID0 Scaling
11 1 2 3 Design and Evaluation of a New Approach to RAID0 Scaling GUANGYAN ZHANG and WEIMIN ZHENG, Tsinghua University KEQIN LI, State University of New York and Tsinghua University 4 5 6 7 8 9 10 11
More informationCLoud Computing is the long dreamed vision of
1 Enabling Secure and Efficient Ranked Keyword Search over Outsourced Cloud Data Cong Wang, Student Member, IEEE, Ning Cao, Student Member, IEEE, Kui Ren, Senior Member, IEEE, Wenjing Lou, Senior Member,
More informationA new probabilistic public key algorithm based on elliptic logarithms
A new probabilistic public key algorithm based on elliptic logarithms Afonso Comba de Araujo Neto, Raul Fernando Weber 1 Instituto de Informática Universidade Federal do Rio Grande do Sul (UFRGS) Caixa
More informationRobust Set Reconciliation
Robust Set Reconciliation Di Chen 1 Christian Konrad 2 Ke Yi 1 Wei Yu 3 Qin Zhang 4 1 Hong Kong University of Science and Technology, Hong Kong, China 2 Reykjavik University, Reykjavik, Iceland 3 Aarhus
More informationThe Steganographic File System
The Steganographic File System Ross Anderson 1, Roger Needham 2, Adi Shamir 3 1 Cambridge University; rja14@cl.cam.ac.uk 2 Microsoft Research Ltd; needham@microsoft.com 3 Weizmann Institute; shamir@wisdom.weizmann.ac.il
More informationStreaming Similarity Search over one Billion Tweets using Parallel LocalitySensitive Hashing
Streaming Similarity Search over one Billion Tweets using Parallel LocalitySensitive Hashing Narayanan Sundaram, Aizana Turmukhametova, Nadathur Satish, Todd Mostak, Piotr Indyk, Samuel Madden and Pradeep
More informationCompetitive Algorithms for VWAP and Limit Order Trading
Competitive Algorithms for VWAP and Limit Order Trading Sham M. Kakade Computer and Information Science University of Pennsylvania kakade@linc.cis.upenn.edu Yishay Mansour Computer Science Tel Aviv University
More informationLONG SHORTTERM MEMORY Neural Computation 9(8):1735{1780, 1997 Sepp Hochreiter Fakultat fur Informatik Technische Universitat Munchen 80290 Munchen, Germany hochreit@informatik.tumuenchen.de http://www7.informatik.tumuenchen.de/~hochreit
More informationThe Disadvantages of Free MIX Routes and How to Overcome Them
The Disadvantages of Free MIX Routes and How to Overcome Them Oliver Berthold 1, Andreas Pfitzmann 1, and Ronny Standtke 2 1 Dresden University of Technology, Germany {ob2,pfitza}@inf.tudresden.de 2 Secunet,
More informationA Survey and Analysis of Solutions to the. Oblivious Memory Access Problem. Erin Elizabeth Chapman
A Survey and Analysis of Solutions to the Oblivious Memory Access Problem by Erin Elizabeth Chapman A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in
More informationEfficient Distributed Backup with Delta Compression
Efficient Distributed Backup with Delta Compression Randal C. Burns Darrell D. E. Long Department of Computer Science Department of Computer Science IBM Almaden Research Center University of California,
More informationCLOUD Computing has been envisioned as the nextgeneration
1 PrivacyPreserving Public Auditing for Secure Cloud Storage Cong Wang, Student Member, IEEE, Sherman S.M. Chow, Qian Wang, Student Member, IEEE, Kui Ren, Member, IEEE, and Wenjing Lou, Member, IEEE Abstract
More informationLightweight Block Ciphers Revisited: Cryptanalysis of Reduced Round PRESENT and HIGHT
Lightweight Block Ciphers Revisited: Cryptanalysis of Reduced Round PRESENT and HIGHT Onur Özen1, Kerem Varıcı 2, Cihangir Tezcan 3, and Çelebi Kocair 4 1 EPFL IC LACAL Station 14. CH1015 Lausanne, Switzerland
More information