Comparing Trade-off Based Models of the Internet

Transcription

1 Fundamenta Informaticae 92 (2009) DOI /FI IOS Press Comparing Trade-off Based Models of the Internet Anthony Spatharis, Ilias Foudalis, Martha Sideri Computer Science Department Athens University Of Economics And Business Patision 76, Athens, Greece Christos Papadimitriou Computer Science Department University of California Berkeley, USA Abstract. We introduce and evaluate several new models of network growth. Our models are extensions of the FKP model, modifying and improving it in various dimensions. In all these models nodes arrive one by one, and each node is connected to previous nodes by optimizing a trade-off between a geometric objective ( last mile cost ) and a topological objective ( position in the network ). Our new models differ from the original FKP model in directions inspired by the real Internet: two or more edges are attached to each arriving node (while the FKP model produces a tree); these edges are chosen according to various criteria such as robustness; edges may be added to the network between old nodes; or only certain fertile nodes (an attribute that changes dynamically) are capable of attracting new edges. We evaluate these models, and compare them with the graph of the Internet s autonomous systems, with respect to a suite of many test parameters (such as average degree, power law exponent, and local clustering rank) proposed in the literature; to this end we have developed the network generation and measurement system Pandora. Keywords: Internet topology, power law, scale-free graphs, network generators, network modeling and simulation, complex networks Address for correspondence: Departmrnt of Computer Science, Athens University of Economics and Business, Patision 76, Athens, Atiki, Greece

2 364 A. Spatharis et al. / Comparing Trade-off Based Models of the Internet 1. Introduction The Internet is a worldwide, publicly accessible collection of many thousands of interconnected computer networks; it is also the platform of a significant and increasing part of humanity s economic and social activity. Understanding the structure of the Internet graph (the graph of the routers, or that of the 20,000 or so autonomous systems comprising the Internet),is a major research frontier in computer science. In particular, it is of great importance to develop realistic generative models of Internet-like graphs, since proposed new protocols and algorithms need to be tested on models that are as realistic as possible. In this regard, the observation, first made in [14], that the degrees of the Internet graph are power law distributed (instead of the normal distribution predicted by traditional random graph models) brought about a revolution in the area, ushering in sophisticated graph models intended to model Internet growth. Indeed, starting with the preferential attachment model [3], many such models have been proposed; see [1, 6, 11, 16, 17] for some surveys, and see [5, 7, 10, 12, 18, 21] for several reviews and bibliographies on scale-free networks from different perspectives. One of the most influential models of network growth is the so called FKP model proposed in [13]. In this model, nodes arrive in discrete times, one after the other, and each node is connected to the node that minimizes a linear combination of two objectives; euclidean distance, and topological (hop) distance from the center of the network (assumed to be the first node). While this model has been shown in [13] to produce power law distributed degrees for large areas of the trade-off, it has also several shortcomings. For example, the produced network is a tree, and in fact one with a huge number of leaves; as a consequence the average degree is two (much less than the average degree of the real Internet). See [4] for a comprehensive critique of the FKP model. In this paper we extend the FKP model in several novel directions, addressing these and other shortcomings. The models proposed differ from the original FKP model in various respects. In all but one of the models each new node is connected to the network via not one but several edges, chosen so to optimize the original trade-off; as a result, the network is no longer a tree. How these edges are chosen among those that have good trade-off values defines several variants of our model. One may insist that the few best edges are added; or that two edges that are, in some precise sense, far from one another. In some models an edge may be added, with some small probability, between nodes of the network other than the one that has just arrived; this reflects real-life situation in which new connections force an already existing node to increase its connectivity to the rest of the Internet. Finally trying to model another real-life phenomenon, we designate some nodes as fertile, an attribute which changes dynamically. New nodes can only connect to nodes that are presently fertile. In a recent authoritative survey of the area [17] it is pointed out that, when it comes to models of the Internet, there is a trade-off between simplicity and realism; that is, how accurately the model reflects properties of the real Internet, and how complicated and unintuitive does it have to be to achieve such closeness. We believe that the models presented here occupy a very attractive region in this trade-off. Importantly, we test these new models experimentally by calculating several key metrics proposed in the literature, such as average degree, power-law degree, clustering coefficient, etc. We use these metrics to compare the various models, one against the other as well as each with previously proposed models, and with the real Internet. This comprehensive testing methodology, as well as the experimental system Pandora for performing these tests that we have created and made available on the Internet, is another contribution of this paper. Our experiments show that these models produce graphs that are in excellent agreement with the Internet with respect to the test parameters.

3 A. Spatharis et al. / Comparing Trade-off Based Models of the Internet 365 The rest of the paper is organized as follows. Section two provides a brief review of the background and related work. In section three the new models are presented and discussed. In section four we present our experimental results, measurements, and comparisons, and we briefly describe the Pandora system. Finally, in section five we discuss our findings and chart future work. 2. Background and Related Work The oldest and most influential model of random graphs is the 1959 G[n,p] model of Erdős and Renyi, in which edges among n nodes are selected independently with probability p; the degrees are thus concentrated by a Gaussian distribution, according to the central limit theorem. However, it was observed in [14] that the degree distribution of the Internet topology exhibits a power law degree distribution: P(k > x) x γ, (1) where γ is the shape parameter or tail exponent. It was later pointed out that, in addition, Internet graphs are highly clustered, among other properties. Many evolutionary models (in which nodes arrive one by one and are somehow connected to the already existing nodes) have been proposed to explain these observations. Of these, the preferential attachment models assume that the arriving node is connected to old ones with probability proportional to their degrees [1, 3]. In the trade-off based models [2, 9, 13] the connections are such that a multi-objective criterion is optimized. Both of these families of models have been shown to generate power law degree distributions, but are lacking with respect to other criteria. We next focus on the FKP model. The original trade-off model FKP [13] is a simple evolutionary model. Each node i (after the first one) is a uniformly random point in the unit square, and attaches itself to the node j that minimizes this weighted sum of two objectives: min j<i (α d ij + ecc(j)). (2) The first term is the Euclidean distance between the two nodes (the last mile cost ), while the second is the eccentricity of j, its distance from the center (assumed for simplicity to be the first node). α is a weight, capturing the relative importance of the two objectives. It is proved in [13] that, for all but extremely small and extremely large values of this parameter, the degree sequence of the resulting graph is power law distributed [4, 13]. The conceptual advantage of the FKP model over, say, the preferential attachment model is that attachment preference is based on a plausible network-motivated objective, as opposed to being postulated. 3. The Network Models There are many reasons why, despite its initial promise, the original FKP model fails to be a good model of the Internet Topology. Since each node attaches itself to the network by a single edge, the resulting graph is a tree; to put it otherwise, the average degree of each node is two, whereas the average degree of the Internet AS graph is 4.6.

4 366 A. Spatharis et al. / Comparing Trade-off Based Models of the Internet 3.1. Best Two (BT) model In the simplest and most basic modification of FKP called Best Two, when a new node i arrives, instead of selecting one node j that minimizes (2), the two best such nodes are chosen, and are connected to node i. This immediately gives the graph a nontree structure and increases the average degree to four, much closer to the Internet s 4.6. One disadvantage of this method is that the two nodes that optimize (2) may be very close, resulting in connections and paths of low diversity. Figure 1. to be 2. Graph generated by the best two algorithm. In all figures, node labels are order of arrival; α is asssumed For example, in Figure 1 when the new node 5 arrives, the FKP criterion first chooses node 2, while the second node chosen will be node Independent path (IP) model In this model when the new node i arrives, first is linked to a node j according to (2). A second edge is attached from the new node i to another node k j that minimizes the following quantity: min k<i,k j (α d ik + Com(j,k)), (3) where Com(j,k) is the number of common nodes of the two shortest paths from j and k to the center. In other words, this model favors second connections that are different from the first. Figure 2. Graph generated by the independent path algorithm.

5 A. Spatharis et al. / Comparing Trade-off Based Models of the Internet 367 Let us see what happens when node 5 enters the graph in Figure 2. The first node selected is node 2. But in contrast with the BT model, the second node selected is node Enhanced models In the real Internet, connections are not created only when a new node arrives. As old nodes acquire more connections and attract more traffic they may seek new upstream connections. This is captured by adding to the previous models the following additional step: After the new node has been connected to the two previous nodes, one of these two previous nodes, say j, is connected, with probability p, to another existing node, called the peering node. The peering node is the one which minimizes (2) for. By adding this feature to the Best of Two and the Independent Path models, one obtains the Enhanced Best Two (EBT) and Enhanced Independent Path (EIP). When p = 0 the non-enhanced models are obtained, while for p = 0.3 the average degree of the resulting graph is 4.6, the same as of that of the real Internet The controlled distance (CD) model In this new model, when a new node i arrives, it connects to an existing node j in the graph, that minimizes (2). A second edge connects node i to an existing node k that minimizes (2), subject to the constraint that the hop distance between nodes j and k is at least a fixed lower bound, such as 4. To implement this algorithm one needs a data structure that maintains the hop distance between any two nodes in the graph. Figure 3. Graph generated by the controlled distance algorithm. For example, in Figure 3 the second node selected by node 4 could be node 1 and not node 2 (as would happen in the BT model, for example), because node 1 has larger hop distance from the first connection, node The Fertile Node (FN) model The graph of the real Internet has a low power law exponent, and only 26% of its nodes as leaves. The FKP model also has a low power law exponent, but the great majority of its nodes are leaves. The Fertile Node (FN) model achieves a low exponent while at the same time maintaining a low number of leaves.

6 368 A. Spatharis et al. / Comparing Trade-off Based Models of the Internet Its motivation comes from the real Internet: Many autonomous systems are initially not available to new customers, but they may become as connectivity demand in their neighborhood grows. Let A 1 be an integer parameter. The FN algorithm maintains a set F of fertile nodes; initially, only the first node is in F. Each new node i > 0 is connnected to the fertile node optimizaing the trade-off: min j F (α d ij + ecc(j)). (4) Finally, if the chosen node j has more than A infertile children, then the infertile child of i that is closest to j becomes fertile and is inserted to F. Figure 4. Graph generated by the fertile node algorithm. In Figure 4 the value of A is two. Thus, when node 0 has more than two infertile children, the closest one (node 1) becomes fertile. 4. Comparison of the Models 4.1. The Pandora System There are many topology generators available. However, while implementing and analyzing the models in this paper, the authors realized that there was no single generator that could accommodate our models with reasonable effort. We therefore implemented our own network topology generator and analyzer, called Pandora, with these characteristics: 1. Large scale: Depending on the algorithm, the generator can produce graphs up to 80, 000 nodes.

7 A. Spatharis et al. / Comparing Trade-off Based Models of the Internet Extensible: Any researcher with a little knowledge on programming can write their own algorithms and metrics and add them to the generator. For example, once the user is accustomed to the features of the generator, the FKP algorithm can be implemented in 5 minutes. 3. Flexible: The generator can import files from inet, pandora and topgen formats and export data to otter and pajek formats. 4. User friendly: The simplicity of the design of the generator minimizes the time for a person to familiarize and use the generator. New users have started writing code within minutes. 5. Portable: Implemented in Java, the generator can run on any platform. For mode details, more experimental results, and for actually using and downloading Pandora, see [15] The Metrics We chose eight different metrics for evaluating and comparing our models; many of these metrics were proposed in [8]. 1. The first three parameters are related to the degree distribution. The average degree (or, equivalently, the number of edges) is the most basic one. 2. The degree exponent γ, found by regression analysis on a log-log plot of the degree distribution. The relative frequency of nodes of degree d is proportional to k γ. 3. The power law max degree is the natural cut-off of the degree probability density function, defined as n 1 γ 1, where n is the number of nodes. 4. Another important aspect of graph structure is the joint degree distribution, looking at the degree distribution among the neighbors of a node with a given degree. Two summary statistics of the joint degree distribution are the average neighbor degree and the assortativity coefficient. The average neighbor degree of a k 1 degree node is equal to: Dmax k 2 =1 k 2 P(k 2 k 1 ). (5) where P(k 2 k 1 ) is a conditional probability that given a node with degree k 1 will be connected with a node of degree k 2 and Dmax is the maximum degree of the graph. Max neighbor degree is the average neighbor degree for k 1 = Dmax. Average neighbor degree of the whole graph is the average of Eq.(5) for all the degrees. 5. The assortativity coefficient quantifies the connection between nodes of similar degree. If nodes of high degree tend to be connected with node of low degree then we say that the graph is dissasortative, if they connect to nodes of similar degree the graph si called assortative. Notice that the assortativity coefficient and the average neighbor degree are very close in meaning: Graphs

8 370 A. Spatharis et al. / Comparing Trade-off Based Models of the Internet with low average neighbor degree or is characterized as dissasortative, and is vulnerable to random failures and virus attacks. On the other hand such graphs are better at traffic monitoring and prevention of DoS attacks [19]. 6. We finally consider two key metrics related to clustering. Mean local clustering is the average value of LC(k), that is the average number of links between neighbors of k-degree nodes. 7. Finally, the clustering exponent is the exponent of the log-log plot of local clustering vs. degree Results Table 1 displays the results of calculating these metrics for the output of five of our models, and comparing them to the same metrics of the actual Internet graph obtained from [8]. We also compare our models with two classic models of internet topology, FKP and a variant of the classical BA model as produced by the inet 3.0 topology generator [22]. Each entry of the table represents the average of 15 runs. To compare the models effectively, the generated graphs have the same number n = 5556 of nodes as the Internet graph data in [8]. Table 1. Model Comparissons. BGP Tables FKP BA EBT CD IP FN EIP Reduced Number of edges Average degree Frequent exponent (γ) Power law max degree Assortativity coefficient Norm. average neighbor degree Norm. max neighbor degree Mean Local clustering Clustering exponent Networks with high average degree tend to be better connected. As it was mentioned before, the enhanced best two and independent path models achieve an average degree of 4.6, which is very close to the real Internet graph data. Since the degree distribution is maybe the most commonly used topology characteristic, in all our models we adjusted parameters so as to agree with the Internet graph on this metric. All our models produce a very good frequent exponent and maximum power law degree. As for the other metrics, the controlled distance, independent path and fertile node models tend to be much closer to the Internet data in terms of assortativity coefficient. Concerning neighbor degree metrics, we notice that most of our models are not close to the Internet with respect of the value of maximum neighbor degree but in terms of average neighbor degree achieve good results. The fertile model is closer to he Internet graph data than the FKP model without fertility according to all metrics, and achieves a very good frequent exponent.

9 A. Spatharis et al. / Comparing Trade-off Based Models of the Internet 371 Finally, the independent path model is the most accurate with respect to the clustering metrics; the fertile model, always generating a tree, is, of course, highly inaccurate in this regard. 5. Conclusions and Future Work The FKP model was a first attempt to take into account the complex and decentralized decision making involved in the evolution of the Internet; its power law behavior can be shown analytically, and its power law exponent is, empirically, quite accurate. On the negative side, the FKP model generates trees with no clustering and too many leaves. The extensions of the FKP model introduced and evaluated in this paper occupy a very interesting point in the accuracy-simplicity trade off, being quite simple and intuitive, while at the same time the producing graphs are rather Internet-like. Furthermore, the experimental methodology (generic and flexible topology generator and standardized metric calculation) used in this paper may be useful in future work. The models proposed are simple enough that some of the crucial metrics may be possible to calculate analytically. We have some preliminary results on the exponent of the fertile node model [20], but much work is needed to be done on this front. Finally, it would be interesting to incorporate in our models deletions of nodes and edges, certainly an element of the real Internet which, to our knowledge, has not been considered in the literature. References [1] Albert, R., Barabási, A.: Statistical Mechanics of Complex Networks, Reviews of Modern Physics, 74, 2002, 47. [2] Alvarez-Hamelin, J. I., Schabanel, N.: An Internet Graph Model Based on Trade-off Optimization, The European Physical Journal B - Condensed Matter and Complex Systems, 38, 2004, [3] Barabási, A., Albert, R.: Emergence of Scaling in Random Networks, SCIENCE: Science, 286, 1999, [4] Berger, N., Bollobás, B., Borgs, C., Chayes, J., Riordan, O.: Degree Distribution of the FKP Network Model, TCS: Theoretical Computer Science, 379, 2007, [5] Bollobás, B., Riordan, O.: Mathematical Results on Scale-free Random Graphs, in: Handbook of Graphs and Networks: From the Genome to the Internet, Bornholdt and Schuster, [6] Bonato, A.: A Survey of Models of the Web Graph, Workshop on Combinatorial and Algorithmic Aspects of Networking (CAAN), LNCS, 1, [7] Brandes, U., Erlebach, T., Eds.: Network Analysis: Methodological Foundations [outcome of a Dagstuhl seminar, April 2004], vol of Lecture Notes in Computer Science, Springer, [8] CAIDA: The Cooperative Association for Internet Data Analysis. In BGP: Internet Data Catalog: [9] Chen, Q., Chang, H., Govindan, R., Jamin, S., Shenker, S., Willinger, W.: The Origin of Power-Laws in Internet Topologies Revisited, INFOCOM, [10] Costa, L., Rodrigues, F., Travieso, G., Boas, A. V.: Characterization of complex networks: A survey of measurements, Advances in Physics, 56, 2005,

10 372 A. Spatharis et al. / Comparing Trade-off Based Models of the Internet [11] Dhamdhere, A., Dovrolis, C.: Ten years in the evolution of the internet ecosystem, Internet Measurement Conference (K. Papagiannaki, Z.-L. Zhang, Eds.), ACM, [12] Dorogovtsev, S., Mendes, J.: Evolution of Networks : from Biological Nets to the Internet and WWW, Oxford University Press, [13] Fabrikant, A., Koutsoupias, E., Papadimitriou, C.: Heuristically Optimized Trade-Offs: A New Paradigm for Power Laws in the Internet, ICALP: Annual International Colloquium on Automata, Languages and Programming, [14] Faloutsos, M., Faloutsos, P., Faloutsos, C.: On Power-Law Relationships of the Internet Topology, ACM SIGCOMM 99, [15] Foudalis, I.: Pandora Topology Generator, [16] Haddadi, H., Rio, M., Iannaccone, G., Moore, A., Mortier, R.: Network topologies: inference, modeling, and generation, Communications Surveys & Tutorials, IEEE, 10, 2008, [17] Krioukov, D., Chung, F., Claffy, K., Fomenkov, M., Vespignani, A., Willinger, W.: The Workshop on Internet Topology (WIT) Report, ACM SIGCOMM Computer Communication, 37(1), December , [18] Mitzenmacher, M., Siganos, G., Faloutsos, M., Faloutsos, P., Faloutsos, C.: Brief History of Generative Models for Power Law and Lognormal Distributions, Internet Math, 1, 2004, [19] Park, K., Lee, H.: On the effectiveness of route-based packet filtering for distributed DoS attack prevention in power-law internets, SIGCOMM, [20] Spatharis, A., Foudalis, I., Sideri, M.: Competition-Induced Tradeoff-based Modeling of the Internet, Technical report, Athens University Of Economics And Business, [21] Strogatz, S.: Exploring Complex Networks, NATURE: Nature, 410, 2001, [22] Winick, J., Jin, C., Chen, Q., Jamin, S.: Inet-3.0 Topology Generator,