Understanding Web Hosting Utility of Chinese ISPs

Transcription

1 Understanding Web Hosting Utility of Chinese ISPs Zhang Guanqun 1,2, Wang Hui 1,2, Yang Jiahai 1,2 1 The Network Research Center, Tsinghua University, 2 Tsinghua National Laboratory for Information Science and Technology (TNList) Beijing, China, PA zgq07@mails.tsinghua.edu.cn, hwang@cernet.edu.cn, yang@cernet.edu.cn Abstract. By the end of 2008, China has 298 millions of Internet users served by 214 ISPs. But little is known to researchers about the details of these ISPs. In this paper, we try to understand web hosting utility of Chinese ISPs. After a brief introduction to ISPs in China, we use a simple and general method to uncover the web hosting information of ISPs including websites and web pages they host. We present a metric to evaluate the web hosting utility based on a new model which relies on public available data. Finally, a ranking of Chinese ISPs is given according to their web hosting utility. We analyze this ranking from the point of view such as IP allocation and geographical features. We believe that results of this work are not only helpful for researchers to understand networks in China, but also beneficial to interdomain capacity planning in China. 1 Introduction The Internet in China is growing faster in the recent years. In 1997, there were only 620,000 Internet users according to the 1 st Statistical Survey Reports on the Internet Development in China [1] published by CNNIC (China Internet Network Information Center). At the end of 2008, the number of users raised to 298 millions [2] which is almost equal to one-quarter of the whole population in China. By the end of 2008, China had become the top one country in the number of Internet users in the world. Internet service providers (ISPs) play an important role in China Internet development. But we know little about the details of Chinese ISPs. By now, few research papers have been published on this topic due to ISP s privacy policy and huge difficulties in AS-level measurement. Most research works [3, 4] related to China Internet only introduced the status of Chinese ISPs briefly according to some published statistical reports. With some exception, Zhou [5] presented an AS-level topology of China Internet based on traceroute data probed from servers of major ISPs in China. Researches on all aspects of Chinese ISPs are very interesting and significant because information about Chinese ISPs is desired by not only researchers but also common Internet users. For example, web hosting utility information and user accessing utility information of one ISP are needed by Internet users to decide which

2 ISP to access. Companies also need this information to choose an ISP where their websites can be hosted with lower cost and fast access by their customers. In this paper, we try to uncover the web hosting utility of Chinese ISPs. Web hosting [6] is a type of Internet hosting service that allows individuals and organizations to provide their own websites accessible via the World Wide Web. In this paper, we assume ISPs which host a large amount of popular web contents are considered high utility in web hosting. Based on this idea, a new model is presented to quantify ISP s web hosting utility in Section 3. This new model relies only on public available measurements and flexible enough to incorporate ISP s private data. The quantified value is called WHU (i.e. Web Hosting Utility). Statistically, the WHU value of one ISP is proportional to the average amount of web traffic sending to outside Internet users within a period of time. This value is significant for inter-as traffic matrix estimation and peering relationship inference between ISPs. Besides the WHU value, the number of websites and web pages one ISP hosted can also be used to complementally describe the web hosting utility of that ISP. In order to avoid huge difficulties in data measurement, similar with paper [7], we use a measurement method to uncover the web hosting utility of ISPs based on search engines and hot search keywords. However, our method pay more attention to the actual Internet conditions in China, and it is easier to understand and realize. Two search engines are employed in our method: Baidu, the leading Chinese language search engine in China, and Google, the most famous search engine in the world. Hot keywords are consulted from search engines websites, and most of them are Chinese characters. With the results obtained from our experiment, we rank the Chinese ISPs according to the number of websites and web pages they hosted as well as the WHU value. We also analyze this ranking from the point of view such as IP allocation and geographical features. To our best of knowledge, this is the first report about the web hosting utility information of Chinese ISPs. We believe that the results of this work are not only helpful for researchers to understand networks in China, but also beneficial to interdomain capacity planning in China. The paper is organized as follows. In Section 2 we introduce the basic information about Chinese ISPs. In Section 3, we describe our new model and the measurement method used. The results of our experiment are presented in Section 4 followed by the result analysis in Section 5. We conclude our work in Section 6. 2 ISPs in China 2.1 AS Numbers Allocated to China APNIC provides IP resource allocation and registration services which support the operation of the Internet in Asia-Pacific area. According to our statistics of AS allocation related data published by APNIC [8], 433 AS numbers (ASNs for short) have been allocated to China. But 55 of 433 AS numbers are not in use at present. In addition, 33 of 433 ASNs are allocated to CNGI (China Next Generation Internet) project which is a set of interconnected IPv6 experimental networks. As CNGI is still

3 in experimental and research stage instead of a commercial Internet and few websites are built on it. For this reason, we will not take these 88 (55+33) ASes into account in inferring the web hosting utility information. 2.2 Status of Chinese ISPs The traditional description of an ISP is a company that offers its customers to access to the Internet. In this paper, we define ISP as an organization which has at least one AS number and provides at least one of three Internet service, including user access, lower tier ISP transit and web hosting. We match ASNs with their corresponding ISPs by collecting AS descriptions from APNIC WHOIS database. According to our statistics, there are 214 ISPs in China. This number is far beyond our imagination. With a little surprise, we find only 108 of 214 are companies which mainly make profit from providing Internet service. The rest 106 ISPs are allocated to broadcasting & TV companies, oil companies, manufacturing companies, sports companies and so on. Obviously, these 108 ISPs are expected to have high utility as web service providers. Unfortunately, little is known about the details of ISPs. The majority of Internet users in China are only familiar with the four most famous ISPs (so-called big four): China Telecom, China Mobile, China Unicom and China Education and Research Network (CERNET). They are all state-owned and the business scope of which cover the whole China. However, the rest of 210 ISPs are strange to us. So it is very interesting and significant to uncover the information of small ISPs in China. 2.3 Imbalanced Allocation At present, 214 ISPs in China totally have 378 ASNs in use. But this allocation is not balanced. Most of ASNs are allocated to several famous ISPs. For example, the big four totally have 112 ASNs which is nearly 30% of the total ASNs of China. In contrast, 86% of ISPs only have one AS number. Table 1 presents the number of ASNs and IP addresses allocated to big four. Table 1. AS numbers and IP address numbers allocated to big four ISPs # of AS Percentage (%) # of IP address [3] Percentage (%) China Telecom ,490, China Mobile ,331, China Unicom ,471, CERNET ,560, total ,854, With no doubt, the big four occupy most of AS and IP resources in China. This imbalanced resource allocation will certainly lead to an imbalance in web contents distribution of ISPs. What we concern is how this imbalance affects the web hosting utility of ISPs and whether this is the only influence factor. We will give answers to these questions in section 6.

4 3 Evaluation Model and Measurement Method 3.1 Model to Quantify Web Hosting Utility The number of websites and web pages can be used to describe the web hosting utility of one ISP, but it is infeasible to obtain the two numbers metrics and accurately. In addition, to evaluate the web hosting utility of ISP, it is better to use a single metric than two. Therefore, we define WHU as the only metric instead of websites or web pages. The WHU values of ISPs are derived from our new model described below. To our best knowledge, this is the first criterion to quantify the web hosting utility of ISPs accurately. The basic idea of our model is: ISPs that host a large amount of hot web contents are considered to have high utility in web hosting. To determine popular web contents on the Internet, we use search engines and hot keywords to help us. In our model, hot web contents are represented by search results returned by a search engine. Specifically, P( k, u ) is defined to denote a piece of hot web content determined by keyword k and URL u returned by a search engine. Before presenting the quantitative model, we first define a set of related notations. We use K to denote the keywords set collected. Nk ( ) is defined as the number of searches for keyword k. This value reflects the popularity of a keyword. We define Uk ( ) as a set of matched URLs returned by search engines when performing a query using keyword k. Cu ( ) denotes the clicked rate of the web page represented by URL u. This value reflects the popularity of a web page. Su ( ) is the size of web page represented by URL u. Wsa( u ) denotes the web server address of URL u. The hotness of web content P( k, u ) is defined by the following equation H( P( k, u)) N( k) C( u) where k K, u U( k) where H( P( k, u )) is the hotness of web content. We use W( P( k, u )) to denote the weighted size of web content P( k, u ). W( P( k, u)) H( P( k, u)) S( u) N( k) C( u) S( u) where k K, u U( k) (2) With the definition above, we define WHU ( x ) in equation (3) to denote the web hosting utility of ISP x. WHU ( x) W( P( k, u)) where Wsa( u) IP( x) (3) k K, u U ( k) IP( x ) is an IP address set allocated to ISP x. With equations (1) ~ (3), we summarize the steps taken to compute the web hosting utility WHU ( x ) for every ISP x in Algorithm 1. Algorithm 1. Computation of WHU ( x ) 1. initialize ( ) WHU x to 0 for every ISP x 2. for each keyword k in keywords set K (1)

5 3. for each URL u in URLs set Uk ( ) 4. whu = Nk ( ) Cu ( ) Su ( ) 5. extract web server address wsa from u 6. map wsa to ISP isp 7. WHU ( isp ) = WHU ( isp ) + whu 3.2 Measurement Method In order to compute the WHU value of ISPs described in Algorithm 1, we use a simple and general measurement method. There are four steps in our method: 1. Collecting hot keywords. Hot keywords and number of searches are collected from top.baidu.com, a website of Baidu. 2. Obtaining matched URLs. For each keyword collected in step 1, we query Google and Baidu to retrieve a set of most closely matched URLs. 3. Extracting web server address from URLs in step 2 by querying DNS servers. 4. Mapping URLs (web content) to ISP hosted according to web server address. We obtained the IP addresses allocated to each ISP in China according to APNIC data. Differently, Chang [7] achieved this mapping using BGP tables. Compared with Chang s method in [7], the measurement method we used in this paper is simpler and more feasible in China. Employing two search engines, we collected more matched URLs. This may help us to uncover small ISPs with fewer web contents. We also avoided difficulties in obtaining BGP routing tables. 3.3 Discussion There are five unknown parameters in Algorithm 1. Three of them can be straightly obtained from the measurement described above. These three parameters are: K, Nk ( ) and Uk ( ). The two parameters left are Su ( ) and Cu ( ). Su ( ) consists of two parts: the size of web page and the size of embedded objects. Embedded objects cannot be ignored as they contribute a large amount of web traffic. We first select ten famous video sharing websites which totally occupied over 90% market in China [9]. Then we crawl the keyword-retrieved URLs individually and extract the embedded objects from the web pages which belong to the top 10 websites. Cu ( ) represents the click rate of URL u. In general, search results with high rank are more frequently clicked. This is a user search behavior habit. We download a 30- day user query logs from the website of Sogou [10], a famous search engine in China. In order to find the relationship between clicked rate and rank of an URL, we plot a log-log graph using 30-day Sogou data in Figure 1(a). The horizontal axis represents the rank of URL, the vertical axis represents the log-value of clicked rate: the ratio of clicked times on that rank to the total clicked times. Figure 1(a) shows that the clicked rates of URLs on the first result page (10 results per-page default) are much higher than results on other pages. In Figure 1(b), we use two straight lines to fit the log-log relationship shown in Figure 1(a). Points in Figure 1(b) present the average values of 30-day data. For the two straight lines, we use

6 Clicked rate of URLs log(clicked rate of URL) equation y=ax+b, where x and y stand for the horizontal axis and vertical axis in Figure 1(b) respectively. Line 1 fits for the top 10 results while Line 2 fits for the rest 80 results. The fitting results are listed in Table 2 where parameters a, b stand for the slope and the ordinate respectively Page 1 Line 1 Page 2 ~ Page 9 Line Rank of URLs (a) Log-log relation graph (b) Curve fitting Fig. 1. Relationship between clicked rate and rank of an URL Table 2. Fitting results log(rank of URL) a b SSE R-square RMSE Line Line Cu ( ) is computed with two equations listed in Table 2. So far, all the inputs needed in Algorithm 1 can be derived from public data and public measurements which mean our model can work without any ISP s proprietary data Experiment & Results In our experiment, we totally collected 1158 hot keywords. These keywords cover most of areas including politics, economics, business, sports, entertainments, and et al. We selected top 50 matched URLs for each hot keyword from Baidu and Google, totally 115,800 URLs. With the measurement method described in section 3.3, we identified 73 different ISPs and 96 ASes in China. 4.1 Websites and Web Pages Hosted by ISPs In our experiment, we obtained websites and web pages distributed on 73 ISPs. Table 3 lists the top 10 ISPs by number of websites and web pages. From Table 3, we can see that the top 3 ISPs have huge advantages in number of websites and web pages compared with other ISPs. With no doubt, they are the top 3 ISPs providing web hosting service in China. Table 3. Top 10 ISPs by number of websites and web pages. Rank ISP # of Websites Rank ISP # of WebPages

7 1 China Telecom China Telecom China Unicom China Unicom CERNET CERNET China Networks Inter- Exchange China Abitcool China Mobile China Networks Inter- Exchange Beijing DianXinTong China Mobile China Abitcool Beijing DianXinTong ZhengZhou GIANT Beijing Net Infinity Alibaba Beijing Zhongguancun Beijing Jingxun Beijing Jingxun Web Hosing Utility of ISPs Using Algorithm 1, we quantified the web hosting utility for 73 ISPs in China we identified. Table 4 lists the top 5 and the bottom 5 ISPs as well as their WHU values. Table 4. Top 5 ISPs and bottom 5 in China according to their WHU values. Rank ISP WHU Value 1 China Unicom CERNET China Telecom China Abitcool China Networks Inter-Exchange China Ministry of Science and Technology Great Wall Broadband China State Post Bureau Shenzhen Information and Network Center China Cultural Heritage Information and Consulting Center From the ranking list, we can see that there is a huge distance between the top 1 and the bottom 1 ISP on web hosting utility. The top 5 ISPs are all networking companies which mainly make profit from providing Internet service. In contrast, 3 of the bottom 5 ISPs belong to government departments. Although China Telecom hosts more websites and web pages than China Unicom and CERNET shown in Table 3, it has lower WHU value than China Unicom and CERNET. In order to explain this puzzle, we list the top 10 websites in Table 5 ranked by total page size they host. We also show the ASNs and their ISPs that host the websites. Table 5. Top 10 websites in total page size Rank Website Type Page Size (KB) ASN ISP 1 tudou.com video sharing China Unicom 2 sina.com.cn general CERNET 3 youku.com video sharing China Abitcool 4 56.com video sharing China Unicom 5 sohu.com general CERNET 6 qq.com general CERNET 7 baidu.com search engine China Abitcool

8 8 ku6.com video sharing China Telecom 9 amazon.cn network shopping Beijing Net Infinity 10 6.cn video sharing China Telecom From Table 5 we can see that the top 10 websites belong to 5 different ISPs respectively. Three top general websites locate in CERNET and two big video sharing websites locate in China Unicom. In contrast, China Telecom only has two small video sharing websites. Some frequently-accessed websites deploy web servers in several ISPs to speed up user access. In this paper, the DNS server we used to extract web server IP addresses from URLs locate in CERNET. This may leads to the missing of some web server IP addresses for some large websites. 5 Discussions From the results listed in Table 3 and Table 4 we notice that the distribution of web content is imbalanced among ISPs in China. The reason for this imbalanced distribution may be various. In this section, we try to interpret the imbalance from the point of IP allocation and geographical features. 5.1 IP Allocation We draw the WHU values and the number of IP addresses of 73 ISPs in Figure 2. Here, the amount of IP is not the total IP addresses allocated to an ISP, because some ASs of an ISP do not host any website or they are not identified by our experiments. In other words, we only count in IP addresses allocated to 96 ASs. Though the number of IP addresses is not decreasing with WHU values proportionally, the correlation coefficient of two lines in Figure 2 is It means that IP allocation may be a possible cause for the imbalanced distribution. We define the density Dx ( ) of an ISP x as the following equation and draw the density curve of 73 ISPs in Figure 3. D( x) 1000 WHU ( x) / IP( x) (4) In Figure 3, some ISPs have extremely high density value such as ISPs ranked in 7 and 11. We call them the dense ISPs as they host larger amount of hot web contents with fewer IP addresses. We list the top 5 dense ISPs in Table 6. Table 6. Top 5 dense ISPs in China Rank WHU rank ISP Density 1 7 Beijing Zhongguancun ZhengZhou GIANT Alibaba China Abitcool Beijing Gu Xiang The top 1 ISP in Table 6 hosts an IT products information website And Alibaba hosts an online shopping website

9 WHU values or Number of IP addresses Density of ISP Nowadays, IT products and online shopping are warmly welcomed in China. These two websites contribute most of the WHU value of Zhongguancun and Alibaba. From this point of view, the WHU of an ISP is also related to the kind of websites it hosts WHU value Number of IP addresses Rank of ISP ISP Fig. 2. Number of IP address vs. WHU values Fig. 3 The density line 5.2 Geographical Features In China, regional economic development and Internet development are imbalanced [12]. Corresponding to this imbalance, China Telecom and China Unicom allocate ASNs for some economically developed areas to promote the Internet development in these areas. We classify AS numbers which belong to the above two ISPs with their geographical position. We try to find the effect of regional difference in imbalanced distribution of hot web contents. Table 7 lists the top 5 regions in China as well as ASNs they have. Table 7. The WHU value of different regions in China Rank Region ASN ISP WHU 1 Beijing 4808, 17620, China Unicom, China Telecom Guangdong 17623, 17816,17622 China Unicom Shanghai 17621,4812 China Unicom, China Telecom Hebei China Telecom Tianjin China Telecom From Table 7 we notice that the distribution of hot web contents is quite related to geographical features. Beijing, Guangdong and Shanghai are not only developed in economy but also strong in Internet construction. We list the statistical data on Internet and economy of the top 3 regions in Table 8. Popularity in Table 8 stands for penetration rate of Internet in each region. Numbers in column 4 and 5 is occupancy. Table 8. GDP data and Internet resource occupancy of the top 3 regions Region Per Capita GDP ( ) [9] Popularity [3] Domain Names [3] Websites [3] Beijing % 21.4% 12.9% Guangdong % 11.3% 15.0% Shanghai % 6.5% 6.2% Nation average % 3.3% 3.3%

10 6 Conclusions The detailed information of Chinese ISPs is significant not only for researchers but also for operators of individual ISPs. In this paper, we focus on the web hosting utility of ISPs. We present a new metric WHU to evaluate the web hosting utility of an ISP based on a simple and general measurement method. We rank Chinese ISPs according to the WHU values as well as number of websites and web pages. The problem of imbalanced distribution of hot web contents among ISPs is serious. The imbalance of IP allocation and regional Internet development in China are partial causes for this problem. In this paper, web server IP addresses for some large websites may miss because we only use the DNS servers located in CERNET. As part of future work, we try to add more measurement points in different ISPs to resolve this problem. References 1. The First Statistical Survey Reports on the Internet Development in China, 2. The 23 rd Statistical Survey Reports on the Internet Development in China, 3. Randolph, K., Chen, Y.: The Internet in China: A Meta-Review of Research. The Information Society. 21(4), (2005) 4. Jonathan, Z., Enhai, W.: Diffusion, Use, and Effect of the Internet in China. Communications of the ACM. 48(4), (2005) 5. Shi, Z., Guo, Z. Guo, Z.: Chinese Internet AS-level Topology. IET Communications. 1(2), (2007) 6. Borka, J.: Web Hosting Market Development Status and Its Value as an Indicator of a Country s E-Readiness. Telecommunications Policy. 32(6), (2008) 7. Hyunseok, C., Sugih, J., Morley, M., Walter, W.: An Empirical Approach to Modeling Inter-AS Traffic Matrices. In: IMC ACM Berkeley, USA (2005) 8. APNIC Reports and Statistics, 9. China Websites Ranking, Sogou User Query Logs, China Statistical Yearbook, He, L., Gui, L., Le, Q.: Spatial-Temporal Analysis of Regional Disparities of Internet in China. Chinese Geographical Science. 14(4), (2004)