The Availability of Commercial Storage Clouds

Transcription

1 The Availability of Commercial Storage Clouds Literature Study Introduction to e-science infrastructure Arjan Borst ccn Grid Computing - University of Amsterdam Software Engineer - WireITup A.Borst@wireitup.nl Roy Bakker ccn Grid Computing University of Amsterdam bakkerr@science.uva.nl October 23, 2008 Abstract The almost unlimited demand of storage capacity for all kinds of projects or applications results in more pervasive solutions. One of those solutions is offered by a new kind of technology called storage clouds. This very promising technique offers an unlimited scalability combined with an availability of 99.9% and you only have to pay for what you are using. In this research we did experiments on Amazon S3 located in the USA and Nirvanix located in Germany. To measure the uptime, connect- and download times we used more than 20 monitoring sites around the world. We compared our results with related work. We found unexpected differences with related research regarding availability. We give a good picture of connect and download times all around the world. 1 Introduction Massive storage solutions for e-science projects like the LHC 1 or VL-e 2 are getting more important then ever before. Continues supply of massive data sets makes it very expensive and difficult to build and maintain your own storage solution in two different ways. The first is the need of being available 24/7 which require maintenance, monitoring and having technical staff standby. The second is to have a scalable solution which results in a high investment. This paper presents a study about the availability offered by a new form of providing storage known as storage clouds. This new kind of internet service works on a pay-per-use base. This means no high investments, long term contracts, technical staff or any other related costs. You only have to pay for what you are using each month without concerns about scale and maintenance. For this research we selected two providers, Amazon and Nirvanix, which are both offering a SLA of 99.9% uptime and claiming to offer an unlimited amount of storage. Primarily we focus on the nonfunctional requirement availability, and therefore we selected three basic availability metrics in our experiment: uptime, connect time and download time. Combining our results with related research [7, 8, 4], we want to determine the current state of fulfilling the non-functional requirement availability by those two storage clouds. The reason for studying both connect- and download times is that we think it is also an essential part of measuring the state of availability. There are many applications which require a certain response or download time to be applicable. 2 Storage Clouds This new kind of on-demand storage has derived his name from the increasingly popular term cloud 1

2 2 3 EXPERIMENT SETUP computing [5]. Unfortunately there is no clear definition [6] and therefore understanding what cloud computing can do or should do. The same problem arise with the term storage cloud, the lack of clear definition and standardization makes it hard to compare and select between different providers. Because of the lack of clarity we defined two criteria which a storage cloud provider should meet to be chosen for this experiment. The first criterion is a SLA of at least 99.9% uptime and the second is the ability to offer a scalable solution. Out of many providers we selected Amazon S3 and Nirvanix. We selected S3 because at the moment of writing S3 is the most popular storage provider. Nirvanix is less famous but the features they offer are very promising and they offer a respectable SLA of 100% when using three different sites. Therefore they could be a very interesting partner for many projects. Of course there are many more providers which meet our criteria. At this moment there is not enough time to study them all. Both providers offer more than one location to store data. Because of time and funding constrains, we selected only one site at both providers. For Amazon S3 we used their site located in the USA and for Nirvanix their site in Germany. 2.1 Pricing and features The main difference between Nirvanix and Amazon S3 is the charge of the different requests. The charge of $ 0.01 for PUT, POST, LIST and GET can really make the difference for selecting a provider. This can be a showstopper when doing many GET (download) requests on files less than 10Kb in combination with high bandwidth usage. As shown in figure 2, this results in more costs than one would expect. At the time of [4], S3 charged for every 1,000 GET requests $ An overview of pricing can be found in figure 1. For a full set of features I refer to the home sites of both storage providers [2, 1]. In figure 3, a comparison of the most important features is shown. They both offer an API covered by good documentation. Also important is the ability to set file permissions. The most noticeable difference is the maximum files size of Nirvanix which can be a respectable 256 GB. Another major difference comparing Nirvanix with S3 is the lack of some basic file system operations, like change, copy and move files or folders. Figure 1: The pricing differences for Amazon S3 and Nirvanix. Cost Type Nirvanix S3 USA Storage (GB) $0.25 $0.15 Incoming (GB) $0.18 $0.10 Outgoing (GB) $0.18 $0.17 Requests (1,000 PUT, POST, LIST) $0.00 $0.01 Requests (10,000 GET) $0.00 $0.01 Figure 2: Pricing (USD/month) for Storage (1KB - 10MB/month) and outgoing traffic (GB/month). Figure 3: Nirvanix. Features provided by Amazon S3 and Feature Nirvanix S3 USA SLA 99.9% 99.9% Max file size 256GB 5GB API yes yes Per File ACL yes yes Full file system functionality yes no 3 Experiment Setup For this experiment we used a SLA monitoring tool [3] which makes it possible to check both storage clouds from 24 different locations around the world. The locations are shown on a world map in figure 4.

3 3 4 MONITORING AMAZON S3 AND NIRVANIX At an interval of 10 minutes one of the 24 control stations performs a check which logs the date, time, station, connection- and download- time. Every time the check is performed, the randomly selected control station will start download a file of 1MB. After downloading the file is automatically tested at some given rules to ensure the integrity of the file. The test will be done for one week and results in 1008 experiments. Figure 4: The monitoring nodes are spread all over the world, to guarantee reliability. One of the main issues involving this kind of experiments is the lack of bandwidth or server capacity at the monitoring site. With this in mind we selected a commercial application which has a good reputation concerning SLA monitoring. As you can imagine, it is very important that the results are flawless, which is also concluded in [7]. When a node reports an error, we will use a second node to verify this by immediately perform the same action. Only a verified fault will count as an error, and an error will be count as downtime. A fault which could not be verified by another monitoring station we call a defect. The reason for doing this is because we want to monitor the uptime of the storage clouds and not the uptime of the connection between the monitoring location and the storage cloud. 4 Monitoring Amazon S3 and Nirvanix 4.1 Availability (Uptime) A combination of both related research and our own experiment is used to determine if both storage clouds can meet their promised uptime of 99,9%. We selected 24 locations which randomly monitored both storage clouds and check if they are available and give the expected response. Before we started, we did some initial testing and found something interesting on our monitoring location in Japan. Somehow the monitoring tool could not resolve the URL of Amazon S3. Because every monitoring site uses the same resolve mechanism, we first thought the country Japan was blocking/shielding connections to Amazons S3 storage cloud. This was a wrong conclusion, the monitoring site in Japan had difficulties with resolving names this means this issue was DNS related. After a week conclusively testing we counted five defects. Four of them where measured at the monitoring station located in Auckland. The defect was reported three times for Nirvanix and one time for S3. The only other defect was reported by a station in Amsterdam. Because of the several defects in Auckland which were reported at both providers, we can conclude that Auckland does not have a stable connection. This we can explain by the physical distance of the cable to the rest of the world. The defect reported by the station in Amsterdam could be something real, because the monitoring site had a connection which timed out. In the log files we couldn t find any explanation, thus the reason why this defect happened remains unknown. At the end of the week both storage providers achieved an uptime of 100%. In a similar test [7] the results where pretty worse for the Amazon S3 located in the USA. At a test site located in Portugal the S3 scored only a poor 88% uptime. All sites suffered from many errors, the node in Texas had an average uptime of which means a failure count of 5. The average availability of all nodes was 98.5%, which means that on average 40 of the total experiments which was A possible explanation of above results we found in [8]. In this research they performed an uptime and performance test. They verified the integrity of data for both tests. The uptime test was very simple because they used one monitoring station with a fairly simple test file of 1MB. The second was far more complicated they did many more experiments with various file sizes till 100MB. They measured in a total of 107,556 experiments only 5 defects. The first experiment observed an availability rate of 95.89%. After investigation why both

4 4 4 MONITORING AMAZON S3 AND NIRVANIX Figure 5: Downloading Results tests gave such a different result, they detected a network fault at the monitoring site of the first test. As our experiment has shown and because of the confirmation we found in research [8], we can conclude that storage providers can deliver their promised uptime of 99.9%. But be aware that the quality depends on the location of both storage provider and storage consumer. 4.2 Download time Besides the availability, we are also interested in the downloading times for our 1MB test file. These measurements give us insight in the performance of the services. We can see in figure 5 that S3 and Nirvanix have more or less the same results for testing nodes located in Europe. For nodes located in the USA, S3 has a noticeable faster download time. Both have lowest download times in the USA, directly followed by Europe. Nodes in Asia and Australia

5 5 4 MONITORING AMAZON S3 AND NIRVANIX Figure 6: Connect Time Results have a much higher download time. The explanation for the results is the physical location of the storage providers. We use the S3 service located in the USA, and the Nirvanix service we use is located in Germany. When we keep this information in mind, we may conclude that S3 performs slightly better at this point. The time measurements for downloading a 1MB file are much larger than we expected. We expected better results from Nirvanix in Europe, since we use the location in Germany. When the files are used in an application that should be responsive, the downloading process takes too long. We obtained a minimal download time of 1544ms, a maximum of 28024ms and an average of 5224ms at Nirvanix, resulting in an average downloading speed of 0.19 MB/sec. S3 had a minimum of 335ms, a maximum of 28761ms and an average of 4535ms which results in an average downloading speed of 0.22 MB/sec. S3 has a better average in this experiment. The standard deviation for S3 is 2853ms, where Nirvanix has a standard deviation of 2652ms. Nir-

6 6 5 CONCLUSION vanix provides a slightly better when we look at the stability of the results, but the standard deviation is still high. In a test [8], performed on the Amazon EC2 cluster where they used larger file sizes (100MB), they obtain an average bandwidth up to 10MB/Sec for one thread. Six threads running on the same virtual machine results in an average throughput of 30MB/Sec. This high performance is achieved because the EC2 cluster is probably located at the same physical location as S3. When the S3 storage would be used for science grids, it would be a good solution if there are computing clusters located near the storage servers. At this point of view, this experiment makes sense. The authors of [8] ran an additional 28 remote tests from 5 different locations in the USA, Germany and France, which is similar to our experiment. They achieved more or less the same results as in our experiment for a 1MB file. In [7], where another similar test has been performed, we also see the same values as obtained from our own experiment. This verifies our results, and we may conclude that the services have not upgraded the bandwidth in order to provide speedup in downloading files. 4.3 Connect time Besides availability and download time, the connect time is an other important issue, especially when small files are used in applications that should provide high responsiveness. The connect time is the time that it took to establish the connection between a machine and the storage service. In some cases where files are less than 100KB, the connect time may be even more than the total downloading time. The monitoring tool is able to log the time required to connect to the server. This may vary depending on route, other traffic on the internet and the physical location of the node that performs the test. In figure 6, the connect times are shown per continent for both test sites. For the 1008 experiments performed against Amazon S3, the overall maximum is 492ms and the minimal is 4ms, with an average of 107ms. For Nirvanix we obtained a maximum of 485ms, a minimum of 5ms and an average of 154ms. From these measurements we may conclude that S3 performs better at the average. S3 has one extremely high drop-out at a testing node in the Netherlands (3088ms), and Nirvanix has one in Denmark (3114ms). We removed these drop-outs from our results in figure 6, but we have to take them in count in our conclusions. As expected, Nirvanix has a better connect time in Europe, and S3 in the USA. In Asia and Australia, both have a high connect time, but the best results are measured for S3. 5 Conclusion As we saw in related research [7, 8], it is not easy to manage a good monitoring infrastructure which is able to give accurate results. Therefore we used the defect and error concept. Looking at our research where we wanted to measure the availability of both storage clouds, it turnouts that this concept works pretty well in our case. One very noticeable thing is the cost structure of S3 which is a little remarkable because when the files are less than 5KB the price for using this service is getting very expensive. Nirvanix on the other hand has a more simplistic pricing structure where you only have to pay for what you are using. Before choosing a provider merely on availability requirements it s advisable to make a full set of requirements including, pricing. Despite some monitored defects at Amsterdam and Auckland we can conclude that both storage clouds meet there promised uptime of 99.9% flawless. Looking at the defects we monitored at location Auckland, we also have to conclude that to have your service being available one have to make sure to select a storage provider which can offer a location which is able to meet your requirements regarding availability in respect to your client s locations. This could mean that you need more than one location to store or retrieve data. There were some incidents with very high latency but the overall connection times monitored by the 24 different locations were fairly stable. Since Nirvanix is located in Germany one would expect to have much better connection time from European monitoring stations and as expected this was the case. S3 on the other did a much better job in their home continent. The downloading speeds differ between both storage providers. On average the downloading

7 7 REFERENCES speed at Nirvanix was more constant. Despite their location, Nirvanix did not outperformed S3 in Europe. S3 on the other hand had, as expected, a far better download time in their home continent. 6 Acknowledgments The authors would like to thank WireITup for giving access to their Watchmouse [3], Amazon [1] and Nirvanix [2] account. References [1] Amazon simple storage service. [October 13, 2008]. [2] Nirvanix storage delivering network. [October 13, 2008]. [3] Watchmouse: a website performance monitoring tool. [October 13, 2008]. [4] J. Broberg, R. Buyya, and Z. Tari. Metacdn: Harnessing storage clouds for high performance content delivery [5] R. Buyya, C.S. Yeo, and S. Venugopal. Marketoriented cloud computing: Vision, hype, and reality for delivering it services as computing utilities [6] G. Gruman and E. Knorr. What cloud computing really means. Infoworld, [7] Ayodele Onibokun and Mayur Palankar. Amazon s3: Black-box performance evaluation. Project Report, [8] M. Palankar, M. Ripeanu, and S. Garfinkel. Amazon s3 for science grids: a viable solution? ACM, 2008.