Faster Cloud Backup: How It's Done

Transcription

1 TECH BRIEF Faster Cloud Backup: How It's Done What You Need to Know One of the main concerns we hear from customers is around the ability to transmit large amounts of data over the Internet with their available bandwidth, or rate of speed. This concern spans the full spectrum of customers from offices managing 500GB of aggregate data and sub 5Mbps upload speeds to large, multi-office organizations with multiple TBs of data and 50Mbps+ upload speeds. To complicate matters, businesses are buying into an industry that is full of competitive backup and recovery solutions that all brand themselves as the fastest or most optimized solution on the market. We too market our product as the fastest solution in the cloud backup space. The problem is if everyone is the fastest then what does the faster really mean?

2 WHAT FASTER CLOUD BACKUP LOOKS LIKE A simple analogy for explaining data transfer over the Internet is relating the process to trucks moving on a freeway. Trucks (data packets) carry boxes (data blocks) from point A to point B. The truck s speed rate is determined by the upload bandwidth speed from point A and the speed limit on the freeway the ingest speed of point B. A fast truck driving at a high speed limit means faster transfer of boxes. A slow truck cruising at a low speed limit translates into slow transfer of boxes. Simple enough. But is it? It would be fairly safe to assume anyone could transfer a sizable amount of data over the internet given enough upload speed. The truth is most customers do not have unlimited bandwidth. And when discussing data transfers and bandwidth, the limiting factor typically is their upload speed, or the speed of their trucks. But you cannot simply throw a faster engine in the truck (upgrade the bandwidth); in fact, there are regions where faster engines are not available even if someone wanted to buy one. So, given that the average customer is trying to move data with a slow truck, what can be done to make their transfer faster? No Throttling Instead of imposing a speed limit (upload bandwidth speed) on a freeway, the trucks (data packets) carrying boxes (data blocks) are free to drive as fast as they are capable of. Transfer Compression & Multithreading The trucks are filled in a way that there is no wasted space (compression). Multiple people load the trucks with a truck always in transit (Multi-threaded WebDav) Local Signature Caching & Subfile Change Detection The truck doesn t have to wait for a response on what to carry (local cache or manifest), enabling rapid change detection and the truck only contains what is needed (sub-file cache). NO At Zetta.net, we focus our resources on efficiency. We do everything we can to make sure that a slow truck moves as much data as possible, and as efficiently as possible. We do this by being more efficient in the following ways: Flash Memory & SSD Storage When the truck reaches its destination, it doesn t have to wait to unload (back-end flash caching). Slow trucks moving as much data as efficiently possible enable faster backup and recovery. 2

3 No Throttling The first way we make your backup faster is that we do not throttle our customers on either uploads or downloads. We also do not charge for bandwidth usage or enforce any kind of bandwidth tiering on a per customer basis. In order to ensure that we provide as much bandwidth as customers can use, Zetta.net utilizes multiple 10 Gbps bandwidth connections into our data centers that are redundant across multiple providers, preventing the failure of a single provider from taking the service down. This has allowed our larger customers to achieve upload speeds in excess of 500 Mbps enabling the transmission of multiple terabytes of changed data within a 24 hour period. A throttle is like imposing a speed limit on a freeway. The trucks might be capable of travelling at 80 MPH, but a limit is imposed saying they can only travel 55 MPH. This may seem obvious but it is surprising how many competitors in the online backup space use bandwidth throttling as a cost saving technique to be price competitive. Transfer Compression & Multithreading The ZettaMirror client utilizes a multi-threaded implementation of the WebDav protocol, as well as network compression to optimize data transfers to the Zetta.net cloud. Multithreading allows for better disk IO utilization which in turn leads to more efficient use of network throughput. Zetta.net uses advance data compression to consolidate blocks of data to significantly reduce the amount of data transferred. Empty or partially full blocks of data take the same amount of space as full blocks of data. The transfer compression prevents empty space from being sent over the network, therefore lowering backup times. Imagine the freeway is being used to move inventory from a storage facility to a grocery store. A single threaded implementation is equivalent to a single employee loading a single truck at the storage facility. Multithreading means having multiple employees loading multiple trucks that will travel on multiple freeway lanes. More employees (threads) loading utilizes the available resources (disk IO) better than a single employee. Loading multiple trucks ensures that there is always a shipment in transit and your bandwidth is being fully utilized at all times. The compression adds onto this by making sure each box (block of data) being loaded on the truck is full. This results in less total boxes being sent, which requires less trucks and less total time. 3

4 Local Signature Caching & Subfile Change Detection This where the bulk of Zetta's speed is derived from compared to the industry. During the initial sync of your data, Zetta.net builds a cache which contains what we refer to as the manifest of your file system. This cache represents the Zetta.net-side state of your data at the end of each sync; this provides the ability during subsequent syncs to compare the current state of the local data against the server-side data, without having to traverse the network to detect changed files. This enables rapid change detection even on file systems scaling into the multi-million file count. With a previous approach, each change would need to be compared by constantly communicating back and forth between the customer s system and the Zetta.net storage. By eliminating this process and enabling the compares to be done locally, Zetta saw a sync time improvement of 10x on a daily basis when transferring incremental changes. The second portion of the caching system is for our sub-file change detection. Zetta.net uses file level changed block detection to determine the blocks that In the analogy of moving items from a storage facility to the grocery store, the manifest cache would be like having a full inventory list at the grocery store maintained at the storage facility. Instead of the storage facility having to ask the grocery store if it needs apples or milk and waiting for a response, the storage facility can simply check the list (manifest), and know what to send. The difference between the two methodologies results in a staggering difference in performance as demonstrated by the 10x improvement our customers have achieved. And if the grocery store needs 2 more boxes of apples, the sub-file cache is the difference between sending just the 2 boxes of apples needed, instead of the entire pallet of 50 boxes that the storage facility has. change within a file, in order to transmit the least amount of data possible when a file does change. This means that if only 10MB of a 100 GB file changes, Zetta.net only moves the 10 MB of data across the network and then patches the change into the version of the file that exists on our end. 4

5 Back End Storage Designed to Work With Front End Software These various methods of making the backup process faster are tied together with our ability to manage our back end infrastructure so as many bottlenecks as possible are removed from the equation. The major bottleneck that comes up with transferring data in and out of any large scale storage infrastructure is network throughput; we handle that as previously mentioned with 10Gbps bandwidth connections (a freeway without a speed limit). The next issue is handling random I/O when writing data to disk, especially when working with extreme amounts of writes coming from multiple sources. This is optimized for transfer through the previously mentioned ZettaMirror methodologies. Zetta.net s back end is designed with a What this means is the loading dock of our store is set up in a way that each shipment can simply be dumped off in a random place on the loading dock (the flash storage) instead of having to spend the time unloading and organizing the shipments. The employees at the store are so fast at finding the items and getting it off the dock and onto the shelves (disk storage), that there is no concern about the loading dock over filling and causing the transport vehicle (the data packet) to get stuck in a traffic jam upon arriving at the store. large scale deployment of flash memory and SSD storage to convert what potentially could be a large disk latency issue with random I/O into what is essentially sequential I/O by utilizing flash memory. We now have slow trucks being used as efficiently as possible. So to answer the original question, what does faster mean and how is Zetta.net faster? Zetta.net provides the most efficient, and thus fastest, solution in the online backup space. CONTACT US Please ask us how we can help you safely backup to the cloud. Get Custom Pricing. Start a Free Trial. Us: sales@zetta.net Call Us: Zetta Inc. is an award-winning provider of enterprise-grade cloud backup, disaster recovery, and archiving.