HIGH-SPEED BRIDGE TO CLOUD STORAGE

Transcription

1 HIGH-SPEED BRIDGE TO CLOUD STORAGE Addressing throughput bottlenecks with Signiant s SkyDrop

2 2 The heart of the Internet is a pulsing movement of data circulating among billions of devices worldwide between computer systems, people, and into and out of cloud infrastructure including cloud storage. Much of that data has a rather boring and neglected life, created only to be forgotten moments later. But a subset is high-valued and even precious from government secrets and personal account information to entire movie files and life-saving research data sets. Those are the files that we protect, analyze, edit, share, backup, and store. And it s all only growing. According to IDC s recent report on the digital universe, by 2020, there will be as many digital bits in existence on the Internet and in storage devices as there are known stars in the universe. And, as we move forward into the future, we will need more storage for the valuable bits we want to keep, as well as much improved digital mechanisms to move them around. Cloud data storage is being rapidly adopted by companies that produce large amounts of data and digital assets, but traditional approaches to file movement to and from the cloud are impeding an otherwise innovative step forward.

3 3 Growing storage needs and the problem of high bandwidth, high latency environments For established companies, the expanding digital universe often equals shrinking on premises storage capacities, and many are looking to the cloud to address growing storage needs. On the startup front, most new software driven companies do not even consider trying to implement enough on-premises storage to meet peak demand; they are including cloud storage and the flexibility of elastic infrastructure from the beginning. In both cases, they know that storage in the cloud offers scalability and global access to their business-critical files. And for some of those companies, getting files into and out of the cloud as quickly as possible usually in order to capitalize on an action serviced by cloud software is also critical. However, the available mechanisms for doing so could use some vast improvements. Typically, the way that content is moved over the Internet, including in and out of cloud object storage, is through HTTP (hypertext transfer protocol) and its underlying TCP (transmission control protocol). Both HTTP and TCP were originally designed in the 1970s and 80s and constantly stall out in high latency environments even those with high bandwidth. What is the difference between bandwidth and latency? Bandwidth is the theoretical maximum rate data can be transferred over a given network or network segment (usually measured in bits/second). And latency is the amount of time it takes for data to travel across the network or network segment (usually measured in thousandths of a second or milliseconds). If you think of the network as a garden hose, the bandwidth is the width of the hose and the latency is the length. What can be confusing here is that latency is not impacted by bandwidth; no matter how big your hose is the data still has to travel the distance. In theory, data travels at the speed of light. But, in practice, switching and other overhead in the network slows it down or creates higher latency. The speed of light is pretty fast; it takes light approximately 1/10th of a second (or 100ms) to travel around the world, but that s without taking into account any switching overhead. And, for longer distances, more switches are involved. The latency when sending data half way around the world and back can be over 1/3 of a second. Even when sending data between two locations in the same building, data can travel the equivalent of across the continent if it s on different service provider s networks; so it s the network distance, rather than the physical distance that s important. Network distance is what causes throughput bottlenecks in most systems, slowing down content transmission even if you ve got a big bandwidth pipe. These throughput bottlenecks can disrupt a successful business model if workflows depend on moving data into and out of cloud object storage. When time to and from cloud storage matters Signiant recently launched SkyDrop to meet the needs of companies dealing with any situations requiring accelerated access to cloud storage, and to fill the technology gap they face between their on-premises storage and the cloud storage provider. Why is that gap such a problem? Improving on traditional Internet file transfer protocols As mentioned above, typically, the way content is moved into and out of cloud object storage is through the good old hypertext transfer protocol or HTTP. For its initial intended purpose, HTTP was optimized for loading web resources

4 4 in a browser. The problem is, when you get very high bandwidth pipes and some distance involved, HTTP and its underlying TCP (Transmission Control Protocol) start to slow down quite dramatically. SkyDrop greatly improves on theses traditional protocols. But before we get into how, let s look at the other ways HTTP s performance problem has been addressed. Content Distribution Networks The way CDNs (content distribution networks) work is by caching content at the edge of the network, closer to the users that are going to consume it. They are typically used for serving the same content to thousands of people distributed around the world or distributed around a particular geography. For example, with a CDN when a user goes to load an image in their web browser or watch a video in their web browser, it s typically not actually being served from a central data center; it s being served from a cached copy at the edge of the network. And while this approach works great for content that is consistent the same image or the same video being served to thousands of people around the world it s not a practical approach when you re either uploading content into the cloud or when you re downloading relatively unique content out of the cloud. Plus, streaming a video like on is very different need than downloading an entire asset as quickly as possible Can t I Use CDN?

5 5 Infrastructure as a Service Other ways to move files: Dedicated pipe Multi-part upload How IaaS reduces latency IaaS options for reducing latency, like dedicated network connections such as Amazon Web Services Direct Connect and Azure s ExpressRoute, basically provide a way to connect up a fixed telecommunications circuit from your enterprise directly into cloud infrastructure. In order to take advantage of this, first you have to go to your friendly neighborhood Telco and order a fixed circuit that runs from your premise into their premise (assuming you re not already co-located in one of their data centers with their application). Then you have to wire that circuit up into your Internet protocol network with firewalls and routers. Then you have to connect it into the Azure or Amazon network. And really, their offering is only that last mile connection between your pipe and their on-premises infrastructure. HISTORIC APPROACHES TO MOVING LARGE FILES TO THE CLOUD AWS Direct Connect Azure Express Route Additionally, various IaaS (infrastructure as a service) vendors offer different solutions to address moving content into and out of the cloud. And one of the most basic of those approaches is to stick your content on some form of physical media and ship it to a provider, who then loads it into their cloud object storage. While shipping hard drives works and you can t beat the bandwidth of a truck, when you have to do this over and over again, it starts to have some serious logistical challenges. And it s not very easy to automate. So, of course, most of us prefer an electronic, network-based approach. For IaaS, these basically fall into two categories. One reduces the latency between the person using cloud object storage and the cloud, and the other compensates for that latency: Not Elastic Not Agile Still have Latency REDUCE LATENCY

6 6 So this approach works in some scenarios, but not if you need lots of points or connections into the cloud. It s also not agile or very elastic: you re deploying fixed network infrastructure to address the problem, which is difficult to change. Plus, you still have latency in your connection, so you haven t completely overcome the limitations of the HTTP protocol in that you re still traveling over some sort of network connection to get to the cloud platform. How IaaS compensates for latency Another option that most IaaS service providers offer is a multi-part upload capability, where you can run multiple HTTP transfers in parallel using a proprietary API to break objects into chunks and upload each of those chunks independently. Now, again, this approach has some benefits. But one downside is that you have to be relatively technically savvy to use the different API that each of the cloud vendors offers for doing this multi-part upload. Multi-part Upload HTTP Different API for Each Cloud Diminishing Returns COMPENSATE FOR LATENCY And you also get diminishing returns as you add parallel streams. When you add one parallel stream, you get twice the throughput. When you add your third parallel stream, you re only getting 50% improvement in throughput. So there s a limiting and diminishing return on adding these parallel HTTP streams, and you still have latency to deal with.

7 7 SkyDrop: How Signiant achieves extremely high transfer rates SkyDrop accelerates file movement into the cloud in two pipelined stages, minimizing latency at both stages of the process. The first stage compensates for latency by using the Signiant Acceleration Protocol. The second stage minimizes latency by running the Signiant transfer server close to the storage. We ve improved on traditional HTTP and TCP by leveraging the parallel HTTP technology used by IaaS providers. If we speak specifically about working with Amazon cloud storage, Amazon EC2 instances have a very low-latency, high-bandwidth connection to the S3 storage. Therefore, parallel HTTP in the cloud works and performs quite well from those instances into object storage. We ve also added Signiant s advanced WAN (wide area network, a network that covers a broad geographical area) acceleration protocol. Essentially, Signiant s accelerated transfer servers are run in the infrastructure as a service provider s environment, while using the Signiant transport protocol to get content into the cloud. SaaS Accelerated Transfer Signiant Transport High Performance Elastic & Agile SKYDROP MINIMIZES LATENCY IN TWO STAGES Through these two pipeline steps Signiant transport protocol over a long distance or metropolitan area IP connection and the parallel multi-part HTTP pipelined on the other end into the cloud object storage we achieve extremely high transfer rates. With speeds up to 200 times faster than standard Internet transfers, SkyDrop also eliminates interruptions in transmission with a featured called checkpoint restart: any interruptions in your file transfers will be automatically restarted, so files will never get stuck en route. Additionally, SkyDrop automatically encrypts all files for the high security transfers.

8 8 More than just Speed: the additional advantages of SkyDrop Managed SaaS SkyDrop, like any true cloud-born software, is run as a managed SaaS offering. Rather than having to go out and deal with machine instances turn them up, make sure they re running, manage them, and monitor them, for instance Signiant runs the servers for you. In contrast with accelerated file transport solutions that are not truly cloud-based, with Signiant you wind up with an elastic high-performance solution that is easy to get started and doesn t have ongoing costs associated with managing your own servers, like applying software updates and monitoring your servers for availability and scale. SaaS control layer SkyDrop s SaaS control layer gives you automatic updates of your web plug-in client, but also allows you to abstract the back end object storage that you re moving content into or out of. As we build out more capabilities in the SaaS control layer, you will have visibility into real time and historic activity reports and have total control over what object storage credentials are being used to upload or download. Automatic scalability With auto-scaling, redundancy, and load balancing built-in, SkyDrop scales automatically for you and can handle your largest bursts of files or a wave of new projects. That s all included in your subscription and you never have to worry about spinning up new servers, instances, or finding the headcount and resources to move large files fast. Automatic software updates Easy deployment & management No server configuration/ procurement by customer Reliability/availability/scalability Signiant manages load balancing, scale out, etc. Flexible client integration With SkyDrop, there s no need to change the way you work. You can install lightweight client apps or command line clients for Mac, Windows, or Linux on a local machine and you are ready to move files. There s also a set of developer tools (SkyDrop C++, SDK, Mobile SDK, or JavaScript API) that can be integrated with your own applications, whether they re web applications, native applications, or mobile applications. SkyDrop subscriptions include the ability to use any of the clients from any location however you want, without any limits or requirements to purchase another client. It s all part of the service. Global access to multiple cloud storage platforms

9 9 SkyDrop is the only solution that makes it easy to move files to multiples cloud platforms from the same system, including Amazon, Microsoft Azure, and Google Cloud Platform. So you can use Amazon S3 today, and if Azure offers a more competitive price tomorrow, you can switch with the same SkyDrop subscription. SkyDrop gives you the freedom to choose the best platform for you business, so you can avoid cloud vendor lock in. SkyDrop supports these platforms automatically, so even if you are using multiple cloud platforms, you never need to purchase, setup and maintain different systems in different clouds. So the Signiant SkyDrop solution is not only fast, it s easy to use, easy to get started with, scales with your requirements, and supports multiple clouds. One of the main reasons it s so easy to use is that it only requires three simple steps to get started. First, install the SkyDrop client software on your premise. Second, use the SkyDrop service to connect to your cloud storage tenancy. Third, start transferring files. Within minutes you are moving content to your cloud object storage in an accelerated, high-performance, and secure manner. Customer s Network Access Multiple Cloud Platforms

10 10 Summary Addressing slow performance in high bandwidth, high latency environments for large-scale data migration to and from cloud storage has been a challenge for businesses. SkyDrop s advanced acceleration technology addresses throughput bottlenecks with a dual pipelined approach, providing transfer rates up to 200 times faster than traditional Internet protocols, and doing so with security encryptions not available in other offerings. As a true SaaS solution, SkyDrop includes automatic updates, auto scaling for large data bursts, flexible client integration, and a control layer into the backend of your cloud-stored files. About Signiant Used by the world s top content creators and distributors, Signiant is the market leader in intelligent file movement software for media and entertainment. The company s powerful software suite optimizes existing enterprise network infrastructure and media technologies to ensure secure digital media exchanges, workflow efficiency and superior user experiences. Visit Increasingly, cloud data storage is becoming a critical aspect of software-defined companies. And, when time is of the essence, the power to efficiently interact with cloud storage provides a definite competitive edge. If your business needs a highspeed bridge to cloud data storage, learn more or try a free trial of SkyDrop or contact us to discuss how SkyDrop and Signiant can help you achieve your cloud data storage goals. p: