Whitepaper How to setup NovaBACKUP DataCenter to backup data to Amazon S3 using Amazon s AWS Storage Gateway
Contents What is Amazon S3?... 3 What is the AWS Storage Gateway?... 4 How to setup Amazon Storage Gateway... 5 Setting up NovaBACKUP DataCenter to utilize the AWS Storage Gateway... 13 How to restore from Amazon S3?... 16 Any Questions left?... 16 Contact Us... 17 Notice... 17 Copyright... 17
The following whitepaper is going to describe what Amazon S3 is, and how with the help of Amazon s AWS Storage Gateway NovaBACKUP DataCenter can backup and restore to Amazon S3 seamlessly. What is Amazon S3? S3 stands for Simple Storage Service and that s exactly what Amazon S3 is, an offsite storage for all kind of data, accessed through a web service interface. The storage includes some amazing features that helps to meet company compliances: Cross-Region Replication Event Notifications Lifecycle Management Encryption Security and Access Management Flexible storage options Because of the high-end technology used in their data centers, Amazon s service is able to deliver HA and redundancy at its best. Transferring single files with a size of up to 5GB at once is not a problem, individual objects can have a size of up to 5TB. In case your complete data is too big to transfer it over a regular internet connection, Amazon s offers include e.g. a dedicated connection between your and their data center. I don t want to talk in depth about the functions, but here are some interesting facts about Amazon S3: The consistency model used for the US region is eventual consistency. Amazon S3 supports user authentication such as bucket policies or ACLs Up- and Download of data via SSL endpoints using the https protocol (further security mechanism available, e.g. SSE, SSE-C) Encrypt your data inside the buckets using SSE-S3, SSE-C, SSE-KMS (HIPAA/ HITECH, and FedRAMP compliant) encryption SSE encryption keys are unique for every object. This key is also encrypted with the master key that changes at least monthly. All keys are stored in separated storages. Using a VPC endpoint, allows to connect only to the S3 storage 1 The checksum used to detect data corruption is Content-MD5 and cycling redundancy checks (CRCs) Amazon doesn t access the data stored except for billing purposes or if required by law. Details regarding Amazon S3 SLA s can be found here: http://aws.amazon.com/s3-sla/ or http://aws.amazon.com/agreement You can find more information about features and functions here: http://aws.amazon.com/s3/ Just remember, every extra feature probably comes with extra costs! In order to access the Amazon S3 storage, it has to be connected with the Amazon Storage Gateway 2. This service ensures that your data is transferred fully SSL encrypted. But the main function is the simple integration to every device. Therefore the Amazon Storage Gateway implements a standard iscsi interface, either as a block disk device or VTL. 1 http://docs.aws.amazon.com/amazonvpc/latest/userguide/vpc-endpoints.html 2 http://aws.amazon.com/storagegateway/
What is the AWS Storage Gateway? The AWS Storage Gateway basically is a virtual machine. In order to connect it to your Amazon services it has to run on a virtual machine. A VMware ESXi v4.1 and newer or a Hyper-V 2008 and newer Hypervisor are sufficient. The service also needs 4 virtual processors assigned to the VM 7.5GB RAM 75GB disk space for the installation Source: http://aws.amazon.com/storagegateway/details/ Security and encryption wise the AWS Storage Gateway encrypts all data in-transit to and from AWS via SSL. All volume and snapshot data stored in AWS using Gateway-Stored Volumes/Gateway-Cached Volumes and all virtual tape data stored in AWS using Gateway-VTL is encrypted-at-rest using Advanced Encryption Standard (AES) 256, a secure symmetric-key encryption standard using 256-bit encryption keys. So you do not need to add other encryption options in order to make sure that the data that you put on the AWS Storage Gateway is encrypted. There are 3 different ways to setup the AWS Storage Gateway, Gateway-Cached Volumes, Gateway- Stored Volumes, and Gateway-Virtual Tape Library. Below is a description of what each configuration does 3 : Gateway-Cached Volumes: You can durably and inexpensively store your primary data in Amazon S3, and retain your frequently accessed data locally. Gateway-Cached Volumes provide substantial cost savings on primary storage, minimize the need to scale your storage onpremises, and provide low-latency access to your frequently accessed data. In addition to storing your primary data in Amazon S3 using Gateway-Cached Volumes, you can also take point-in-time snapshots of your Gateway-Cached volume data in Amazon S3, enabling you to make spaceefficient versioned copies of your volumes for data protection and various data reuse needs. 3 http://aws.amazon.com/storagegateway/faqs/
Gateway-Stored Volumes: In the event you need low-latency access to your entire data set, you can configure your gateway to store your primary data locally, and asynchronously back up point-in-time snapshots of this data to Amazon S3. Gateway-Stored volumes provide durable and inexpensive off-site backups that you can recover locally or from Amazon EC2 if, for example, you need replacement capacity for disaster recovery. Gateway-Virtual Tape Library (Gateway-VTL): With Gateway-VTL you can have a limitless collection of virtual tapes. Each virtual tape can be stored in a Virtual Tape Library backed by Amazon S3 or a Virtual Tape Shelf backed by Amazon Glacier. The Virtual Tape Library exposes an industry standard iscsi interface which provides your backup application with on-line access to the virtual tapes. When you no longer require immediate or frequent access to data contained on a virtual tape, you can use your backup application to move it from its Virtual Tape Library to your Virtual Tape Shelf in order to further reduce your storage costs. How to setup Amazon Storage Gateway The following will go through how to setup the AWS Storage Gateway (ASG), and in this case it will be configured as a Gateway-Chached Volumes storage gateway. This way the data that is sent to the ASG locally will be synced with S3, and since we are just using this for backup storage only the most recent backed up data will be available on the local appliance but everything will be available on S3. Here is a good video on how to get started with the Amazon Storage Gateway if you want to get a video overview: https://www.youtube.com/watch?v=bb8nk0owjbu. The following configuration assumes you already have an Amazon AWS account setup and have admin access to that account. The first thing you need to do is log into the AWS web console then go into the Storage Gateway screen.
After you go into that screen, assuming you have no other ASG deployed, you need to deploy a new AWS Storage Gateway. Then you need to choose the type of configuration your ASG will be.
What type of virtualization you will utilize, and in this case it will be VMware based. It will then give you the option to download the AWS Storage Gateway virtual machine files, or if you already have it downloaded you can just continue. The next steps will go through deploying the virtual machine itself into your environment, and their wizard will step you through this process if you have never done it before.
After you finish those steps the next thing you need to do is setup the local disk storage for the ASG. Amazon gives you a calculator to try to determine a good idea of how big your disks need to be. All of the allocation of the disks need to be done on your hypervisor, in this case VMware. Make sure to go through the Step-by-Step instructions as you do have to do a couple things different than normal to get the disks to work correctly. You need to create at least 2 volumes, one for the upload volume and one for the cache volume.
Once you have your disks added to the virtual machine go ahead and start the virtual machine. Once it is up and running you want to get the local ip address of the virtual machine, as the next screen requires that local ip address in order to activate the gateway. Then you can give the gateway a name so that when you are in the AWS console you know which gateway you are looking at.
Once you have done that you should have an active gateway, but now we need to add some volumes to it and set it up. The first thing we need to do is to create a volume. When you create the volume the wizard will ask you which volumes you want to be the upload and cache volumes.
After that you can configure it to alert you when those volumes become full, and then it will ask you how big you want the actual volume for the AWS Storage Gateway to be. This will be how big of volume is presented via iscsi to the machine that is mounting it.
The wizard will then ask if you want to setup CHAP authentication on the iscsi target, and finally after that it will complete the configuration of the gateway.
Setting up NovaBACKUP DataCenter to utilize the AWS Storage Gateway Now that you have the AWS Storage Gateway setup and running, you need to get it connected with NovaBACKUP DataCenter so you can start sending data to it. In this case the configuration that will be detailed below is with a Windows Server 2012 R2, but Linux and other Windows operating systems that include iscsi connectivity will work just fine. The basics of the configuration here is to add the iscsi target to the machine, add a volume, and do a quick format. After Windows sees the drive then you can configure the storage in NovaBACKUP DataCenter as a disk pool. The first thing you need to do is to go into the iscsi configuration under control panel administrative tools. Once there add the local address of the AWS Storage Gateway.
Once that is added you need to connect to the target on the targets tab.
After you have connected to the target, go into drive manager to create the volume and start the quick format.
Once that drive is accessible within Windows go into the NovaBACKUP DataCenter UI and create a new disk pool so that you can backup or clone to your newly configured AWS Storage Gateway. At this point your newly created AWS Storage Gateway is going to act like any other disk pool for NovaBACKUP DataCenter. Anything that you put on this volume be it backup or other files will now be automatically sent up to Amazon S3 via the AWS Storage Gateway. How to restore from Amazon S3? One of the concerns is how to get that data back from Amazon S3 once it is no longer on the local AWS Storage Gateway s cache. This is seamless and you have to do basically nothing to tell the gateway to go out and grab data from S3. If you start a restore within NovaBACKUP DataCenter from a backup file that is no longer on the gateway locally the gateway will actually stream data from S3 for you to NovaBACKUP DataCenter. So it will be completely transparent to you as the user when it is downloading data from S3 or when coming from the local cache. Obviously from local cache it will restore faster since it does not have to stream from S3. Any Questions left? If you want to learn more about how to connect NovaBACKUP DataCenter to an Amazon S3 storage, feel free to contact our Enterprise Team directly at: Tel.: +1 805.579.5445 M-F 9AM-5PM PDT Email: DCInfo@novastor.com
Contact Us NovaStor Software AG Poststr. 18 CH-6301 Zug, Switzerland Tel +41 (41) 712 31 55 Fax +41 (41) 712 21 56 NovaStor Corporation 29209 Canwood St. Agoura Hills, CA 91301 USA Tel +1 (805) 579 6700 Fax +1 (805) 579 6710 NovaStor GmbH Neumann-Reichardt-Str. 27-33 D-22041 Hamburg, Germany Tel +49 (40) 638 09 0 Fax +49 (40) 638 09 29 Notice Information in this document is subject to change without notice. NovaStor makes no representations or warranties with respect to the contents of this document and specifically disclaims any implied warranties of merchantability or fitness for any particular purpose. Further, NovaStor reserves the right to revise this publication and to make changes without obligation to notify any person or organization of such revisions or changes. NovaStor is not responsible for any charges incurred by Amazon s services. Please check Amazon s pricing here http://aws.amazon.com/storagegateway/pricing/ Copyright Under copyright laws, the contents of this document may not be copied, photocopied, reproduced, translated or reduced to any electronic medium or machine-readable form, in whole or in part, without prior written consent of NovaStor. Trademarks NovaBACKUP is a registered trademark of NovaStor. Windows is a registered trademark of Microsoft Corporation. Amazon and AWS are registered trademarks of Amazon.com Inc. and Amazon Web Services Inc 4 4 http://aws.amazon.com/trademark-guidelines/