Red Hat Enterprise Virtualization Disaster Recovery John Herr, RHCA Senior Software Engineer, Red Hat Aaron Weitekamp Senior Software Engineer, Red Hat
What is this session about?
This session is about Disaster Recovery An overview of the failover process. The methods available to create a failover. The Red Hat Enterprise Virtualization 3.3 REST API. Automating the failover process.
Presenters John Herr (joherr@redhat.com) Senior Software Engineer, Red Hat Aaron Weitekamp (aweiteka@redhat.com) Senior Software Engineer, Red Hat Both work in the Systems Engineering Team at Red Hat. This team identifies high-value solution stacks based on input from Sales, Marketing, and Engineering teams and develops reference architectures for customers.
Failover Environment No two environments are the same
Environment Description Required Mirrored Storage Red Hat Enterprise Virtualization Manager Data Domain Optional Mirrored Storage ISO Domain Export Domain Non-Mirrored Storage DNS/DHCP server
Failing Over The Process
Failover Steps 1.Ensure storage domain data is available 2.Ensure name resolution for the RHEV Manager host name 3.Bring up RHEV Manager at second site 4.Add second sites hypervisors to the datacenter 5.Modify the storage configuration 6.Bring up VMs
Preparing Move storage domain data Mirrored data storage (Ideal Solution) Name resolution
Interfaces to Control Failover
Failover Methods Web User Interface (partial) REST API rhevm-shell rhevm-sdk-python rhevm-sdk-java Third Party Applications
Web User Interface (portal)
RHEVM-Shell Wrapper around the REST API calls Data is passed and returned in a human readable format Uses either UUIDs or Common Names to access resources Scriptable
RHEVM-Shell Setup $ cat /root/.rhevmshellrc [cli] autoconnect = True autopage = True [ovirt-shell] username = admin@internal timeout = -1 extended_prompt = False url = https://rdr-rhevm.sitea.example.org/api insecure = False filter = False session_timeout = -1 ca_file = /root/ca.crt dont_validate_cert_chain = False key_file = None password = password cert_file = None
RHEVM-Shell Example Interactive use of shell $ rhevm-shell [RHEVM shell (connected)]# add datacenter --name DC-iSCSI --storage_type iscsi --version-major 3 --version-minor 3 Non-Interactive use of shell $ rhevm-shell <<< "add datacenter --name DC-iSCSI --storage_type iscsi --version-major 3 --version-minor 3 --expect '201-created'"
RHEVM-Shell Example - Output id name status-state storage_type : ecd6d665-9488-40eb-808d-66ad12126c5e : DC-iSCSI : uninitialized : iscsi supported_versions-version-major: 3 supported_versions-version-minor: 3 version-major : 3 version-minor : 3
REST API Calls are made through a web portal as GET, PUT, POST, etc requests. Data is passed and returned in XML format Uses UUIDs to access resources Curl can be used to submit API requests Scriptable
REST API Example $ wget --no-check-certificate https://rdr-rhevm.sitea.example.org:/ca.crt $ curl --cacert ca.crt --user 'admin@internal:password' \ --header "Accept: application/xml" \ --header "Content-type: application/xml" \ --request "POST" \ --data '<data_center> <name>dc-iscsi</name> <storage_type>iscsi</storage_type> <version minor="1" major="3"/> </data_center>' \ https://rdr-rhevm.sitea.example.org/api/datacenters
REST API Example - Output <?xml version="1.0" encoding="utf-8" standalone="yes"?> <data_center href="/api/datacenters/1727e963-bd5c-4e43-99c5- d79fd1635103" id="1727e963-bd5c-4e43-99c5-d79fd1635103"> <name>dc-iscsi</name> <storage_type>iscsi</storage_type> <version major="3" minor="1"/> <supported_versions> <version major="3" minor="2"/> <version major="3" minor="1"/> <version major="3" minor="3"/> </supported_versions> <status> <state>uninitialized</state> </status> </data_center>
RHEVM software development kits rhevm-sdk-python, rhevm-sdk-java Libraries for interacting with the REST API
API Methods
Disaster Recovery APIs /api/storageconnections/connection_uuid path, iqn, address, port, user, password /api/networks/network_uuid name, description, ip, vlan, stp and display /api/hosts /api/hosts/host_uuid/nics/nic_uuid network, ip and boot_protocol
Update iscsi Storage Connection rhevm-shell [RHEVM shell (connected)]# update storageconnection e545772d-0e34-4db6-8e6f-db076ef8a139 --target iqn.2001-05.com.equallogic:0-1cb196-7de60c201-2e19b5f809c5310b-rdriscsi-b id : e545772d-0e34-4db6-8e6f-db076ef8a139 address: 172.31.143.200 port : 3260 target : iqn.2001-05.com.equallogic:0-1cb196-7de60c201-2e19b5f809c5310b-rdr-iscsi-b type : iscsi
Update iscsi Storage Connection REST API curl --silent --cacert rhevm.cer \ --user "admin@internal:2bor!2b" \ --header "Accept: application/xml" \ --header "Content-type: application/xml" \ --request "PUT" \ 1cb196-7c160c201-08f9b5f80995310b-rdr-iscsi-a</target> </storage_connection>" \ --data "<storage_connection> <target>iqn.2001-05.com.equallogic:0- https://rdr-rhevm.sitea.example.org/api/storageconnections/e545772d- 0e34-4db6-8e6f-db076ef8a139
Disaster Recovery Process
Failover Steps Bring up RHEV Manager Fix networking Keep original hostname Prevents need to recreate certificates Add hypervisors to RHEV Manager and approve them Modify storage connection using API Start the Vms Test for functionality
Mirror Data from Primary Site to Backup Site This is hopefully already done using mirroring If not, the data domain needs restored to the backup site Could take a while
Name Resolution Create a dns entry for the RHEV Manager This entry should resolve the original FQDN to an address on the backup site's network Will require zone for original site
Bring up the RHEV Manager Fix networking New IP Address (can be automatically taken care of with DHCP) May need to edit /etc/udev/rules.d/70-persistent-net.rules Keep original hostname to prevent the need to re-create certificates
Hypervisors Add the new hypervisors to RHEV Manager Add configure hypervisor nics if needed Power on the Vms Test the Vms for functionality
New Backup API Methods
Backup APIs /api/vms/vm_uuid/snapshots /api/vms/vm_uuid/snapshots/snap_uuid/disks /api/vms/vm_uuid/disks
Backup Process Create a snapshot of the VMs disk Attach the snapshot onto a backup appliance VM Backup the data on the disk Detach the snapshot from the appliance With the API, all this can be controlled via the appliance VM
Resources
Summit Sessions of Interest Configuring a disaster-resilient Red Hat Enterprise Virtualization environment Wednesday, 2:30-3:30 Red Hat Enterprise Virtualization: Overview & roadmap Wednesday, 3:40 4:40 Red Hat Enterprise Virtualization Hypervisor roadmap Thursday, 9:45 10:45 Red Hat Enterprise Virtualization deep dive Wednesday, 4:50 5:50
Summit Demos Symantec Tintri
Reference Architectures Red Hat Customer Portal Provides tar balls of scripts used in reference architecture if available https://access.redhat.com Support -> Knowledgebase -> Reference Architectures Red Hat Resource Library No scripts available http://www.redhat.com/resourcelibrary/reference-architectures/
Thanks Please complete the session survey