1 Live und in Farbe Live Migration André Przywara CLT 2010
2 Agenda (Live) Migration explained (Why? Limits!) Xen and KVM usage Details Memory synchronization QEMU device state transfer Host considerations (CPU features) Cross Vendor Migration QEMU block device transfer Project Remus (Xen) Demo!
3 Guest Migration move a virtual machine from one host to another offline: power down the guest, copy files, restart comparable to a reboot migration: halt the guest, copy state, wake up again minimal downtime live migration: copy state in background, switch at one (almost) no downtime at all
4 Reasons for migration Load balancing: freeing loaded hosts in favor of idle ones Upgrade / update / planned downtime migrate to a spare machine, rework the host, migrate back to the original one Roaming eternal desktop Uptime, uptime! desktop is running on a server, migrated to the respective client workstation Replacing older machines You name it!
5 How does it work? Host has full control over the guest Can read/write/protect memory Devices are (usually) also virtualized Host controls CPU usage similar to OS vs. application Steps: host de-schedules the guest host copies memory content over network host copies device state over network old host signals new host to take over
6 Limits of migration disk images should be accessible through a SAN, NAS, NFS can also be copied / synced (DRBD) no downgrade of CPU features maybe start with features disabled? No device pass-through Network connectivity must prevail Resources should match (memory, vcpus) Matching software versions (devices!)
7 Xen / KVM usage Xen: via xm tool $ xm migrate <domid> <newhost> xend must be running on both sides KVM: on target: $ qemu incoming tcp:0:<port> on source: via QEMU monitor (qemu) migrate tcp:<host>:<port> Need to have the exact same guest parameters on the command line (management app!)
8 Memory synchronization Problem: transferring RAM image takes time e.g.: 40 MB/s = 25 sec too long for live migration solution: start copying (in background) write protect already copied pages on page fault: allow r/w again, mark page as dirty repeat: copying dirty pages until no more left number of tries exhausted: halt guest and copy rest 1 2 3' 4
9 QEMU device state transfer QEMU devices used for Xen and KVM each device has a VMStateDescription describes the data that holds the complete state variables will be dumped to the stream contains version information (backward compatible) QEMU will iterate through all devices sends the device name and instance number executes a pre_save callback function dumps the device' state to the stream (TCP)
10 QEMU device state dump QEMU magic version stage section ID device name : d c : 6f 63 6b : d : : ff 00 f0 c3 e2 00 f : 53 ff 00 f0 53 ff 00 f0 53 ff 00 f0 53 ff 00 f0 version no. QEVM...bl ock......ram s... S...S...S...S... instance no.
11 QEMU block device transfer Recent QEMUs can transfer the block device No need for a shared storage (qemu) migrate -b tcp:<host>:<port> Can also migrate overlay only (-i) Uses same approach like RAM transfer Works like this: Transfer data in chunks of 1 MB Each chunk is preceded by a 64bit address Allows gaps Each chunk has the block device name in it
12 Project Remus (Xen) High availability using migration Continuously migrating the guest Avoids slowdown by snapshotting Only commits results when transmitted Snapshot frequency about every 25ms Running machine slightly in the past part of Xen 4.0
13 Host considerations Applications and libraries rely on a consistent set of CPU features (like SSEx) CPU instruction set may change at migration no downgrade! (loss of a feature) upgrade can be hidden (CPUID masking) least common denominator in a migration pool dictates the feature set of all guests KVM: use -cpu kvm64 migration pool should be well defined before starting the guest
14 Cross Vendor Migration Migrating from an Intel box to an AMD box (and vice versa ;-) allows for bigger migration pools avoids vendor lock in maps mostly to different CPU generations but subtle differences: x87 FPU rounding on some instructions (e.g. for fsin, deprecated) sysenter/syscall support in compat mode (emulation upstream) slightly different guest state checks (fixed) Model specific registers (MSRs) (fixed) Both Xen and KVM support it now!
15 Demo! Live! In Color! Using KVM (qemu-kvm , kernel ) Migration between servers, using VNC Windows XP 32 guest with running Passmark stress test application Between Intel C2Q and AMD Opteron
16 Demo screenshot
17 References Project Remus: Cross Vendor Migration: QEMU live migration:
Windows Server 2008 R2 Hyper-V Live Migration Table of Contents Overview of Windows Server 2008 R2 Hyper-V Features... 3 Dynamic VM storage... 3 Enhanced Processor Support... 3 Enhanced Networking Support...
Parallels Cloud Server 6.0 Installation Guide November 11, 2014 Copyright 1999-2014 Parallels IP Holdings GmbH and its affiliates. All rights reserved. Parallels IP Holdings GmbH Vordergasse 59 8200 Schaffhausen
Parallels Cloud Server 6.0 Readme September 25, 2013 Copyright 1999-2013 Parallels IP Holdings GmbH and its affiliates. All rights reserved. Contents About This Document... 3 About Parallels Cloud Server
ACTi Central Management System User s Manual Version 1.1.05.06 2012/10/12 Table of Contents 1 Overview 5 Software Architecture... 5 2 Installation 6 Requirements... 6 Operating System... 6 Hardware System
Remus: High Availability via Asynchronous Virtual Machine Replication Brendan Cully, Geoffrey Lefebvre, Dutch Meyer, Mike Feeley, Norm Hutchinson, and Andrew Warfield Department of Computer Science The
NVR 3.0 System Administrator s Manual For V3.0.02 Version 2013/03/21 About This Manual Target Audience This manual is intended for System Administrators who are responsible for installing and setting up
Parallels Cloud Server 6.0 Readme May 21, 2015 Copyright 1999-2015 Parallels IP Holdings GmbH and its affiliates. All rights reserved. Parallels IP Holdings GmbH Vordergasse 59 8200 Schaffhausen Switzerland
Parallels Cloud Server 6.0 Readme February 05, 2015 Copyright 1999-2015 Parallels IP Holdings GmbH and its affiliates. All rights reserved. Parallels IP Holdings GmbH Vordergasse 59 8200 Schaffhausen Switzerland
Parallels Deploying Microsoft Clusters in Parallels Virtuozzo-Based Systems Copyright 1999-2008 Parallels, Inc. ISBN: N/A Parallels Holdings, Ltd. c/o Parallels Software, Inc. 13755 Sunrise Valley Drive
HP StoreOnce Catalyst and HP Data Protector 7 Implementation and Best Practice Guide Release 1 Executive Summary This guide is intended to enable the reader to understand the basic technology of HP StoreOnce
Kerio Control Administrator s Guide Kerio Technologies 2011 Kerio Technologies s.r.o. All rights reserved. This guide provides detailed description on configuration and administration of Kerio Control,
Backup and Recovery of SAP Systems on Windows / SQL Server Author: Version: Amazon Web Services sap- on- firstname.lastname@example.org 1.1 May 2012 2 Contents About this Guide... 4 What is not included in this guide...
EaseUS Todo Backup User guide - 1 - Table of contents Welcome... - 5 - Overview... - 6 - How to use this manual... - 7 - New in EaseUS Todo Backup 6.5... - 8 - Getting started... - 9 - Hardware requirements...
INTRODUCTION TO LINUX CLUSTERING DOCUMENT RELEASE 1.1 Copyright 2008 Jethro Carr This document may be freely distributed provided that it is not modified and that full credit is given to the original author.
Allworx OfficeSafe Operations Guide Release 6.0 No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopy,
Trusted VM Snapshots in Untrusted Cloud Infrastructures Abhinav Srivastava 1, Himanshu Raj 2, Jonathon Giffin 3, Paul England 2 1 AT&T Labs Research 2 Microsoft Research 3 School of Computer Science, Georgia
Acronis Backup & Recovery 11.5 Update 2 Installation Guide Applies to the following editions: Advanced Server Server for Windows Virtual Edition Server for Linux Advanced Server SBS Edition Workstation
Basic System Administration ESX Server 3.0 and VirtualCenter 2.0 Basic System Administration Revision: 20090213 Item: VI-ENG-Q206-219 You can find the most up-to-date technical documentation at: http://www.vmware.com/support/pubs
LevelOne User Manual ACC-2000 KVM IP Console Module Ver. 1.1 1 / 87 Certificates Ver. 1.0.0-0709 FCC This equipment has been tested and found to comply with Part 15 of the FCC Rules. Operation is subject
Chapter 12 Disaster Recovery, Business Continuity, and Backup Disaster Recovery (DR) takes many forms, and the preceding chapter on DRLB covers a small part of DR. Actually, DRLB is more a preventative
RAIDXPERT USER MANUAL Version 2.1 RAIDXpert User Manual Copyright 2010 Advanced Micro Devices, Inc. All Rights Reserved. Copyright by Advanced Micro Devices, Inc. (AMD). No part of this manual may be reproduced