1 Virtualization for Data Centers of Today & Tomorrow Jim Rymarczyk IBM Fellow, Chief Virtualization Technologist August 2008
2 IT Complexity the need for IT transformation
4 The IT Infrastructure Complexity and Cost Problem Business processes as services Topologies of federated services must be mapped onto large numbers of diverse physical and virtual resources Sea of heterogeneous servers, storage, networks and their virtualization Businesses spend a large fraction of their IT budgets on data center resource management rather than on valuable applications and business processes Enterprises report that IT operational overhead = 70% of their IT budget and growing... leaving precious few resources for new initiatives. Forrester, 2007
5 1 Today s Enterprise Application Complexity M a inframe apps - B lue PC/NT apps - G reen Unix apps - Y ellow 3rd party interface - Orange Lines: Colors have no special meaning. They are to help make the diagram easier to read. For More Information: See the database containing information about each application: Application V4.mdb AIS Reports AIS Calendar Vendor Setup Budget Analysis Tool Insertions Orders Vendor Maintenance Printer Maintenance Printer PO Process Servers (Imaging) S20-Sales Polling Application Diagram V4 NEW Soundscan Sterling VAN NPD Group Mesa Data Roadshow AIG Warranty Guard Mailbox (Value) I06 - Customer S01 - Sales Order Corrections I13- Auto Replenishment Depository Banks UAR - Universal Account Reconcilliation Page 1 of 2 I17 Customer Perceived In-Stock I15 Hand Scan Apps Due Dates Stores & Mrkts Broadcast Filter General Maintenance Invoice App Print Costing E13 E3 Interface Fringe PO I06 Warehouse Management Smart Plus Smart Plus Launcher M03 - Millennuim 3.0 S04 - Sales Posting P16 - Tally Sheet M02 - Millennium S07 - Cell Phones D01 Post Load Billing I03 Return to Vendor S06 - Credit App Stock Options L02-Resource Scheduling (Campbell) P15 EES Employee Change Notice Equifax A04 - Cust Refund Chks I12 Entertainment Software E01-EDI S09 - Digital Satellite System L01-Promo Analysis P14 On-line New Resumix Cobra CTS ACH Prodigy Banks - ACH and Pos to Pay Plan Administrators (401K, PCS, Life, Unicare, Solomon Smith Barney) E02-Employee Purchase Scorecard - HR P09 Bonus/HR Hire Entry P09 - P17 Cyborg Intercept V03- Mkt Reactions POS S08 - Vertex Sales Tax S03-Polling P01- Employee Masterfile Frick Co V04-Sign System Spec Source SKU Tracking NPD, SoundScan Spec Source V01-Price Management System AAS CTO2.Bestbuy. com U18 - CTO B01 - Stock Status I05 Inventory Info I04 Home Deliveries I09 Cycle Counts I10 Cycle Physical Inventory S02 - Layaways S11 - ISP Tracking I11 Price Testing L60 MDF Coop I01 PO Receiving V02-Price Marketing Support I02 - Transfers Supplier Compliance Washington, RGIS, Ntl Bus Systems X92-X96 Host to AS400 Communication SKU Performance SKU Selection Tool ASIS Store Monitor Texlon 3.5 U16-Texlon I35 - CEI Rebate Transfer ELT PowerSuite Sign System CopyWriter's Workspace I18 SKU Rep G02 - General Ledger B02 Merchandise Analysis I55 SKU Information K02 Customer Repair Tracking NARM I14 Count Corrections EDI Coordinator I07 Purchase Order I35 Early Warning System Valley Media Arthur Planning Ad Expense Store Scorecard Store Budget Reporting BMP - Bus performance Mngt Merch Mngr Approval Batch Forcasting Ad Measurement AIMS Admin AIMS A05 - AP Journal Entry Tool Kit Ad Launcher Data Warehouse (Interfaces to and from the Data Warehouse are not displayed on this diagram) AIMS Reporting PSP In-Home Repair Warranty Billing System Star Repair C02 - Capital Projects F06 - Fixed Assets Optika SiteSeer US Bank Recon File Connect 3 Connect 3 Connect 3 PDF Transfe Reports Cash Over/ Short S05 - House Charges ICMS Credit Cellular Rollover Cash Receipts/Credit OTHER APPS - PC AP - Collections/Credit TM - Credit Card DB Misc Accounting/Finance Apps - PC/NT COBA (Corp office Budget Assistant) PCBS(Profit Center Budget System) Merchandising Budget INVENTORY CONTROL APPS - PC Code Alarm Debit Receivings Devo Sales Display Inventory In Home Junkouts Merchandise Withdrawl Promo Credits RTV Accrual Shrink AP Research - Inv Cntrl AP Research-Addl Rpts Book to Perpetual Inventory Close Out Reporting Computer Intelligence Data Count Corrections Cross Ref for VCB Dnlds Damage Write Off Debit Receivings DFI Vendor Database Display Inventory Reconcil Display Inventory Reporting INVENTORY CONTROL APPS - PC DPI/CPI IC Batching Inventory Adj/Count Correct Inventory Control Reports Inventory Levels Inventory Roll Merchandise Withdrawl Open Receivings PI Count Results PI Time Results from Inv Price Protection Sales Flash Reporting Shrink Reporting SKU Gross Margin SKU Shrink Level Detail USM VCB Downloads ACCTS REC APPS - PC 990COR Bad Debt Benefical Fees Beneficial Reconcil JEAXF JEBFA JEBKA JEDVA JESOA JEVSA JEVSF NSF TeleCredit Fees Prepared by Michelle Mills Source: IBM client engagement experience Not shown: physical and virtual resources used
6 Basic Forces Will Drive Increases In IT Complexity The numbers of systems deployed will continue to grow rapidly, driven largely by: New applications (Web-based apps, surveillance, operational asset mgmt., etc.) Improving hardware price/performance The diversity of IT products will increase as competing suppliers continue to introduce new applications, systems, and management software products Today s innovations become tomorrow s legacy The coupling of IT components is extensive and increasing, driven by application tiering, advances in high-performance standard networks, The virtualization of resources will affect existing IT processes and can lead to virtual server sprawl Introducing virtualization can have significant hidden costs, and requires considerable skills, planning, and discipline
7 Virtualization its current and expanding IT role
8 Virtualization Functions and Benefits Virtual Resources Virtual Resources Sharing Aggregation Resources Examples: LPARs, VMs, virtual disks, VLANs Benefits: Resource utilization, workload manageability, flexibility, isolation Resources Examples: Virtual disks, IP routing to clones Benefits: Management simplification, investment protection, scalability Resource Type Y Virtual Resources Virtual Resources Emulation Insulation Resource Type X Resources Add, Replace, or Change Resources Examples: Arch. emulators, iscsi, virtual tape Benefits: Compatibility, software investment protection, interoperability, flexibility Examples: Spare CPU subst., CUoD, SAN-VC Benefits: Continuous availability, flexibility, software investment protection
9 Server Virtualization Hierarchy Virtual Resources Implementation Methods Examples System Stack Applications Virtual runtimes (application containers) Middleware provides JVM, J2EE, or CLR application containers Multiple middleware instances act as one WebSphere VE Microsoft and Sun JVMs BEA Liquid VM Middleware Operating Systems Virtual operating systems (application containers) OS creates virtual OS environment per app. Each container has its own name space, files, root, z/os Address Spaces AIX 6.1 WPARs Solaris Containers HP-UX Syst. Res. Part. MS SoftGrid Hardware Virtual servers (virtual machines / LPARs) Virtual I/O Virtual networks Virtual devices (CPUs, memory, I/O adapters, ) Hypervisors HW partitioning Virtual I/O Servers Self-virtualizing I/O adapters In-memory VLANs zseries PR/SM and z/vm Power Hypervisor HP vpars, Integrity VM Sun Logical Domains VMware, MS Hyper-V, Xen, KVM, Virtual Iron, There are three levels in the system stack at which virtualization is often done: (1) hardware virtualization, (2) OS virtualization, and (3) middleware virtualization Concurrent use of virtualization at these levels requires coordinated management
10 Virtualized IT Envisioned Benefits Future Virtualized IT IT Without Virtualization Web Servers Virtual Environment App Servers App/DB Servers Virtual resources are easier to deploy, grow, move, Virtual resources, configurations, and workloads are decoupled and insulated from physical environment End Users App Servers Virtual Virtual Virtual Application Application Application Server Server Server App Servers App/DB Server Virtual Clients Virtual Servers Virtual Storage Virtual Networks Virtualization IT Resources for a Medium Business Rigid configurations Fixed resources per server Low server utilization Wasted energy and floor space HW changes impact SW assets Servers managed individually Decouples Virtual and Physical Environments Physical Environment Physical resource changes can be made without impact to running IT workloads Improved HW utilization and energy efficiency Blades SMP Servers Storage Servers and Storage Network Hardware Virtualization technologies with emerging management software will significantly improve the efficiency, resiliency, responsiveness, and greenness of IT data centers.
11 Expanding IT Role Of Virtualization A new IT era is dawning in which virtualization will provide major new IT benefits The industry is investing to make this happen and it will unfold over the next 3-5 years The resulting high value to customers will render non-virtualized IT obsolescent Virtual Systems Virtual LANs Virtual Web Virtual Server Web Web Server Server Virtual App Server App Server Database Virtual Database Server Server Future: Virtual IT Configurations Virtual Disks Virtual resources are fully self-describing objects Adjustable dynamically Movable while active Durable over HW generations Storable and versionable in shared libraries Distributable as appliances Pools of physical systems are manageable as one system Emerging: Multi-System Virtualization Mgmt. Software Maturing: Physical Systems with Local Virtualization V SMP Servers Virtual Servers Blades Virtual Storage Storage Servers and Storage Virtual Networks V V V Network Hardware Emerging Benefits: Better SW investment protection Simplified HA\DR/PD solutions Improved resource optimization Ready-to-run packaged software Provides virtual resource mgmt. and virtualization-based solutions spanning physical resource pools Today s Benefits: Better hardware utilization Improved IT agility Lower power consumption
12 Stages of IT Infrastructure Transformation Strategic Imperatives Evolve data centers from ad-hoc to well-architected Better manage the IT complexity end-to-end service mgmt Reduce the IT complexity to be managed via ensembles Virtualization has major benefits but introduces more complexity Abstraction and Pooling Virtual Resource Objects (Servers, Storage, Networks) Integration and Simplification New Enterprise Data Center Service Mgmt. Green IT Ensemble Ensemble SOA Physical Consolidation Local Virtualization V Mainframe or Unix Server V Linux Server Windows Server Networks Storage Better hardware utilization Improved IT agility Lower power consumption V V V Multi-System Virtualization V Servers Resource Pools V Networks V Storage Better SW investment protection Simplified HA solutions Improved resource optimization Ready-to-run packaged software Resilience Ensemble TVDC Cloud Computing Reduced scale-out complexity Integrated autonomic mgmt Dynamic energy optimization Business resilience foundation Continuing Advances Continuing Advances (I/O, network, resilience, performance, )
13 Applying Simplification First Principles to IT Virtualize as many resources as possible Allows the elimination of dependencies between virtual and physical resources Modularize the data center decouple resources and management roles by type Decouple servers, storage, and network controllers Decouple server management from storage management Decouple data management from physical storage management Provide compatibility across families and generations of hardware resources Enable virtual resources to function unchanged despite changes to underlying hardware Group physical resources into pools of like types Provide virtual resource mobility within these pools Make groups of resources look like single resources to users and resource managers Hide their internal design complexities and their parallelism Provide a single system management image per pool Apply the first principles to the management software itself Decouple management of virtual resources from management of physical resources Use hierarchy of management software
14 Key Complementary Methods for IT Simplification 1 Service oriented architecture technologies frame business processes as services, facilitating deployment, composition, reuse, modular change, agility, efficiency, Business Processes as Services 2 Service management software spans the diversity of heterogeneous physical and virtual resources, providing unified cross-platform management in support of SOA Services registry, service life cycle mgmt., image libraries, multi-tier composition, provisioning, autonomic optimization, network services, security services, Sea of Heterogeneous Servers, Storage, Networks and Their Virtualization Ensemble Ensemble E E 3 Ensembles and scalable servers replace multitudes of individual servers An ensemble is a pool of like systems that is manageable as a single system It integrates compatible networked systems, virtualization, and management functions It scales from few to many servers, while having management complexity and cost like that of a single system essentially independent of the ensemble size
15 Ensembles a new integrated systems approach
16 Ensemble Components Server Ensemble Example OS OS OS Hypervisor Server Workload Mobility Ensemble Manager OS OS OS Hypervisor Server Ensemble definition: a pool of like systems that is manageable as a single system An Ensemble generally consists of the following components: A pool of compatible system nodes (e.g., N physical servers; need not be homogeneous) Virtual resource mobility within an ensemble and with compatible ensembles The networks which interconnect the ensemble nodes (may be local / optimized) Resource virtualizers (hypervisors, I/O virtualizers, storage virtualizers, ) An ensemble manager appliance that provides platform management for the ensemble virtual and physical resources Tools for planning, ensemble creation, P2V migration, image mgmt. & composition, Ensemble-local automated optimization software of performance, availability, energy usage, security, with intelligent defaults Multi-system services (locking, caching, message queuing, ) may be integrated with some ensembles
17 The Benefits of Server Ensembles Individual Servers OS OS OS OS OS OS OS OS OS Server Ensemble Workload Mobility OS OS OS Hypervisor Server Hypervisor Server Hypervisor Server Ensemble Manager Hypervisor Server # of Things to Manage Create, Test, and Maintain Management Automation Management Interfaces # of Consoles Data Center Mgmt. Arch. N virtual servers; M physical servers Do it yourself; few assemblies are alike Add-on software, custom scripts, Many individual knobs and variables Separate consoles for physical & virtual Monolithic; spans d. c. heterogeneity N virtual servers; one physical ensemble Standard off the shelf assemblies Built-in optimizations; intelligent defaults Menus of selectable standard behaviors Single console; in-context functionality Hierarchical; pool-level modularity Ensembles will significantly reduce IT resource management complexity and cost, and improve IT functionality, in consumable data center increments
18 Cloud Computing the next phase of Internet-based sharing
19 What Is Cloud Computing? Cloud computing is an emerging style of computing in which applications and data are provided as services to users over the Web. The services provided can be available globally, always on, low in cost, on demand, massively scalable, pay as you grow, Consumers of the services need only care about what the service does for them, not how it is implemented Monitor & Manage Resources Datacenter Infrastructure Access Services Service Consumers Cloud Administrator IT Cloud Examples: Amazon Elastic Compute Cloud EC 2 IBM Research Compute Cloud RC 2 Component Library Publish & Update Components Component Vendors / Software Publishers
20 Evolution of Sharing on the Internet Services Sharing Resource Sharing Cloud Computing Everything as a service over the Web: SaaS, utility computing, IT services, Ubiquitous and always available Grid Computing Standards and software for sharing of remote resources and collaboration Mainly used for highly scalable HPC jobs The Grid Information Sharing The World Wide Web HTML page format, HTTP protocol, and Mosaic browser for document exchange Initially in universities; worldwide adoption Network Sharing Inter-Networking and the Internet Inter-Networking of regional networks with TCP/IP Began to replace regional alternatives Worldwide adoption Networking Networks Multiple regional networks linking computers Initially at universities and national labs
21 Cloud Computing Examples Amazon EC 2, S3 Xen / SLES based; 3 basic virtual machine sizes; various applications and application build tools Google Proprietary; virtualization in OS image IBM / Google / Hadoop Cloud Google Map/Reduce application on different infrastructure IBM Research Compute Cloud (RC 2 ) Self-service, Web-based cloud for computer science research Includes System p and System x resources University of North Carolina, Virtual Compute Lab Xen / Linux based; used for teaching and computer science research
22 The IBM HiPODS Team Building Clouds Today
23 Research Computing Cloud RC 2 - Overview A living lab to advance research strategies Provides self service on demand delivery solution for research computing resources Integrates existing assets and products using SOA RC 2 Self Service Portal Business Process Workflow Application Business Process Workflow Mgmt Leverages Tivoli products Zero touch support for the full life cycle of service delivery Order creation Approval process notification Automated provisioning Monitoring Capacity Mgmt Provisioning Metering & Monitoring & Orchestration Rating Virtualized Infrastructure
25 Summary IT data center complexity and costs are continuing to increase and have already reached a crisis level A breakthrough in data center architecture and management is needed Virtualization will become pervasive and play a vital role in enabling the required data center transformations Today it improves hardware utilization, energy efficiency, and agility but it impacts most IT processes and introduces new complexities / costs Going forward, virtualization will be used to reduce management costs, modularize data center architecture, and facilitate ongoing improvement Integration will be key to achieving IT simplification Server, storage, and network ensembles will be used as pre-built scalable modular data center building blocks Cloud Computing will become widely used for IT services delivery globally across the Web and locally within enterprise data centers It will yield a range of attractive Web-based services platforms
26 IBM s Vision: The New Enterprise Data Center An evolutionary new model for efficient IT delivery New economics: Virtualization with optimized systems and networks to break the lock between IT resources and business services Rapid service delivery: Service management enables visibility, control and automation to deliver quality service at any scale Aligned with business goals: Real-time integration of transactions, information and analytics and delivery of IT as a service
27 New Enterprise Data Center Stages of Adoption Green Physical consolidation and optimization Business Resiliency / Security Virtualization of individual systems Systems, network and energy management Service Management Highly virtualized resource pools ( ensembles ) Integrated IT service management Green by design Virtualization of IT service Business-driven service management Service-oriented delivery of IT
February 2009 Seeding the Clouds: Key Infrastructure Elements for Cloud Computing Page 2 Table of Contents Executive summary... 3 Introduction... 4 Business value of cloud computing... 4 Evolution of cloud
An Oracle White Paper May 2010 Oracle Cloud Computing 1 Executive Overview Cloud computing is a significant advancement in the delivery of information technology and services. By providing on demand access
W H I T E P A P E R Cloud Computing Raising Geospatial Technology to the Cloud: Intergraph Strategy for Leveraging Cloud-based Resources Contents 1. Introduction... 1 2. Cloud Computing... 2 2.1. Clustering...
Sun Cloud Computing TABLE OF CONTENTS Table of Contents Cloud Computing at a Higher Level............................................... 4 Why Cloud Computing?.......................................................
IT@Intel White Paper Intel Information Technology Business Solutions June 2010 An Enterprise Private Cloud Architecture and Implementation Roadmap The private cloud is a shared multi-tenant environment
Cloud Computing Tutorial CLOUD COMPUTINGTUTORIAL by tutorialspoint.com tutorialspoint.com i ABOUT THE TUTORIAL Cloud Computing Tutorial Cloud Computing provides us a means by which we can access the applications
Cloud optimize your business Windows Server 2012 R2 Published: October 7, 2013 Contents 1 Trends 3 Windows Server: cloud optimize your business 5 Windows Server 2012 R2 capability overview 5 Server virtualization
RESPONSIVE ENTERPRISE COMPUTING SERVICES Cloud Solutions A Silver Lining for Intelligence Missions Salient Federal Solutions Proprietary Information For comments or questions regarding this white paper,
Smart IBM Solutions for High Performance Engineering Clouds: Advanced Performance for Complex Engineering Problems Delivered in a Flexible and Secure Mode through Private and Private-Hosted Clouds. Sponsored
Clouds IT Microsoft Private Cloud- Making It Real Contents Copyright information... 3 The Microsoft Cloud OS... 4 The Private Cloud in your datacenter.... 5 Windows Server 2012 built from the cloud up...
BUILT FOR THE FUTURE, READY NOW. Clouds Microsoft Private Cloud: Evaluation Guide Copyright Information 2013 Microsoft Corporation. All rights reserved. This document is provided "as-is." Information and
Introduction to Cloud Computing architecture White Paper 1st Edition, June 2009 Abstract Cloud computing promises to increase the velocity with which applications are deployed, increase innovation, and
Red Hat Cloud Foundations Reference Architecture Edition One: Private IaaS Clouds Version 1.0 April 2010 Red Hat Cloud Foundations Reference Architecture Edition One: Private IaaS Clouds 1801 Varsity Drive
Front cover IBM SmartCloud: Building a Cloud Enabled Data Center Redguides for Business Leaders Pietro Iannucci Manav Gupta Learn how to choose the infrastructure as a service (IaaS) solution that best
Iron Networks Microsoft Fast Track Architecture Reference Architecture Technical White Paper IronPOD Product Family Prepared by Iron Networks Engineering Robert Keith Revision 1.0.9 Published April 2014
Front cover IBM Data Center Networking Planning for Virtualization and Cloud Computing Drivers for change in the data center IBM systems management networking capabilities The new data center design landscape
Cloud Computing Payback An explanation of where the ROI comes from November, 2009 Richard Mayo Senior Market Manager Cloud Computing email@example.com Charles Perng IBM T.J. Watson Research Center firstname.lastname@example.org
Analysis of Provisioning Dependable Spot Virtual Machines in Cloud Environments RamSanthosh Gopala Krishna Guptha MSc Computing and Management Session 2011/2012 The candidate confirms that the work submitted
Front cover IBM SmartCloud: Becoming a Cloud Service Provider Redguides for Business Leaders Joe McIntyre Learn about the IBM CCRA Cloud Service Provider adoption pattern Understand the key cloud service
CIO White Paper April 2009 Staying aloft in tough times Why smart, innovative businesses are turning to cloud computing Table of Contents 3 Introduction Cloud computing a smarter way to work 4 Innovation
Masaryk University Faculty of Informatics Master Thesis Database management as a cloud based service for small and medium organizations Dime Dimovski Brno, 2013 2 Statement I declare that I have worked
Cloud Computing: Paradigms and Technologies Ahmed Shawish and Maria Salama Abstract Cloud Computing has recently emerged as a compelling paradigm for managing and delivering services over the internet.
Price. Performance. Advantage. Red Hat Enterprise Linux on the HP ProLiant DL980 Server 2 introduction 2 red hat enterprise linux and the hp proliant dl980 4 HP ProLiant DL980 G7 servers 4 Red Hat Enterprise
Enterprise Cloud Infrastructure FOR DUMmIES ORACLE SPECIAL EDITION by Michael Wessler, OCP & CISSP Enterprise Cloud Infrastructure For Dummies, Oracle Special Edition Published by John Wiley & Sons, Inc.