Cloud Computing with Red Hat Solutions Sivaram Shunmugam Red Hat Asia Pacific Pte Ltd sivaram@redhat.com
Linux Automation Details Red Hat's Linux Automation strategy for next-generation IT infrastructure includes: Red Hat Enterprise Linux Integrated Virtualization technologies Virtualization Management technologies Red Hat Enterprise MRG 3
The Concept of the CLOUD Web 2.0 Amazon EC2 Google App Engine Utility Computing Software-as-a-Service (SaaS) Hardware-as-a-Service (HaaS) So where is Red Hat? Partner Applications JBoss Middleware MySQL Database Red Hat Enterprise Linux CPU, RAM, DISK online 4
Red Hat Enterprise Linux 5 Latest generation of the world's leading commercial open source environment Comprehensive OEM and ISV certifications Leadership quality, performance and security Supports Intel & AMD 32-bit/64-bit, Itanium2, IBM Power and mainframe Includes virtualization for x86, x86-64 and Itanium2 architectures Includes Clustering and Cluster File Systems 5
Integrated Virtualization Based on Xen Hypervisor (RHEL) Embedded hypervisor based on KVM Hypervisor (ovirt) Almost native performance (para-virtualization) Integrated into the operating system Fully Supported end to end No Patch-lag No separate certifications for hardware and software Uses existing Linux devices drivers Dynamically allocate resources CPUs, memory, disks, networking Ability to Migrate guests between hosts (Live Migration) High Availability for Virtual Machines Shared Data for Virtual Machines / Physical Hosts 6
Performance and Scalability Red Hat provides the most scalable and high performance virtualization platform Para-virtualization (optimized kernel) provides best virtualization performance RHEL guests typically run 20% to 100% faster on RHEL than on traditional virtualization solutions Some workloads can see up to a 400% performance improvement Any workload can be virtualized even highly IO intensive eg, databases Full Virtualization RHEL provides enhanced drivers for fully virtualized guests Includes Linux and Windows virtual machines Para-virtualized drivers Significant performance increases with fully virtualized guests 7
Benefits of Red Hat Virtualization RHEL provides the most scalable and flexible Virtualization platform Allows organizations to virtualize all work loads Removes barriers to virtualizing high performance enterprise applications Eliminates the economic constraints on virtualization Virtualize any workload without added cost Reduces complexity Deploy a single platform that covers all your virtualization need Includes support for Windows XP, 2000 and 2003 Use consistent tools to deploy, manage and deploy your entire infrastructure 8
libvirt Provides a standard management interface Hypervisor agnostic Will work with multiple hypervisors Stable API Shield users from hypervisor changes Consistent tools across hypervisors eg. Same interface for Xen, QEMU, KVM, OpenVZ, etc Scriptable Provides APIs for developers / tool vendors Allows vendors to build cross platform tools 9
Virtualization Management Tools 10
Red Hat Network A systems management platform designed to provide complete lifecycle management of the operating system and applications. A single solution for lifecycle management of compute resources Installing and provisioning new systems Supports Physical and Virtual systems Updating/patching operating system Install and patch custom software Manage configuration files Monitoring performance Redeploying for a new purpose 11
ovirt project delivers complete virtualization solution Hypervisor Built on Linux kernel Managed using standards based tools Based on libvirt, CIM and LDAP From Red Hat, the community or ISVs Virtualization Management Platform Cross platform multiple hypervisors Built on open standards LDAP, Kerberos, Libvirt, etc Scalable From 1 node to tens of thousands of nodes From small host cluster to cloud computing infrastructure Secure Integrated policy and audit framework 12
Virtualization Alone is Not Enough: Challenges and Requirements for Today's Large-Scale Computing Most enterprises are running more and more distributed applications and workloads and dealing with an increasing amount of distributed data Many enterprises have increasing demands for computing power while simultaneously needing to cut down on physical resource consumption (e.g. power, space) Many enterprises have variable needs for their amounts of computing power Virtualization and consolidation can only get you so far if you have significant variability in computing requirements or server loads Some enterprises can never get enough computing capacity 13
Promises and Challenges of Cloud Computing Cloud computing promises to deliver many of the benefits of virtualization via an external provider Flexibility, scalability, on-demand utilization, efficiency, etc. Many companies are looking at a variety of cloud-based offerings to address their IT Infrastructure challenges Research: Folding@Home, BOINC, Fedora Nightlife Enterprise: Amazon EC2, Google App Engine There are important issues to consider when evaluating cloud infrastructure, including: Do I need to integrate the services with anything local in my datacenter or organization? Do I have any particular requirements about data security? How large are my datasets? Are there any issues with platform lock-in? Do I have any requirements or SLA's for how quickly my results return? Do I want to add capacity to my IT infrastructure or outsource my IT infrastructure? 14
About Red Hat Enterprise MRG Integrated platform for high performance distributed computing High speed, interoperable, open standard Messaging Deterministic, low-latency Realtime kernel High performance & throughput computing Grid scheduler for distributed workloads, 15
About MRG Grid Brings advantages of scale-out and flexible deployment to any application Delivers better asset utilization, allowing applications to take advantage of all available computing resources Dynamically provisions additional peak capacity for Christmas Rush -like situations Executes across multiple platforms and in virtual machines Schedules from sub-second jobs to long-running batch jobs Provides seamless and flexible High Throughput Computing (HTC) and High Performance Computing (HPC) across Local grids Remote grids Remote clouds (Amazon EC2) Cycle-harvesting from desktop PCs 16
MRG Grid is Based on Condor MRG Grid is based on the Condor Project created and hosted by the University of Wisconsin, Madison Red Hat and the University of Wisconsin have signed a strategic partnership around Condor: University of Wisconsin makes Condor source code available under OSI-approved open source license Red Hat & University of Wisconsin jointly fund and staff Condor development oncampus at the University of Wisconsin Red Hat and the University of Wisconsin's partnership will: Add enhanced enterprise features, management, and supportability to Condor and MRG Grid Add High Throughput Computing capabilities to Linux 17
Red Hat is Initially Adding to Condor: Enterprise Supportability Break out Condor from statically-linked blob to multiple well-maintained and individually patchable rpm's Web-Based Management Console Unified management across all of MRG for job, system, and workload management/monitoring AMQP Messaging Integration Enable job submission to Condor via AMQP Messaging clients Enable sub-second, low-latency scheduling for sub-second jobs Virtualization Support via libvirt Integration Support scheduling of virtual machines on Linux using libvirt API's Amazon EC2 Integration Enable automatic EC2 provisioning, job submission, results storage, teardown via Condor scheduler Runs as a job, so it can be a dependency for other jobs or executed based on rules (e.g. add capacity in EC2 if local grid out of capacity) 18
Automating Virtualization with Red Hat Enterprise MRG MRG's Grid Scheduler can automatically and fully manage virtual machines via libvirt: start, stop, migrate, etc Integrating virtualization with MRG enables enterprises to: Dynamically partition their data center based on workload priorities and load Drive higher utilization by leveraging Condor's Compute On Demand to schedule lower priority jobs onto idle servers Drive higher utilization by optimizing resource matching via virtualization. e.g. Schedule a Windows job onto a Linux resource Schedule virtual machines onto idle desktop workstations and live migrate them to other machines when the workstations are no longer idle Red Hat Enterprise MRG builds on Red Hat's integrated virtualization to provide the tools to build your own internal cloud and treat your IT infrastructure as a generic computational grid 19
MRG and Amazon EC2: The Most Flexibility and Best of all Worlds for Grid Deployments Transparently move between or blend local grids with cloud-based grids Red Hat Enterprise Linux application certifications hold whether local or in Amazon EC2 Red Hat Enterprise MRG provides a unified management interface across local grids, remote grids, and the cloud Dynamically add capacity in EC2 to dedicated grids where there is a need to maintain local infrastructure but there is a desire for flexible capacity e.g. Financial Services, Government, Healthcare Schedule entirely in EC2 through MRG when you only have occasional needs for grids. Easily add local capacity to your MRG Grid when necessary e.g. Internet, Manufacturing, Research No lock-in to local or cloud infrastructure 20
MRG and Amazon EC2 Demo Video 21
Enter SCS Grid... Red Hat is one of the integral platform members as part of the SCS-HP Grid Consortium As part of the consortium, we are bringing infrastructure solutions to the Grid Operating System platform Red Hat Network for systems management, provisioning and monitoring Applications through Partner ISV solutions Customer Relationship Management Retail Management System SMS Gateways More on the SAAS track tomorrow. 22
Enterprise Cloud Architecture Operating System Hardware Embedded or Virt Hypervisor Orchestration VM OS VM OS VM OS VM OS VM OS Grid Scheduler VM OS VM OS Management 23
Enterprise Cloud Architecture Operating System RHEL Condor RHEL JBoss MySQL Postgres Hardware RHEL / ovirt Embedded or Virt Hypervisor Orchestration AMQP + ovirt VM OS RHEL VM OS RHEL VM OS RHEL VM OS RHEL VM OS RHEL Grid Scheduler VM OS RHEL VM OS RHEL Management RHN 24
Summary Red Hat is leveraging Linux Automation to deliver next-generation IT infrastructure Includes Integrated Virtualization that provides a scalable and high performance virtualization platform Provides virtualization management tools and open API's with libvirt Provides grid scheduling capabilities for virtualization and cloud computing Run any application, anywhere, anytime Red Hat Enterprise Linux Virtualization: http://www.redhat.com/rhel/virtualization/ Red Hat Enterprise MRG: http://www.redhat.com/promo/mrg/ 25