White Paper PROSPHERE: DEPLOYMENT IN A VITUALIZED ENVIRONMENT Abstract This white paper examines the deployment considerations for ProSphere, the next generation of Storage Resource Management (SRM) from EMC. The use of VMware tools to deploy and manage ProSphere is discussed, as well as the advantages of using a virtual application. August 2011
Copyright 2011 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. VMware is a registered trademark of VMware, Inc. in the United States and/or other jurisdictions. All other trademarks used herein are the property of their respective owners. Part Number h8896 2
Table of Contents Executive Summary... 4 Audience... 4 Overview and Terminology... 4 Virtualization Advantages... 5 of ProSphere... 6 Prerequisites... 7 Process... 9 Artifacts... 11 Updating ProSphere... 13 Conclusion... 15 References... 16 3
Executive Summary The desire to contain IT management costs is one of the motivations for the spread of virtualization across the network, application and compute stack. The ability to deploy more applications with less hardware and physical space results in lower capital and operational expenses. With the maturity of virtualization technologies comes the improvement of tools to manage the entire lifecycle of applications, from deployment to retirement. Advances in hardware and software have also improved the performance of virtualized applications when compared to physical deployments, making the virtualization decision even more compelling. The time has come for a new variety of management software for corporations of all sizes that takes advantage of the strengths of virtualization and has the ability to manage both the traditional physical environment, as well as virtualized cloud infrastructures. EMC ProSphere is a new class of Storage Resource Management software with a deployment architecture specifically targeted to leverage virtualization advances and dramatically lowering the cost of ownership. Audience The intended audience for this document is anyone interested in deploying ProSphere (e.g. system and storage administrators), system implementers (e.g. EMC certified Solutions Architects), and EMC partners. To help with comprehension of the product architecture, the reader should review the ProSphere Overview white paper, prior to reading this paper. Overview and Terminology Before embarking on a discussion of deployment, a foundation in terminology is necessary. Industry and product virtualization terminology is often used inconsistently, which can quickly lead to confusion. This paper will refer to the following terms throughout. The references listed at the end of this white paper are good sources for additional information. Virtual Machine (VM) A VM is a computer environment that instead of being implemented in hardware, like a physical computer, is implemented in software. A Virtual Machine runs its own operating system and applications. Multiple virtual machines can concurrently operate on the same physical host system. vapp vapps are typically complex and multi-tier applications. A vapp is a logical entity that represents one or multiple virtual machines. vapps are typically created to provide 4
complete software capability. They are self-documenting and are manageable by software provided on the virtualization platform. Open Virtualization Format (OVF) The Distributed Management Task Force (DMTF) OVF specification, co-authored by industry virtualization leaders, including VMware, describes an open, secure, portable, efficient and extensible format for the packaging and distribution of software to be run in virtual machines. Just Enough Operating System (JeOS) JeOS is a deployment of an operating system with a limited set of packages. The deployed software provided is explicitly included to meet the intended need of the application that is deployed on the host. RPM Package Manager (RPM) RPM provides the system administrator with the ability to manage a related group of objects with ease. Among other capabilities, RPM provides the administrator with the ability to easily install, erase, upgrade, and verify computer packages. A software package consists of an archive of files, along with information about the package, such as its name, version, release, and package architecture. Virtualization Advantages The following lists some of the key advantages of virtualization of the hosts that run ProSphere. Time to value By using a virtualized infrastructure, ProSphere can be quickly deployed with minimal intervention and preparation, dramatically decreasing the deployment timeline. This provides value from the product as quickly as possible. Simplified management Management of ProSphere is far simpler on a virtualized platform than on a nonvirtualized platform. Control over the virtualized resources is centralized within vsphere, allowing it to manage allocation of these resources for the guest operating systems. For example, changes to resources like CPU and memory utilization are as simple as a mouse click. Updates to virtual machines are easily performed with controlled and auditable updates, using standard packaging software such as RPM. Backup and recovery of a vapp can be accomplished with all the native and third party tools available for a vsphere deployment. Snapshots allow the system to be 5
backed up while the VM is powered on. The VM can then be restored, if necessary, due to failure or the need to rollback to an earlier release. In addition, due to vmotion capabilities that VMware provides, migration of ProSphere to a new physical infrastructure or location is easily accomplished within vsphere. Lower development, distribution and support costs The VMs are packaged with a particular version of the operating system and distributed, maintained, updated and managed as a unit. Because of this packaging, the ProSphere application runs in a completely controlled environment. This protects it from adverse interactions with unrelated applications. In particular, this type of deployment isolates ProSphere from changes made in different versions of an operating system than the one on which it was created and tested. The potential for problems due to these interactions is greatly diminished and by reducing problems found in the customer environment, the cost of ownership is greatly decreased. Standardization of deployment improves the quality of the product due to the identical deployments of the product across customers. Fewer variables in the deployed solution (e.g. hardware, vsphere/esx versions) reduce the potential for defects. Due to the controlled operating environment, all application conflicts are worked out during the product development cycle, as the virtual machine is a closed system. This means that once deployed, there is no ability for third party software to be installed. of ProSphere ProSphere is packaged as a vapp, containing three virtual machines and configured for deployment using the vapp s Open Virtualization Format (OVF). ProSphere is deployed through the vsphere administrator interface using the OVF file, which describes the vapp. The initial deployment of ProSphere consists of these virtual machines: ProSphere Appliance Historical Database Discovery Engine The ProSphere Appliance is the foundation of the system. All of ProSphere s application components and discovered information are contained in this VM. The Historical Database hosts the EMC Greenplum database, which holds the historical performance data used to populate dashboards and performance charts. This same database instance is extendible and is capable of hosting other historical information. 6
The Discovery Engine leverages agent-less technology to discover a customer s environment and collect performance information. It utilizes a variety of techniques to gather required data. These mechanisms are lightweight, typically considered less intrusive than agent based discovery systems and where possible standards-based. Agent-less technology enables quicker time to value. It also reduces the amount of time customers need to spend managing ProSphere. For more details, please refer to the white paper: ProSphere Discovery and Monitoring for the Modern Datacenter. In some very large data centers, additional discovery engines are required. In order to meet the scalability needs of these data centers, a separate OVF template has been created to allow for easy deployment of additional Discovery Engine VMs. Figure 1 - Architecture Prerequisites This section contains requirements for installation of ProSphere, as well as some information that should be understood at the outset of a ProSphere installation. At a minimum, a vsphere 4.0 and ESX/ESXi 4.0 (or higher) environment is required for deployment. It is not possible to deploy the ProSphere vapp directly to an ESX server as only vsphere itself has the ability to deploy from an OVF template. To maintain consistency and time synchronization, all ESX servers should be configured to use the same Network Time Protocol (NTP) server. 7
VMware has made the creation of VMs easy, by providing a wizard to handle configuration of the VM. It prompts the user to supply primarily networking information. Each of the ProSphere VMs needs to be provided with an IP address (along with a gateway and network mask). This address should have a corresponding hostname registered in the DNS prior to deployment. It is important that the IP addresses not be reused. In order to avoid this, once used, they should be removed from the resource pool in which they are defined. Figure 2 - Deploy OVF screen Each VM has specific CPU, memory and disk requirements predefined in the OVF template. The explicit resource requirement information is specified in the ProSphere 8
Preparing for document. The number of CPUs (i.e., 2 or 4) and amount of memory required (i.e. 4GB - 8GB) varies by VM. The disk requirements also vary (i.e., 20GB for the Discovery Engine, and 200-500 GB for the ProSphere Appliance and Historical Database VMs). The values for each of these parameters are specified in the ProSphere OVF template. These values were determined based on a significant amount of ProSphere testing. Actual disk usage will vary based on the size of the data center environment managed, as well as the amount of data collected. The specified requirements should satisfy the majority of environments. Whether ProSphere is deployed to a single physical ESX Server or an ESX cluster, the VMware environment being used should have the required resources available prior to deployment. All credentials for the SAN elements to be discovered should be readily available so that access profiles and discovery policies can be configured immediately after deployment completes. The elements to be managed should be properly configured (e.g. WMI for Windows and SSH for Unix and Linux) with the appropriate software (e.g. packages with sar and iostat commands for performance data) deployed. For SAN discovery, the HTTP or HTTPS credentials for the Brocade SMI agent, Connectrix Manager Data Center Edition, or Connectrix Manager Converged Network Edition should be available. Cisco discovery uses SNMP and the v1/v2/v3 credentials will be needed. EMC storage arrays are discovered using the Solutions Enabler SMI-S Provider, which should be deployed and configured to manage the arrays prior to the deployment process. The host IP address, access port, username, and password for the provider are required to configure discovery by ProSphere. Process The deployment process for ProSphere is quite simple. At a high-level, the user needs to: Navigate to the Deploy OVF template option of the vsphere Client UI Specify the location of the ProSphere OVF template and its related, co-located VMDK files Supply the wizard dialog variables Start ProSphere The ProSphere OVF and VMDK files are downloaded from EMC Powerlink. They can be presented several ways to the vsphere client (e.g. using a local file share or a web server URL). Once the files are accessible, the deployment of ProSphere begins by selecting File and Deploy OVF template in the vsphere UI. 9
Figure 3 - Start of deployment Once the OVF template location and the required details are entered, the deployment process begins. The information that needs to be entered includes: The ESX Host or Cluster where ProSphere will be deployed The Resource Pools to be used The Datastore where the VM files will be stored Choice of thick or thin provisioned disks (thick disks will provide better performance, although thin provisioned disks have the advantage of unattended expansion). Networking information for each virtual machine The deployment process copies the OVF template and Virtual Machine Disk Format (VMDK) files to the target ESX environment. time will vary based on network bandwidth and the ESX computing resource availability. It is suggested that these be kept in mind before deployment. In all but the extreme cases, deployment should complete within an hour. A manual startup of the vapp is required. Each VM may be started individually; however, it is easier to start the vapp. The VMs can start in any order. Once the deployment is complete, the vapp is available for use. One can test the vapp by clicking on the available link in vsphere Client UI or by entering the host name of the ProSphere Application in a browser. The ProSphere login page should appear. 10
Artifacts This section is provided to explain how the ProSphere vapp is defined and the significance of OVF template in this process. This is helpful in providing an understanding of the significance of resource definitions, how the resources are bound to the application prior to execution and how these definitions can be changed. VMware Studio is a virtual appliance that helps transform software applications into virtual machines. This build process generates the virtual machines that comprise the ProSphere application. These virtual machines are based on build profiles that specify the Application Packages and ISO image of the software to be installed. The VM requirements (e.g. CPU and memory) are defined at build time and embedded in the VM image. There is no option in the deployment process to change required resource parameters (e.g. number of vcpus). If these parameters need to be changed due to the extensive size of the data center, they must be changed when the VM is powered off. Figure 4 - VMware Studio appliance packaging VMware Studio produces virtual appliances in OVF, the packaging format used for ProSphere. The OVF file contents are updated at deployment time based on the 11
information provided by the installer (e.g. hostname) and as determined by the VMware application management instrumentation (VAMI). The following part of this section contains some subsections of the OVF file s contents. They are provided to serve as simple examples of the type of information contained in the OVF. The file used to deploy the ProSphere vapp is more complicated than the subsections shown here (and consequently is much larger). <PlatformSection> <Kind>VMware ESXi</Kind> <Version>4.1.0</Version> <Vendor>VMware, Inc.</Vendor> <Locale>en</Locale> </PlatformSection> <PropertySection> <Property oe:key="storageresourcemanager.version" oe:value="1.0.0.0.1708"/> </PropertySection> <ve:ethernetadaptersection> <ve:adapter ve:mac="00:50:56:af:0c:36" ve:network="lglor"/> </ve:ethernetadaptersection> In the above example: PlatformSection contains the specific version of the virtualization platform and the vendor name. PropertySection includes run-time provided properties. The ProSphere application version is displayed. The oe:key attribute provides the name of the attribute field (StorageResourceManager.Version). The oe:value is the ProSphere version. EthernetAdapterSection Other values are set at deployment time (e.g. the VM s MAC address). As mentioned above, the OVF template is used by vsphere to deploy the VM. The configuration details contained in the template are used for deployment only and are not used again after the VM is created. For example, the Item element below is part of the VirtualHardwareSection element. This XML element states that this particular virtual machine requires four virtual CPUs. This information is used by vsphere to allocate the correct type and quantity of resources during deployment. 12
<Item> </Item> <rasd:allocationunits>hertz * 10^6</rasd:AllocationUnits> <rasd:description>number of virtual CPUs</rasd:Description> <rasd:elementname xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cimschema/2/cim_resourceallocationsettingdata">4 virtual CPU</rasd:ElementName> <rasd:instanceid xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cimschema/2/cim_resourceallocationsettingdata">1</rasd:instancei> <rasd:reservation>0</rasd:reservation> <rasd:resourcetype>3</rasd:resourcetype> <rasd:virtualquantity>4</rasd:virtualquantity> <rasd:weight>5000</rasd:weight> In this section, the key artifacts used when a version of ProSphere is deployed were reviewed. In the next section, we will overview how versions of ProSphere can be updated, using new deployments of these artifacts. Updating ProSphere A ProSphere vapp may be updated using the VMware vcenter Update Manager. This software provides an easy to use method for distributing ProSphere enhancements and patches from a centralized location. Use of the Update Manager to implement patch management ensures that the data center can keep the cost of ownership low, while at the same time ensuring deployments are secure and in compliance. Patch management assessments and remediation can be automated for both online and offline virtual machines. This provides a higher level of compliance to patch standards than would be available in more traditional deployments. The remainder of this section will outline the update process. 13
Figure 5 - Update Model The vsphere Client is used to connect to the vcenter Server instance that needs to be updated. Then the user navigates to the Update Manager Tab. Figure 6 - Update process Update files consist of a set of Linux RPMs and an associated manifest that lists the files and their versions. They may be downloaded directly from EMC Powerlink, hosted locally or mounted as an image directly to the VM. Once the update location is set, Update Manager can then scan the state of the deployed ProSphere VMs, as well as the server s guest operating systems (VM 14
running its own operating system), compare the list of updates that it has available to the deployed packages and determine what, if any, update is available. The Update Manager UI gives administrators visibility into the patch status of the entire deployment. By navigating to the update page on each VM, the status of each VM can be viewed. This capability dramatically simplifies the patch management process, while helping protect the data center against bugs and security vulnerabilities. Contrary to some expectations, there is no requirement to have a VMware administrator perform the updates. This makes the process as simple and quick as possible. Updates can be scheduled or performed through a user-initiated process. Users performing this task can do so from the Update Manager tab, choosing remediate on the objects that need updating. If the process is to be scheduled, this is done from the Scheduled Task pane. It is important to remember that all VMs of the vapp must be updated using the Update Manager process. Conclusion ProSphere deployment provides a simple and fast deployment experience to enable the customer to achieve value out of the product as quickly as possible. Advances in virtualization technology and standards like OVF allow ProSphere to be easily deployed, configured and migrated. By using standard operating system and packaging software, these applications are updated quickly and simply, enabling patch level and security compliance to be attained with minimal intervention. The ability to deploy and update the ProSphere vapp is now in the hands of users of various skill levels, which helps to reduce the operational expenses of the organization. The capital expenses of a ProSphere deployment are also reduced as physical hardware can be shared across multiple applications including ProSphere, which requires only a virtualized environment and no dedicated hardware. 15
References DMTF http://www.dmtf.org/ Open Virtualization Format (OVF) - http://www.vmware.com/appliances/gettingstarted/learn/ovf.html RPM - http://rpm.org/ SSH - http://en.wikipedia.org/wiki/secure_shell SMI-S - http://www.snia.org/tech_activities/standards/curr_standards/smi SNMP - http://en.wikipedia.org/wiki/simple_network_management_protocol VMware Glossary - http://www.vmware.com/pdf/master_glossary.pdf VMware vsphere - http://www.vmware.com/products/vsphere/overview.html VMware vcenter Update Manager - http://www.vmware.com/products/updatemanager/overview.html WMI - http://msdn.microsoft.com/en-us/library/aa394582(v=vs.85).aspx WS-Management - http://www.dmtf.org/standards/wsman 16