Lab Validation Report EMC ProSphere End-to-End Cloud Storage Resource Management By Vinny Choinski with Bob Laliberte January 2012
Lab Validation: EMC ProSphere 2 Contents Introduction... 3 Background... 3 EMC ProSphere... 4 ESG Lab Validation... 5 Getting Started... 6 Configuration and Discovery... 9 Performance Monitoring and Alerting... 12 ESG Lab Validation Highlights... 15 Issues to Consider... 15 The Bigger Truth... 16 Appendix... 17 ESG Lab Reports The goal of ESG Lab reports is to educate IT professionals about emerging technologies and products in the storage, data management and information security industries. ESG Lab reports are not meant to replace the evaluation process that should be conducted before making purchasing decisions, but rather to provide insight into these emerging technologies. Our objective is to go over some of the more valuable feature/functions of products, show how they can be used to solve real customer problems and identify any areas needing improvement. ESG Lab's expert third-party perspective is based on our own hands-on testing as well as on interviews with customers who use these products in production environments. This ESG Lab report was sponsored by EMC. All trademark names are property of their respective companies. Information contained in this publication has been obtained by sources The Enterprise Strategy Group (ESG) considers to be reliable but is not warranted by ESG. This publication may contain opinions of ESG, which are subject to change from time to time. This publication is copyrighted by The Enterprise Strategy Group, Inc. Any reproduction or redistribution of this publication, in whole or in part, whether in hard-copy format, electronically, or otherwise to persons not authorized to receive it, without the express consent of the Enterprise Strategy Group, Inc., is in violation of U.S. Copyright law and will be subject to an action for civil damages and, if applicable, criminal prosecution. Should you have any questions, please contact ESG Client Relations at (508) 482.0188.
Introduction Lab Validation: EMC ProSphere 3 This ESG Lab Validation report documents the results of hands-on testing of the EMC ProSphere storage resource management solution. Testing explored how ProSphere can help simplify management, identify problem areas, and ensure application performance in otherwise complex storage environments. The report focuses on the easy deployment, agent-less discovery, and end-to-end storage environment visibility elements of ProSphere. Background Organizations need to be able to respond to rapidly changing market dynamics, customer requirements, and competitive threats. Many are building out private cloud infrastructures in order to achieve the requisite level of agility. One of the foundational technologies of a private cloud is server virtualization, so it shouldn t be a surprise to learn that increasing the use of server virtualization technology has been the top IT priority for the last two years in a row according to ESG research. 1 This, in turn, is driving increased growth in networked storage environments, which are critical to enabling virtualized deployments to be highly mobile and available. In fact, ESG research indicates that the increased use of SANs was the number one impact server virtualization has on storage environments. This rapid growth in server virtualization and SAN deployment is affecting the management of these increasingly complex and abstracted environments. In most cases, organizations end up changing management tools and processes. And this goes beyond just the server environment to include storage and network environments (see Figure 1). 2 Figure 1. The Impact of Virtualization on Management Process and Tools 70% 60% 50% 40% 30% Which of the following best describes how virtualization technology is impacting your organization s management processes and tools? (Percent of respondents) Impact on server management Impact on storage management Impact on network management 44% 39% 31% 53% 55% 61% 20% 10% 4% 5% 8% 0% Significant impact our strategy is to migrate all or most technology-specific management functionality to our virtualization software platform Some impact we will migrate some technology-specific management functionality to our virtualization software platform, but we will continue to leverage existing management tools and processes Little or no impact we will use the technology-specific management functionality in our virtualization software to a minor extent and it will not fundamentally replace our existing management tools and processes Source: Enterprise Strategy Group, 2011. As organizations continue to evolve their environments and deploy highly virtualized cloud infrastructures in production, easy to deploy, simple to use management tools that provide end-to-end configuration and performance visibility will become increasingly critical. 1 Source: ESG Research Report, 2011 IT Spending Intentions Survey, January 2011. 2 Source: ESG Research Report, The Evolution of Server Virtualization, November 2010.
Lab Validation: EMC ProSphere 4 EMC ProSphere EMC ProSphere is an end-to-end storage resource management application designed for quick, easy, and costeffective deployment to enable a rapid return on investment. With ProSphere, storage professionals can create a detailed view of their storage infrastructure from the host, through the SAN, to the backend storage environment. Figure 2. EMC ProSphere ProSphere is designed to be deployed in a VMware environment as a virtual appliance or vapp. For ease of deployment, it s packaged as an OVF template the can be downloaded from the EMC support website. To further simplify and reduce the management burden, it leverages an agent-less discovery and data ingest paradigm. ProSphere gets its performance data, alerts, and end-to-end visibility using standard protocols like SNMP and SMI- S for switches and arrays and WMI, SSH, or vcenter for physical and virtual hosts. ProSphere is designed to be easily deployed as a virtual appliance in a VMware environment. The discovery and data ingest processes are agent-less, leveraging SMI-S, SNMP and manufacturer APIs. ProSphere can be integrated with Microsoft AD and the LDAP protocol. Customers with active EMC ControlCenter support contracts can use ProSphere at no cost. EMC ControlCenter performance data can be imported into ProSphere. Administrators can manage multiple ProSphere instances without logging into each site separately.
ESG Lab Validation Lab Validation: EMC ProSphere 5 ESG Lab performed hands-on evaluation and testing of ProSphere at an EMC facility in Hopkinton, MA. Testing was designed to demonstrate how EMC ProSphere intuitively provides end-to-end resource management for storage environments. Also of interest were ease of use and implementation as well as agent-less host, SAN, and array data collection. The configuration used for ESG Lab testing is shown in Figure 3. On the left side of the figure are the three VMware virtual machines that made up the EMC ProSphere operating environment. These VMs consist of the Historical Database, Discovery Appliance, and the Storage Resource Manager. On the right side of the figure are the test bed objects that were discovered and monitored during the ESG Lab testing. EMC VMAX and CLARiiON arrays were discovered, as were Cisco and Brocade SAN switches. The host environment consisted of physical and virtual Windows servers as well as Solaris 9 and Solaris 10 servers. A LAN with multiple VLANs provided connectivity between the ProSphere environment and the objects being monitored. Figure 3. The ESG Lab Test Bed
Lab Validation: EMC ProSphere 6 Getting Started Getting started with EMC ProSphere begins with obtaining and installing the OVF template packaged code to bring the environment to the initial login screen. This was accomplished by downloading the ProSphere application software from the EMC web support portal 3 and installing the application into an existing VMware environment. ESG Lab Testing ESG Lab began testing ProSphere by copying the downloaded OVF formatted application code to the test VMware vsphere environment. ESG Lab used the Deploy OVF Template option to select and install the ProSphere application. Figure 4 shows the summary of entries including hostnames, networks, and other common system configuration settings that were selected as part of the template-driven install process. The settings were confirmed and the Finish Install tab was selected. The ProSphere installation process completed in approximately 30 minutes and resulted in the creation of a virtual application with three virtual machines. Figure 4. Install from OVF 3 https://powerlink.emc.com
Lab Validation: EMC ProSphere 7 ESG Lab used the vsphere client to verify the creation of the application. Figure 5 shows the ProSphere application (ESG-ProSphere) and the three associated virtual machines (Discovery Appliance, Historical Database, and Storage Resource Manager). Power on was selected at the virtual application level to start the three virtual machines in the correct order. Figure 5. vsphere View Next, ESG Lab performed an upgrade of the ProSphere application using VMware Studio as shown in Figure 6. Figure 6. Update After Install In preparation for the ProSphere vapp upgrade, ESG Lab configured a CD/DVD drive to the VMs. This enabled access to the ISO formatted upgrade image. VMware Studio was then launched from a browser for each VM to conduct the upgrade.
Lab Validation: EMC ProSphere 8 Finally, ESG Lab connected to the ProSphere vapp using the browser-based GUI to validate login access and confirm proper operation. Figure 7 shows the Administration view from the ProSphere user interface. From this view, user account and security certificates can be added and managed. The Administration view also enables integration configuration for LDAP and Microsoft Active Directory. Figure 7. ProSphere GUI Why This Matters Even in a fairly static environment, managing storage can be difficult. Introducing virtualization at the host and storage levels can make efficient, proactive management seem almost impossible. Storage management tools are of little use if they add more management complexity than the storage itself. Traditionally, storage management application deployments have been extremely complex, time-consuming, and costly to roll out. ESG Lab found that as part of the ProSphere redesign, EMC placed a heavy emphasis on ease of deployment and rapid time to value. The ability to rapidly deploy will enable organizations to recognize value from the software on the very first day and, in highly dynamic environments, every day after that. Deployed as a vapp in VMware vsphere environments, EMC ProSphere is ready to leverage protection capabilities already built into your virtual environment such as snapshots, vmotion, and VMware High Availability.
Lab Validation: EMC ProSphere 9 Configuration and Discovery Configuration is the process of setting up ProSphere to ingest data for performance analysis and storage environment alerting. Discovery is the process of identifying and ingesting performance and operating metrics as well as configuration data for the storage environment components or objects including hosts, switches, and storage arrays. With ProSphere, this process does not require an agent on each discrete object. It enables the administrator to create custom end-to-end views of the storage environment. ESG Lab Testing ESG Lab launched the ProSphere user interface and selected the Discovery icon to start the discovery configuration process. The first step in setting up each discovery job was defining its access credentials. Access credentials are a set of properties that define how ProSphere will connect to an object on the network. These property sets contain permission settings for security and define the connection type used for data discovery. Any access credential can be designated as global. Standard global access credentials can be defined and used for similar object types to streamline the discovery process by allowing a discovery job to automatically find and pick an appropriate, previously defined global credential. This process separates the creation and management of access credentials from that of discovery jobs. Server administrators can manage their credentials within the ProSphere environment while storage teams can then use those credentials to manage discovery jobs and meet security requirements for access management. As shown in Figure 8, the Access Credentials tab was selected from within the discovery view. ESG Lab leveraged this configuration tab to set up discovery permission for a number of objects in the environment. Discovery access was defined for EMC storage arrays; Brocade and Cisco SAN switches; and Windows, UNIX, and VMware hosts. Figure 8. Access Credentials Next, ESG Lab configured a number of different discovery jobs. As shown in Figure 9, ESG Lab configured a job to discover an EMC storage array. First, the scope of the job was set by selecting the object type and where on the network to look. In this case, the hostname of the server running the EMC Array SMI-S was selected, along with the object type of array. Then the previously-defined access credentials were selected for the new discovery job. The last step was defining the discovery schedule.
Lab Validation: EMC ProSphere 10 It should be noted that a range of IP addresses can also be used for discovery to help simplify the process in large dynamic infrastructures such as rapidly growing virtual environments. This option also allows for excludes to be set for objects that a user does not want to include in discovery. Figure 9. Discovery Jobs Figure 10 shows the results of a successful discovery job for host type objects. Here, ESG Lab selected the Discovered Host tab from the objects list view. Also available are switches, arrays, and fabric level object views. Figure 10. Object List
Lab Validation: EMC ProSphere 11 Finally, ESG Lab configured groups of objects. As shown on the left of Figure 11, ESG Lab configured three groups (ESG Lab Windows Servers, ESG Lab ESX, and ESG Lab Physical Hosts). The groups were used to create more manageable views. Large environments could have hundreds or thousands of servers, making the objects list view difficult to use when searching for a specific object of interest. Groups can also be used to view objects with interdependencies such as a group of servers or the server, switch, and storage that support a specific application. Figure 11. Groups Three group classifications are available in ProSphere. The three groups are defined as system groups, simple groups, and smart groups. System groups are automatically created by the application to group objects by type. Simple groups can be created and used for a collection of objects that rarely change. Objects must be manually added to simple groups. Smart groups can be used to dynamically scan and group objects based on user-defined parameters. With smart groups, objects can be properly grouped for analysis based on business requirements such as mission-critical application or utilization for bill back purposes. Why This Matters ESG Lab found that setup and configuration of EMC ProSphere was intuitive and straightforward. By leveraging industry-standard protocols like SMI-S and SNMP, the need to install and manage individual agents on each object under management is eliminated. This agent-less discovery paradigm greatly improves ease of management and reduces the possibility of missing critical components in rapidly evolving storage environments. ESG Lab believes the ease of use and intuitive approach of the ProSphere GUI, along with its agent-less discovery process, can greatly shorten the time spent managing, tuning, and re-architecting storage environments. Features like smart groups can help automate management by dynamically grouping objects for analysis as business-related storage resources are added and changed.
Lab Validation: EMC ProSphere 12 Monitoring, Alerting, Troubleshooting, and Analysis Performance monitoring is the ability to select, view, and analyze detailed information for objects in the storage infrastructure. ProSphere can create end-to-end topology views and detailed performance charts. Alerting is the ability to monitor SAN and storage objects in the infrastructure by setting thresholds and notification. Combined, performance monitoring and alerting help accelerate the identification and remediation of performance bottlenecks and component failures. ESG Lab Testing ESG Lab explored the performance capabilities of ProSphere by selecting a virtual machine in the storage environment and analyzing the supporting components from the host through the storage. Figure 12 shows a topology view from the virtual machine on the left side of the figure all the way through the SAN in the middle to the storage hosting the VM datastore on the far right. The high level overview was created by selecting specific objects of interest from the host object list. ESG Lab used a drill-down process from this view to analyze detailed information of each component. Hosts, switches, and arrays in the topology view can be expanded into a detailed view by clicking on the desired icon. Figure 12. Topology View In addition to simply selecting items from the object lists, ProSphere has sophisticated search capabilities. ESG Lab used the intelligent filtering feature of the search functionality to quickly and easily set views into different objects without having to scroll through the entire list to find a specific object. Intelligent search functionality becomes more important as the environment grows organically or through the federation of multiple ProSphere sites. Geographically separated ProSphere implementations can be federated through synchronization, a process that creates a synchronized data set between sites. The search feature can retrieve data in a federated environment eliminating the need to log in to each site and enabling users in different locations to perform analysis and troubleshooting of remote environments.
Lab Validation: EMC ProSphere 13 Next, ESG Lab reviewed the performance metrics of the VM selected in the topology view. As shown in Figure 13, ESG was able to analyze and review resource utilization, response times, and performance characteristics for the virtual machine. Similar views are provided for physical hosts as well. Figure 13. Host Performance As shown in Figure 14, ESG Lab also monitored performance data on the EMC array containing the datastore for the selected VM. ESG Lab noted a number of storage performance metrics (e.g., response time, IO, and throughput) that can be used to quickly analyze the health of the supporting array. Figure 14. Array Performance
Lab Validation: EMC ProSphere 14 Lastly, ESG Lab used the alerts tab to configure thresholds and then monitor storage and switch metrics in the test environment. As shown in Figure 15, utilization levels were set to flag warnings and critical conditions when a defined percent was exceeded. Alert levels can be set for the following metrics: Switch port link utilization Symmetrix port utilization Symmetrix host director utilization CLARiiON SP utilization CLARiiON SP % dirty pages Figure 15. Alerts Setting up and monitoring thresholds allows for easy, at-a-glance identification of performance issues in the storage environment. A quick review of the alerts page will show if a switch or array component in the environments has exceeded defined parameters. ProShpere can then be used to view storage component mapping and identify and analyze physical hosts or virtual machines that might be impacted by changes in SAN performance. Why This Matters ESG research indicates that storage scalability and performance are significant challenges for a growing number of organizations, especially those that embrace virtualization technology. These companies are continuously challenged to cost-effectively meet the performance requirements of applications. ESG Lab found that by leveraging the end-to-end visibility and threshold alerting available in ProSphere, identifying and remediating storage infrastructure issues was quick and easy. The ability to view a virtual storage group from the host to the array it sits on, combined with automated threshold alerting for the associated storage components, helps paint a clear picture for accurate analysis and diagnosis.
Lab Validation: EMC ProSphere 15 ESG Lab Validation Highlights The installation process for ProSphere was quick, easy, and intuitive. ESG Lab downloaded the application images from EMC s website, moved them to the VMware environment, and easily navigated through the OVF template install process, which took about 30 minutes. ESG Lab validated the efficiency of the agent-less discovery process driven by ProSphere. Without installing a single agent, ESG Lab was able to easily set up and discover over 100 physical and virtual hosts, four storage area networks, and six storage arrays. This was accomplished by leveraging SMI-S and SNMP for storage and switch objects and WMI, SSH, and vcenter connectivity for physical and virtual hosts. With ProSphere, ESG Lab confirmed that EMC can offer a comprehensive topology view from the host to the LUN on the supporting array, enabling end-to-end visibility of resources for the storage or system administrator. ESG Lab confirmed the ability to set up alerting for switch and storage components in the monitored environment. Thresholds can be set that notify the administrator when critical or warning conditions occur in the environment, allowing corrective action to be quickly taken to remediate the condition. Issues to Consider It should be noted that currently, the historical database cannot be pruned. Best practice should dictate monitoring disk capacity for this application component. Also, to avoid corruption, the historical database requires an orderly shutdown. EMC recommends that ProSphere be started and stopped from the vapp level to maintain database consistency. Special attention should be paid to best practice recommendations when using VMware snapshots in federated ProSphere environments. If one site in a synchronized environment is rolled back to a snapshot, an error will be returned when users try to access discovered object detail. To avoid this, each site must be rolled back to a corresponding snapshot. Since EMC ProSphere was in its first GA release at the time of the validation, ESG expects the application to gain new features over time and hopes to see mixed object group views at the dashboard level.
The Bigger Truth Lab Validation: EMC ProSphere 16 In an effort to respond more quickly to changing business needs, IT is transitioning to highly virtualized and cloud environments. To ensure a successful transition, storage administrators need to leverage management software that was specifically built for these new data center paradigms. This software should include comprehensive management of the networked storage environment and provide visibility into the virtual server environment, as these previously siloed technology domains are now more tightly integrated and interdependent. ESG Lab validated that EMC ProSphere s efficient architecture enables it to be deployed easily, accelerating the time to deliver value for organizations transitioning to cloud environments. It is able to provide end-to-end views and the performance information required to ensure applications will run in an optimized environment while meeting demanding SLAs. ESG Lab believes EMC has made it easy for clients currently running ControlCenter to test and then migrate to ProSphere thanks to its ability to ingest ControlCenter performance data. EMC has also allowed ControlCenter clients with active maintenance contracts to deploy ProSphere at no cost. As organizations evolve their highly virtualized infrastructures into scalable cloud solutions, they will need management solutions that can adapt to these dynamic and agile environments. By designing and architecting management software for the Cloud Computing Era, EMC enabled ProSphere to effectively manage and optimize cloud storage environments.
Appendix Lab Validation: EMC ProSphere 17 Table 1. ESG Lab Test Bed Detail EMC ProSphere Storage Resource Manager Version: 1.0.0.1.266 Guest OS: Suse Linux Enterprise 11 (64-bit) Historical Database Version: 1.0.0.1.267 Guest OS: Suse Linux Enterprise 11 (64-bit) Discovery Appliance Version: 3.0.1.1.263 Guest OS: Suse Linux Enterprise 11 (64-bit) Storage (3) EMC VMAX Arrays Microcode: 5874.248.194 (1) EMC VMAX Array Microcode: 5875.198.148 (1) EMC CX4 480 Array Microcode: 4.28.5.706 (1) EMC CX4 960 Array Microcode: 4.28.706 Brocade Cisco Solaris Solaris Windows ESX SAN Hosts Model: 5100 Firmware: v6.2.1 Model: DS-C9124-K9 Firmware: 4.2(7) HW: Oracle Sun Fire 280R OS: Version 9 HW: Oracle Sun Fire 280R OS: Version 10 HW: Dell PowerEdge OS: Windows Server 2003 Enterprise HW: Dell PowerEdge 2950 OS: ESX 4.0.0
20 Asylum Street Milford, MA 01757 Tel:508.482.0188 Fax: 508.482.0218 www.enterprisestrategygroup.com