Jeremy Canady. IBM Systems and Technology Group ISV Enablement November 2013

Transcription

1 Using VMware VASA with IBM Storwize family and IBM XIV Using the VASA support provided by IBM Storwize and IBM XIV to automate virtual machine storage placement Jeremy Canady IBM Systems and Technology Group ISV Enablement November 2013 Copyright IBM Corporation, 2013

2 Table of contents Abstract...1 Introduction...1 Prerequisites...1 IBM storage systems...1 IBM Storwize family... 2 IBM XIV... 3 VMware vsphere...3 IBM Storage Integration Server...3 VMware vsphere APIs for Storage Awareness (VASA)...4 Installation and configuration...4 IBM Storage Integration Server installation... 4 VASA configuration... 5 Add storage arrays to IBM Storage Integration Server... 7 Configure array credentials... 8 Add storage array to IBM Storage Integration Server... 9 VASA integration use cases...10 Multiple development stage example Dev stage QA stage Load-test Multiple development environments and stages Summary...19 Resources...20 About the author...20 Trademarks and special notices...21

3 Abstract This white paper provides an overview of how the support of VMware vsphere APIs for Storage Awareness (VASA) provided by the IBM Storwize family and IBM XIV Storage System can be used to provide smarter and more reliable deployment of virtual machines. Introduction Integration between management, automation, and hardware systems has become a requirement to support the ever-evolving software-defined environments and virtual infrastructures. Users expect the solutions to have a higher level of awareness regarding the infrastructure when deploying workloads to software-defined environments or virtual environments. To address this, VMware has created the vsphere APIs for Storage Awareness (VASA). In short, VASA is an application programming interface (API) that allows vsphere to be aware of the underlying capabilities of each storage volume provided by the storage system. Thanks to the new IBM Storage Integration Server, the IBM Storwize family now provides support for VASA. This paper provides an overview of the new IBM Storage Integration Server and VASA, and how the new VASA support can be used with virtual machine (VM) storage policies to automate the placement of virtual machines on Storwize-backed data stores. Prerequisites To successfully understand and implement the IBM Storage Integration Server with VASA support, the reader needs to have familiarity with several technologies and needs to complete the following prerequisite tasks. Technology skills prerequisites VMware vcenter Server management and administrator IBM Storwize management IBM XIV Storage System management Linux command-line familiarity Implementation prerequisites Operational VMware vsphere 5.1 or later environment Virtual machine or physical server running a compatible Linux distribution IBM XIV or IBM Storwize storage system configured and zoned to a VMware vsphere environment IBM storage systems IBM provides a range of storage systems designed to meet challenges. IBM storage systems support many advanced storage functionalities such as IBM Real-time Compression, automated tiering, storage virtualization, and thin provisioning. These advanced functions, combined with the performance and reliability expected of an IBM solution, result in better system performance and lower IT costs. 1

4 IBM Storwize family The IBM Storwize family consists of a set of virtualized storage systems designed to reduce cost, simplify management, provide cost saving advanced functionality, and provide high scalability. To meet this challenge, IBM offers a range of storage systems running on the same core Storwize code base. IBM System Storage SAN Volume Controller (SVC) IBM System Storage SAN Volume Controller provides a modular and highly scalable storage virtualization solution. SVC allows customers a single point of management for all the storage in their IT infrastructure. SVC also augments the virtualized storage systems by providing advanced storage features such as Real-time Compression and automated tiering, to name a few. For additional information about IBM SVC, refer to the following URL: ibm.com/systems/storage/software/virtualization/svc/ IBM Storwize V7000 The IBM Storwize V7000 storage system provides block storage enhanced with enterprise-class features to midrange customer environments. The Storwize V7000 system can scale up to 240 drivers per control enclosure. Additionally, up to four control enclosures can be clustered, allowing the Storwize V7000 system to scale up to 906 drives. For additional information about the IBM Storwize V7000 system, refer to the following URL: ibm.com/systems/storage/disk/storwize_v7000/index.html IBM Storwize V7000 Unified The IBM Storwize V7000 storage system combines the block storage capabilities of the Storwize V7000 system with advanced file storage capabilities to form a single system for greater ease of management and efficiency. The IBM Storwize V7000 Unified system contains the same 2U drive enclosures of the Storwize V7000 system and two 2U file modules. For additional information about the IBM Storwize V7000 Unified, refer to the following URL: ibm.com/systems/storage/disk/storwize_v7000/index.html IBM Flex System V7000 Storage Node IBM Flex System V7000 Storage Node is a high-performance block-storage solution that has been designed to integrate directly with IBM Flex System. IBM Flex System V7000 Storage Node provides advanced storage capabilities, such as IBM System Storage Easy Tier, IBM Real-Time Compression, thin provisioning, and more. For more information the IBM Flex System V7000 Storage Node, refer to the following URL: ibm.com/systems/flex/storage/v7000/index.html IBM Storwize V5000 The IBM Storwize V5000 system provides cost-efficient midrange storage. Built from the same technology as the IBM SAN Volume Controller and IBM Storwize V7000 systems, the IBM Storwize V5000 system offers advanced storage features in a cost-conscious package. For additional information about the IBM Storwize V5000 system, refer to the following URL: ibm.com/systems/hk/storage/disk/storwize_v5000/index.html 2

5 IBM Storwize V3700 The IBM Storwize V3700 system is an entry-level storage system designed for ease of use and affordability. Built from the same technology used in all of the Storwize family, the Storwize V3700 system offers some of the advanced features that can be found in other Storwize models. For more information about the IBM Storwize V3700 system, refer to the following URL: ibm.com/systems/storage/disk/storwize_v3700/index.html IBM XIV IBM XIV is an enterprise-class disk storage system designed to provide hotspot-free high performance with extreme ease of use. XIV features a grid design that allows for automated tuning to provide peak performance without administrator intervention. Designed for five-nines reliability, the XIV system provides peace of mind with its self-healing capability and rebuilds within an hour. For additional information about the IBM XIV Storage System, refer to the following URL: ibm.com/systems/storage/disk/xiv/index.html VMware vsphere VMware vsphere is a virtualization platform capable of transforming a traditional data center, consisting of industry-standard hardware, into a shared mainframe-like environment. Through the use of virtualization technology, hardware resources are abstracted into pools that can be allocated to a variety of workloads. VMware vsphere is the core enabling technology of VMware s software-defined environment. IBM Storage Integration Server The IBM Storage Integration Server is a centralized service platform that provides a consolidated method for IBM storage system monitoring, automation, and provisioning. IBM Storage Integration Server provides a centralized and standardized location for independent software vendor (ISV) frameworks and platforms to integration with a diverse range of IBM storage systems. At the time of publishing this paper, IBM Storage Integration Server supports storage-level integration into VMware vsphere through the VMware vsphere Web Client and VASA. IBM Storage Integration Server provides the foundation for integration with future IBM systems and ISV solutions. IBM Storage Integration Server consists of a Linux server running as either a virtual machine or a physical server. IBM Storage Integration Server acts as the intermediary between the IBM storage systems and the ISVs framework or platform. You can find additional details regarding the installation and configuration in the Installation and configuration section of this paper. Figure 1 provides a visual overview of the IBM Storage Integration Server solution. 3

6 Figure 1: Integration of IBM storage systems with a VMware environment VMware vsphere APIs for Storage Awareness (VASA) VASA is an API designed to allow vsphere to gain insight into the storage systems that it uses. VASA allows vsphere to discover the functional capabilities of a volume, such as if the volume has Real-time Compression enabled. This information can then be used to help automate the placement of virtual machines. Installation and configuration Installation of IBM Storage Integration Server and configuration of VASA is a two-part process. First, IBM Storage Integration Server is installed onto the server, then each component is configured. After the configuration is complete, IBM storage arrays can be added to IBM Storage Integration Server for management. IBM Storage Integration Server installation This section provides a quick overview of the IBM Storage Integration Server installation process. For detailed instructions, refer to the IBM Storage Information Server user guide. 4

7 Requirements Server running a compatible version of Red Hat Enterprise Linux (RHEL) IBM Storage Integration Server compatible web browser Shell access to the server as a privileged user IBM Storage Integration Server installation package IBM Storage Integration Server installation Download the IBM Storage Integration Server installation package. Unpack the installation files. Install all Red Hat Package Manager (RPM) packages in the proper order. VASA configuration With the installation of IBM Storage Integration Server, the VASA provider is already installed and operational. To begin using the VASA provider, two additional steps must be completed. First, the VASA provider must be registered with any vcenter servers that will be using it. Then, storage arrays must be added to IBM Storage Integration Server. Instructions to add storage arrays are provided in the subsequent sections. Before the VASA provider can be registered with vcenter, a secret must be set. The secret is a user name and password combination that will be used by vcenter to authenticate the VASA provider. The same secret will be used for all connecting vcenter servers. To configure the secret, log in to the VASA web GUI and navigate to the VASA tab. Next, click Change Secret as shown in Figure 2. Figure 2: Option to change VASA secret In the Set Secret dialog box, enter a valid user name and password and click Set. The configuration dialog box is shown in Figure 3. 5

8 Figure 3: Set Secret dialog box To register the VASA provider with vcenter, log in to the vcenter Web Client and navigate to the vcenter Management tab. Click the Add storage provider button, as shown in Figure 4. Figure 4: Add storage provider button A new dialog box appears asking for the various connectivity parameters. The name can be anything the user would like. One suggestion is to use the fully qualified domain name (FQDN) of the IBM Storage Integration Server. In the User name and Password fields, you need to enter the user name and password provided during the VASA secret creation. Finally, the URL is the URL to the VASA provider on the IBM Storage Integration Server. The URL is shown along with the prompt in Figure 5. URL: IBM Storage Integration Server Server FQDN or IP>:8443/services/vasa1 6

9 Figure 5: vsphere Web Client storage (New Storage Provider window) After the provider has been registered, and the initial discovery complete, the Storage Providers window now displays the storage systems that the VASA provider is providing information for. No storage systems will be listed until they are added to IBM Storage Integration Server (which is explained in a later section). Figure 6 shows the Storage Providers window after storage systems have been added to IBM Storage Integration Server. Figure 6: vsphere Web Client storage providers page with storage systems Add storage arrays to IBM Storage Integration Server IBM Storage Integration Server is capable of managing multiple IBM storage systems from a single installation. The single installation allows consumers of IBM Storage Integration Server services, such as VASA, to have a single point of contact for multiple storage systems. This reduces configuration 7

10 complexity and management overhead. For this to function, all storage systems that IBM Storage Integration Server provides services for must be connected to the IBM Storage Integration Server. Configure array credentials To add a storage system to IBM Storage Integration Server, credentials must first be defined. Log in and navigate to the Arrays tab. Click Set Credentials (as shown in Figure 7) to configure the login credentials for the storage system. IBM Storage Integration Server uses a single set of credentials to access all storage systems. Administrators need to make sure that proper user accounts exist on all storage systems that connects to IBM Storage Integration Server. Figure 7: IBM Storage Integration Server Option to set array credentials In the Set Credentials dialog box (shown in Figure 8), enter the user name and password that has been configured on all the storage systems. 8

11 Figure 8: IBM Storage Integration Server setting array credentials Add storage array to IBM Storage Integration Server To add a storage system to IBM Storage Integration Server, first log in to the IBM Storage Integration Server web GUI and then navigate to the Arrays tab. Click Add in the Storage Arrays section as shown in Figure 9. Figure 9: IBM Storage Integration Server adding storage array 9

12 In the Add Array dialog box (shown in Figure: 10), select the appropriate storage system type and then provide the FQDN or the IP address for the storage system. Clicking Add will add the storage array to IBM Storage Integration Server. At this point, IBM Storage Integration Server begins providing VASA information for the newly added storage array to any registered vcenter servers. Figure: 10 IBM Storage Integration Server adding an array After the array has been added, the array window displays basic information regarding the array (as shown in Figure 11). During the initial addition phase, a small set of information is gathered by the IBM Storage Integration Server. The periodic background collection process runs at a later time to collect detailed information. Figure 11: IBM Storage Integration Server with storage arrays added VASA integration use cases The VASA integration provided by IBM Storage Integration Server allows vcenter to gain additional insight into each storage volume backed by an IBM storage system. The IBM VASA provider supports two profiles, BlockDeviceProfile and CapabilityProfile. The BlockDeviceProfile allows vcenter to inquire about the block device. The CapabilityProfile allows VCenter to inquire about the capabilities of the volumes presented by the device allowing vcenter to know, for example, if the volume is using Real-time Compression. 10

13 The primary use case for this information is to use it with VM storage policies. VM storage policies are friendly names that can be assigned to a set of storage capabilities and tags. VM storage policies allow administrators to define a friendly name such as Development that automatically selects the proper data stores. To further illustrate how VASA and VM storage policies can be used, a couple of example use cases have been provided. Multiple development stage example For this example, assume a portion of the virtual environment has been dedicated to software development. The development team uses three development stages, dev, QA, and load-test. Administrators have designed specific storage solutions for each environment. Dev stage The dev stage is where developers develop new features and functions. This stage must support a multitude of temporary virtual machines for each developer but does not require top-level performance. To accommodate this, the storage administrators have designed and deployed a set of storage pools backed by 10k SAS drives. Capacity: Large Performance: Medium Solution: Disk pools containing 10k SAS drives. QA stage The QA stage is where the quality assurance teams take the completed software and run it through a battery of tests. Various development instances have now been refined to a much smaller subset of instances. This reduces the overall capacity needs. The QA tests do not test at full load but they do need to test for reasonable performance under the light testing load. To accommodate this, the administrators have decided to provide disk pools containing 10k SAS drives with Easy Tier. Capacity: Medium Performance: High Solution: Disk pools containing 10k SAS drives with Easy Tier. Load-test The load-test stage is the final stage of the software development cycle. The QA approved software build is deployed onto near production like hardware and loaded with above production load. Only a single instance of the software makes it to the load-test stage at a time, so capacity requirements are low. However, for the load test to be accurate, the backing hardware needs to be like production. To accommodate this need, administrators have selected to use a disk pool containing solid-state drives. Capacity: Small Performance: Extreme Solution: Disk pools containing solid-state drives. The solution is now designed and implemented. During usage, administrators discover issues that are becoming a larger problem. During VM allocation, administrators, or self-service developers, have begun incorrectly placing VMs. The correct placement of a VM relies on the proper naming of the data 11

14 store and the understanding of the naming scheme. In some cases, data stores have been mislabeled upon creation or migrated to different storage pool without the label being changed. Additionally, it is suspected that some developers are intentionally placing development VMs on production storage to increase local performance. To correct this issue, administrators have decided to use the VASA support of their IBM storage and VM storage policies. Administrators create three separate storage policies, Dev Storage, QA Storage, and Load-test Storage. Each profile is configured with a rule-set that limits storage to the appropriate type of storage. Figure 12 displays the configuration for the QA Storage VM storage profile. Figure 12: VM storage profile rule creation When a new VM is being deployed, administrators and self-service users can select the proper profile. The user will be presented with a list of data stores that are compatible with the selection. This can be seen in Figure 13. Figure 13: VM storage selection with the QA Storage profile Besides properly locating virtual machines upon creation, the new configuration also allows for auditing the compliance. In the event a VM is relocated to the wrong data store, it will be flagged for administrators to take the appropriate action. This can be seen in Figure

15 Figure 14: Not compliant VM Multiple development environments and stages For this example, the previous example is expanded to support multiple development environments. In the previous example, a portion of the virtualized infrastructure was dedicated to software development. Storage was automatically selected for each development stage based on VASA information and VM storage policies. For this example, assume that there are now three development teams working on separate pieces of software. Each team needs to have dedicated storage resources and multiple levels of performance. A representation of the resulting hierarchy is provided in the following list. Product A Development Team - Product A Development - Product A QA - Product A Load-test Product B Development Team - Product B Development - Product B QA - Product B Load-test Product C Development Team - Product C Development - Product C QA - Product C Load-test As before, storage has been selected for each stage of development. Administrators now face the issue of how to separate the storage between the development teams. Although the load-test stage can use the same type of storage between development teams, they should not use the same physical storage. The solution is to use the tagging feature of VMware vsphere. 13

16 Tagging has become a common feature of many services. Blogging software allows writers to tag their posts, photo management software allows users to tag their photos, and even some software has forgone folders and instead allows users to tag . Tags are a simple text identifier that can be attached to any object. Objects can contain any number of tags. The tags then allow users to quickly select objects based on whether the object has a tag associated to it. The tagging implementation in vsphere allows administrators to attach tags to hosts, data stores, virtual machines, and many other objects. These tags allow administrators the ability to quickly sort and select the objects they are interested in. This tagging feature can be used by VM storage policies to help automatically select the appropriate data stores. Figure 15 and Figure 16 give a visual representation of how tags can be used to select objects. Figure 15: Tagging example one Figure 16: Tagging example two The vsphere tag implementation includes categories, which are an additional component that might not be found in many other tagging solutions. By default, a tag can be attached to one of many 14

17 different types of objects within vsphere. Categories allow administrators to provide some restriction to specific tags. For example, you can have a tag that is only appropriate to a virtual machine. You use this tag to select a set of virtual machines to be automatically restarted. Without categories, the tag can be applied to an object that does not need to be restarted, such as a data store. Categories also provide an additional important functionality, the ability to modify the cardinality of the tags in the category. This allows an object to be associated with only one tag from the category. For example, an administrator can tag every virtual machine based on its location. Without this limitation, a virtual machine can be tagged with multiple locations. To solve the problem with multiple development environments, tagging can be used to select the proper data store. VASA capabilities can then be used to select the correct storage from the subset created by the tag selection. This can be seen in the following list representation. Product A Development vsphere Tag o o o Product A Development VASA (SAS) Product A QA VASA (Easy Tier) Product a Load-test VASA (SSD) Product B Development vsphere Tag o o o Product B Development VASA (SAS) Product B QA VASA (Easy Tier) Product B Load-test VASA (SSD) Product C Development vsphere Tag o o o Product C Development VASA (SAS) Product C QA VASA (Easy Tier) Product C Load-test VASA (SSD) To use the tagging feature, the tags must first be created. After navigating to the tags window in the vsphere Web Client, click New Tag. In the New Tag dialog box, enter the relevant tag information. For this use case, administrators will create a new tag for every product. They will also place the tag in a category devoted to development storage. The category will be set to One tag per object to prevent a data store from being associated with tags for multiple development teams. Creating the tag for Product A Storage can be seen in Figure

18 Figure 17: vsphere New Tag dialog box With the tag created, it must be applied to all data stores that are dedicated to that development team. There is no need to tag the data stores differently to accommodate for the development stages. The selection of storage will be handled by VASA. To tag a data store, navigate to the data store and then click Actions Assign Tag, as shown in Figure

19 Figure 18: Data store Actions menu Select the proper tag and click Assign. Figure 19: Data store tag assignment window The tag has now been created and assigned to the proper data stores. The final step is to define the new VM storage profile for each environment. The same steps are taken as in the previous example but this time a tag is added. To add a tag, click Add tag-based rule (as shown in Figure 20). Figure 20: VM storage policy rule set window 17

20 Selecting the proper tag will add it to the rule. This can be seen in Figure 21. Figure 21: VM storage policy rule set window After completion, the VM storage policies now use two pieces of information when determining proper placement of a virtual machine. First, it looks for data stores that have assigned tags that match the rule. Next, it uses VASA to determine whether the backing storage supports the capabilities that the rule requires. The result is a listing of only the data stores that are appropriate for that virtual machine. 18

21 Summary The IBM Storwize family provides a wide range of advanced storage features that can provide storage administrators the ability to optimize the storage for specific workloads, providing better performance and utilization. Although storage administrators might design optimal configurations for a specific workload, the disconnect between the workload placement and storage capabilities can introduce errors. By using the new VASA support for the IBM Storwize family and VMware vsphere s VM storage policies, administrators can automatically classify storage, automate the proper placement of workloads, and audit the placement at a later time. 19

22 Resources The following websites provide useful references to supplement the information contained in this paper: IBM Systems on PartnerWorld ibm.com/partnerworld/systems Virtual Loaner Program ibm.com/systems/vlp IBM Redbooks ibm.com/redbooks IBM Publications Center IBM Storage Integration Server Documentation pic.dhe.ibm.com/infocenter/strhosts/ic/index.jsp?topic=%2fcom.ibm.help.strghosts.doc% 2Fhsg_isis_1.1.0.html VMware vsphere Support Center About the author Jeremy Canady is an IT specialist in IBM System and Technology Group ISV Enablement Organization. He has more than 8 years of experience in IT administration and over 4 years of experience working with VMware virtualization products. 20

23 Trademarks and special notices Copyright IBM Corporation References in this document to IBM products or services do not imply that IBM intends to make them available in every country. IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at UNIX is a registered trademark of The Open Group in the United States and other countries. Other company, product, or service names may be trademarks or service marks of others. Information is provided "AS IS" without warranty of any kind. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Information concerning non-ibm products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-ibm list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-ibm products. Questions on the capability of non-ibm products should be addressed to the supplier of those products. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller for the full text of the specific Statement of Direction. Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here. Photographs shown are of engineering prototypes. Changes may be incorporated in production models. 21

24 Any references in this information to non-ibm websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk. 22