Bridging the Gap between Servers and Storage with Veritas Operations Manager and Storage Foundation

Transcription

1 November 2010 By Jerome M Wendt DCIG, LLC 7511 Madison Street Omaha NE O Bridging the Gap between Servers and Storage with and Storage Foundation 2011 DCIG LLC. All rights reserved.

2 Bridging the Gap between Servers and Storage with and Storage Foundation Table of Contents Executive Summary 1 Navigating the Enterprise Storage Management Maze 2 The Need to Centralize Storage Management 2 Centralizing Storage Foundation Management under Veritas Operations Manager (VOM) 2 Detailed Data Collection through Storage Foundation s DMP 3 VOM s Deep Insight into Enterprise Storage Arrays Completes the Detailed View 3 Creating the Broad Storage Overview 4 Wizard Driven Storage Provisioning through VOM Templates 4 VOM Fast Tracks Volume Migrations 4 VOM Helps Automate Staying Thin 5 VOM Bridges the Server-Storage Gap DCIG LLC. All rights reserved. p. ii

3 Bridging the Gap between Servers and Storage with and Storage Foundation Executive Summary Storage management is one of the most complex tasks in enterprises today. This is due to a number of reasons: server consolidation, server growth and server virtualization alongside feature-rich storage arrays that store application data. However it was not supposed to be this complicated. The introduction of more features into storage arrays that match specific application requirements was instead supposed to result in better performing applications that had the right data on the right storage tier at the right time. Instead, what we see is a growing gap between the tier of storage that applications need and what tier of storage is actually assigned to them. This has largely come about because administrators are struggling to find the time to match the right tier of storage capacity with an application s requirements due to the number of servers they have to manage. Even then, if they do find the time, they may not have the right tier of storage capacity available for that application since they do not have the tools to forecast storage growth. Bridging this server-storage management gap requires that enterprise organizations have the appropriate storage management tools at their disposal. This paper will examine: The complex storage management maze that administrators must navigate The now inherent need that enterprise organizations have to centrally monitor, manage and report on their storage infrastructure Why the capabilities of existing storage management software such as Storage Foundation must be extended to satisfy these new demands How enterprises can centralize their Storage Foundation management under How Veritas Operation Manager can collect detailed information from both servers and storage arrays to build a comprehensive view of the storage environment How Veritas Operation Manager can centralize, simplify and expedite the provisioning of storage to new servers, the migration of volumes to other storage arrays and reclaim storage on existing servers Why is a necessary next step for organizations to take in helping to bridge their growing server-storage management gap 2011 DCIG LLC. All rights reserved. p. 1

4 Bridging the Gap between Servers and Storage with and Storage Foundation Navigating the Enterprise Storage Management Maze Server and storage administration has come a long way and gotten a lot more complicated since its humble beginnings. Whereas servers and storage were once sold and managed as a single entity, storage has since moved outside of servers in the form of external, purpose-built storage arrays that are connected to servers via storage networks. This has led to the introduction of storage array features to match application requirements. While matching application requirements to specific storage features works relatively well with a few number of servers, it becomes more complex to efficiently manage in networked storage environments. Administrators in today s enterprise environments must now navigate a maze of application requirements, storage networking protocols and storage array features in order to allocate the right type of storage for each application. This leaves little room for error. Ramifications include allocating the wrong storage capacity to an application resulting in anything from overprovisioned and wasted storage resources to, in worst case scenarios, degradations in application performance and even unexpected application outages. Just some of the considerations involved with provisioning storage to application servers and then managing it in these environments include: Accounting for the tens, hundreds or even thousands of both physical and virtual application servers that may access storage capacity on one or more storage arrays Identifying the specific storage requirements for each application Documenting and tracking the mix of features on each storage array Provisioning storage with right set of features to each application Reporting on storage utilization once it is allocated But even as networked storage environments have gotten more complex and time consuming to manage with little margin for error, internal and external forces are creating new demands for faster and more efficient management of this environment. The Need to Centralize Storage Management Server consolidation and server virtualization initiatives are speeding up not slowing down. As this occurs, the number of server and storage administrators left to centrally manage storage is staying flat or even decreasing. To remain efficient in their storage management responsibilities, many of these individuals have already deployed Symantec s Veritas Storage Foundation. Storage Foundation provides them with a common set of commands that work the same way across a variety of enterprise operating systems (AIX, HP-UX, Linux, Solaris, and Windows) along with a GUI so they can centrally manage storage on any of these OS platforms. But Storage Foundation by itself requires administrators to log on to each server to execute its commands or access its GUI. So while administrators want to continue to leverage the commonality of Storage Foundation s features, logging onto each server is no longer practical. Today s environment instead demands that the administration of Storage Foundation be expanded such that administrators can centrally: Provision the right storage to the right application quickly and efficiently Identify the features available on each storage array as well as on each storage array volume Perform storage management tasks in a risk free manner This is where Symantec s comes into play. Centralizing Storage Foundation Management under Symantec s (VOM) is a complimentary software tool available from Symantec that any organization using Storage Foundation or Veritas Cluster Server can install on a virtual or physical server. VOM still uses Storage Foundation to do storage management on individual application servers. However, it eliminates the need for server or storage administrators to log on to each server. VOM instead leverages existing security permissions such as Active Directory or LDAP to authenticate administrators DCIG LLC. All rights reserved. p. 2

5 Once an administrator is authenticated, he or she is granted the appropriate permissions to access and manage any server that has the Storage Foundation High Availability for UNIX or Storage Foundation for Windows High Availability software installed on it. This is all done without the administrator needing to log onto each individual server. has rolebased user access that allows for administrators to have management access to servers only they are allowed to manage. Here is a sample of operations that are offered in : Monitors storage allocation and utilization on Storage Foundation managed servers Visualizes the storage infrastructure so administrators can see what file system are assigned to each application and on which storage array these volumes reside Monitors and visualizes their clustered server environment Understands the underlying features and properties associated with each file system Manage and control the storage capacity assigned to each server Generates reports on storage utilization and capacity growth Detailed failover reports for Veritas Cluster Server environments Detailed Data Collection through Storage Foundation s DMP The depth of information that VOM collects from both Storage Foundation and the storage arrays across the storage infrastructure is where VOM s full value becomes apparent. One of the major challenges of centralizing the management of storage across multiple servers is gathering sufficient data to build an in-depth picture of how each storage array volume is configured and then in what ways it is presented to one or more application servers. To build this detailed view, VOM can gather data on: How the storage array volume is presented to the server. Enterprise storage arrays can present the same volume to one or more servers on multiple paths. So unless VOM understands that the storage array volume can be configured this way, it may wrongly conclude that the server has more storage capacity assigned to it than it really does. How path load balancing and fail over are configured. Since each server may have 2 or more paths to a single storage array volume, path load balancing and failover settings need to be preconfigured, set and then monitored so in the event of network disruption, path failure or periods of congestion, the server responds appropriately. What the underlying properties of the storage array volume are. Understanding what disks (SSD, FC, SAS, or SATA) are used to construct each volume and which RAID level is used to protect these disks is needed so each storage array volumes can be appropriately categorized as to what tier it is and then assigned to each application. VOM takes two tactics to collect this data. The option VOM universally uses is to leverage Storage Foundation s native Dynamic Multi-Pathing (DMP) feature to gather data about storage array volumes presented to the server. DMP gathers information about each volume via standard SCSI commands such as the name of the storage array on which the volume resides; the name by which the volume is known and recognized to both the server and storage array; and, whether the volume is thinly provisioned. This data is gathered and sent to the VOM management server so it can build a comprehensive view of each storage array volume for all application servers using Storage Foundation. However, to gain the deep insight such as what underlying tier of disk is used by each volume as well as the RAID level associated with those disks, VOM uses another technique. VOM s Deep Insight into Enterprise Storage Arrays Completes the Detailed View Enterprise storage arrays from providers such as EMC, Hitachi Data Systems (HDS) and IBM provide much more detailed information about each storage array volume than what can be obtained from just SCSI commands. By communicating directly with these vendors storage arrays, VOM can gather additional details about their volumes, such as: The physical properties of the disk on which the volume resides to include SSD, FC, SAS and SATA. The logical device mappings so an administrator can match the storage array volume name with the storage array s internal logical device (LDEV) name. The RAID properties of the underlying disk on which the storage array volume resides. The internal storage array groups from which the volume may be constructed. Any replication of that volume that may take place between two storage arrays Array snapshot information for individual storage array volumes 2011 DCIG LLC. All rights reserved. p. 3

6 VOM captures this very detailed storage array information from HDS storage arrays by communicating with the HDS HiCommand management server. To gather this information from supported EMC and IBM storage arrays, VOM uses the command line interfaces available on their storage arrays. In the case of the EMC Symmetrix DMX series, VOM uses its SymCLI while on the IBM XIV Storage System it uses IBM s XCLI. VOM then combines this detailed storage array data with the data it previously gathered from Storage Foundation s DMP to build a complete end-to-end detailed view into an organization s storage environment. Creating the Broad Storage Overview However, enterprise organizations need more than just detailed end-to-end views into their storage environment. Organizations also require reports that provide a broad overview of their storage environment to include: How much total storage capacity is allocated to each application How much storage capacity is available and used by each application What type of storage capacity is being used by each application Which applications are experiencing the fastest storage growth VOM is able to use the information that it gathers from Storage Foundation and storage arrays to build these high level trending and capacity planning reports that enterprise organizations need to plan for future growth and upcoming storage purchases. Using these reports, enterprises can now more confidently determine and/or forecast the following for Storage Foundation managed hosts: When applications are running low on storage capacity If there is excess storage capacity in their environment so that can be provisioned first When they need to buy more storage capacity How much storage capacity they need to buy What type of storage capacity they need to buy Wizard Driven Storage Provisioning through VOM Templates In addition to reporting, VOM also offers templates to more quickly and effectively provision storage for new servers. Without VOM, provisioning storage for any new server is still a manual task as it requires administrators to first run a set of commands and then optimize the file system for I/O and caching. So to help streamline and automate these tasks VOM offers a complimentary Provisioning Storage Add-on. This Add-on gives administrators the ability to initially create storage provisioning templates from an existing Veritas file system and volume manager configuration or from scratch. Using the VOM Provisioning Storage wizard an administrator selects the server, a Storage Foundation disk group and a template. The template then takes storage capacity out of the Storage Foundation disk group, assigns it to the application, creates a volume or volumes and then formats each volume s file system according to the template s policies. Administrators can create multiple templates so they can create Gold templates that are optimized for use with specific applications. For instance, if storage capacity from an existing disk group is being assigned to a new Oracle RAC application server, the template may include policies that first configure the disk group as two volumes and then optimally tunes the file system on each one for the RAC logs and the other for the RAC database tables. This VOM Provisioning Storage Add-on reduces the time that administrators need to configure storage on new application servers from minutes to seconds. Further, the availability of these storage templates in VOM enables storage architects to initially define and configure the policies in each of these templates. This keeps the variance between how settings for different applications are implemented in data centers to a minimum while also making it possible for storage architects to delegate this responsibility to others in the organization. VOM Fast Tracks Volume Migrations Provisioning storage for new application servers is not the only repetitious task that consumes the time of administrators. Migrating volumes from one storage array to another is one more example of a task that occurs frequently and can consume large amounts of their time. VOM takes this challenge on as well. Under VOM administrators still leverage Storage Foundation s native ability to dynamically migrate volumes from one storage array to another. However using VOM administrators can concurrently and intelligently manage volume migrations across multiple application servers. To simplify and expedite this task, VOM provides a Volume Migration wizard. Administrators use this wizard to select the specific volumes on the source storage array that need to be migrated and then the volumes on the destination storage array to which the data will be migrated. Before actual movement of data begins, VOM does an impact analysis to assess how the volumes, disks, applications, and even the business entities associated with this data migration 2011 DCIG LLC. All rights reserved. p. 4

7 might be affected. In this way, administrators can anticipate and correct any potential problems. Administrators can also use the detailed information that VOM provides about the volumes on the storage array to ensure that the volumes to which the data is being migrated meets the application needs for availability, performance, protection and tier of storage. Once the impact analysis and storage evaluation are complete, the volume migration can begin. VOM calls upon Storage Foundation to dynamically migrate volumes from one storage array to another. This volume migration from one storage array to another can occur even as the application(s) accessing the volume remain in production and, since volume migrations are being centrally managed by VOM, multiple migrations can be initiated and managed at the same time across multiple servers in the environment. VOM s volume migration feature can also be called upon to optimize volume placement within an array. For instance, administrators may want to leverage VOM to migrate volumes residing on higher performing FC or SAS disks to SATA disks or vice versa. Alternatively, they may want to leverage this feature to move a volume from thick to thin storage in order to reclaim excess, unused storage capacity on the thick volume. VOM Helps Automate Staying Thin A final area worth noting is VOM s ability to help enterprises control their storage capacity consumption on storage arrays. Thin provisioning is a feature available on about 55% of enterprise midrange arrays and all high end enterprise storage arrays. 1 Thin provisioning feature enables administrators to over provision storage capacity to servers without incurring the penalty of allocating real storage capacity. Using thin provisioning, only storage capacity that is used by the application is actually reserved. The challenge with using thinly provisioned volumes is keeping these volumes thin over time. Prime examples are database administrators who create and then delete tables or users who save files to file servers and then later delete or archive them. In both of these examples, when the new tables or files are saved, additional storage capacity on thinly provisioned volumes is reserved. But when these tables or files are deleted or archived, the thinly provisioned storage capacity that was consumed when this data was saved is not automatically returned to the available storage pool. Instead this freed storage capacity continues to be held in reserve unless some action is taken to return it to the available storage pool. VOM again provides a mechanism for helping enterprises centrally automate staying thin. To do so, VOM leverages Storage Foundation and its Thin Reclamation API to keep storage volumes thin that are located on storage arrays from providers like Compellent, EMC, Fujitsu, HP, IBM and NetApp. The Thin Reclamation API recognizes which storage array volume are thinly provisioned. It identifies reserved but unused blocks of storage on these volumes, reclaims that storage capacity and returns it to that storage array s pool of available storage capacity. However the Thin Reclamation API without VOM can only be executed to run on single servers. VOM builds upon Storage Foundation s Thin Reclamation API by offering a Thin Provisioning Add-on that gives organizations three ways to centrally create and set policies to automate staying thin. Individual storage arrays. VOM leverages its integration with enterprise storage arrays from EMC, HDS and IBM and Storage Foundation on the servers to identify allocated but unused blocks on these storage arrays. These blocks are then returned to that storage array s free storage pool of thinly provisioned storage capacity. Further, using VOM, an administrator can click on an array thin pool and issue a command to reclaim the allocated but unused storage capacity on all of the hosts connected to that pool of storage capacity. Business entity. VOM supports the creation of objects that can include servers or storage arrays and then be associated with a specific business entity like a department or business unit. Policies can then be set up under VOM so storage capacity associated with these objects is automatically reclaimed and returned to the free thin provisioning pools Hosts. Through VOM s integration with Storage Foundation, administrators can use VOM to select multiple file systems on multiple hosts. Then using Storage Foundation s Thin Reclamation API, VOM can schedule the detection of free space on these hosts, the reclamation of this free space and the return of that storage capacity to the available thinly provisioned storage pool on the storage array. VOM Bridges the Server-Storage Gap Enterprises have confidently used Storage Foundation for years to simplify and expedite their storage management tasks by providing them with a common set of commands and a familiar GUI when working across a range of enterprise operating systems DCIG LLC. All rights reserved. p. 5

8 But the combination of server consolidations, server growth and server virtualization has outstripped their ability to continue to efficiently manage storage using Storage Foundation alone. Administrators no longer have the time to go from server to server to individually manage storage on each server using Storage Foundation. They now need a central management console to make it practical to monitor, manage and report on storage. VOM bridges that gap. It gathers and provides the detailed information that administrators need in a central location about individual storage arrays, storage array volumes and storage configurations to make specific decisions. Organizations are rapidly evolving and, as they do, their storage management practices need to change to help them remain efficient and effective. VOM provides organizations with the opportunity to do so by helping them stay one step ahead of these demands and giving them the ability to centrally monitor, manage and report on their storage. But what makes VOM and all of its Add-ons so appealing, is that they are available at no charge to Storage Foundation and Veritas Cluster Server users so they can bridge this widening server-storage gap and move to this next stage of centralized storage management. This may involve something as simple as being alerted when a specific application server is running out of space so more storage capacity can be added. Or it may be more complex such as understanding what tiers of storage are available so when new application servers come online, wizards can be leveraged to ensure the right tier of storage is assigned and that the file system is properly configured. VOM also provides organizations with the broad overview that they need of their storage environment. VOM enables them to track how storage consumption is growing, what storage capacity is unused, what applications are experiencing the most growth, what tiers of storage are needed, and then leverage that information to make informed decisions as to what type of storage to buy. 1. Wendt, Jerome M DCIG Midrange Array Buyer s Guide. PDF About DCIG DCIG analyzes software, hardware and services companies within the storage and ESI industries. DCIG distributes industry, company and product analysis by way of viral marketing and community building using the burgeoning BLOG infrastructures created worldwide. DCIG, LLC 7511 Madison Street Omaha NE dciginc.com 2011 DCIG, LLC. All rights reserved. All other brands or products are trademarks or registered trademarks of their respective holders and should be treated as such. The information, product recommendations and opinions made by DCIG LLC are based upon public information and from sources that DCIG LLC believes to be accurate and reliable. However since market conditions change, the information and recommendations are made without warranty of any kind. All product names used and mentioned herein are the trademarks of their respective owners. DCIG LLC assumes no responsibility or liability for any damages whatsoever (including incidental, consequential or otherwise) caused by one s use or reliance of this information or the recommendations presented or for any inadvertent errors which this document may contain. Any questions please call DCIG LLC at (402)