The Economics of File-based Storage

Transcription

1 Technology Insight Paper The Economics of File-based Storage Implementing End-to-End File Storage Efficiency By John Webster March, 2013 Enabling you to make the best technology decisions

2 The Economics of File-based Storage: Implementing End-to-End File Storage Efficiency 1

3 The Economics of File-based Storage: Implementing End-to-End File Storage Efficiency 2 Executive Summary Storage efficiency has a major impact on enterprise IT s ability to live within budgetary constraints even as new data sources and new applications increase the pressure on existing storage resources. Improving efficiency provides economic benefits by providing sustainable ways to plan for and implement storage capacity in the long-term and with minimal disruption. This paper focuses on enterprise data center NAS environments and examines three significant contributors to storage efficiency in more detail. These include: Consolidation from many to fewer hardware and software resources Active archiving migration of less active/inactive data from primary to active archival platforms Rationalizing data protection and making it more sustainable by eliminating inactive/stale data from the backup process A strategy for consolidating NAS siloes and implementing a primary archive (also known as active archive) that is integrated with a scalable NAS platform is now recommended. As an example, we outline how Hitachi NAS (HNAS) platforms integrated with Hitachi Content Platform (HCP) can serve as a foundation for a highly scalable and efficient file storage architecture. Driving Enterprise Storage Efficiency Initiatives: Data Growth Collides with Efficiency Mandates Enterprise storage architects and administrators are traditionally caught between two conflicting forces. One is the ever-present need to store increasing amounts of data. The other comes in the form of the absolute requirement to live within IT budgetary constraints. It s a way of life for them that is exacerbated by competition from lower-cost cloud services providers. And now a new dimension has been added: a requirement imposed from the IT executive level to maximize storage efficiency. We have seen mandates from CIOs to make better use of currently installed but unused capacity first before new storage capacity is added. These efficiency mandates come at a critical time. Across vertical industry segments we can identify needs to store data from an increasing number of new sources. In healthcare there is a growing storage of high resolution, digital imagery, as well as the use of digital audio/video to capture physician/patient interactions. In retail we see a desire to analyze data from web-based social media sources and combine the results with existing customer data a process now referred to as customer sentiment analysis. Banking, insurance, and financial services firms are drawing a wider circle around data points that can be correlated to show patterns of fraudulent behavior.

4 The Economics of File-based Storage: Implementing End-to-End File Storage Efficiency 3 In the face of new data sources and new applications increasing the pressure on existing storage resources, improving efficiency yields economic benefits by providing sustainable ways to plan for and implement storage capacity in the long-term and with minimal disruption. File Storage Efficiency Initiatives that have Measurable TCO and ROI Benefits For IT staffs, measuring storage efficiency is seen as a component in controlling costs, both capital and operational over at least a three-year time period usually expressed as Total Cost of Ownership (TCO). Measuring storage efficiency both before and after and correlating the results to TCO can yield the ROI of an efficiency initiative. Storage efficiency is also one of the key items in Data Center Optimization projects. What constitutes the elements to consider for maximizing storage efficiency will vary from one production data center to the next and discussing them all is beyond the scope of this paper. Here we focus on enterprise data center NAS environments and examine three significant contributors to storage efficiency in more detail. These include: Consolidation from many to fewer hardware and software resources including snapshot, cloning, and remote data replication software licenses Active archiving migration of less active/inactive data from primary to active archival NAS platforms Making the data protection environment more efficient by reducing backup data volumes NAS Consolidation NAS is often implemented on a per application basis and therefore siloed meaning that NAS platforms (hardware and software) are dedicated to each application. Consolidating many of these existing data siloes to a single, high capacity, high performance NAS platform can dramatically reduce the overall costs related to enterprise NAS in a number of ways: The overall cost of storage hardware is reduced by eliminating hardware redundancy and reducing costs associated with power, cooling and data center floor space Substantially fewer software licenses for storage and data management services including those for local and remote data replication functions, element managers, etc. Hardware maintenance costs are reduced Management processes are simplified leading to lower management expenses and reduced risk of data loss due to human error Data protection and security is enhanced lowering the cost to achieve these two objectives

5 The Economics of File-based Storage: Implementing End-to-End File Storage Efficiency 4 The greater the single image capacity of the consolidation platform, the greater the impact on the cost factors noted above. Primary Archiving We note that one of the first places to look to free-up space on primary NAS platforms is via the migration of data to a primary archive. The efficiency principle we are advancing here is simply this: if data that is either stale or infrequently accessed is migrated from high performance primary to high density secondary disk storage that is nevertheless available to applications when needed, space is created for new, active data as it is generated by applications. Figure 1 (below) demonstrates this principle. Figure 1. Capacity growth with and without primary archiving. Results above assume that primary storage is 62% utilized and that 50% of primary storage is migrated to active archival storage on a yearly basis. Note in Figure 1 above, the capacity of primary file storage platforms without an active archiving process applied to them grows at a faster rate than that of primary storage with active archiving. Additionally, because all primary storage data is typically included in data protection processes, the growth of backup storage is reduced as well though the implementation of active archiving. Perhaps more importantly, applying this principle has as a direct and beneficial impact on the overall IT budget as applied to the enterprise storage environment. This is shown in Figure 2 below.

6 The Economics of File-based Storage: Implementing End-to-End File Storage Efficiency 5 Figure 2. Fully burdened yearly storage costs (primary, backup and primary archive). Calculations include hardware capacity, maintenance, administration, floor space, power and cooling costs. In Figure 2 we note that migrating infrequently accessed and dormant data to a primary archive offers a number of benefits including: Reductions in the fully burdened cost of primary storage Reduced growth of backup storage infrastructure, thereby lowering data protection costs while enhancing business continuance capabilities Enhanced management efficiency resulting from the ability for a single administrator to manage a multiple of archival storage vs. primary storage Therefore, an investment in active archiving storage can result in a significant return (ROI) depending on the costs of primary and backup storage and the amount of data migrated off of primary storage. It can also help to preserve the performance characteristics of primary storage over time. An Enterprise NAS Consolidation and Primary Archive Strategy To achieve the benefits of a file storage efficiency initiative, we believe that enterprise storage administrators should first develop an archiving practice as a logical approach to dealing with capacity

7 The Economics of File-based Storage: Implementing End-to-End File Storage Efficiency 6 demand. We believe that the establishment of an archive practice would provide immediate and continued economic payback. Storage administrators can start by identifying the NAS filers that are targets for replacement by a single, highly scalable platform. This process can include NAS filers at remote and branch locations assuming that the target NAS consolidation platform supports these locations via a device often referred to as a cloud onramp. Next, put together a data migration and decommissioning schedule so as to implement the file storage consolidation. Once data has been consolidated, one can then begin to migrate occasionally active/inactive data off of primary to active archival storage. However, IT needs to first review current operational practices and assess application user impact. On the operational side, this includes: Determining operational changes necessary when implementing an archive strategy. Addressing current procedural limitations affecting archiving process automation. Assessing data protection improvements resulting from less data to back up due to archiving. Modifying disaster recovery operations to include archived data. On the application side, business application owners should be interviewed for several purposes: Understand from their perspective the changes in applications planned or in process, and the amount of information produced or retained Review retention requirements including data governance and regulatory requirements Draw them into the decision process as their endorsement can have a major impact on success of the initiative. Storage administrators will then need to identify data sets that require retention, establish the requirements for retention (lifecycle, compliance, etc.) and apply them as policy statements to the active archival storage environment. When selecting storage platforms used to put in place a file storage efficiency initiative, IT should include the following characteristics: High performance so as to support large numbers of users served by multiple applications. Performance required should be assessed on the number applications, number of application users, and the criticality of the applications. Scalable to billions of files/objects even if the starting point is relatively low. Internal data protection capabilities to make recoveries more responsive to the needs of application users and relieve pressure on the existing data protection environment.

8 The Economics of File-based Storage: Implementing End-to-End File Storage Efficiency 7 For the primary archive platform, the ability to index and search across the entire platform and any archive data replicas residing in remote sites. These capabilities are essential to quickly finding objects as needed for compliance or legal discovery requests without the expensive, laborious and error-prone process of spinning through tapes stored in an offsite vault. The Hitachi Approach to File Storage Efficiency Through acquisition supplemented by internal development, Hitachi has assembled a file storage product portfolio that demonstrates a rational, sustainable approach to managing capacity growth which we believe will accelerate over the next three to five years. The products outlined below promote a highly efficient and economic use of the different types of storage now available to enterprise IT including SSD, high performance disk, and high capacity disk, while preserving the enterprise-grade availability and reliability characteristic Hitachi is known for. They can also be integrated to create a storage lifecycle continuum starting with file creation and ending with long-term file retention and archiving. HNAS HNAS is a next generation, N-way clustered NAS platform with the ability to scale to 16PB in a single namespace. HNAS, acquired with Hitachi acquisition of BlueArc, was originally designed to meet the scalability and performance demands of the media and entertainment, oil and gas exploration, and other high performance computing (HPC) usages. Since that time, Hitachi has extended its applicability to enterprise data centers by adding additional features and capabilities. Significant characteristics that contribute to the scalability of HNAS while preserving storage efficiency include: Cluster Namespace a virtual file system that creates a global namespace across nodes in the HNAS cluster. The Cluster Namespace provides a means to scale beyond the 256TB physical capacity available on each cluster node. Snapshots schedule, implement and recover point in time, copy on write snapshots. A succession of snapshots is maintained and individual snaps can be backed up, restored or deleted. Storage administrators can restore individual files from any snapshot. Incremental Data Replication allows administrators to set up a scheduled, incremental, system internal backup of a volume or a directory using snapshots. Scripting is supported which enables

9 The Economics of File-based Storage: Implementing End-to-End File Storage Efficiency 8 automated functions before or after replication. By using snapshots for system-internal data replication only changed files are replicated which minimizes capacity growth related to data protection. BlueArc JetClone creates writeable, file-level clones, a capability that is particularly useful in virtual server and virtual desktop environments. BlueArc JetClone writes only change data when subsequent clones of virtual machines are created meaning that the very large amounts of data that are common among VMs is not duplicated leaving only modified data to consume additional capacity as clones are created. BlueArc JetMirror for asynchronous replication of objects and files to remote HNAS systems. BlueArc JetMirror creates sub-file containers that include data objects, the master index object, and metadata objects. When a file is replicated for example, the target HNAS system will reassemble the objects involved in the file replication a more efficient and faster way to remotely replicate files than typical file-based replication. Multiple HNAS targets are supported as well meaning that HNAS can support multi-site disaster recovery schemes and storage system-based content distribution. Intelligent File Tiering migrates data between tiers of storage within the HNAS system and to other storage systems that are connected via Ethernet using NFSv3 protocol including Hitachi Content Platform (HCP-see below for details). Tiering moves files bi-directionally to the most appropriate internal storage tier whether it be SSD s, Fibre Channel, SAS and ATA drives or externally to HCP. Migration is based on based on user-defined criteria and HNAS will transparently migrate data when the criteria are met. Policy templates are included which provide selection based on many criteria. In addition, a preview or what if tool is available to show the expected effects of the migration. Hitachi Data Discovery Suite (HDDS) offers the ability to search, text mine, identify, collect and preserve data across NAS and archival storage systems including HNAS, HCP, and Windows file servers. HDDS facilitates automated file tiering as well as file management processes related to compliance and legal discover. Primary Data Deduplication a software-only, active data deduplication engine is now included to reduce the physical capacity required for all HNAS content. This could significantly increase the effective storage capacity of HNAS depending on content types and add an additional level of storage efficiency to those mentioned above. Primary data deduplication is now available for HNAS and Hitachi Unified Storage file module systems. HCP Hitachi Content Platform (HCP) is a storage solution for the active and long-term archiving of fixed content data under policy controls. HCP targeted at archiving unstructured data across multiple

10 The Economics of File-based Storage: Implementing End-to-End File Storage Efficiency 9 industries and market segments with the capability to meet multiple regulatory requirements around records retention. HCP manages fixed content in an active archive an object-based, online archive where the information is readily available but is under control of HCP system for retention and security according to administrator-defined policies and external regulatory requirements. Object metadata is administrator definable such that different objects can be associated with one another. For example, all content relating to a specific project including s, Microsoft SharePoint files, images, etc. can be correlated. HCP architecture uses clustered nodes based on standard server technology coupled with Hitachi storage and software. The minimum is two nodes with the ability to scale up to 96 nodes. Data is distributed among the nodes in the cluster for load balancing. Files stored in HCP are accessed over an Ethernet network with remote file system protocols and can be integrated with HNAS as a source for file migration to HCP (see above for more information on HNAS). The advanced features expected of an enterprise archiving system are available with HCP and include: Data Authenticity as files are ingested, a digital fingerprint including a hash code is created and saved in the metadata. The hash code is selectable. Choices include MD-5, SHA-1, SHA-256. Upon file retrieval, the digital fingerprint is created again and checked against the original to validate the integrity of the file. Versioning files are set into a WORM state after ingestion. Subsequent storage of a file with the same name will create a new version. Retrievals may specify specific versions of the file. Access Protection supports file access controls using AD (Active Directory) and LDAP (Lightweight Directory Access Protocol). Using HCP administrative GUI, users and groups can be granted access to files. Available permission masks are applied at the system level, tenant level, and the namespace level. Optional Encryption for data at rest stored on disk storage attached to HCP cluster nodes. Setting encryption is an installation option (all or nothing). The encryption key is encrypted itself and distributed among nodes for security. Data transfers to other HCP systems are also encrypted. Management nodes use SSL and HTTPS and the system provides for self-signed certificates. Multiple Copies Up to four copies of files may be made across nodes within HCP cluster. Drive Spin-down individual HCP drives can be automatically powered-off ( spun down ) after an administrator-defined period of inactivity as a power saving measure.

11 The Economics of File-based Storage: Implementing End-to-End File Storage Efficiency 10 Compression / Single Instancing the hash code (digital fingerprint) used for data authenticity checking is also used for single instancing (file-level deduplication). Only a single copy of a file in a namespace will be stored. Data Shredding digital overwrite of data is supported for expiration and deletion of objects. Search an optional search feature that is provided as part of a Content Discovery Option. With the Content Discovery Option, data that is ingested is indexed and a role-base search engine is available through a Web GUI. Retrievals of files to a server and setting legal holds may be done as a result of the search. Search console usage is supported across all data in HCP namespace. Search in the context of the Content Discovery option is supplemental to Hitachi Data Discovery Suite mentioned above. Cloud Onramp an optional appliance called Hitachi Data Ingestor (HDI) can be used in remote locations to link file storage at these locations to HCP. HDI serves as both a NAS device for applications at the remote site, and a migration engine pushing inactive files back to HCP. Archive Data Protection Archived data within HCP namespace can be protected for recovery in the case of a disaster by making multiple copies either on different storage systems locally or via remote replication to another HCP. Additionally, data may be backed up to another storage system (disk or tape) using NDMP. In addition to the backup of archive data, an optional remote mirror to another HCP feature is available. With the remote mirroring feature, information is synchronized between HCP at a primary site and HCP at a secondary site. For disaster protection and business continuance, the remote mirroring feature will provide the availability required. Conclusion File storage consolidation has obvious and measurable benefits that are expressible in terms of TCO and ROI. Less obvious are some of the ancillary benefits that aren t captured by these measurements. For example, consolidation of data such that it is accessible to many applications as opposed to having siloed data sets dedicated to specific applications promotes a data governance policy often referred to as a single version of the truth. This is important to those within the enterprise who are trying to bring rational control over the multiple versions of electronic documents in use at any given time, as well as their disposition and access privileges. We also note that the implementation of a primary archive platform can also have benefits beyond those of making the primary and backup storage environments more cost-efficient. The primary archive can be made available to multiple applications including existing data warehouses and allow for the

12 The Economics of File-based Storage: Implementing End-to-End File Storage Efficiency 11 potential to support new analytics applications if required. As a result of the Big Data analytics phenomenon, there is a growing sense that value can be extracted from archival data sets. We now see enterprises IT feeding archival data to a broader range of traditional BI and new analytics platforms such as Hadoop. Having index and search services resident in an archival storage platform allows for transferring to the analytics application only the data that is relevant to a particular process, allowing for faster delivery of results to analytics applications users. Hitachi Data Systems has long been an advocate of storage efficiency and has a well-established a corporate initiative for explaining how storage efficiency contributes to storage economics. For Hitachi Data Systems storage economics at the customer level is expressed in the characteristics of their storage systems and software to provide better TCO and ROI vs. competing systems without those characteristics. For example, we noted that HNAS integrated intelligent tiering is represented as storing data on the most efficient hardware technology (i.e. different classes of disks and SSDs) managed by automated migration expressed in software. For customers, Hitachi Data Systems believes that the measurement of storage efficiency is really an economics measurement (called econometrics ) featuring year over year comparisons to show the effect of continuous improvement. The implementation of HNAS integrated with HCP as a storage continuum from file creation to final disposition is a laudable demonstration of the Hitachi ability to apply storage econometrics to file-based storage for medium to very large scale enterprises. About Evaluator Group Evaluator Group Inc. is dedicated to helping IT professionals and vendors create and implement strategies that make the most of the value of their storage and digital information. Evaluator Group services deliver in-depth, unbiased analysis on storage architectures, infrastructures and management for IT professionals. Since 1997 Evaluator Group has provided services for thousands of end users and vendor professionals through product and market evaluations, competitive analysis and education. Follow us on Copyright 2013 Evaluator Group, Inc. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or stored in a database or retrieval system for any purpose without the express written consent of Evaluator Group Inc. The information contained in this document is subject to change without notice. Evaluator Group assumes no responsibility for errors or omissions. Evaluator Group makes no expressed or implied warranties in this document relating to the use or operation of the products described herein. In no event shall Evaluator Group be liable for any indirect, special, inconsequential or incidental damages arising out of or associated with any aspect of this publication, even if advised of the possibility of such damages. The Evaluator Series is a trademark of Evaluator Group, Inc. All other trademarks are the property of their respective companies.