File System Archiving

Transcription

1 Focus on EMC By Tony Asaro Senior Analyst February 2006 Copyright The Enterprise Strategy Group, Inc. All Rights Reserved.

2 ESG has found that many companies are currently implementing a configuration of intelligent tiers in their storage environments. These intelligent tiers consist of various storage systems and disk-types with different cost, performance, and protection levels. Software tools are then used to move data between these different tiers based on frequency of access, performance, and data type. Intelligent tiered storage allows companies to archive , database, and file storage, resulting in improved cost savings, performance, and protection. File system archiving is rapidly emerging as one of the primary applications for intelligent tiered storage networks. The reason for this is that file storage, also known as unstructured data, is made up of many different forms of data, since it can be generated by a myriad of applications. Files may come in the form of word processing documents, spreadsheets, presentations, images, videos, audio files, scanned documents, x-rays, etc. The sheer quantity of file data is creating a chain of inefficiency that can have major implications on business. The amount of unused files on any storage system will vary. According to SNIA, the average amount of dormant files on storage systems is approximately 80 percent. This means that 80 percent of your files can be moved to a tier that is less expensive and easier to manage. For instance, if a storage system contains 10 TB of files, then 8 TB of that can be moved from primary storage to a digital archiving solution such as the EMC Centera. However, even if just 20 percent of your files are dormant, that may still represent a significant amount. The goal of file system archiving is to move files off of primary storage systems and put them on lower cost, easy to manage archival storage. The benefits of implementing file system archiving within an intelligent tiered storage environment include: Lower Capital Costs. A great deal of file data is created and rarely or never accessed over time. However, these files still take up capacity on primary storage. To keep up with growing demands, companies will purchase more capacity for their Tier One storage systems. This capacity comes at a premium. In some cases, companies will go even further and buy entirely new storage systems to meet the rapid growth of file storage. By implementing file system archiving with different tiers of storage, companies can reclaim capacity on their primary storage systems by moving unused files to digital archive storage. This can significantly reduce cost by lowering the cost per MB for that capacity. Additionally, it can slow down or eliminate the need to acquire additional primary storage systems. Improved Backup Performance and Better Data Protection. Keeping only active data on Tier One storage improves backup performance, which results in better protection of data. Reducing the backup window also reduces risk. If a backup takes 16 hours to complete, a file lost during that period of time may not be recoverable. Having less data to back up shrinks the backup window and therefore reduces the risk of data loss, which results in better protection of data. Additionally, both recovery point objectives (RPO) and recovery time objectives (RTO) are improved because there is less primary data to backup and recover. Reduced Administration Costs. By implementing file system archiving and streamlining the amount of files stored on primary storage systems, companies can simplify the administration of those systems. For example, new storage capacity will need to be provisioned less frequently, since a great deal of primary data is moved from tier one to archival storage. Additionally, files archived to a lower tier typically require less management, especially in an archival storage system designed for ease of use, like the EMC Centera. Improved Application Performance. Moving unused or infrequently accessed files to a lower tier of storage frees up resources on primary storage systems. Since the primary storage system stores fewer files and less data, it can serve read and write requests much more quickly. This improves overall application performance. Improved Data Access. Digital archives can grow to store massive amounts of files. These files can be created by a number of different applications, including document, spreadsheet, PDF, and presentation creation programs, as well as custom applications. Being able to provide rapid search and retrieval of these files will make the digital archive much more useful, just as search engines make accessing data on the Internet more productive. Enterprise Strategy Group Page 1

3 In this report, we discuss intelligent tiered storage, with file system archiving as a key component. The focus of this report is on EMC solutions, including its Centera digital archiving storage system solution. Additionally, we discuss different EMC and third-party tools to move data from SAN and NAS primary storage systems to the Centera as a digital archive for files. These tools include EMC Centera File Archiver, EMC DiskXtender, Arkivio, and CaminoSoft. We also discuss EMC VisualSRM, which evaluates and analyzes files to see what types of files have not been accessed for long periods of time in multi-vendor storage environments. The Need to Archive In a recent study on digital archiving, ESG Research found that the main reasons that companies want to archive their file-based storage is to perform search and retrieval of data, get better access to the intellectual property (IP), meet compliance regulations, reduce storage capacity of primary storage, improve the workflow process, increase the performance of backup, reduce the cost of backup, and improve server performance. This study found that 27 percent of companies surveyed stored over 10 million objects (an object is defined as a file and its metadata) of unstructured data, 11 percent stored over 50 million objects, and 7 percent stored over 100 million objects 1. Reduce Storage Capacity One of the primary reasons companies want to implement disk-based archives is to reduce the amount of stored content on their primary storage systems. There are a number of benefits to this, including cutting costs, optimizing performance, simplifying management of primary storage systems, and improving data protection,and reliability. Implementing a disk-based archive for file systems reduces the amount of capacity required on primary storage. However, by using a disk-based archive, users actually get access to more capacity. In a sense, home directories can be limitless, combining primary and archival storage without runaway costs. The process is transparent to users and allows system administrators to minimize client side management significantly. Compliance Federal and local government mandates require companies to meet standards for retaining certain types of data for specified periods of time. Companies and organizations must be able to produce that information as part of a discovery process in a reasonable fashion. Further, the data must be proven to be unaltered since it is considered electronic evidence. Based on these requirements, customers are implementing digital archival storage that supports write once, read many (WORM) capability and user definable policies that set retention and disposition periods. Improved Workflow Process A digital archive provides a central repository for files that could have been scattered across multiple storage systems. Additionally, a digital archive with search capabilities provides faster access to relevant files, even those created by different applications. Further, removing unused files from primary storage systems results in fewer files to scan through, also making it easier for users to find relevant data quicker. Backup Performance Since there are fewer files stored on the primary storage system, there is less data to back up. Therefore, backup performance improves, reducing backup times significantly, and lowering the probability of backup errors. Some backup applications back up all of the data on your storage system even if it has not been altered. Since the majority of data on storage systems is dormant, then most of this data has been backed up already. This process is extremely wasteful, costly, and risky. Backup Cost Companies often keep multiple copies of full backups on tape and disk backup targets and retain these backups for a period of time to meet service level agreements. Additionally, many companies keep multiple full backups for a period of 30 days or more. Moving unused files to a digital archive reduces the amount of primary file data that needs to be backed up. This reduces the amount of media required and results in lower costs. 1 ESG Research: Digital Archiving Report, January Enterprise Strategy Group Page 2

4 ESG Research found that 83 percent of the 485 customers surveyed either have or will deploy a digital archive. The level of interest in digital archiving was consistent across companies off all sizes ranging from less than 1,000 employees to those with over 20,000. A digital archive storage system is an important component of an intelligent tiered storage network and the implementation of file system archiving. Enterprise Strategy Group Page 3

5 Intelligent Tiered Storage Many companies currently have in place within their data centers different tiers of storage with multiple storage systems from various vendors. While these customers have tiered storage environments, they may lack the ability to move data transparently between these different tiers based on user defined policies. In order to do this, a level of intelligence through the use of software tools is required. The combination of different tiers of storage and software creates an intelligent tiered storage network. Discrete Storage Systems Islands of Inefficiency In this example, Company ABC has three primary storage systems, one SAN-based and the other two NAS systems from different vendors. Each storage system is being used by various departments for a wide range of applications. As Figure One illustrates, each storage system is a discrete solution that has no interrelationship with one another. There is no easy way to move data between these storage systems. Additionally, since each storage system is Tier One, the cost difference between them is marginal. Each storage system is experiencing rapid data growth, which in turn creates stress on the company to protect all its data, meet backup windows, provide optimal performance, and keep capital costs under control. Figure One: SAN and NAS Storage Systems FC or iscsi SAN Storage System EMC DMX NAS Storage System NetApp FAS NAS Storage System EMC NSX 60% Full 80% Full 40% Full The rapid growth of data in primary storage systems creates a domino effect that impacts the entire data management process: The rapid growth of files stored increases the complexity of managing primary storage. Backup windows increase, making the backup process longer and exposing the company to greater risk. More data is backed up, including duplicates, which requires more tapes and creates a more complex media management process. A complex media management process increases the risk of human error, which can lead to data loss. Acquiring more tapes increases capital expenditure and may require more shipments to an off-site vaulting facility. The process of recovering data becomes more complex and time consuming since there are more tapes both on and off-site. Application performance begins to slow down and results in system administrators tuning and analyzing performance. More storage capacity is required for storage systems, creating additional expenditures. If storage capacity keeps growing with continued rapidity, the company may need to buy additional storage systems for an even greater cost and an increase in management complexity. Enterprise Strategy Group Page 4

6 File system archiving can significantly reduce the burden and greatly assist alleviating the issues described above. Companies can conduct an assessment of their environments by using the EMC VisualSRM tool. This process will provide companies with an understanding of the various data types, origin of files by department, and dates that files were last accessed and modified. Additionally, VisualSRM can provide insight on personal files such as MP3 files, photographs, etc., that can be moved to the archive tier. This step will give companies insight into how many files they can move, which can be used as a basis to determine cost savings and improvements in operational efficiencies. EMC Centera Archival Storage The EMC Centera is a scalable storage system that can be used as a digital archive. The Centera enables customers to more affordably create a digital archive that is easy to install, grow, and manage. The Centera is a disk-based solution that can be used for applications that previously relied on the inefficiencies and hassles of tape, CD-ROM, and microfiche to archive file storage. Additionally, the EMC Centera is arguably better suited than traditional SAN and NAS storage systems for archival storage due to its self-managing characteristics, transparent scalability, and lower total cost of ownership (TCO). Additionally, EMC provides search capability to quickly find and retrieve files. These characteristics are essential as the digital archive grows to hundreds of thousands, millions, and even billions of files. Each Centera node is a server that implements multiple hardware resources, including memory, bandwidth, and disk capacity. The Centera can keep scaling by adding nodes into the cluster: from four to 32 in a single cabinet, and up to eight cabinets. And regardless of whether the Centera has four or 32 nodes or even more -- it is logically ONE system. This means that it is as easy to manage with four nodes as it is with 32, with 100 files as it is 100 million. This makes the EMC Centera one of the easiest storage systems to manage, which is essential for a digital archival solution. The EMC Centera supports powerful functionality that stores only one copy of a file even if there are for example - 20 copies migrated to it. Centera will identify duplicate files, delete them and maintain an index of all of the data and its associated meta-data. For example, twenty or more copies of a single PowerPoint presentation might exist on shared network drives. Imagine a 5 MB PowerPoint presentation multiplied by 20 copies. The amount of capacity would equal 100 MB. The amount of capacity taken up with duplicate files can be astounding. There is no reason to keep 20 copies of duplicate, data and the Centera can provide significantly more efficient capacity utilization than (primary) NAS or SAN storage that does not support this capability. Implementing It is one thing to assume that a large number of files have not been accessed but having actual quantitative data that tracks access can be extremely useful. EMC provides a service that evaluates how frequently files have been accessed to assess how much a company can benefit by implementing file system archiving. EMC found that based on over 180 assessments they performed, an average of 67 percent of total files stored in primary storage systems had not been accessed in 120 days or more. This means that for each 10,000 files, 6,700 can be moved to a lower cost tier of storage, based on frequency of access. ESG believes that this is extremely common and that the majority of files are not accessed very often after their initial creation, if at all. As discussed earlier, Company ABC has three different primary storage systems, including one SAN-attached and two NAS systems. Company ABC installed the EMC Centera to act as a digital archive with the goal of moving any files that have not been accessed in thirty days off their primary storage systems. Files that have not been accessed in that time are moved to the Centera as an online process with no user interruption or application downtime. Even when the file is migrated from its primary storage system to Centera, applications and users can still access all of the files instantly. From the users and applications perspective, the files appear to be where they always were, although in reality the files are replaced by stubs. Additionally, if the files are accessed again after they ve been moved to the Centera, Company ABC also can set a policy to move the files back to their primary storage system. Enterprise Strategy Group Page 5

7 SAN and DAS with DiskXtender SAN-based storage systems work at the block level with no knowledge of files or file attributes. In order to move files stored on SAN-based storage, EMC provides its DiskXtender software, which supports Linux, Unix, and Windows platforms. The EMC DiskXtender software is installed on each of the application servers and provides transparent file movement to the Centera from SAN- or even DAS-based storage systems. A large number of EMC customers have implemented DiskXtender and the Centera for just this purpose. Figure Two: LAN SAN or DAS Celerra to Centera Archiving EMC provides another policy engine called Centera FileArchiver. This is a low cost, simple policy engine that moves data between EMC Celerra NAS to EMC Centera disk archive storage based on properties such as the age and/or size of the file. According to EMC, the customer interest and acquisition of Centera FileArchiver has been extremely high. EMC claims that the large majority of its customers want simple but strong glue between these two products. EMC is now bundling the Celerra and Centera as a combined solution, bringing together a primary NAS tier and a disk-archive tier for best-in-class services with the right mix of price and performance. Non-EMC NAS Migrations Customers using NetApp and any of the other NFS or CIFS-based NAS solutions in the market can use software from Arkivio. The Arkivio auto-stor software is an EMC certified third-party tool used to transparently migrate data from any NAS solution to the Centera. The auto-stor solution is agent-less and will perform discovery of file system storage throughout the network. The Arkivio auto-stor solution is policy-based and will automatically move data from, for example, NetApp storage (see Figure Three) to the Centera, based on user defined policies. The Arkivio configuration for NetApp to Centera archiving is also available through the EMC Select Program. Enterprise Strategy Group Page 6

8 Novell NetWare A popular platform for File and Print serving, Novell NetWare still has a large number of customers world wide. EMC Centera ISV partners CaminoSoft and Moonwalk enable customers to move NetWare files to the Centera while keeping these files accessible and online. Again, the archiving of these files is transparent to users and can be done as an online process. In addition, as a number of customers consider transitioning from NetWare environments to either Windows or Linux, features within CaminoSoft and Moonwalk environments provide an easier migration path. Mainframe Joins the Party The mainframe still sits at the core of many companies running mission-critical applications. The EMC HSM Migrator is software that is installed on z/os mainframes and integrated with IBM DFSMShsm. This capability enables companies to include mainframe datasets as part of file system archiving within an intelligent tiered storage environment. All of the same benefits apply with mainframe environments as with open systems. Remote Replication Another key aspect of leveraging Centera for is the ability to automatically replicate files off-site to other locations. This can be used as a tape or optical replacement strategy and/or disaster recovery process. Centera Replication provides a variety of asynchronous replication options including star and chain. Leveraging Centera s Single Instance technology, the amount of data replicated across the WAN is reduced compared to traditional storage replication. The Centera has self healing features that, in combination with replication, have led many customers to forego performing backups. Figure Three: Centera Remote Replication Enterprise Strategy Group Page 7

9 ESG s View File growth far exceeds other data types in terms of sheer capacity requirements and the quantity of files being created. File system archiving is emerging as one of the core applications to implement a disk-based archive solution. ESG Research recently found that 27 percent of companies surveyed stored over 10 million objects (an object is defined as a file and its metadata) of file system data, 11 percent stored over 50 million objects, and 7 percent stored over 100 million objects 2. The EMC Centera is the leading disk-based archive solution today and was built specifically to be easy to manage, provide a high level of scalability, and offer intelligent search capability as well as WORM and sophisticated retention and disposition policies. It has also been integrated with a number of data mover tools provided by EMC and other third-party vendors. ESG is seeing rapid adoption of archival storage as a part of an intelligent tiered storage environment. ESG Research found that 83 percent of companies surveyed either have installed or are planning to implement a disk-based archive solution. The combination of a lower cost, scaleable, easy to manage archive storage system with intelligent software to move data based on frequency of access, file name, and file size offers significant value. Companies are archiving , databases, and files in order to reduce capacity on primary storage systems, improve backup processes, enable search for faster access and retrieval of data, maintain compliance with federal mandates, improve workflow, and ultimately reduce cost. With the average amount of dormant files on storage systems at approximately 80 percent 3, there is a great deal of value to be gained with file system archiving. Customers who are evaluating file system archiving can easily start with a simple assessment to quickly understand how much static data resides on their primary storage. ESG has found that customers are saving thousands to millions of dollars annually by implementing intelligent tiered storage. EMC Centera-based solutions that include the appropriate policy engine software offer excellent options for customers considering intelligent tiered storage in their organization. All trademark names are property of their respective companies. Information contained in this publication has been obtained by sources The Enterprise Strategy Group (ESG) considers to be reliable but is not warranted by ESG. This publication may contain opinions of ESG, which are subject to change from time to time. This publication is copyrighted by The Enterprise Strategy Group, Inc. Any reproduction or redistribution of this publication, in whole or in part, whether in hard-copy format, electronically, or otherwise to persons not authorized to receive it, without the express consent of the Enterprise Strategy Group, Inc., is in violation of U.S. copyright law and will be subject to an action for civil damages and, if applicable, criminal prosecution. Should you have any questions, please contact ESG Client Relations at (508) ESG Research: Digital Archiving Analysis conducted from October to December SNIA Enterprise Strategy Group Page 8