An IT Briefing produced by. Archiving. Sponsored By:

Transcription

1 An IT Briefing produced by

2 By George Crump 2008 TechTarget BIO George Crump is President and founder of Storage Switzerland, an IT analyst firm focused on the storage and virtualization segments. With 25 years of experience designing storage solutions for data centers across the United States, Crump has seen the birth of such technologies as RAID, NAS, and SAN. Prior to founding Storage Switzerland, he was CTO of one of the nation's largest storage integrators, where he was in charge of technology testing, integration, and product selection. This IT Briefing is based on an Arrow/TechTarget Webcast,. This TechTarget IT Briefing covers the following topics: Introduction About Storage Switzerland Why The Archive Problem From Compliance to Litigation Storage Management Backup and Recovery Backup Archives Archive Solutions Integrator Advice Software Recommendations Hardware Recommendations Conclusion Common Questions Copyright 2008 George Crump. All Rights Reserved. Reproduction, adaptation, or translation without prior written permission is prohibited, except as allowed under the copyright laws. About TechTarget IT Briefings TechTarget IT Briefings provide the pertinent information that senior-level IT executives and managers need to make educated purchasing decisions. Originating from our industry-leading Vendor Connection and Expert Webcasts, TechTarget-produced IT Briefings turn Webcasts into easy-to-follow technical briefs, similar to white papers. Design Copyright TechTarget. All Rights Reserved. For inquiries and additional information, contact: Dennis Shiao Director of Product Management Webcasts, TechTarget dshiao@techtarget.com

3 Introduction This document addresses why archiving continues to be an important subject for integrators to raise with their customers, provides an overview of current management problems and some potential solutions, and finally offers advice on how integrators should be approaching this market. About Storage Switzerland Storage Switzerland is an independent analyst group that focuses primarily on the data storage and IT virtualization marketplaces. Storage Switzerland provides strategic consulting to manufacturers, resellers, integrators, and end users of data storage and IT virtualization technologies. They have a complete testing facility to validate products and solutions, and publish articles and white papers online in SearchStorage, ByteandSwitch, SearchData- Backup, and SearchStorageChannel. Their website, is updated weekly and contains over 70 articles, white papers, and product reviews from their laboratories. Why archiving is not a heavily penetrated market, despite publicity to the contrary. Most current customers are regulated by the SEC. Government regulations drove their implementation. Now the market is moving beyond regulation into retention and management for litigation readiness. Furthermore, continues to present a storage management and data protection problem, which an archive can help address. Some integrators may believe that the low-hanging fruit has already been picked. While it is true that large companies who were forced to make an archiving decision have already done so, significant opportunities exist in the litigation readiness and management spaces. A greater concern, from an integrator perspective, is the confusion over what should be retained and for how long. This confusion results in long, unproductive conversations with 1 IT Briefing: customers about retention problems. However, once integrators are educated about the issue, they can add significant value for their customers. Finally, customers may believe they can address storage management issues by updating their packages or restricting their users. The most common solution is to restrict the volume of per user. Some restrictions can be so small as to allow only a single day s traffic. At the other end of the spectrum, systems have improved abilities to retain large volumes of for a long time. However, this creates data protection problems as well as other issues addressed later in this document. The Archive Problem Figure 1 shows a graph of the compounded annual growth rate of the archiving market. The market is estimated to grow at about a 40% compound annual growth rate through Figure 1 also shows that archiving is an untapped market. The market is not restricted to upgrades for existing systems. In the next three years, the percentage of companies implementing archiving is expected to increase from about 22% to about 61% of available organizations. These statistics from Osterman Research indicate significant potential growth and therefore significant opportunity in the market today. From Compliance to Litigation The shift from compliance to litigation is the driving force behind this growth. Companies forced into compliance adopted archiving early. Litigation affects a broader section of the market. Sixty percent of companies with 10 or more employees have faced more than one lawsuit in their first 10 years of existence. Ninety-five percent of all discovery requests last year involved delivering a range of s. The company forced to deliver under discovery may not be the one being litigated against. For example, if a vendor is under litigation, that vendor s customers may be required to deliver all correspondence with the vendor. Finally, 50% of all litigation costs come from fulfilling pretrial discovery requests. Significant money is being spent to fulfill pretrial requests, even

4 Figure 1 if the company involved is not part of the lawsuit. Some current headlines include: Wachovia Fined $2.3 Million for Not Complying With SEC 17a Bank of America Securities to Pay SEC $10 Million Fine Morgan Stanley Fined $15 Million for Violations SEC Fines Seven More Broker/Dealers New headlines appear every week, a situation that will most likely continue. Storage Management System administrators continue to set mailbox restrictions, which in turn force users to export data to.pst files, a practice often endorsed by those same system administrators. Typically,.pst files are stored on the network, creating two problems. The first is a backup problem. One customer with a 50 MB restriction on user mailboxes, but with 14,000 users, had over 30 terabytes of.pst files stored on dedicated file servers or network-attached storage (NAS) devices. It became a major backup problem because one addition to a.pst file forced the backup application to back up the entire file. It was also a major storage problem. Second, the company is not protecting itself from litigation discovery by storing data in.pst files. These files can easily be requested, imported into a discovery application, indexed, and searched for specific information. Restricting mailbox size simply trades storage management problems in the application for storage management problems on the storage side. Another major issue around restricting mailbox size is loss in user productivity. The Osterman study referenced above showed that average users spent 1.5 to 2 hours a day not writing or replying to but simply managing their , deciding whether to put the in a.pst file, move it into a folder, or delete it. Another ramification is that, if a limitation is imposed on mailbox size, the users who reach it cannot send or receive until they reduce their mailbox size. Suddenly this becomes their top priority, regardless of other business issues. 2 IT Briefing:

5 The other option is to let the store grow. Modern applications allow for significantly larger stores than in the past, making this a possibility. However, unlimited growth will affect overall system performance and become a data protection challenge as well. Approximately 60% of decision makers in the Osterman study cited the growth in storage as a serious or very serious problem. Backup and Recovery Backup and recovery in the Exchange environment has improved significantly over the past 3 to 4 years. Most backup applications can do some sort of message-level backup and recovery, but with a few exceptions it is still a very slow process. Additionally, the number of point solutions designed to protect or enhance the Exchange environment has exploded. In many cases, these are additions to existing applications designed to give them some Exchange intelligence. For example, in Snapshots it might be an Exchange module that puts the Exchange database into a backup mode to get a clean snapshot of the message store. The multiplicity of point solutions creates multiple protection processes, all of which must be managed, which in itself is a problem. archiving can to a large extent address and in many cases eliminate the need for point solutions. Backup Archives Many customers think they can rely on their tape or disk-to-disk backup as their mechanism for archiving. These customers need to understand a typical process for recovery from a backup. First the customer needs to set up a target server. Then they have to determine which files cover the date range of the discovery request. In many cases, discovery covers every sent in the past relating to the case, which can be a large range. It can be very difficult to identify the needed files. Backup applications keep indexes cross-referencing files to tapes. These indexes can get very large and it is common practice to purge them after a period of time. If the file has been purged, the entire tape has to be rescanned into the backup application. If the correct tape has been selected, the application can search for relevant files, which can be extremely slow. If not, another tape has to be rescanned and the process repeated. Unfortunately, in some cases companies believe they have found the correct backups and extracted all relevant s. Then opposing counsel presents an they missed, and they look like they are hiding something, or are even fined. While most users understand at some level the process of recovery from a backup, it is useful to walk through the steps involved in a manual recovery of data. Archive Solutions A common archive architecture includes an archive server. As s are received they are copied in real time or on a schedule to the archive server. That server stores the on a database often called a container file to a local, relatively high-speed, very reliable disk. These are not the types of disks bought cheaply on the Internet, because they have to be extremely reliable. The containers generally store a preset number of s. When the container is full, another container is created and linked to the previous container. As containers age and become less relevant, they potentially can be moved to a less expensive storage medium. Traditionally, this has been tape or optical, but can also be one of the focused archived disk technologies that exist today. The point is that, with containers, users can move data to less expensive storage as time goes on. Two general classes of solutions are available today. At a high level, the debate is between hosted or internal solutions. Hosted solutions are interesting especially for smaller businesses. In a hosted solution, s are sent simultaneously to the customer s and to the provider s server. The provider stores, organizes, and indexes the s as they arrive. In a discovery, the provider assembles the data. The advantages of a hosted solution are lower up-front costs and no need for the internal staff and resources required to deploy and manage an internal archiving solution. The disadvantages are that costs increase in the long term, because more and more data is stored over time. Furthermore, most users resist handing over corporate assets, especially s, to a third party. Legitimate technology concerns exist over confirmation that the has arrived at the hosted site, as well as the bandwidth requirements between the sites. It is difficult for an integrator to participate unless they are also a provider. Integrator participation, if any, is in the form of an agent s fee. Finally, anecdotal evidence exists about slow search-andretrieval response times. Of these concerns, the primary ones are the increasing costs over time and handing over corporate assets. The primary advantage of an internal solution is that all components are purchased and owned by the customer. Integrators can have a significant role to play; however, the selection process can be confusing. 3 IT Briefing:

6 Many archive solutions are available, with new solutions and upgrades to existing solutions released every year. One customer had reviewed 15 different archive solutions. Customers also have to choose between archive-specific and integrated solutions, which are discussed in more detail later. The initial cost of an internal solution is higher because of the equipment and software that must be purchased, as well as implementation costs, but it is typically less expensive in the long term than a hosted solution. Finally, users must choose between a wide range of software solutions, hardware solutions, and now turnkey appliance solutions that integrate hardware and software into a single black box solution. Integrator Advice In this market, integrators still need to help their customers understand why they should implement archiving. Litigation is a viable reason. One challenge is from those customers who say, for example, that they only retain for 30 days the short-term retention problem. An illustration similar to that shown in Figure 2 can help counter this argument. It shows the life of a typical , illustrating how an can begin as a single exchange between two users on the company server. The recipient then forwards the to other users, who may then copy it to a.pst file, a personal account, or a laptop. As shown in Figure 3, deleting the on the server does remove it from that server. However, the persists in a number of other locations, two of which, illustrated by the blue dots, may not be accessible. The problem with a short-term deletion policy is that users may end up deleting the evidence they need to defend themselves. For example, a company was the subject of a human resources-related lawsuit regarding inappropriate . The recipient had forwarded the offending to a private account and so had a copy of it. The company had no archive in place and had deleted any evidence of the file. By accident, the company found the user s.pst file on a laptop that had not been reissued. In that.pst file, the company found an in which the user had forwarded the offending to a friend saying, Isn t this funny? Obviously the user was not offended. If the company had had an archive application, it could have easily found every instance of the , instead of relying on luck. In some cases, the discussion moves to the legal issues around a short-term retention policy. In many cases, it is frankly illegal. However, in most cases the Figure 2 4 IT Briefing:

7 Figure 3 company is simply removing its ability to defend itself, as the relevant s almost certainly exist somewhere else. Long-term retention is another challenge. Here the company decides to keep s forever, to always be protected. This strategy exposes the entire company history, but it is more defensible than short-term deletion. However the archiving application must offer sophisticated search and index capabilities, so that the index files are kept small and manageable. The final challenge is the hardest to overcome. For lack of a better term, call it Our legal counsel said. In this case, IT simply hands the problem over to legal. While it is no longer an IT issue, the company is still potentially at risk. One approach might be to try to talk directly with legal. Within IT, another approach is to shift the focus to storage management or data protection and recovery. Within storage management, the impact of.pst files on the network and on backup and the impact of very large stores are of considerable concern. Because applications can create very large stores, IT must buy more expensive primary storage and provide more server bandwidth. Data protection and recovery conversations can focus on the value of continuous protection solutions and application-aware snapshots. Software Recommendations In general, software falls into the category of either integrated or archive-specific tools. CommVault Software offers an example of the integrated approach. CommVault has made a large central application and connected backup, archiving, file archiving, and other functionality to it. This solution s value is that the same metadata database is used for backup and for archiving. The Intradyn archive appliance offers an ideal solution for integrators who are new to archiving or do not have a large professional services staff. This is a black box that connects to the network and simply starts archiving data. This solution allows the integrator to begin participating in the archive space, especially for small- to medium-sized businesses. Archive-specific tools include Mimosa, Symantec, and EMC Software. While Symantec and EMC have backup and other applications, the applications are 5 IT Briefing:

8 not fully integrated. Mimosa is a relatively new player on the market, focused solely on archiving. They have also built a file archive engine that leverages the archive, so they can generate multiple copies of similar files. Approximately 14 other archive applications exist, but Mimosa, Symantec, and EMC Software are ones that Storage Switzerland sees as being best received by end users. Hardware Recommendations Regarding hardware, users should not simply install the most inexpensive disk array available. Disk archive solutions designed specifically for this type of environment include EMC Centerra and Permabit Enterprise Disk Archive. These are examples of Content Addressable Storage (CAS)-based systems. These systems fingerprint the file and leverage that fingerprint to reduce duplicate data. The fingerprint also supports data-integrity checking over time. These systems are also scalable. Both the EMC and Permabit systems are grid-architected solutions, where storage can be increased by adding nodes to the grid cluster. They can grow into the high terabyte to petabyte size. A wide range of well-known tape and optical solutions also exist. While many customers are moving to disk-based archiving solutions, for some, tape solutions still make sense. The only concern is that media format changes over time. Tapes today may not be readable 10 years from now. Integrators should be sure to include a stop point in the plans they offer, where the customer migrates to a new technology at a point when both old and new technologies are available. Conclusion archiving remains an excellent market for storage integrators. Integrated appliances offer easier ways to get into the market. Software applications are more mature. Integrators no longer need to be as heavily involved in regulatory affairs as in the past. Certainly customer perceptions offer challenges to be overcome, but reliable solutions exist for integrators to recommend. 6 IT Briefing:

9 Common Questions Question: You said that the number of companies who will invest in archiving will increase from 22% of the available market to approximately 60% by Have integrators and resellers who are not actively involved in this business missed the boat? Answer: No, I do not think so. Clearly, the early movers have some advantages, especially related to the SEC and regulated businesses. However, second generation solutions are easier to learn, understand, and implement. In a fast growing market such as this, I think you could become a valuable participant even if you are starting from ground zero. Question: You mentioned the second generation of some of these products. Are these products mature enough to be reliable? Answer: That is a very good question. Data loss occurs at all levels whether the applications are new or mature. Generally it occurs infrequently. In many cases, it is a result of user errors, which are hard to admit, or integrator errors, which are even harder to admit. Furthermore, it is something that is very difficult for an integrator to test in a lab. During implementation, integrators will have to deal with a far larger volume of than most have available for testing. When choosing a solution, it is best to check references other than those the solution provider offers. Research their installed base and go directly to some customers, or talk with resellers who are not your direct competitors. Question: Most people think that backup is the key, but the critical aspect is recovery. However, recovery often seems to be an afterthought. Answer: Absolutely. We have often interviewed customers who never tested recovery. Integrators who are fortunate enough to have labs should test as many de-archives as possible. When speaking with customers, ask them if they have had any problems with recovery. At the same time, recognize that no product works flawlessly every time. Understand that mistakes will happen and your job as an integrator is to understand how those mistakes occur and help your end users avoid those situations. Question: Assuming a customer wants an internal archiving system, does it make sense to select an integrated over an archive-specific solution based on customer size? Answer: The physical size of the end user matters less than the specific characteristics of the archive they require. Whether the solution is integrated or standalone, look at the strength of the archive application as a standalone entity, and then count everything else as a bonus. If your issue is how many s per day the system can handle, do not believe the solution provider s numbers but talk with their customers. Question: Then it is a matter of prioritizing your needs and basing your decision on your needs? Answer: Absolutely. If the loss of an is cataclysmic, then you should be prepared to sacrifice some speed for 100% integrity. If speed is more important then, odd as it sounds, you might want to sacrifice some integrity. Question: Do you have any final thoughts? Answer: If you already have an existing archive application, now might be a good time to consider other options. Existing products have been upgraded and new entrants in the field have products that might be worth considering. Especially if your archiving solution has not worked particularly well for you, this might be a good time to revisit it. 7 IT Briefing:

10 About TechTarget We deliver the information IT pros need to be successful. TechTarget publishes targeted media that address your need for information and resources. Our network of technology-specific Web sites gives enterprise IT professionals access to experts and peers, original content, and links to relevant information from across the Internet. Our events give you access to vendor-neutral, expert commentary and advice on the issues and challenges you face daily. Our magazines give you in-depth analysis and guidance on the critical IT decisions you face. Practical technical advice and expert insights are distributed via specialized e-newsletters, video TechTalks, podcasts, blogs, and wikis. Our Webcasts allow IT pros to ask questions of technical experts. What makes TechTarget unique? TechTarget is squarely focused on the enterprise IT space. Our team of editors and network of industry experts provide the richest, most relevant content to IT professionals. We leverage the immediacy of the Web, the networking and face-to-face opportunities of events, the expert interaction of Webcasts, the laser-targeting of e-newsletters, and the richness and depth of our print media to create compelling and actionable information for enterprise IT professionals. Arrow_07_2008_ IT Briefing: