Global Headquarters: 5 Speen Street Framingham, MA USA P F

Transcription

1 Global Headquarters: 5 Speen Street Framingham, MA USA P F W H I T E P A P E R M a n a g i n g D a t a G r o w t h a n d M o n e t i z i n g I n f o r m a t i o n V a l u e : C o m p e t i n g i n t h e E x p a n d i n g U n i v e r s e o f D i g i t a l I n f o r m a t i o n a n d B i g D a t a Sponsored by: Isilon Storage Division of EMC Corp. Richard L. Villars October 2011 Benjamin Woo E X E C U T I V E S U M M A R Y The continued expansion of business-critical information and unstructured rich content within extended enterprises is changing the storage dynamic in a wide range of industries and organizations. Extracting value from this expanding universe of digital information, primarily through the use of Big Data storage solutions, is becoming a core business mandate. IT organizations must place a high priority on boosting the efficiency and reliability of their platforms for collecting, storing, protecting, and delivering this information. IDC defines Big Data technologies as a new generation of technologies and architectures, designed to economically extract value from very large volumes of a wide variety of data, by enabling high-velocity capture, discovery, and/or analysis. The IT community has spent many years virtualizing hardware abstracting hardware components from each other. In more recent years, advances have been made in abstracting applications from their underlying hardware. With Big Data, the opportunity arises to now liberate data from any single application and infrastructure. It enables enterprises to create new and innovative models for serving their customers. Ultimately, Big Data technologies will enable enterprises to discover and create strategic long-term organizational and economic value. The top storage challenges that your organization will face with existing dispersed storage assets (e.g., file servers and standard NAS devices) include: Inefficient use of deployed storage assets Continuous need for administrator intervention (rebalancing loads) Expensive and time-consuming upgrade/replacement cycles Difficulties in managing information in high-value Big Data analytics environments For many organizations (especially those in content-driven industries as well as cloud-based storage service providers themselves), addressing these limitations requires the deployment of a new class of storage solutions (scale-out storage solutions) that are optimized for data ingest, storage, and access. Aggressive use of innovative technologies such as EMC Isilon's scale-out NAS architecture will allow your IT organization to implement a number of best practices that will boost storage asset use, reduce operational overhead, and meet high data availability expectations.

2 I N T E R SEC T I O N O F I T, D A T A, A N D C O N T E N T I N T H E E X P A N D I N G D I G I T A L U N I V E R S E The creation, organization, and distribution of files and rich content are driving most of the growth in organizations' storage investments. In 2010, organizations around the world spent $16.6 billion on storage solutions to store the documents, images, videos, and machine-generated data records created by their employees, partners, and customers. This level of spending translated into 10,700 petabytes of disk storage capacity deployed for file-based data in Given current spending and capacity growth rates, organizations worldwide will be spending $24 billion in 2014 to deploy 67,150 petabytes (67 exabytes) of capacity to store all this information. The main question for IT executives, however, isn't, "Are we deploying too much capacity?" Instead, most will be asking, "Are we deploying enough capacity, and are we using that capacity effectively?" N o E n d i n S i g h t : F o r c e s D r i v i n g t h e I n f o r m a t i o n E x p l o s i o n Organizations ranging from small and medium-sized businesses (SMBs) to the largest enterprises and service providers are experiencing an explosion in the volume of content generated, accessed, and stored. Some of the forces driving this explosion apply equally across all sectors, while others pertain to specific industry transformations. Diversifying Information Assets and Uses Organizations ranging from SMBs to large enterprises: Rely on , collaboration tools, and Web sites to communicate and conduct business with customers and business partners Collect, store, and analyze more information about products, customers, and transactions Digitize records, design documents, and other types of unstructured data to boost efficiency, offer new services, and comply with evolving government regulations This increasing dependence on unstructured data types (e.g., documents, records, images, and video) for competitive differentiation as well as effective operations is causing an explosion in demand for file-based storage capacity versus more traditional block-based storage capacity. As more enterprises invest in Big Data technologies, the ability to reference files, rather than blocks, will become critical in unlocking the value in unstructured data over time. Regulations and legislation are requiring the maintenance of many data objects (many of which are stored in files). Through Big Data technologies, enterprises can discover and create competitive advantage necessary for rapid growth and increased profits. 2 # IDC

3 The rapid growth in file-based storage shipments is posing a number of challenges for IT and storage administrators: Achieving economies of scale, operational efficiency, comprehensive disaster recovery and business continuity, and information governance through file server/nas consolidation Complying with best practices and government regulations by properly retaining and storing file-type information over a predetermined period of time Reducing management complexity and eliminating performance and capacity management constraints to align storage infrastructure investments with the demands of the business Long-Term Value of Information Organizations are increasingly creating, storing, and accessing more information for long periods of time. This need has been driven by regulations and legislation and the increasing number of regulated industries. Over time, firms have come under increasing pressure to store and retain large amounts of data in content depots to quickly retrieve this information when needed. For a growing number of organizations, the ability to perform business analysis and discovery on existing data is expected to result in new business development and the creation of competitive advantage. Firms are mining their historical data to analyze and extract data for market intelligence, product planning, and inventory planning. In R&D environments, reuse of historical information can yield vast savings in time and effort, which in turn saves money and in some cases provides competitive advantage by shrinking the time required to bring products to market. Intelligent Industries and Industry Digitization The most profound and long-lasting driver of the content explosion is the current and expanding "digitization" of entire industries, including: Mobile communications (applications for smartphones and tablets) Utilities (smart grid) Legal services (ediscovery and review) Life sciences (genomic sequencing, protein sequencing) Healthcare (digital x-rays, electronic medical records) Media/entertainment (HD conversion, digital production, digital delivery) These developments are driving an IT investment wave by "intelligent industries" and organizations whose business is almost entirely based on the storage, organization, long-term retention, timely retrieval, and near continuous analysis of data and other unstructured content IDC #

4 For example, in the healthcare industry, the ability not only to manage patient records within a PACS but also to perform analytics within those records (in an anonymous fashion) for the purpose of medical advancement, using Big Data technologies, can yield dramatic possibilities for society. The primary job of IT teams in these industries is building, managing, and ultimately expanding the appropriate access to data stored in large content depots. Immense amounts of analytics can be performed on this data to enable strategic long-term value creation for enterprises. As an aggregate community, data in content depots and other unstructured data accounted for 32% of all storage capacity shipped in 2010 and will account for 49% of all datacenter storage capacity shipped into organizations by While specific requirements will vary by industry (e.g., a utility that is ingesting sensor data from hundreds of thousands of meters every five minutes versus media/entertainment companies that need to concurrently stream a song or movie to millions of mobile users), they all need storage solutions that can: Support reliable and faster access to significantly more and larger files for much longer periods of time Ensure that the retention and sharing of content is both cost-efficient and secure Enhance (aka monetize) the value of unstructured data through more sophisticated business intelligence and analytics solutions Big Data: Information Everywhere, But Where's the Knowledge? One of the major forces driving the data explosion and industry digitization is the proliferation of devices located at the periphery of the network (e.g., embedded sensors, smartphones, tablet computers, and portable diagnostic systems). All of these data generators create new opportunities to "extract more value" in areas such as human genomics, healthcare, oil and gas, search, video surveillance, and finance, to name just a few. Over the next several years, we will be entering the age of "Big Data." If you did a little investigating inside some of your own business units, you would likely be surprised at what is already there in terms of Big Data projects. Thanks to a sustained and dramatic decline in the costs of compute power and storage capacity, it is possible for some bright staff in your organization to effectively deal with specific data variety, volume, and velocity problems. Even better, they can do so while still being able to take advantage of the knowledge (get value). As an organization makes the transition from Big Data as a number of isolated "junior science projects" to Big Data as a core business resource, concerns about the impact on current and future datacenters will increase. Big Data creates infrastructure challenges that extend from data creation to data collection to data storage and analysis. The key questions for you as CIO or IT architect are: How do we get started, and then what will it take to set up an enterprise-class Big Data environment? 4 # IDC

5 How big of an impact can Big Data have on our current storage environment in terms of capacity, performance, and data management? What organizational and technical challenges might inhibit our ability to get the maximum value from Big Data investments? IDC believes that organizations that are best able to make real-time business decisions using Big Data solutions will thrive, while those that are unable to embrace and make use of this shift will increasingly find themselves at a competitive disadvantage in the market and face potential failure. L i m i t a t i o n s o f E x i s t i n g S t o r a g e S o l u t i o n s i n L a r g e C o n t e n t a n d B i g D a t a E n v i r o n m e n t s The transformation of IT environments through greater use of virtualization and the continued expansion in information creation, consumption, analysis, and retention will have a dramatic impact on enterprises across a wide range of industries and geographies in the next five years. This shift is already affecting how companies assess storage solutions in terms of system performance, operational efficiency, and system intelligence. Areas where many existing storage solutions are part of the problem, not part of the answer include: Dispersal of storage assets (primarily file servers and standalone NAS platforms) within datacenters and across the extended enterprise Inability to intelligently optimize/tier information assets based on either performance or capacity requirements Difficulties in classifying, reorganizing, and rapidly analyzing original content and metadata associated with newly created and archived files Limited (and expensive options) to set up business continuity, disaster recovery, and data migration processes and policies for very large (multipetabyte) content and Big Data pools New technologies such as multicore servers, unified 10GbE network fabrics, converged IT infrastructure, and cloud-based services are already at the heart of many organizations' next-generation IT plans. In the very near future, the management, organization, and continuous mining of large content pools will become an equally important task for many datacenter administrators. The greater use of robust "Big Data" solutions in ongoing business processes is where these two developments will intersect. Enabling a distributed, scale-out architecture, particularly as it relates to data storage, protection, and access, is the foundational building block to being able to evolve organizations from application-focused to data/information-focused enterprises. In this data-intense marketplace, selecting and deploying a scalable and flexible storage platform that can address the diverse, but interdependent information management needs of your organization will be critical IDC #

6 P I C K I N G T H E R I G H T S T O R A G E S O L U T I O N F O R B U I L D I N G L O N G - T E R M S U C C E S S W I T H B I G D A T A While not all companies, regardless of size, are in industries experiencing exponential data growth, the scale of content growth across all companies is significant. If you haven't done so already, your organization will need to consider new approaches for dealing with data growth and active access to archive data in the very near future. For some companies, especially small and midsize organizations, this transition will involve greater use of cloud-based storage services. For many others (especially organizations in content-driven industries as well as cloud-based storage service providers themselves), it will require deployment of a new class of storage solutions (scale-out storage solutions) that are optimized for data ingest, storage, and access. What are scale-out storage solutions? Scale-out solutions use a distributed file system or object store to span multiple disk storage modules while presenting a single name space. The scale-out architecture such as that used in EMC Isilon's scale-out NAS platform allows for flexible scalability in performance and capacity independent of each other. Regardless of the size of your environment, you need to consider a set of fundamental, common elements when evaluating a scale-out solution: Multidimensional scalability. Solutions that offer elastic scalability will arm customers with the ability to manage rapidly expanding and varying loads. Many content environments start with modest beginnings but can quickly overwhelm the initial environment. Scalability isn't just about expanding hardware capacity, however. In content-intensive environments, scalability must also encompass throughput, file size, and file volume. The best solutions will allow you to scale each dimension independently to ensure the optimal configuration for your specific needs. Storage efficiency. Solutions in this area will enable easy adoption of a continually expanding range of storage asset optimization technologies. These capabilities include data reduction technologies (e.g., data deduplication, compression, thin provisioning) and data optimization technologies (e.g., automated data tiering with SSDs and capacity-optimized HDDs). The best solutions will allow you to introduce or recalibrate these capabilities without major disruptions or system migrations. Intelligent information management. Solutions in this area will allow organizations to continuously observe data usage changes, analyze usage patterns, and reorganize information pools in near real time (often referred to as Big Data). The best solutions will provide tight integration with advanced metadata, index, and analytics solutions (developed internally or provided by a third-party developer) that enable more intelligent information management as volumes of data sets grow and new use cases proliferate. 6 # IDC

7 With these capabilities, your IT organization can implement a number of best practices that will boost storage asset use, reduce operational overhead, and meet high data availability expectations, including: Enabling faster and nondisruptive addition of capacity or performance in more cost-effective increments Leveraging automatic data tiering to maximize asset use and dramatically reducing the need for administrator intervention Reducing or delaying spending on high-performance storage capacity and cutting average storage capacity acquisition cost though greater use of high-capacity/ low-cost disk storage It is difficult to predict an organization's data storage needs over time. The combination of data capacity efficiency technologies and the compounding growth in data (from both a compliance perspective and a data generation perspective) means that the ability to scale capacity and performance, either independently or in parallel, will be necessary for any enterprise that wants to leverage Big Data technologies to unlock the as yet untapped value of the data it owns. The remainder of this white paper examines the scale-out storage solutions from Isilon (a division of EMC) and assesses how well they address organizations' growing need for storage systems that manage new Big Data applications and increasingly complex content creation, access, and archiving requirements. E M C I S I L O N S C A L E - O U T S T O R A G E S O L U T I O N S F O R B I G D A T A EMC Isilon is a global leader in the development and deployment of scale-out storage solutions for a wide range of content-driven organizations and large content depots in industries such as financial services, government, public cloud services, healthcare/life sciences, and media/entertainment. As of early 2011, it had over 2,000 customers worldwide. Isilon's stated mission is to remove the barriers that exist between businesses and their data by helping customers get to the information they need, manipulate it, and act on it. It seeks to meet this goal by delivering a scale-out storage platform with near limitless capacity, high levels of performance, and advanced data services that make the storage and delivery of content simpler and more efficient. Isilon asserts that its hallmark architectural characteristics are an ideal solution for the challenges posed by Big Data storage requirements. I s i l o n ' s S c a l e - O u t P l a t f o r m : D e l i v e r i n g S c a l e i n M u l t i p l e D i m e n s i o n s The foundation of Isilon's scale-out storage solution (the Isilon IQ cluster) is the OneFS operating system, currently in its sixth generation. It provides a unified software foundation for the basic storage services (file system, volume manager, and RAID) that can be distributed across multiple storage hardware nodes, tuned for different requirements IDC #

8 At present, Isilon delivers six specialized hardware nodes: S-Series IOPS-Intensive Applications (including SSD support) X-Series High Concurrent and Sequential Throughput Workflows NL-Series High Density/High Capacity for Archival Content Performance Accelerator Independent Scaling of Throughput EX Capacity Extension Independent Scaling of Capacity Backup Accelerator High-Speed and Scalable Backup and Restore Solution The combination of OneFS and these tuned hardware nodes allows organizations to scale independently across multiple dimensions (capacity, I/O performance, throughput) while managing the entire content pool as a single entity. The advantages to this approach include: Reduced management staff overhead. Organizations can manage petabytes of storage capacity with a single FTE administrator. Highly tunable and scalable capacity and throughput levels. Organizations can deploy up to 15 petabytes of capacity, achieving up to 85GBps of throughput and supporting over 1 million IOPS in a single file system, as validated by the SPECsfs2008 benchmarking test, released in June Rapid system expansion and upgrades. Administrators can nondisruptively add capacity and/or performance resources within 60 seconds so that they no longer need to buy/overprovision capacity months or years in advance of actual need. Evergreen (persistent) storage. Using the OneFS autobalance feature, IT teams can quickly and nondisruptively migrate data from older nodes (due for replacement) to new, higher-performing or more cost-effective nodes as they are introduced. I s i l o n ' s A p p l i c a t i o n S o f t w a r e P o r t f o l i o : E n a b l i n g S t o r a g e E f f i c i e n c y a n d I n f o r m a t i o n M a n a g e m e n t Providing a cost-effective and future-proof scale-out storage platform isn't sufficient to meet the increasingly complex needs of content-intensive organizations. IT teams in these organizations also require a complementary set of application and management software solutions to address operational and administrative challenges. 8 # IDC

9 Isilon also provides a suite of advanced application and management solutions to address customers' storage efficiency and information management requirements. The solutions include the following: SmartQuotas provides the ability to set up, control, and limit storage usage subpools in Isilon scale-out storage (IQ cluster) to meet storage challenges for different content types and use cases. SmartPools provides automated tiering, workflow isolation, higher utilization, and independent scalability for different content pools in a single Isilon IQ cluster. SyncIQ includes policy-based file replication management tools to quickly and efficiently move mission-critical data through a globally distributed network for load balancing or business continuity/disaster recovery. SmartConnect simplifies the management and maximizes the performance of large numbers of clients that access content from Isilon IQ clusters. SnapShotIQ enables faster and more efficient snapshot scheduling to provide more flexibility in meeting recovery point and recovery time goals. InsightIQ includes advanced analytics/reporting tools to optimize applications, correlate workflow and network events, and more accurately forecast future storage needs. Aspera for Isilon IQ is a predictable, high-performance wide area file and content delivery solution designed to move large files long distances (jointly developed/provided by Aspera and Isilon). I n d u s t r y a n d I T S o l u t i o n s The most important characteristic of content-driven organizations is the diversity of their needs, based upon industry and IT function being addressed. Isilon understands that while providing a single, flexible platform is critical, it's useful only if tuned properly to address your organization's unique requirements. Currently, Isilon and its partners provide customers with assessment, design, and implementation services targeting the follow use cases: Industry-specific solutions Energy Financial quantitative analysis Geospatial imaging Government (research/reconnaissance) Higher education IC design and EDA 2011 IDC #

10 Internet and cloud services Life sciences powers the activities of the world's foremost research facilities Media and entertainment IT solutions Archiving, backup, and recovery Cloud storage ediscovery Disaster recovery and business continuance High-performance computing Home directories and file sharing Virtualization These industries and these IT solutions are the driving forces behind the rapid evolution toward Big Data architectures. Initially, many Big Data implementations will be specific to solving a particular business problem. Ultimately, these same industries will be applying Big Data technologies to the generation of strategic long-term organizational and economic value. C h a l l e n g e s / O p p o r t u n i t i e s f o r E M C I s i l o n Given continued rapid data growth and the increasing role of rich content in organizations' new application and services plans, the storage and information management challenges posed by Isilon's content-driven customers and prospects will only increase in the coming years. Isilon needs to address a number of requirements as it expands its role in organizations' content-intensive environments: Continue to improve underlying storage hardware capacity, performance, and power management efficiencies through technologies such as data compression, more tunable/intelligent automated data tiering, and support for even denser/more power-efficient HDD solutions Establish closer technical and business ties with leading content creation, distribution, and analytics (Big Data) solutions suppliers that will make it easier for customers to fully exploit the information stored within Isilon clusters Extend the reach of the Isilon solutions to better address the growing content needs of SMBs as well as large enterprises (This effort should include expanding the ecosystem of cloud-based storage service provider partners.) 10 # IDC

11 E S S E N T I A L G U I D A N C E The digital universe will continue to expand, and organizations will spend more time and money collecting, storing, and monetizing fast-growing data sets and rich pools of content for the foreseeable future. IDC expects companies such as EMC Isilon to develop and deliver increasing numbers of tools and practices to help the digital universe's guardians the CIOs of the world deal with the here and now of their own particular content depots and Big Data investments. In the end, however, a CIO's challenges will be not just technical how to find, manage, and protect information but also organizational. The CIO must take a leadership role in driving the adoption of new information-taming technologies and practices, such as EMC Isilon's scale-out storage solutions. Making the leap will require organizational change, not just a few new systems or more software. The success of many enterprises in the coming years will be determined by how successful the CIO is in driving the required enterprisewide adjustment to the new realities of the data- and content-driven enterprise. C o p y r i g h t N o t i c e External Publication of IDC Information and Data Any IDC information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right to deny approval of external usage for any reason. Copyright 2011 IDC. Reproduction without written permission is completely forbidden IDC #