EMC in the Era of Big Data and Cloud

Analysis from The Wikibon Project January 2011 EMC in the Era of Big Data and Cloud Stuart Miniman, David Vellante, David Floyer

As we enter the new decade, storage veteran EMC is 30 years old. In the 1990s, EMC was a Wall Street darling that helped to create the storage systems industry. In the 2000s, EMC greatly broadened its portfolio through acquisitions. We are looking at this announcement as an indicator as what will define EMC going forward, as we enter the era of big data and cloud. EMC s January 18, 2011, storage announcement the first of this big data era is one of its broadest ever. It includes over 40 new products and touches most of the company s traditional platform lines, from Symmetrix at the mission critical high end to a new midrange architecture that further evolves EMC s flagship offerings in the sweet-spot of the market. Today s announcement also demonstrates both steady improvements and new capabilities within EMC s relatively new Backup and Recovery Systems (BRS) division. In addition, the announcement included a surprise in the form of the VNXe, a brand new platform that dramatically simplifies enterprise-class storage to a level not previously seen by EMC customers. Product Line Update Enterprise Storage (Symmetrix) Midrange Storage Enginuity 5875 includes: FAST VP (Virtual Pools; i.e., sub- LUN tiering) and dozens of other improvements including faster provisioning and increased performance VNX platform converges CLARiiON (block) and Celerra (file) into a single modular hardware architecture SMB/ROBO Storage New platform VNXe starts at < $10k Backup and Recovery Table 1: Summary of EMC s announcement Increased performance and capacity of Data Domain solutions plus the introduction of Data Domain Archiver, which consolidates backup and archiving to a common platform We are exiting a decade of rapid storage innovation where several VC-backed companies shipped products positioned to chip away at EMC s core Symmetrix and CLARiiON product lines. 3PAR, Compellent, XIV, EqualLogic and LeftHand Networks all set out to offer enterprise-class features in a simplified package. This served as an important catalyst, lighting a competitive fire under EMC and pressing the industry leader to respond by including feature sets to facilitate simplified provisioning and storage management (e.g., thin provisioning and automated tiering). Ironically, these companies are gone now as their intellectual property and businesses have been subsumed by larger companies. Meanwhile, EMC remains the #1 supplier of external storage systems from a revenue perspective. What does this announcement say about EMC s product strategy? It says that despite big picture marketing campaigns from EMC and its competitors (e.g., virtualization, private clouds, unified storage), EMC is emphasizing technology advances Wikibon.org 1 of 6

that provide real, tactical benefits to customers and minimize disruption to the installed base. This announcement underscores that EMC s broad portfolio is not consolidating; rather, it is expanding to include core products and new entrants such as the VNXe. In addition, EMC s aggressive acquisition strategy means that customers can expect continued diversity in the product line as seen with new Data Domain capabilities for backup, recovery and archiving; and in the not-too-distant future with new scale out NAS offerings from the Isilon acquisition. The bottom line from a product standpoint is that EMC is developing new capabilities to target growth opportunities (e.g., SMB/ROBO, archiving), while at the same time co-opting many of the industry s recent innovations around simplicity and automation making them features of products within its large portfolio. What does the announcement mean to storage users? There are several major takeaways from this announcement from a customer perspective that involve technology dislocations and changes to the nature of storage competition going forward; specifically: Transforming skill sets. The consumerization of IT trend continues unabated. Virtualization, automation and self-service are at the center of the response from traditional IT companies such as EMC. The mandate for delivering IT-as-a-service (ITaaS) requires simpler IT and on-demand models are becoming compulsory. This is leading to a generalization of IT skills versus decades of emphasis on specialized technical capabilities, particularly at the lower end of the market. The VNXe announcement underscores this trend and is an excellent example of a simplified storage platform developed for generalists in the SMB space, not storage specialists. In a different way, flash storage and FAST VP is simplifying and automating the movement of data between storage tiers, freeing administrators to work on more strategic tasks. A changing storage hierarchy. While the storage industry always delivers more capacity, technology is shifting in non-traditional ways. We re seeing a major move toward flash-based architectures (versus disk-only systems) and a continuance of the narrowing of tape use cases. The adoption of technologies such as FAST VP and the Data Domain Archiver are examples of products attempting to capitalize on these shifts and users must plan for increased flash exploitation throughout the storage hierarchy. As well, disk is increasingly attractive in use cases beyond primary storage and will continue to steadily encroach on tape. Innovation through software. EMC and many other firms are betting the ranch on commodity hardware based on Intel platforms. The nature of competition in storage is shifting from hardware to software. Increases in functionality used to require major hardware upgrades and big platform announcements. While bigger, better faster hardware will continue to ship, functional updates through software will be the primary value domain of storage companies. Indeed, the vast majority of value in today s announcements (specifically) and storage announcements generally involve lines of software code much more so than new hardware capabilities. Wikibon.org 2 of 6

Where Does Cloud Fit with Unified Storage? The industry is going through what Nicholas Carr calls The Big Switch a move from traditional data centers filled with equipment to a more agile, flexible, responsive, self-service set of offerings that have a transparent cost to the end user: cloud computing. Public cloud vendors provide IT services with scalable solutions and dynamic costs similar to an electrical utility. Unlike electricity, there is a significant difference to an IT department related to the security, management, performance and control of an outsourced solution. Traditional data center vendors have created private cloud offerings, which provide many of the same benefits as public clouds through the use of virtualization and optimized, converged infrastructure. The goal is not necessarily to have the same cost as a public cloud, but to be able to provide the business transparency so that comparisons of the requirements and costs can be made allowing for informed decisions that balance risk and reward. Amazingly, EMC, a storage company, is the central player in this transition due to its acquisition of VMware in 2003. While EMC knew that VMware technology shifted intelligence and dollars away from servers, no one was predicting that hypervisors would become the new OS of the enterprise. Virtualization has had the largest impact on architectural design for servers, network and storage in the last decade. Not only are the individual technologies affected, but the inherent abstraction of virtualization causes collisions of domains which require that customers and vendors break-down the walls between infrastructure silos to allow for the deployment of services rather than just boxes. Unified storage is a natural fit for virtualization. Specifically, hypervisor deployments have been the rising tide that has lifted both block and file networked storage, with customers installing a mix of NFS, iscsi and FC/FCoE (a majority of customers are running multiple network protocols). Customers require unified storage to be a flexible, multi-purpose building block that can be used across a broad spectrum of applications (including, but not limited to, virtualized environments). While EMC has offered some type of unified storage option for years (e.g., block behind a NAS head), the release of the VNX family creates an integrated and more competitive offering. As virtualization has matured, it is moving beyond a tool to increase server utilization rates towards enabling IT to be consumed as a service (ITaaS). For ITaaS to be realized, solutions need to be more than just simplified, they also need to be automated. Simplification and automation touch many aspects of the storage solution including automated tiering of storage (SSD/HDD) where the system self-regulates the resources. On balance, smart storage systems are far more efficient at managing data placement than humans. Every vendor in the industry has been working on simplification of storage management over the past decade, mostly with incremental improvements to existing legacy platforms. One of the more interesting pieces of EMC s new entry level VNXe solution is that the management of the system was built from the ground up, similar to the startups that emerged last decade. This means that not only could the latest design tools be used, but rather than taking storage language and simplifying it (e.g., simplifying LUN management), the VNXe design starts from the application point of view. This resulted in a solution that not only an IT generalist, but even a business person working with Exchange Wikibon.org 3 of 6

or SQL could utilize. In short, the VNXe provides an ITaaS solution for the underserved SMB and remote office marketplaces. How does this Announcement Impact the Data Center as a Whole? Along with the growth of VMware, EMC has expanded its reach into the data center as a whole. The introduction of the VCE joint venture with Cisco, and the Atmos initiative have given EMC integrated stack offerings that reflect the trends of simplification, unification and integration of former IT components into larger building blocks. Wikibon has looked at these trends in detail and has modeled their impact on data center budgets. The main conclusion of this research is that simplification, unification and integration are very strong trends within IT, and will have a profound impact on IT budgets. The major impacts of these trends are: 1. The cost of supporting existing traditional applications will be reduced by 39% over five years; 2. These cost savings will go into additional external services, including cloud services; 3. The percentage of budget spend on external services will rise from 14% to 53% over the five year period. A summary of the transformation of IT budgets is shown in Figure 1. The additional benefits of this trend are: 1. The improvement in time to implement new projects and applications will be significantly reduced; 2. The business risks of new projects will be reduced from later and faster implementation, and the use of external services. Figure 1: Transformation of IT Budget as a Result of Technology Simplification, Unification & Integration. Source: Wikibon 2011 Wikibon.org 4 of 6

The availability of simplified stacks will come from large IT vendors who will put together their own integrated stacks, and coalitions of vendors who will join together to create integrated stacks based on simplified components. EMC is simplifying and unifying its products, has key components of the stack with storage and the hypervisor, and is partnering with other IT vendors to bring this vision to market. What about Big Data? The big data era is upon us and virtually every storage announcement will be assessed in this context going forward. Traditionally, the emphasis on data has been about the difficulty of managing explosive growth. The industry in our view is undergoing a bit flip where data is becoming a strategic asset. We ve heard this story before however the difference this time around is that new technologies actually exist to take advantage of massive amounts of data and turn it into an information asset. The Hadoop movement is an example where organizations are using new techniques to manage massive amounts of data. At Hadoop World 2010 in October, for instance, the average reported database size of attendees was a whopping 115TB. This is game-changing. Analytics is an obvious example where firms in financial services, health care and government are taking advantage of the ability to process, analyze and act upon data not just sampling data but entire data sets; in near real time. Beyond analytics, companies are monetizing big data, especially Web firms such as Twitter, Facebook, Zynga and many others. The other major trend in the context of big data is, Little Data. Specifically, the typical home in a developed country contains 4-5 microprocessors and up to 100 microcontrollers. In addition, according to various market research firms, more than one billion cell phones shipped in 2010. The prediction for 2011 is it will be the year of the smart phone with Apple shipping more than 100M iphones and several vendors bringing sub $100 Android handsets to the market. These smart devices are becoming a connected set of sensors creating huge volumes of data in the form of video, text messages, social gestural data and the like. Wikibon will be watching these trends closely. Our fundamental premise is that every organization is creating some form of big data and this data will have increasing value to all organizations. Of particular interest to our community are EMC s recent acquisitions of Greenplum and Isilon, both directly involved in the big data movement. How EMC manages these acquisitions and other moves it makes in this space will determine its position in the big data arms race and also guide many decisions at IT organizations. There are three relevant points related to big data storage that bear watching: 1. Big data will require infrastructure that is simple, efficient and inexpensive to manage, yet provides access to multiple platforms and data sources simultaneously. As such, virtualization and storage optimization techniques will be required to exploit big data especially where legacy infrastructure is involved; Wikibon.org 5 of 6

2. The big data trend will heavily utilize flash across the storage hierarchy to deliver real- or near-real-time performance for users. This will be seen in the disk system and at the memory level of the hierarchy; 3. Real-time, big data analysis will not be done from tape-based systems; increasingly disk-based architectures will be the norm for analyzing massive archives. Action Item: EMC s big announcement signals a new period of change in the storage business. While much of the announcement underscores the need to make near-term tactical decisions (e.g., which midrange platform to deploy and how to get the best price), practitioners should use this announcement as an opportunity to begin evolving toward ITaaS, understanding how to leverage big data and where EMC fits in the broader spectrum of infrastructure players required to deliver on this vision. Wikibon.org 6 of 6