THE ANSWERS TO YOUR TOP THREE QUESTIONS ABOUT PROTECTING BIG DATA

Transcription

1 THE ANSWERS TO YOUR TOP THREE QUESTIONS ABOUT PROTECTING BIG DATA Does Big Data Have to Be Big to Warrant New Protection Practices? Can Big Data Be Too Big to Back Up? Is Building a Protected Data Lake a Lesson in Compromise? DOES BIG DATA HAVE TO BE BIG TO WARRANT NEW PROTECTION PRACTICES? By many media and vendor accounts, Big Data is simply that: big data. Large volumes of structured or unstructured data, the likes of which are generally associated with companies in data-crunching industries like oil and gas, seismology, genomics and finance. Even Wikipedia defines Big Data as the collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing. HANDOUT

2 The question is: Does Big Data have to be big? And if it does, how big is big exactly? Petabytes? Exabytes? Zettabytes? Bigger? And if size is the determining factor, is Big Data, then, only a big-company, select-industry concern? The answer to all three is no. Two great articles one from the folks at SearchCIO and the other from The Harvard Business Review (HBR) discuss this very topic. In the SearchCIO article, editor Harvey Koeppel says thinking about big data as a bigger version of little data puts us at a significant disadvantage when it comes to understanding what it is; how it can be harnessed; and the enormous value that it has created already and will continue to create with respect to how we live, work and play. This type of thinking limits the discussion and application of Big Data to a world that existed two, three or more years ago. It doesn t account for the huge technology changes that are transforming our lives and businesses today, and it doesn t account for the big value of the data being created, the protection this necessitates, or the millions of pockets of small data that organizations, big and small, will analyze and take action against that are a reality of the Internet of Things. As Andrew McAfee and Erik Brynjolfsson point out in the HBR piece Big Data: The Management Revolution, velocity (i.e., the speed of data creation) and variety (i.e., the diverse sources of data) have as much, if not more, to do with Big Data than the volume of data that s actually created. In this way, all organizations are Big Data generators. Some just create more of it, faster, and from more sources. Therefore, it isn t really how big Big Data is, but how it is leveraged for businessbenefit that really matters. It s all about using Big Data intelligently, explain McAfee and Brynjolfsson. As a proof point, McAfee and Brynjolfsson, along with a research team from the MIT Center for Digital Business, McKinsey s business technology office, and Wharton, surveyed 330 North American companies about their Big Data practices. The survey found that companies that were data-driven were 5% more productive and 6% more profitable than those that weren t. So, don t let semantics hold your organization back. Whether you re a large enterprise with massive data volumes, a midsize organization with pockets of distributed data or a small business with data volumes that are large for your environment, your data has big value and requires improved levels of protection. Doing so will not only help reduce risk and costs, but also drive business growth and revenue. CAN BIG DATA BE TOO BIG TO BACK UP? Is Big Data too big to back up? Is the new Boeing Dreamliner too big for safety inspections? Was the Titanic too big to need lifeboats? Are some banks too big to follow basic asset safety rules? Was Gmail too big to lose data? Definitely not. But that doesn t mean you can back up Big Data with traditional backup techniques. Big Data s volume, variety, and velocity otherwise known as the three V s of Big Data require data protection teams to transform their approach. Volume scalability to accommodate massive amounts of data. As data capacity increases, traditional backup and recovery windows become unmanageable. This is not a new challenge, but Big Data accelerates the pain. Full backup and recovery become virtually irrelevant when you approach windows of multiple days. Incremental backups will scale only if you leverage intelligence in the data source; on its own, a backup agent cannot find the new data fast enough. Granular recoveries become challenging either due to the size of the objects to recover, or due to the process of sifting through trillions of objects to locate the object. The answer to the volume challenge is snapshot and replication technology such as that offered through the EMC Data Protection Suite with EMC

3 WHAT IS A DATA LAKE? A data lake is a platform that provides for the storage, management and analysis of Big Data by bringing together data in realtime, near real-time or in batch, and from disparate sources across multiple protocols. A key requirement for data lakes is extremely high levels of processing performance, and simple, costeffective and secure management of massive volumes of data. The data lake provides a shared platform for analytics and data-driven applications. It lets you build solutions for IT operations and security as well as for business analytics. The data lake provides: A standardized set of mechanisms to capture data and store it in a cost-effective storage layer. Distributed compute power to analyze monitoring data, build models and perform real-time analysis and scoring. Mechanisms to integrate with existing enterprise monitoring solutions and visualization tools. NetWorker Snapshot Management. Snapshots are created and managed while ensuring there is minimal disruption to the production environment, and then subsequently backed up to deduplicated disk, in this case EMC Data Domain systems. The deduplicated disk synthesizes the incremental changes into full, space-efficient, native format backup copies. Snapshot and replication is the best way to scale to meet Big Data s volume. Variety ability to protect a variety of data types with a single solution. As the types of data increase, so do the applications that create and use that data. Therefore, it is increasingly difficult for the backup team to deliver a onesize-fits-all centralized solution for all of those applications. Backup tools can t build application-intelligent agents fast enough to cover all customers use cases. Furthermore, the application owners expect more visibility and control over their backups; they don t want to be relegated to bystanders in the protection of their data. The only way to cope with these changes is for backup teams to offer a variety of services to the application teams, to help them meet protection SLAs. In some cases, the backup team will run the backups. In others, they will provide backup storage, cataloging, and reporting. Or alternatively, they may offer only cataloging and reporting. In other words, they need to transform their backup environment and behave like service providers to their Big Data customers. For this to work, however, the team needs infrastructure that can work with the variety inherent in any enterprise, such as EMC Data Domain Systems and EMC Data Protection Advisor. Velocity the performance and speed to protect rapidly growing data. Big Data users want high-performance access to their data whenever and wherever they need it. Therefore, there is no backup window. Backup must minimize its resource impact on the production environment. Second, recoveries must happen quickly. Again, nothing is faster than leveraging protected data directly off protection storage. Deduplicated disk may not have the I/O performance of primary disk but it can still provide performance that enables quick access to your data. Ultimately, there s nothing faster than instantly accessing your data, instead of waiting hours for recoveries to complete. In other words, Big Data protection is all about snapshot and replication to deduplicated disk, with the backup team shifting into a service provider role. Not surprisingly, our most successful customers are following that same transformation for their traditional data. Whether your backup transformation is being driven by building a private cloud, Big Data, absorbing multiple acquisitions, or just doing more with less, the transformation is the same. Bottom line: Big Data is not too big to back up but you ve got to be willing to change. IS BUILDING A PROTECTED DATA LAKE A LESSON IN COMPROMISE? First-Hand Lessons Learned from Stephen Manley, CTO, EMC Core Technologies Two years ago, EMC Core Technologies set a goal to replace much of the on-premise, human-intensive management of data protection with cloud-based, machine-learning algorithms. EMC s data protection systems and software (e.g., EMC Data Domain, EMC Avamar, EMC NetWorker) send information about us via phone home support. We planned to create a data lake (see sidebar What Is a Data Lake ) to enable us to rapidly expand our analytics and optimize customers systems.

4 While we ve made progress especially in predicting system capacity limits and disk drive failures, each step has been much more challenging than we had expected. Getting the Data You Need We found only two real challenges in getting the data to fill our lake: getting ANY data and getting the RIGHT data. Any organization, including EMC, has silos, which make it challenging to gather data: Stephen Manley is Chief Technology Officer of EMC Core Technologies. Read more of his insights on The Protection Continuum and follow him on Product silos Each of the companies that EMC has acquired has a unique way of sending, storing, and utilizing phone-home support data. Convincing different groups to change is hard. IT vs. CTO IT wants to own the data for security, compliance, and control. CTO wants to fold, spindle, and manipulate the data in ways that we can t predict. This creates tension. After two years, we have our own copy of the Avamar and Data Domain phone-home support data. We only made it this far by making personal connections to overcome organizational boundaries. Next, we discovered that the phone-home support data does not have everything we need. Despite publishing vast swaths of telemetry, the product developers did not put the information we need into the support logs (how could they? We just figured out that we needed it!) Since waiting for product changes and customer upgrades takes too long, we are building tools that, with our customers permission, allow us to dynamically gather the rest of the data we need. A Trusted Data Lake Is a Protected Data Lake Trust means many things to many people when we talk about information infrastructure. For some, it is security, protection, and availability. For others, it s about evolving quickly to meet user needs and adopting technology ranging from columnar databases to Hadoop to Spark. For others, it s about ensuring that they will not encounter development roadblocks. This has led to our own struggle with who owns our data lake: CTO We have complete control and can do what we want. But, as our work becomes business critical, we burn precious resources doing shadow IT. And, we do it poorly. IT The right way to get product-level support protection, availability, security, and 24x7 support. But the reduced agility and control make us wary. External cloud provider Currently, we believe that the data is too large and too sensitive to place in any external cloud. Engineering operations Our middle ground. More responsive than IT, but less mature in its service offerings. Currently, we prototype in CTO, hand off run books to Engineering Operations, and continue to negotiate with IT. There are no right answers just compromises that we can live with. Driving Investment Translating the data lake excitement into investment has proven most challenging. Other EMC executives love to talk about our Greenplum database, Hadoop cluster, advanced analytics, and machine learning algorithms. But when it comes to putting skin in the game, the room gets very quiet and empty. After all, they ve got core products to evolve and they can t sacrifice people and budget. Meanwhile, like other organizations leveraging Big Data (e.g., marketing), we struggle to directly tie our efforts to revenue. For example, our analytics can predict that a Data Domain system will run out of capacity within 90 days. Then, a sales representative works with the customer to move data, change retention periods, or purchase new equipment. While our work improves the customer s experience, the

5 sales representative will point out that they d have sold the new Data Domain capacity without us. How do we drive investment? The same way we always have. Convince people that we solve a key business problem, track our metrics for success, and never stop selling the project. Net-Net Data lakes are incredibly useful tools, but there is no magic to be found in them. You ll face challenges consolidating the data ponds into a lake. You ll have to make tradeoffs between agility and stability. You ll struggle to get the business to back up its excitement with actual investment. Like us, you need to identify a business challenge, sell the idea constantly, cherish every small victory, make bridges across the organization, and, most importantly, think differently about how you do data protection. For more information about Big Data and other protection topics, be sure to visit emc.com/protectionleader and The Protection Continuum blog. CONTACT US To learn more about how EMC products, services, and solutions can help solve your business and IT challenges, contact your local representative or authorized reseller, visit or explore and compare products in the EMC Store. EMC 2, EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. VMware is a registered trademark or trademark of VMware, Inc., in the United States and other jurisdictions. Copyright 2014 EMC Corporation. All rights reserved. Published in the USA. 12/14 Handout H EMC believes the information in this document is accurate as of its publication date. The information is subject to change without notice.