Managing Big Data Exhaust Controlling Big Application Data Growth for ERP, MDM, and Other Key Enterprise Functions Solution Brief
Managing Big Data Exhaust Revision: 2 August 2012 You can find the most up- to- date technical documentation at: http://www.delphix.com/support The Delphix Web site also provides the latest product updates. If you have comments about this documentation, submit your feedback to: help@delphix.com 2012 Delphix Corp. All rights reserved. The Delphix logo and design are registered trademarks of Delphix Corp. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies. Delphix Corp. 275 Middlefield Road, Suite 50 Menlo Park, CA 94025 www.delphix.com 2
This is even more valuable than server virtualization because my number one asset is my data, not my computing infrastructure. - Tony Young, CIO, Informatica In many ways, applications are the face of business. Apps are how customers often interact with a firm, and if they work well, the business looks streamlined and intelligent. If application processes are broken or slow, the business looks equally bad. Most organizations rely on applications to operate the business and to capture new revenue, and spending on application portfolios continues to grow. Increased apps create ever more data, and while big data commonly refers to particular types of analytical operations on large data sets, enterprise applications create a large amount of data exhaust as those apps move through their lifecycle. While corporate data centers have applied virtualization technologies to reduce actual exhaust (i.e. physical server and cooling output), they have struggled to manage the data exhaust from the apps running in these data centers. Big Applications Create Big Data Exhaust For most organizations, the major applications that power business operations are not going away anytime soon. Large companies have made significant investments in ERP, CRM, and other applications and many are unable or unwilling to replace or migrate these apps. Big apps, therefore, continue to create large amounts of data exhaust, which drags along significant cost and delay to application projects. Some common areas where big data exhaust causes headaches include: Application Database Copies A typical ERP program at a large firm might include multiple instances, each with databases over 3 TB in size. Processes to deploy new modules and new functions, or to support production with bug fixes, patches, etc. all generate multiple copies. One Delphix customer has dozens of copies of its SAP landscape databases, totaling over 300 TB in size. The time required to create and update the copies, as well as to back them up and restore them, causes months of delay to overall Many environments required, many copies created Production 27 Databases 3 TB Dev/Test Copies 810 Databases 90 TB application initiatives. In certain industries, firms are managing multiple petabytes (PB) of copies and backups of application data. " Downstream Data Quality Upstream applications generate data at different rates, and new apps deployed mid- cycle cause data generation spikes as well. When the data is pulled together in downstream systems, collection timing may create inaccurate totals or missing items. Marketing programs, financial operations, and customer support are 3
key areas that are often affected by data quality issues from upstream applications. Data Integration Large data sets from applications are difficult to integrate in master data management or other business intelligence projects. Integration teams have limited access to source data, and delays in access plus orchestration issues can double project schedules. Since data warehouses are regularly extended and updated with new types of source data, the problem continues past the production cutover of the MDM system. Copy proliferation, data quality, and data integration issues are just a few of the common problems from big data exhaust. Unfortunately, the impact of these problems is significant, as big application data enables effective business operations and analysis. Example: Business Intelligence Master Data Management projects are usually focused on creating a single view of corporate customers. Due to mergers & acquisitions, decentralization, and separate product decisions, customer information often resides in multiple production systems. For example, some info is in the front office CRM app, some in the back office ERP app, still more in the Web commerce system, etc. An MDM project pulls data from these systems, cleans and maps it into a common schema, then fills a data warehouse with the integrated data. The goal is more effective cross- sell, better customer support, and higher revenues. Effectively integrating multiple large data sets from source applications is difficult for two reasons. First, the BI team doesn t own the source apps and so has limited windows for accessing the systems. One app might be undergoing an upgrade, has code changes open, and so can t allow access. Another app owner will allow access, but not until after quarter end. A third app owner will provide data, but only last month s data. Second, once the BI team has the required data, the sets may not be synchronized. Now one set of orders for a customer doesn t match another set of orders and the history isn t correct. Problems accessing and orchestrating application data sets for BI cause significant delays. In reality, many projects take twice as long as expected, due to data issues. To address schedule delays, BI teams may reduce testing in order to hit schedule requirements, resulting in poor initial data quality and perception of failure by business users. Example: Quarter-end Closing Impacts from big application data can also affect highly visible areas such as quarter- close. As data volumes from applications grows, the number of data errors tends to grow as well. Resolving data errors often requires updates to applications, re- tests, and confirmations. The time required form resolve data issues not only delays a firm s ability to close the books in a timely manner, but also increases the likelihood of restating financial results in a future quarter. 4
One large, fast growing firm reported an average of 12 significant data errors in the final month of each quarter. New apps deployed during the quarter had more data errors, and ensuring that all data had been submitted took extra effort. Resolving these took three weeks, resulting in a 21- day close term, to ensure that results reported to the SEC were accurate. (This customer later used Delphix to accelerate quarter- end testing and shortened its quarter close from 21 to 4 days.) Big application data exhaust creates data quality, data synchronization, and data distribution issues. Delphix: Turning Big App Data Exhaust into Agile Data Delphix does for enterprise databases what server virtualization products did for data centers: replace redundant, expensive hardware with flexible software that gives application teams greater control over their projects. By virtualizing the many exhaust copies of big application databases, Delphix greatly reduces the impact to shared infrastructure and shared resources that enterprise applications have historically casued. While today s enterprise application developers are accustomed to getting their own virtual machines to Delphix replaces complex hardware with intelligent software to virtualize enterprise databases Produc'on) Development) Unit)Test) QA) Cutover)Test) Repor'ng) 10:1 Consolidate 10+ DBs into the Space of 1 VDB Virtual DBs perform, function like physical On-Demand Database Provisioning, Integration, Refresh, and Rollback enable isolated testing and experimentation, they often must share physical databases and step on each other s toes, which can result in delays to project timelines and release schedules. By combining a comprehensive data management stack with data virtualization, Delphix cuts through the bloat of big application data to accelerate application initiatives. Delphix provides value to application teams in three areas: Control Data Sprawl and Data Set Proliferation Today, application teams usually make 5-10 copies of production databases, generating tens or hundreds of extra terabytes of storage, and are struggling to manage these copies. Using Delphix database virtualization, firms can instead create more copies, one for every developer, with full data and functionality, but without the time, cost and effort of physical database copy management. Delphix users essentially get the benefit of unlimited copies, without the sprawl and costs. For example, StubHub (a subsidiary of ebay) used Delphix to give its developers hundreds of full copies of production databases, without the process overhead and costs that so many actual physical copies would create. 5
Accelerate Implementations and Upgrades Upgrading an ERP system is a major task, often creating several intermediate Delphix Increases ERP Upgrade Productivity environments for testing, for Before: After: ERP application teams delivered 2 certification, etc. Refreshing and re- ERP teams delivered 11 projects in 6 months projects in 6 months testing these copies takes time, Provisioning and refresh took 80 Provisioning and refresh takes hours per occurrence 30 minutes or less delaying projects significantly. Delphix creates virtual copies of 550%$ existing databases, enabling project More Project teams to apply patches and upgrade Output scripts, quickly test, rollback and re- test, in a cost- effective and timely manner. For example, Pella Windows used Delphix to eliminate 50% of the steps required (as well as 85% of the storage costs) to create and update Oracle EBS databases during an application upgrade. Orchestrate Multiple Data Sets for Faster Integration and Better Data Quality Through its data orchestration capabilities, Delphix can not only create virtual copies of multiple production databases, but can sync all of those copies to any point in time, down to the second. This dramatically accelerates data integration work typical in master data management and data warehousing projects. For example, Informatica used Delphix to cut 50% of the schedule for an internal master data management implementation, simply by giving the MDM team virtual copies of source databases and auto- syncing all of the copies. Summary Application growth creates data exhaust, slowing down projects, increasing costs, and impacting business. Major applications such as ERP, large data warehouses, and increasingly critical web systems all require many copies of production data, for faster development, better testing, and improved analysis. Delphix can dramatically accelerate application projects, reducing not only time and cost, but also schedule risk. Delphix reduces app teams dependency on shared IT resources and impact on shared infrastructure. Delphix is accelerating application portfolios at leading firms including Macys, Comcast, Qualcomm, P&G, Facebook, Wind River, ebay, and many others. To learn how Delphix can improve your major application initiatives, please visit www.delphix.com. 2012 Delphix Corp. All rights reserved. 6