DATA INTEGRATION, MASTER DATA MANAGEMENT, AND DATA VIRTUALIZATION. Best Practices Series

Transcription

1 Attunity PAGE 14 ENABLING BIG DATA ANALYTICS BY REPLICATING ORACLE DATA TO GREENPLUM Composite Software PAGE 15 DATA VIRTUALIZATION PROVIDES AGILITY AND EFFICIENCY TO QUALCOMM Melissa Data PAGE 16 CONTACT RESOLUTION: ENHANCING THE SINGLE CUSTOMER VIEW Denodo Technologies PAGE 17 DATA VIRTUALIZATION SMART MOVE FOR HIGHER ROA! Oracle PAGE 18 BIG DATA DETOUR AHEAD: NEW APPROACHES IN DATA INTEGRATION Progress DataDirect PAGE 21 TAMING DATA CHAOS: SIMPLIFY DATA INTEGRATION SAP PAGE 22 MAKING INFORMATION GOVERNANCE A REALITY FOR YOUR ORGANIZATION Continuent PAGE 26 HIGHLY-AVAILABLE MYSQL AND REAL-TIME BIG DATA WITH CONTINUENT TUNGSTEN Delphix PAGE 27 AGILE DATA TRANSFORMS MASTER DATA MANAGEMENT HiT Software, Inc. PAGE 28 TIPS FOR DATA INTEGRATION, REPLICATION, AND SYNCHRONIZATION BENEFITS OF CHANGE DATA CAPTURE DATA INTEGRATION, MASTER DATA MANAGEMENT, AND DATA VIRTUALIZATION Best Practices Series

2 12 SEPTEMBER 2012 DBTA Insight at Last: Master Data Management Emerges to Tackle Big Data For a long time, data integration has been the holy grail for data organizations, promising a single, accurate picture of relevant data from across the enterprise, regardless of original source or format. Now, with the rise of new approaches including master data management (MDM) and data virtualization, there is hope that this goal is within reach. But the challenges keep on coming, and lately, there has been a surge in unstructured data that may fall outside MDM realms. Just when architects thought they had integrated all their enterprise applications, their businesses started using SaaS apps, Zeb Mahmood, principal of product and strategy at SnapLogic, tells DBTA. And more recently, the challenge has not been around where the data resides on-premise or in the cloud but with the nature of the new data in the enterprise: the unstructured, high velocity, high volume big data. Data integration is never complete. The problem is that large portions of this new class of information are not visible to decision makers. Most is passing through unnoticed, David Flammia, product owner of LP Insights at LivePerson, Inc., tells DBTA. The primary reason behind that is the complexity of the raw data formats, and the general inability to reasonably convert the data into a meaningful, business intelligence-oriented type of format. And when data analysis does occur within a business, it mostly happens disparately; that is, data source analysis happens independently amongst all channels, so the insights found are very narrow in scope, and limited in value. Thus, observers concur that there is still plenty of work to be done before organizations truly bring together the disparate data pulsing through their systems and networks. The reality is that today's organizations have a ways to go with unstructured data management, integration, and protection, David Gibson, vice president of strategy for Varonis, tells DBTA. People aren t MDM offers great promise as a way to address thorny data integration problems. adequately managing unstructured data because it s a hard data set to get a handle on. In a recent survey we did, we found that 67% of respondents said that senior management in their organizations either don t know where all company data resides or is not sure. It s really hard to manage something when you don t know where it is, who is using it, who has access to it, and what it contains. It s time to expand data integration efforts to address the new realities of big and unstructured data, Scott Gidley, senior director of R&D and data management for SAS, tells DBTA. The most common approach is ETL [extract-transform-load]. And while this is a solid, viable option, there are numerous data integration options available today, including ELT [extractload-transform] to consider. It s important to weigh all options. Enter MDM, emerging as a strategy that may help deliver unity in enterprise data. The uptake of MDM has been strong and the market continues to mature, Rick Clements, program director for MDM and big data strategy at IBM, tells DBTA. MDM has gone past the early adopter stage and is now in the early majority with more organizations across all industries and sizes realizing value and returns on investment. Where is this return on investment coming from? Analytics and business intelligence are the low-hanging fruit for organizations attempting data integration efforts. MDM can play a key role here as well. An MDM roadmap is needed, Umesh Karpe, vice president of the data warehousing and business intelligence practice for igate, tells DBTA. Such a roadmap needs to include multiple entities for the MDM repository, covering subjects such as customers, suppliers, products, sites, and in some cases even finance, Karpe says. The MDM repository becomes the golden copy and clean certified data is used across the organization for operational and analytical applications. The challenge of data integration and the promise of MDM doesn t stop at the organization s walls, either. Information is fundamentally federated and that information is most valuable when it is presented as part of the active

3 DBTA SEPTEMBER transaction and represents the most recent picture possible, Steve Jones, global head for master data management at Capgemini, tells DBTA. This includes external information such as social media and supplier catalogs, as well as internal systems. The new thinking is about providing a consistent federated view rather than fighting against how companies work with technology. This shift in thinking on information has highlighted the need to identify the unique customers, suppliers, products, and other key elements across systems, rather than simply try and consolidate into single unachievable repositories. Ultimately, MDM offers great promise as the best way to address thorny data integration problems, for both structured and unstructured data stores. MDM considerably reduces the spaghetti of interfaces across business applications and hence optimizes business processes and outcomes, KR Sanjiv, senior vice president and global head of analytics and information management for Wipro Technologies, tells DBTA. MDM can eliminate a significant number of integration points. While MDM offerings currently on the market manage unstructured data in a comprehensive way, it's not a showstopper either, says SnapLogic s Mahmood. It s not a big limitation, he explains. The unstructured data of interest for MDM is not large unstructured text or machine-generated big data, but mostly images and PDFs, which can be managed by integration with content management systems. Wipro s Sanjiv agrees, pointing out that MDM tools lack in sophistication when it comes to managing content and unstructured data, and only provide rudimentary storage and retrieval capabilities. However, MDM is essential for unstructured data, IBM s Clements points out. We see information that is mined from unstructured social media data being integrated and mastered as part of customer-centric initiatives, he adds. Still, MDM must be well-thought out and planned to adequately address the wide range of organizational needs. And more often than not, support needs to come from the top, says Mahmood. MDM requires buy-in from across different business units, and it s not easy to get agreement across the board unless the initiative is sponsored by the VP or CIO, he says. It s hard to quantify benefits of MDM, and there are always questions about which business unit should pay for it. Often as well, MDM proponents fail to articulate a compelling business case for MDM, Sanjiy states. Successful MDM implementations are the ones that have been able to establish a clear line of sight of the business benefits, combined with business ownership. Data virtualization also holds promise as a way to finally achieve data integration. Plus, there s often a lack of common understanding of the information organizations need to integrate, says IBM s Clements. For example, we ve seen companies that have dozens of different definitions of the same terms, including terms that aren t part of the domains they ve targeted for MDM. One company had 37 different definitions for Employee ID across multiple divisions. This leads to big data quality issues, as the wrong information shows up in applications and reports and leads to poor decisions downstream. Often as well, MDM efforts tend to remain confined within technical realms, without being communicated to or appreciated by business users. MDM adoption is focused too much around technical implementation and too little around the business operational challenges, says Capgemini s Jones. When companies focus on the technical side they tend to find that the projects spiral with the master data repository becoming more of a data dumping ground. You simply end up with a big database with some expensive data quality. Data quality, in fact, needs to be part of MDM from the start, says SAS Gidley. One thing is certain if a company s data quality isn t in top shape with solid data governance in place, then MDM will not succeed. This problem is only exacerbated with the rise of big data initiatives. In some ways, organizations are taking steps backward in their overall data management practices as they struggle just to deal with increasing volumes. Data virtualization in which information is abstracted away from underlying systems and databases also holds great promise as a way to finally achieve data integration. Data virtualization can help spot trends and help validate initial data findings early on in a data integration process, says Capgemini s Jones. However, Jones and other industry experts caution against rushing into data virtualization too quickly as it enables early-view reporting of critical data points that may have not been properly vetted for quality or relevance. SAS Gidley notes that data virtualization has been a concept on the industry s radar for some time now, but has not yet proven to be more costeffective than traditional data integration techniques. However, new trends may drive more organizations to consider data virtualization, he adds. Data privacy and security issues may prevent companies from physically or logically moving data, so virtualization allows them to do integration in a less invasive way. In addition, data virtualization helps companies that want to understand massive amounts of data without going through the processing efforts typically required in traditional data integration efforts. Then, there s the capability data virtualization provides to enable a semantic view of data from various systems. This is appealing as organizations look to more quickly make data available to business analysts and decision makers.

4 14 SEPTEMBER 2012 DBTA Sponsored Content Enabling Big Data Analytics By Replicating Oracle Data to Greenplum BARRIERS TO BIG DATA ANALYTICS To enhance your organizational competitiveness and profitability, business users must derive new insights from their data. However, Big Data also means major technical obstacles: Large volumes of heterogeneous information create technical limitations. Traditional approaches to data warehousing are faltering due to Big Data deluge. As IT professionals try to deliver information to business users in a timely way, they report challenges related to database and system performance, the lack of data standards, data mapping, managing and delivering unstructured data, and more. Traditional approaches to data integration don t work well. When data integration requires only simple transformations, ETL tools can be overly complicated as well as ineffective at delivering data updates as rapidly as business users require. Alternatively, if data transformations are needed, it is often more efficient to perform that work in powerful databases like EMC Greenplum, rather than in an ETL tool. A NEW APPROACH FOR MAKING BIG DATA ACCESSIBLE EMC Greenplum and Attunity Replicate for Greenplum offer a powerful operational data warehousing solution for organizations faced with Big Data challenges. As the industry-leading platform for in-database analytics, EMC Greenplum enables enterprises to manage, store, and analyze terabytes to petabytes of data. But storage and management are only one part of the Big Data picture. IT teams also need a fast and easy way to load diverse data sources into EMC Greenplum. Attunity Replicate for Greenplum s change data capture (CDC) technology streams changes from source databases like Oracle to EMC Greenplum with very low impact on the source system. Users gain the following benefits: High-performance loads and continuous data capture. The intuitive Click-2-Replicate graphical user interface for designing and monitoring replication tasks. Support for a wide range of data sources, including mainstream relational databases, like Oracle and SQL Server, and legacy systems. Automatic schema generation and implementation of metadata changes on the target. Transparent data type transformations between source and target databases. Attunity Replicate leverages Greenplum s Scatter/Gather Streaming TM technology to remove data loading bottlenecks related to large-scale analytics and data warehousing. Attunity Replicate for Greenplum provides full automation for four tasks: (1) schema generation and data type mapping, (2) full load of source tables from databases like Oracle using high-speed data extraction and loading, (3) incremental load of changes made to source tables, and (4) application of schema changes (DDL) made to the source tables. SOLVE BIG DATA CHALLENGES WITH ATTUNITY REPLICATE FOR GREENPLUM Attunity Replicate for Greenplum solves Big Data Challenges for Oracle users: Efficient data integration for Big Data analytics. Through change data Through change data capture (CDC), Attunity Replicate for Greenplum changes the processing paradigm and only works with information that has changed in the database. This improves data integration efficiency and increases the power of Big Data analytics. Would you like to learn more about getting the most value from your Big Data? Download Attunity Replicate for EMC Greenplum: Enabling Big Data Analytics with High Performance Data Replication, a technical white paper. Download today at Low latency data offloading. Data warehouses built for highperformance analytics can quickly become overwhelmed with Big Data. Attunity Replicate for Greenplum can be used to offload older data from Oracle to EMC Greenplum operational data stores.with CDC, changes are replicated with low latency. Minimal impact on source performance. Attunity s zero footprint technology doesn t require any software installation on the source. As a result, the performance impact on sources like Oracle and others is minimal. No specialized skills or knowledge necessary. Attunity s user-friendly Click-2-Replicate user interface doesn t require coding, specialized Oracle DBA skills, or in-depth knowledge of the Oracle database structure. ATTUNITY Learn more by downloading the new white paper at attunity.com/oracle2greenplum or visit

5 Sponsored Content DBTA SEPTEMBER Data Virtualization Provides Agility and Efficiency to Qualcomm Qualcomm Incorporated is a global leader in next-generation mobile technologies. The company manufactures chipsets, licenses technology and provides communications services worldwide, primarily to the telecommunications industry. More than 180 telecommunications equipment manufacturers license Qualcomm inventions worldwide. In 1989, Qualcomm introduced CDMA (Code Division Multiple Access), a technology for wireless and data products that provided an important foundation for the evolution of today s 3G (third generation) wireless technologies. BUSINESS NEEDS DRIVE TECHNOLOGY DECISIONS In 2009, Enterprise Architecture (EA), a group within Qualcomm s IT organization, identified data virtualization as a compelling concept that could address some major business challenges facing Qualcomm. Among the challenges is the rapidly changing business environment in which Qualcomm operates. We are constantly challenged to get things done faster in order to maintain our leadership position in the mobile technology market. This is a tremendous struggle for any organization, but it hits especially hard when the industry Figure 1. Qualcomm Data Virtualization Architecture changes as fast as ours does, says Mark Morgan, IT Manager in the EA group. The second challenge is effectively managing multiple terabytes of data and ensuring the integrity of the data. Qualcomm was moving significant amounts of data among multiple systems, including data warehouses and data marts. In some cases, the same data might be stored in 10 different systems. We were just shuffling the data around and it wasn t giving us any benefit. But we had no other option at the time, explained Morgan. Therefore, the potential for increased agility and speed of execution was a key component and benefit of a data virtualization approach and a critical success factor for Qualcomm. COMPOSITE SOFTWARE DATA VIRTUALIZATION PLATFORM SELECTED Qualcomm s solution is an enterprisewide data virtualization layer built with Composite s Data Virtualization Platform. The primary use for data virtualization is the ability to make data available to multiple applications without having to copy and move the data. The Qualcomm Data Virtualization Solution Architecture is shown in Figure 1. Instead of spending six months on an integration project to bring the data physically together in one place to make the application functional, Morgan s group can build on the existing repository and virtual views within Composite and simply plug the application into the data virtualization layer. If any required data is missing, we can get access to that within a few days, says Morgan. SIGNIFICANT BENEFITS WITH RAPID PAYBACK Having this level of independence between applications and data sources dramatically reduces the maintenance effort required to keep up with changing business requirements. A key benefit is the ability to get feedback to customers faster. Before Composite, significant time was spent on identifying the requirements and then developing the application and the data integration piece. It could take three months for the customer to see anything and only then would we find out it wasn t what they needed. With Composite, Qualcomm can model the application in a few days and show it to the customer. Further, using data virtualization has lowered Qualcomm s IT support costs by replacing the need to rely solely on traditional integration technologies. Data virtualization has improved the efficiency of Qualcomm s data management in two ways. One is the fact that the company no longer has to store, manage and synchronize the same data in multiple locations. This has also enabled Qualcomm to establish stronger ownership of data as part of its governance effort. In the past, ownership of data was a big challenge. Once data is copied, who owns the data gets fuzzy. In a data virtualization environment, ownership of the data is well-defined because it stays with the data. Data virtualization has definitely been worth the effort and investment. Not only has it saved us money, but it has accelerated our ability to meet business demands, says Morgan. COMPOSITE SOFTWARE, INC. is the data virtualization gold standard.

6 16 SEPTEMBER 2012 DBTA Sponsored Content Contact Resolution: Enhancing the Single Customer View Know your customer is the mantra of companies everywhere. Achieving that single view of the customer is an ultimate goal, as it will help improve customer service and marketing, enhance business intelligence, and meet regulatory compliance. The ability to answer questions about your customers who they are, which products they own, what their history is, and how profitable they are provides a true competitive edge. Most companies already possess the data that would enable them to gain these insights. But it s often fragmented and scattered, held in incompatible formats, in legacy systems, or not well integrated, and frequently changing making data hygiene a challenging, yet vital component for the accuracy of a single view. Data quality tools and techniques are imperative to the process, and today these technologies are fueling the maturity of an organization s data quality initiatives. CONTACT RESOLUTION, EXPLAINED A single view can only be attained with consistent, current, and accurate data, however, it can be difficult to determine whether two or more records refer to the same customer. For instance, a customer s name in one database might be represented by a nickname in another. She might have a valid address in one data set, but a different address in another. The various customer information fields for her might be missing or inconsistently filled out. Without a way to determine the data interconnectivity between similar records, it is difficult to consolidate duplicate records and match accurate data to the actual customer. Fortunately, a new process called contact resolution can help companies make clear connections within and across data sets. The contact resolution process works by linking historical snapshots of each component of contact data and identifying the most updated records (i.e. the most current street address on file). Basically, the process brings forward the entire customer record to its most current form, enhancing the single view of the customer. CROSSING THE Ts AND DOTTING THE Is Often, companies rely on the mere validation of their contact data. Validation is a rules-based approach used to determine whether data fits the required format. It will determine if the data is reasonable and possible. For example, validating a phone number would require that there are 10 digits and no letters in the string. However, this traditional process does not necessarily mean that the data is correct, which falls short of true data quality. Another approach involves verifying data by comparing it discretely against reference data to determine its accuracy. Using this process, an organization can match its data against USPS data to identify which customer addresses are deliverable, which can be corrected, and which are undeliverable. The same process of matching customer data to reference datasets can be used to verify and correct addresses, telephone numbers, full names, and other contact data. The benefits of verified contact data include reduced return mail, postage savings, improved response and conversions for marketing campaigns, and an increase in customer satisfaction. CONTACT RESOLUTION TAKES DATA TO A WHOLE NEW LEVEL But having verified contact data still leaves questions to be answered. For example, in our database, a customer, Jane Doe, is associated with this address: 100 Main Street, Anytown, California We can verify that address exists and is deliverable, but how do we know Ms. Doe still lives there? Or that she ever lived there? The same goes for the telephone numbers in her record, and her address. The beauty of contact resolution is knowing that your contacts are reachable. The contact resolution process compares data from multiple sources, including USPS records, telco data, title information, and other public and proprietary information both current and historical to first associate and connect verified contact data to a specific customer. Now an organization has the confidence that a direct marketing or telemarketing campaign will reach the intended customers and prospects because they are reachable. QUALITY DATA IS COMPLETE DATA Almost every CRM system is missing one or more pieces of data about a given customer. A data quality vendor can fill these gaps with verifiable street addresses, addresses, phone numbers, and associated demographics. With an arsenal of accurate, reachable, and complete contact data, organizations can now perform more holistic, meaningful analysis of their customers to create additional opportunities for growth. Over the past two decades, a changing attitude has gradually altered our perception of what is meant by quality information. Today, the concept goes beyond data cleansing, standardization and enhancement, to include contact resolution. With the ability to help resolve different representations of a record and link all touch points of customer data together contact resolution provides the means to achieve and enrich a true single view of the customer. MELISSA DATA CORP.

7 Sponsored Content DBTA SEPTEMBER Data Integration is not new... in fact, it s been an endless money pit! Even after spending millions, companies face data challenges that are only getting bigger. What s missing and why? And, how do you measure the value of a new integration technology that promises the elusive right information to the right user at the right time? Data Virtualization Smart Move for Higher ROA! SILOED DATA INTEGRATION In the past, each data solution focused on one problem or data type ETL /data warehousing for BI and analytic needs, ESB for transactional data movement between applications, and search/ knowledge management for unstructured information. Good idea then. But when data volumes and complexity explode and business processes are much more dynamic and interactive, just stretching the old solutions beyond their focus doesn t work. Siloed heavy-weight tools don t make unified data access possible, but instead proliferate data, complicate governance, and kill business agility. A different approach is needed. THE NEED FOR DATA VIRTUALIZATION A new data strategy to build a Unified Virtual Data Layer for agile data provisioning across all data types and consumers is gaining momentum. Data Virtualization (DV) works with existing infrastructure to provide a strategic solution for a virtualized, abstracted data layer across all data assets. And on its own, DV adds functionality to your toolkit e.g., real-time data, access to complex data types (structured/ unstructured), integrating Cloud/Web data, Self-service, linked data, etc. that is useful to meet tactical projects needing agile, real-time integration. A GREAT TEAM PLAYER DV enables the unified virtual data layer which allows reuse of views across analytical and operational projects where previously, separate workflows existed. But this does not imply a rip-and-replace of existing systems but rather that DV connects and complements existing tools to build an overall agile data infrastructure. It extends the data warehouse with realtime and unstructured data, creates a semantic view to share across multiple BI tools or further ETL processes, and provisions data services to self-service BI users.dv is also used to combine master data in MDM systems with dynamic operational data to enable single view applications. For operational uses, ESBs can now focus on process orchestration and leverage integrated data services that DV provides which speeds time-to-solution and flexibility. DV can handle mixed workloads with high performance by combining realtime query optimization with caching and scheduled/etl pre-fetch. The DV layer provides unified data governance to track data lineage, security and monitoring. A VALUABLE INDIVIDUAL CONTRIBUTOR In addition to virtualization/ abstraction across the data landscape, DV platforms add utility to your integration toolkit for tactical projects, connecting disparate data sources, transform and publish on-demand integrated results with much less effort and time required. This meets business real-time data needs and increases agility because virtual views are changed very quickly, and it costs less since data replication is minimized. MEASURE THE ROA, NOT ROI But how do you justify this investment under a difficult economic climate? The answer is in the Return on (Data) Assets (ROA)! Many Denodo customers have already invested millions in databases, warehouses, data feeds, storage and integration technologies, often without realizing the promised benefits. So, while adopting DV, they focused on ROA instead of ROI and proved that a 5% increase in data usage would return 20 40x their investment in DV because it unlocks existing data assets, combines disparate data, delivers it with speed to many more users from CEO to rank-andfile workers, from internal applications to partners, all while minimizing data replication and management costs. DATA VIRTUALIZATION IS VALIDATED DV is used by top companies including Cisco, Biogen Idec, Oppenheimer Funds, Eli Lilly, Vodafone, Telefonica, BBVA, Nippon Steel Solutions and others providing data agility directly in support of big data analytics, cloud integration, single view applications, and much more. To learn more about these cases, visit DENODO TECHNOLOGIES is the leader in Data Virtualization. Please contact info@denodo.com or call to discuss your next project.

8 18 SEPTEMBER 2012 DBTA Sponsored Content Big Data Detour Ahead: New Approaches in Data Integration Dain Hansen, Director of Product Marketing for Oracle Fusion Middleware You don t need to be a data scientist to know that big data is all the vogue in today s IT landscape. Big data has created a happy detour one which has forced us to navigate differently the ways in which we integrate and manage our data. One of the reasons that big data is so compelling is that it addresses a universal challenge that impacts every one of us. Whether it is healthcare, financial, manufacturing, government, or retail big data presents a pressing problem for many industries: how can so much information be processed so quickly to deliver the complete picture? With big data we re tapping into newer information that didn t exist before: social data, weblogs, sensor data, complex content, and more. What also makes big data revolutionary is that it turns traditional information architecture on its head, putting into question commonly accepted notions of where and how data should be aggregated processed, analyzed, and stored. This is where Hadoop and NoSQL come in new technologies which solve new problems for managing unstructured data. But before we discard everything that we know and get burdened in big data buzzwords, we shouldn t forget what s already running in today s IT environment. Today s business analytics, data warehouses, enterprise applications, and even many social, mobile, cloud applications still rely almost exclusively on structured data or what we d like to call enterprise data. This dilemma is what today s IT leaders are up against: what are the best ways to bridge enterprise data with big data? And what are the best strategies for dealing with the complexities of these two unique worlds? This is what data integration is all about. And, in this article we ll outline four important new approaches for using data integration techniques to reap the benefits of big data technologies. 1) Leverage existing tools and skill-sets 2) Quality first 3) Real-time analytics 4) Integrate the platform LEVERAGE EXISTING TOOLS AND SKILL-SETS FOR AN INTEGRATED SOLUTION While Hadoop technologies are compelling in their own right, a word of caution that not every big data technology may actually be the right tool for every data processing job. When investigating these emerging technologies, why not leverage the existing resources in information management to meet some of the end-goals? More advanced data integration tools are becoming integrated in such a way that designing ETL and developing such big data transformations (in MapReduce or Hive) can be implemented in a single design environment. Data Integration tools are evolving to support new forms of connectivity to source in NoSQL, HDFS. This is as opposed to keeping these two worlds separate and apart from one another. The advantages of a single solution allow you to address not only the complexities of mapping, accessing, and loading big data but also correlating your enterprise data and this correlation may require integrating across mixed application environments. The correlation is key to taking full advantage of big data and requires a single unified tool that can straddle both environments. Oracle provides such a unified tool strategy with its single design time environment in Oracle Data Integrator for loading, transforming both enterprise data and big data. As a component of Oracle Big Data Connectors, Oracle Data Integrator Application Adapter for Hadoop provides native Hadoop integration together with Oracle Data Integrator. The knowledge modules can be used to build Hadoop metadata within Oracle Data Integrator, load data into Hadoop, transform data within Hadoop, and load data easily and directly into Oracle Database utilizing Oracle Loader for Hadoop. Once the data is processed and organized on the Hadoop cluster, Oracle Data Integrator loads the data directly into Oracle Database utilizing the Oracle Loader for Hadoop. QUALITY FIRST Big data sources consist of many different types and in many different forms. How can anyone be sure of the quality of that data? In the big data scenario, data quality is important because of the multitude of data sources. Multiple data sources make it difficult to trust the underlying data. Being able to quickly and easily identify and resolve any data discrepancies or missing values in an automated fashion is beneficial to the applications and systems that use this information.

9 Sponsored Content DBTA SEPTEMBER One scenario that comes up frequently is a business intelligence or data warehousing application, where the data being extracted from the operational systems is after some basic analysis too dirty to bring into the warehouse in a useful way. We ve all seen those types of data horror stories where lax application data entry standards have allowed less than ideal data into the transactional application. One of the most basic rules of data management is that data is usually fine in the first system or context where you enter it or use it. But when you try to extract it from that system and use it more broadly, in this case an ERP system, all the warts are revealed. Oracle s strategy for data quality is Oracle Enterprise Data Quality. Today this product integrates with Oracle Data Integrator products to enable data analysts to investigate, identify and resolve data quality issues by discovering and analyzing anomalies. Mastering and stewardship are also part of Oracle s Enterprise Data Quality product strategy since Oracle Master Data Management products are directly integrated with Oracle Enterprise Data Quality for both customer and product data. REAL-TIME ANALYTICS One of the key expectations we have for big data and our information architecture is to yield faster, better and more real-time analytics. That appeal of processing so much information quickly is why the Hadoop technologies may have originally been invented. But is big data nothing more than glorified batch? Can it meet the same velocity mandates as real-time replication for structured data? At the heart of big data precepts are good old fashion batch processing underpinnings in extraction, transformation, loading. We just call it different with terms like reduction or federation. But what s differently is that big data batch is done much faster and with different styles of data, unstructured, semi-structured data, and also it s applied on different non-relational systems. All of these elements contribute to velocity, volume and variety. So yes, some of the big data processing is implicitly fast. Let s turn to the structured world of enterprise data for a moment. Real-time replication is one of the most important and next-generation approaches to the business analytics and data warehousing industry. It s one of the most efficient ways to get information over to our analytics tools from the multiple sources and systems where data lives, and of course, get it there instantly without impact on the original source systems. What a business user may see on a business analytics dashboard is dependent on how the data is loaded, transformed, cleansed, and ultimately mastered into many different applications. But how timely the data is is a question that will still need to be asked whether it is big data or traditional enterprise data. And that gets to the heart of it. We need both. These technologies need to work together. Real-time and big data are very much sisters, brothers to developing next-generation information architectures that can help businesses overcome the challenges of managing a data explosion. Otherwise the speed advantage to indexing realms of big data will be undone by sluggish ETL processing that it s dependent on. Big data can be processed at high volume with high velocity. Combine this power with real-time solutions in replication, change data capture, synchronization, and the integration to in-memory business analytics tooling, and you have what amounts to the compelling advantages of real-time business analytics. Oracle s solution for Real-time Business Analytics [for structured data] is achieved by using Oracle GoldenGate, Oracle Data Integrator and Oracle Business Analytics together. Within Oracle Business Analytics is Oracle Exalytics, the industry s first in-memory BI machine designed to achieve explosive performance of BI applications. Oracle GoldenGate is an integral part of the real-time business analytics use case in that it accomplishes real-time data replication and capture, hence ensuring

10 20 SEPTEMBER 2012 DBTA Sponsored Content that applications have the data they need immediately. INTEGRATED PLATFORMS Taking all the miscellaneous technologies around big data which are new to many organizations and making them each work with one another is challenging. Making them work together in a production-grade environment is even more daunting. Integrated systems can help an organization radically simplify their big data architectures by integrating the necessary hardware and software components to provide fast and cost-efficient access, and mapping, to NoSQL and HDFS. Combined hardware and software systems can be optimized for redundancy with mirrored disks, optimized for high availability with hot-swappable power, and optimized for scale by adding new racks with more memory and processing power. Take it one step further and you can use these same systems to build out more elastic capacity to meet the flexibility requirements big data demands. Oracle is uniquely qualified to combine everything needed to meet the big data challenge including software and hardware into one engineered system. The Oracle Big Data Appliance is an engineered system that combines optimized hardware with the most comprehensive software stack featuring both the Cloudera Distribution including Apache Hadoop and specialized solutions developed by Oracle to deliver a complete, easy-to-deploy solution for acquiring, organizing and loading big data into Oracle Database. It is designed to deliver extreme analytics on all data types, with enterprise-class performance, availability, supportability and security. With Oracle Big Data Connectors and Oracle Data Integrator, the solution is tightly integrated with Oracle Exadata and Oracle Database, so that data can be analyzed data with extreme performance. CONCLUSION Big data continues to be the center of attention take away the hype and there s a key takeaway. For years, companies have been running their critical business infrastructure and building business insights based on transactional data stored in relational databases. Beyond that critical data, however, is a potential treasure trove of less structured data that can be mined for useful information. Companies that are seeking ways to capitalize on the hidden potential of big data need to consider data integration technologies to help bridge the gap and correlate that data across the enterprise. Bridging the two worlds of big data and enterprise data means considering solutions that are complete, based on emerging Hadoop technologies (as well as traditional), and are poised for success through integrated design tools, improved data quality, real-time analytics, and integrated platforms. Leveraging these types of best practices translates to improved productivity, lowered TCO, IT optimization, and better business insights. Only Oracle provides the most complete, integrated, and real-time data integration solution that can help bridge the two worlds of big data and enterprise data to accelerate adoption and yield greater returns for these emerging technologies. ORACLE For more information on Oracle Data Integration see our website:

11 Sponsored Content DBTA SEPTEMBER Taming Data Chaos: Simplify Data Integration Each year, the role of Information Technology becomes more complex. Virtually every IT organization of any size follows the latest trends, introducing new platforms, applications and infrastructure on an ongoing basis. At the same time, most continue to support applications and platforms that may be decades old. Technologies across this wide spectrum from Rich Internet Applications (RIAs) to mainframes, from.net to COBOL, and from Web Services deployments to legacy Electronic Data Interchange (EDI) coexist within today s data centers, each one adding unique challenges to the overall task of IT support. Valuable business information is locked within these diverse systems, and IT organizations are confronting the twin problems of accessing and integrating data. At the same time, the vast majority of IT organizations are struggling with funding and facing the need to balance dwindling budgets against rising costs. Data transformation and integration are key areas where specialized products can yield impressive time and cost efficiencies. BUSINESS APPLICATIONS AND THE CHALLENGE OF HETEROGENEOUS DATA Under the covers, virtually every integration project is really about data sending, receiving, and transforming data across applications, companies, and customers. Supply chain applications exchange order information with suppliers and billing information with customers. The complexity surrounding such seemingly simple activities can be overwhelming, and a typical EDI integration, for example, takes months to complete. Every company s data is different, and, because of ongoing mergers and acquisitions, many companies are actually managing and integrating multiple internal data sets. In today s companies, data can exist on disparate platforms (UNIX, Windows, mainframe), in dissimilar Relational Databases (SQL Server, Oracle, MYSQL), and in multiple formats (Microsoft Excel spreadsheets/office documents, proprietary vendor files). With little or no consistency between businesses, or even between departmental data within a single company, virtually every integration is unique and requires custom programming. Integrating unlike platforms, data stores, and data formats is a complex business, and this complexity is one factor driving today s high development and support costs. For many medium-sized to enterprise-sized businesses, application heterogeneity is also a factor contributing to data chaos. Diverse data, stored in multiple locations and on multiple platforms, can be accessed and consumed by Java and.net applications, SOA services, REST and Scripting applications, and legacy software. This diversity takes its toll. For developers and architects, it requires ongoing training as well as substantial manual coding. It also has performance implications, since it is impossible to predict which applications are requesting the same data concurrently. Finally, it drives up development costs. Data schemas constantly change to meet the changing needs of the business, and multiple programs must be modified with each change to a data source. In short, while applications built on these technologies promise high value and competitive differentiation, they do not come cheaply. Transforming data into business-focused information massaging it into a form that is useful to the business is an expensive proposition. A NEW ERA OF FLEXIBILITY AND TRANSFORMATION Progress DataDirect is one of the leaders in this space. By easing the pain of integrating diverse applications yet delivering time, cost, and performance efficiencies by providing easy access to data stored in virtually any format including Relational Database (RDBMS) files, XML, Web Services, Microsoft Excel/Office, flat files, and EDI documents. Progress DataDirect solutions: Reduce the time required to develop applications that need to consume data from various data sources like EDI, relational, non-relational, and legacy systems. Eliminate the need for custom coding, which is expensive to develop, difficult to maintain, and is difficult to reuse. Ensure lower total cost of ownership by providing lightweight data integration components that integrate seamlessly with existing middleware stack. Allow companies to focus on revenue generating core competencies, and to focus costly developer resources on solving business problems and not implementation details. Improve operational efficiencies by eliminating costly manual or paperdriven processes and by providing a solution that can interoperate with legacy and modern IT infrastructures. Allow ISVs to extend their solution and overall customer footprint by enabling support for their customers data formats and data sources. Multiple roles within a company will find these solutions useful. Development organizations are the obvious choice, as many are finding that heterogeneous technology is impacting productivity. Architects find them equally valuable, as it gives them leeway to effectively utilize data from new sources. Using the capabilities, businesses can embark on a new era of flexibility fueled by faster development turnaround times and new versatility in terms of data utility. Progress DataDirect solutions capitalize on years of experience developing solutions for RDBMS access, and build on industry standards. Optimally positioned for both integration projects and coding projects that rely on data from multiple sources, these solutions are designed for simple deployment and ease of use. PROGRESS DATADIRECT For more information, visit

12 22 SEPTEMBER 2012 DBTA Sponsored Content Making Information Governance a Reality for Your Organization MAXIMIZE THE VALUE OF ENTERPRISE INFORMATION Enterprise information underpins all business operations and decisions, and without good information, opportunities to seize competitive advantage are missed. But managing information is no simple task, especially as it explodes in volume and variety. Best-run organizations view information as a strategic asset and use information governance initiatives to deliver consistent, accurate, information across the enterprise. Technology and business process advances in recent years have led to an information explosion. This information comes from internal sources such as business process applications, productivity suites mainly containing unstructured content such as s or documents, and external sources including social media. Enterprise information is growing at an unprecedented rate as are the prospects to exploit it for competitive advantage. Best-run companies know that those who can seize the potential of enterprise information will have the edge in effectiveness, innovation, and profitability. It is certainly no secret that managing information well is critical for business success. According to a Forbes Insight survey of over 200 business and IT leaders, 95% of organizations agree that strong information management is critically important. The survey also found that fragmented data ownership is a common and significant roadblock to enterprise information management programs. 1 Reliable, accurate information forms the foundation for making sound decisions quickly and taking advantage of opportunities. Without it, businesses may move too slowly to enter promising markets, lose sales due to dissatisfied customers, or run afoul of regulations. To avoid these pitfalls, companies need a better way of managing information to improve business effectiveness and reduce risk. Information governance initiatives can provide trusted, secure, highquality information to support business operations, growth, and innovation by addressing data ownership fragmentation. Information governance initiatives provide a disciplined framework for organizations to establish the right people, processes, policies, and metrics to oversee enterprise information and add value to the business. Many organizations have taken an ad hoc approach to information governance, with poor documentation of policies and rules, lack of consistent processes across the organization, and an inability to monitor progress. In the past, technological solutions for managing information often failed to give lineof-business owners self-service access to their own information to analyze and improve it. Recent innovations, however, provide a better way for businesses to govern information. User-friendly interfaces automate the ability to measure, monitor, and enforce information policies and standards across heterogeneous systems enabling lines of business to take control of and responsibility for their own information. THE IMPORTANCE OF INFORMATION GOVERNANCE DATA QUALITY EMPOWERS THE BUSINESS Information-related problems can cost companies like yours millions of dollars most companies report losing more than $5 million annually, and one-fifth of companies estimate losses in excess of $20 million per year, according to the Forbes Insight survey mentioned earlier. Information issues often are caused by insufficient ownership of data by lines of business, inadequate governance practices, lack of technological support, insufficient executive involvement, and lack of collaboration between the business and IT. Leading organizations have dramatically increased their focus on information governance as a discipline to support enterprise information management projects such as master-data management, data quality management, and data integration projects. Business issues that highlight the need for information governance include data quality, financial restatements, regulatory compliance, customer relations, and business process gaps. 1 Managing Information in the Enterprise: Perspectives for Business Leaders, Forbes Insight survey, Information Quality Information quality is a primary concern for information governance initiatives.

13 Sponsored Content DBTA SEPTEMBER Poor-quality information such as missing customer contact information or multiple product IDs for the same item in different systems can proliferate throughout the operational landscape and have far-reaching consequences. Information issues can spread quickly within and across applications, throughout the systems that facilitate virtually all critical business operations. The effects of poor-quality information are not limited within company walls. Business users outside your organization such as customers and suppliers often have self-service access to the organization s systems. This fosters customer loyalty, improves supplier relationships, and reduces costs but it also exposes internal process issues and information flaws. Simply put, businesses need good information to run well. But by the time information quality issues come to the fore, the problem is often widespread. For example, in mergers and acquisitions, combining company information often results in duplicate customer name and address information. This duplication may lead to poor customer service, ineffective marketing campaigns, and missed sales opportunities. Financial Restatements Financial restatements can affect company reputation, negatively impact the stock price, and result in regulatory fines. Most restatements are caused by invalid or missing operational or financial information. Government regulations in this area directly affect internal controls on organizational processes and reporting practices, adding complexity to the situation. Restatements divert management and employee attention, time, and energy. Regulatory Compliance Managing compliance is crucial. Reports to stakeholders must be precise and timely. Information must be kept as long as retention laws mandate. Businesses must support enterprise-wide compliance with standards including International Financial Reporting Standards (IFRS), the Sarbanes-Oxley Act, and generally accepted accounting principles (GAAP). Manual and homegrown approaches, such as the use of spreadsheets or legacy applications, are not up to the task of facilitating compliance. There is simply too much information to govern, the speed of business is too fast, and manual processes inevitably lead to errors, duplication, and wasted efforts. Customer Relations Acquiring, retaining, and servicing customers form the backbone of success for any company. But information redundancies and errors can stymie sales and marketing efforts and lead to reduced sales effectiveness, high marketing costs, unreliable analysis, and service issues such as out-of-stock items, invoice errors, and excessive call center queries. Business Process Gaps Inefficient processes caused by incorrect or redundant information not only add costs but also can affect customer relationships and hamper the ability to seize business opportunities. For example, an incorrect price in your systems will cause a billing error, resulting in a collection process and the lack of timely payment. In the end, you have to provide a cash discount, and the customer will have an unsatisfactory experience. With accurate information, you can reduce such occurrences and improve productivity, reduce operational costs, and adapt to changing market conditions. BUILDING THE BUSINESS CASE FOR INFORMATION GOVERNANCE SHOW THE POTENTIAL VALUE OF AN INFORMATION GOVERNANCE INITIATIVE In many cases, companies can turn business process failures into opportunities to jump-start information governance efforts. You can begin by addressing the specific financial impacts of information issues and identifying compliance problems, which are easy to measure by regulatory fines levied. For example, consider a situation where bad information has compromised business processes critical to shipping activities, resulting in failure to deliver product and the associated financial loss. The discovery of this acute issue can be the impetus for launching an information governance program, including support for ongoing resource assignment and a data road map. During a business process reengineering with an enterprise resource planning (ERP) solution, a company might discover that too many employees are involved in maintaining master data, leading to inefficiency and errors. In such a case, centralizing masterdata maintenance and reducing the number of employees responsible for information can result in optimized business processes, reduced risk, and better decision making. An organization with many regional offices or franchises can implement a governance initiative to align information elements across the enterprise and enforce consistent rules and practices. The resulting improvement in information quality can create a considerable return on investment for the business. With a solid information management and governance strategy, you can involve key stakeholders in defining information parameters for example, establishing what constitutes valid point-of-sale data. By enlisting people who are most familiar with the information, along with businessprocess-engineering experts, you can come to agreement on how specific types of information should flow through the enterprise. This helps align and improve information across the organization, building effectiveness and profitability. STEPS TO IMPLEMENTING INITIATIVES A Foundation for Successful Information Governance While there is no single route to establishing an information governance practice, certain steps are key to successful initiatives. Understand Why Information Governance Matters to You First off, you need to recognize why information governance is important to

14 24 SEPTEMBER 2012 DBTA Sponsored Content your success and identify information issues in the organization. As discussed above, the most common issues are poor information quality, financial restatements, compliance issues, customer issues, and business process gaps. Involve the Right People Successful information governance initiatives have positive effects throughout the company and involve stakeholders from various departments and roles. This is true for strategic IT initiatives, such as cloud implementations, and for strategic business initiatives including business intelligence implementations, expense reductions, and mergers and acquisitions. Executive sponsorship for cross-organizational alignment and funding is critical for success. You need to target executives with the most to gain from well-managed information and start the conversation by discussing specific business issues. In addition to executive support, you ll need a knowledgeable team with line-of-business representatives from marketing, sales, supply chain, finance, manufacturing, IT, and so on. The governance team establishes policies, processes, definitions, standards, and metrics with the most effective key performance indicators. You need subject-matter experts in the relevant business processes, and process owners who can create and update critical, shared strategic information. Line-of-business owners help define the accuracy and usefulness of information in meeting business goals. You also need to include people who are well versed in internal auditing, risk management, and compliance and privacy issues especially if your information governance initiative is tied to legal requirements. IT involvement is crucial for successful initiatives, with several key roles including data architects, data modelers, and database analysts. Data architects ensure the various elements of the data management strategy and solutions come together, including databases, tools, and other technologies. Data modelers work with data stewards and data architects, translating business definitions and taxonomy into logical and physical IT models. Database analysts translate data models into physical layouts in databases and implement and oversee database changes and operations. Information stewards, an emerging role in many organizations, drive information ownership and accountability policies. Ideal stewards are people who best understand the information and its value to the business; they are not (usually) programmers or part of the IT department. Information stewards monitor data fitness across the enterprise to improve the overall quality of information. Define Information Policies and Procedures A comprehensive information governance program must address the policies needed for all information, including both structured to unstructured types. Policies encompass security, responsibilities, and ownership; legal obligations; information quality and lifecycle; and interactions with the governance team. Procedures ensure that policies are enforced to drive data quality and information lifecycle management. For example, a policy concerning customer master data could mandate a minimum number of information quality checks, audit trails, and multistep approval processes. Another policy could define the minimum amount of time that financial information must be retained before it expires. While many organizations understand the value of high-quality information for running their business applications, the need to harness the unstructured content that now floods into most organizations within a business process is often not recognized. Integrated content management avoids siloed governance for unstructured and structured content, ensuring efficient and high-quality business process execution. The applications that most organizations use for enterprise resource planning, customer relationship management, and supply chain management are typically designed to manage structured, transactional information such as addresses, customer numbers, and order numbers. With integrated enterprise content management, you can effectively govern and support document and records management, collaboration, archiving, scanning, and information retrieval. Identify Processes and Systems That Create and Update Information It is essential to identify and manage information creation, updating, and deletion processes, as well as the transfer of data from one system or application to another. Governance policies and processes must be aligned with corresponding business processes to support overall strategic goals. For example, sales and marketing processes are aligned with policies governing customer information. Supply chain processes are aligned with governance polices for data entities related to suppliers and materials information. Monitor Compliance and Establish Remediation Processes Monitoring and remediation drive compliance and business change. Information stewards identify areas that may require business change and perform root cause analysis. The core governance team collects and prioritizes incidents of breaches in policy and procedures, and it works with information stewards to investigate and propose possible solutions. Compliance and remediation activities include end-user training; process automation, creation, and modification; and new solution implementation. SUPPORT AND AUTOMATE GOVERNANCE OF YOUR INFORMATION SAP SOLUTIONS FOR ENTERPRISE INFORMATION MANAGEMENT Information governance initiatives support strategic business goals by providing trusted, consistent, and secure data and software solutions are essential to achieving these goals. Solutions supporting governance must provide a range of functionality

15 Sponsored Content DBTA SEPTEMBER including information stewardship, data integration and quality management, master-data management, workflow and rules management, enterprise content management, information lifecycle management, and business intelligence. SAP solutions for enterprise information management deliver a comprehensive, integrated way for you to automate and enforce information governance policies and standards, providing the reporting and analysis that your company needs to monitor effectiveness. With SAP solutions, you can empower the business to own and manage its information with intuitive solutions for better information stewardship. You can govern information in the business process to optimize operational performance and help ensure compliance. And you can establish trust in structured and unstructured information by helping ensure information quality throughout its lifecycle. The SAP solutions facilitate information governance initiatives and drive performance and efficiency. For example, the solutions help provide trusted information to optimize supplier relationships by improving purchasing and reducing cost of goods sold. Warehouses can use higher-quality bar-code data to speed shipping. Power plant operators can use carbon input data to facilitate compliance. Distributors can use geospatial data to improve delivery routes and timing. SAP solutions for enterprise information management include the following. The SAP Information Steward software empowers business users throughout the organization to assess data quality, understand the lineage and impact of data across systems, and define business definitions for information. It enables them to create rules to cleanse information and use dashboards to continuously measure and monitor data quality. The SAP Data Services software provides an information management foundation for moving, improving, governing, and unlocking the value of enterprise information from structured and unstructured sources. The software includes functionality for extract, transform, and load (ETL); data quality; data profiling; metadata management; and text analytics. SAP provides embedded data quality functionality for SAP Business Suite software and third-party applications so that you can enforce governance policies in the business process. The SAP NetWeaver Master Data Management component provides an open solution that supports the consolidation and syndication of master data from any data source and type, such as customer, product, material, or employee. The SAP Master Data Governance application centralizes the creation and management of customer, material, supplier, and financial master data for your SAP Business Suite software. The SAP NetWeaver Information Lifecycle Management component enables you to set retention rules and retain business records for different periods of time according to policy or legal requirements. This holds true for structured and unstructured content, for live and legacy systems, and for SAP and non-sap solutions. You also can collect and preserve records related to ongoing legal cases. The SAP Extended Enterprise Content Management application by OpenText allows you to efficiently manage unstructured information (such as attachments and word processing documents) along with structured information (such as application data) in the context of business processes. The SAP HANA platform enables highperformance in-memory computing that gives your enterprise the ability to instantly explore and analyze huge volumes of data in real time. Big data analytics requires that you deliver trusted information that is effectively governed. The SAP NetWeaver Process Orchestration software provides IT organizations with a framework of tools to design, model, implement, run, monitor, operate, and improve business processes flexibly throughout their lifecycle. The SAP BusinessObjects Business Intelligence suite gives people in roles throughout the company self-service access to relevant information. Business users can make better decisions based on fact-based, quality information, regardless of where the data resides. YOUR NEXT MOVE A good way to start is with a reasonably sized project that suits your information needs and your budgetary constraints. You can implement the project and track metrics to demonstrate your success. Then, with buy-in from people from C-level executives to line-of-business managers, you can expand governance initiatives throughout the enterprise. Learn More In today s information-saturated environment, businesses need to maximize the value of enterprise information to improve efficiency and effectiveness. But managing information and processes across interconnected applications and systems is a complex endeavor. With information governance initiatives supported by SAP solutions, you can make the most of your enterprise information, empower people across the business with information access and ownership, improve information transparency, and enable collaboration. To learn more about how SAP solutions for enterprise information management can support your governance initiatives, contact your SAP representative or visit us online at

16 26 SEPTEMBER 2012 DBTA Sponsored Content Highly-Available MySQL and Real-Time Big Data with Continuent Tungsten MySQL, the world s most popular open source database, is a pioneer in data management in the web economy. System builders choose MySQL for its simplicity, robustness, and low cost but with the advent of Big Data face a new set of challenges. Continuent Tungsten, an advanced clustering and replication solution for open source databases, helps MySQL users handle the demands of huge data sets and new real-time reporting requirements while preserving the features that make MySQL great in the first place. ORGANIZE RELATIONAL DATA INTO SCALABLE ARRAYS Big Data is a problem for every sort of relational DBMS, and no more than for MySQL. Software-as-a-service applications for market automation, intrusion detection, and mobile applications now commonly manage 10s of terabytes of data on MySQL. Continuent Tungsten clusters with advanced connectivity and load balancing make it easy to split data into arrays of MySQL instances each managing a few terabytes apiece. The horizontal scaling model allows business to cope with the growing deluge of data by scaling applications and data incrementally using a divide-and-conquer approach. Continuent Tungsten-based systems process 100s of millions of transactions daily with multiple options to scale capacity as data sets increase. STAY AVAILABLE WHILE GROWING FAST Agility is a watchword for modern IT the ability to adapt applications to new requirements quickly while maintaining high levels of availability to users. Continuent Tungsten clustering keeps transaction processing systems continuously available through software upgrades and maintenance. Continuent Tungsten also automatically protects businesses from show-stopping host, network, and site failures as well as human mistakes that can lead to chaos in growing businesses. With Continuent Tungsten, you can deal with future needs without neglecting basic promises to keep systems running and support current business needs. GET SAFELY INTO THE CLOUD Cloud computing offers businesses the ability to build out systems quickly without up-front capital investment or huge IT departments. Continuent Tungsten s software-only architecture works equally well in on-premise, managed hosting and cloud-deployments. Easy and fast installation allows users to install and configure new cloud-based clusters and replication links in minutes, as well as to provision copies of data rapidly from backups. Continuent Tungsten supports both multi-master as well as primarydisaster recovery configurations to spread data between data centers as well as Amazon (AWS) Regions and Availability Zones for maximal protection against large-scale failures. Continuent Tungsten clustering protects individual databases in replication chains to keeps data flow up and running even if individual servers fail. Continuent Tungsten s own resource requirements are modest, so you can keep cloud resources focused on data processing rather than running infrastructure. REPLICATE TO ANALYTICS IN REAL-TIME One of the biggest changes in analytic processing is not just that users have more data but that they want to extract actionable information from it more quickly. True Big Data is therefore characterized by real-time reporting, which creates an urgent need to move data quickly from MySQL into solutions like Vertica, MySQL-based column stores like Calpont InfiniDB, and Hadoop that can process complex queries quickly. Continuent Tungsten s replication extracts data asynchronously by reading committed transactions from the log and can also run off-board. The result is fast replication with minimal load on production MySQL systems. On the reporting side, Continuent Tungsten supports fast, parallel batch loading to quickly populate data into column stores like Vertica and distributed file systems like Hadoop. Finally, Continuent Tungsten handles a wide range of data replication topologies to allow users to fan-in streams of data from dozens of MySQL operational stores to data warehouses. REMAIN UPWARDS-COMPATIBLE WITH NEW BIG DATA SOLUTIONS The Big Data field is changing very rapidly, with new solutions appearing almost weekly. MySQL is a good bet for building extremely fast operational systems with strong transaction support but exposes users to the danger of creating systems that cannot integrate quickly with new types of analytics or new database storage systems. Continuent Tungsten offers the key benefit that it enhances MySQL and integrates data warehouses without requiring application rewrites. You can adapt your current systems to meet new Big Data requirements without migrating data from existing stores. Moreover, Continuent Tungsten itself is modular and highly configurable. It can support rapid addition of new data store types as well as replication topologies. SUMMARY Big Data is creating an exciting world for database practitioners and users alike. Continuent Tungsten provides the plumbing to allow users to implement flexible and robust solutions to manage rapidly growing volumes of transaction data while maintaining extremely high availability. At the same time, Continuent Tungsten provides the real-time integration necessary to use real-time analytics effectively and provide fast answers to important business questions. Isn t that what all users want? CONTINUENT

17 Sponsored Content DBTA SEPTEMBER Agile Data Transforms Master Data Management Over time, as companies acquire other firms or create new divisions, data sets tend to become distributed across multiple systems. The goal of Master Data Management (MDM) is to reintegrate that data into unified views: a single view of customers, of risk, of products, of sales, etc. Better visibility results in better operations, manifested in higher sales conversion rates, better customer service scores, or lower operating costs. COMMON MDM OBSTACLES In practice, however, MDM can be costly and slow to deploy. Every MDM project hits a similar wall: the data it needs lives elsewhere and is owned by others. Getting access to that data, at the right time and repeatedly, is difficult. The obstacle is not always technology; organizational processes get in the way. Each department that owns a data set manages that data according to its own business needs. However, the combined result causes major problems for teams that want to access that information. Too often, data warehousing and MDM initiatives face project delays and poor initial data quality. For example, one system owner might allow access to his database, but not until a current project is completed. Another system owner might be making schema updates, and can only provide an older copy of the data. A third system owner is happy to provide data sets, but only once a month. Yet another will provide data, but it s from an old version of Oracle and the new MDM app will be on a much newer version. Once the data is collected, the data sets are not in sync, and customer history doesn t quite match up, order history might be incorrect, outstanding tickets don t appear, etc. Ensuring that the data is correlated accurately requires additional work, re-collection at different time points, and an overall delay in the project schedule. Of course, the MDM system will support a new sales compensation plan, being rolled out in Q3, so MDM must be released in Q2. The warehousing team is forced to cut back on testing and exclude certain data elements, so when the project ships, initial data quality is not what it should be, and users complain. Sound familiar? DELPHIX ACCELERATES MASTER DATA MANAGEMENT Delphix transforms MDM by decoupling data access from the underlying infrastructure. Through virtualization of relational databases, Delphix gives MDM project teams greatly increased data access and project control, while reducing the impact and dependency on shared infrastructure. Delphix software is typically installed on a per-project, per-application basis, and can dramatically shorten the time required to create full, synchronized data sets for integration testing. Once in place, Delphix reduces the MDM team s impact and dependence on shared IT infrastructure. Through self-service, application engineers can create, refresh, and rewind databases for development and testing. CASE STUDY: INFORMATICA As an example, Informatica, a market leader in MDM solutions, used Delphix in conjunction with its own products to accelerate an internal MDM project schedule by 50%, delivering a MDM solution 6 months early, with higher initial quality. Using Delphix, the Informatica team created full-function virtual copies of six different production customer databases. These copies are kept in sync with production and can be provisioned to any point in time, with all copies can be synced to the second. As a result, the data warehousing team was able to work at its own rapid pace on the MDM project, without disrupting operations at the production systems. ETL code could be tested, tweaked, and verified quickly, independent production operations. A project that was estimated to take 12 months was instead delivered in 6 months, primarily due to accelerated access to the source data and orchestration across the data sets. Creating integrated virtual copies with Delphix enabled the Informatica MDM team to make the best use of its own world-class tools and expertise. Tony Young, Informatica CIO, states: The cost savings and reduction in complexity we have achieved with Delphix have been unprecedented. Last year, Ventana Research gave Informatica an IT Leadership award in the area of IT Performance Management for the success the company achieved using Delphix for its MDM initiative. While server virtualization has brought significant time and cost benefits to the data center, those same benefits have not been realized at the database tier. Delphix increases application team agility, alters the economics of database management, and provides massive real-world benefits to global leaders including Facebook, Qualcomm, Macys, P&G, KLA Tencor, Comcast, and others. DELPHIX To learn more, please go to:

18 28 SEPTEMBER 2012 DBTA Sponsored Content Tips for Data Integration, Replication, and Synchronization Benefits of Change Data Capture Data management issues span data entry to data archiving, however minimizing the work and time to manage data lifecycles and being flexible in extracting maximum benefit from data give you competitive advantage. This can mean automating tasks or strategically managing your data. In a perfect world, you should do both. Frequently, data needs to be converted, separated, copied (replicated) or migrated. This part of the lifecycle is typically repeated multiple times with different parameters. Some businesses consider this a simple task and allot minimal investment and time to it, believing that it should be done manually. But this is one area where using a reliable and powerful data replication tool will save hours/days and thousands of dollars in man-hours and lost business. Change Data Capture (CDC) is defined as moving only what s changed in the data, rather than entire data sets, to update a downline database. The following scenarios benefit from using data replication with CDC and logreading. REAL-TIME DATABASE UPDATES AND SYNCHRONIZATION Passing data between operational applications such as ERP, CRM and MRP is no easy task. Data needs to be reconfigured, re-mapped, sorted and then transported in a timely fashion to best support applications and business performance. Good data migration/ conversion and automated data movement with CDC (real-time or batch) can improve inventory turns, customer retention, sales process and financial management. REAL-TIME REPORTING AND ANALYTICS Everyone knows that fresh data is the key to the best reporting and analytics. Tools that automatically deliver fresh data to the reporting application without disturbing critical systems such as production systems or other operational applications are now mature and proven solutions that can include data replication and CDC. EFFICIENT DATA WAREHOUSE LOADING If your business relies on data warehousing for analytics and reporting, the how and when data gets loaded are key. Too many steps cost you in time and decision-making. Save time by using a data replication product that is already integrated with your data warehouse (or appliance) and designed to load data based on your parameters, out of the box, using CDC. FAST UPDATES FOR COLUMNAR DATABASES If you are working with columnar databases for data processing speed, why forfeit time in data updates? Using a data replication tool integrated with columnar databases minimizes data update time by automatically transforming data between formats and providing ongoing updates without manual intervention. DATA UPDATES TO DISCONNECTED DATABASES OVER THE CLOUD If you are sharing data between offices, plants, retail outlets or other remote locations, you need a data replication tool that supports many databases and platforms, and can handle data updates and CDC without dependence on connected lines. Handling data updates on remote sites and passing that data through the Cloud requires a reliable, safe and secure process that is minimally invasive to systems, lightweight and cost-effective. DATA UPDATES TO CLOUD DATABASES Many businesses now look to Cloud databases for large datasets, storage and retrieval elasticity and cost-efficiency. But how about sharing data with Cloud databases? This is complicated, particularly when the data systems are not relational. Using a data replication solution with CDC efficiently moves large data sets to and from the Cloud systems with the least amount of effort and time. INFORMATION GOVERNANCE GOALS Businesses require data replication between each data transfer or migration touchpoint in order to achieve Information Governance. Selectively automating the transformations, data movement and scheduling significantly reduces time to achieve data quality, and delivers an overall faster time-to-value in data management. Decreasing data cycles, minimizing the effects of data movement and reducing man-hours all contribute to a better bottom line. For more tips, see HiT SOFTWARE, INC. a BackOffice Associates, LLC Company info@hitsw.com