CIC Audit Review: Experian Data Quality. Guidance for use of data quality tools to leverage data advantage

Transcription

1 CIC Audit Review: Experian Data Quality Guidance for use of data quality tools to leverage data advantage March 2014

2 Table of contents 1. Challenge Overview Experian Data Quality at a glance Experian Data Quality Tools Data quality tools review and strategy Standards and openness support Roadmap directions Competitive Differentiation CIC Analysis Experian Data Quality summary benefits End User Directions Guidance strategies Tooling 10 Foreword Companies are beginning to realise the value of clean data to business operations. To maintain the quality of the data, there needs to be a continuous process that ensures new data is validated as it arrives in the organisation and existing data is monitored to prevent data drift. This requires a mix of policies and tools that are embedded as part of the data workflow. This audit looks at the software tools available from Experian Data Quality to automate and validate data to maintain great data quality. Ian Murphy, Principal Analyst, Creative Intellect Consulting Bola Rotibi, Research Director, Creative Intellect Consulting Creative Intellect Consulting is an analyst research, advisory and consulting firm focused on software development, delivery and lifecycle management across the Software and IT spectrum along with their impact on, and alignment with, business. Read more about our services and reports at

3 1. Challenge Overview Companies are investing significant sums of money and time into building a data quality strategy. Driving this is the need to leverage their existing data assets to find new competitive opportunities. Without accurate data, they run the risk of wasting money, missing opportunities and alienating potential business. Before any company can create a reliable data quality process, they need to consider investment in data quality tools and reference data as part of a broader data quality strategy. These help to ensure that any data entered and held by the company is accurate, reliable and most importantly fit for purpose. Understanding the value of data means a change from the current grab anything approach to a trust and verify approach. Too many companies are focused on creating a mass of data assets that can then be analysed to find a new competitive edge for their organisation. If that information cannot be relied upon to be accurate, any knowledge or insight derived from it cannot, in any reasonable way, be considered valuable or something on which competitive advantage can be built. At the heart of the problem is the way customer prospect data is captured, cleaned and then enhanced. There are many examples of the impact of poor data quality affecting UK businesses today. Attempting to engage with customers who have passed away or moved away, is not an uncommon mistake. This drives unnecessary cost and reputational risks for businesses whilst undermining their competitive edge. Achieving all of this, however, has been seen in the past as expensive and time consuming which leads to a failure to maintain clean data. This means that companies who have invested in a data quality initiative run the very real risk of seeing that money wasted as data becomes inaccurate over time. This process is called data drift and is what data quality tools, as part of a data quality process, are designed to prevent. The reasons for data drift are many. Data enters the company from many sources, some of it is keyed directly into systems by staff, partners and customers. While other data can be purchased from third parties. This data is often unchecked and inaccuracies in one set of data is transferred to others thereby undermining the data quality process. Those companies that realise this is a long term challenge that requires a commitment to continuous data validation, will drive more operational efficiency and have a much higher customer satisfaction ratings as a direct result. From a financial perspective, happy customers mean greater business efficiency and higher revenues. 1.1 Experian Data Quality at a glance Experian Data Quality (formerly QAS) has built up exceptional market coverage assisting customers with their unique data challenges. Their vision is to ensure that all of the data their customers hold is fit for purpose and quality is maintained overtime. The company provides a comprehensive toolkit for data quality projects combining software with a broad scope of reference data assets and services. The Experian Data Quality mission is to enable customers to make the right decisions from accurate and reliable data. The size and scope of data management projects varies considerably but the common factor in all ventures is unlocking operational efficiency and improving customer engagement and boosting the bottom line. The core capabilities of Experian Data Quality lie in the following areas: Data quality strategy and processes Profiling and visualisation Cross channel data validation (Address, , mobile) Cleansing and deduplication Data enrichment at point of contact or retrospectively Governance and monitoring CIC Audit Review: Experian Data Quality- 3

4 In table 1 below, we provide an outline of the important categories that help to define the characteristics and strength for a robust data cleansing and quality portfolio. Figure 1 provides Creative Intellect Consulting s rating review of the Experian Data Quality s contact data offerings. Table 1: Key categories and characteristics for data cleansing and quality support Category Criteria Usability International Data support Tooling strength and capability Solution support Partner Ecosystem External Reference Data support Integrations and integration support Deployment Options Evaluation Definition This evaluates the ease of use of the applications and tools that customers will use to maintain their data. Of specific importance will be the focus given to user experience features. This evaluates the extent, breadth and sophistication of the access to international reference datasets and the support given for use and validation. This evaluates the breadth and sophistication of the data quality tools provided and the range of delivery mechanisms offered. This evaluates the level of technical support available to customers in the case of product functionality challenges. This evaluates the sophistication of the partner network and the extent to which support for partners, to add value and leverage the portfolio services, is provided. It also encompasses the variability of the partner program support (online or otherwise) and the flexibility by which partners can engage e.g. premium, standard etc. This evaluates the extent, breadth and sophistication of the access to third party reference datasets and the support given for use and validation. This evaluates the support for widely used enterprise business applications such as ERP and CRM application platforms/systems. This evaluates the range of deployment options for the tools and reference datasets available to customers. Figure 1: Chart representing Creative Intellect s rating review of the Experian Data Quality Tools portfolio Score guide Usability 0 Laggard 3 Market mean 6 Market Leading Deployment Options International Data support Integration and integration balance 1 0 Tooling strength and capability External Reference Data support Solution support 4 - CIC Audit Review: Experian Data Quality Partner Ecosystem

5 1.2 Experian Data Quality Tools Cleaning data once is not enough. Data quality management is a continuous process of improvement across the data lifecycle. Experian Data Quality tools cover three key stages for the management of customer data: Capture: the data across multiple channels ( , phone, physical address) as it enters the organisation and immediately validate it. Utilising these tools in conjunction with the right datasets significantly reduces error at the point of collection, minimising risk to core customer facing applications. Clean: the data as part of a continuous process to ensure that quality is maintained; identify changes in customer status making sure records are updated; understand where duplicate records exist to eliminate redundancy and improve response rates to customer issues and needs. Enhance: is about moving beyond the data that has been captured and adding value to it. The use of external data assets can help fill in gaps. Identify information essential to marketing campaigns and make customer data richer and more focused. While data quality tools focus on data accuracy, data quality management requires additional functionality such as profiling, visualisation and monitoring that is delivered through the Experian Data Quality Platform ( It s important to note that the use of data quality tools is no longer something that is only available to large enterprises. The arrival of cloud based solutions means that any company can create and manage a continuous data quality process for their data. 1.3 Data quality tools review and strategy Experian Data Quality have a rich history in the contact data/name and address data space leading to a range of off the shelf software tools and tested reference data assets. Each of the tools can be used independently of eachother and the datasets can be purchased as a whole or as separate parts. This provides a flexible set of solutions to customers. Of particular note is the support for International datasets supported through both on premise and cloud delivery options. Experian Data Quality has staff and partners operating in over 220 countries around the world. For 20 countries, including large European countries, Australia, Canada, Singapore and the United States, it has sourced data directly from governments and can provide a number of enhanced datasets. In addition, it has partnerships with companies in a further 200 countries where those partners source and validate the data for Experian Data Quality customers. A full breakdown of the different tools within the Experian Data Quality tools portfolio can be seen on the Experian Data Quality website. Table 2 below provides a summary list of key functionalities. Table 2: Summary of Experian Data Quality Tooling Capabilities Capability Capture Functionality Review Address capture which uses simply a postcode and building number to allow correctly formatted addresses to be imported from the Experian managed data files. Capture of contact data through web forms with support for developers to integrate Experian components into their software for accessing local copies of the Experian address datasets. Deployment Options On premise and Cloud. In both cases, the underlying dataset is updated regularly by Experian, ensuring maximum accuracy. CIC Audit Review: Experian Data Quality- 5

6 Table 2: Summary of Experian Data Quality Tooling Capabilities (cont...) Capability Clean Functionality Review Removal of duplicated data entries Validation of addresses Checking of mobile phone numbers to ensure correct formatting and that the number is in service Validation of business addresses against the Experian National Business Database and address datasets Validation against International datasets Validate against not-yet-built datasets Batch validation of internal and third party datasets for continuous data cleansing and suppression. Deployment Options On premise Programmatic APIs Cloud delivery for real time cleaning of contact data such as , address and mobile validation. Enhance Adds additional business customer data e.g., telephone and fax numbers, business registration numbers, Standard Industry Classification (SIC) codes Adds electoral register data associated with residential addresses Builds on the name and address data to provide a wider profile of the contact (e.g. a broad range of government managed datasets, Ordnance Survey Code Point and ADDRESS-POINT for greater accuracy of locations, Experian Mosaic for profiling customers) Batch enhancement of internal and third party datasets through datasets such as Experian Mosaic for advanced marketing purposes Enhance campaigns to support not yet built datasets International dataset enhancements. Appended at the point of capture or retrospectively On premise Cloud options 6 - CIC Audit Review: Experian Data Quality

7 1.4 Standards & openness support To make it easy for customers to integrate a capture, clean and enhance approach with their existing data, Experian supports a wide range of software standards and common programming languages. This includes XML, J2EE,.NET, Java, C#, C, C++, VB.NET, JSP, PHP, ASP languages and SQL. All Experian Data Quality solutions support either ODBC or JDBC making it possible to connect with the widest possible set of data sources. 1.5 Roadmap directions Experian Data Quality has already established its own cloud-based Software as a Service (SaaS) solution that allows customers to do real-time data validation. During 2014, we expect to see Experian Data Quality continue to strengthen in this area through extending the reference datasets that they offer and giving customers a greater ability to enhance data. Irrespective of customer size, this is an easy to use method of data cleansing. Like many software companies, Experian Data Quality has a lot of point solutions. With their SaaS solution well established and the company rebranding complete, we expect them to package up existing capabilities providing customers with a joined up off the shelf solution. The most likely route will be through the delivery of a Data Quality Suite or at least, a set of product bundles. This will enable customers to choose exactly what suits their business and provide them with the ability to extend the cleansing and more importantly the data enhancement that they need. With an increasing amount of data coming onto the market via Governments and private companies, a third area of growth will be OpenData. Experian Data Services are well placed to buy in more reference datasets, clean and enhance them to add significant value. This is something that their competitors may well struggle to compete with. 1.6 Competitive differentiation Gaining competitive advantage from customer information requires data that is highly accurate. Those companies that attempt to treat data as a one stop, quick fix, will soon see their data quality deteriorate. They will then be faced with additional high costs to remediate that data. Experian Data Quality has a range of software and data assets that make it possible to capture and validate all data as it enters the company irrespective of where it comes from. Once the data is in the company system the validation does not stop. Using tools such as QAS Batch and Unify, datasets can be validated and deduplicated. The breadth of the Experian datasets allows companies to add significant enhancements to the data that they hold. Examples are the Experian Mosaic dataset which adds in key marketing data and the Experian business data sets which add in telephone, fax and business category data. Further enhancement is possible by using other government datasets that Experian Data Quality has licenced. At its most extreme this can manifest itself through the use of Experian propensity data a great example of this is in the healthcare sector where it is used to ascertain an individuals communication preference alongside the risk of failing to attend an appointment. Companies no longer operate in a single national market so there is a high demand for Internationalisation of data assets. A significant differentiator for Experian Data Quality is its ability to provide customers with access to a wide range of international data that they can validate against. While it is possible for customers to source the international data themselves, having it delivered as part of a single coherent solution, makes it easier to utilise, less prone to error and more cost effective. It is also worth noting that Experian has accelerated prebuilt integrations into enterprise applications such as SAP, Oracle and Salesforce.com that can manage data quality from an end to end perspective. See more information on Experian Data Quality in this space. Experian Data Quality has already established its own cloud-based Software as a Service (SaaS) solution that allows customers to do real-time data validation. CIC Audit Review: Experian Data Quality- 7

8 2. CIC Analysis Experian Data Quality holds a privileged position in the data quality market as a wholly owned business of one of the leading information services companies in the world - Experian. This is an organisation with a very broad range of data quality tools and reference data assets that enable the process of Capture, Clean and Enhance. Customers can choose to do data quality in several ways. Some will prefer to buy on premise tools while others will be happy to work with cloud based services and redirect data validation to the Experian servers. Some may choose a combination of both depending on how these tools fit into their overall data quality strategy. Customers implementing a data quality initiative and who are struggling with defining their data quality processes, can engage Experian Data Quality to help build the right processes and make sure the tools are implemented in the most effective manner. Data is not just about the initial capture process. Experian Data Quality has tools that ensure a continuous validation of data, especially where that data is subject to change in such a way that the enterprise may not easily identify. Examples are bereavements, address changes, company bankruptcy and the increasing use of preference services to reduce direct marketing. By providing the ongoing suppression sets to identify and update records, Experian ensures that its customers can rely on their data. An area that Experian has invested heavily in is validation for business users. Timely and accurate communication is critical for any business. The impact of getting it wrong is not just lost business opportunities. Reputational damage through poor or failed communication and blacklisting due to poor sender reputation can have a severe effect on the ability of a company to talk to its customers. There is another choice and it is one that we would caution customers against considering and that is to deploy an in-house built solution referencing raw versions of datasets such as PAF (the postcode address file). Customers that choose to roll their own are faced with the need to constantly update the datasets, to build their own integration engines and to devise their own enhancements. This is more than just a simple data management process and cannot be overcome through the use of analytics, especially as the analytics would need to resolve data errors during initial capture. Another major challenge for roll your own is the complexity of identifying ongoing changes to data that customers may not notify the organisation about. Experian Data Quality has built its reputation around its reference data assets. It has simplified and enhanced data from other sources such as the Post Office and the UK Office for National Statistics. The company has brought those reference data assets into a coherent offering that is easily accessed from its own software and made it simple for developers to call the validation routines in their code. A major benefit that Experian Data Quality brings to customers is the support for International data. Much of this data is directly sourced by the company and where it has no direct access to local data, or where there are legal issues over access, it has sought to work with local partners to expand its offering. With business at all levels, from SME to enterprise, increasingly having an International element, this ability to support non-national data, in the same way as domestic data, is a significant bonus. What Experian Data Quality brings to users of its software and data is simplicity of use, standardised interfaces, the ability to integrate with their existing applications and robust security. 2.1 Experian Data Quality Summary benefits Experian Data Quality tools and datasets make it easy for customers to validate data as it is captured and then do continuous data validation and enhancement. Without the ability to capture, clean and enhance, it is not possible to build any lasting data quality process. This would What Experian Data Quality brings to users of its software and data is simplicity of use, standardised interfaces, the ability to integrate with their existing applications and robust security. 8 - CIC Audit Review: Experian Data Quality

9 leave companies spending a lot of money for one-off data cleansing projects that will inevitably degrade as data drift occurs over time. For companies who want to gain competitive advantage from big data and analytics, poor data quality will significantly inhibit this. Experian Data Quality tools can be installed onsite or accessed through the Experian cloud. This provides a highly flexible set of options that can meet the data validation demands of any business. Experian holds an extensive set of data for validation purposes. Some of the data is wholly owned by Experian, while other datasets are drawn from public bodies and then enhanced. All of the datasets can be used to validate and enhance customer data to ensure that information held is as accurate as possible. With such a very large range of reference datasets, it is possible to do everything from checking names and address through to building a highly focused hub of data for marketing needs. The key benefits of Experian Data Quality tools are: A wide range of data capture tools and reference datasets that can speed up the capture and cleaning of data. Reference datasets that enable customers to improve and add value to the data they own. International datasets that enable companies to operate across national borders. Ability to identify and remove duplicate data to reduce unnecessary storage and improve data quality. These benefits ensure that companies can reduce and even eliminate the risk of data drift from their business, lowering cost and raising customer satisfaction levels. 3. End User Directions Cleansing data and keeping it clean is no longer a highly complex, expensive process requiring significant amounts of IT resource. Developers have invested a lot of time trying to reduce errors at point of capture but all too often this has relied on drop down boxes for specific fields. Using reliable, trustworthy data sources, it is possible to standardise and eliminate the majority of mistakes that happen during data entry. Capture is not enough, however, to prevent data drift, even over the short term. There needs to be a process that continuously validates and checks data as well as enhances this data. To achieve this, companies need to invest in both data quality tools and in reliable datasets. 3.1 Guidance strategies There are key actions that you should take to create a robust data quality process. This next section provides the core areas of focus for action. People, Process and Business Engagement Understand your data quality process within your organisation: Many companies struggle to clearly define their data quality processes, which makes it hard to invest in the right solution and tools. Quantify the standard of data quality within your architecture understand the scale of the problem that exists: Without knowing the accuracy of your existing data, it is hard to define a strategy to improve it. Carry out a test cleansing of a subset of existing data as well as newly captured data. At the same time, set a level of acceptability for the business. Should data be 100%, 80%, 60% or less accurate? Work with the business to prioritise the areas for improvement: The obvious place to start is with new data entering the business. Beyond that, identify where poor data is causing additional costs or loss of reputation. Make sure that wherever you start, the process covers all data in that area. Think about the deployment options that are right for your business: There are two options, on premise or cloud using Software as a Service (SaaS) to deliver the latter. This is a decision about cost, management and effectiveness and IT as well as the business units should be engaged early in this choice. CIC Audit Review: Experian Data Quality- 9

10 Think about the global reach of your organization: If you engage outside a single country then it is important to make sure that you have international data cleansing capabilities. Choosing a partner with global coverage might not be easy but it is far easier than trying to gather the data and do it yourself or working with multiple data partners. Think about how you would like to enrich your data: If your data is just about names and addresses, then there is probably little enrichment to be done apart from accuracy. If it is to be used by marketing teams or as part of a wider application set, you need to look at the various options that are available in terms of reference datasets. Have an education program that trains: Keeping data clean is not just about the tools, processes and reference datasets. It requires user education and a clear commitment to ensuring that data is not used unless there is proof it has been properly handled and cleansed. 3.2 Tooling Data quality tools and reference data that can validate data both at point of entry and throughout the data lifecycle, creates clean data that can be relied upon. They need to cover the following areas: Validation of data at input: Data validation at input is used to speed up data entry with the goal of reducing the risk of bad information by using high quality reference data assets to help complete forms. International validation of data: Being able to ask for the data using the correct names and then validating it against the right national database is essential. Deduplication: Being able to compare data and remove the duplicate data saves time, money and space on systems as well as ensuring the customer gets a more coherent response. Managing do not contact lists: Identifying customers on Telephone or Mailing Preference Lists who should not be contacted, is not just good customer experience it is a regulatory requirement. Get it wrong and it can lead to big fines. Corporate data: Validating that a company exists, still trading and is creditworthy is not just good business practice, it is essential to prevent fraud. Data about the state of a company constantly needs to be checked in order to determine whether they are a trustworthy supplier or customer. Extensive datasets provide higher quality data: The comprehensiveness and accuracy of datasets used to validate data is critical to ensure that any ongoing data quality processes can be relied upon. Getting the right tools and reference data in place to capture, validate and continuously monitor data changes is the difference between trusted data and poor data. If the tools and datasets are solid, accurate and usable, the result will be data that drives the business forward not data that requires continual manual remediation. Using the right tools, reference datasets and processes continuously will ensure that the data coming into the company is clean and stays clean CIC Audit Review: Experian Data Quality

11 Creative Intellect Consulting is an analyst research, advisory and consulting firm focused on software development, delivery and lifecycle management across the Software and IT spectrum along with their impact on, and alignment with, business. Read more about our services and reports at Creative Intellect Consulting Ltd 2013 CIC Audit Review: Experian Data Quality- 11