Master Data Management

Transcription

1 A White Paper by Bloor Research Authors : Philip Howard & Andy Hayler Publish date : January 2008

2 Data governance is vital to the success of an MDM project. Collaborative, analytical and operational MDM projects have quite different profiles Andy Hayler This document is sponsored by

3 In search of data s Holy Grail page 1 If you have ever worked in a large organisation, especially a multinational one, you will know the frustration of trying to get meaningful, timely information about business performance. How profitable is a certain account? How effective was the last promotional campaign? Which is our best channel partner? Which product lines really make the most money? The problem is not a lack of data companies are drowning in it. The difficulty stems from the fact that complex organisations have multiple ways of dealing with business issues. Different countries have differing market concerns and priorities, so set up additional systems to collect the data they need. This results in multiple business units each having differing figures, and each claiming their numbers are correct. This makes intelligent decision making difficult. Most organisations struggle with gathering operational information around a single business entity that has multiple touch points. As consumers we are used to receiving multiple mailshots from the same company, or moving house and discovering that changing one s home address with a bank is less than a seamless operation. Companies wishing to gain a consistent view of their customers, for example, have struggled with reconciling data in numerous operational systems. Another source of difficulty in a complex organisation can be the management of either new or updated business information that requires multiple stages or people to be involved. Introducing a new product category (or changing an existing one) may require stages of consultation and sign-off and have impacts on many systems once it is agreed. The heart of these problems lies in the handing of shared data and it should be self evident that it is better to run your business if this shared data is accurate, consistent and up-to-date rather than relying on data that is inaccurate, inconsistent and out-of-date. However, the amount of work involved in ensuring this is often under-estimated by business managers. A recent survey asked business managers to estimate how many separate sources of data there were in their (large) organisation for any particular piece of information (for example, a customer record). Typical responses were three or four and at most a handful. In practice, investigation proved that this was an underestimate by an order of magnitude: most of the companies in the survey had 30 to 40 sources of customer data and, in some cases, many more than this.

4 So what exactly is master data? page 2 Essentially, master data is data that needs to be shared by multiple systems or business processes. For example a third party company may be a customer of yours, but it actually may also supply another part of your organisation, and so also be a vendor. A product line that your company sells may have a unique code, which is stored in a marketing system, but is also used by systems in manufacturing, sales and logistics. Examples of master data include customer, product, asset, location, employee, organisational unit, legal entity and chart of accounts. Data that is not shared is not master data e.g. a record of a specific sales transaction is important, but in itself it typically will not be shared with other systems. Critically, transaction data itself is stable you go into a store and buy a bar of chocolate at a certain time and day for a given price. That transaction happened and does not change, but the context of the transaction may. In a few weeks time that store may be switched into a new sales region, the chocolate bar may be reclassified by marketing into a new luxury foodstuffs hierarchy, the store itself may even be closed or sold on, but that transaction still happened on that day at that time. This is a vital distinction, because it is the way that master data changes that causes the business problems we have alluded to. If only marketing never reclassified products and the organisation never changed then it would be relatively easy to compare the profitability of chocolate by region this June v last June for example. But if the store changed regions in February, and some toffee snacks that used to be dealt with separately are now classed as chocolate as of last October, then things get more slippery. The situation is actually worse than this since, in addition to the problem of change, it is likely that there were multiple, competing, definitions from the outset. One of the core issues around shared data is that the way that a thing is classified is actually dependent on the perspective of the person classifying it. If you consider a chocolate bar and you work in marketing, you think about the brand, the packaging, the pricing. If you work in logistics you care about the weight and dimension of the bar, for example how many fit in a box or on a palette. If you work in production you care about the formulation and manufacturing process associated with the bar as well as health and safety issues. The reason that there are often dozens of versions of master data is that these are actually all valid perspectives, so there is no single version that will necessarily satisfy everyone. Moreover, this also applies to customers: sales people, for example, may want to see sales to a corporate customer broken down by geography while product managers wanted to see customer information by product category. Thus, except in specific instances, the term single view of the customer is a misnomer. Master data issues can also occur because a business transaction causes a chain of activities, for example an internet sales order causing a dispatch to be made, as well as an invoice. The data associated with this single transaction may be held in different systems or modules within an ERP system, potentially with master data associated with the transaction held in different places. If, for example, the logistics department restructure the way they classify delivery addresses of customers, does this change seamlessly ripple through to the way that customers are classified in the sales or finance systems? Squeezing the balloon There have been several attempts to fix this single version of the truth issue. To a large extent, the ERP dream was sold on a basis of automating and unifying business processes, allowing management to easily make crossborder and cross-business unit performance comparisons. In practice most large enterprises have numerous separate ERP implementations (in some cases hundreds) in which much data is still dealt with in different ways. Some of these initiatives have brought significant rationalisation of applications, but as long as there is just one other system beyond the boundary of a solitary ERP instance then data issues remain. One multinational we spoke to owns up to having over 600 separate applications beyond its (multiple) SAP instances, for example. The single instance ERP consolidation project is a pipedream for most large organisations; such projects are so complex that they are often scheduled to last five or more years, by which time business requirements have moved on. Even after such a project is finished, there will still be independent applications left, since no ERP system covers every aspect of an organisation s needs. Data warehouses were, in part, an attempt to address the issue of unified, consistent information. Yet how many large organisations have a single, consistent data warehouse across the enterprise? In reality, issues of scalability, organisational complexity and inflexibility of technology have resulted in organisations having a smorgasbord of data warehouses. Their data is frequently outdated and distrusted by the business, which have sponsored shadow systems at a local level to give them what they really need. In all this, data lies at the heart of the problem. Data is duplicated by different departments, countries, functions and systems, each with their own specific needs and reasons. Isolated and sustained initiatives can result in harmony at some level: a project to standardise international product codes, or unique vendor ids. Yet such initiatives can be frustrated by change imposed from outside. Companies have to deal with data that is not just internally generated, but in many cases is supplied from third parties (suppliers, customers, government agencies and so on). Moreover, many initiatives of this type have a particular focus and business sponsor, and the person running that project understandably does not wish to be delayed by having to co-operate with other departments, since such work for the greater good may conflict with project priorities. In a few cases there are initiatives to try and tackle such issues on an industry basis. UCCnet (the Uniform Code Council standards body for product information in Supply Chains) is one example in the case of product information, but such efforts are by no means complete or universal. Moreover, even if an organisation has perfectly consistent, clean procurement data (say), what happens if it takes over or merges with another company that does not? For large multinationals such acquisitions can be a quarterly or even monthly event, yet merging the data and systems that result can take years. For all these reasons, enterprise data architects can feel as if they are hamsters running on a wheel that just keeps spinning faster.

5 So what exactly is master data? page 3 Data governance In recent years it has been realised that ERP and data warehouses are insufficient to really tackle the problem of inconsistent, inaccurate and unreliable data (whether for use in a call centre, say, or for business performance analysis). In particular there is a growing awareness that the processes that create and update corporate data need to be addressed if the data dragon is ever to be slain. This involves understanding, documenting and controlling the business rules that surround the creation of new business classifications (such as a new customer code, a new product line or brand, an updated hierarchy of engineering assets or organisational structure). The scope of data governance needs to extend to the semantics used in systems, to ensure consistency in the way that data is handled. vendors have swiftly moved to claim that they always had MDM capabilities. In some cases they are right, but we have also seen the rise of a new breed of technologies aimed specifically at the problem, rather than the mere rebranding of something that already existed. How can we make sense of these different technologies? Many organisations have attempted to address the governance of business processes and data, but few have genuinely succeeded. Documenting business models in some sort of data dictionary, or even on PowerPoint, is a useful start and can have many benefits but it is, essentially, a passive activity and gets quickly outdated. What is needed is for such descriptions of processes to be intimately linked with the systems that they involve. Data governance is, of course, an umbrella term that is much broader than master data per se. For example, it encompasses archiving policy (when and to which media do we archive data and how long do we have to retain it for), and compliance with data protection laws and security policies as well as the quality and accessibility of data. Recent government initiatives (such as Sarbanes-Oxley and the forthcoming EuroSox) have given impetus to such work, as has our increasingly litigious society, mandating the retention of data for evidentiary purposes. So, the governance of master data is only a part of the picture, which we will discuss in due course. Master data management So, if master data is essentially shared business data, then what is master data management exactly? It is a combination of processes and technology that help us manage such data better. This includes: processes: data stewards/governance groups, business rules and workflow (the last two of which are enabled by relevant technologies). technology: master data repositories, data integration technologies, data quality tools and so on. The master data management (MDM) market has grown up, at least using that term, since about Of course prior to this there were systems associated with managing master data, but such systems were often custom built (processes using spreadsheets and for example), or called something else (customer hubs, data dictionaries and so forth). MDM is now generally taken to be the term that encompasses all of these approaches regardless of the domain (customers, products, locations and so on) being addressed. As the term MDM has become recognised and industry analysts and media have started to discuss the space, Figure 1: A complete MDM solution A complete MDM solution would encompass the elements shown in Figure 1. Let us peek a little deeper into each of these boxes. Master data governance Master data governance is shown separately on the diagram (as Governance ) since this is not a technology it is an organisational issue, with people and processes rather than a piece of software. Some of the technology, such as workflow, may help with these processes, but governance is a business, not a technical, issue. In order to take control of their information, some companies are creating data governance units that are assigned responsibility for resolving data conflicts that cross organisational boundaries. A data steward might work in finance but be responsible for the procurement hierarchy used by manufacturing; a similar data steward working in marketing may have responsibility for customer classification, even though the same customer data is used by the logistics unit, and also by customer support in operations. Such units are responsible for the master data of an organisation, and these responsibilities need to be documented and managed. It is fair to say that, without the involvement of business staff around data governance issues such as these, a master data management (MDM) initiative that is instead driven by an internal IT department is doomed to failure. A key to improving the quality of master data is to resolve interdisciplinary differences e.g. when marketing and engineering classify the same thing differently. An IT person caught in the crossfire between two business departments probably does not have the knowledge, and certainly not the authority, to take charge and make a decision on which will be the golden copy. Hence data governance cannot be an afterthought in an MDM initiative. It needs to be at the heart of it technology alone cannot solve master data issues.

6 So what exactly is master data? page 4 Governance support The creation, update and retirement of master data definitions come down to business processes. For example a marketing manager may be the notional owner of the product hierarchy used in a company (for example a chocolate bar is a type of snack food, which itself is a type of foodstuff ). However to change a definition, say to add a new product as a chocolate bar to the international product hierarchy is something which the marketing manager may need to validate with regional colleagues, and may need sign off from the VP of marketing. In a more complex case there may even need to be several draft proposals of a new hierarchy, which need to be circulated and reviewed before being signed off and published. A complete MDM tool allows some automation of this process, perhaps by generating web-based screens that show the sequence of drafts and authorisation steps necessary for this procedure. It should also keep track of the progress of master data as it moves through these processes. An alternative approach would be to link up to a third party workflow tool and implement such workflow automation outside the direct control of an MDM technology. Ideally an MDM product should be able to offer a complete audit trail of the various stages of master data creation and update, and allow for the possibility of tracking back versions of the master data and business rules throughout their lifecycle. Some tools can do this, but many cannot, so if this kind of facility is important to you then you need to check carefully when evaluating as to whether the software meets your needs. Business rules The distinction between the storage of what we have termed business rules and data rules is important. The rules that drive the update of master data can themselves be stored in a repository and may include: rules around business processes derived data business hierarchies. To continue the same example, a rule that says VP x in marketing has sign-off on a new version of a marketing hierarchy, but marketing managers y and z can create draft hierarchies for peer review and approval, is an example of a business rule that may be stored. Derived data contains definitions of data that may not exist in operational systems but may be the consolidation of such data, for example global net profit may draw upon data from many systems and have quite complex rules describing how net profit is calculated. Some MDM repositories can deal with such business rules, others cannot. Business hierarchies are familiar: asset hierarchies, breakdowns of organisational units, groupings of marketable products into categories and so on. Some vendors have much more sophisticated hierarchy management abilities that others. For example, you should bear in mind that not all business hierarchies are simple affairs such as a country has many states, each of which has many cities. In some cases hierarchies can have varying levels e.g. bill of materials is a well-known example of this. In other situations you may have unbalanced hierarchies, which need to be mapped together. Some MDM products can deal well with such complex cases, others cannot. You need to ensure that the complexities of your own business are dealt with by the tools you are considering. Some vendors supply pre-built data and business models with their products; for example this is often the case with product information management products, and may also be true of customer data integration (CDI) vendors. In such cases you need to balance the advantage in time-saving that a pre-built model may bring with the possible problems that can occur if the vendor model does not exactly mirror your own business processes. In such cases how easy is it to modify the vendor model in a controlled way, if at all? Data rules A different type of repository may be used to store rules about data, which may include things such as the following. Validation rules (such as x must be numeric and <100 or must be a valid US state ). Dependency rules (for example, if oil well type is exploration then the fields can be dry or success ; if the well type is production then the status can be producing or abandoned ). Matching rules (to identify potentially duplicate data). Internationalisation of names or attributes (for instance, William and Willem, Bleu, Blau and Blue). Statistics about data quality. Data from operational systems, even corporate ERP systems, is by no means guaranteed to be accurate, despite every attempt at validation at source. Data should be accurate, correct, current, complete and relevant, but this is not always the case. A class of technologies (data quality tools) exist which are adept at helping to spot potential data quality issues. These technologies profile data in order to understand potential data quality challenges, highlighting potential issues. They may allow the creation of rules against which data should conform, for example customer numbers in the US should be numeric and start with a 7, and can then apply such rules to samples of operational data to highlight mismatches. Other examples might be validating postal code data or range checking. Some technologies allow data to be compared across systems to spot potential matches or mismatches, for example is A. Hayler in one system perhaps the same as Andy Hayler in another, an A. D. Hayler in yet another? Some tools allow much more elaborate links to be spotted such as dependencies between fields in different systems (does an order number always have a customer number associated with it?) as well as providing useful statistics about data that allow data quality to be measured in an ongoing fashion.

7 So what exactly is master data? page 5 Some of these data quality elements are included in certain MDM technologies as features, while others have such features bundled up and sold as stand-alone data quality products (possibly sourced from third parties). Some MDM technologies store just data rules, some store master data, some store business rules. It is not within the scope of this document to discuss vendor technologies, but this distinction should help you understand the scope of a vendor s MDM solution. By understanding the distinction between business rules, data rules, and the data itself, you can decide which technology is best suited to your needs. Data provision This aspect of functionality within an MDM application covers how master data is to be accessed by people. This includes: reporting search browsing security (different roles having different level of access) publication for open access (for instance, XML). Some MDM vendors provide more comprehensive data provision support than others. For example, if a vendor has a persistent master data store, then this may or may not have an open interface to the data such as an ODBC interface for reporting, as opposed to allowing access only via proprietary tools. Master data storage At the heart of many MDM projects is a repository or database in which golden copy master data is held, such as the list of international customer accounts, the master product and asset hierarchies and so forth. Some MDM repositories go further and can keep track of partial or incomplete master data that is gathered from other systems before being reviewed and approved, keeping an audit trail of the stages involved as the conflicting source master data makes its transformation into a new golden copy. This distinction is useful in understanding why a data warehouse is not a true MDM repository, and vice versa. A data warehouse, because its purpose is to produce reliable information, should have only clean consolidated data stored within it (there may also be staging areas and operational data stores that feed the warehouse with raw data). Yet an MDM repository should be able to keep track of such incomplete data and track it through the stages of it becoming golden, ideally retaining an audit trail of the steps involved. We reiterate that master data goes beyond single domains such as customer and product. When constructing a model of your own master data, you may not wish to start from scratch. Some vendors provide pre-built models of their own that are embedded within their applications. You may want to take advantage of work on how to do data modelling that is already in the public domain, such as ISO 15926, which actually started in the manufacturing domain but can be applied to virtually any area and may be helpful in modelling master data categories. The MDM repository may itself then provide feeds to other systems, for example to a data warehouse, or perhaps via an API to an enterprise service bus (perhaps through an SOA service), which may in turn supply other systems. Some MDM vendors have repositories which are able to deal with business rules as described, some store an actual copy of master data, and some merely have a registry which points to where master data is stored elsewhere in the enterprise. We will return to this point later. Others focus on what we call data rules. Not all MDM repositories are created equal. Some have elaborate support for time-variance, audit trails to keep track of changes, identity management, security rules and much more. Others may be more limited, and it is important that you check that your needs are met by the vendor repository that you are considering.

8 Data movement & synchronisation page 6 There are two aspects to data movement: getting the data into the MDM system in the first place and then using it once it is there. In the first instance, ETL (extract, transform and load) tools would be the normal method of introducing master data and, as these technologies are well known, we will not discuss them further here. However, we would point out that it is sensible to treat ETL as a separate technology layer from MDM in case you opt to change providers at any time. For example, we know one company that initially designed its own MDM system, then moved to a packaged MDM solution and then moved again to a different MDM supplier. Fortunately, they had invested in an ETL platform separately and were able to reuse the work that they had done with this tool. A similar argument might also apply to data quality tools. On the other side of the coin, once you manage to construct your golden copy master data then you need to be able to link this up with other systems. At the least, such data needs to be provided as a feed, whether via an SOA service, an API or a batch file. Some technologies go much further and allow near real-time synchronisation with other systems, for example aligning customer records across multiple operational environments. As well as messaging technology, some vendors go as far as to store the golden copy of, say, customer account records, in a database hub, which in turn can serve up this data to other systems. Some vendors have pre-built connectors to leading EAI tools, while others may rely on proprietary synchronisation technology. Such approaches need to be able to deal with the full set of master data related to a business change e.g. a system about customers still needs to be able to handle the change of master data related to customer, such as organisation. As noted earlier, a business transaction can cause a chain of master data impacts that may go beyond the initial system in which the transaction is registered. Architectural choices There are a number of distinct use cases we observe in MDM projects today that are quite different in nature. These can be broadly split into: collaborative MDM analytical MDM operational MDM. (note that in operational MDM you may not need to store the master data persistently; instead storing identifiers only may be sufficient to support interfacing master data from operational systems via looking up cross-references) Collaborative MDM concentrates on use cases, which focus on the creation and update of critical business rules and master data and the processes which surround that. The example earlier around the classification of chocolate bars would fit into this category but there are many others. A non-marketing example is one we encountered at a global bank. In this, a whole series of authority levels exist with regards to consolidating the quarterly results of the various subsidiaries at a corporate level. In this case the definition of terms such as gross margin were standardised at the global level. Some managers are authorised to make some kinds of accounting adjustments but not others; some people can only see information at a certain level of detail and so on. In a similar case, the handling of time was very important: the draft results that will go off to the regulator and the capital markets are extremely sensitive for a few days and need tight security procedures, yet days later are public knowledge. An extension of the collaborative approach is where data rules are covered as well as business rules. In such an example the rules associated with data, such as validation rules for a product code, are also handled within the project. Many Product Information Management (PIM) products can be thought of as fitting into this category, where there is significant human collaboration and also detailed data rules. Analytical MDM concentrates on trying to improve the quality of master data at the enterprise level so that better business decisions can be taken. For example, in a large enterprise it is frequently very difficult to know how profitable an enterprise account is due to the number of touch points and definitions of that company within internal systems, as well as lack of clarity around cost allocation. One famous investment bank that we spoke to admitted that it had no idea about how much business was done globally with a large customer, or how profitable that business really was. Analytical MDM projects do not focus on fixing data at the operational level, but are concerned about improving the quality and consistency of master data so that enterprisewide views can be taken to support decision making. Figure 2: Coverage of different MDM styles

9 Data movement & synchronisation page 7 In the case of both analytical and operational applications of MDM, the use cases are about establishing common business rules that define master data, and identifying where this data is in the organisation. In this approach it is assumed that there will, for all practical purposes, always be multiple copies of certain types of master data physically present; for example in multiple ERP, CRM or supply chain systems. These applications do not attempt to actually convert data in those operational systems but seek to map or stitch them together. For example there may be a: global customer account in the MDM repository, but also records such as: ERP customer id CRM customer ID Supply chain customer id... along with the links to join these together: Global customer ERP customer id CRM customer id Supply chain customer id All sorts of additional information about this single customer entity may be stored in the master data repository, but as a minimum there are links back to how this same customer is identified in the various operational systems. In this way when, for example, sales to a certain customer are considered at the global level, the MDM repository knows where to go to find this consolidated information. This mapping could be shared with a data warehouse application to allow consolidated analysis of account data at the global level, for example. However, operational MDM is more ambitious than this, going beyond merely knowing where the various versions of master data reside. In this type of use case, the MDM technology actually stores the global customer id in a central hub, and drives that id out into other operational systems via EAI (enterprise application integration) or other synchronisation technologies. When a new customer account is created, the operational systems must synchronise with the master data hub to ensure that there is no duplicate customer record created. Some technologies automate the application of data rules within this. For example, if a clerk tries to create a new customer record in an ERP system, then a matching rule is automatically applied to check whether there may already be a customer record for that customer (perhaps by matching the billing address). If potential duplicates are detected, then the technology will offer an existing suggested match to the clerk before creating a new customer id. This sort of operational MDM is technically quite demanding. Not only is it operating in near real-time, there is no matching technology that ensures 100% success, so there needs to be processes set up to go back and deal manually with records which are misclassified. Typically such operational uses of MDM are to be seen in customer hubs. In theory, such approaches could extend one day to all types of master data domains, but this is a non-trivial task given the sheer volume of data that is stored using existing identifiers in operational systems. The effort to actually convert such data across an enterprise is not something to be considered lightly. However, the benefits of synchronising important classes of master data such as suppliers or customers may justify significant efforts in this area.

10 Where to start? page 8 MDM is a journey rather than a destination. An MDM project that finishes on a certain date will at best have partial success, since organisations need to treat master data not as a one-off problem to be fixed. Rather, companies need to consider the whole life cycle of the creation, updating and retirement of master data, just as they would do for physical assets like trucks. Also, it is clear that you are tackling a moving target, with new master data being created and updated every day. As with any project, business justification is important. The scope of the area that you tackle will determine how difficult or otherwise it is to justify an MDM initiative. The business issues associated with master data will vary by industry and by company, but should not be hard to articulate. Examples are: ineffective marketing caused by poor customer data, costly interfaces due to needless duplication of data about assets, inability to gain a single view of global accounts. All these have very real business costs and so solving them should result in hard dollar return on investment. In terms of where to start, then it is logical to begin with an area that has a high priority for your business, one with clear payback if the master data problem is fixed. This might be around customer or product, but may be around a whole array of other master data types. Some companies we have seen have started their first MDM initiative around improving the quality of employee data, or chart of accounts, or assets. Bear in mind that if you pick a particular master data domain then you will have to deal with the fact that this data is used across a range of applications, as shown in Figure 3. It is important to balance the needs of a strategic approach to MDM with the practicalities of solving genuine business issues. For example, if you start with a particular small but urgent area (say, dealing with customer data to improve customer churn) it is important to educate the business sponsors that this project is not going to solve all MDM issues in the company, and that further investment will be needed. On the other hand, it is impractical to deal with all master data at once. Consequently the mantra should be think big, but start small. Tackling a specific, high priority area (for example, reducing customer returns due to poor delivery and order information, or improving your product catalogue) is more realistic than boiling the ocean by trying to tackle all classes of master data at once. Yet care should be taken to ensure that the approach taken in this specific case is something that can be replicated on a broader basis. For example, if you implement a technology that can only deal with one class of master data (a customer or product hub, say) then when you try and deal with financial asset data you may need a separate technology. Moreover, some technologies have far better scalability than others. An MDM repository that is good at dealing with a broad range of master data may struggle to deal with a high volume of updates being thrown at it in near real-time. Therefore it is important that you consider the likely range and scale of what you are trying to achieve in your MDM project. Figure 3: Domain/application matrix Master data about product, for example, will be stored in multiple applications, and needs to be linked in some way. There is also the dimension of geography or organisation to be considered. For a multinational, it is likely that a suite of applications is deployed in each country, or at least in a regional cluster of countries. In Figure 4, there are three local stacks of applications that connect to a global level. Customer data is used in all four, so the MDM project needs to address the standardisation of data across each geography, as well as within each application suite. You will also need to consider just how ambitious are your goals. At one end of the scale, creating a passive registry of where master data is sourced is non-invasive and useful, but does not attempt to actually improve the business rules or master data itself. By contrast, a hub aiming to be the system of record for an enterprise is much more ambitious but may bring greater benefits. Some vendors paint a vision of a world where such hubs are not only the system of record but become the place where new master data is created, removing core functionality from traditional transaction systems. There are very few (if any) examples of this in real projects, due to the sheer scale of existing investment. It is much more likely, in our view, that even operational hub applications will co-exist with existing operational applications such as ERP systems. Figure 4: Global and local level stacks

11 Where to start? page 9 Even if your company operates within a single geography, you need to remember that you also have to deal with data outside of your own organisation that is authored elsewhere: for example D&B company data or credit data, consumer data from companies like Acxiom or Experian. The management of how updates happen to this external data is part of MDM. Some vendors have specific pre-built connectors to such data. Best practice suggests that it is best to plan an architecture that can cope with all geographic/organisational levels but to start with one; that is, get the MDM application running happily in one country and then replicate this elsewhere, connecting the enterprise steadily rather than trying to do everything via a big bang implementation. Such an approach takes time but has much lower risk of failure than an all-or-nothing enterprise-wide project. An iterative approach that will eventually join up the current islands of master data also needs to consider how to deal with multiple master data domains. If you use one technology to deal with customer data and another to deal with product data you will soon discover that there are, in fact, many other types of master data as well (asset, location, employee ). You want to avoid replacing current stove-piped applications with stove-piped master data repositories. This point has often been missed in early MDM projects where, in the haste to solve a particular problem, approaches have been used that cannot scale properly to other master data categories. After initial resistance to this message by vendors (especially those who specialised in one type of master data) we are seeing encouraging recent signs of the penny having dropped. Most vendors (at least in their roadmaps) are starting to acknowledge that master data goes beyond just product and customer. evolutionary approach has obvious attractions. The downside is that you need some external technology such as EAI to ensure that the registry and the source systems to which it refers are kept synchronised. Collaborative MDM projects will usually require a master data repository which goes beyond a registry in that it can store relevant business rules ( who can sign off this document ) as well as maintaining a golden copy of the master data itself. In such an architecture the master data itself will likely be authored elsewhere (in source systems), but the MDM repository may feed back its golden copy master data to operational or other systems in order to gradually improve the state of the master data over time. The repository may also feed golden copy master data into a corporate data warehouse, so is the natural choice for analytic MDM projects. The most extreme architecture is where master data hubs aim to replace the creation and maintenance of master data in operational systems entirely. This is an ambitious and relatively invasive approach whereby, for example, a customer hub becomes the place where customer data is maintained and, via a messaging layer, feeds this into other systems such as ERP applications. Clearly this approach needs to be carefully considered in terms of interfaces with existing systems, which would otherwise expect to be considered the definitive source of master data. Moreover, such a design will usually be deployed in near real-time and so may be quite demanding in terms of operational performance. In reality, even hub deployments that aim to become the system of record for master data (such as customer) will generally end up co-existing with pre-existing sources of data such as ERP systems. Implementation choices It is not within the scope of this document to discuss MDM technology in any depth; that will be the subject of separate research material. However it is worth being aware of the quite different implementation styles that vendors can offer. The nature of the type of project that you are tackling (whether it is collaborative, analytical or operational) will help determine the best fit. At one end of the scale are what some have termed registries, essentially latter-day data dictionaries that record where master data is created and maintained via pointers, but do not attempt to directly store master data, or to change the processes around it. This approach is non-invasive and may be a useful way of documenting the state of master data. This may also aid data quality initiatives by providing a useful address book of source master data. However, such an approach does not lend itself to projects where you are trying to modify, at a fundamental level, the way in which master data itself is maintained. That said, the use of a simple registry does have the very significant benefit that it can be implemented very quickly. A registry-style implementation can be up and running in a matter of weeks, while a full scale hub-based deployment (see below) is likely to take much longer. A registry can also be a useful stepping stone towards a hub. For example, we know of companies that have started with a registry but, as new master data has been defined, that has been added to the registry rather than source systems. This sort of An important decision that you will need to make is the trade off between MDM platforms, which can handle all types of master data domains, and specialist applications which focus just on one area, such as a PIM or CDI solution. Clearly it is likely that a solution that deals just with one type of data, and may come with a pre-built data model and set of business rules, may be more effective in its context or quicker to implement than a generic application. On the other hand such specialist applications may not be able to adapt to other master data domains easily, or at all. Hence there is an architectural trade-off to be considered, based on your particular business needs.

12 Which vendor(s)? page 10 At present no vendor, even the industry giants, offers a complete solution that tackles all types of master data and all the types of need that your organisation may have (for example, in throughput volume, or perhaps in the complexity of hierarchies it can handle) and provides comprehensive functionality in the areas that this report has discussed, all from a single platform. However many vendors of all sizes are investing heavily in MDM technology so, over time, you can expect software to mature and become more stable and grow in breadth of functionality. There has already been significant M&A activity as larger vendors seek to buy in and integrate innovative technology that they did not have themselves, and this is likely to continue as customers become more demanding. You should ask vendors about their medium term roadmap to see whether their vision is aligned with your needs. You should also ask vendors about their partnering strategy. For example does a company that is strong at storing business rules also have a data quality component? If not, does it have an established partnership with a vendor that does? Given that no one vendor offers a complete solution right now (though they are promised), we see significant scope for partnerships between vendors that are strong in one area but not another. Conclusions In this overview report we have set out MDM terminology and can draw a number of conclusions based on early customer experiences. Data governance is vital to the success of an MDM project. There is an important distinction between business rules, data rules and data itself. Collaborative, analytical and operational MDM projects have quite different profiles. An architected approach needs to consider master data beyond customer and product. Links between different geographies and organisations need to be planned for. Think big, start small is a sensible approach on the MDM journey. No one vendor currently offers a total solution. Above all, in a market that barely existed in 2003 and which now has literally dozens of vendors claiming MDM capability, caveat emptor is the rule when evaluating software vendor claims. A slick PowerPoint presentation may disguise a software product that, indeed, itself runs best in PowerPoint form rather than on any other operating system. Challenge vendors to talk to reference customers and ensure that the capabilities that vendors claim are actually things that are relevant to your particular needs.

13 Bloor Research overview About the author Bloor Research has spent the last decade developing what is recognised as Europe s leading independent IT research organisation. With its core research activities underpinning a range of services, from research and consulting to events and publishing, Bloor Research is committed to turning knowledge into client value across all of its products and engagements. Our objectives are: Save clients time by providing comparison and analysis that is clear and succinct. Update clients expertise, enabling them to have a clear understanding of IT issues and facts and validate existing technology strategies. Bring an independent perspective, minimising the inherent risks of product selection and decision-making. Communicate our visionary perspective of the future of IT. Founded in 1989, Bloor Research is one of the world s leading IT research, analysis and consultancy organisations distributing research and analysis to IT user and vendor organisations throughout the world via online subscriptions, tailored research services and consultancy projects. Philip Howard Research Director - Data Philip started in the computer industry way back in 1973 and has variously worked as a systems analyst, programmer and salesperson, as well as in marketing and product management, for a variety of companies including GEC Marconi, GPT, Philips Data Systems, Raytheon and NCR. His practice area encompasses anything to do with data and content and he has five further analysts working with him in this area. While maintaining an overview of the whole space Philip himself specialises in databases, data management, data integration, data quality, data federation, master data management, data governance and data warehousing. He also has an interest in event stream/complex event processing. In addition to the numerous reports Philip has written on behalf of Bloor Research, Philip also contributes regularly to www. IT-Director.com and com and was previously the editor of both Application Development News and Operating System News on behalf of Cambridge Market Intelligence (CMI). He has also contributed to various magazines and published a number of reports published by companies such as CMI and The Financial Times. Andy Hayler Independent Consultant Andy is an established software industry authority, an independent strategy consultant advising corporations, venture capital firms and software companies. He is the founder of Kalido, which under his leadership was the fastest growing business intelligence vendor in the world in Kalido was recognised as an innovator in data warehousing, and then launched arguably the first true master data management product, a market which at the time did not exist but is now a well recognised and fast growing industry. Andy was the only European named in Red Herring s Top 10 Innovators of He was a pioneer in blogging with his award winning Andy On Enterprise Software blog. Andy started his career with Esso, working in a number of technology roles before moving to Shell. He was Technology Planning Manager of Shell UK, then Principal Technology Consultant for Shell International. He later established a global information management consultancy, which under his leadership grew to 300 staff.

14 Copyright & disclaimer This document is copyright 2008 Bloor Research. No part of this publication may be reproduced by any method whatsoever without the prior consent of Bloor Research. Due to the nature of this material, numerous hardware and software products have been mentioned by name. In the majority, if not all, of the cases, these product names are claimed as trademarks by the companies that manufacture the products. It is not Bloor Research s intent to claim these names or trademarks as our own. Likewise, company logos, graphics or screen shots have been reproduced with the consent of the owner and are subject to that owner s copyright. Whilst every care has been taken in the preparation of this document to ensure that the information is correct, the publishers cannot accept responsibility for any errors or omissions.

15 Suite 4, Town Hall, 86 Watling Street East TOWCESTER, Northamptonshire, NN12 6BS, United Kingdom Tel: +44 (0) Fax: +44 (0) Web: