Navigating the cloud of Big Data hype: lessons to live by. January 2015

Transcription

1 Navigating the cloud of Big Data hype: lessons to live by January 2015

2 INFORMATION DRIVES SOUND ANALYSIS, INSIGHT, AND ACTION. Navigating the cloud of Big Data hype: lessons to live by Inform - Exit the data/it dilemma Transform - Lessons Outperform - Fact-based management Inform Exit the data/it dilemma When you look up cloud in a thesaurus, it s interesting what turns up. Mist, fog and haze appear as common nouns. Confuse, darken and obscure appear on the verb list. These are apt metaphors for the all too common discussions between business teams and the data analysts and technology teams that support them. The analyst / tech team - I think we should put everything on the cloud. I recommend we go down the jamjim route. The backend is great. Really scalable. I m worried about latency. The business - Why should everything be on the cloud? I just want to know what sales figures are likely to be. We ve spent $1M but we still don t have data we can trust. That analysis is clearly wrong. I don t want to be on the news about some data security issue Why can t we have something simple like Smoogle? Developments in data collection, storage, processing, and platforms continue apace. More solutions on the market make the data management and technology landscape confusing and difficult to navigate for both IT departments and the business groups they serve. As the demand for, and availability of, solutions that harness the power of data analytics increase, this tension between business and technology will only continue to heighten. What is needed is to strike a balance between technical capabilities and genuine business needs. It s well known that hyped technology rarely lives up to its lofty expectations, and all too often a particular technology is selected based upon criteria far removed from actual business objectives. Business groups need to clearly define and communicate the return they expect while IT must set realistic goals for delivery and functionality. Hype and a lack of a common language are stumbling blocks that businesses and IT departments face when trying to maximise the utility of data and technology to support business success. Both groups are keen to achieve the benefits of leveraging data and technology, but struggle to make it happen. Organisations producing information on a large scale are often tempted by the promise of new data analysis technologies and their ability to improve operational performance. New software 1

3 capabilities appeal to IT, while business teams desire tools that will create long-term value and growth. When different teams are speaking different languages and viewing problems from different perspectives, where is the middle ground where transformation can take place? Transform Optimity Advisors has identified six core lessons for leaders to take on board in order to help them maintain or build competitive advantage in the market by using data and technology to build customer loyalty, locate areas for improvement and make better decisions earlier. These are based on our 25 years of experience working with global public and private organisations to solve complex technology and data problems in a broad range of industries. Lesson 1: Take control of information management insight engine Knowing your sales volume, accounts receivable and customer satisfaction are just a few of the basic metrics that are available and used by businesses. Frequently however analytical tasks which are perceived to be more complex, such as assessing the income risk of different market segments, or projecting future demand based on demographics and other factors, are frequently outsourced to specialist providers who provide a black box solution algorithms and analysis which are not transparent. Whilst this outsourcing strategy may have some short term value, an organisation which does not own, and have control of the way in which it understands how it assesses risk and allocates time and resources, or controls the pace and nature of improvement, leaves itself vulnerable. This could be because information providers will be selling the same or similar approaches or may even go out of business. There are many examples of companies now in the world of media, retail, and transportation who have seen the opportunity afforded by analyses which can help businesses identify not only how they have performed in the past, or currently, but also how it looks as though they will perform in the future. In the board room this means changing the conversation from one focussed on what was achieved months before and setting up working groups to investigate what should be done, to one focussed on having facts at the fingertips concerning under or over performing lead indicators and risk areas - more like a weather forecast. This type of analytical approach is frequently called predictive analytics and the insights it creates we have likened to an engine in the way that it keeps running and needs to be well maintained. In order to create such an engine business groups and data/ IT teams need to have a shared view of what this insight engine could deliver, have the skills available to deliver these insights, and then create an agreed route map. There are also cultural challenges to overcome to ensure that the analyses and tools created promote the right type of risk based discussion in the business as weather forecasts are not always correct, but continue to improve through greater insights into how weather patterns work. 2

4 Lesson 2: Work hard to get the data in working condition More often than not, initial investigation of data reveals it to be less of a silver bullet than anticipated. Data might be delivered too late to be actionable, or key elements may be missing altogether. Data may be disorganised or contain strange values. These challenges are particularly common when surfacing data that has gone unused in storage for months or years. We have two options when faced with dirty data: to get frustrated or to take steps to resolve the issues. Organisations that reap the benefits of data are the ones that commit to working with facts and establish a governance program with new data-centric roles such as Chief Data Officers and data managers to negotiate across the business: what the data means, how it is defined and how it is used. These roles cut across the business, including finance, sales and marketing, and have the focus to ensure that the single version of the trusted facts to operate the business are available. Lesson 3: Make sure analysis and systems are flexible and responsive to changing circumstances Whilst getting the data in working condition is the foundation to a successful fact based business, in life and in business the only constant is change. Customer expectations and needs change, new data are collected, businesses merge or are bought, supply chains alter, behavioral insights improve, and data processing power rises inexorably. Relying solely on tried and tested analyses and IT in this environment will inevitably limit the success of any business. Planning for improvement by investing in data architecture and innovations in analysis allows businesses to stay ahead of the game by experimenting and quickly delivering valuable and surprising insights. In terms of IT, regular feedback by end users, contracts focussing on delivering outcomes in staged cycles as opposed to Big Design Up Front approaches will also build in the flexibility needed and help ensure that the business is in the lead, rather than falling behind. For business groups and data/it this frequently means developing a more iterative and flexible approach to working with business transformation and solution providers, focussing on a long term vision and negotiating business benefits incrementally, as opposed necessarily to creating one off big specifications. Lesson 4: Face the facts about customers For many years, market surveys, satisfaction forms and focus groups were seen as the way to understand customers. The reliance was on selfreported behaviours and immediate reactions as opposed to facts concerning what people actually did. Now, customer insights are captured digitally at a whole new level of volume and granularity. Customer sentiment and behaviour have become the lifeblood of business strategy by linking data at the individual level and incorporating new unstructured data sources from social media to create new insights. One example of building and incorporating a wide swatch of customer metrics is the new 3

5 way that concert tickets are sold. It used to be that an orchestra would focus their customer insight on audience demographics and frequency of attendance. Now, companies examine a whole new set of insights - how and when tickets were purchased, where people arrive from, what food they ate before the performance, how they engaged with the performance through media. Having access to this type of information can help inform how and where to maximise customer loyalty, develop evangelists and gain new customers through targeted nudges which may influence the way that people feel about a business, product or service. Getting hold of these facts, and managing them responsibly and consistently is now possible and those businesses that unlock these insights are seeing rewards in customer loyalty and market share. For business groups and data/it this means being more creative about the data sources that could be used/become available, negotiating access, and then working through the data governance and security considerations. This provides a platform to undertake the analysis to understand more about the existing customer base, how customer loyalty could be expanded and how potentially new customers could be gained. Lesson 5: Invest in data architecture Some believe that only analysts should have access to data because it might be misinterpreted or misused by business groups. This says less about the data and more about communication. Understanding and communicating risk can be challenging, but many organisation resolve the issue by identifying and communicating key metrics using well-designed visualisations that are easy to interpret. This is an educational and design journey that requires on-going discussion between analysts and business leaders. Businesses that successfully leverage data, focus on the data architecture. These are the policies, rules or standards that govern what data is collected, how it is stored, arranged, integrated, and put to use by stakeholders inside and outside of the organisation. A key element of a quality data architecture concerns the automation and delivery of key buckets of information to help business groups understand performance and goals. Classically these information buckets cover reputation, quality, revenue, cost, profit, efficiency, pipeline and contractual/legal compliance. Business rules can be created as part of the architecture, which trigger different warning signals. This approach means that business users can identify areas for extra focus at a glance by incorporating predictive analytics and established business rules into the data architecture program. A further benefit of this automation approach is that valuable data scientist time can be freed up from performing repetitive manual tasks producing monthly board packs, by automating data loading, cleaning and processing. For business groups and data/it, this means committing to developing a data architecture that supports business success and is fit for the future, as well as establishing a route map to deliver this architecture. This will inevitably mean working through a range of people and process issues, alongside the data and technology challenges. 4

6 Lesson 6: Moneyball your business This story, frequently recounted, focused on the Oakland Athletics baseball team, which had very limited resources compared with their competition in Major League Baseball. Yet, they managed to get to the playoffs in both 2002 and 2003 by rejecting perceived wisdom, creating new insights and applying these new data to help inform decisions and achieve their competitive objectives. To do this the Athletics put their trust in creating new hypotheses on what contributed to success, undertaking statistical analysis and then using these insights to shape their roster and trading policy. As a result they were able to pick up undervalued (and less expensive) players who went on to perform well. A similar approach has now been taken on by most, if not all, teams. Just imagine if the same approach could be used in other environments such as the most cost effective way to reduce risk for different patient groups, maximise sales, gain new leads or achieve learning outcomes. Some businesses have started working in this way. Some have automated standard reporting functions and created new intelligence skunkworks - groups of data scientists and programmers testing ways in which people react to different interventions, identifying new variables of real value and embedding them in the business on a day to day basis. activities deliver the greatest return on investment. If these analyses tell a different story than was expected, the insights should be taken on board by the business to influence which clients it targets, what service lines it operates and how it engages the market. Outperform Committing to fact-based management Many consistently high performing businesses create fact-based management with one version of the truth and intelligence skunkworks to deliver new insights. The journey is not easy, and requires a team effort involving different aspects of businesses that have traditionally worked in silos and don t speak the same language. Finding a way to overcome this language barrier and design a good route map incorporating not just data and technology, but also people and process issues will unlock business potential. For the business groups and data/it teams this means setting aside intuition and conventional wisdom and approaching the business with an open mind. Experiments can then be designed and undertaken - testing assumptions and hypotheses with the data in order to better understand what is driving business success, and what For more information, contact: Andrew Beale Partner e: andrew.beale@optimityadvisors.com t: +44 (0)

7 Matrix Knowledge formally joined the global consultancy group Optimity Advisors in September As its European arm, the newly combined business trades as Optimity Matrix to run the public policy arm of Optimity Advisors global operations. For more info go to: Optimity Matrix and Matrix Knowledge are trading names of TMKG Limited (registered in England and Wales under registration number ) and its subsidiaries: Matrix Decisions Limited (registered in England and Wales under registration number ); Matrix Insight Limited (registered in England and Wales under registration number ); Matrix Evidence Limited (registered in England and Wales under registration number ); Matrix Observations Limited (registered in England and Wales under registration number ); and Matrix Knowledge Group International Inc. (registered in Maryland, USA under registration number D ). Disclaimer: In keeping with our values of integrity and excellence, Optimity Matrix has taken reasonable professional care in the preparation of this document. Although Optimity Matrix has made reasonable efforts, we cannot guarantee absolute accuracy or completeness of information/ data submitted, nor do we accept responsibility for recommendations that may have been omitted due to particular or exceptional conditions and circumstances. Confidentiality: This document contains information which is proprietary to Optimity Matrix and may not be disclosed to third parties without prior agreement. Except where permitted under the provisions of confidentiality above, this document may not be reproduced, retained or stored beyond the period of validity, or transmitted in whole, or in part, without Optimity Matrix s prior, written permission. TMKG Ltd, 2015

8 London Brussels Washington DC New York Los Angeles Optimity Matrix 1st Floor Kemp House City Road London EC1V 2NP w: Follow us on Twitter: Join us on LinkedIn: