September 9 11, 2013 Anaheim, California Data Rich, Insight Poor: Un-confusing Your Confused Analytics Team

Transcription

1 September 9 11, 2013 Anaheim, California Data Rich, Insight Poor: Un-confusing Your Confused Analytics Team Manish A. Shah, Director of Strategy & Analytics, InterContinental Hotels Group

2 ASUG Trivia Question 1 The Analyst, the first book addressing modern analysis, was first published in what year? A B C D

3 ASUG Trivia Question 2 Which of these companies have not owned the tool Xcelsius? A. Business Objects B. Crystal Software C. Infommersion D. Becarra Solutions 3

4 ASUG Trivia Question 3 Business Objects first product launched in 1990 was called: A. The Gilligan B. The Skipper C. The Professor D. The Ginger 4

5 ASUG Trivia Question 4 The original due date to deliver this presentation to the ASUG Conference Planning Team was: A. June 22 B. July 15 C. July 29 D. Sept 9 5

6 6 My pet-peeve

7

8

9 Source: IHG PSP

10

11

12

13

14

15

16

17

18

19

20 Source: IHG PSP

21 Source: IHG PSP

22

23 My pet-peeve When a speaker is doing nothing more than advertising their product 23

24 Today s Conversation. Analytics Reform Some questions? Why do our analysts not have a single version of the truth for each metric? Why do our analysts spend all their time on data pulling and not thinking? Why can't I get insights of our data? Why does it take so long to secure and analyze our own data? 24

25 Today s Conversation. Analytics Reform a project in review: Metrics how do we solve for single truths on key metrics Moving our support team from being data pullers to thinkers (strategists) Data issues galore how and when to know to fix something Off-shoring when and how do we use it appropriately Organization alignment what org changes are needed to measure a ROI 25

26 26 Today s Conversation Optimizing Our Team First started with interviews and a survey

27 Original State Analyst Time Spent When we interviewed our various analytics groups we found this Original State Data Pulling / Defining Aggregation / Cleansing Reporting Analytics Insight Development Future State Vision Strategy Development 2 ISSUES

28 Key Findings Time Consuming Issues Our Analysts spent considerable time on potentially non-value added activities Top Issues Uncovered - Time: Analysts spent considerable time on senior leadership ad hoc activities fire drills Analysts spent considerable time pulling manipulating data for ad hocs Analysts spent considerable time answering questions for other analysts Analysts spent excessive time obtaining precise numbers for directional decisions Analysts spent excessive time on recurring reports

29 Key Findings Knowledge Gap Issues Analysts had knowledge gaps for frequently asked questions. Top Issues Uncovered - Knowledge: Analysts did not share data methodologies, short cuts, and best practices Analysts were not aligned on definitions and metric calculations Senior executives would have different contradicting metrics at leadership meetings

30 Key Findings Analysts Opinions We need to meet more often to express different views, discuss, debate and then move forward with solutions. There needs to be a repository for definitions, report documentation that would allow us to spend less time figuring it out each time. If we had an in-depth inventory of current reporting, we wouldn't have to spend so much time responding to ad hoc requests. We need an Audit Committee to ensure all metrics and reports follow similar truths We need FAQ documents with step-by-step guides and common methodologies There is no cross-communication now. I have my own distribution list that I use We need to get everyone on the same page, work together better and share knowledge and information. We have trouble getting data from the owners of the data. No one has time to help.

31 Key Findings Current Activities There were three top buckets of analyst activities identified in project interviews. 1. Annual Activities Goal / Budget Development 2. Monthly Activities Goal / Revenue / Leadership Updates Operational Activities 3. Ad Hoc Activities External Communication of Value Brand Campaign Analysis Helping Other Analysts Hotel Issue Resolution (The Fire Drills) Product Development Strategy Development When asked how much time they currently spend on analysis and thinking, analysts self-reported between 10% and 25% Best Practices Puts It At 70%

32 Key Findings - Time Spent on Activities Analysts spent considerable time data pulling and preparing presentations for ad hoc requests. Creating SAS Data Sets Data Quality/Clean-up Administrative Tasks Preparing Recommendations Report Design Report Generation Defining/Clarifying Requirements Preparing Presentations Ad hoc requests Data Analysis/Identifying Trends Data gathering/pulling 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% # of respondents who do this activity

33 Key Findings - Formal Training More than 60% of analysts have had little training since joining IHG and 80% of analysts had no prior training in the analytic tools that IHG uses. Good Training prior to IHG, none since 19% None, Minimal or Informal Training at IHG 43% 38% Some or Good Training at IHG

34 Key Findings Reports Overload There were 178 documented reports produced and regularly executed. ~

35 Key Findings Title Overload While there is little differentiation in tasks across jobs, there were 22 job descriptions. It is difficult to determine what the actual job responsibilities are from the below titles: Analyst Web Analytics Analyst Web Analytics Specialist Financial Analyst Financial Specialist Reporting Specialist Reporting Manager Analytics Analyst Senior Business Analyst Senior Analyst Senior Analyst Specialist Strategy Analytics Manager Statistical Analyst Senior Operations Research Analyst Operations Research Analyst Analytics Consultant Lead Analytics Consultant

36 So, does this all sound familiar? 36

37 Time for a PROJECT 37

38 Project Overview The project was conducted in four phases SVOT (Single Truth) Productivity Dashboard Organization BusinessObjects As Primary Solution

39 Phase 1 - Single Version of the Truth The first deliverable included the aggregation and agreement of the most used metrics. Steps Taken: Consolidated all metrics Developed definitions and documented calculations Conducted iterative reviews until all key metrics were agreed to Outcome: Identified owner of each metric Communicated definition, calculation, source system / person Core team agreed to comply with communicated metric definitions 249 Metrics Brand-site Conversion Average Daily Rate Booking Sources Tagging Parameters Visits Sessions

40 Phase 1 SVOT Document The SVOT deliverable defines the most commonly used universes (data sets) along with explaining the connection between them.

41 Phase 1 SVOT Document The SVOT deliverable also includes detailed instructions on how to calculate more than 100 of the most commonly used measures and metrics.

42 Phase 1 - Best Practices Business Objects Web Universe Question: If a customer visits page A, B, and C which page should take the credit? A B C $100 Answer: Page A, B, and C should all take credit equally Revenues Page A $100 B $100 C $100 TOTAL REVENUES $100 Knowing exactly which page drove the customer action is not always possible Therefore, double counting is a standard practice in web analytics Companies such as Delta Air Lines, Toshiba, Circuit City, REI, and Fairmont all consistently attribute their Web revenues to all pages customers view in a single session (source: Coremetrics)

43 Phase 2 - Productivity Steps Taken: Interviewed core team on key data pain areas Leveraged feedback from Leadership team interviews and project sponsors to understand top areas of data concern Prioritized list of concerns / pain areas Outcome: The development of 4 new aggregated and cleansed data sources for: Daily Channel Revenues Daily revenues by channel by hotel (brand / region) Call Volume Data Call volume by date by toll-free number Booking Source Data Aggregated revenues by booking source Coremetrics Data Aggregated web analytics data

44 Phase 2 Productivity Monthly Query The second set of deliverables included the design and development of data sets for the most difficult and often confusing sets of data.

45 Phase 2 Productivity Daily Query The second set of deliverables included the design and development of data sets for the most difficult and often confusing sets of data.

46 Phase 3 Executive Dashboard The design and development of an executive level dashboard depicting a balanced approach to measuring the activities was the third phase. Steps Taken: Interviewed Leadership team on top metrics Chose dashboard tool Business Objects Xlsesius Completed iterations of dashboard organization

47 Phase 3 - Dashboard Goals, Principles, and Trade-Offs Goal: To provide leadership with a view of the previous month s quality and quantitative of bookings against resources deployed Guiding Principles: Choose a balanced selection of key measures Ensure measures are aligned with our strategic themes Allow drill down-capabilities Ensure a foundation for future iterations Build in a tool which allows scalability Trade-Offs: Quantity of Measures vs. Number of Pages Simple vs. Complex Measures Retrospective vs. Forward Looking Measures Use of Graphics vs. Numerical Depiction of Measures Comparison of Measures against YOY vs. Target Two Options Were Proposed: A Single Page Scorecard With 6 to 9 Key Measures A Multiple Page Report With 10 to 20+ Key Measures

48 Phase 3 Dashboard Phasing Now: A multiple page scorecard using tabs to simulate drill downs to align with current data warehouse constraints Future: The end result would be a full drill-down capable scorecard depicting all key dimensions (brand, hotel, channel, region, market, and major city) Now August 2008 Distribution Marketing Executive Scorecard AMERICAS EMEA ASIAPACIFIC Future

49 Phase 3 Final Dashboard

50 Phase 4 - Organization Key steps for optimizing the analytics organization: Establish role clarification and expectations (e.g. data analyst vs. business analyst) Project Activities: developed roles and definitions including skills and training Develop an organization that supports sharing knowledge and methodologies Project Activities: identified the storage tools for shared work and subject matter experts Develop a production philosophy, where repeatable reports are developed and designed within the analyst community and moved over to Manila to run Project Activities: developed a process and governance for using Manila as a production shop Form a stronger link between IT for development of reporting tables Project Activities: identified touch points users/owners

51 Phase 4 Organization - Analytics Titles Leadership Director: Manages group of analysts; Provides direction and thought leadership; Supports Strategy Data Analysis Data Specialist Designs and develops new datasets Works with IT and 3 rd party providers of data Focus on data quality and maintains code Senior Business Analyst SME and thought leader in a specific set of data Solves more complex business needs Business Analyst General business skills to conduct ad hoc analysis, spot trends, and make recommendations Statistical Analyst Primary responsibility is to build predictive models and forecasts Reporting Internal Reporting Products Team Survey stakeholders to gather requirements Works with IT to prioritize reports Reporting Analyst (Development) Work with users to design and develop reports, queries, and datasets Documents and moves reports to production Reporting Analyst (Production) Runs reports in production

52 Phase 4 Organization Changes Due to the heavy use of contractors, IHG risked not retaining knowledge so the Manila team was expanded as IHG employees. LEGEND Team Americas Asia Pacific EMEA Channel RM PM Services Res Off Shore

53 Phase 4 Organization Training Analysts recognized that they needed training in analytics Existing analyst training gap On-board training tools Specific Training Needs Access 12% Coremetrics 12% SQL 12% Data Know ledge 24% Most Needed SAS 47% Business Objects 53% 0% 10% 20% 30% 40% 50% 60% % of respondents

54 Phase 4 - Training: Proposed New Hire Training A solid foundation need not be time consuming nor expensive Month One Months Two - Three Months Three - Six Months Six - Eighteen Training 1.Company Orientation (HR) 2.Individual Performance Planning (DM/HR) 3.Work Group Orientation (DM) 4.Introduction to DM 5.Introduction to DM Analytics 6.New hire mentor assigned 1. EDW Overview ( IT) 2. Analytics Overview 3. Business Objects or SAS Basic (area dependent) 1. Booking Source Data In-Depth 2. Other data source Coremetrics Travelclick etc. 1. Advance classes in BO, SQL or SAS as needed for job requirements 2. On-the-job mentoring in specific functional assignments and tools Documents/ Classes Analyst Notebook Analytics Overview DM Data External classes Basic SAS or BO selfstudy Coremetrics Mini Manual and IHG class Coremetrics webinars

55 Phase 4 Plans on Sharing Analysts surveyed focused on sharing information as a key ingredient for future success. Affinity Groups 14% Newsletters/Updates 27% Reoccurring Conference Calls 32% Single person as governance manager 32% Formal request process/tracking 41% Group list 45% Shared space - Deliverables/Templates 59% Data governance committee 73% Semi-annual or regular analyst summit 82% Shared space - Knowledge sharing/best practices 86% 0% 20% 40% 60% 80% 100%

56 RECAP 56

57 We had a Tough Original Analytics Situation Where did you get that data point from? I have something completely different. It took you a day to find out how many customers came to our web-site last week! I use SAS! I use Business Objects! Can we see how we are performing as a whole? - Visually - I am not supposed to be doing that! Maybe XYZ can do it. What are those? ISSUES Multiple Truths Lack of Productivity Executive Reporting Organization / Roles / Training

58 So, we ran a four-phase project SVOT (Single Truth) Productivity Dashboard Organization

59 To move our analysts to become more thinkers and shapers Original State Data Pulling / Defining Aggregation / Cleansing Reporting Analytics Insight Development Strategy Development Future State Vision

60 And leverage Business Objects to drive efficiency And minimize fire drills. 60

61 61 Presented By: Manish A. Shah Director of Strategy & Analytics InterContinental Hotels Group Q&A

62 Thank you for participating. Please provide feedback on this session by completing a short survey via the event mobile application. SESSION CODE: 0704 Learn more year-round at