Data & Analytics Success Stories That Any Organization Can Achieve

Transcription

1 Data & Analytics Success Stories That Any Organization Can Achieve Eugene Kolker, PhD

2 3 Key Themes 1. Transforming data and analytics (D & A) capabilities into a vital core service. 2. Building a step-by-step process of engagement with your clients. 3. Success stories that can be applied in every organization. 2

3 Data to Knowledge to Action and Outcomes Beyond Data Surfing - Towards Data Acting! 3

4 Key Strategies & PPT-DA Approach 3 Strategies: 1. Build Data & Analytics (D&A) to improve organization performance and for competitive advantage. 2. Adopt and follow project management best practices along with delivering great science! 3. Hire data scientists who can carefully and compassionately communicate analysis findings without bias, agenda, or ego. PPT-DA Approach: 1. People, 2. Process, 3. Technology (D&A) 4

5 D&A Are Your Product / Your Business / YOU! D&A are key strategic assets for (1) improvement, (2) growth, and (3) competitive advantage; Utilizing D&A aligns to strategic objectives; D&A should be thought of as a core service similar to IT, HR, and Finance; and D&A are synergistic with and complementary to IT, not redundant. 5

6 cdoanalytics.org Set Executive Expectations Carefully Successful D&A Programs Take Time to Develop! 6 Analytics: The Art and Science of the Better, Computer World Technology Briefings Business Intelligence.

7 Hybrid Architecture Advanced Analytics Repositories, Databases EDW External Data Hadoop 7 Adapted from Gartner

8 Key Determinants of Project Success Failed Project Successful Project Process Not defined Well-defined Customers Identified Concerned stakeholders Clear customers Core Question Free floating anxiety Hypothesis Shared Vision No expectations Fully articulated People Communication Opaque Transparent 8 Data Science High quality High quality Actionable Insights Yes, but too much to handle Actionable Bottom line: People make the Difference! D&A

9 What Each Party Brings to the Project Client Brings: Experience and Data Final Decision on Project Direction Implementation and Outcomes D&A Team Brings: Data Processing and Analytics Expertise Optional Decision Pathways and Predicted Outcomes Ongoing Measurements and Comparative Effectiveness 9

10 Always Manage Projects Professionally Identification and Qualification Team learns the needs and goals from the client Team assesses the availability of data and resources Confirmation Client and D&A Team define the project purpose and scope Planning and Implementation Client and D&A Team approve the project specifics, including: scope, deliverables, cost, schedule, and outcome measures Team completes the analysis and presents the results to the client cdoanalytics.org Measurement and Documentation Client owns the data-empowered decision and outcomes Team can complete an effectiveness comparison and analysis of outcomes 10

11 11 3 Steps: Steps to Empower People Through D & A 1. Identifying the needs of the enterprise, people pain points, and business growth opportunities. 2. Establishing processes that support and are supported by the people. 3. Selecting the appropriate technology to align with your people and processes. PPT-DA Approach: 1. People, 2. Process, 3. Technology (D&A)

12 PPT-DA Approach People Business needs, pain points (challenges/opportunities), and growth Competitive advantage, long-term changes, and survival Process Best practices in consulting and business Technology R, Java, Python, SQL, EDW, Hadoop, AWS, and Azure Data Leverage both internal and external data, social media Analytics A/B Testing, regression analysis, modeling, and optimization 12

13 How Data Scientists Can Operate Like Consultants Need to take the time to listen and get to know your clients & their objectives Appreciate the clients knowledge and expertise on subject matter Complete robust data science for a full unbiased 360-view Concentrate on communicating well: Clearly define projects with clients. Specify deliverables and expected outcomes; Utilize clients expertise and experience to inform the analysis; and Present clear data stories. Do not oversell or push your client may have social, political, or financial reasons why they can t make changes don t take it personally 13

14 3 Learning Approaches Talk to your peers, incl. going to meetings like this Follow-up best practices, incl. reports by Gartner, IBM, and Experian Read books, e.g. The Wolf in CIO s Clothing by T. Nunno (defensive and offensive approaches / light and dark sides) 14

15 3 Success Stories That Can Be Applied to Any Organization 1. D&A to Action: Starbucks 2. A/B Testing (of Software Tools) 3. Performance Benchmarking (IF have time) 15

16 1. Improving Chai Experience??? 16

17 cdoanalytics.org 1. Improving Chai Experience, 2 17

18 2. A/B Comparison of Buy vs. Build Project Background: Severe Clinical Deterioration (SCDs) of a child s health is a very frightening situation for families and care givers alike. We are constantly looking for tools that can help this situation, however, live testing of such tools in a care setting is risky and prohibitively expensive. Our leadership called upon D&A Team to come up with a solution. Project Description: 1. Can an external vendor tool (Buy) be used to predict SCDs? and 2. How does Buy perform in a head-to-head comparison of SCH s internally developed tool (Build) for predicting SCDs? 18

19 Side-by-Side Comparison of Buy vs. Build Q1: Can Buy predict SCDs? Q2: Head-to-head Buy vs. Build? True Positive Rate Buy Build 19 Buy Scores Buy scores for patients with no SCDs and with SCDs strongly overlap. False Positive Rate Buy & Build perform similarly when identifying SCDs.

20 Hybrid Architecture, 2 Advanced Analytics Repositories, Databases EDW External Data Clusters 20

21 Buy vs. Build Comparison Summary Buy Build SCD Prediction Equivalent Equivalent Pediatric Utility Uncertain Large decrease in SCDs* EHR Integration Yes* No Algorithm Closed Open/Transparent Cost Unknown Unknown Development Unknown Unknown Maintenance Unknown Unknown *These are self-reported assessments: they are not tested in this project. 21

22 Key Findings & Outcomes Findings: 1. Buy had a false positive rate between 90% and 95%. 2. Buy and Build were equivalent at predicting SCDs. 3. Buy would require significant development and training work to implement in hospital operations. Outcomes: Based on this analysis, leadership concluded that: 1. Buy wasn t usable for predicting SCDs, 2. Build should be integrated into EHR, and 3. Buy can be tested for another purpose. 22

23 Why Buy vs. Build Succeeded Well-defined expectations and shared vision; Transparent and structured communication led to strong leadership engagement; and Disciplined process and quality analysis led to succinct and actionable outcomes. SUCCESS STORY I go to D&A Team to help me answer the toughest, most complex questions. Their projects and recommendations, including work on software tool comparison, saved the hospital an incredible amount of time and money. Wes Wright, Corporate CTO, Sutter Health 23

24 3. Performance Benchmarking cdoanalytics.org Each year SCH participates in USNWR. These national rankings bring patients to our hospital. CDO Analytics uses the results to benchmark SCH against other hospitals and identify areas for improvement. In 2013, D&A Team did not complete the project due to lack of resources. In 2014, D&A Team s analysis identified a downward trend that led to the creation of a new enterprise-wide program. 24

25 3. Performance Benchmarking, 2 Program Background: USNWR submission requires considerable time and effort. The process had a low ROI as it was not aligned with strategic planning and did not fill the need for system-wide benchmarking. This new Benchmarking Improvement Program was created to address these issues. Program Description: 1. What is the baseline of SCH s current USNWR process? 2. Where are the opportunities for process improvement? 3. How can SCH use the USNWR results to improve care? 25

26 Hybrid Architecture, 3 Advanced Analytics Repositories, Databases EDW External Data R, Python 26

27 Overall Findings SCH does well in the Structure component (e.g. quality improvement, patient and family services, and specialized clinics and programs) The categories below offer opportunities for improvement Category Weight Range # of SLs affected Commitment to Best Practices ICU Infections Rates Surgical Outcomes/ Survival Rates Patient Management Readmissions

28 Key Findings from Analysis Hospital incorporated improving the ICU infection rate into strategic planning. Infection rate has decreased since 2012 SUCCESS STORY The improvements that the D&A Team identified were applicable enterprise-wide as well as on a medical-service-line level Their work is of immense value because it helped us improve patient care. Dr. David Fisher, SVP and CMO (Ret.) 28

29 Overall Trends 2015 cdoanalytics.org When compared to forecasted rank, 2015 actual rank improved dramatically. This change was primarily driven by Outcomes. Forecasted Rank Actual Rank 29

30 SCH Rank Changes Department 2013 rank 2014 rank 2015 rank Change (2014 to 2015) Cancer 6* 7* 5* +2 Cardiology Endocrinology Gastroenterology * +12 Neonatology ** +23 Nephrology 4** 5* 3** +2 Neurology Orthopedics Pulmonology * +12 Urology Average Rank Honor Roll Rank --- (25) --- (28) **Top 5% of hospitals for , *Top 10% of hospitals for For details: 2013, 2014, 2015

31 Key Lessons Learned: D&A are YOU! 1. Customer and core questions identified for a well-defined project. 2. Maintain communication and high sponsor engagement with cadence and transparency throughout the process. 3. Great data science to deliver actionable insights towards shared vision. PPT-DA Approach: People-Process-Technology (D&A) Bottom line: People make the Difference! 31

32 Thanks to Team! Roger Higdon, Natali Kolker, Libby Montague, Beth Stewart, Imre Janko, Maggie Lackey, Rob Arnold, John Choieniere, Mary Eckert, and Aaron Lai Mark Del Beccaro, Tom Hansen, Wes Wright Questions? Thank You! 32