STAR WARS AND THE ART OF DATA SCIENCE

Transcription

1 STAR WARS AND THE ART OF DATA SCIENCE MELODIE RUSH, SENIOR ANALYTICAL ENGINEER CUSTOMER LOYALTY Original Presentation Created And Presented By Mary Osborne, Business Visualization Manager At 2014 SAS Global Forum

2

3 FIRST A TRUTH. Han didn t shoot first, he shot. Greedo never had a chance.

4 WHAT IS DATA SCIENCE? Drew Conway, Project FLA

5 UNDERSTANDING THE WAYS OF THE FORCE

6 HACKING SKILLS AND THE DATA SCIENTIST JEDI

7 THE DATA JEDI S DISCIPLINE SAS DATA INTEGRATION STUDIO Point-and-click Build and edit data Manage metadata Collaboration

8 WHY USE SAS DATA INTEGRATION SERVER / STUDIO? Centralized Metadata Audit trail & consistency Data governance Change Management Checkpoint / Restart Documentation Impact Analysis

9 THE DATA INTEGRATION PROCESS 1. Create a reference to the physical file. 2. Specify the delimiters and file parameters. 3. View the output.

10 A NEW HOPE AS A PRELIMINARY SAS DATASET!

11 THE DATA INTEGRATION PROCESS FLOW

12 ADD AN IDENTIFIER

13 APPEND THE SCRIPTS INTO TRILOGY

14 UNDERSTANDING UNSTRUCTURED DATA Context is king. Jargon exists everywhere. Error is expected and accepted. Language changes over time. Geographic locale can have a significant impact.

15 THE DATA JEDI S LIGHTSABER SAS CONTEXTUAL ANALYSIS Classification Clustering Term and concept mapping Rule generation and building Scoring

16 WHAT IS SAS CONTEXTUAL ANALYSIS? 1. Classify large document collections for better retrieval and reporting. 2. Visualize and discover trends within text data 3. Eliminate lengthy and error-prone manual document reviews 4. Hybrid approach combines machine learning and user refinement 5. Automatically generate linguistic rules, enhanced with machine/ user input 6. Adds unstructured data sources to analytics investigations 7. Useful for social media, MS Office files, PDFs, , surveys, contact centre conversations and more.

17 SAS TEXT ANALYTICS TECHNIQUES TEXT MINING SENTIMENT ANALYSIS Classify documents and specific attributes/features as having positive, negative, or neutral/mixed tone Explore textual data to uncover valuable patterns, themes, and insights CONTENT CATEGORIZATION Automatically identify or extract content that matches predefined criteria to more easily search by, report on, and model/segment by important themes SAS Text Analytics INFORMATION RETRIEVAL Copyright 2012, SAS Institute Inc. All rights reserved. INTEGRATED ANALYTICS Integrate structured and unstructured data for enhanced: Forecasting Optimization Predictive Modeling Network Analysis 19

18 SAS TEXT ANALYTICS TEXT MINING Explore textual data to uncover valuable patterns, themes, and insights CONTEXTUAL ANALYSIS Information Retrieval Automatic Topic Detection Content Categorization Entity Extraction Sentiment Analysis SAS Text Analytics INTEGRATED ANALYTICS Integrate structured and unstructured data for enhanced: Forecasting Optimization Predictive Modeling Network Analysis Copyright 2012, SAS Institute Inc. All rights reserved. 20

19 SAS CONTEXTUAL ANALYSIS CONTEXTUAL ANALYSIS PROCESS Natural Language Processing Discovery Machine Learning Topics, Insights, Relationships, Taxonomies, Scored Documents How does it work? JL H Human Input Unstructured Text Data Context Operations Further Analysis

20 THE FIRST CLUE TO DATA INITIALLY GONE TO THE DARK SIDE

21 THE RAW DATA REVISITED

22 THE RAW DATA REVISITED

23 DATA PREPARATION, EPISODE II

24 DEFINE YOUR SYNONYMS

25 THE SYNONYM LIST APPLIED

26 VISUAL EXPLORATION TERM MAPS

27 VISUAL EXPLORATION TERM MAPS

28 VISUAL EXPLORATION TERM MAPS

29 VISUAL EXPLORATION TERM MAPS

30 VISUAL EXPLORATION TERM MAPS

31 VISUAL EXPLORATION TOPIC EXTRACTION

32 AUTOMATIC RULE GENERATION - OVERVIEW

33 AUTOMATIC RULE GENERATION THE RULE

34 CUSTOMIZING CATEGORIES AND RULE SETS

35 RULE CREATION

36 SCORING BRINGS ADDITIONAL STRUCTURE TO THE UNSTRUCTURED DATA

37 THE DATA JEDI S MIND TRICKS SAS VISUAL ANALYTICS Speed Approachable Analytics Big Data Manipulation Integration

38 SAS VISUAL ANALYTICS PLAYING A ROLE IN THE ANALYTICS LIFECYCLE Visual Analytics is the lens for viewing the results of analytic insights and creating new ones Provides incredibly fast data visualization and analytic visualization on massive amounts of data (or any amount of data) Simplifies insights sharing and collaboration between groups Empowers broader user communities to accelerate model development activities Decreases time to insight.

39 SAS VISUAL ANALYTICS OVERVIEW

40 BENEFITS INCREASE INSIGHTS ACROSS THE ORGANIZATION Enterprise AND Self-service Easy to use Analytics Work with more data Reporting and Dashboards Mobile BI Collaboration

41 DISTRIBUTION OF SPOKEN LINES

42 ON-THE-FLY HIERARCHIES INTERIOR LOCATION

43 ON-THE-FLY HIERARCHIES EXTERIOR LOCATION

44 SCORED OUTPUT THE FORCE VS. THE DARK SIDE

45 SCORED OUTPUT - WEAPONS

46 SCORED OUTPUT PLANETARY SYSTEMS

47 APPROACHABLE ANALYTICS WORD CLOUD

48 BESIDES THE FACT THAT STAR WARS IS COOL, HOW DOES THIS APPLY TO BUSINESS? Data preparation is vital to all analytical processes. Text is omnipresent. Text analysis is more than just sentiment. Data Jedi are few and need to be appreciated, not stymied. If you don t have a good strategy for getting the results into the hands of the powers that be, you truly are lost.

49 Denver International Airport celebrates May 4th

50 Denver International Airport celebrates May 4th

51 QUESTIONS? MAY THE FORCE BE WITH YOU! Connect with me: LinkedIn: