STAR WARS AND THE ART OF DATA SCIENCE MELODIE RUSH, SENIOR ANALYTICAL ENGINEER CUSTOMER LOYALTY Original Presentation Created And Presented By Mary Osborne, Business Visualization Manager At 2014 SAS Global Forum
FIRST A TRUTH. Han didn t shoot first, he shot. Greedo never had a chance.
WHAT IS DATA SCIENCE? Drew Conway, Project FLA
UNDERSTANDING THE WAYS OF THE FORCE
HACKING SKILLS AND THE DATA SCIENTIST JEDI
THE DATA JEDI S DISCIPLINE SAS DATA INTEGRATION STUDIO Point-and-click Build and edit data Manage metadata Collaboration
WHY USE SAS DATA INTEGRATION SERVER / STUDIO? Centralized Metadata Audit trail & consistency Data governance Change Management Checkpoint / Restart Documentation Impact Analysis
THE DATA INTEGRATION PROCESS 1. Create a reference to the physical file. 2. Specify the delimiters and file parameters. 3. View the output.
A NEW HOPE AS A PRELIMINARY SAS DATASET!
THE DATA INTEGRATION PROCESS FLOW
ADD AN IDENTIFIER
APPEND THE SCRIPTS INTO TRILOGY
UNDERSTANDING UNSTRUCTURED DATA Context is king. Jargon exists everywhere. Error is expected and accepted. Language changes over time. Geographic locale can have a significant impact.
THE DATA JEDI S LIGHTSABER SAS CONTEXTUAL ANALYSIS Classification Clustering Term and concept mapping Rule generation and building Scoring
WHAT IS SAS CONTEXTUAL ANALYSIS? 1. Classify large document collections for better retrieval and reporting. 2. Visualize and discover trends within text data 3. Eliminate lengthy and error-prone manual document reviews 4. Hybrid approach combines machine learning and user refinement 5. Automatically generate linguistic rules, enhanced with machine/ user input 6. Adds unstructured data sources to analytics investigations 7. Useful for social media, MS Office files, PDFs, email, surveys, contact centre conversations and more.
SAS TEXT ANALYTICS TECHNIQUES TEXT MINING SENTIMENT ANALYSIS Classify documents and specific attributes/features as having positive, negative, or neutral/mixed tone Explore textual data to uncover valuable patterns, themes, and insights CONTENT CATEGORIZATION Automatically identify or extract content that matches predefined criteria to more easily search by, report on, and model/segment by important themes SAS Text Analytics INFORMATION RETRIEVAL Copyright 2012, SAS Institute Inc. All rights reserved. INTEGRATED ANALYTICS Integrate structured and unstructured data for enhanced: Forecasting Optimization Predictive Modeling Network Analysis 19
SAS TEXT ANALYTICS TEXT MINING Explore textual data to uncover valuable patterns, themes, and insights CONTEXTUAL ANALYSIS Information Retrieval Automatic Topic Detection Content Categorization Entity Extraction Sentiment Analysis SAS Text Analytics INTEGRATED ANALYTICS Integrate structured and unstructured data for enhanced: Forecasting Optimization Predictive Modeling Network Analysis Copyright 2012, SAS Institute Inc. All rights reserved. 20
SAS CONTEXTUAL ANALYSIS CONTEXTUAL ANALYSIS PROCESS Natural Language Processing Discovery Machine Learning Topics, Insights, Relationships, Taxonomies, Scored Documents How does it work? JL H Human Input Unstructured Text Data Context Operations Further Analysis
THE FIRST CLUE TO DATA INITIALLY GONE TO THE DARK SIDE
THE RAW DATA REVISITED
THE RAW DATA REVISITED
DATA PREPARATION, EPISODE II
DEFINE YOUR SYNONYMS
THE SYNONYM LIST APPLIED
VISUAL EXPLORATION TERM MAPS
VISUAL EXPLORATION TERM MAPS
VISUAL EXPLORATION TERM MAPS
VISUAL EXPLORATION TERM MAPS
VISUAL EXPLORATION TERM MAPS
VISUAL EXPLORATION TOPIC EXTRACTION
AUTOMATIC RULE GENERATION - OVERVIEW
AUTOMATIC RULE GENERATION THE RULE
CUSTOMIZING CATEGORIES AND RULE SETS
RULE CREATION
SCORING BRINGS ADDITIONAL STRUCTURE TO THE UNSTRUCTURED DATA
THE DATA JEDI S MIND TRICKS SAS VISUAL ANALYTICS Speed Approachable Analytics Big Data Manipulation Integration
SAS VISUAL ANALYTICS PLAYING A ROLE IN THE ANALYTICS LIFECYCLE Visual Analytics is the lens for viewing the results of analytic insights and creating new ones Provides incredibly fast data visualization and analytic visualization on massive amounts of data (or any amount of data) Simplifies insights sharing and collaboration between groups Empowers broader user communities to accelerate model development activities Decreases time to insight.
SAS VISUAL ANALYTICS OVERVIEW
BENEFITS INCREASE INSIGHTS ACROSS THE ORGANIZATION Enterprise AND Self-service Easy to use Analytics Work with more data Reporting and Dashboards Mobile BI Collaboration
DISTRIBUTION OF SPOKEN LINES
ON-THE-FLY HIERARCHIES INTERIOR LOCATION
ON-THE-FLY HIERARCHIES EXTERIOR LOCATION
SCORED OUTPUT THE FORCE VS. THE DARK SIDE
SCORED OUTPUT - WEAPONS
SCORED OUTPUT PLANETARY SYSTEMS
APPROACHABLE ANALYTICS WORD CLOUD
BESIDES THE FACT THAT STAR WARS IS COOL, HOW DOES THIS APPLY TO BUSINESS? Data preparation is vital to all analytical processes. Text is omnipresent. Text analysis is more than just sentiment. Data Jedi are few and need to be appreciated, not stymied. If you don t have a good strategy for getting the results into the hands of the powers that be, you truly are lost.
Denver International Airport celebrates May 4th
Denver International Airport celebrates May 4th
QUESTIONS? MAY THE FORCE BE WITH YOU! Connect with me: LinkedIn: https://www.linkedin.com/in/melodierush Twitter: @Melodie_Rush