Tomer Shiran VP Product Management MapR Technologies. November 12, 2013

Transcription

1 Predictive Analytics with Hadoop Tomer Shiran VP Product Management MapR Technologies November 12,

2 Me, Us Tomer Shiran VP Product Management, MapR Technologies MapR Enterprise-grade Hadoop distribution Apache Hadoop + infrastructure and management innovation > 500 paying customers EMEA offices in UK, Germany, Sweden and France (HQ in London) Twitter: #mapr 2

3 Hadoop Job Growth 3

4 Agenda Examples Data-driven solutions Obtaining big training data Recommendation with Mahout and Solr Operational considerations 4

5 Recommendation is Everywhere Recommend sales opportunities to partners $40M revenue in year 1 1.5B records per day Using MapR Media and Advertising e-commerce Enterprise Sales 5

6 Classification is Everywhere 600+ variables considered for every IP address Billions of data points Using MapR Identify anomalous patterns indicating fraud, theft and criminal activity Stop phishing attempts Using MapR Fortune 100 Telco Customer 360 application Each customer is scored and categorized based on all their activity Data from hundreds of streams and databases Using MapR IP address blacklisting Fraud Detection Customer 360 Scoring & Categorization 6

7 Data-Driven Solutions Physics is simple: f = ma; E=mc 2 Human behavior is much more complex Which ad will they click? Is a behavior fraudulent? Why? Don t look for complex models that try to discover general rules The size of the dataset is the most important factor Simple models (n-gram, linear classifiers) with Big Data A. Halevy, P. Norvig, and F. Pereira. The unreasonable effectiveness of data. IEEE Intelligent Systems, 24(2):8-12,

8 The Algorithms Are Less Important 8

9 Focus on the Data Most algorithms come down to counting and simple math Invest your time where you can make a difference Getting more data can improve results by 2x eg, add beacons everywhere to instrument user behavior Tweaking an ML algorithm will yield a fraction of 1% Data wrangling Feature engineering Moving data around 9

10 Obtaining Big Training Data Can t really rely on experts to label the data Doesn t scale (not enough experts out there) Too expensive So how do you get the training data? Crowdsourcing Implicit feedback Obvious features User engagement 10

11 Using Crowdsourcing for Annotation R. Snow, B. O'Connor, D. Jurafsky, and A. Ng. Cheap and fast but is it good? Evaluating non-expert annotations for natural language tasks. EMNLP, Quantity: $2 for 7000 annotations (leveraging Amazon Mechanical Turk and a flat world ) 11 Quality: 4 non-experts = 1 expert

12 Using Obvious Features for Annotation :) :( 12

13 Leveraging Implicit Feedback Users behavior provides valuable training data Google adjusts search rankings based on engagement Did the user click on the result? Did the user come back to the search page within seconds? Most recommendation algorithms are based solely on user activity What products did they view/buy? What ads did they click on? T. Joachims, L. Granka, B. Pan, H. Hembrooke, F. Radlinski, and G. Gay. Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search. ACM TOIS, 25(2):1{27,

14 Increasing Exploration Can t learn much about results on page 2, 3 or 4! ctr We need to find ways to increase exploration to broaden our learning rank Exploration of the second page 14

15 Result Dithering Dithering is used to re-order recommendation results Re-ordering is done randomly Dithering is guaranteed to make off-line performance worse Dithering also has a near perfect record of making actual performance much better Made more difference than any other change 15

16 Simple Dithering Algorithm Generate synthetic score from log rank plus Gaussian s = logr + N(0,ε) Pick noise scale to provide desired level of mixing Typically: Δr r expε ε 0.4, 0.8 [ ] Oh use floor(t/t) as seed so results don t change too often 16

17 Example:ε = Each line represents a recommendation of 8 items The non-dithered recommendation would be 1, 2,, 8

18 Example:ε = log 2 = Each line represents a recommendation of 8 items The non-dithered recommendation would be 1, 2,, 8

19 Recommendations with Mahout and Solr 19

20 What is Recommendation? The behavior of a crowd helps us understand what individuals will do 20

21 Batch and Real-Time We can learn about the relationship between items every X hours These relationships don t change often People who buy Nikon D7100 cameras also buy a Nikon EN-EL15 battery What to recommend to Bob has to be determined in real-time Bob may be a new user with no history Bob is shopping for a camera right now, but he was buying a baby bottle an hour ago How do we do that? Mahout for the heavy number crunching Solr/Elasticsearch for the real-time recommendations 21

22 Real-Time Recommender Architecture Complete history (log files/tables) Co-occurrence (Mahout) Indexing (Solr) Item metadata Index shards Recent history for single user Web tier Search (Solr) Recommendation Note: All data lives in the cluster 22

23 Recommendations Alice Alice got an apple and a puppy Charles Charles got a bicycle 23

24 Recommendations Alice Alice got an apple and a puppy Bob Bob got an apple Charles Charles got a bicycle 24

25 Recommendations Alice Bob? What else would Bob like? Charles 25

26 Log Files Alice Charles Charles Alice Alice Bob Bob 26

27 History Matrix Alice Bob Charles 27

28 Co-Occurrence Matrix: Items by Items Q: How do you tell which co-occurrences are useful? A: Let Mahout do the math 28

29 Indicator Matrix: Anomalous Co-Occurrences Result: The marked row will be added to the indicator field in the item document 29

30 Indicator Matrix That one row from indicator matrix becomes the indicator field in the Solr document used to deploy the recommendation engine. id: t4 title: puppy desc: The sweetest little puppy ever. keywords: puppy, dog, pet indicators: (t1) Note: Data for the indicator field is added directly to meta-data for a document in Solr index. You don t need to create a separate index for the indicators. 30

31 Internals of the Recommender Engine Q: What should we recommend if new user listened to 2122:Fats Domino & 303:Beatles? A: Search the index with indicator_artists:2122 OR indicator_artists:

32 Internals of the Recommender Engine 1710:Check Berry is the top recommendation 32

33 Toolbox for Predictive Analytics with Hadoop Mahout Use it for Recommendations, Clustering, Math Vowpal Wabbit Use it for Classification (but it s harder to use) SkyTree Commercial (not open source) implementation of common algorithms 33

34 Operational Considerations Snapshot your raw data before training Reproducible process MapR snapshots (Beware of HDFS so-called snapshots not consistent ) Leverage NFS for real-time data ingestion Train the model on today s data Learning schedule independent from ingestion schedule Look for HA, data protection, disaster recovery Predictive analytics increases revenue or reduces cost Quickly becomes a must-have, not a nice-to-have 34

35 Starting a Project or Moving to Production? Read the Hadoop Buyer s Guide Free hard copies at the MapR booth Or read it online: Come to our office hour (right now, 2:45pm) Will be joined by Ted Dunning, Mahout Committer and author of Mahout In Action Contact MapR Data Science team that can help you with Hadoop and predictive analytics 35

36 Thank You 36