Liktap 2012 Search Engine

Size: px

Start display at page:

Download "Liktap 2012 Search Engine"

Oscar Stephens
3 years ago
Views:

1 Related Searches at LinkedIn Mitul Tiwari Joint work with Azarias Reda, Yubin Park, Christian Posse, and Sam Shah LinkedIn 1

2 Who am I 2

3 Outline About LinkedIn Related Searches Design Implementation Evaluation 3

4 LinkedIn by the numbers 175M+ members 2+ new user registrations per second 4.2 Billion people searches in Billion page views in Q million monthly active users in Q

5 Broad Range of Products Profile Search People You May Know News Skills Hiring Solutions

6 Related Searches at LinkedIn Millions of searches everyday Goal: Build related searches system at LinkedIn To help users to explore and refine their queries 6

7 Related Searches at LinkedIn

8 Related Searches Design Implementation Evaluation 8

9 Related Searches Design Implementation Evaluation 9

10 Design 10

11 Design Signals Collaborative Filtering Query-Result Click graph Overlapping terms 10

12 Design Signals Collaborative Filtering Query-Result Click graph Overlapping terms Length-bias 10

13 Design Signals Collaborative Filtering Query-Result Click graph Overlapping terms Length-bias Ensemble approach for unified recommendation 10

14 Design Signals Collaborative Filtering Query-Result Click graph Overlapping terms Length-bias Ensemble approach for unified recommendation Practical considerations 10

15 Design: Collaborative Filtering Q1 Q2 Q3 Q4 Time Searches correlated by time Searches done in the same session by the same user Collaborative filtering: implicit feedback TFIDF scoring to take care of popular queries (e.g. `Obama ) 11

16 Design: Query-Result Clicks Q1 R1 Qn Rm Searches correlated by result clicks 12

17 Design: Overlapping Terms Q1 Software Developer Q2 Software Engineer Searches with overlapping terms TFIDF scoring to give importance to terms 13

18 Design: Length Bias Insight: clicks on suggestions one term longer 14

19 Design: Length Bias Insight: clicks on suggestions one term longer Corresponds to refining the initial query Statistical biasing model to score a longer query higher

20 Design: Length Bias Insight: clicks on suggestions one term longer Corresponds to refining the initial query Statistical biasing model to score a longer query higher

21 Design: Ensemble Approach Need to generate unified recommendation dataset Analysis to figure out engagement of each signal Attempted ML approach Minimal overlap across different signals 16

22 Design: Ensemble Approach Step-wise unionization Importance based on individual signal performance First, collaborative filter Second, queries correlated by query-result clicks Third, queries overlapping terms 17

23 Design: Practical Considerations System designed for public consumption Strong profanity filters Need to deal with misspellings Languages Remove spammy search queries 18

24 Related Searches Design Implementation Evaluation 19

25 Implementation Challenge Scale 175M+ members Billions of searches Terabytes of data to process 20

26 Implementation Search Backend HDFS Hadoop Front End Kafka Related Searches Backend Metaphor Voldemort Kafka: publish-subscribe messaging system Hadoop: MapReduce data processing system Azkaban: Hadoop workflow management tool Voldemort: Key-value store

27 Implementation: Workflow Query Preprocessing Pair Formation Graph Formation Token Identification CF Scoring QRQ Scoring Partial Scoring Step-wise Union Length Bias Filter Step-wise Union

28 Related Searches Design Implementation Evaluation 23

29 Evaluation Performance of each signal and combination How does the system scale? 24

30 Evaluation Cont d Offline evaluation Precision-Recall Online evaluation A/B testing to measure engagement Performance evaluation 25

31 Offline Evaluation Correct set: set of searches performed by a user in the following K minutes, here K=10 26

32 Online Evaluation Used A/B testing Metrics Coverage: queries with recommendations Impressions: # of recommendations shown Clicks: Clicks on recommendations Click-through rate (CTR): Clicks per impression 27

33 Online Evaluation

34 Evaluation: System Runtime 29

35 Details Metaphor: a System for Related Search Recommendations, Azarias Reda, Yubin Park, Mitul Tiwari, Christian Posse, and Sam Shah. In Proceedings of the CIKM,

Building Data Products using Hadoop at Linkedin. Mitul Tiwari Search, Network, and Analytics (SNA) LinkedIn

Building Data Products using Hadoop at Linkedin Mitul Tiwari Search, Network, and Analytics (SNA) LinkedIn 1 1 Who am I? 2 2 What do I mean by Data Products? 3 3 People You May Know 4 4 Profile Stats: