Text mining for tourism

Text mining for tourism The world of typical high quality restaurants in Piedmont Roberto Fontana ATR Osservatorio Turistico Regione Piemonte Agenzia Regionale per la promozione Turistica del Piemonte Osservatorio Turistico Regionale The Observatory on tourism of Piedmont in co-operation with: SAS ITALIA EPAT-FIPE PIEMONTE 1

The team and the software tools Agenzia Regionale per la promozione Turistica del Piemonte Osservatorio Turistico Regionale Roberto Fontana Cristina Bergonzo Emanuela Giorgini SAS software Sabina Silani Sara Galli Michela Giacomini Enterprise Miner Enterprise Guide Francesca Martinengo Cristina Giraudo SEUGI21, 18 June 2003 N. 2

Introduction! Tourism flows generated by the interest for food & drink are extremely important for Piedmont.! In this context typical high quality restaurants (THQRs) play a major role.! In May 2002, the Observatory on Tourism of Piedmont, in co-operation with SAS and EPAT-FIPE Piemonte, started a joint research project on THQRs.! This work summarizes some of the results that have been achieved so far! For further information please refer to the contact point reported at the end of this document SEUGI21, 18 June 2003 N. 3

Work objectives 1. To build an atlas of typical high quality restaurants (THQRs) in Piedmont, combining the large numbers of restaurant guides available in bookshops 2. To compare Piedmont with the other Italian Regions in terms of number of THQRs 3. To study customer s preferences and to get an estimate of the use of restaurant places in different periods of the year SEUGI21, 18 June 2003 N. 4

Methodological approach! To acquire as many as possible restaurant guides (possibly in electronic format to facilitate the analysis)! To use multivariate statistical methods and software (including state-of-the-art tools for text mining) to extract relevant information from them! To conduct surveys, interviewing restaurants owners and their customers. The first phase focused on top restaurants. SEUGI21, 18 June 2003 N. 5

Progress of work! For 6 guides, the number of restaurants that have been reviewed for every Italian Region, has been determined.! The electronic format of the chapters related to Piedmont of three major restaurant guides has been acquired and analyzed.! Top restaurants owners have been interviewed in June, September and October 2002. SEUGI21, 18 June 2003 N. 6

Some Results An analysis of the offer based on some restaurant guides 7

STEP 0: THE Giallo Dat@ s DATA! The first preliminary step was based on the analysis of data provided by Giallo Dat@! Giallo Dat@ is a line of services of Consodata S.p.A., a company of the Seat Pagine Gialle Group, operating in Customer Relationship Management.! Giallo Dat@ offers a comprehensive database of 9 million households and 27 million individuals across Europe, all collected through national household surveys, for marketing intelligence needs.! Available data were the number of four different types of restaurants for each Italian province: " Ristoranti " Trattorie " Ristoranti Tipici " Pizzerie SEUGI21, 18 June 2003 N. 8

Giallo Dat@ Distribution of Restaurants among Italian Regions Regione Ristoranti Trattorie Ristoranti tipici Pizzerie Totale Abruzzo 1.020 95 75 608 1.798 Basilicata 254 25 22 126 427 Calabria 837 50 32 465 1.384 Campania 2.449 186 109 1.300 4.044 Emilia Romagna 2.511 831 136 1.661 5.139 Friuli Venezia Giulia 494 631 46 414 1.585 Lazio 3.219 732 261 2.344 6.556 Liguria 1.443 523 112 706 2.784 Lombardia 4.494 1.764 303 2.989 9.550 Marche 1.103 111 49 762 2.025 Molise 193 16 16 85 310 Piemonte 2.388 644 166 1.334 4.532 Puglia 1.311 184 74 1.046 2.615 Sardegna 921 63 35 733 1.752 Sicilia 1.450 434 104 1.110 3.098 Toscana 2.655 447 140 1.533 4.775 Trentino Alto Adige 882 105 42 353 1.382 Umbria 585 99 34 375 1.093 Valle d'aosta 225 19 7 62 313 Veneto 2.071 1.610 134 2.368 6.183 ITALY 30.505 8.569 1.897 20.374 61.345 SEUGI21, 18 June 2003 N. 9 Fonte: Giallo Dat@ - maggio 2002

GIALLO DAT@ - Distribution of Restaurants among Italian Regions RISTORANTI TIPICI LOCALI [n] LOCALI [%] LOCALI [%cum] RISTORANTI LOCALI [n] LOCALI [%] LOCALI [%cum] PIZZERIE 0 100 200 300 400 LOCALI [n] 303 15. 97 15. 97 261 13. 76 29. 73 166 8. 75 38. 48 140 7. 38 45. 86 136 7. 17 53. 03 134 7. 06 60. 09 112 5. 90 66. 00 109 5. 75 71. 74 104 5. 48 77. 23 75 3. 95 81. 18 74 3. 90 85. 08 49 2. 58 87. 66 46 2. 42 90. 09 42 2. 21 92. 30 35 1. 85 94. 15 34 1. 79 95. 94 32 1. 69 97. 63 22 1. 16 98. 79 16 0. 84 99. 63 7 0. 37 100. 00 LOCALI [%] LOCALI [%cum] 2989 14. 67 14. 67 2368 11. 62 26. 29 2344 11. 50 37. 79 1661 8. 15 45. 95 1533 7. 52 53. 47 1334 6. 55 60. 02 1300 6. 38 66. 40 1110 5. 45 71. 84 1046 5. 13 76. 98 762 3. 74 80. 72 733 3. 60 84. 31 708 3. 47 87. 79 608 2. 98 90. 77 465 2. 28 93. 06 414 2. 03 95. 09 375 1. 84 96. 93 353 1. 73 98. 66 126 0. 62 99. 28 0 1000 2000 3000 4000 5000 4494 14. 73 14. 73 3219 10. 55 25. 28 2655 8. 70 33. 99 2511 8. 23 42. 22 2449 8. 03 50. 25 2388 7. 83 58. 08 2071 6. 79 64. 86 1450 4. 75 69. 62 1443 4. 73 74. 35 1311 4. 30 78. 65 1103 3. 62 82. 26 1020 3. 34 85. 61 921 3. 02 88. 62 882 2. 89 91. 52 837 2. 74 94. 26 585 1. 92 96. 18 494 1. 62 97. 80 254 0. 83 98. 63 225 0. 74 99. 37 193 0. 63 100. 00 LOCALI [n] LOCALI [%] SEUGI21, 18 June 2003 N. 10 85 0. 42 99. 70 62 0. 30 100. 00 TRATTORIE LOCALI [%cum] 1764 20. 59 20. 59 1610 18. 79 39. 37 831 9. 70 49. 07 732 8. 54 57. 61 644 7. 52 65. 13 631 7. 36 72. 49 523 6. 10 78. 60 447 5. 22 83. 81 434 5. 06 88. 88 186 2. 17 91. 05 184 2. 15 93. 20 111 1. 30 94. 49 105 1. 23 95. 72 99 1. 16 96. 87 95 1. 11 97. 98 63 0. 74 98. 72 50 0. 58 99. 30 25 0. 29 99. 59 19 0. 22 99. 81 16 0. 19 100. 00 0 1000 2000 3000 0 200 400 600 800 1000 1200 1400 1600 1800

Italian Restaurants Giallo Dat@ Regione Abruzzo Basilicata Molise Emilia Romagna Lazio Liguria Lombardia Piemonte Puglia Sicilia Toscana Umbria Friuli Venezia Giulia Veneto Calabria Campania Marche Sardegna Trentino Alto Adige Valle d'aosta Ristoranti 85.71% 84.39% 85.78% 72.20% 76.42% 69.44% 68.50% 74.67% 83.56% 72.94% 81.89% 81.48% 42.19% 54.29% 91.08% 89.25% 87.33% 90.38% Trattorie 7.98% 8.31% 7.11% 23.89% 17.38% 25.17% 26.89% 20.14% 11.73% 21.83% 13.79% 13.79% 53.89% 42.20% 5.44% 6.78% 8.79% 6.18% Ristoranti tipici 6.30% 7.31% 7.11% 3.91% 6.20% 5.39% 4.62% 5.19% 4.72% 5.23% 4.32% 4.74% 3.93% 3.51% 3.48% 3.97% 3.88% 3.43% TOTALE RISTORANTI RISTORANTI TIPICI E TRATTORIE 1190 3478 4212 2078 6561 3198 1569 1988 3242 1171 3815 2744 1263 1019 85.71% 10.20% 4.08% 1029 SEUGI21, 18 June 2003 N. 11 89.64% 7.57% 2.79% 251 301 225 718 919

ITALIAN REGIONS: PERCENTAGES OF DIFFERENT TYPES OF RESTAURANTS SEUGI21, 18 June 2003 N. 12

STEP 1: DATA FROM RESTAURANT GUIDES! Apart from the classification of restaurants into 4 different categories, Giallo Dat@ include all types of restaurants! Therefore the second step has been to consider restaurant guides commonly available in bookshops.! Indeed this kind of guides, even if with different criteria, reviews only high quality restaurants! The guides (2002 edition) that have been considered are:! Gambero Rosso! Espresso! Michelin! Veronelli! Slowfood! Accademia! Touring Club! As for Giallo Dat@, the number of restaurants for each Italian region has been taken into account SEUGI21, 18 June 2003 N. 13

Restaurant guides- Distribution of Restaurants among Italian Regions Regioni Abruzzo Basilicata Calabria Campania Emilia Romagna Friuli Venezia Giulia Lazio Liguria Lombardia Marche Molise Piemonte Puglia Sardegna Sicilia Toscana Trentino Alto Adige Umbria Valle d'aosta Veneto Gambero Rosso 69 35 51 128 220 89 235 127 316 82 16 200 101 70 151 266 95 55 23 163 Accademia Veronelli Slow Food Michelin Espresso Giallo Data 100 24 57 62 57 25 11 26 17 26 42 18 42 44 47 113 100 63 114 50 278 140 110 313 207 95 57 87 75 86 141 94 68 180 215 126 123 53 186 164 315 256 112 522 324 97 50 86 77 73 31 7 13 11 18 242 177 139 308 230 82 58 60 94 132 66 31 42 63 88 90 80 68 106 173 270 204 136 366 216 79 89 48 92 79 99 32 42 66 65 Ristoranti + Ristoranti Tipici + Trattorie 1190 301 919 2744 3478 1171 4212 2078 6561 1263 225 3198 1569 1019 1988 3242 1029 718 19 17 14 25 40 251 SEUGI21, 18 June 2003 N. 14 314 139 112 332 198 3815

Restaurant guides- Distribution of Restaurants among Italian Regions SEUGI21, 18 June 2003 N. 15

Restaurant guides- Distribution of Restaurants among Italian Regions SEUGI21, 18 June 2003 N. 16

Restaurant guides- Distribution of Restaurants among Italian Regions! Piedmont appears in the first positions in terms of number of THQRs reviewed by the guides! The densities of THQRs, computed as the ratio between: - the number of THQRs reviewed by the guide - the number of restaurants listed in GIALLODAT@ confirms the relevant percentages of THQRs with respect to all the restaurants in Piedmont SEUGI21, 18 June 2003 N. 17

Restaurant guides- Densities SEUGI21, 18 June 2003 N. 18

Restaurant guides- Densities SEUGI21, 18 June 2003 N. 19

Restaurant guides- Absolute values vs. densities Numbe r of Re staurants Densities SEUGI21, 18 June 2003 N. 20

STEP 2: IN DEPTH ANALYSIS OF SOME RESTAURAUNT GUIDES! Veronelli Editor, Slowfood Editor and Touring Club provided their restaurant guides in electronic format (more precisely the chapter concerning Piedmont)! The availability of the electronic format of the guides opens the way to the use of multivariate statistical methods and tools, including state-of-the-art software for text mining! The pages below briefly summarize the achieved results using standard statistical tools and text mining tools on Veronelli s and Slowfood s guide. SEUGI21, 18 June 2003 N. 21

I Ristoranti di Veronelli SEUGI21, 18 June 2003 N. 22

La guida di Veronelli some statistics Veronelli distributions over the territory N: 51 %: 30 N: 34 %: 20 N: 24 %: 14 N: 25 %: 15 N: 4 %: 2 N: 13 %: 8 N: 11 %: 7 N: 6 %: 4 The guide reviews 168 restaurants of Piedmont (2002 edition) SEUGI21, 18 June 2003 N. 23

I Ristoranti di Veronelli Prices distribution The price (wine excluded) goes between 15 and 83 Mean price is around 39 SEUGI21, 18 June 2003 N. 24

I Ristoranti di Veronelli Covers distribution Almost 35% of restaurants has a number of places between 33 and 45 SEUGI21, 18 June 2003 N. 25

La guida di Veronelli the menu 7 and 8 are the most common values for the number of specialties served in these restaurants SEUGI21, 18 June 2003 N. 26

I Ristoranti di Veronelli Some indicators 14 restaurants out of 168 receive 3 chef s hats Num. of chef s hats Almost 55% of restaurants puts particular attention to wine and receives from 1 to 3 bottles Num. of bottles SEUGI21, 18 June 2003 N. 27

From data analysis to text analysis! Restaurant guides contain both quantitative information, like prices and number of covers, and qualitative information (the description of restaurants)! Multivariate statistical methods for cluster analysis combined with tools for text mining give the possibility to explore the huge amount of information hidden in textual descriptions! The restaurant descriptions which are published on Veronelli and SlowFood books have been studied and classified! The classification is based on both textual and quantitative information using text mining techniques from SAS Text Mining Solution. SEUGI21, 18 June 2003 N. 28

Typical structure of a restaurant guide Restaurant guides contain both quantitative information and qualitative information Quantitative data include prices, number of seats, opening hours Classical statistical methods are suitable to mine them Text describes, in an unstructured way, the restaurant. Recent advances in text mining give the possibility to explore it. SEUGI21, 18 June 2003 N. 29

Text Mining What is it?! Discovering and using knowledge that exists in a document collection as a whole Knowledge SEUGI21, 18 June 2003 N. 30

SAS unique position in Text Mining Text Mining is Data Mining Gartner 2002 " Transform unstructured data into structured data Text Parsing " Reduce Dimension of data while keeping relevant information " Integrate new structured data with traditional structured data for data mining SEUGI21, 18 June 2003 N. 31

Text Mining Quantitative and qualitative data can be jointly analysed, providing a better view of the world of restaurants Organised Textual data SEUGI21, 18 June 2003 N. 32

Text Mining Process Reading text files Text parsing Dimension reduction Text analysis SEUGI21, 18 June 2003 N. 33

Text Mining Process! Textual data are transformed into frequency matrix of words inside the documents. This data can be integrated with quantitative information and used in a Data Mining process. Word Doc1 Doc2 Doc3 Doc4 word1 1 0 3 4 word2 2 4 0 6 word3 0 0 9 2 word4 2 7 12 8 Document SEUGI21, 18 June 2003 N. 34

Clustering Cluster procedures allow to group all the restaurants SEUGI21, 18 June 2003 N. 35

I ristoranti di Veronelli! In the next few pages some results of the text mining process on the Ristoranti di Veronelli guide are briefly synthesized! The main objective has been to divide all the restaurants into a manageable number of different clusters! Both quantitative and qualitative characteristics have been taken into account to describe each restaurant! Each cluster is homogeneous in the sense that it contains all the restaurants that have similar characteristics SEUGI21, 18 June 2003 N. 43

I Ristoranti di Veronelli - Cluster Analysis Using clustering procedures, the 168 restaurants have been divided into 6 clusters 1 - Spacious Typical Restaurants Many seats Good prices Wide selection of courses and desserts Local wines Local dishes 4 - Creative Cooking Few seats Few courses Confectionery No parking facilities Reservation adviced 6 - Prestigious Restaurants High prices Very high vote and many chef s hats Excellent wines Confectionery Wide selection of wines and spirits Few seats 2 Novelty Not typical 3 - Typical and Rural Restaurants Local wines Wide selection of cheeses Good prices Soups 5 Very Good Typical Restaurants Good prices High vote and many chef s hats Excellent local wines Confectionery Good choice of cheeses and spirits In this context typical means closed to the values of territory in which the restaurant lives SEUGI21, 18 June 2003 N. 44

Restaurants distribution in clusters CLUSTER It is easy to look at all the restaurants that belong to a group In this example we have selected group 6 (prestigious restaurants) SEUGI21, 18 June 2003 N. 45

Means grid plot from Veronelli inputs Creative Cooking Restaurants To understand which are the distinguishing features of cluster 4, a comparison between the cluster 4 values and the population values is carried out for all the variables. Indeed the input means grid plot compares the data set input means (in blue) to those from cluster 4 (in violet). We defined cluster 4 as Creative Cooking because the value of this variable is higher than the overall restaurants. Besides, the variables col1-col60 represent words frequency in documents. Indeed, the analysis are made by using both textual and quantitative information. It is easier to look at textual analysis in a different way (see below). SEUGI21, 18 June 2003 N. 46

Means grid plot from Veronelli inputs Very good Typical Restaurants A similar procedure has been adopted for all the clusters We defined cluster 5 as Very good Typical restaurants because the restaurants that belong to this group tend to be defined as typical in the guide ( Typical=si is higher than the overall restaurants). Also the number of chef s hats is large. SEUGI21, 18 June 2003 N. 47

Categorical Variables Another way to look at clusters is as follows. The slice variable displays the categorical variable typical (cyan). The height variable displays the interval variable vote. The cluster 2 ( Novelty ) has no typical restaurants and the cluster with the highest vote from Veronelli is number 6 ( Prestigius Restaurants ). SEUGI21, 18 June 2003 N. 48

Restaurants profiles It could be also useful to compare clusters among them The table displays the percentage of categorical variables in clusters The table displays min, max and mean values of the interval variables in clusters SEUGI21, 18 June 2003 N. 49

Words frequencies in clusters Clusters can be compared also from textual point of view. chocolate elegant The table displays for each cluster the percentage of words frequencies in cluster. fillet oven Highest frequencies Lowest frequencies nuts piedmontese SEUGI21, 18 June 2003 N. 50

Osterie d Italia - Slowfood SEUGI21, 18 June 2003 N. 51

Osterie d Italia Some statistics Osterie d Italia 2002 - Slowfood! The Osterie d Italia guide reviews 139 restaurants of Piedmont (2002 edition)! Langhe, Roero and Monferrato confirm their importance in terms of concentration of restaurants 1 esercizio restaurant 2 esercizi restaurants 3 esercizi restaurants 9 esercizi restaurants SEUGI21, 18 June 2003 N. 52

Osterie d Italia Prices distribution The price (wine excluded) goes between 14 and 33. Mean price is around 25 SEUGI21, 18 June 2003 N. 53

Osterie d Italia covers distribution 50% of osterie has a number of covers between 45 and 80 SEUGI21, 18 June 2003 N. 54

Osterie d Italia the menu 7 is the most common value for the number of specialties served in an Osteria SEUGI21, 18 June 2003 N. 55

Osterie d Italia the classification bottles A particular attention to wines distinguishes 63 restaurants snail 21 restaurants receive the snail cheeses Cheese plays a major role in the menu of 31 restaurants novelty 21 restaurants are new entries for this guide SEUGI21, 18 June 2003 N. 56

Osterie d Italia Clusters Analysis 1 Cheap Osteria New Osteria Many seats Good prices 2 Summer Season Many external seats High prices Specialized in starters, cheeses, cakes, creams and wines 3 Typical Osteria Few seats Good prices New Osteria Few dishes choice Specialized in starters and second meat dishes 4 - Snail Restaurants High prices Snail Excellent cheeses, pasta and sweets Few seats SEUGI21, 18 June 2003 N. 57

Restaurant distribution in clusters Easy identification of restaurants belonging to a group In this example we have selected group 2 (Summer Season) SEUGI21, 18 June 2003 N. 58

Means grid plot from Slow Food inputs Summer Season The input means grid plot compares the data set input means (_ALL_) to those from cluster 2. We defined cluster 2 as Summer Season because the value external seats is higher than the overall restaurants. SEUGI21, 18 June 2003 N. 59

Means grid plot from Slow Food inputs Snail Restaurants The input means grid plot compares the data set input means (_ALL_) to those from cluster 4. We defined cluster 4 as Snail Restaurants because the restaurants of this group tend to get the snail ( chiocciola=1 is higher than the overall restaurants). SEUGI21, 18 June 2003 N. 60

Categorical Variables The row variable displays the categorical variable Chiocciola (snail). The slice variable displays the categorical variable Bottles. The height variable displays the interval variable Prices. The cluster with the highest prices from Slow Food is number 4. SEUGI21, 18 June 2003 N. 61

Restaurants profiles The table displays the percentage of categorical variables in clusters The table displays min, max and mean values of the interval variables in clusters SEUGI21, 18 June 2003 N. 62

Words frequencies in clusters starters meat classic rabbit cream cakes greens summer cheeses The table displays for each cluster the percentage of words frequencies in clusters. Highest frequencies Lowest frequencies SEUGI21, 18 June 2003 N. 63

Top Restaurants Survey Some Results 64

Top restaurants survey Ristoranti Espresso, Michelin, Gambero Rosso 1 restaurant esercizio 2 restaurants esercizi esercizi 8 restaurants! Top restaurants were defined according to the following procedure: 1. Three guides (2002 Edition) have been chosen " Espresso, " Gambero Rosso, " Michelin 2. A restaurant is considered top if it appears in at least two of the above guides! Using the definition above 66 Top Restaurants were identified in Piedmont! They were asked to fill up three questionnaires (Jun, Sep, Oct)! The rate of response has been close to 50% SEUGI21, 18 June 2003 N. 65

Top restaurants survey The most suggested Menu is first time menu and then a la carte menu. The most common food style is the fusion style followed by the local style. SEUGI21, 18 June 2003 N. 66

Top restaurants survey The average number of covers is about 62, but only the 12% of restaurants has more than 100 places. Most of the restaurants are elegant and lovely. SEUGI21, 18 June 2003 N. 67

Top restaurants survey The floor staff counts on an average of 2.94 people. The kitchen staff counts on an average of 3.50 people. SEUGI21, 18 June 2003 N. 68

Top restaurants survey In floor staff there are very high qualified people like the sommellier (25%), the maitre (15%) and the commis (15%). In kitchen staff there are very high qualified people like the chef (25%), the cook (15%) and the pastry cook (15%). SEUGI21, 18 June 2003 N. 69

Top restaurants survey The 25% of clients come from abroad: Switzerland and Germany are the first countries. Most of the Italian people come from Piedmont and from Lombardy. SEUGI21, 18 June 2003 N. 70

Top restaurants survey Saturday Sunday Saturday Sunday Saturday Sunday Sunday Saturday Saturday Friday Sunday Friday Sunday Friday Saturday Saturday Saturday Sunday Friday Sunday In June the net use of covers available was less than 50%. In September net use of the covers increased significantly. In particular, during Saturday evenings, peak values of more than 70% were registered. SEUGI21, 18 June 2003 N. 71

Top restaurants survey Covers net use in October Sabato In October, during Saturday evenings, net use of covers is always above 70%. Venerdì Domenica Venerdì Domenica Sabato Venerdì Domenica Sabato Venerdì Domenica Sabato SEUGI21, 18 June 2003 N. 72

Conclusion 73

Future work! To extend the analysis to other guides - the guide from Touring Club has already been acquired in electronic format - Editors that would like to provide their guides are welcome!! To complete end extend the Top restaurants survey, including interviews to customers SEUGI21, 18 June 2003 N. 74

The team and the software tools Agenzia Regionale per la promozione Turistica del Piemonte Osservatorio Turistico Regionale Roberto Fontana Cristina Bergonzo Emanuela Giorgini SAS software Sabina Silani Sara Galli Michela Giacomini Enterprise Miner Enterprise Guide Francesca Martinengo Cristina Giraudo SEUGI21, 18 June 2003 N. 75

Acknowledgments Authors wish to thank:! Slowfood Editore, Veronelli Editore and Touring Club Editore, for having provided the electronic format of the sections of their guides related to Piedmont! Giallo Dat@ for having provided the number of restaurants published in their book, divided by Italian province! All the top restaurants owners for having filled up the questionnaire SEUGI21, 18 June 2003 N. 76

Contact! To ask for further information please feel free to contact Roberto Fontana Osservatorio Turistico Regione Piemonte Via Magenta 12 I-10128 TORINO Italy roberto.fontana@regione.piemonte.it Phone: +39.011.432.2479 SEUGI21, 18 June 2003 N. 77

THANKS FOR YOUR ATTENTION!