- PDF Free Download

Size: px

Start display at page:

Download ""

Mitchell Maximilian Morrison
10 years ago
Views:

1 Future Proof Analytics Techniques for Web 2.0 Applications Near Real Time Support Techniques Constantine J. Aivalis* - Technological Education Institute Crete, Anthony C. Boucouvalas ** - University of Peloponnese Tripoli s: *[email protected], **[email protected]

2 Keywords Analytics E-Commerce Web Analytics Real Time Log File Analysis Tagging Systems RIA 2

3 Contents of the Presentation Introduction Web Analytics Comparison of Methodologies The Problem The Solution Architecture Functionality Results Customer Behavioral Model Graph Measurements Current Work Applications Conclusion 3

4 Introduction WWW is today's common business platform. E-Commerce infrastructure must be reliable, robust and scalable. Web systems produce huge amounts of user activity data that often stay unused. User activity data must be converted to information. Intelligent Customer classification allows better customized services and increases sales. 4

5 Analytics As a Component 5

6 Basic Models of Analytics Technologies Log File Analysis Systems, Tagged Systems, Network Data Collection Devices and Hybrid Systems. 6

7 Web Analytics Analysis of log files Log files contain very detailed information about each request. The data have to be carefully selected. Page Tagging Page Tagging requires an extra web server, to whom the visitors browser is automatically sent. This server collects the log data generated by this visit and stores it to a specific data base for each site, based on an account number. Network Data Collection Devices Sniffers, Black boxes that capture IP packages. Hybrid Methods Combine Analysis of log files and Tagging, in order to reduce the disadvantages of each method. 7

8 Available Vendors Google Analytics (Urchin) Microsoft adcenter Analytics (DeepMetrix) Yahoo Web Analytics (indextools) Clickstream.com Adobe Web Analytics (Omniture) IBM Unica NetInsight ChartBeat Inc. Hitmatic 8

9 Comparison of Methodologies Source: Brian Clifton Web Traffic Data Sources & Vendor Comparison 9

10 Google Analytics JavaScript <meta name="google-site-verification" content="xxxxxxxxxxxx" /> <script type="text/javascript"> var _gaq = _gaq []; _gaq.push(['_setaccount', 'UA-XXXXXXXX-Y']); _gaq.push(['_trackpageview']); (function() { var ga = document.createelement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = (' == document.location.protocol? ' : ' + analytics.com/ga.js'; var s = document.getelementsbytagname('script )[0]; s.parentnode.insertbefore(ga, s); })(); </script> '.google- 10

11 The Problem E-shops often operate in blind folded fashion. Only successful sales transactions are visible to the administration and management. Most e-commerce systems have no built-in performance measuring mechanisms. Only registered-customer actions are taken into consideration. Visitor majority may not be customers yet. Their behavior has to be analyzed in order to win them. Access log files include all interaction data details. Manual access log file scrutinizing is too inconvenient to be performed on regular basis. 11

12 Rotating Access Log Files 12

13 Access Log file Sample 13

14 Initial Architecture of the System 14

15 Initial Functionality of the System 15

16 Extended System 16

17 GWT Applications Access Log "GET /LogDB/ HTTP/1.1" 304 /LogDB/ 117 "GET /LogDB/LogDB.css HTTP/1.1" 304 /LogDB/LogDB.css 19 "GET /LogDB/com.art.logdb.LogDB/com.art.logdb.LogDB.nocache.js HTTP/1.1" "GET /LogDB/com.art.logdb.LogDB/1D630481E14BBD07DE7ED3D963A012CE.cache.html HTTP/1.1" "POST /LogDB/com.art.logdb.LogDB/MySQLConnection HTTP/1.1" "POST /LogDB/com.art.logdb.LogDB/MySQLConnection HTTP/1.1" "POST /LogDB/com.art.logdb.LogDB/MySQLConnection HTTP/1.1" "POST /LogDB/com.art.logdb.LogDB/MySQLConnection HTTP/1.1" "POST /LogDB/com.art.logdb.LogDB/MySQLConnection HTTP/1.1"

18 MySQL General Log File :55:20 3 Connect root@localhost on w2p 3 Query /* mysql-connector-java ( Revision: ${bzr.revision-id} ) */SHOW VARIABLES WHERE Variable_name ='language' OR Variable_name = 'net_write_timeout' OR Variable_name = 'interactive_timeout' OR Variable_name = 'wait_timeout' OR Variable_name = 'character_set_client' OR Variable_name = 'character_set_connection' OR Variable_name = 'character_set' OR Variable_name = 'character_set_server' OR Variable_name = 'tx_isolation' OR Variable_name = 'transaction_isolation' OR Variable_name = 'character_set_results' OR Variable_name = 'timezone' OR Variable_name = 'time_zone' OR Variable_name = 'system_time_zone' OR Variable_name = 'lower_case_table_names' OR Variable_name = 'max_allowed_packet' OR Variable_name = 'net_buffer_length' OR Variable_name = 'sql_mode' OR Variable_name = 'query_cache_type' OR Variable_name = 'query_cache_size' OR Variable_name = 'init_connect' :55:21 3 Query /* mysql-connector-java ( Revision: ${bzr.revision-id} ) 3 Query SHOW COLLATION 3 Query SET NAMES utf8mb4 3 Query SET character_set_results = NULL 3 Query SET autocommit=1 3 Query select * from user where username = 'asdasd' and password = 'aa' 1 Query show global status :55:25 1 Query show global status :55:30 3 Query select * from user where username = 'costis' and password = 'foobar' Future Proof Analytics Techniques for Web 2.0 Applications 18

19 Near Real Time Analytics Tools 19

20 Components & Data Flow 20

21 Trigger for Order Producer CREATE TABLE `transmitted` ( `transmittedid` int(11) NOT NULL AUTO_INCREMENT, òrdernumber` int(11) DEFAULT NULL, `transmitted` bit(1) DEFAULT NULL, PRIMARY KEY (`transmittedid`) ) ENGINE=InnoDB AUTO_INCREMENT=27 DEFAULT CHARSET=utf8; USE `kona5502`; DELIMITER $$ CREATE TRIGGER `fixorders` AFTER INSERT ON òrders` FOR EACH ROW -- trigger body BEGIN insert into `kona5502`.`transmitted` values (null, NEW.òrders_id`, 0); END 21

22 Data Queuing Sequence Diagram 22

23 Customer Behavioral Model Graph 23

24 Measurements 24

25 Measurements Order Values/Numbers Visits Time spent per product or service Accesses per product or service Orders per Product or service Bots visited Visitors Uncompleted ordering sessions Profitable customer groups Profitable products or services Overall profits Promotion impact Speed Performance metrics 25

26 Current Research Bot Behavioral Analysis Bot Classification Use off ibeacons in order to allow Analytics Techniques to be applied to physical stores Predictive Algorithms 26

27 Thank you Constantine J. Aivalis Anthony Boucouvalas 27

Is Your Google Analytics Data Accurate?

Is Your Google Analytics Data Accurate? September 18, 2013 Presented By Amin Shawki Analytics Manager Andy Gibson Digital Marketing Analyst 1. 1 Billion+ pageviews/year in sites analyzed and supported