BIG DATA MEET PHARMACEUTICAL INDUSTRY: AN

Size: px
Start display at page:

Download "BIG DATA MEET PHARMACEUTICAL INDUSTRY: AN"

Transcription

1 BIG DATA MEET PHARMACEUTICAL INDUSTRY: AN APPLICATION ON SOCIAL MEDIA DATA Caterina Liberati 1, Paolo Mariani 1 1 Catrina Liberati, Paolo Mariani, Department of Economics, Management and Statistics, University of Milano-Bicocca caterina.liberati@unimib.it; paolo.mariani@unimib.it KEYWORDS: Data Analysis; Big Data, Pharmaceutical industry. 1 An introduction Big Data represent the new frontier of data analysis. There is still substantial confusion between having a lot of data available and operate on Big Data. In order to clarify such misunderstanding, we can take into account seven characteristics that allow us to highlight differences and to uncover peculiarities of each type of data (Table 1): Table 1: Data vs Big Data: the 7 V V Data Big Data Volume Megabyte MB 10 6 Zettabyte ZB Velocity Static Real time Variety Structured and rarely integrated from different sources Structured e unstructured- Not integrated. Collected from different sources Value High Not verified Veridicity High Low Validity High Limited and with high obsolescence due to time Nowadays businesses are attempting to employ Big Data in their operative contexts because they recognize the innovative and strategic aspects of such source of information. The pharmaceutical industry, for example, although has a classical perspective in manipulating data, is exploring this new contest (Santoro, E., 2009). Inspired by a report produced by Cubeyou on Facebook data, referred to the pharmaceutical sector (Cubeyou, 2014), we analyzed microdata applying statistical technique for sparse matrices reducing. The present work describes the survey, the data collected and the modeling employed, which is based on a pre-process done

2 with Principal Component Analysis. Finally, part of results is shown in the last section. 2 The Goal and the Data The goal of the Cubeyou report is to help businesses understand which are their customers or potential ones and what they do. It also highlights how helping business in structuring marketing activities and how to make informed marketing decisions in the areas of media and content. In such report, instances observed were selected among users of the social media pages, websites and forums that write about drugs and health. Some pages visited are: Bristol - Myers Squibb; Amgen; Boehringer Ingelheim; Schering Plough; Baxter International; Takeda Pharmaceutical; AIFA, just to cite a few. The research was conducted on 5607 Italian subjects, at the end of 2014, hunting all interactions between people and brands, products and services (shares, likes, tweets, pins, posts, ) on Facebook, and classifies them into categories. The algorithm employed in the report transforms the data in meaningful customer information as personalities, psychographics, location, hobbies, interests, media (Kosinski et al, 2013) and it has been derived on dataset set including Facebook Likes provided by over 58,000 volunteers, together with their detailed demographic profiles, and the results of several psychometric tests. Cubeyou raw data are stored on a cloud platform with 20 servers active on Amazon Web Services (AWS) infrastructure. Thanks to Hadoop2 and Hbase, more than 5TB distributed database can be stored and updated daily. Three main tables are used to store different types of data. The behavioural table with more than 8 billions rows stores users actions on Facebook pages. Each row contains a single user by page interaction. The User demographic table contains unstructured data about users profiles. The Pages demographic table contains unstructured data about Facebook pages. Starting from this infrastructure, for each project data are then extracted with queries based on users profile and behaviour. Extraction process aggregates data by Facebook users and creates a user by variable matrix stored in a structured dataframe. Therefore the logistic/linear regression employed to predict individual psycho-demographic profiles 1 benefits from such external survey responses. The ambitious goal of our work is to propose a strategy to explore and synthesize the same raw data set but using only the Facebook information in order to aid businesses to make their strategic decisions in terms of communication or activities and targets. The matrix analysed has 5607 rows (Facebook users) and 159 dummy columns (pages visited/liked). In order to reduce the dimensionality of the data we exploited the subjects classification which distinguish among: Pet Lovers; Outdoor 1 Psychographics classifies people, using their interests, attitudes, habits, values and opinions, not only on their objective demographic characteristics to better understand what drives them or could drive them to purchase and engage with the company. It is based on the assumption that the types of products and brands an individual purchases will reflect who that person is and how he lives.

3 Enthusiast; Techies; Car Lovers; Book Lovers; Social Activist; Gamers; Movie Lovers; Politically Active; Sport Lovers; Fashion Lovers; Music Lovers; Travel Lovers; Public Figures Followers; Food Lovers; Home Decorators-amp DIYs; Beauty and Wellness Aware; Business People; Housekeepers. In our application we focus on the selection of public figures usable to address a media campaign. 3 Methodology and results Before considering techniques or models we have to face with the question: Are such data Big Data? Yes, if we consider the origin, not if we consider that present format. The data, actually, have been collapsed in a matrix using a map reduce technique (like Hadoop) and now are completely manageable. The second question to consider is: How to elaborate this data set? We suggest performing a pre-processing as if the data have a big volume. The data matrix dimensions have been reduced, generating a contingency table of 'like'. Therefore, we obtained 19 psychographics profiles (in row) and 140 topics (in column). In this way we have squeeze the data volume. An alternative choice could have been the Multiple Correspondence Analysis but the results interpretation might have been difficult. On this new data set we performed a Principal Component Analysis (Bolasco S., 2010) 2, retaining only those factors with eigenvalue greater or equal to 1. In such way we generated a principal plane which explain the 93.65% with KMO equal to and Bartlet equal to The first factor named Hedonism (47,97 % of explained inertia) reports: Outdoor Enthusiast; Car Lovers; Gamers; Movie Lovers; Sports Lovers; Music Lovers; Beauty and wellness Aware. The second factor, called Commitment, (45.68 % of explained inertia) reports: Pet Lovers; Techies; Book Lovers; Activist company; Politically Active; Business People; Home Decorators. Other profiles as Housekeepers; Public Figures Followers; Fashion Lovers; Food Lovers; Travel Lovers are transversal respect to the 2 axis. Using only the Italian public figures we can explain the peculiarities of the social media pages users, websites and forums that show an interest in drugs and health. In the first quadrant of the factor plane we found: Marco Travaglio; Fiorello; Beppe Grillo, Luciana Litizzetto. Hedonism and Commitment are both positive. Hedonism negative and Commitment positive, the second quadrant is filled in by Gino Strada; Papa Francesco; Massimo Gramellini. Hedonism and Commitment are both negative into third quadrant: Sonia Peronaci e Giulio Golia. Hedonism positive and Commitment negative in the fourth quadrant filled in by BAZ Marco Bazzoni; Belen Rodriguez; Paolo Bitta; Alessia Marcuzzi; Alessandro Borghese (Figure 1).

4 Figure 1: ACP Plan Hedonism vs Commitment (Source: Elaboration on Cubeyou data November 2014) The choice of the testimonial by a business will take into account the psychographic profiles preferences here visualized, in order to provide coherent brand image. 4 Further developments Social data offer a set of information associated with an user, which can be declared, if they are spontaneous self-profiling by the user (sex, age, work place) or gathered, if they are derived from the interactions with other contents, business pages, users etc (Moubarak G. et alii, 2010). Social data are Big Data when the seven V are respected. In our case the application was performed after preelaborations, and then the usage of standard statistical models were legitimated. Our results confirm how Big Data can meet pharmaceutical industry and highlight how small is the cluster of people interested in pharmaceutical environment. Into this cluster most of the users are interested in wellness in general and not in drugs or pathologies. We limited the analysis to the public figures but the further information available from the survey could be modeled at a later stage. The results illustrated are very interesting, especially if we consider the context: pharmaceutical industries are attempting to measure adherence on therapies and R&D area. To this purpose marketing department (Greene j. A., Kesselheim, A. S. 2010) of such companies are focusing their attention to Big Data. References BOLASCO, S Analisi multidimensional dei dati. Metodi, strategie e criteri di interpretazione. Roma: Carocci

5 Cubeyou 2014, The Pharmaceutical Industry How Pharmaceutical Market Customers Benchmark Against Average Italian People. Industry Report, 11/2014, Cubeyou Inc., Redwood City, CA, USA GREENE J. A., KESSELHEIM, A. S Pharmaceutical Marketing and the New Social Media. The New England Journal of Medicine, november 25. KOSINSKI M., STILLWELL D. & GRAEPEL, T Private traits and attributes are predictable from digital records of human behaviour. MOUBARAK G., GUIOT A., BENHAMOU Y. & HARIRI S Relationship and its impact on the doctor-patient Facebook activity of residents and fellows. J Med Ethics. doi: /jme SANTORO, E Web 2.0 e Medicina: come social network, podcast, wiki e blog trasformano la comunicazione, l assistenza e la formazione in sanità. Milano: Il pensiero scientifico Editore.

International Journal of Advancements in Research & Technology, Volume 3, Issue 5, May-2014 18 ISSN 2278-7763. BIG DATA: A New Technology

International Journal of Advancements in Research & Technology, Volume 3, Issue 5, May-2014 18 ISSN 2278-7763. BIG DATA: A New Technology International Journal of Advancements in Research & Technology, Volume 3, Issue 5, May-2014 18 BIG DATA: A New Technology Farah DeebaHasan Student, M.Tech.(IT) Anshul Kumar Sharma Student, M.Tech.(IT)

More information

Keywords Big Data; OODBMS; RDBMS; hadoop; EDM; learning analytics, data abundance.

Keywords Big Data; OODBMS; RDBMS; hadoop; EDM; learning analytics, data abundance. Volume 4, Issue 11, November 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Analytics

More information

Online Media Kit 2014-FCC_OnlineMediaKit 12/4/2014 8:56 AM Page 1 nline Odvertising A

Online Media Kit 2014-FCC_OnlineMediaKit 12/4/2014 8:56 AM Page 1 nline Odvertising A Online Advertising The Network Forum Communications Company online advertising network includes 46 websites across Minnesota, Wisconsin, North Dakota, and South Dakota. Our entire network reaches 2+ Million

More information

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB Planet Size Data!? Gartner s 10 key IT trends for 2012 unstructured data will grow some 80% over the course of the next

More information

Transforming the Telecoms Business using Big Data and Analytics

Transforming the Telecoms Business using Big Data and Analytics Transforming the Telecoms Business using Big Data and Analytics Event: ICT Forum for HR Professionals Venue: Meikles Hotel, Harare, Zimbabwe Date: 19 th 21 st August 2015 AFRALTI 1 Objectives Describe

More information

5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014

5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014 5 Keys to Unlocking the Big Data Analytics Puzzle Anurag Tandon Director, Product Marketing March 26, 2014 1 A Little About Us A global footprint. A proven innovator. A leader in enterprise analytics for

More information

DATA MINING TECHNIQUES SUPPORT TO KNOWLEGDE OF BUSINESS INTELLIGENT SYSTEM

DATA MINING TECHNIQUES SUPPORT TO KNOWLEGDE OF BUSINESS INTELLIGENT SYSTEM INTERNATIONAL JOURNAL OF RESEARCH IN COMPUTER APPLICATIONS AND ROBOTICS ISSN 2320-7345 DATA MINING TECHNIQUES SUPPORT TO KNOWLEGDE OF BUSINESS INTELLIGENT SYSTEM M. Mayilvaganan 1, S. Aparna 2 1 Associate

More information

Alexander Nikov. 5. Database Systems and Managing Data Resources. Learning Objectives. RR Donnelley Tries to Master Its Data

Alexander Nikov. 5. Database Systems and Managing Data Resources. Learning Objectives. RR Donnelley Tries to Master Its Data INFO 1500 Introduction to IT Fundamentals 5. Database Systems and Managing Data Resources Learning Objectives 1. Describe how the problems of managing data resources in a traditional file environment are

More information

Introduction to Predictive Analytics. Dr. Ronen Meiri ronen@dmway.com

Introduction to Predictive Analytics. Dr. Ronen Meiri ronen@dmway.com Introduction to Predictive Analytics Dr. Ronen Meiri Outline From big data to predictive analytics Predictive Analytics vs. BI Intelligent platforms What can we do with it. The modeling process. Example

More information

Outline. What is Big data and where they come from? How we deal with Big data?

Outline. What is Big data and where they come from? How we deal with Big data? What is Big Data Outline What is Big data and where they come from? How we deal with Big data? Big Data Everywhere! As a human, we generate a lot of data during our everyday activity. When you buy something,

More information

Advanced Big Data Analytics with R and Hadoop

Advanced Big Data Analytics with R and Hadoop REVOLUTION ANALYTICS WHITE PAPER Advanced Big Data Analytics with R and Hadoop 'Big Data' Analytics as a Competitive Advantage Big Analytics delivers competitive advantage in two ways compared to the traditional

More information

Statistics for BIG data

Statistics for BIG data Statistics for BIG data Statistics for Big Data: Are Statisticians Ready? Dennis Lin Department of Statistics The Pennsylvania State University John Jordan and Dennis K.J. Lin (ICSA-Bulletine 2014) Before

More information

DIGITAL MARKETING STRATEGIES Leveraging The Back-End Tools

DIGITAL MARKETING STRATEGIES Leveraging The Back-End Tools DIGITAL MARKETING STRATEGIES Leveraging The Back-End Tools Professional Background RACING INDUSTRY EXPERIENCE: First Job Out of Undergrad: - Arlington Park, Assistant to the VP of Marketing - Sponsorship

More information

Doing Multidisciplinary Research in Data Science

Doing Multidisciplinary Research in Data Science Doing Multidisciplinary Research in Data Science Assoc.Prof. Abzetdin ADAMOV CeDAWI - Center for Data Analytics and Web Insights Qafqaz University aadamov@qu.edu.az http://ce.qu.edu.az/~aadamov 16 May

More information

Foundations of Business Intelligence: Databases and Information Management

Foundations of Business Intelligence: Databases and Information Management Foundations of Business Intelligence: Databases and Information Management Problem: HP s numerous systems unable to deliver the information needed for a complete picture of business operations, lack of

More information

COMP9321 Web Application Engineering

COMP9321 Web Application Engineering COMP9321 Web Application Engineering Semester 2, 2015 Dr. Amin Beheshti Service Oriented Computing Group, CSE, UNSW Australia Week 11 (Part II) http://webapps.cse.unsw.edu.au/webcms2/course/index.php?cid=2411

More information

Big Data Analytics. Prof. Dr. Lars Schmidt-Thieme

Big Data Analytics. Prof. Dr. Lars Schmidt-Thieme Big Data Analytics Prof. Dr. Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Institute of Computer Science University of Hildesheim, Germany 33. Sitzung des Arbeitskreises Informationstechnologie,

More information

HIGH PERFORMANCE ANALYTICS FOR TERADATA

HIGH PERFORMANCE ANALYTICS FOR TERADATA F HIGH PERFORMANCE ANALYTICS FOR TERADATA F F BORN AND BRED IN FINANCIAL SERVICES AND HEALTHCARE. DECADES OF EXPERIENCE IN PARALLEL PROGRAMMING AND ANALYTICS. FOCUSED ON MAKING DATA SCIENCE HIGHLY PERFORMING

More information

Open source large scale distributed data management with Google s MapReduce and Bigtable

Open source large scale distributed data management with Google s MapReduce and Bigtable Open source large scale distributed data management with Google s MapReduce and Bigtable Ioannis Konstantinou Email: ikons@cslab.ece.ntua.gr Web: http://www.cslab.ntua.gr/~ikons Computing Systems Laboratory

More information

PRACTICAL DATA MINING IN A LARGE UTILITY COMPANY

PRACTICAL DATA MINING IN A LARGE UTILITY COMPANY QÜESTIIÓ, vol. 25, 3, p. 509-520, 2001 PRACTICAL DATA MINING IN A LARGE UTILITY COMPANY GEORGES HÉBRAIL We present in this paper the main applications of data mining techniques at Electricité de France,

More information

Industry 4.0 and Big Data

Industry 4.0 and Big Data Industry 4.0 and Big Data Marek Obitko, mobitko@ra.rockwell.com Senior Research Engineer 03/25/2015 PUBLIC PUBLIC - 5058-CO900H 2 Background Joint work with Czech Institute of Informatics, Robotics and

More information

Business IntelliSENSE Social and Internal Data Marriage for Smarter Decisions

Business IntelliSENSE Social and Internal Data Marriage for Smarter Decisions 2014 BA Convention Business IntelliSENSE Social and Internal Data Marriage for Smarter Decisions May 2014 2014 BA Convention 2 Big Click Data to edit Drivers Master text styles Big Data Analytics is a

More information

Copyright 2014. This report and/or appended material may not be partly or completely published or

Copyright 2014. This report and/or appended material may not be partly or completely published or Aalborg University Copenhagen Semester: 4 th Title: Personalized Medicine based on patient journals and family medical history records Aalborg University Copenhagen A.C. Meyers Vænge 15 2450 København

More information

How to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning

How to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning How to use Big Data in Industry 4.0 implementations LAURI ILISON, PhD Head of Big Data and Machine Learning Big Data definition? Big Data is about structured vs unstructured data Big Data is about Volume

More information

Suresh Lakavath csir urdip Pune, India lsureshit@gmail.com.

Suresh Lakavath csir urdip Pune, India lsureshit@gmail.com. A Big Data Hadoop Architecture for Online Analysis. Suresh Lakavath csir urdip Pune, India lsureshit@gmail.com. Ramlal Naik L Acme Tele Power LTD Haryana, India ramlalnaik@gmail.com. Abstract Big Data

More information

Two-Phase Data Warehouse Optimized for Data Mining

Two-Phase Data Warehouse Optimized for Data Mining Two-Phase Data Warehouse Optimized for Data Mining Balázs Rácz András Lukács Csaba István Sidló András A. Benczúr Data Mining and Web Search Research Group Computer and Automation Research Institute Hungarian

More information

Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013

Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013 Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013 Housekeeping 1. Any questions coming out of today s presentation can be discussed in the bar this evening 2. OCF is

More information

CAP4773/CIS6930 Projects in Data Science, Fall 2014 [Review] Overview of Data Science

CAP4773/CIS6930 Projects in Data Science, Fall 2014 [Review] Overview of Data Science CAP4773/CIS6930 Projects in Data Science, Fall 2014 [Review] Overview of Data Science Dr. Daisy Zhe Wang CISE Department University of Florida August 25th 2014 20 Review Overview of Data Science Why Data

More information

BIG DATA Alignment of Supply & Demand Nuria de Lama Representative of Atos Research &

BIG DATA Alignment of Supply & Demand Nuria de Lama Representative of Atos Research & BIG DATA Alignment of Supply & Demand Nuria de Lama Representative of Atos Research & Innovation 04-08-2011 to the EC 8 th February, Luxembourg Your Atos business Research technologists. and Innovation

More information

An analysis of Big Data ecosystem from an HCI perspective.

An analysis of Big Data ecosystem from an HCI perspective. An analysis of Big Data ecosystem from an HCI perspective. Jay Sanghvi Rensselaer Polytechnic Institute For: Theory and Research in Technical Communication and HCI Rensselaer Polytechnic Institute Wednesday,

More information

The Data Mining Process

The Data Mining Process Sequence for Determining Necessary Data. Wrong: Catalog everything you have, and decide what data is important. Right: Work backward from the solution, define the problem explicitly, and map out the data

More information

Big Data. Fast Forward. Putting data to productive use

Big Data. Fast Forward. Putting data to productive use Big Data Putting data to productive use Fast Forward What is big data, and why should you care? Get familiar with big data terminology, technologies, and techniques. Getting started with big data to realize

More information

Chapter 6. Foundations of Business Intelligence: Databases and Information Management

Chapter 6. Foundations of Business Intelligence: Databases and Information Management Chapter 6 Foundations of Business Intelligence: Databases and Information Management VIDEO CASES Case 1a: City of Dubuque Uses Cloud Computing and Sensors to Build a Smarter, Sustainable City Case 1b:

More information

OPINION MINING IN PRODUCT REVIEW SYSTEM USING BIG DATA TECHNOLOGY HADOOP

OPINION MINING IN PRODUCT REVIEW SYSTEM USING BIG DATA TECHNOLOGY HADOOP OPINION MINING IN PRODUCT REVIEW SYSTEM USING BIG DATA TECHNOLOGY HADOOP 1 KALYANKUMAR B WADDAR, 2 K SRINIVASA 1 P G Student, S.I.T Tumkur, 2 Assistant Professor S.I.T Tumkur Abstract- Product Review System

More information

QLIKVIEW DEPLOYMENT FOR BIG DATA ANALYTICS AT KING.COM

QLIKVIEW DEPLOYMENT FOR BIG DATA ANALYTICS AT KING.COM QLIKVIEW DEPLOYMENT FOR BIG DATA ANALYTICS AT KING.COM QlikView Technical Case Study Series Big Data June 2012 qlikview.com Introduction This QlikView technical case study focuses on the QlikView deployment

More information

A New Era Of Analytic

A New Era Of Analytic Penang egovernment Seminar 2014 A New Era Of Analytic Megat Anuar Idris Head, Project Delivery, Business Analytics & Big Data Agenda Overview of Big Data Case Studies on Big Data Big Data Technology Readiness

More information

BIG DATA What it is and how to use?

BIG DATA What it is and how to use? BIG DATA What it is and how to use? Lauri Ilison, PhD Data Scientist 21.11.2014 Big Data definition? There is no clear definition for BIG DATA BIG DATA is more of a concept than precise term 1 21.11.14

More information

The big data business model: opportunity and key success factors

The big data business model: opportunity and key success factors MENA Summit 2013: Enabling innovation, driving profitability The big data business model: opportunity and key success factors 6 November 2013 Justin van der Lande EVENT PARTNERS: 2 Introduction What is

More information

Data Warehouse (DW) Maturity Assessment Questionnaire

Data Warehouse (DW) Maturity Assessment Questionnaire Data Warehouse (DW) Maturity Assessment Questionnaire Catalina Sacu - csacu@students.cs.uu.nl Marco Spruit m.r.spruit@cs.uu.nl Frank Habers fhabers@inergy.nl September, 2010 Technical Report UU-CS-2010-021

More information

Beyond GDP and new indicators of well-being: is it possible to sum up objective and subjective data in the perspective of constructing new indicators?

Beyond GDP and new indicators of well-being: is it possible to sum up objective and subjective data in the perspective of constructing new indicators? Beyond GDP and new indicators of well-being: is it possible to sum up objective and subjective data in the perspective of constructing new indicators? Filomena Maggino Università degli Studi di Firenze

More information

Raul F. Chong Senior program manager Big data, DB2, and Cloud IM Cloud Computing Center of Competence - IBM Toronto Lab, Canada

Raul F. Chong Senior program manager Big data, DB2, and Cloud IM Cloud Computing Center of Competence - IBM Toronto Lab, Canada What is big data? Raul F. Chong Senior program manager Big data, DB2, and Cloud IM Cloud Computing Center of Competence - IBM Toronto Lab, Canada 1 2011 IBM Corporation Agenda The world is changing What

More information

NoSQL Databases. Institute of Computer Science Databases and Information Systems (DBIS) DB 2, WS 2014/2015

NoSQL Databases. Institute of Computer Science Databases and Information Systems (DBIS) DB 2, WS 2014/2015 NoSQL Databases Institute of Computer Science Databases and Information Systems (DBIS) DB 2, WS 2014/2015 Database Landscape Source: H. Lim, Y. Han, and S. Babu, How to Fit when No One Size Fits., in CIDR,

More information

Mining Text Data for Useful Information in Higher Education John Zilvinskis Indiana University

Mining Text Data for Useful Information in Higher Education John Zilvinskis Indiana University Mining Text Data for Useful Information in Higher Education John Zilvinskis Indiana University Institutional Researchers Credo We have not succeeded in answering all our problems indeed we sometimes feel

More information

CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing. University of Florida, CISE Department Prof.

CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing. University of Florida, CISE Department Prof. CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing University of Florida, CISE Department Prof. Daisy Zhe Wang Data Science Overview Why, What, How, Who Outline Why Data Science?

More information

Big Data Technologies Compared June 2014

Big Data Technologies Compared June 2014 Big Data Technologies Compared June 2014 Agenda What is Big Data Big Data Technology Comparison Summary Other Big Data Technologies Questions 2 What is Big Data by Example The SKA Telescope is a new development

More information

Monitis Project Proposals for AUA. September 2014, Yerevan, Armenia

Monitis Project Proposals for AUA. September 2014, Yerevan, Armenia Monitis Project Proposals for AUA September 2014, Yerevan, Armenia Distributed Log Collecting and Analysing Platform Project Specifications Category: Big Data and NoSQL Software Requirements: Apache Hadoop

More information

Foundations of Business Intelligence: Databases and Information Management

Foundations of Business Intelligence: Databases and Information Management Foundations of Business Intelligence: Databases and Information Management Content Problems of managing data resources in a traditional file environment Capabilities and value of a database management

More information

The Next Wave of Data Management. Is Big Data The New Normal?

The Next Wave of Data Management. Is Big Data The New Normal? The Next Wave of Data Management Is Big Data The New Normal? Table of Contents Introduction 3 Separating Reality and Hype 3 Why Are Firms Making IT Investments In Big Data? 4 Trends In Data Management

More information

Business Intelligence. A Presentation of the Current Lead Solutions and a Comparative Analysis of the Main Providers

Business Intelligence. A Presentation of the Current Lead Solutions and a Comparative Analysis of the Main Providers 60 Business Intelligence. A Presentation of the Current Lead Solutions and a Comparative Analysis of the Main Providers Business Intelligence. A Presentation of the Current Lead Solutions and a Comparative

More information

Big Data and Analytics: Challenges and Opportunities

Big Data and Analytics: Challenges and Opportunities Big Data and Analytics: Challenges and Opportunities Dr. Amin Beheshti Lecturer and Senior Research Associate University of New South Wales, Australia (Service Oriented Computing Group, CSE) Talk: Sharif

More information

Sentiment Analysis. D. Skrepetos 1. University of Waterloo. NLP Presenation, 06/17/2015

Sentiment Analysis. D. Skrepetos 1. University of Waterloo. NLP Presenation, 06/17/2015 Sentiment Analysis D. Skrepetos 1 1 Department of Computer Science University of Waterloo NLP Presenation, 06/17/2015 D. Skrepetos (University of Waterloo) Sentiment Analysis NLP Presenation, 06/17/2015

More information

dm106 TEXT MINING FOR CUSTOMER RELATIONSHIP MANAGEMENT: AN APPROACH BASED ON LATENT SEMANTIC ANALYSIS AND FUZZY CLUSTERING

dm106 TEXT MINING FOR CUSTOMER RELATIONSHIP MANAGEMENT: AN APPROACH BASED ON LATENT SEMANTIC ANALYSIS AND FUZZY CLUSTERING dm106 TEXT MINING FOR CUSTOMER RELATIONSHIP MANAGEMENT: AN APPROACH BASED ON LATENT SEMANTIC ANALYSIS AND FUZZY CLUSTERING ABSTRACT In most CRM (Customer Relationship Management) systems, information on

More information

Hadoop Technology for Flow Analysis of the Internet Traffic

Hadoop Technology for Flow Analysis of the Internet Traffic Hadoop Technology for Flow Analysis of the Internet Traffic Rakshitha Kiran P PG Scholar, Dept. of C.S, Shree Devi Institute of Technology, Mangalore, Karnataka, India ABSTRACT: Flow analysis of the internet

More information

OLAP and Data Mining. Data Warehousing and End-User Access Tools. Introducing OLAP. Introducing OLAP

OLAP and Data Mining. Data Warehousing and End-User Access Tools. Introducing OLAP. Introducing OLAP Data Warehousing and End-User Access Tools OLAP and Data Mining Accompanying growth in data warehouses is increasing demands for more powerful access tools providing advanced analytical capabilities. Key

More information

The 3 questions to ask yourself about BIG DATA

The 3 questions to ask yourself about BIG DATA The 3 questions to ask yourself about BIG DATA Do you have a big data problem? Companies looking to tackle big data problems are embarking on a journey that is full of hype, buzz, confusion, and misinformation.

More information

ROME, 17-10-2013 BIG DATA ANALYTICS

ROME, 17-10-2013 BIG DATA ANALYTICS ROME, 17-10-2013 BIG DATA ANALYTICS BIG DATA FOUNDATIONS Big Data is #1 on the 2012 and the 2013 list of most ambiguous terms - Global language monitor 2 BIG DATA FOUNDATIONS Big Data refers to data sets

More information

METADATA DRIVEN INTEGRATED STATISTICAL DATA PROCESSING AND DISSEMINATION SYSTEM

METADATA DRIVEN INTEGRATED STATISTICAL DATA PROCESSING AND DISSEMINATION SYSTEM METADATA DRIVEN INTEGRATED STATISTICAL DATA PROCESSING AND DISSEMINATION SYSTEM By Karlis Zeila Central Statistical Bureau of Latvia Abstract The aim of this report is to introduce participants with the

More information

Chapter 1. Contrasting traditional and visual analytics approaches

Chapter 1. Contrasting traditional and visual analytics approaches Chapter 1 Understanding Big Data Analytics In This Chapter Defining Big Data Understanding Big Data Analytics Contrasting traditional and visual analytics approaches The era of Big Data is upon us. The

More information

IT and CRM A basic CRM model Data source & gathering system Database system Data warehouse Information delivery system Information users

IT and CRM A basic CRM model Data source & gathering system Database system Data warehouse Information delivery system Information users 1 IT and CRM A basic CRM model Data source & gathering Database Data warehouse Information delivery Information users 2 IT and CRM Markets have always recognized the importance of gathering detailed data

More information

Volume 3, Issue 8, August 2015 International Journal of Advance Research in Computer Science and Management Studies

Volume 3, Issue 8, August 2015 International Journal of Advance Research in Computer Science and Management Studies Volume 3, Issue 8, August 2015 International Journal of Advance Research in Computer Science and Management Studies Research Article / Survey Paper / Case Study Available online at: www.ijarcsms.com An

More information

So What s the Big Deal?

So What s the Big Deal? So What s the Big Deal? Presentation Agenda Introduction What is Big Data? So What is the Big Deal? Big Data Technologies Identifying Big Data Opportunities Conducting a Big Data Proof of Concept Big Data

More information

Open & Big Data for Life Imaging Technical aspects : existing solutions, main difficulties. Pierre Mouillard MD

Open & Big Data for Life Imaging Technical aspects : existing solutions, main difficulties. Pierre Mouillard MD Open & Big Data for Life Imaging Technical aspects : existing solutions, main difficulties Pierre Mouillard MD What is Big Data? lots of data more than you can process using common database software and

More information

Course 103402 MIS. Foundations of Business Intelligence

Course 103402 MIS. Foundations of Business Intelligence Oman College of Management and Technology Course 103402 MIS Topic 5 Foundations of Business Intelligence CS/MIS Department Organizing Data in a Traditional File Environment File organization concepts Database:

More information

How To Handle Big Data With A Data Scientist

How To Handle Big Data With A Data Scientist III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution

More information

Big Data a threat or a chance?

Big Data a threat or a chance? Big Data a threat or a chance? Helwig Hauser University of Bergen, Dept. of Informatics Big Data What is Big Data? well, lots of data, right? we come back to this in a moment. certainly, a buzz-word but

More information

You should have a working knowledge of the Microsoft Windows platform. A basic knowledge of programming is helpful but not required.

You should have a working knowledge of the Microsoft Windows platform. A basic knowledge of programming is helpful but not required. What is this course about? This course is an overview of Big Data tools and technologies. It establishes a strong working knowledge of the concepts, techniques, and products associated with Big Data. Attendees

More information

The Scientific Data Mining Process

The Scientific Data Mining Process Chapter 4 The Scientific Data Mining Process When I use a word, Humpty Dumpty said, in rather a scornful tone, it means just what I choose it to mean neither more nor less. Lewis Carroll [87, p. 214] In

More information

Visual Data mining SAS/SPECTRAVIEW Software

Visual Data mining SAS/SPECTRAVIEW Software Visual Data mining SAS/SPECTRAVIEW Software :HOFRPH Annie Postic / Bengt Bengtsson SAS Institute Introduction SAS/SPECTRAVIEW software Advanced Visualization Technology! Data Mining Turning data into profits!,qwurgxfwlrq

More information

Oracle Big Data SQL Technical Update

Oracle Big Data SQL Technical Update Oracle Big Data SQL Technical Update Jean-Pierre Dijcks Oracle Redwood City, CA, USA Keywords: Big Data, Hadoop, NoSQL Databases, Relational Databases, SQL, Security, Performance Introduction This technical

More information

Big Data: Study in Structured and Unstructured Data

Big Data: Study in Structured and Unstructured Data Big Data: Study in Structured and Unstructured Data Motashim Rasool 1, Wasim Khan 2 mail2motashim@gmail.com, khanwasim051@gmail.com Abstract With the overlay of digital world, Information is available

More information

Customer Classification And Prediction Based On Data Mining Technique

Customer Classification And Prediction Based On Data Mining Technique Customer Classification And Prediction Based On Data Mining Technique Ms. Neethu Baby 1, Mrs. Priyanka L.T 2 1 M.E CSE, Sri Shakthi Institute of Engineering and Technology, Coimbatore 2 Assistant Professor

More information

Web Archiving and Scholarly Use of Web Archives

Web Archiving and Scholarly Use of Web Archives Web Archiving and Scholarly Use of Web Archives Helen Hockx-Yu Head of Web Archiving British Library 15 April 2013 Overview 1. Introduction 2. Access and usage: UK Web Archive 3. Scholarly feedback on

More information

Data Visualization Techniques

Data Visualization Techniques Data Visualization Techniques From Basics to Big Data with SAS Visual Analytics WHITE PAPER SAS White Paper Table of Contents Introduction.... 1 Generating the Best Visualizations for Your Data... 2 The

More information

PEER REVIEW HISTORY ARTICLE DETAILS VERSION 1 - REVIEW. Elizabeth Comino Centre fo Primary Health Care and Equity 12-Aug-2015

PEER REVIEW HISTORY ARTICLE DETAILS VERSION 1 - REVIEW. Elizabeth Comino Centre fo Primary Health Care and Equity 12-Aug-2015 PEER REVIEW HISTORY BMJ Open publishes all reviews undertaken for accepted manuscripts. Reviewers are asked to complete a checklist review form (http://bmjopen.bmj.com/site/about/resources/checklist.pdf)

More information

DATA WAREHOUSE E KNOWLEDGE DISCOVERY

DATA WAREHOUSE E KNOWLEDGE DISCOVERY DATA WAREHOUSE E KNOWLEDGE DISCOVERY Prof. Fabio A. Schreiber Dipartimento di Elettronica e Informazione Politecnico di Milano DATA WAREHOUSE (DW) A TECHNIQUE FOR CORRECTLY ASSEMBLING AND MANAGING DATA

More information

Knowledge Discovery from patents using KMX Text Analytics

Knowledge Discovery from patents using KMX Text Analytics Knowledge Discovery from patents using KMX Text Analytics Dr. Anton Heijs anton.heijs@treparel.com Treparel Abstract In this white paper we discuss how the KMX technology of Treparel can help searchers

More information

Chapter 6 8/12/2015. Foundations of Business Intelligence: Databases and Information Management. Problem:

Chapter 6 8/12/2015. Foundations of Business Intelligence: Databases and Information Management. Problem: Foundations of Business Intelligence: Databases and Information Management VIDEO CASES Chapter 6 Case 1a: City of Dubuque Uses Cloud Computing and Sensors to Build a Smarter, Sustainable City Case 1b:

More information

Comparison of Non-linear Dimensionality Reduction Techniques for Classification with Gene Expression Microarray Data

Comparison of Non-linear Dimensionality Reduction Techniques for Classification with Gene Expression Microarray Data CMPE 59H Comparison of Non-linear Dimensionality Reduction Techniques for Classification with Gene Expression Microarray Data Term Project Report Fatma Güney, Kübra Kalkan 1/15/2013 Keywords: Non-linear

More information

Big Data Analytics and Healthcare

Big Data Analytics and Healthcare Big Data Analytics and Healthcare Anup Kumar, Professor and Director of MINDS Lab Computer Engineering and Computer Science Department University of Louisville Road Map Introduction Data Sources Structured

More information

By Ken Thompson, ServQ Alliance

By Ken Thompson, ServQ Alliance Instant Customer Analytics using only Excel By Ken Thompson, ServQ Alliance There are some excellent specialized data analytics, data mining and data visualization tools available however we should not

More information

Understanding Your Customer Journey by Extending Adobe Analytics with Big Data

Understanding Your Customer Journey by Extending Adobe Analytics with Big Data SOLUTION BRIEF Understanding Your Customer Journey by Extending Adobe Analytics with Big Data Business Challenge Today s digital marketing teams are overwhelmed by the volume and variety of customer interaction

More information

DEMYSTIFYING BIG DATA. What it is, what it isn t, and what it can do for you.

DEMYSTIFYING BIG DATA. What it is, what it isn t, and what it can do for you. DEMYSTIFYING BIG DATA What it is, what it isn t, and what it can do for you. JAMES LUCK BIO James Luck is a Data Scientist with AT&T Consulting. He has 25+ years of experience in data analytics, in addition

More information

Green Big Data. A Green IT / Green IS Perspective on Big Data

Green Big Data. A Green IT / Green IS Perspective on Big Data Green Big Data A Green IT / Green IS Perspective on Big Data Agenda 1. Starting Point and Research Question 2. Subject of Analysis 3. Research Methodology 4. Results 5. Conclusion Green Big Data 2 Starting

More information

BIG DATA CHALLENGES AND PERSPECTIVES

BIG DATA CHALLENGES AND PERSPECTIVES BIG DATA CHALLENGES AND PERSPECTIVES Meenakshi Sharma 1, Keshav Kishore 2 1 Student of Master of Technology, 2 Head of Department, Department of Computer Science and Engineering, A P Goyal Shimla University,

More information

Can we Analyze all the Big Data we Collect?

Can we Analyze all the Big Data we Collect? DBKDA/WEB Panel 2015, Rome 28.05.2015 DBKDA Panel 2015, Rome, 27.05.2015 Reutlingen University Can we Analyze all the Big Data we Collect? Moderation: Fritz Laux, Reutlingen University, Germany Panelists:

More information

Oracle Advanced Analytics 12c & SQLDEV/Oracle Data Miner 4.0 New Features

Oracle Advanced Analytics 12c & SQLDEV/Oracle Data Miner 4.0 New Features Oracle Advanced Analytics 12c & SQLDEV/Oracle Data Miner 4.0 New Features Charlie Berger, MS Eng, MBA Sr. Director Product Management, Data Mining and Advanced Analytics charlie.berger@oracle.com www.twitter.com/charliedatamine

More information

Foundations of Business Intelligence: Databases and Information Management

Foundations of Business Intelligence: Databases and Information Management Foundations of Business Intelligence: Databases and Information Management Wienand Omta Fabiano Dalpiaz 1 drs. ing. Wienand Omta Learning Objectives Describe how the problems of managing data resources

More information

The primary goal of this thesis was to understand how the spatial dependence of

The primary goal of this thesis was to understand how the spatial dependence of 5 General discussion 5.1 Introduction The primary goal of this thesis was to understand how the spatial dependence of consumer attitudes can be modeled, what additional benefits the recovering of spatial

More information

Data Visualization Techniques

Data Visualization Techniques Data Visualization Techniques From Basics to Big Data with SAS Visual Analytics WHITE PAPER SAS White Paper Table of Contents Introduction.... 1 Generating the Best Visualizations for Your Data... 2 The

More information

Big Data in the Nordics 2012

Big Data in the Nordics 2012 Big Data in the Nordics 2012 A survey about increasing data volumes and Big Data analysis among private and governmental organizations in Sweden, Norway, Denmark and Finland. Unexplored Big Data Potential

More information

Big Data from a Database Theory Perspective

Big Data from a Database Theory Perspective Big Data from a Database Theory Perspective Martin Grohe Lehrstuhl Informatik 7 - Logic and the Theory of Discrete Systems A CS View on Data Science Applications Data System Users 2 Us Data HUGE heterogeneous

More information

Ubuntu and Hadoop: the perfect match

Ubuntu and Hadoop: the perfect match WHITE PAPER Ubuntu and Hadoop: the perfect match February 2012 Copyright Canonical 2012 www.canonical.com Executive introduction In many fields of IT, there are always stand-out technologies. This is definitely

More information

EDM THE AUDIENCE ANALYSIS

EDM THE AUDIENCE ANALYSIS EDM THE AUDIENCE ANALYSIS Methodology Our goal was to understand the unique behavioral tendencies and topics of conversation among electronic dance music (EDM) fans. Additionally, we wanted to surface

More information

A PRACTICAL GUIDE TO MODERN MARKETING ANALYTICS

A PRACTICAL GUIDE TO MODERN MARKETING ANALYTICS A PRACTICAL GUIDE TO MODERN MARKETING ANALYTICS How marketing analytics becomes the next competitive weapon in building stronger customer relationships PUBLISHED BY US Headquarters StrongView Systems,

More information

DATA SCIENCE CURRICULUM WEEK 1 ONLINE PRE-WORK INSTALLING PACKAGES COMMAND LINE CODE EDITOR PYTHON STATISTICS PROJECT O5 PROJECT O3 PROJECT O2

DATA SCIENCE CURRICULUM WEEK 1 ONLINE PRE-WORK INSTALLING PACKAGES COMMAND LINE CODE EDITOR PYTHON STATISTICS PROJECT O5 PROJECT O3 PROJECT O2 DATA SCIENCE CURRICULUM Before class even begins, students start an at-home pre-work phase. When they convene in class, students spend the first eight weeks doing iterative, project-centered skill acquisition.

More information

BIG DATA TECHNOLOGY. Hadoop Ecosystem

BIG DATA TECHNOLOGY. Hadoop Ecosystem BIG DATA TECHNOLOGY Hadoop Ecosystem Agenda Background What is Big Data Solution Objective Introduction to Hadoop Hadoop Ecosystem Hybrid EDW Model Predictive Analysis using Hadoop Conclusion What is Big

More information

Big Data Buzzwords From A to Z. By Rick Whiting, CRN 4:00 PM ET Wed. Nov. 28, 2012

Big Data Buzzwords From A to Z. By Rick Whiting, CRN 4:00 PM ET Wed. Nov. 28, 2012 Big Data Buzzwords From A to Z By Rick Whiting, CRN 4:00 PM ET Wed. Nov. 28, 2012 Big Data Buzzwords Big data is one of the, well, biggest trends in IT today, and it has spawned a whole new generation

More information