Exploratory Data Analysis with #codemash

Size: px
Start display at page:

Download "Exploratory Data Analysis with R. @matthewrenze #codemash"

Transcription

1 Exploratory Data Analysis with #codemash

2 Motivation The ability to take data to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it that s going to be a hugely important skill in the next decades, because now we really do have essentially free and ubiquitous data. So the complimentary scarce factor is the ability to understand that data and extract value from it. Hal Varian, Google s Chief Economist The McKinsey Quarterly, Jan 2009

3 Source: Dice 2014 Tech Salary Survey Results

4 A Flood of Data is Coming... Source: Source: Wikipedia Sink or Swim

5 Overview Introduction to R Data Munging Descriptive Statistics Data Visualization Beyond R and EDA

6 How Does This Apply to Me? As a software developer, I often: Perform log file analysis Analyze software performance Analyze code metrics for code quality Detect anomalies in source data Transform or clean data files to make them usable Help decision makers make decisions based on data

7 About Me Independent software consultant Education B.S. in Computer Science B.A. in Philosophy Community Pluralsight Author ASPInsider Public Speaker Open-Source Software

8 Introduction to R

9 What is R? Open source Language and environment Numerical and graphical analysis Cross platform

10 What is R? Active development Large user community Modular and extensible extensions and best of all

11 FREE

12 FREE

13 Source:

14

15 Source:

16 Code Demo

17 Data Munging

18 Data Munging Transforming data Raw data to usable data Data must be cleaned first

19 Data Munging Tasks Renaming variables Data type conversion Encoding, decoding, or recoding data Merging data sets Transforming data Handling missing data (imputing) Handling anomalous values

20 Loading Data in R File-based data Web-based data Databases Statistical data And many more CSV XML

21 Cleaning Data This step is often the: Most difficult Most time consuming TIP: Record all steps

22

23

24 Column with wrong name Rows with missing values Runtime column has units Revenue in multiple scales Wrong file format

25 Code Demo

26

27 Descriptive Statistics

28 Descriptive Statistics Describe data Provides a summary aka: Summary statistics Movie Runtime Statistic Value (minutes) Minimum 38 1 st Quartile 93 Median 101 Mean rd Quartile 113 Maximum 219

29 Statistical Terms Observations Variables Qualitative variable Quantitative variable ID Date Customer Product Quantity John Pizza John Soda Jill Salad Jill Milk Miko Pizza Miko Soda Sam Pizza Sam Milk 1

30 Number of Variables Types of Analysis Number of variables Univariate Bivariate Multivariate Qualitative Univariate Analysis Quantitative Univariate Analysis Type of variables Qualitative Quantitative Qualitative Bivariate Analysis Qualitative & Quantitative Bivariate Analysis Quantitative Bivariate Analysis Multivariate Analysis Type of Variable(s)

31 Univariate Analysis One variable Qualitative Frequency Quantitative Central tendency Dispersion

32 Bivariate Analysis Qualitative Joint frequency Quantitative Two variables Predictor Outcome Measures Covariance Correlation

33

34 Cowboys & Space Invaders: The Musical Extended Edition

35 Code Demo

36 Cowboys & Space Invaders: The Musical Extended Edition

37 Data Visualization

38 Data Visualization Visual data representation For human pattern recognition Map dimensions to visual characteristics

39 Data Visualization ID Date Customer Product Quantity John Pizza John Soda Jill Salad Jill Milk Miko Pizza Miko Soda Sam Pizza Sam Milk 1

40 Number of Variables Types of Data Visualizations Number of variables Univariate Bivariate Multivariate Qualitative Univariate Analysis Quantitative Univariate Analysis Type of variable(s) Qualitative Quantitative Qualitative Bivariate Analysis Qualitative & Quantitative Bivariate Analysis Quantitative Bivariate Analysis Multivariate Analysis Type of Variable(s)

41 Cowboys & Space Invaders: The Musical Extended Edition

42 Code Demo

43 The Warlord of the Rings : An Unexpected Adventure Feature Length PG

44 Beyond R and EDA

45 This is just the tip of the iceberg! This is just the tip of the iceberg!

46 Advanced Data Analysis with R Cluster Analysis Statistical Modeling Dimensionality Reduction Analysis of Variance (ANOVA)

47 Source: Nathan Yau (

48 Data Mining and Machine Learning with R

49 Alternatives to R for EDA

50 Source: Microsoft

51

52

53

54

55

56 Where to Go Next R website: R Studio: Revolutions: Flowing Data: R-Blogger:

57

58 Conclusion

59 Conclusion Introduction to R Data munging Descriptive statistics Data visualization Beyond R & EDA

60 Feedback Feedback is very important to me! One thing you liked? One thing I could improve?

61 Contact Info Matthew Renze Website:

d3.js Data-Driven Documents Scott Murray, Jerome Cukier & Jeffrey Heer VisWeek 2012 Tutorial

d3.js Data-Driven Documents Scott Murray, Jerome Cukier & Jeffrey Heer VisWeek 2012 Tutorial d3.js Data-Driven Documents Scott Murray, Jerome Cukier & Jeffrey Heer VisWeek 2012 Tutorial How much data (bytes) did we produce in 2010? 2010: 1,200 exabytes Gantz et al, 2008, 2010 2010: 1,200 exabytes

More information

Lecture 2: Descriptive Statistics and Exploratory Data Analysis

Lecture 2: Descriptive Statistics and Exploratory Data Analysis Lecture 2: Descriptive Statistics and Exploratory Data Analysis Further Thoughts on Experimental Design 16 Individuals (8 each from two populations) with replicates Pop 1 Pop 2 Randomly sample 4 individuals

More information

Introduction to Data Visualization

Introduction to Data Visualization Introduction to Data Visualization STAT 133 Gaston Sanchez Department of Statistics, UC Berkeley gastonsanchez.com github.com/gastonstat/stat133 Course web: gastonsanchez.com/teaching/stat133 Graphics

More information

?????? Data Analytics

?????? Data Analytics ?????? Data Analytics Prof. Dr.-Ing. Lars Linsen Prof. Dr. Adalbert FX Wilhelm Fall 2015 0. Organizational Stuff 0.1 Syllabus and Organization Data Analytics 3 Course website http://www.faculty.jacobsuniversity.de/llinsen/teaching/??????.htm

More information

research topics in Data Visualization Jeffrey Heer Stanford University

research topics in Data Visualization Jeffrey Heer Stanford University research topics in Data Visualization Jeffrey Heer Stanford University Set A Set B Set C Set D X Y X Y X Y X Y 10 8.04 10 9.14 10 7.46 8 6.58 8 6.95 8 8.14 8 6.77 8 5.76 13 7.58 13 8.74 13 12.74 8 7.71

More information

Silvermine House Steenberg Office Park, Tokai 7945 Cape Town, South Africa Telephone: +27 21 702 4666 www.spss-sa.com

Silvermine House Steenberg Office Park, Tokai 7945 Cape Town, South Africa Telephone: +27 21 702 4666 www.spss-sa.com SPSS-SA Silvermine House Steenberg Office Park, Tokai 7945 Cape Town, South Africa Telephone: +27 21 702 4666 www.spss-sa.com SPSS-SA Training Brochure 2009 TABLE OF CONTENTS 1 SPSS TRAINING COURSES FOCUSING

More information

business statistics using Excel OXFORD UNIVERSITY PRESS Glyn Davis & Branko Pecar

business statistics using Excel OXFORD UNIVERSITY PRESS Glyn Davis & Branko Pecar business statistics using Excel Glyn Davis & Branko Pecar OXFORD UNIVERSITY PRESS Detailed contents Introduction to Microsoft Excel 2003 Overview Learning Objectives 1.1 Introduction to Microsoft Excel

More information

4.1 Exploratory Analysis: Once the data is collected and entered, the first question is: "What do the data look like?"

4.1 Exploratory Analysis: Once the data is collected and entered, the first question is: What do the data look like? Data Analysis Plan The appropriate methods of data analysis are determined by your data types and variables of interest, the actual distribution of the variables, and the number of cases. Different analyses

More information

A Prototype System for Educational Data Warehousing and Mining 1

A Prototype System for Educational Data Warehousing and Mining 1 A Prototype System for Educational Data Warehousing and Mining 1 Nikolaos Dimokas, Nikolaos Mittas, Alexandros Nanopoulos, Lefteris Angelis Department of Informatics, Aristotle University of Thessaloniki

More information

Avigdor Gal Technion Israel Institute of Technology

Avigdor Gal Technion Israel Institute of Technology Avigdor Gal Technion Israel Institute of Technology Tutorial Big data integration Applications of big data integration Current challenges and future research directions Big data is a game changer From

More information

Data Analysis and Statistical Software Workshop. Ted Kasha, B.S. Kimberly Galt, Pharm.D., Ph.D.(c) May 14, 2009

Data Analysis and Statistical Software Workshop. Ted Kasha, B.S. Kimberly Galt, Pharm.D., Ph.D.(c) May 14, 2009 Data Analysis and Statistical Software Workshop Ted Kasha, B.S. Kimberly Galt, Pharm.D., Ph.D.(c) May 14, 2009 Learning Objectives: Data analysis commonly used today Available data analysis software packages

More information

11/17/2015. Learning Objectives. What Is Data Mining? Presentation. At the conclusion of this presentation, the learner will be able to:

11/17/2015. Learning Objectives. What Is Data Mining? Presentation. At the conclusion of this presentation, the learner will be able to: Basic Data Mining and Analysis for Program Integrity: A Primer for Physicians and Other Health Care Professionals Presentation Learning Objectives At the conclusion of this presentation, the learner will

More information

Strategies For Setting Up Your Organisation For Success With Big Data. Kevin Long Business Development Director Teradata

Strategies For Setting Up Your Organisation For Success With Big Data. Kevin Long Business Development Director Teradata Strategies For Setting Up Your Organisation For Success With Big Data Kevin Long Business Development Director Teradata Agenda Developing a big data strategy and plan that is aligned with your organisation

More information

SkySpark Tools for Visualizing and Understanding Your Data

SkySpark Tools for Visualizing and Understanding Your Data Issue 20 - March 2014 Tools for Visualizing and Understanding Your Data (Pg 1) Analytics Shows You How Your Equipment Systems are Really Operating (Pg 2) The Equip App Automatically organize data by equipment

More information

Chapter 3: Data Mining Driven Learning Apprentice System for Medical Billing Compliance

Chapter 3: Data Mining Driven Learning Apprentice System for Medical Billing Compliance Chapter 3: Data Mining Driven Learning Apprentice System for Medical Billing Compliance 3.1 Introduction This research has been conducted at back office of a medical billing company situated in a custom

More information

Additional sources Compilation of sources: http://lrs.ed.uiuc.edu/tseportal/datacollectionmethodologies/jin-tselink/tselink.htm

Additional sources Compilation of sources: http://lrs.ed.uiuc.edu/tseportal/datacollectionmethodologies/jin-tselink/tselink.htm Mgt 540 Research Methods Data Analysis 1 Additional sources Compilation of sources: http://lrs.ed.uiuc.edu/tseportal/datacollectionmethodologies/jin-tselink/tselink.htm http://web.utk.edu/~dap/random/order/start.htm

More information

Why Taking This Course? Course Introduction, Descriptive Statistics and Data Visualization. Learning Goals. GENOME 560, Spring 2012

Why Taking This Course? Course Introduction, Descriptive Statistics and Data Visualization. Learning Goals. GENOME 560, Spring 2012 Why Taking This Course? Course Introduction, Descriptive Statistics and Data Visualization GENOME 560, Spring 2012 Data are interesting because they help us understand the world Genomics: Massive Amounts

More information

Getting the global picture

Getting the global picture Jesús M. González Barahona, Gregorio Robles GSyC, Universidad Rey Juan Carlos, Madrid, Spain {jgb,grex}@gsyc.escet.urjc.es Oxford Workshop on Libre Software 2004 Oxford, UK, June 25th Overview 1 Overview

More information

Marketing Research Core Body Knowledge (MRCBOK ) Learning Objectives

Marketing Research Core Body Knowledge (MRCBOK ) Learning Objectives Fulfilling the core market research educational needs of individuals and companies worldwide Presented through a unique partnership between How to Contact Us: Phone: +1-706-542-3537 or 1-800-811-6640 (USA

More information

In this presentation, you will be introduced to data mining and the relationship with meaningful use.

In this presentation, you will be introduced to data mining and the relationship with meaningful use. In this presentation, you will be introduced to data mining and the relationship with meaningful use. Data mining refers to the art and science of intelligent data analysis. It is the application of machine

More information

GETTING STARTED WITH R AND DATA ANALYSIS

GETTING STARTED WITH R AND DATA ANALYSIS GETTING STARTED WITH R AND DATA ANALYSIS [Learn R for effective data analysis] LEARN PRACTICAL SKILLS REQUIRED FOR VISUALIZING, TRANSFORMING, AND ANALYZING DATA IN R One day course for people who are just

More information

DATA EXPERTS MINE ANALYZE VISUALIZE. We accelerate research and transform data to help you create actionable insights

DATA EXPERTS MINE ANALYZE VISUALIZE. We accelerate research and transform data to help you create actionable insights DATA EXPERTS We accelerate research and transform data to help you create actionable insights WE MINE WE ANALYZE WE VISUALIZE Domains Data Mining Mining longitudinal and linked datasets from web and other

More information

A GENERAL TAXONOMY FOR VISUALIZATION OF PREDICTIVE SOCIAL MEDIA ANALYTICS

A GENERAL TAXONOMY FOR VISUALIZATION OF PREDICTIVE SOCIAL MEDIA ANALYTICS A GENERAL TAXONOMY FOR VISUALIZATION OF PREDICTIVE SOCIAL MEDIA ANALYTICS Stacey Franklin Jones, D.Sc. ProTech Global Solutions Annapolis, MD Abstract The use of Social Media as a resource to characterize

More information

Introduction to Agile and Scrum

Introduction to Agile and Scrum Introduction to Agile and Scrum Matthew Renze @matthewrenze COMS 309 - Software Development Practices Purpose Intro to Agile and Scrum Prepare you for the industry Questions and answers Overview Intro

More information

Microsoft Services Exceed your business with Microsoft SharePoint Server 2010

Microsoft Services Exceed your business with Microsoft SharePoint Server 2010 Microsoft Services Exceed your business with Microsoft SharePoint Server 2010 Business Intelligence Suite Alexandre Mendeiros, SQL Server Premier Field Engineer January 2012 Agenda Microsoft Business Intelligence

More information

ETL Anatomy 101. Tom Miron Systems Seminar Consultants Madison, WI. An ETL-like process is key for many reporting and analysis systems

ETL Anatomy 101. Tom Miron Systems Seminar Consultants Madison, WI. An ETL-like process is key for many reporting and analysis systems ETL Anatomy 101 Tom Miron Systems Seminar Consultants Madison, WI 1 What Is ETL? Extract, Transform, Load ETL is often associated with Data Warehouse/Mart But An ETL-like process is key for many reporting

More information

Applying Process Mining to IT Big Data

Applying Process Mining to IT Big Data Approved for Public Release; Distribution Unlimited. 14-0899 Applying Process Mining to IT Big Data Richard F. Eng PRINCE2, PMP, CSQE, CRE, CQE 14 March 2014 The author s affiliation with The MITRE Corporation

More information

Week 1. Exploratory Data Analysis

Week 1. Exploratory Data Analysis Week 1 Exploratory Data Analysis Practicalities This course ST903 has students from both the MSc in Financial Mathematics and the MSc in Statistics. Two lectures and one seminar/tutorial per week. Exam

More information

DESCRIPTIVE STATISTICS AND EXPLORATORY DATA ANALYSIS

DESCRIPTIVE STATISTICS AND EXPLORATORY DATA ANALYSIS DESCRIPTIVE STATISTICS AND EXPLORATORY DATA ANALYSIS SEEMA JAGGI Indian Agricultural Statistics Research Institute Library Avenue, New Delhi - 110 012 seema@iasri.res.in 1. Descriptive Statistics Statistics

More information

Information Visualization WS 2013/14 11 Visual Analytics

Information Visualization WS 2013/14 11 Visual Analytics 1 11.1 Definitions and Motivation Lot of research and papers in this emerging field: Visual Analytics: Scope and Challenges of Keim et al. Illuminating the path of Thomas and Cook 2 11.1 Definitions and

More information

Improving the Performance of Data Mining Models with Data Preparation Using SAS Enterprise Miner Ricardo Galante, SAS Institute Brasil, São Paulo, SP

Improving the Performance of Data Mining Models with Data Preparation Using SAS Enterprise Miner Ricardo Galante, SAS Institute Brasil, São Paulo, SP Improving the Performance of Data Mining Models with Data Preparation Using SAS Enterprise Miner Ricardo Galante, SAS Institute Brasil, São Paulo, SP ABSTRACT In data mining modelling, data preparation

More information

Course Descriptions: Undergraduate/Graduate Certificate Program in Data Visualization and Analysis

Course Descriptions: Undergraduate/Graduate Certificate Program in Data Visualization and Analysis 9/3/2013 Course Descriptions: Undergraduate/Graduate Certificate Program in Data Visualization and Analysis Seton Hall University, South Orange, New Jersey http://www.shu.edu/go/dava Visualization and

More information

Cleaned Data. Recommendations

Cleaned Data. Recommendations Call Center Data Analysis Megaputer Case Study in Text Mining Merete Hvalshagen www.megaputer.com Megaputer Intelligence, Inc. 120 West Seventh Street, Suite 10 Bloomington, IN 47404, USA +1 812-0-0110

More information

Geostatistics Exploratory Analysis

Geostatistics Exploratory Analysis Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa Master of Science in Geospatial Technologies Geostatistics Exploratory Analysis Carlos Alberto Felgueiras cfelgueiras@isegi.unl.pt

More information

Tools and Analysis for Business Strategy. Course Philosophy. Background on the Analytics Revolution

Tools and Analysis for Business Strategy. Course Philosophy. Background on the Analytics Revolution Tools and Analysis for Business Strategy Professor Jason Snyder (jsnyder@anderson.ucla.edu) Mondays: 1:00-3:50 & 7:00-10:00 (C315) Office Hours: By appointment Course Philosophy In this class, students

More information

2015 Workshops for Professors

2015 Workshops for Professors SAS Education Grow with us Offered by the SAS Global Academic Program Supporting teaching, learning and research in higher education 2015 Workshops for Professors 1 Workshops for Professors As the market

More information

Creating Page Layouts using SharePoint Designer or Visual Studio

Creating Page Layouts using SharePoint Designer or Visual Studio Creating Page Layouts using SharePoint Designer or Visual Studio Becky Bertram MCSD, MCAD MCTS WSS Development MCTS MOSS Development www.beckybertram.com What is Web Content Management? According to Wikipedia:

More information

Exploratory Data Analysis for Ecological Modelling and Decision Support

Exploratory Data Analysis for Ecological Modelling and Decision Support Exploratory Data Analysis for Ecological Modelling and Decision Support Gennady Andrienko & Natalia Andrienko Fraunhofer Institute AIS Sankt Augustin Germany http://www.ais.fraunhofer.de/and 5th ECEM conference,

More information

Data Exploration Data Visualization

Data Exploration Data Visualization Data Exploration Data Visualization What is data exploration? A preliminary exploration of the data to better understand its characteristics. Key motivations of data exploration include Helping to select

More information

A Presenta*on from Big Data 22 February 2013

A Presenta*on from Big Data 22 February 2013 A Presenta*on from Big Data 22 February 2013 Big Data Analytics: avoiding the pitfalls with robust analytics Steve Cohen In4mation insights All copyright owned by The Future Place and the presenters of

More information

Microsoft Azure Machine learning Algorithms

Microsoft Azure Machine learning Algorithms Microsoft Azure Machine learning Algorithms Tomaž KAŠTRUN @tomaz_tsql Tomaz.kastrun@gmail.com http://tomaztsql.wordpress.com Our Sponsors Speaker info https://tomaztsql.wordpress.com Agenda Focus on explanation

More information

CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing. University of Florida, CISE Department Prof.

CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing. University of Florida, CISE Department Prof. CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing University of Florida, CISE Department Prof. Daisy Zhe Wang Data Science Overview Why, What, How, Who Outline Why Data Science?

More information

CAP4773/CIS6930 Projects in Data Science, Fall 2014 [Review] Overview of Data Science

CAP4773/CIS6930 Projects in Data Science, Fall 2014 [Review] Overview of Data Science CAP4773/CIS6930 Projects in Data Science, Fall 2014 [Review] Overview of Data Science Dr. Daisy Zhe Wang CISE Department University of Florida August 25th 2014 20 Review Overview of Data Science Why Data

More information

03 The full syllabus. 03 The full syllabus continued. For more information visit www.cimaglobal.com PAPER C03 FUNDAMENTALS OF BUSINESS MATHEMATICS

03 The full syllabus. 03 The full syllabus continued. For more information visit www.cimaglobal.com PAPER C03 FUNDAMENTALS OF BUSINESS MATHEMATICS 0 The full syllabus 0 The full syllabus continued PAPER C0 FUNDAMENTALS OF BUSINESS MATHEMATICS Syllabus overview This paper primarily deals with the tools and techniques to understand the mathematics

More information

BNG 202 Biomechanics Lab. Descriptive statistics and probability distributions I

BNG 202 Biomechanics Lab. Descriptive statistics and probability distributions I BNG 202 Biomechanics Lab Descriptive statistics and probability distributions I Overview The overall goal of this short course in statistics is to provide an introduction to descriptive and inferential

More information

Analytics on Big Data

Analytics on Big Data Analytics on Big Data Riccardo Torlone Università Roma Tre Credits: Mohamed Eltabakh (WPI) Analytics The discovery and communication of meaningful patterns in data (Wikipedia) It relies on data analysis

More information

Business Statistics. Successful completion of Introductory and/or Intermediate Algebra courses is recommended before taking Business Statistics.

Business Statistics. Successful completion of Introductory and/or Intermediate Algebra courses is recommended before taking Business Statistics. Business Course Text Bowerman, Bruce L., Richard T. O'Connell, J. B. Orris, and Dawn C. Porter. Essentials of Business, 2nd edition, McGraw-Hill/Irwin, 2008, ISBN: 978-0-07-331988-9. Required Computing

More information

2. A typical business process

2. A typical business process I. Basic Concepts on ERP 1. Enterprise resource planning (ERP) Enterprise resource planning (ERP) is the planning of how business resources (materials, employees, customers etc.) are acquired and moved

More information

An introduction to using Microsoft Excel for quantitative data analysis

An introduction to using Microsoft Excel for quantitative data analysis Contents An introduction to using Microsoft Excel for quantitative data analysis 1 Introduction... 1 2 Why use Excel?... 2 3 Quantitative data analysis tools in Excel... 3 4 Entering your data... 6 5 Preparing

More information

an introduction to VISUALIZING DATA by joel laumans

an introduction to VISUALIZING DATA by joel laumans an introduction to VISUALIZING DATA by joel laumans an introduction to VISUALIZING DATA iii AN INTRODUCTION TO VISUALIZING DATA by Joel Laumans Table of Contents 1 Introduction 1 Definition Purpose 2 Data

More information

BEHAVIOR BASED CREDIT CARD FRAUD DETECTION USING SUPPORT VECTOR MACHINES

BEHAVIOR BASED CREDIT CARD FRAUD DETECTION USING SUPPORT VECTOR MACHINES BEHAVIOR BASED CREDIT CARD FRAUD DETECTION USING SUPPORT VECTOR MACHINES 123 CHAPTER 7 BEHAVIOR BASED CREDIT CARD FRAUD DETECTION USING SUPPORT VECTOR MACHINES 7.1 Introduction Even though using SVM presents

More information

Diagrams and Graphs of Statistical Data

Diagrams and Graphs of Statistical Data Diagrams and Graphs of Statistical Data One of the most effective and interesting alternative way in which a statistical data may be presented is through diagrams and graphs. There are several ways in

More information

Data Mining: Exploring Data. Lecture Notes for Chapter 3. Introduction to Data Mining

Data Mining: Exploring Data. Lecture Notes for Chapter 3. Introduction to Data Mining Data Mining: Exploring Data Lecture Notes for Chapter 3 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 1 What is data exploration? A preliminary

More information

Anomaly detection. Problem motivation. Machine Learning

Anomaly detection. Problem motivation. Machine Learning Anomaly detection Problem motivation Machine Learning Anomaly detection example Aircraft engine features: = heat generated = vibration intensity Dataset: New engine: (vibration) (heat) Density estimation

More information

Essentials of Marketing Research

Essentials of Marketing Research Essentials of Marketing Second Edition Joseph F. Hair, Jr. Kennesaw State University Mary F. Wolfinbarger California State University-Long Beach David J. Ortinau University of South Florida Robert P. Bush

More information

Data exploration with Microsoft Excel: analysing more than one variable

Data exploration with Microsoft Excel: analysing more than one variable Data exploration with Microsoft Excel: analysing more than one variable Contents 1 Introduction... 1 2 Comparing different groups or different variables... 2 3 Exploring the association between categorical

More information

Some vendors have a big presence in a particular industry; some are geared toward data scientists, others toward business users.

Some vendors have a big presence in a particular industry; some are geared toward data scientists, others toward business users. Bonus Chapter Ten Major Predictive Analytics Vendors In This Chapter Angoss FICO IBM RapidMiner Revolution Analytics Salford Systems SAP SAS StatSoft, Inc. TIBCO This chapter highlights ten of the major

More information

Data Visualization. History, present and challanges. Giorgio Uboldi @giorgiouboldi. DensityDesign Research Lab @densitydesign

Data Visualization. History, present and challanges. Giorgio Uboldi @giorgiouboldi. DensityDesign Research Lab @densitydesign Università degli Studi di Milano-Bicocca - November 20th 2015 Data Visualization History, present and challanges Giorgio Uboldi @giorgiouboldi DensityDesign Research Lab @densitydesign TEAM Prof. Paolo

More information

The Need for Training in Big Data: Experiences and Case Studies

The Need for Training in Big Data: Experiences and Case Studies The Need for Training in Big Data: Experiences and Case Studies Guy Lebanon Amazon Background and Disclaimer All opinions are mine; other perspectives are legitimate. Based on my experience as a professor

More information

System Management. 2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice

System Management. 2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice System Management Jonathan Cyr System Management Product Line Manager Udi Shagal Product Manager SiteScope Sudhindra d Tl Technical Lead Performance Manager 2010 Hewlett-Packard Development Company, L.P.

More information

ADD-INS: ENHANCING EXCEL

ADD-INS: ENHANCING EXCEL CHAPTER 9 ADD-INS: ENHANCING EXCEL This chapter discusses the following topics: WHAT CAN AN ADD-IN DO? WHY USE AN ADD-IN (AND NOT JUST EXCEL MACROS/PROGRAMS)? ADD INS INSTALLED WITH EXCEL OTHER ADD-INS

More information

Data exploration with Microsoft Excel: univariate analysis

Data exploration with Microsoft Excel: univariate analysis Data exploration with Microsoft Excel: univariate analysis Contents 1 Introduction... 1 2 Exploring a variable s frequency distribution... 2 3 Calculating measures of central tendency... 16 4 Calculating

More information

Introduction Course in SPSS - Evening 1

Introduction Course in SPSS - Evening 1 ETH Zürich Seminar für Statistik Introduction Course in SPSS - Evening 1 Seminar für Statistik, ETH Zürich All data used during the course can be downloaded from the following ftp server: ftp://stat.ethz.ch/u/sfs/spsskurs/

More information

Analytics: The Future of Security

Analytics: The Future of Security Analytics: The Future of Security Yong Qiao, Vice President of Software Engineering & Chief Security Architect, MicroStrategy Agenda Introduction: Security Analytics Usher Analytics What is Usher Analytics?

More information

Dimensionality Reduction: Principal Components Analysis

Dimensionality Reduction: Principal Components Analysis Dimensionality Reduction: Principal Components Analysis In data mining one often encounters situations where there are a large number of variables in the database. In such situations it is very likely

More information

By Ken Thompson, ServQ Alliance

By Ken Thompson, ServQ Alliance Instant Customer Analytics using only Excel By Ken Thompson, ServQ Alliance There are some excellent specialized data analytics, data mining and data visualization tools available however we should not

More information

Course Text. Required Computing Software. Course Description. Course Objectives. StraighterLine. Business Statistics

Course Text. Required Computing Software. Course Description. Course Objectives. StraighterLine. Business Statistics Course Text Business Statistics Lind, Douglas A., Marchal, William A. and Samuel A. Wathen. Basic Statistics for Business and Economics, 7th edition, McGraw-Hill/Irwin, 2010, ISBN: 9780077384470 [This

More information

Software/Applications Programmer Technical Writer E-Commerce Manager. Computer and Electronics Repair Interactive Media Developer

Software/Applications Programmer Technical Writer E-Commerce Manager. Computer and Electronics Repair Interactive Media Developer Information Technology Career Majors Description: Information Technology (IT) is the study, design, development, implementation, support or management of computer-based information systems, particularly

More information

Final Project Report

Final Project Report CPSC545 by Introduction to Data Mining Prof. Martin Schultz & Prof. Mark Gerstein Student Name: Yu Kor Hugo Lam Student ID : 904907866 Due Date : May 7, 2007 Introduction Final Project Report Pseudogenes

More information

Data, Measurements, Features

Data, Measurements, Features Data, Measurements, Features Middle East Technical University Dep. of Computer Engineering 2009 compiled by V. Atalay What do you think of when someone says Data? We might abstract the idea that data are

More information

A Correlation of. to the. South Carolina Data Analysis and Probability Standards

A Correlation of. to the. South Carolina Data Analysis and Probability Standards A Correlation of to the South Carolina Data Analysis and Probability Standards INTRODUCTION This document demonstrates how Stats in Your World 2012 meets the indicators of the South Carolina Academic Standards

More information

January 26, 2009 The Faculty Center for Teaching and Learning

January 26, 2009 The Faculty Center for Teaching and Learning THE BASICS OF DATA MANAGEMENT AND ANALYSIS A USER GUIDE January 26, 2009 The Faculty Center for Teaching and Learning THE BASICS OF DATA MANAGEMENT AND ANALYSIS Table of Contents Table of Contents... i

More information

Capturing Meaningful Competitive Intelligence from the Social Media Movement

Capturing Meaningful Competitive Intelligence from the Social Media Movement Capturing Meaningful Competitive Intelligence from the Social Media Movement Social media has evolved from a creative marketing medium and networking resource to a goldmine for robust competitive intelligence

More information

Customer Analytics. Turn Big Data into Big Value

Customer Analytics. Turn Big Data into Big Value Turn Big Data into Big Value All Your Data Integrated in Just One Place BIRT Analytics lets you capture the value of Big Data that speeds right by most enterprises. It analyzes massive volumes of data

More information

The Forgotten JMP Visualizations (Plus Some New Views in JMP 9) Sam Gardner, SAS Institute, Lafayette, IN, USA

The Forgotten JMP Visualizations (Plus Some New Views in JMP 9) Sam Gardner, SAS Institute, Lafayette, IN, USA Paper 156-2010 The Forgotten JMP Visualizations (Plus Some New Views in JMP 9) Sam Gardner, SAS Institute, Lafayette, IN, USA Abstract JMP has a rich set of visual displays that can help you see the information

More information

Data Science and Business Analytics Certificate Data Science and Business Intelligence Certificate

Data Science and Business Analytics Certificate Data Science and Business Intelligence Certificate Data Science and Business Analytics Certificate Data Science and Business Intelligence Certificate Description The Helzberg School of Management has launched two graduate-level certificates: one in Data

More information

Chapter 4 Displaying and Describing Categorical Data

Chapter 4 Displaying and Describing Categorical Data Chapter 4 Displaying and Describing Categorical Data Chapter Goals Learning Objectives This chapter presents three basic techniques for summarizing categorical data. After completing this chapter you should

More information

DMDSS: Data Mining Based Decision Support System to Integrate Data Mining and Decision Support

DMDSS: Data Mining Based Decision Support System to Integrate Data Mining and Decision Support DMDSS: Data Mining Based Decision Support System to Integrate Data Mining and Decision Support Rok Rupnik, Matjaž Kukar, Marko Bajec, Marjan Krisper University of Ljubljana, Faculty of Computer and Information

More information

Analyzing and interpreting data Evaluation resources from Wilder Research

Analyzing and interpreting data Evaluation resources from Wilder Research Wilder Research Analyzing and interpreting data Evaluation resources from Wilder Research Once data are collected, the next step is to analyze the data. A plan for analyzing your data should be developed

More information

COGNOS 8 Business Intelligence

COGNOS 8 Business Intelligence COGNOS 8 Business Intelligence QUERY STUDIO USER GUIDE Query Studio is the reporting tool for creating simple queries and reports in Cognos 8, the Web-based reporting solution. In Query Studio, you can

More information

Foundation of Quantitative Data Analysis

Foundation of Quantitative Data Analysis Foundation of Quantitative Data Analysis Part 1: Data manipulation and descriptive statistics with SPSS/Excel HSRS #10 - October 17, 2013 Reference : A. Aczel, Complete Business Statistics. Chapters 1

More information

MultiAlign Software. Windows GUI. Console Application. MultiAlign Software Website. Test Data

MultiAlign Software. Windows GUI. Console Application. MultiAlign Software Website. Test Data MultiAlign Software This documentation describes MultiAlign and its features. This serves as a quick guide for starting to use MultiAlign. MultiAlign comes in two forms: as a graphical user interface (GUI)

More information

Business Statistics MBA 2010. Course Outline

Business Statistics MBA 2010. Course Outline Business Statistics MBA 2010 Course Outline Lecturer: Catalina Stefanescu A305, ext 8846, cstefanescu@london.edu Secretary: Kate Pelling S347, ext 8844, kpelling@london.edu Overview The objective of this

More information

This chapter reviews the general issues involving data analysis and introduces

This chapter reviews the general issues involving data analysis and introduces Research Skills for Psychology Majors: Everything You Need to Know to Get Started Data Preparation With SPSS This chapter reviews the general issues involving data analysis and introduces SPSS, the Statistical

More information

Is a Data Scientist the New Quant? Stuart Kozola MathWorks

Is a Data Scientist the New Quant? Stuart Kozola MathWorks Is a Data Scientist the New Quant? Stuart Kozola MathWorks 2015 The MathWorks, Inc. 1 Facts or information used usually to calculate, analyze, or plan something Information that is produced or stored by

More information

White Paper April 2006

White Paper April 2006 White Paper April 2006 Table of Contents 1. Executive Summary...4 1.1 Scorecards...4 1.2 Alerts...4 1.3 Data Collection Agents...4 1.4 Self Tuning Caching System...4 2. Business Intelligence Model...5

More information

Data Analytics in Organisations and Business

Data Analytics in Organisations and Business Data Analytics in Organisations and Business Dr. Isabelle E-mail: isabelle.flueckiger@math.ethz.ch 1 Data Analytics in Organisations and Business Some organisational information: Tutorship: Gian Thanei:

More information

How to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning

How to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning How to use Big Data in Industry 4.0 implementations LAURI ILISON, PhD Head of Big Data and Machine Learning Big Data definition? Big Data is about structured vs unstructured data Big Data is about Volume

More information

Machine Learning using MapReduce

Machine Learning using MapReduce Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous

More information

Text Mining - Scope and Applications

Text Mining - Scope and Applications Journal of Computer Science and Applications. ISSN 2231-1270 Volume 5, Number 2 (2013), pp. 51-55 International Research Publication House http://www.irphouse.com Text Mining - Scope and Applications Miss

More information

Republic Polytechnic School of Information and Communications Technology C355 Business Intelligence. Module Curriculum

Republic Polytechnic School of Information and Communications Technology C355 Business Intelligence. Module Curriculum Republic Polytechnic School of Information and Communications Technology C355 Business Intelligence Module Curriculum This document addresses the content related abilities, with reference to the module.

More information

Introduction to Longitudinal Data Analysis

Introduction to Longitudinal Data Analysis Introduction to Longitudinal Data Analysis Longitudinal Data Analysis Workshop Section 1 University of Georgia: Institute for Interdisciplinary Research in Education and Human Development Section 1: Introduction

More information

How To Identify Technical Debt In Java (Tty) On A Microsoft Powerbook (V0.2.2) On An Ipa (Microsoft) Microsoft Microsoft (Powerbook) On Microsoft.Com (V1

How To Identify Technical Debt In Java (Tty) On A Microsoft Powerbook (V0.2.2) On An Ipa (Microsoft) Microsoft Microsoft (Powerbook) On Microsoft.Com (V1 visualization and representation for scientific analysis Seminar Foundations in Empirical Software Engineering Dominik Münch Institut für Informatik Software & Systems Engineering ization The use of computer-supported,

More information

All Visualizations Documentation

All Visualizations Documentation All Visualizations Documentation All Visualizations Documentation 2 Copyright and Trademarks Licensed Materials - Property of IBM. Copyright IBM Corp. 2013 IBM, the IBM logo, and Cognos are trademarks

More information

Exploratory data analysis (Chapter 2) Fall 2011

Exploratory data analysis (Chapter 2) Fall 2011 Exploratory data analysis (Chapter 2) Fall 2011 Data Examples Example 1: Survey Data 1 Data collected from a Stat 371 class in Fall 2005 2 They answered questions about their: gender, major, year in school,

More information

EXPLORATORY DATA ANALYSIS

EXPLORATORY DATA ANALYSIS CHAPTER 3 EXPLORATORY DATA ANALYSIS HYPOTHESIS TESTING VERSUS EXPLORATORY DATA ANALYSIS GETTING TO KNOW THE DATA SET DEALING WITH CORRELATED VARIABLES EXPLORING CATEGORICAL VARIABLES USING EDA TO UNCOVER

More information

INTRODUCING DATA ANALYSIS IN A STATISTICS COURSE IN ENVIRONMENTAL SCIENCE STUDIES

INTRODUCING DATA ANALYSIS IN A STATISTICS COURSE IN ENVIRONMENTAL SCIENCE STUDIES INTRODUCING DATA ANALYSIS IN A STATISTICS COURSE IN ENVIRONMENTAL SCIENCE STUDIES C. Capilla Technical University of Valencia, Spain CCAPILLA@EIO.UPV.ES Education in methods of applied statistics is important

More information

ANALYTICS IN ACTION WINNING IN ASSET MANAGEMENT DISTRIBUTION WITH PREDICTIVE SALES INTELLIGENCE WHITEPAPER

ANALYTICS IN ACTION WINNING IN ASSET MANAGEMENT DISTRIBUTION WITH PREDICTIVE SALES INTELLIGENCE WHITEPAPER ANALYTICS IN ACTION WINNING IN ASSET MANAGEMENT DISTRIBUTION WITH PREDICTIVE SALES INTELLIGENCE WHITEPAPER ANALYTICS IN ACTION WINNING IN ASSET MANAGEMENT DISTRIBUTION WITH PREDICTIVE SALES INTELLIGENCE

More information

COM CO P 5318 Da t Da a t Explora Explor t a ion and Analysis y Chapte Chapt r e 3

COM CO P 5318 Da t Da a t Explora Explor t a ion and Analysis y Chapte Chapt r e 3 COMP 5318 Data Exploration and Analysis Chapter 3 What is data exploration? A preliminary exploration of the data to better understand its characteristics. Key motivations of data exploration include Helping

More information