CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing. University of Florida, CISE Department Prof.

Transcription

1 CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing University of Florida, CISE Department Prof. Daisy Zhe Wang

2 Data Science Overview Why, What, How, Who

3 Outline Why Data Science? What is Data Science? What are some prominent examples of Data Science? How to become a Data Scientist? Who are hiring Data Scientists Now? 3

4 Why Data Science? 4

5 The Dawn of Big Data Google, Yahoo today Web Search and Computational advertising Google: 35,000 searches/sec Yahoo! scale: 600 million users per month, 4 billion clicks per day, 25 terabytes of data collected every day Netflix 2007 Movie recommendations, netflix prize 100 million ratings, 500,000 users, 18,000 movies Amazon 2003 Product recommendations, reviews 29 million customers, millions of products Word Economic Forum 2011 at Davos Personal data digital data created by and about people represents a new economic asset class, touching all aspects of society.

6

7 How Big is Your Data? Kilobyte (1000 bytes) Megabyte ( bytes) Gigabyte ( bytes) Terabyte ( bytes) Petabyte ( bytes) Exabyte ( bytes) Zettabyte ( bytes) Yottabyte ( bytes) 7

8 5 Vs of Big Data Raw Data: Volume Change over time: Velocity Data types: Variety Data Quality: Veracity Information for Decision Making: Value

9 Cloud Computing Cloud computing is a style of computing where scalable and elastic IT-enabled capabilities are delivered as a (pay-asyou-go) service to external customers using Internet technologies. -- Gartner IT Glossary Cloud Computing is a new term for a long-held dream of computing as a utility -- Above the Clouds,

10 Cloud Computing = Cloud + SaaS Cloud computing refers to both: Cloud: The hardware and system software in the datacenters that provide those services. Public Cloud (Utility Computing) vs. Private Cloud SaaS: The applications delivered as services over the Internet, and Cloud Computing started around 2006 Big Data and Data Science (Big Data Analytics) started around

11 Current Trends Applications has bigger data and need more advanced analysis Example: Web, Corporate documents and s Natural Language Processing Example: Social Media Network/Graph Analysis IT Infrastructure moving to Cloud Computing Data Science arise given this application pull and technology push 11

12 What is Data Science? 12

13 Data Science A Definition Data Science is the science which uses computer science, statistics and machine learning, visualization and humancomputer interactions to collect, clean, integrate, analyze, visualize, interact with data to create data products. 13

14 Goal of Data Science Turn data into data products.

15 Data to Data Products Transaction Databases Fraud Detection Wireless Sensor Data Smart Home Text Data, Social Media Data Product Review and Consumer Satisfaction Software Log Data Automatic Trouble Shooting Genotype and Phenotype Data New treatment for Cancer

16 Other Data Products Financial products for investment or retirement funds Legal profession uses e-discovery tool for retrieval and review of legal documents Political campaign management Sports (e.g., Oakland baseball team) Remote Sensing for Environment Monitoring

17 What are some prominent examples of Data Science? 17

18 Data Products Google Web Search Google Ads News Recommendation Engine Google Maps Currently one of the best if not the best IT company to work for. (Google event on Jan 21/22)

19 Data Products Netflix Personalized Movie Ratings Movie Recommendations Similar Movies Movie Categories (e.g., 80 s movie with a strong female lead, Kung Fu movies) BlockBuster is out of the business

20 Data Products LinkedIn/Facebook People you may know Applications you may like Jobs/Events you might be interested Classifier for bad users and bad content With high accuracy, Facebook can guess whether you are single or married Who does not have LinkedIn/Facebook Account?

21 Data Products Twitter Text Analysis Spam Filter/Similarity Search User Sentiment/Satisfaction/Feedback News Breakout Trend and Topics 200 million users as of 2011, generating over 200 million tweets and handling over 1.6 billion search queries per day

22 Data Products Splunk Degradation, Failure Detection Identify Security Breach Event Monitoring Troubleshoot Tools Cross-platform Event Correlation Founded 2004, IPO in 2012

23 How to become a Data Scientist? 23

24 The Life of Data (state-of-the-art) Users Interface Collect Clean Integrate Analysis Visualization Data Sources

25 Challenges in Data Science Preparing Data (Noisy, Incomplete, Diverse, Streaming ) Analyze Data (Scalable, Accurate, Realtime, Advanced Methods, Probabilities and Uncertainties...) Represent Analysis Results (i.e. data product) (Story-telling, Interactive, explainable )

26 Skill Set of a Data Scientist Data Management Data collection, storage, cleaning, filtering, integration Large-scale Parallel Data Processing Parallel computing Statistics and Machine Learning Data modeling, inference, prediction, pattern recognition Interface and Data Visualization HCI design, visualization, story-telling

27 Who are hiring Data Scientists Now? 27

28 Sexy Job in the next 10 years The sexy job in the next ten years will be The ability to take data to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it that s going to be a hugely important skill. -- Hal Varian, Google Chief Economist, 2009

29 Who s hiring Data Scientist? IT companies: Google, Twitter, Lexis/Nexis, Facebook, Pivotal/EMC Media and Financial sectors Fox, CNN, NYT, Bloomburg, Research: Biology, Medicine, Physics, Psychology, Information office in government and corporations Law firms: e-discovery tools

30 Books on Data Science 30

31 Additional Reading Pointers Data Science Summit (Strata) ( ) Kaggle Competitions ( Data Science course at Berkeley & Corsera ( ) 31

32 Summary Why now: Dawn of Big Data, Need for Advanced Analytics and Cloud Computing What is it: Data Data Product, many examples incl. Google, Netflix, Splunk, LinkIn How to become: Data management, parallel computing and data processing, statistical machine learning, and visualization skills Life of Data Who are hiring: Data Scientists are in great demands, from industry to government to science. 32