Introduction to Big Data! with Apache Spark" UC#BERKELEY#
|
|
|
- Wesley Baldwin
- 7 years ago
- Views:
Transcription
1 Introduction to Big Data! with Apache Spark" UC#BERKELEY#
2 Course Goals" This Lecture" Brief History of Data Analysis" Big Data and Data Science Why All the Excitement?" Where Big Data Comes From"
3 Course Goals" 1. Learn about Data Science "» Where Big Data Comes from"» Observation and Experimentation"» The Elements of Data Science" Data Acquisition" Data Preparation" Analysis" Data Presentation" Data Products"
4 Course Goals" 2. Learn how to perform Data Science"» Understanding Data Quality"» Cleaning and manipulating datasets"» Using and parsing data representations "» Using basic Machine Learning algorithms and libraries"» Writing big data applications "» Performing Exploratory Data Analysis"
5 Course Goals" 3. Learn to write Apache Spark programs"» History and development"» Conceptual model"» How the Spark cluster model works"» Spark essentials (transformations, actions,! persistence, broadcast variables, accumulators,! Key-Value pairs, pyspark API)"» Debugging Spark programs"» Using Spark mllib for Machine Learning "
6 Brief Data Analysis History" R. A. Fisher"» 1935: The Design of Experiments " correlation does not imply causation! W. E. Demming"» 1939: Quality Control " Images: "
7 Brief Data Analysis History" Peter Luhn"» 1958: A Business Intelligence System " " John W. Tukey"» 1977: Exploratory Data Analysis" Howard Dresner"» 1989: Business Intelligence " Images: " " "
8 Brief Data Analysis History" Tom Mitchell"» 1997: Machine Learning book " Google"» 1996: Prototype Search Engine " Data-Driven Science ebook"» 2007: The Fourth Paradigm " Images: " " "
9 Brief Data Analysis History" Peter Norvig"» 2009: The Unreasonable Effectiveness of Data " Exponential growth in! data volume"» 2010: The Data Deluge " Images: " "
10 Data Makes Everything Clearer (part I)?" Seven Countries Study (Ancel Keys) "» Started in 1958, followed13,000 subjects total for 5-40 years" "
11 Data Makes Everything Clearer (part I)?" Seven Countries Study (Ancel Keys) "» Started in 1958, followed13,000 subjects total for 5-40 years" Significant controversy" Only studied subset of 21 countries with data" Failed to consider other factors (e.g., per capita annual sugar consumption in pounds)" 15" 40" 60" "
12 Big Data: Why all the Excitement?" Nowcasting vs Forecasting" Example Google Flu Trends:"» February 2010 detected outbreak two weeks ahead of CDC data"» Initially 97% accurate but overestimated during including one interval in period where GFT was off by 2x"» New models are estimating which cities are most at risk for spread of the Ebola virus" "
13 Why All the Excitement?" USA 2012 Presidential Election" "
14 Big Data and Election 2012 (cont.)" that was just one of several ways that Mr. Obama s campaign operations, some unnoticed by Mr. Romney s aides in Boston, helped save the president s candidacy. In Chicago, the campaign recruited a team of behavioral scientists to build an extraordinarily sophisticated database!! that allowed the Obama campaign not only to alter! the very nature of the electorate, making it younger! and less white, but also to create a portrait of shifting! voter allegiances. The power of this operation stunned! Mr. Romney s aides on election night, as they saw! voters they never even knew existed turn out in! places like Osceola County, Fla. " New York Times, Wed Nov 7, 2012"
15 Example: Facebook Lexicon" New Year s Eve" Halloween" Weekend"
16 Facebook availability in new countries and languages" Example: Facebook Lexicon"
17 Data Makes Everything Clearer (part II)?" Extrapolating the best fit model into the future predicts a rapid decline in Facebook activity in the next few years. " "
18 Data Makes Everything Clearer (part II)?" Google Trends searches for MySpace " Two Figures from the paper" " Searches for Facebook "
19 Data Makes Everything Clearer (part II)?" In keeping with the scientific principle "correlation equals causation," our research unequivocally demonstrated that Princeton may be in danger of disappearing entirely. " "
20 Data Makes Everything Clearer (part II)?" and based on Princeton search trends:" " This trend suggests that Princeton will have only half its current enrollment by 2018, and by 2021 it will have no students at all, " "
21 Data Makes Everything Clearer (part II)?" While we are concerned for Princeton University, we are even more concerned about the fate of the planet Google Trends for air have also been declining steadily, and our projections show that by the year 2060 there will be no air left:" "
22 Where Does Big Data Come From?" It s all happening online could record every:"» Click"» Ad impression"» Billing event"» Fast Forward, pause, "» Server request"» Transaction"» Network message"» Fault"» "
23 Where Does Big Data Come From?" User Generated Content (Web & Mobile)"» Facebook"» Instagram"» Yelp"» TripAdvisor"» Twitter"» YouTube"» "
24 Where Does Big Data Come From?" Health and Scientific Computing" Images: " " "
25 Graph Data" Lots of interesting data has a graph structure:" Social networks" Telecommunication Networks" Computer Networks" Road networks" Collaborations/Relationships" " Some of these graphs can get quite large! (e.g., Facebook user graph)" 33"
26 Log Files Apache Web Server Log" uplherc.upl.com-.-.-[01/aug/1995:00:00: ]-"get-/-http/1.0" uplherc.upl.com-.-.-[01/aug/1995:00:00: ]-"get-/images/ksclogo.medium.gif- HTTP/1.0" uplherc.upl.com-.-.-[01/aug/1995:00:00: ]-"get-/images/mosaic.logosmall.gif- HTTP/1.0" uplherc.upl.com-.-.-[01/aug/1995:00:00: ]-"get-/images/usa.logosmall.gif-http/ 1.0" ix.esc.ca2.07.ix.netcom.com-.-.-[01/aug/1995:00:00: ]-"get-/images/launch. logo.gif-http/1.0" uplherc.upl.com-.-.-[01/aug/1995:00:00: ]-"get-/images/world.logosmall.gif- HTTP/1.0" slppp6.intermind.net-.-.-[01/aug/1995:00:00: ]-"get-/history/skylab/ skylab.html-http/1.0" piweba4y.prodigy.com-.-.-[01/aug/1995:00:00: ]-"get-/images/launchmedium.gif- HTTP/1.0" tampico.usc.edu-.-.-[14/aug/1995:22:57: ]-"get-/welcome.html-http/1.0"
27 Machine Syslog File" dhcp :cs100_1>-syslog-.w-10- Feb--3-15:18:11-dhcp Evernote[1140]-<Warning>:-.[EDAMAccounting- read:]:-unexpected-field-id-23-with-type-8.--skipping.- Feb--3-15:18:11-dhcp Evernote[1140]-<Warning>:-.[EDAMUser-read:]:- unexpected-field-id-17-with-type-12.--skipping.- Feb--3-15:18:11-dhcp Evernote[1140]-<Warning>:-. [EDAMAuthenticationResult-read:]:-unexpected-field-ID-6-with-type Skipping.- Feb--3-15:18:11-dhcp Evernote[1140]-<Warning>:-. [EDAMAuthenticationResult-read:]:-unexpected-field-ID-7-with-type Skipping.- Feb--3-15:18:11-dhcp Evernote[1140]-<Warning>:-.[EDAMAccounting- read:]:-unexpected-field-id-19-with-type-8.--skipping.- Feb--3-15:18:11-dhcp Evernote[1140]-<Warning>:-.[EDAMAccounting- read:]:-unexpected-field-id-23-with-type-8.--skipping.- Feb--3-15:18:11-dhcp Evernote[1140]-<Warning>:-.[EDAMUser-read:]:- unexpected-field-id-17-with-type-12.--skipping.- Feb--3-15:18:11-dhcp Evernote[1140]-<Warning>:-.[EDAMSyncState-read:]:- unexpected-field-id-5-with-type-10.--skipping.- Feb--3-15:18:49-dhcp com.apple.mtmd[47]-<Notice>:-low-prioritythinning-needed-for-volume-Macintosh-HD-(/)-with-18.9-<=-20.0-pct-free-space--
28 Internet of Things:! Example Measurements" Humidity vs. Time meters" 33m: 111" 32m: 110" 30m: 109,108,107" 20m: 106,105,104" 10m: 103, 102, 101" Rel Humidity (%) Temperature (C) Temperature vs. Time 8 7/7/03 9:40 Redwood tree humidity and temperature at various heights" 7/7/03 7/7/03 7/7/03 7/7/03 7/8/03 7/8/03 7/8/03 7/8/03 7/8/03 7/8/03 7/9/03 7/9/03 7/9/03 7/9/03 13:11 16:43 20:15 23:46 3:18 6:50 10:21 13:53 17:25 20:56 0:28 4:00 7:31 11:03 Date
29 Internet of Things: RFID tags" California FasTrak Electronic Toll Collection transponder" Used to pay tolls" Collected data! also used for! traffic reporting"» " "
30 What Can You do with Big Data?" Crowdsourcing " + Physical modeling + Sensing + Data Assimilation" =" "
Software Engineering for Big Data. CS846 Paulo Alencar David R. Cheriton School of Computer Science University of Waterloo
Software Engineering for Big Data CS846 Paulo Alencar David R. Cheriton School of Computer Science University of Waterloo Big Data Big data technologies describe a new generation of technologies that aim
Healthcare data analytics. Da-Wei Wang Institute of Information Science wdw@iis.sinica.edu.tw
Healthcare data analytics Da-Wei Wang Institute of Information Science wdw@iis.sinica.edu.tw Outline Data Science Enabling technologies Grand goals Issues Google flu trend Privacy Conclusion Analytics
Opportunities and Limitations of Big Data
Opportunities and Limitations of Big Data Karl Schmedders University of Zurich and Swiss Finance Institute «Big Data: Little Ethics?» HWZ-Darden-Conference June 4, 2015 On fortune.com this morning: Apple's
The Melvyl Recommender Project: Final Report
Performance Testing Testing Goals Current instances of XTF used in production serve thousands to hundreds of thousands of documents. One of our investigation goals was to experiment with a much larger
Managing Big Data with Hadoop & Vertica. A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database
Managing Big Data with Hadoop & Vertica A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database Copyright Vertica Systems, Inc. October 2009 Cloudera and Vertica
Spark Application Carousel. Spark Summit East 2015
Spark Application Carousel Spark Summit East 2015 About Today s Talk About Me: Vida Ha - Solutions Engineer at Databricks. Goal: For beginning/early intermediate Spark Developers. Motivate you to start
Why Big Data Analytics?
An ebook by Datameer Why Big Data Analytics? Three Business Challenges Best Addressed Using Big Data Analytics It s hard to overstate the importance of data for businesses today. It s the lifeline of any
How has Web 2.0 reshaped the presidential campaign in the United States?
How has Web 2.0 reshaped the presidential campaign in the United States? Dennis Anderson, Ph.D. Professor of Information Systems Pace University, New York, NY, USA dennis.danderson@gmail.com Abstract In
How To Handle Big Data With A Data Scientist
III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution
Challenges and Lessons from NIST Data Science Pre-pilot Evaluation in Introduction to Data Science Course Fall 2015
Challenges and Lessons from NIST Data Science Pre-pilot Evaluation in Introduction to Data Science Course Fall 2015 Dr. Daisy Zhe Wang Director of Data Science Research Lab University of Florida, CISE
Chapter 7. Using Hadoop Cluster and MapReduce
Chapter 7 Using Hadoop Cluster and MapReduce Modeling and Prototyping of RMS for QoS Oriented Grid Page 152 7. Using Hadoop Cluster and MapReduce for Big Data Problems The size of the databases used in
Are You Ready for Big Data?
Are You Ready for Big Data? Jim Gallo National Director, Business Analytics February 11, 2013 Agenda What is Big Data? How do you leverage Big Data in your company? How do you prepare for a Big Data initiative?
Social network downtime in 2008 February 2009
Social network downtime in 2008 February 2009 This is a survey of the 2008 website availability (uptime) of 15 of the world s largest social networks. It includes analysis of the accumulated downtime of
Creating, Solving, and Graphing Systems of Linear Equations and Linear Inequalities
Algebra 1, Quarter 2, Unit 2.1 Creating, Solving, and Graphing Systems of Linear Equations and Linear Inequalities Overview Number of instructional days: 15 (1 day = 45 60 minutes) Content to be learned
Machine- Learning Summer School - 2015
Machine- Learning Summer School - 2015 Big Data Programming David Franke Vast.com hbp://www.cs.utexas.edu/~dfranke/ Goals for Today Issues to address when you have big data Understand two popular big data
BIG DATA, MAPREDUCE & HADOOP
BIG, MAPREDUCE & HADOOP LARGE SCALE DISTRIBUTED SYSTEMS By Jean-Pierre Lozi A tutorial for the LSDS class LARGE SCALE DISTRIBUTED SYSTEMS BIG, MAPREDUCE & HADOOP 1 OBJECTIVES OF THIS LAB SESSION The LSDS
CAS CS 565, Data Mining
CAS CS 565, Data Mining Course logistics Course webpage: http://www.cs.bu.edu/~evimaria/cs565-10.html Schedule: Mon Wed, 4-5:30 Instructor: Evimaria Terzi, evimaria@cs.bu.edu Office hours: Mon 2:30-4pm,
A Pharmacometrician s Perspective for Utilization of Big Data
Is There a Role of Big Data in Drug Development Decisions? ACoP6 Oct. 5, 2015 Crystal City, VA A Pharmacometrician s Perspective for Utilization of Big Data Marc R. Gastonguay, Ph.D. President & CEO Metrum
International Journal of Advanced Engineering Research and Applications (IJAERA) ISSN: 2454-2377 Vol. 1, Issue 6, October 2015. Big Data and Hadoop
ISSN: 2454-2377, October 2015 Big Data and Hadoop Simmi Bagga 1 Satinder Kaur 2 1 Assistant Professor, Sant Hira Dass Kanya MahaVidyalaya, Kala Sanghian, Distt Kpt. INDIA E-mail: simmibagga12@gmail.com
Tutorial: Big Data Algorithms and Applications Under Hadoop KUNPENG ZHANG SIDDHARTHA BHATTACHARYYA
Tutorial: Big Data Algorithms and Applications Under Hadoop KUNPENG ZHANG SIDDHARTHA BHATTACHARYYA http://kzhang6.people.uic.edu/tutorial/amcis2014.html August 7, 2014 Schedule I. Introduction to big data
The Berkeley AMPLab - Collaborative Big Data Research
The Berkeley AMPLab - Collaborative Big Data Research UC BERKELEY Anthony D. Joseph LASER Summer School September 2013 About Me Education: MIT SB, MS, PhD Joined Univ. of California, Berkeley in 1998 Current
Big Data Research in the AMPLab: BDAS and Beyond
Big Data Research in the AMPLab: BDAS and Beyond Michael Franklin UC Berkeley 1 st Spark Summit December 2, 2013 UC BERKELEY AMPLab: Collaborative Big Data Research Launched: January 2011, 6 year planned
A Berkeley View of Big Data
A Berkeley View of Big Data Ion Stoica UC Berkeley BEARS February 17, 2011 Big Data is Massive Facebook: 130TB/day: user logs 200-400TB/day: 83 million pictures Google: > 25 PB/day processed data Data
At The Crossroads of Marketing and Technology. Top 6 Tips for Success in the Digital World
At The Crossroads of Marketing and Technology Top 6 Tips for Success in the Digital World Introduction As a small business, you have enough challenges keeping your head above water and focusing on how
Extreme Computing. Big Data. Stratis Viglas. School of Informatics University of Edinburgh sviglas@inf.ed.ac.uk. Stratis Viglas Extreme Computing 1
Extreme Computing Big Data Stratis Viglas School of Informatics University of Edinburgh sviglas@inf.ed.ac.uk Stratis Viglas Extreme Computing 1 Petabyte Age Big Data Challenges Stratis Viglas Extreme Computing
The Data Engineer. Mike Tamir Chief Science Officer Galvanize. Steven Miller Global Leader Academic Programs IBM Analytics
The Data Engineer Mike Tamir Chief Science Officer Galvanize Steven Miller Global Leader Academic Programs IBM Analytics Alessandro Gagliardi Lead Faculty Galvanize Businesses are quickly realizing that
BiDAl: Big Data Analyzer for Cluster Traces
BiDAl: Big Data Analyzer for Cluster Traces Alkida Balliu, Dennis Olivetti, Ozalp Babaoglu, Moreno Marzolla, Alina Sirbu Department of Computer Science and Engineering University of Bologna, Italy BigSys
Collaborations between Official Statistics and Academia in the Era of Big Data
Collaborations between Official Statistics and Academia in the Era of Big Data World Statistics Day October 20-21, 2015 Budapest Vijay Nair University of Michigan Past-President of ISI vnn@umich.edu What
ISSN: 2320-1363 CONTEXTUAL ADVERTISEMENT MINING BASED ON BIG DATA ANALYTICS
CONTEXTUAL ADVERTISEMENT MINING BASED ON BIG DATA ANALYTICS A.Divya *1, A.M.Saravanan *2, I. Anette Regina *3 MPhil, Research Scholar, Muthurangam Govt. Arts College, Vellore, Tamilnadu, India Assistant
Data Warehousing in the Age of Big Data
Data Warehousing in the Age of Big Data Krish Krishnan AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD * PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO Morgan Kaufmann is an imprint of Elsevier
BASHO DATA PLATFORM SIMPLIFIES BIG DATA, IOT, AND HYBRID CLOUD APPS
WHITEPAPER BASHO DATA PLATFORM BASHO DATA PLATFORM SIMPLIFIES BIG DATA, IOT, AND HYBRID CLOUD APPS INTRODUCTION Big Data applications and the Internet of Things (IoT) are changing and often improving our
International Journal of Engineering Research ISSN: 2348-4039 & Management Technology November-2015 Volume 2, Issue-6
International Journal of Engineering Research ISSN: 2348-4039 & Management Technology Email: editor@ijermt.org November-2015 Volume 2, Issue-6 www.ijermt.org Modeling Big Data Characteristics for Discovering
COMP9321 Web Application Engineering
COMP9321 Web Application Engineering Semester 2, 2015 Dr. Amin Beheshti Service Oriented Computing Group, CSE, UNSW Australia Week 11 (Part II) http://webapps.cse.unsw.edu.au/webcms2/course/index.php?cid=2411
Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013
Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013 Housekeeping 1. Any questions coming out of today s presentation can be discussed in the bar this evening 2. OCF is
Big Data a threat or a chance?
Big Data a threat or a chance? Helwig Hauser University of Bergen, Dept. of Informatics Big Data What is Big Data? well, lots of data, right? we come back to this in a moment. certainly, a buzz-word but
Searching the Social Network: Future of Internet Search? alessio.signorini@oneriot.com
Searching the Social Network: Future of Internet Search? alessio.signorini@oneriot.com Alessio Signorini Who am I? I was born in Pisa, Italy and played professional soccer until a few years ago. No coffee
Kafka & Redis for Big Data Solutions
Kafka & Redis for Big Data Solutions Christopher Curtin Head of Technical Research @ChrisCurtin About Me 25+ years in technology Head of Technical Research at Silverpop, an IBM Company (14 + years at Silverpop)
MapReduce Jeffrey Dean and Sanjay Ghemawat. Background context
MapReduce Jeffrey Dean and Sanjay Ghemawat Background context BIG DATA!! o Large-scale services generate huge volumes of data: logs, crawls, user databases, web site content, etc. o Very useful to be able
Introduction to Data Mining and Machine Learning Techniques. Iza Moise, Evangelos Pournaras, Dirk Helbing
Introduction to Data Mining and Machine Learning Techniques Iza Moise, Evangelos Pournaras, Dirk Helbing Iza Moise, Evangelos Pournaras, Dirk Helbing 1 Overview Main principles of data mining Definition
Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum
Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum Siva Ravada Senior Director of Development Oracle Spatial and MapViewer 2 Evolving Technology Platforms
Data and Machine Architecture for the Data Science Lab Workflow Development, Testing, and Production for Model Training, Evaluation, and Deployment
Data and Machine Architecture for the Data Science Lab Workflow Development, Testing, and Production for Model Training, Evaluation, and Deployment Rosaria Silipo Marco A. Zimmer Rosaria.Silipo@knime.com
Big Systems, Big Data
Big Systems, Big Data When considering Big Distributed Systems, it can be noted that a major concern is dealing with data, and in particular, Big Data Have general data issues (such as latency, availability,
Cybersecurity Analytics for a Smarter Planet
IBM Institute for Advanced Security December 2010 White Paper Cybersecurity Analytics for a Smarter Planet Enabling complex analytics with ultra-low latencies on cybersecurity data in motion 2 Cybersecurity
From Spark to Ignition:
From Spark to Ignition: Fueling Your Business on Real-Time Analytics Eric Frenkiel, MemSQL CEO June 29, 2015 San Francisco, CA What s in Store For This Presentation? 1. MemSQL: A real-time database for
Presidential Nominations
SECTION 4 Presidential Nominations Delegates cheer on a speaker at the 2008 Democratic National Convention. Guiding Question Does the nominating system allow Americans to choose the best candidates for
Big Data and Analytics: Challenges and Opportunities
Big Data and Analytics: Challenges and Opportunities Dr. Amin Beheshti Lecturer and Senior Research Associate University of New South Wales, Australia (Service Oriented Computing Group, CSE) Talk: Sharif
Big Data Processing. Patrick Wendell Databricks
Big Data Processing Patrick Wendell Databricks About me Committer and PMC member of Apache Spark Former PhD student at Berkeley Left Berkeley to help found Databricks Now managing open source work at Databricks
International Journal of Innovative Research in Computer and Communication Engineering
FP Tree Algorithm and Approaches in Big Data T.Rathika 1, J.Senthil Murugan 2 Assistant Professor, Department of CSE, SRM University, Ramapuram Campus, Chennai, Tamil Nadu,India 1 Assistant Professor,
Advance Diploma in Digital. Marketing. Full Time Part Time Online.
Advance Diploma in Digital Marketing Full Time Part Time Online www.zodrox.com Become a Digital Marketing Expert and Reach New Highs INDIA S MOST PROMINENT DIGITAL MARKETING & SKILL DEVELOPMENT INSTITUTE
BIG DATA & DATA SCIENCE
BIG DATA & DATA SCIENCE ACADEMY PROGRAMS IN-COMPANY TRAINING PORTFOLIO 2 TRAINING PORTFOLIO 2016 Synergic Academy Solutions BIG DATA FOR LEADING BUSINESS Big data promises a significant shift in the way
What s next for the Berkeley Data Analytics Stack?
What s next for the Berkeley Data Analytics Stack? Michael Franklin June 30th 2014 Spark Summit San Francisco UC BERKELEY AMPLab: Collaborative Big Data Research 60+ Students, Postdocs, Faculty and Staff
Monitis Project Proposals for AUA. September 2014, Yerevan, Armenia
Monitis Project Proposals for AUA September 2014, Yerevan, Armenia Distributed Log Collecting and Analysing Platform Project Specifications Category: Big Data and NoSQL Software Requirements: Apache Hadoop
Associate Professor, Department of CSE, Shri Vishnu Engineering College for Women, Andhra Pradesh, India 2
Volume 6, Issue 3, March 2016 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Special Issue
Analytics in the Cloud. Peter Sirota, GM Elastic MapReduce
Analytics in the Cloud Peter Sirota, GM Elastic MapReduce Data-Driven Decision Making Data is the new raw material for any business on par with capital, people, and labor. What is Big Data? Terabytes of
"BIG DATA A PROLIFIC USE OF INFORMATION"
Ojulari Moshood Cameron University - IT4444 Capstone 2013 "BIG DATA A PROLIFIC USE OF INFORMATION" Abstract: The idea of big data is to better use the information generated by individual to remake and
Automatically Finding Performance Problems with Feedback-Directed Learning Software Testing
Automatically Finding Performance Problems with Feedback-Directed Learning Software Testing Mark Grechanik University of Illinois at Chicago Chen Fu and Qing Xie Accenture Technology Lab Good Performance
Transforming the Telecoms Business using Big Data and Analytics
Transforming the Telecoms Business using Big Data and Analytics Event: ICT Forum for HR Professionals Venue: Meikles Hotel, Harare, Zimbabwe Date: 19 th 21 st August 2015 AFRALTI 1 Objectives Describe
Introduction to Big Data Training
Introduction to Big Data Training The quickest way to be introduce with NOSQL/BIG DATA offerings Learn and experience Big Data Solutions including Hadoop HDFS, Map Reduce, NoSQL DBs: Document Based DB
Cloud Computing at Google. Architecture
Cloud Computing at Google Google File System Web Systems and Algorithms Google Chris Brooks Department of Computer Science University of San Francisco Google has developed a layered system to handle webscale
Customer Segmentation in the Age of Big Data
Customer Segmentation in the Age of Big Data By: Michael Million Moving Beyond Traditional Segmentation Traditional customer segmentation is at the heart of every marketing organization, giving companies
Introduction to Big Data! with Apache Spark" UC#BERKELEY#
Introduction to Big Data! with Apache Spark" UC#BERKELEY# This Lecture" The Big Data Problem" Hardware for Big Data" Distributing Work" Handling Failures and Slow Machines" Map Reduce and Complex Jobs"
IC05 Introduction on Networks &Visualization Nov. 2009. <mathieu.bastian@gmail.com>
IC05 Introduction on Networks &Visualization Nov. 2009 Overview 1. Networks Introduction Networks across disciplines Properties Models 2. Visualization InfoVis Data exploration
Graph Database Proof of Concept Report
Objectivity, Inc. Graph Database Proof of Concept Report Managing The Internet of Things Table of Contents Executive Summary 3 Background 3 Proof of Concept 4 Dataset 4 Process 4 Query Catalog 4 Environment
INDIVIDUAL RESEARCH PAPER
Lane Department of Computer Science and Electrical Engineering 12/2/2013 WEST VIRGINIA UNIVERSITY INDIVIDUAL RESEARCH PAPER CS 480: Senior Design Group 3- LightSpeed Chelsea Skotnicki Benjamin M. Statler
Program Visualization for Programming Education Case of Jeliot 3
Program Visualization for Programming Education Case of Jeliot 3 Roman Bednarik, Andrés Moreno, Niko Myller Department of Computer Science University of Joensuu firstname.lastname@cs.joensuu.fi Abstract:
Infrastructures for big data
Infrastructures for big data Rasmus Pagh 1 Today s lecture Three technologies for handling big data: MapReduce (Hadoop) BigTable (and descendants) Data stream algorithms Alternatives to (some uses of)
Using Big Data to Generate New Insights about Your Workforce Sirota Science Webinar Series
August 2013 Using Big Data to Generate New Insights about Your Workforce Sirota Science Webinar Series Welcome to Our 2013 Sirota Science Webinar Series SirotaScience Webinar Series Regular review of current
14.30 Introduction to Statistical Methods in Economics Spring 2009
MIT OpenCourseWare http://ocw.mit.edu 14.30 Introduction to tatistical Methods in Economics pring 2009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Databricks. A Primer
Databricks A Primer Who is Databricks? Databricks was founded by the team behind Apache Spark, the most active open source project in the big data ecosystem today. Our mission at Databricks is to dramatically
HBase Schema Design. NoSQL Ma4ers, Cologne, April 2013. Lars George Director EMEA Services
HBase Schema Design NoSQL Ma4ers, Cologne, April 2013 Lars George Director EMEA Services About Me Director EMEA Services @ Cloudera ConsulFng on Hadoop projects (everywhere) Apache Commi4er HBase and Whirr
HUAWEI Advanced Data Science with Spark Streaming. Albert Bifet (@abifet)
HUAWEI Advanced Data Science with Spark Streaming Albert Bifet (@abifet) Huawei Noah s Ark Lab Focus Intelligent Mobile Devices Data Mining & Artificial Intelligence Intelligent Telecommunication Networks
The Internet of Things and Big Data: Intro
The Internet of Things and Big Data: Intro John Berns, Solutions Architect, APAC - MapR Technologies April 22 nd, 2014 1 What This Is; What This Is Not It s not specific to IoT It s not about any specific
How Companies are! Using Spark
How Companies are! Using Spark And where the Edge in Big Data will be Matei Zaharia History Decreasing storage costs have led to an explosion of big data Commodity cluster software, like Hadoop, has made
BIG DATA ANALYTICS For REAL TIME SYSTEM
BIG DATA ANALYTICS For REAL TIME SYSTEM Where does big data come from? Big Data is often boiled down to three main varieties: Transactional data these include data from invoices, payment orders, storage
The Youth Vote in 2012 CIRCLE Staff May 10, 2013
The Youth Vote in 2012 CIRCLE Staff May 10, 2013 In the 2012 elections, young voters (under age 30) chose Barack Obama over Mitt Romney by 60%- 37%, a 23-point margin, according to the National Exit Polls.
The Effectiveness of Internet Grassroots Movements
ROGERS STATE UNIVERSITY The Effectiveness of Internet Grassroots Movements Communication Research Methods Josh Gammon Fall 2010 This assignment was my first major research project at RSU. In this article
Using Data Mining for Mobile Communication Clustering and Characterization
Using Data Mining for Mobile Communication Clustering and Characterization A. Bascacov *, C. Cernazanu ** and M. Marcu ** * Lasting Software, Timisoara, Romania ** Politehnica University of Timisoara/Computer
Video Analytics and Security
Video Analytics and Security Using video data to improve both safety and ROI March 2008 / White Paper Make the most of your energy Summary I. Executive Summary... 3 II. Captured on Video. Now What?...
Rational Asset Manager 7.2 Editions and Licensing
Rational Asset Manager 7.2 Editions and Licensing Derek D. Baron, ddbaron@us.ibm.com Product Manager, Rational Asset Manager 2009 IBM Corporation IBM Corporation 200 The information contained in this presentation
RELEVANT TO ACCA QUALIFICATION PAPER P3. Studying Paper P3? Performance objectives 7, 8 and 9 are relevant to this exam
RELEVANT TO ACCA QUALIFICATION PAPER P3 Studying Paper P3? Performance objectives 7, 8 and 9 are relevant to this exam Business forecasting and strategic planning Quantitative data has always been supplied
Database Systems. National Chiao Tung University Chun-Jen Tsai 05/30/2012
Database Systems National Chiao Tung University Chun-Jen Tsai 05/30/2012 Definition of a Database Database System A multidimensional data collection, internal links between its entries make the information
CS 493N: Big Data Engineering - Overview
CS 493N: Big Data Engineering - Overview Introduction Data Structures for Big Data Data structures Basic Search Algorithms for Big Data Machine Learning and Predictive Analytics Intro to machine learning
Day 10: Text Mining from Social Media
Day 10: Text Mining from Social Media Paul Nulty Essex Summer School 2014 August 1, 2014 Why social media data? Volume: 500M registered users, 400M tweets per day (March 2013), Facebook has 1.15billion
Hadoop implementation of MapReduce computational model. Ján Vaňo
Hadoop implementation of MapReduce computational model Ján Vaňo What is MapReduce? A computational model published in a paper by Google in 2004 Based on distributed computation Complements Google s distributed
Oracle Big Data Handbook
ORACLG Oracle Press Oracle Big Data Handbook Tom Plunkett Brian Macdonald Bruce Nelson Helen Sun Khader Mohiuddin Debra L. Harding David Segleau Gokula Mishra Mark F. Hornick Robert Stackowiak Keith Laker
Getting personal: The future of communications
Getting personal: The future of communications Neil Wholey LGinsight and Head of Research and Customer Insight at Westminster City Council @neilwholey Accuracy of opinion polls http://thefutureplace.type
Databricks. A Primer
Databricks A Primer Who is Databricks? Databricks vision is to empower anyone to easily build and deploy advanced analytics solutions. The company was founded by the team who created Apache Spark, a powerful
The big data revolution
The big data revolution Friso van Vollenhoven (Xebia) Enterprise NoSQL Recently, there has been a lot of buzz about the NoSQL movement, a collection of related technologies mostly concerned with storing
INTRO TO BIG DATA. Djoerd Hiemstra. http://www.cs.utwente.nl/~hiemstra. Big Data in Clinical Medicinel, 30 June 2014
INTRO TO BIG DATA Big Data in Clinical Medicinel, 30 June 2014 Djoerd Hiemstra http://www.cs.utwente.nl/~hiemstra WHY BIG DATA? 2 Source: http://en.wikipedia.org/wiki/mount_everest 3 19 May 2012: 234 people
Large-Scale Data Sets Clustering Based on MapReduce and Hadoop
Journal of Computational Information Systems 7: 16 (2011) 5956-5963 Available at http://www.jofcis.com Large-Scale Data Sets Clustering Based on MapReduce and Hadoop Ping ZHOU, Jingsheng LEI, Wenjun YE
Outline. What is Big data and where they come from? How we deal with Big data?
What is Big Data Outline What is Big data and where they come from? How we deal with Big data? Big Data Everywhere! As a human, we generate a lot of data during our everyday activity. When you buy something,
The Big Data Paradigm Shift. Insight Through Automation
The Big Data Paradigm Shift Insight Through Automation Agenda The Problem Emcien s Solution: Algorithms solve data related business problems How Does the Technology Work? Case Studies 2013 Emcien, Inc.
Suresh Lakavath csir urdip Pune, India lsureshit@gmail.com.
A Big Data Hadoop Architecture for Online Analysis. Suresh Lakavath csir urdip Pune, India lsureshit@gmail.com. Ramlal Naik L Acme Tele Power LTD Haryana, India ramlalnaik@gmail.com. Abstract Big Data
Introduction to Parallel Programming and MapReduce
Introduction to Parallel Programming and MapReduce Audience and Pre-Requisites This tutorial covers the basics of parallel programming and the MapReduce programming model. The pre-requisites are significant
Reflection Report International Semester
Reflection Report International Semester Studying abroad at KTH Royal Institute of Technology Stockholm 18-01-2011 Chapter 1: Personal Information Name and surname: Arts, Rick G. B. E-mail address: Department:
CDH AND BUSINESS CONTINUITY:
WHITE PAPER CDH AND BUSINESS CONTINUITY: An overview of the availability, data protection and disaster recovery features in Hadoop Abstract Using the sophisticated built-in capabilities of CDH for tunable
Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料
Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料 美 國 13 歲 學 生 用 Big Data 找 出 霸 淩 熱 點 Puri 架 設 網 站 Bullyvention, 藉 由 分 析 Twitter 上 找 出 提 到 跟 霸 凌 相 關 的 詞, 搭 配 地 理 位 置
Big Data. How it is Transforming Learning and Talent Development
Big Data How it is Transforming Learning and Talent Development Agenda 1. Big Data Background 2. Big Data in Talent and Learning Analytics 3. Examples and Getting Started Big Data Defined Big Data.. The
THE FIELD POLL. By Mark DiCamillo, Director, The Field Poll
THE FIELD POLL THE INDEPENDENT AND NON-PARTISAN SURVEY OF PUBLIC OPINION ESTABLISHED IN 1947 AS THE CALIFORNIA POLL BY MERVIN FIELD Field Research Corporation 601 California Street, Suite 210 San Francisco,
New ways to a secure IT Management
New ways to a secure IT Management Comprehensive IT Performance & SIEM A strategic imperative IT is the key to the business strategy implementation and success. Organizations can get essential added value