Laurence Liew General Manager, APAC Economics Is Driving Big Data Analytics to the Cloud
Big Data 101 The Analytics Stack Economics of Big Data Convergence of the 3 forces Big Data Analytics in the Cloud with CloudR
Who we are Leading provider of commercial analytics platform based on open source R statistical computing language Our Software Delivers Power: Distributed, scalable high performance advanced analytics Productivity: Easier to build and deploy analytic applications Enterprise Readiness: Multi-platform Our Services Deliver Knowledge: Our experts enable you to be experts Time-to-Value: Our QuickStart projects give you a jumpstart Guidance: Our customer support team is here to help you Our Philosophy Customer-centric innovation Easy to do business with Customers 200+ Global 2000 Global Presence North America / EMEA / APAC Global Industries Served Financial Services Digital Media Government Health & Life Sciences High Tech Manufacturing Retail Telco 3
Revolution Confidential 200 Corporate Customers and Growing Finance & Insurance Healthcare & Life Sciences Academic & Gov t Consumer & Info Svcs Manuf & Tech 4
Centre of Excellence (CoE) Partner with iles to create new IPs in big data analytics in Singapore Conduct and run Big data analytics training/workshops to promote the use and adoption of big data technologies and analytics We will have our data scientist and developers work alongside our collaboration partners.
Centre of Attachment To accelerate formation of data science team within organization Analytics/statistics skills Big data infrastructure skills such as Hadoop and HPC clusters 3-months program consisting of: 1. Classroom training spread over 2 months 2. Inter-spaced with practical hands-on and guidance and 1-on-1 consultations with Revo s data science team 3. 1 month project work to deploy model into organization s infrastructure
Backdrop - Massive Data Volumes Exabytes 3D/4D Seismic Realtime Telemetry Machine Sensors Communication Logs Petabytes Systems Logs Vehicle Monitoring Geospatial ESRI Video And Imagery Terabytes Gigabytes Cost Records Volumes ERP Logistics Summary Operating Statistics Incidents Alarms Daily Activity Reports Text Instructions Workorders Reports Increasing Volume, Variety and Velocity 7 Decision Management Solutions, 2013
Volume Variety Velocity What s big data?
Big Data is big. Data set so large it cannot be managed in conventional database with acceptable performance and at acceptable cost. Volume What s big data?
Big Data is messy. 70-90% of all data generated lacks predefined structure or is difficult to map into a conventional data model. Variety What s big data?
Big Data moves. ICU: predict patient events FICO: flag suspect transactions Oreo: Superbowl ad from Tweets Retail: push in-store offers What s big data? Velocity
Big Data 101 The Analytics Stack Economics of Big Data Convergence of the 3 forces Big Data Analytics in the Cloud with CloudR
Next Generation Big Data Analytics Players??? ANALYTICS HDD -> SSD -> In-Memory INFRASTRUCTURE AND DATABASES 14
Hadoop The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers 1 node = 12TB 10 nodes = 120TB 100 nodes = 1.2PB 15
Hadoop Dell PowerEdge Servers 16
R or Revo R video goes here
= Language + Analytics Statistical data analysis programming language Huge library algorithms for data access, manipulation, analysis & graphics
Data Analytics Workflow INGEST DISTILL & ANALYZE CONSUME
Write Once. Deploy Anywhere. Hadoop Hortonworks Cloudera, Intel EDW Teradata ConnectR DeployR Clustered Systems Workstations & Servers Linux HPC Windows HPC Desktop Server Linux ScaleR DistributedR In the Cloud Microsoft Azure Amazon AWS (CloudR) DESIGNED FOR SCALE, PORTABILITY & PERFORMANCE 20
Big Data 101 The Analytics Stack Economics of Big Data Convergence of the 3 forces Big Data Analytics in the Cloud with CloudR
Why Now? THE PERFECT STORM CONVERGENCE OF
Data Science - The Tool 23
Computer Science - The infrastructure DISRUPTIVE TECHNOLOGY 1. Commodity Hardware 2. Open source Linux Hadoop R
Computer Science - Attack of the Exponentials 1TB: $14M in 1980. ~ $4.70 $9 99GFlops Cloud is the launching pad for data startups. 25
Management Science - The Data Scientist 20% 20% 60% Magic Statistics Communications 26
Management Science - The Team Data Integration Mashups Applications Models Visualization Predictions Uncertainty Problems Data Sources Credibility Effective Data Applications Drew Conway http://www.dataists.com/2010/09/the-data-science-venn-diagram/ 27
Big Data 101 The Analytics Stack Economics of Big Data Convergence of the 3 forces Big Data Analytics in the Cloud with CloudR
The Cloud More than buying VMs PaaS/APIs SaaS Per hour pricing infrastructure in 10mins upon sign-up CHEAP Enabling innovations and focusing on your core IPs
Analytics Platform as a Service Hi-Mem instances HPC Clusters Analytics Platform as a Service Hadoop Clusters Databases
HPA Benchmarking comparison* Logistic Regression LEADING LEGACY ANALYTICS SOFTWARE Rows of data 1 billion 1 billion 1 billion Parameters just a few 7 7 Time 80 seconds 44 seconds 95 seconds Data location In memory On disk On disk Nodes 32 5 8 Cores 384 20 24 RAM 1,536 GB 80 GB 120GB Revolution R is faster on the same amount of data, despite using approximately a 20 th as many cores, a 20 th as much RAM, a 6 th as many nodes, and not preloading data into RAM. Months Weeks 10 minutes
CloudR
Thank you Revolution Analytics is the leading commercial provider of software and support for the popular open source R statistics language. E: Laurence.liew@revolutionanalytics.com W: www.revolutionanalytics.com 33