Teradata Unified Big Data Architecture
|
|
- Ezra Barker
- 8 years ago
- Views:
Transcription
1 Teradata Unified Big Data Architecture
2 Agenda Recap the challenges of Big Analytics The 2 analytical gaps for most enterprises Teradata Unified Data Architecture - How we bridge the gaps - The 3 core elements of the architecture - Teradata s solutions in the architecture Bring it all together Teradata, Teradata Aster, and Hadoop. 2
3 Recap of the Big Data Analytics Challenge
4 New and Emerging Sources of Data Petabytes Terabytes User Generated Content User Click Stream Web logs Offer history BIG DATA Mobile Web Sentiment Web A/B testing Dynamic Pricing Social Network External Demographics Business Data Feeds Gigabytes Megabytes CRM Segmentation Offer details Customer Touches Affiliate Networks Search marketing Behavioral Targeting HD Video And using an RDBMS/SQL alone is difficult or impossible ERP Purchase detail So it s the data, right? Support Yes Contacts Purchase record Dynamic Funnels Payment record So it s the analytics, right?. Yes So it s the need for iterative visualisation. Yes Or it is just that it cannot be expressed in SQL Yes Speech to Text Product/Service Logs SMS/MMS 4
5 Big Data Analytics MORE Analytics on ALL the data Enabling All Users, All Tools and Any Data for Capture to Analysis Java, C/C++, Pig, Python, R, SAS, SQL, Excel, BI, Visualisation, etc. Discover and Explore Reporting and Execution in the Enterprise Capture, Store and Refine Audio/ Video Images Docs Text Web & Social Machine Logs CRM SCM ERP 5
6 The Big Data Architecture Today Has Gaps Engineers Gap 1: Analysts Data Scientists Quants Business Analysts Java, C/C++, Pig, Python, R, SAS, SQL, Excel, BI, Visualisation, etc. MapReduce (Processing) Gap 2: File system lacks optimisers, data locality, indexes Data Warehouse Database and Analytic Processing Layer Data Storage and Refining Audio/ Video Images Text Web and Social Machine Logs CRM SCM ERP 6
7 Teradata Unified Big Data Architecture for the Enterprise Engineers Data Scientists Quants Business Analysts Java, C/C++, Pig, Python, R, SAS, SQL, Excel, BI, Visualisation, etc. Aster MapReduce Portfolio Teradata SQL Analytics Portfolio Discovery Platform SQL-H Integrated Data Warehouse SQL-H Capture, Store, Refine Audio/ Video Images Text Web and Social Machine Logs CRM SCM ERP 7
8 Teradata Aster Discovery Platform 5.10 Fastest path to big data apps and new business insights Analysts Customers Business Data Scientists Interactive & Visual Big Data Analytic Apps Develop SQL-H Teradata RDBMS Data Acquisition Module Unpack Pivot Apache Log Parser Data Preparation Module Pathing Graph Statistical Analytics Module Flow Viz Hierarchy Viz Affinity Viz Viz Module Attensity Zementis SAS, R Partner & Add-On Modules Growing the Development Bucket 70+ pre-built functions for data acquisition, preparation, analysis & visualization Richest Add-On Capabilities: Attensity, Zementis, SAS, R Visual IDE & VM-based dev environment: develop apps very fast Process SQL SQL-MapReduce Platform Services (e.g. query planning, dynamic workload management, security ) SQL-MapReduce framework Analyze both multi-structured complex and relational data Store Row Store Column Store Integrated hardware and software appliance Relational-data architecture can be extended for non-relational types and procedural M-R analytics 8
9 Big Data Apps in Days not Weeks or Months DATA SOURCES ASTER DISCOVERY PORTFOLIO Hadoop Data PACKAGED BIG ANALYTICS APPS CUSTOM BIG ANALYTICS APPS Analysts Multi- Structured Data Structured Data Data Acquisition Module Hadoop access Teradata access RDBMS access Data Preparation Module Data Adaptors Data Transformers - JSON, XML, Apache, etc Analytics Module Statistical Pattern Matching Pathing Graph Algorithms Text Visualisation Module Flow Visualizer Hierarchy Flow Sankey Affinity More. Customers Business More OLTP DBMS s Data Scientists 9
10 MapReduce vs. SQL - Reduce Function Data output from Mass Spectrometer Detecting centroids of peaks is highly complex using SQL as it is not a set based operation 10
11 Almost 800 lines of complex SQL 11 SELECT file_id,scan_id,ren_tm,ms_lvl,mz,i AS n_,sum(i) OVER (PARTITION BY file_id, ms_lvl, ren_tm ORDER BY mz ASC ROWS BETWEEN 1 PRECEDING AND 1 PRECEDING) AS p_i,(case WHEN (i > 0) THEN 1 ELSE 0 END) AS Ind,(Ind - SUM(ind) OVER (PARTITION,(weighted_peak_mz BY file_id, * ms_lvl, chrg) / ren_tm ORDER BY mz ASC ROWS BETWEEN AS delta_mz 1 PRECEDING AND 1 PRECEDING)),CAST((CASE,CASE WHEN ( B = 1 THEN CSUM(1,Ind) WHEN B (CASE = 0 AND WHEN Ind = 1 THEN 0 ELSE NULL END) AS DECIMAL(38,0)) SUM((weighted_peak_mz AS CurveID * chrg)) OVER (PARTITION BY file_id, ms_lvl ORDER BY Weighted_peak_mz, scan_id ROWS FROM dd_stg.mzml BETWEEN 1 PRECEDING AND 1 PRECEDING) WHERE ms_lvl = 1 BETWEEN ((weighted_peak_mz * chrg) - delta_mz) AND ((weighted_peak_mz * chrg) + delta_mz) ) WITH DATA THEN 'Y' PRIMARY INDEX (mz) ELSE NULL END) = 'Y' SELECT file_id,scan_id,ren_tm,ms_lvl,mz OR,i (CASE WHEN,CASE WHEN ind = 1 THEN SUM(CurveID+Mark) OVER (PARTITION BY file_id, ms_lvl, ren_tm ORDER BY mz, ind ROWS UNBOUNDED PRECEDING) SUM((weighted_peak_mz * chrg)) OVER (PARTITION BY file_id, ms_lvl ORDER BY Weighted_peak_mz, scan_id ROWS BETWEEN ELSE 1 FOLLOWING NULL END AS AND CurveNum SELECT A.file_id,A.ren_tm,A.scan_id,A.ms_lvl,A.CurveNum 1 FOLLOWING) A.Weighted_Peak_mz,A.ren_tm,A.sum_i FROM (SELECT file_id,scan_id,ren_tm,ms_lvl,mz,n_i BETWEEN ((weighted_peak_mz AS i,a.ren_tm - B.ren_tm AS Diff_Ren_Tm * chrg) - delta_mz) AND ((weighted_peak_mz * chrg) + delta_mz) THEN 'Y',CASE,A.Weighted_Peak_mz - B.Weighted_Peak_mz AS Diff_WP WHEN ELSE NULL,B.CurveNum AS L_CurveNum ( (CASE END) = 'Y',B.Weighted_Peak_mz AS L_Weighted_Peak_mz WHEN n_i OR - p_i > 0 THEN 1,B.ren_tm AS L_ren_tm WHEN n_i (CASE - p_i < WHEN 0 THEN -1,B.sum_i AS L_Sum_I ELSE 0 SUM((weighted_peak_mz * chrg)) OVER (PARTITION BY file_id, ms_lvl ORDER BY Weighted_peak_mz, scan_id ROWS FROM DD_STG.S2_WEIGHTED_CURVE AS A END) BETWEEN - 2 PRECEDING AND 2 PRECEDING) INNER JOIN DD_STG.S2_WEIGHTED_CURVE AS B SUM(CASE,A.Weighted_Peak_mz - B.Weighted_Peak_mz BETWEEN ((weighted_peak_mz AS Diff_WP * chrg) - delta_mz) AND ((weighted_peak_mz * chrg) + delta_mz) ON THEN 'Y' (A.Weighted_Peak_mz - B.Weighted_Peak_mz) BETWEEN AND ,B.CurveNum WHEN n_i - AS p_i > 0 THEN 1 L_CurveNum AND A.ren_tm WHEN n_i ELSE = - p_i NULL B.ren_tm,B.Weighted_Peak_mz < 0 THEN -1 AS L_Weighted_Peak_mz AND END) A.CurveNum ELSE 0 = 'Y' <> B.CurveNum,B.ren_tm AS L_ren_tm AND B.max_i > ( END) OVER OR * A.max_i),B.sum_i (PARTITION BY file_id, ms_lvl, ren_tm AS ORDER BY mz ASC L_Sum_I ROWS BETWEEN 1 PRECEDING AND 1 FROM PRECEDING) (CASE DD_STG.S2_WEIGHTED_CURVE WHEN AS A INNER JOIN DD_STG.S2_WEIGHTED_CURVE ) = 2 THEN 1 ELSE 0 AS B ON (A.Weighted_Peak_mz - B.Weighted_Peak_mz) BETWEEN AND END AS AND Mark A.ren_tm = B.ren_tm,Ind AND A.CurveNum <> B.CurveNum,B AND B.max_i > ( * A.max_i),CurveID ) AS J LEFT JOIN DD_TAB.CHARGE_STATES AS C ON CAST(J.Diff_WP AS DECIMAL(18,2)) = CAST(C.chrg_mz_diff AS DECIMAL(18,2))
12 Procedural code declared to the Aster as new new MapReduce function called PeakPick while (inputiterator.advancetonextrow()) { currintensity=inputiterator.getdoubleat(5); maxintensity=0.0; //Initialise Temp Array for (int i=0; i <= 50; i++){ curvearray[0][i]=0; curvearray[1][i]=0; if (overlapflag==1){ count = 1; else { count = 0; //Find start of Curve, lastintensity is 0 //or previous lastintensity is higher than lastintensity overlapping peaks (double peak curve) if (currintensity > 0 && lastintensity == 0 overlapflag==1){ //Populate Temp Array with Curve points and find maxintensity to derive threshold while (currintensity > 0){ if(maxintensity < currintensity) maxintensity=currintensity; if (overlapflag==1){ overlapflag=0; curvearray[0][count-1]=overlapmz; curvearray[1][count-1]=overlapintensity; PI = overlapintensity; currintensity=inputiterator.getdoubleat(5); curvearray[0][count]=inputiterator.getdoubleat(4); curvearray[1][count]=inputiterator.getdoubleat(5); count++; inputiterator.advancetonextrow(); PI2 = PI; PI = currintensity; 12 currintensity=inputiterator.getdoubleat(5); if (currintensity > PI && PI2 > PI){ //Overlapping Peak found, store MZ and Intensity and start new Curve for next Iteration overlapflag=1; overlapmz=inputiterator.getdoubleat(4); overlapintensity=inputiterator.getdoubleat(5); break; //Process Temp Array to create intermediate metrics while (curvearray[1][curvecount] > 0){ if (curvearray[1][curvecount] > intensitythreshold){ if (maxmz < curvearray[0][curvecount]){ maxmz=curvearray[0][curvecount]; if (minintensity > curvearray[1][curvecount] minintensity == 0){ minintensity=curvearray[1][curvecount]; if (minmz > curvearray[0][curvecount] minmz == 0){ minmz=curvearray[0][curvecount]; sumintensity=sumintensity+curvearray[1][curvecount]; summz=summz+curvearray[0][curvecount]; summzbyintensity=summzbyintensity+(curvearray[0][curvecou nt]*curvearray[1][curvecount]); curvepoints++; curvecount++;
13 SQL MapReduce Reduce Function In Teradata Aster SQL-MR code run by analyst becomes trivial SELECT * FROM PeakPick (ON SELECT * FROM STG.MassSpecLoad) Parameters can easily be included in the function and exposed to the analyst In Hadoop, command line interface means Engineers involved at all times 13
14 TERADATA UNIFIED DATA ARCHITECTURE Data Scientists Quants Customers / Partners Front-Line Workers Engineers Business Analysts Executives Operational Systems LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS Big Data Analytics DISCOVERY PLATFORM INTEGRATED DATA WAREHOUSE Enterprise Analytics CAPTURE STORE REFINE Big Data Management 14 AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
15 The Integrated Data Warehouse Single View of the Business, Cross-Functional SQL based Business Analysts Knowledge Workers Customers/Partners Marketing Executives Front-line Workers Operational Systems Structured schema Productionised Analytics Active BUSINESS INTELLIGENCE DATA MINING APPLICATIONS Complex mixed workloads Highest service level goals Highest resilience 1000 users INTEGRATED DATA WAREHOUSE 15
16 The Discovery Environment Project-led view of data approach for big analytics Business Analysts Data Scientists Power Analysts Rules Discovery Big Analytics using SQL-MR Schema-Lite Interactive Discovery Analytics Load fast, act fast, fail fast analytical workload SQL AND MAP-REDUCE BIG ANALYTICS DATA VISUALISATION Interactive Limited service levels Resilience 10 s users DISCOVERY PLATFORM 16
17 Hadoop Big Data Management Lowest Cost Storage footprint NoSchema design, load raw files Power Analysts Data Scientists IT Professionals Single use Systems MapReduce based Deep history and 1 st level data transformations SPECIAL PURPOSE ANALYTIC TRANSFORMATIONS REGULATORY Simple single use workloads Batch and open source analytics High Data Availability service level goal CAPTURE STORE REFINE High resilience 17
18 TERADATA UNIFIED DATA ARCHITECTURE Data Scientists Quants Customers / Partners Front-Line Workers Engineers Business Analysts Executives Operational Systems LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS Big Data Analytics DISCOVERY PLATFORM INTEGRATED DATA WAREHOUSE Enterprise Analytics CAPTURE STORE REFINE Big Data Management 18 AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
19 Unified Data Architecture Give Any User Any Analytic on Any Data To leverage Big Data you must give all the business analysts in your organization the right analytical tool on all the existing and new data available Unified Data Architecture - architecture that leverages the right technology on the right analytical problems - leveraging best-of-breed technologies Big Data Analytics Teradata and Aster harness the business value of Big Data. Every company needs both a Data Warehouse and a Discovery Platform Big Data Management Hadoop for landing, storing, and refining data Democratise Big Data and Maximise Enterprise Adoption 19
What is Big Data? Mark Whitehorn, Co-Founder, Penguinsoft Consulting Ltd. Global Sponsor:
What is Big Data? Mark Whitehorn, Co-Founder, Penguinsoft Consulting Ltd. Global Sponsor: It s all about me Prof Mark Whitehorn Chair of Analytics School of Computing University of Dundee Scotland Consultant
More informationBIG DATA: FROM HYPE TO REALITY. Leandro Ruiz Presales Partner for C&LA Teradata
BIG DATA: FROM HYPE TO REALITY Leandro Ruiz Presales Partner for C&LA Teradata Evolution in The Use of Information Action s ACTIVATING MAKE it happen! Insights OPERATIONALIZING WHAT IS happening now? PREDICTING
More informationTeradata s Big Data Technology Strategy & Roadmap
Teradata s Big Data Technology Strategy & Roadmap Artur Borycki, Director International Solutions Marketing 18 March 2014 Agenda > Introduction and level-set > Enabling the Logical Data Warehouse > Any
More informationWelcome. Host: Eric Kavanagh. eric.kavanagh@bloorgroup.com. The Briefing Room. Twitter Tag: #briefr
The Briefing Room Welcome Host: Eric Kavanagh eric.kavanagh@bloorgroup.com Twitter Tag: #briefr The Briefing Room Mission! Reveal the essential characteristics of enterprise software, good and bad! Provide
More informationUNIFY YOUR (BIG) DATA
UNIFY YOUR (BIG) DATA ANALYTIC STRATEGY GIVE ANY USER ANY ANALYTIC ON ANY DATA Scott Gnau President, Teradata Labs scott.gnau@teradata.com t Unify Your (Big) Data Analytic Strategy Technology excitement:
More informationADVANCED ANALYTICS AND FRAUD DETECTION THE RIGHT TECHNOLOGY FOR NOW AND THE FUTURE
ADVANCED ANALYTICS AND FRAUD DETECTION THE RIGHT TECHNOLOGY FOR NOW AND THE FUTURE Big Data Big Data What tax agencies are or will be seeing! Big Data Large and increased data volumes New and emerging
More informationINVESTOR PRESENTATION. Third Quarter 2014
INVESTOR PRESENTATION Third Quarter 2014 Note to Investors Certain non-gaap financial information regarding operating results may be discussed during this presentation. Reconciliations of the differences
More informationInvestor Presentation. Second Quarter 2015
Investor Presentation Second Quarter 2015 Note to Investors Certain non-gaap financial information regarding operating results may be discussed during this presentation. Reconciliations of the differences
More informationINVESTOR PRESENTATION. First Quarter 2014
INVESTOR PRESENTATION First Quarter 2014 Note to Investors Certain non-gaap financial information regarding operating results may be discussed during this presentation. Reconciliations of the differences
More informationBig Data and Your Data Warehouse Philip Russom
Big Data and Your Data Warehouse Philip Russom TDWI Research Director for Data Management May 7, 2013 Sponsor Speakers Philip Russom TDWI Research Director, Data Management Chris Twogood VP, Product and
More informationHarnessing the Value of Big Data Analytics
Big Data Analytics Harnessing the Value of Big Data Analytics How to Gain Business Insight Using MapReduce and Apache Hadoop with SQL-Based Analytics By: Shaun Connolly, VP, Corporate Strategy, Hortonworks
More informationArtur Borycki. Director International Solutions Marketing
Artur Borycki Director International Solutions Agenda! Evolution of Teradata s Unified Architecture Analytical and Workloads! Teradata s Reference Information Architecture Evolution of Teradata s" Unified
More informationBig Data Realities Hadoop in the Enterprise Architecture
Big Data Realities Hadoop in the Enterprise Architecture Paul Phillips Director, EMEA, Hortonworks pphillips@hortonworks.com +44 (0)777 444 3857 Hortonworks Inc. 2012 Page 1 Agenda The Growth of Enterprise
More informationHarnessing the Value of Big Data Analytics
Harnessing the Value of Harnessing the Value of By: Shaun Connolly, Vice President, Corporate Strategy, Hortonworks Steve Wooledge, Sr. Director, Product Marketing, Teradata How to Gain Business Insight
More informationWhy Consumer Empowerment is moving retailers from Product Centricity to Customer Centricity
Why Consumer Empowerment is moving retailers from Product Centricity to Customer Centricity CONSUMER TRANSFORMATION TIMELINE Transformation Phase Transformation Phase Transformation Phase Transformation
More informationSAS and Teradata Partnership
SAS and Teradata Partnership Ed Swain Senior Industry Consultant Energy & Resources Ed.Swain@teradata.com 1 Innovation and Leadership Teradata SAS Magic Quadrant for Data Warehouse Database Management
More informationDiscovering Business Insights in Big Data Using SQL-MapReduce
Discovering Business Insights in Big Data Using SQL-MapReduce A Technical Whitepaper Rick F. van der Lans Independent Business Intelligence Analyst R20/Consultancy July 2013 Sponsored by Copyright 2013
More informationData Warehouse Hadoop. Shimpei Kodama 2015/9/29
Data Warehouse Hadoop Shimpei Kodama 2015/9/29 of DWH 1979 Founded 77+ Counties 2,600+ Customers 11,000+ Employees GNo1 L 95% Top 20 Communications 90% Top 20 Finance 75% Top 20 Retail 70% Top 20 Travel
More informationBIG Data Analytics Move to Competitive Advantage
BIG Data Analytics Move to Competitive Advantage where is technology heading today Standardization Open Source Automation Scalability Cloud Computing Mobility Smartphones/ tablets Internet of Things Wireless
More informationBig Data: Making Sense of it all!
Big Data: Making Sense of it all! Jamie Engesser E-mail : jamie@hortonworks.com Page 1 Data Driven Business? Facts not Intuition! Data driven decisions are better decisions its as simple as that. Using
More informationIntegrating a Big Data Platform into Government:
Integrating a Big Data Platform into Government: Drive Better Decisions for Policy and Program Outcomes John Haddad, Senior Director Product Marketing, Informatica Digital Government Institute s Government
More informationOracle Big Data Strategy Simplified Infrastrcuture
Big Data Oracle Big Data Strategy Simplified Infrastrcuture Selim Burduroğlu Global Innovation Evangelist & Architect Education & Research Industry Business Unit Oracle Confidential Internal/Restricted/Highly
More informationRamesh Bhashyam Teradata Fellow Teradata Corporation bhashyam.ramesh@teradata.com
Challenges of Handling Big Data Ramesh Bhashyam Teradata Fellow Teradata Corporation bhashyam.ramesh@teradata.com Trend Too much information is a storage issue, certainly, but too much information is also
More informationUNLEASHING THE VALUE OF THE TERADATA UNIFIED DATA ARCHITECTURE WITH ALTERYX
UNLEASHING THE VALUE OF THE TERADATA UNIFIED DATA ARCHITECTURE WITH ALTERYX 1 Successful companies know that analytics are key to winning customer loyalty, optimizing business processes and beating their
More informationHow To Learn To Use Big Data
Information Technologies Programs Big Data Specialized Studies Accelerate Your Career extension.uci.edu/bigdata Offered in partnership with University of California, Irvine Extension s professional certificate
More informationThe Future of Data Management
The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah (@awadallah) Cofounder and CTO Cloudera Snapshot Founded 2008, by former employees of Employees Today ~ 800 World Class
More informationUp Your R Game. James Taylor, Decision Management Solutions Bill Franks, Teradata
Up Your R Game James Taylor, Decision Management Solutions Bill Franks, Teradata Today s Speakers James Taylor Bill Franks CEO Chief Analytics Officer Decision Management Solutions Teradata 7/28/14 3 Polling
More informationThe Enterprise Data Hub and The Modern Information Architecture
The Enterprise Data Hub and The Modern Information Architecture Dr. Amr Awadallah CTO & Co-Founder, Cloudera Twitter: @awadallah 1 2013 Cloudera, Inc. All rights reserved. Cloudera Overview The Leader
More informationA Next-Generation Analytics Ecosystem for Big Data. Colin White, BI Research September 2012 Sponsored by ParAccel
A Next-Generation Analytics Ecosystem for Big Data Colin White, BI Research September 2012 Sponsored by ParAccel BIG DATA IS BIG NEWS The value of big data lies in the business analytics that can be generated
More informationNavigating Big Data business analytics
mwd a d v i s o r s Navigating Big Data business analytics Helena Schwenk A special report prepared for Actuate May 2013 This report is the third in a series and focuses principally on explaining what
More informationBig Data, Start Small! Dr. Frank Säuberlich, Director Advanced Analytics (Teradata International) 26 th May 2015
Big Data, Start Small! Dr. Frank Säuberlich, Director Advanced Analytics (Teradata International) 26 th May 2015 Agenda Introduction Big Data And The Emergence Of The Logical Data Warehouse Architecture
More informationSurfing the Data Tsunami: A New Paradigm for Big Data Processing and Analytics
Surfing the Data Tsunami: A New Paradigm for Big Data Processing and Analytics Dr. Liangxiu Han Future Networks and Distributed Systems Group (FUNDS) School of Computing, Mathematics and Digital Technology,
More informationEfficient Big Data Analytics using SQL and Map-Reduce
Efficient Big Data Analytics using SQL and Map-Reduce Pekka Kostamaa, VP of Engineering and Big Data Lab ACM Fifteenth International Workshop On Data Warehousing and OLAP DOLAP 2012 Conference, Maui, Hawaii
More informationHow To Use Big Data For Business
Big Data Maturity - The Photo and The Movie Mike Ferguson Managing Director, Intelligent Business Strategies BA4ALL Big Data & Analytics Insight Conference Stockholm, May 2015 About Mike Ferguson Mike
More informationThe Future of Data Management with Hadoop and the Enterprise Data Hub
The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah Cofounder & CTO, Cloudera, Inc. Twitter: @awadallah 1 2 Cloudera Snapshot Founded 2008, by former employees of Employees
More informationGanzheitliches Datenmanagement
Ganzheitliches Datenmanagement für Hadoop Michael Kohs, Senior Sales Consultant @mikchaos The Problem with Big Data Projects in 2016 Relational, Mainframe Documents and Emails Data Modeler Data Scientist
More informationBIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES
BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES Relational vs. Non-Relational Architecture Relational Non-Relational Rational Predictable Traditional Agile Flexible Modern 2 Agenda Big Data
More informationBig Data Just Noise or Does it Matter?
Big Data Just Noise or Does it Matter? Opportunities for Continuous Auditing Presented by: Solon Angel Product Manager Servers The CaseWare Group. Founded in 1988. An industry leader in providing technology
More informationConsistent, Reusable Analytics for Big Data: The Hallmark of Analytic Applications
I D C T E C H N O L O G Y S P O T L I G H T Consistent, Reusable Analytics for Big Data: The Hallmark of Analytic Applications May 2015 Adapted from IDC FutureScape: Worldwide Big Data and Analytics 2015
More informationCERULIUM TERADATA COURSE CATALOG
CERULIUM TERADATA COURSE CATALOG Cerulium Corporation has provided quality Teradata education and consulting expertise for over seven years. We offer customized solutions to maximize your warehouse. Prepared
More informationArchitecting for Big Data Analytics and Beyond: A New Framework for Business Intelligence and Data Warehousing
Architecting for Big Data Analytics and Beyond: A New Framework for Business Intelligence and Data Warehousing Wayne W. Eckerson Director of Research, TechTarget Founder, BI Leadership Forum Business Analytics
More informationCollaborative Big Data Analytics. Copyright 2012 EMC Corporation. All rights reserved.
Collaborative Big Data Analytics 1 Big Data Is Less About Size, And More About Freedom TechCrunch!!!!!!!!! Total data: bigger than big data 451 Group Findings: Big Data Is More Extreme Than Volume Gartner!!!!!!!!!!!!!!!
More informationMike Maxey. Senior Director Product Marketing Greenplum A Division of EMC. Copyright 2011 EMC Corporation. All rights reserved.
Mike Maxey Senior Director Product Marketing Greenplum A Division of EMC 1 Greenplum Becomes the Foundation of EMC s Big Data Analytics (July 2010) E M C A C Q U I R E S G R E E N P L U M For three years,
More informationSAP and Hortonworks Reference Architecture
SAP and Hortonworks Reference Architecture Hortonworks. We Do Hadoop. June Page 1 2014 Hortonworks Inc. 2011 2014. All Rights Reserved A Modern Data Architecture With SAP DATA SYSTEMS APPLICATIO NS Statistical
More informationBig Data Can Drive the Business and IT to Evolve and Adapt
Big Data Can Drive the Business and IT to Evolve and Adapt Ralph Kimball Associates 2013 Ralph Kimball Brussels 2013 Big Data Itself is Being Monetized Executives see the short path from data insights
More informationAGENDA. What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story. Our BIG DATA Roadmap. Hadoop PDW
AGENDA What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story Hadoop PDW Our BIG DATA Roadmap BIG DATA? Volume 59% growth in annual WW information 1.2M Zetabytes (10 21 bytes) this
More informationHIGH PERFORMANCE ANALYTICS FOR TERADATA
F HIGH PERFORMANCE ANALYTICS FOR TERADATA F F BORN AND BRED IN FINANCIAL SERVICES AND HEALTHCARE. DECADES OF EXPERIENCE IN PARALLEL PROGRAMMING AND ANALYTICS. FOCUSED ON MAKING DATA SCIENCE HIGHLY PERFORMING
More informationHow To Analyze Data In A Database In A Microsoft Microsoft Computer System
Big Data Technical Workshop Sept 24 Minneapolis, MN Data Discovery, Modern Architecture & Visualization Big Data Discovery Demo: Financial Services Customer Journey 1 9/25/2014 AGENDA Key Functionality
More informationData Lake In Action: Real-time, Closed Looped Analytics On Hadoop
1 Data Lake In Action: Real-time, Closed Looped Analytics On Hadoop 2 Pivotal s Full Approach It s More Than Just Hadoop Pivotal Data Labs 3 Why Pivotal Exists First Movers Solve the Big Data Utility Gap
More information5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014
5 Keys to Unlocking the Big Data Analytics Puzzle Anurag Tandon Director, Product Marketing March 26, 2014 1 A Little About Us A global footprint. A proven innovator. A leader in enterprise analytics for
More informationUnleashing the Potential of your Social Media and CRM Data. Markus Hirsch Sales Manager
Unleashing the Potential of your Social Media and CRM Data Markus Hirsch Sales Manager To get the RIGHT INFORMATION to the RIGHT PLACE, at the RIGHT TIME and put it in the RIGHT CONTEXT to make the WORLD
More informationThe 4 Pillars of Technosoft s Big Data Practice
beyond possible Big Use End-user applications Big Analytics Visualisation tools Big Analytical tools Big management systems The 4 Pillars of Technosoft s Big Practice Overview Businesses have long managed
More informationOracle Big Data SQL Technical Update
Oracle Big Data SQL Technical Update Jean-Pierre Dijcks Oracle Redwood City, CA, USA Keywords: Big Data, Hadoop, NoSQL Databases, Relational Databases, SQL, Security, Performance Introduction This technical
More informationHarnessing the power of advanced analytics with IBM Netezza
IBM Software Information Management White Paper Harnessing the power of advanced analytics with IBM Netezza How an appliance approach simplifies the use of advanced analytics Harnessing the power of advanced
More informationBig Data and Your Data Warehouse Philip Russom
Big Data and Your Data Warehouse Philip Russom TDWI Research Director for Data Management April 5, 2012 Sponsor Speakers Philip Russom Research Director, Data Management, TDWI Peter Jeffcock Director,
More informationHDP Hadoop From concept to deployment.
HDP Hadoop From concept to deployment. Ankur Gupta Senior Solutions Engineer Rackspace: Page 41 27 th Jan 2015 Where are you in your Hadoop Journey? A. Researching our options B. Currently evaluating some
More informationIntegrated Big Data: Hadoop + DBMS + Discovery for SAS High Performance Analytics
Paper 1828-2014 Integrated Big Data: Hadoop + DBMS + Discovery for SAS High Performance Analytics John Cunningham, Teradata Corporation, Danville, CA ABSTRACT SAS High Performance Analytics (HPA) is a
More informationAligning Your Strategic Initiatives with a Realistic Big Data Analytics Roadmap
Aligning Your Strategic Initiatives with a Realistic Big Data Analytics Roadmap 3 key strategic advantages, and a realistic roadmap for what you really need, and when 2012, Cognizant Topics to be discussed
More informationArchitecting for the Internet of Things & Big Data
Architecting for the Internet of Things & Big Data Robert Stackowiak, Oracle North America, VP Information Architecture & Big Data September 29, 2014 Safe Harbor Statement The following is intended to
More informationBig Data and Analytics in Government
Big Data and Analytics in Government Nov 29, 2012 Mark Johnson Director, Engineered Systems Program 2 Agenda What Big Data Is Government Big Data Use Cases Building a Complete Information Solution Conclusion
More informationArchitectures for Big Data Analytics A database perspective
Architectures for Big Data Analytics A database perspective Fernando Velez Director of Product Management Enterprise Information Management, SAP June 2013 Outline Big Data Analytics Requirements Spectrum
More informationAn Integrated Analytics & Big Data Infrastructure September 21, 2012 Robert Stackowiak, Vice President Data Systems Architecture Oracle Enterprise
An Integrated Analytics & Big Data Infrastructure September 21, 2012 Robert Stackowiak, Vice President Data Systems Architecture Oracle Enterprise Solutions Group The following is intended to outline our
More informationAchieving Business Value through Big Data Analytics Philip Russom
Achieving Business Value through Big Data Analytics Philip Russom TDWI Research Director for Data Management October 3, 2012 Sponsor 2 Speakers Philip Russom Research Director, Data Management, TDWI Brian
More informationA Tour of the Zoo the Hadoop Ecosystem Prafulla Wani
A Tour of the Zoo the Hadoop Ecosystem Prafulla Wani Technical Architect - Big Data Syntel Agenda Welcome to the Zoo! Evolution Timeline Traditional BI/DW Architecture Where Hadoop Fits In 2 Welcome to
More informationHortonworks & SAS. Analytics everywhere. Page 1. Hortonworks Inc. 2011 2014. All Rights Reserved
Hortonworks & SAS Analytics everywhere. Page 1 A change in focus. A shift in Advertising From mass branding A shift in Financial Services From Educated Investing A shift in Healthcare From mass treatment
More informationHOW TO DO A SMART DATA PROJECT
April 2014 Smart Data Strategies HOW TO DO A SMART DATA PROJECT Guideline www.altiliagroup.com Summary ALTILIA s approach to Smart Data PROJECTS 3 1. BUSINESS USE CASE DEFINITION 4 2. PROJECT PLANNING
More informationManaging Big Data with Hadoop & Vertica. A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database
Managing Big Data with Hadoop & Vertica A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database Copyright Vertica Systems, Inc. October 2009 Cloudera and Vertica
More informationEnd Small Thinking about Big Data
CITO Research End Small Thinking about Big Data SPONSORED BY TERADATA Introduction It is time to end small thinking about big data. Instead of thinking about how to apply the insights of big data to business
More informationDatenverwaltung im Wandel - Building an Enterprise Data Hub with
Datenverwaltung im Wandel - Building an Enterprise Data Hub with Cloudera Bernard Doering Regional Director, Central EMEA, Cloudera Cloudera Your Hadoop Experts Founded 2008, by former employees of Employees
More informationFrancois Ajenstat, Tableau Stephanie McReynolds, Aster Data Steve e Wooledge, Aster Data
Deep Data Exploration: Find Patterns in Your Data Faster & Easier Curt Monash, Founder and President, Monash Research Francois Ajenstat, Tableau Stephanie McReynolds, Aster Data Steve e Wooledge, Aster
More informationHow To Turn Big Data Into An Insight
mwd a d v i s o r s Turning Big Data into Big Insights Helena Schwenk A special report prepared for Actuate May 2013 This report is the fourth in a series and focuses principally on explaining what s needed
More informationTableau s Place in a Big Data Architecture DAMA, Tableau User Group Meeting November 13, 2014
s Place in a Big Data Architecture DAA, User Group eeting November 13, 2014 Agenda BI/DW Workload Categories & Three Integration odels Capability odels Architecture Patterns Summary Q & A 2 Workload Categories
More informationOracle Big Data Discovery Unlock Potential in Big Data Reservoir
Oracle Big Data Discovery Unlock Potential in Big Data Reservoir Gokula Mishra Premjith Balakrishnan Business Analytics Product Group September 29, 2014 Copyright 2014, Oracle and/or its affiliates. All
More informationHow to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning
How to use Big Data in Industry 4.0 implementations LAURI ILISON, PhD Head of Big Data and Machine Learning Big Data definition? Big Data is about structured vs unstructured data Big Data is about Volume
More informationEnd to End Solution to Accelerate Data Warehouse Optimization. Franco Flore Alliance Sales Director - APJ
End to End Solution to Accelerate Data Warehouse Optimization Franco Flore Alliance Sales Director - APJ Big Data Is Driving Key Business Initiatives Increase profitability, innovation, customer satisfaction,
More informationHow To Handle Big Data With A Data Scientist
III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution
More informationVIEWPOINT. High Performance Analytics. Industry Context and Trends
VIEWPOINT High Performance Analytics Industry Context and Trends In the digital age of social media and connected devices, enterprises have a plethora of data that they can mine, to discover hidden correlations
More informationWhat is a Petabyte? Gain Big or Lose Big; Measuring the Operational Risks of Big Data. Agenda
April - April - Gain Big or Lose Big; Measuring the Operational Risks of Big Data YouTube video here http://www.youtube.com/watch?v=o7uzbcwstu April, 0 Steve Woolley, Sr. Manager Business Continuity Dennis
More informationThe Bloor Group. The Pillars of Data Science
The Pillars of Data Science The Three Pillars DOMAIN KNOWLEDGE The DS/DA needs to know the business STATISTICAL SKILLS Knowing how to use software to analyze data TECHNOLOGY KNOWLEDGE Knowing how to leverage
More informationHow To Create A Business Intelligence (Bi)
Oracle Business Analytics Overview Markus Päivinen Business Analytics Country Leader, Finland May 2014 1 Presentation content What are the requirements for modern BI Trend in Business Analytics Big Data
More informationGetting Started Practical Input For Your Roadmap
Getting Started Practical Input For Your Roadmap Mike Ferguson Managing Director, Intelligent Business Strategies BA4ALL Big Data & Analytics Insight Conference Stockholm, May 2015 About Mike Ferguson
More informationWHITE PAPER. Harnessing the Power of Advanced Analytics How an appliance approach simplifies the use of advanced analytics
WHITE PAPER Harnessing the Power of Advanced How an appliance approach simplifies the use of advanced analytics Introduction The Netezza TwinFin i-class advanced analytics appliance pushes the limits of
More informationMoving From Hadoop to Spark
+ Moving From Hadoop to Spark Sujee Maniyam Founder / Principal @ www.elephantscale.com sujee@elephantscale.com Bay Area ACM meetup (2015-02-23) + HI, Featured in Hadoop Weekly #109 + About Me : Sujee
More informationAre You Ready for Big Data?
Are You Ready for Big Data? Jim Gallo National Director, Business Analytics April 10, 2013 Agenda What is Big Data? How do you leverage Big Data in your company? How do you prepare for a Big Data initiative?
More informationNoSQL for SQL Professionals William McKnight
NoSQL for SQL Professionals William McKnight Session Code BD03 About your Speaker, William McKnight President, McKnight Consulting Group Frequent keynote speaker and trainer internationally Consulted to
More informationwww.intelligentbusiness.biz mferguson@intelligentbusiness.biz Twitter: @mikeferguson1
Welcome to Today s Web Seminar! March 15, 2011 12:00PM ET Sponsored by: Hosted by: Eric Kavanagh is the host of DM Radio and Information Management's Webcasts. He is a veteran journalist and consultant
More informationBIG DATA TECHNOLOGY. Hadoop Ecosystem
BIG DATA TECHNOLOGY Hadoop Ecosystem Agenda Background What is Big Data Solution Objective Introduction to Hadoop Hadoop Ecosystem Hybrid EDW Model Predictive Analysis using Hadoop Conclusion What is Big
More informationHow To Scale Out Of A Nosql Database
Firebird meets NoSQL (Apache HBase) Case Study Firebird Conference 2011 Luxembourg 25.11.2011 26.11.2011 Thomas Steinmaurer DI +43 7236 3343 896 thomas.steinmaurer@scch.at www.scch.at Michael Zwick DI
More informationESS event: Big Data in Official Statistics. Antonino Virgillito, Istat
ESS event: Big Data in Official Statistics Antonino Virgillito, Istat v erbi v is 1 About me Head of Unit Web and BI Technologies, IT Directorate of Istat Project manager and technical coordinator of Web
More informationYu Xu Pekka Kostamaa Like Gao. Presented By: Sushma Ajjampur Jagadeesh
Yu Xu Pekka Kostamaa Like Gao Presented By: Sushma Ajjampur Jagadeesh Introduction Teradata s parallel DBMS can hold data sets ranging from few terabytes to multiple petabytes. Due to explosive data volume
More informationReference Architecture, Requirements, Gaps, Roles
Reference Architecture, Requirements, Gaps, Roles The contents of this document are an excerpt from the brainstorming document M0014. The purpose is to show how a detailed Big Data Reference Architecture
More informationBig Data, Data Analytics and Actuaries. Adam Driussi, Quantium
Big Data, Data Analytics and Actuaries Adam Driussi, Quantium Companies are collecting data like never before 3 Leading to massive volumes of data for analysis Volume Petabytes 2.5 PB Walmart database
More informationHow the oil and gas industry can gain value from Big Data?
How the oil and gas industry can gain value from Big Data? Arild Kristensen Nordic Sales Manager, Big Data Analytics arild.kristensen@no.ibm.com, tlf. +4790532591 April 25, 2013 2013 IBM Corporation Dilbert
More informationBig Data on Microsoft Platform
Big Data on Microsoft Platform Prepared by GJ Srinivas Corporate TEG - Microsoft Page 1 Contents 1. What is Big Data?...3 2. Characteristics of Big Data...3 3. Enter Hadoop...3 4. Microsoft Big Data Solutions...4
More informationIntegrating Hadoop. Into Business Intelligence & Data Warehousing. Philip Russom TDWI Research Director for Data Management, April 9 2013
Integrating Hadoop Into Business Intelligence & Data Warehousing Philip Russom TDWI Research Director for Data Management, April 9 2013 TDWI would like to thank the following companies for sponsoring the
More informationAdvanced In-Database Analytics
Advanced In-Database Analytics Tallinn, Sept. 25th, 2012 Mikko-Pekka Bertling, BDM Greenplum EMEA 1 That sounds complicated? 2 Who can tell me how best to solve this 3 What are the main mathematical functions??
More information<Insert Picture Here> Extending Hyperion BI with the Oracle BI Server
Extending Hyperion BI with the Oracle BI Server Mark Ostroff Sr. BI Solutions Consultant Agenda Hyperion BI versus Hyperion BI with OBI Server Benefits of using Hyperion BI with the
More informationHow To Write A Bigbench Benchmark For A Retailer
BigBench Overview Towards a Comprehensive End-to-End Benchmark for Big Data - bankmark UG (haftungsbeschränkt) 02/04/2015 @ SPEC RG Big Data The BigBench Proposal End to end benchmark Application level
More informationBig Data Analytics Platform @ Nokia
Big Data Analytics Platform @ Nokia 1 Selecting the Right Tool for the Right Workload Yekesa Kosuru Nokia Location & Commerce Strata + Hadoop World NY - Oct 25, 2012 Agenda Big Data Analytics Platform
More information