BMW11: Dealing with the Massive Data Generated by Many-Core Systems. Dr Don Grice IBM Corporation

Size: px
Start display at page:

Download "BMW11: Dealing with the Massive Data Generated by Many-Core Systems. Dr Don Grice. 2011 IBM Corporation"

Transcription

1 BMW11: Dealing with the Massive Data Generated by Many-Core Systems Dr Don Grice

2 IBM Systems and Technology Group Title: Dealing with the Massive Data Generated by Many Core Systems. Abstract: Multi-core and Many-core architectures are enabling computing systems that are more powerful than ever. The amount of data being generated by these systems is becoming an issue in several areas, including storage of results, movement of intermediate and final results, and the ability to consume the data and transform it into 'information'. As we move forward we need to be developing HW and SW methods to deal with this massive data explosion. Data reduction/simplification and real time analytics will involve more computation but may be one of the most promising methods for dealing with this flood of newly generated data.

3 TYPES OF DATA MANIPULATION COMPUTE INTENSIVE DATA INTENSIVE NETWORK INTENSIVE 3

4 Styles of Massively Parallel Workloads Data Intensive (Data At Rest) Data Intensive (Data Needs to Move) Hadoop/MapReduce (BigInsights) Data at Rest*: High Volume Mixed Variety Low Velocity Input Data Mappers Reducers Global Analytics: View of All Data Required Data Must be Moved Higher Velocity Network is Critical = compute node (*pre-partitioned) Output Data Data Intensive : Data in Motion (Streaming) Data in Motion: High Velocity Mixed Variety High Volume* (*over time) SPL, C, Java Reactive Analytics Extreme Ingestion Compute Intensive (Data Generators) C/C++, Fortran, MPI, OpenMP Long Running Small Input Massive Output Generative Modeling Extreme Physics 4

5 Styles of Massively Parallel Workloads Embarassingly Parallel Data Intensive (Data At Rest) Network Dependent Data Intensive (Data Needs to Move) Hadoop/MapReduce (BigInsights) Data at Rest*: High Volume Mixed Variety Low Velocity Input Data Mappers Reducers Global Analytics: View of All Data Required Data Must be Moved Higher Velocity Network is Critical = compute node (*pre-partitioned) Output Data Data Intensive : Data in Motion (Streaming) Data in Motion: High Velocity Mixed Variety High Volume* (*over time) SPL, C, Java Reactive Analytics Extreme Ingestion Compute Intensive (Data Generators) C/C++, Fortran, MPI, OpenMP Long Running Small Input Massive Output Generative Modeling Extreme Physics 5

6 Data Intensive Applications (Large Data)

7 New Big Data Brings New Opportunities, Requires New Analytics Exa Peta Up to 10,000 Times larger Data Scale Data Scale Tera Giga Mega Kilo Data at Rest Traditional Data Warehouse and Business Intelligence Data in Motion Up to 10,000 times faster yr mo wk day hr min sec ms μs Occasional Frequent Real-time Decision Frequency Telco Promotions 100,000 records/sec, 6B/day 10 ms/decision 270TB for Deep Analytics DeepQA 100s GB for Deep Analytics 3 sec/decision Smart Traffic 250K GPS probes/sec 630K segments/sec 2 ms/decision, 4K vehicles

8 Petascale Analytics, Appliances and Ecosystem Big Data is the new resource. The new opportunity is Big Analytics. Every Smarter Planet solution will depend on it. Market leadership in the Era of Analytics will be taken by the first player to deliver high volumes of easyto-use Smarter Planet solutions. Ultimate success will require a Petascale Analytics Appliance and a rich ecosystem of data, algorithms and skills. Smarter Planet Faster Decisions Deeper Insights

9 Maximum Insight Requires Combining Deep and Reactive Analytics Exa Deep Analytics Government and Telco industries are leading this trend Peta Hypotheses Deep Predictions Data Scale Data Scale Tera Giga Mega Integration History Traditional Data Warehouse and Business Intelligence Integration Feedback Observations Reality Fast Directly integrating Reactive and Deep Analytics enables feedbackdriven insight optimization Actions Integration Reactive Analytics Kilo yr mo wk day hr min sec ms μs Occasional Frequent Real-time Decision Frequency

10 Watson Feb. 14 / 15 / 16 IBM Research built a computer system that is able to compete with humans at the game of Jeopardy: Human vs. Machine contest. Named Watson, the computer is designed to rival the human mind Answering questions in natural language poses a grand challenge in computer science, and the Jeopardy! clue format is a great way to showcase: Broad range of topics, such as history, literature, politics, popular culture and science Nature of the clues, requires analyzing subtle meaning, irony, riddles and other complexities Based on the science of Question Answering (QA); differs from conventional search Natural Language / Human Interactions Critical for implementing useful business applications such as: Medical diagnosis Regulatory compliance Customer relationship management Help desk support

11 Compute Intensive Workloads (Traditional HPC ) 11

12 IBM Systems and Technology Group Fundamental Issues with Large Scale HPC Compute Intensive Workloads Power Efficiency TCO Programmability and Scaleout Frequency is Plateaued More Parallelism is needed Balanced BWs are required for sustained Perf Shared Memory Model vs I/O Accelerator Model Availability and Reliability More Circuitry is required Technology Scale makes it worse Design for Availability is required Data Management and Cost of Storing/Moving Data Time Steps & Checkpoints Storage Cost, Energy Cost, BW, Latency Life Cycle Management

13 Amount of Data Generated Growing Much Faster Than BW to Store or Retrieve it Example: 100x improvement in Machine Performance Core Frequency has Plateaued 100x Performance -> >100x more cores Memory per core ~ constant? -> >100x more memory Checkpoint Data Increase >100x Plus frequency may increase due to reliability changes Time Step Data Increase at least 100x? (with Performance) Disk and Tape BWs are basically Plateaued (~100MB/s) Compression Methods are not improving much Only provides ~2x BW boost at most in any event Capacity Growing at 20-30% CGR but not BW Amount of Disk/Tape needs to grow >100x to match BW Some relief possible with Write Duty Cycle Utilization Cache locally and take full interval to write it out Pre-stage Reads 13

14 Example of Data Volume Gap Example of Data Volume Gap Growing for Commercial Users BW Gap is even larger! 4% CAGR 14

15 Data Centric Computing Network Register Stack Server(s) CPU s Data Set Size Increases Downward Server(s) CPU s Domain Functional Units SMP Bus Memory Memory OS/SMP I/O FLASH I/O FLASH High Speed Cluster Network High Speed Cluster Network Cluster/ System LAN/SAN WAN? Local Storage Node Disk, Tape? Flash? Remote Storage Node LAN/SAN.. WAN? Disk or Tape Local Storage Node Remote Storage Node Multi-Cluster Grid? Disk or Tape 15

16 SUMMARY Data Volume and BW is Exploding in many Areas Multicore/Many Core Compute Intensive Systems Are generating more data and faster than ever before Also using more Memory due to Frequency Stabilization Data Storage BWs are not improving much Balance of Compute to I/O and Storage will need to shift Compute Intensive Workloads will also interact with Data Intensive Workloads in Workflow environments Data Life Cycle Management, Prestaging and Intelligent Writing will become increasingly more important as machines grow in capability 16

17 IBM Systems and Technology Group Thank you......any Questions?

Data Centric Computing Revisited

Data Centric Computing Revisited Piyush Chaudhary Technical Computing Solutions Data Centric Computing Revisited SPXXL/SCICOMP Summer 2013 Bottom line: It is a time of Powerful Information Data volume is on the rise Dimensions of data

More information

IBM Centennial Getting ready for a Smarter Planet & Big Data

IBM Centennial Getting ready for a Smarter Planet & Big Data IBM Centennial Getting ready for a Smarter Planet & Big Data First Byte Symposium 1st Anniversary of LSDF for Life Sciences at Bioquant Heidelberg 26 Mai 2011 Dieter Münk Vice President IBM WW Storage

More information

Von Social Media zum Social Business Ein Megatrend für die Geschäftswelt

Von Social Media zum Social Business Ein Megatrend für die Geschäftswelt Stephan Schneider Executive Technology Briefer 07/11/2013 Von Social Media zum Social Business Ein Megatrend für die Geschäftswelt Our experiences are changing in the new Social world How I Buy Interacting

More information

Data Centric Systems (DCS)

Data Centric Systems (DCS) Data Centric Systems (DCS) Architecture and Solutions for High Performance Computing, Big Data and High Performance Analytics High Performance Computing with Data Centric Systems 1 Data Centric Systems

More information

Beyond Watson: The Business Implications of Big Data

Beyond Watson: The Business Implications of Big Data Beyond Watson: The Business Implications of Big Data Shankar Venkataraman IBM Program Director, STSM, Big Data August 10, 2011 The World is Changing and Becoming More INSTRUMENTED INTERCONNECTED INTELLIGENT

More information

IBM Data Warehousing and Analytics Portfolio Summary

IBM Data Warehousing and Analytics Portfolio Summary IBM Information Management IBM Data Warehousing and Analytics Portfolio Summary Information Management Mike McCarthy IBM Corporation mmccart1@us.ibm.com IBM Information Management Portfolio Current Data

More information

Beyond von Neumann. Dr. Mark E. Dean, PhD Fisher Distinguished Professor UTK College of Eng. 2009 IBM Corporation

Beyond von Neumann. Dr. Mark E. Dean, PhD Fisher Distinguished Professor UTK College of Eng. 2009 IBM Corporation Beyond von Neumann Dr. Mark E. Dean, PhD Fisher Distinguished Professor UTK College of Eng. 2009 IBM Corporation Businesses are dying of thirst in an ocean of data 90% of the world s data was created in

More information

Accelerating Hadoop MapReduce Using an In-Memory Data Grid

Accelerating Hadoop MapReduce Using an In-Memory Data Grid Accelerating Hadoop MapReduce Using an In-Memory Data Grid By David L. Brinker and William L. Bain, ScaleOut Software, Inc. 2013 ScaleOut Software, Inc. 12/27/2012 H adoop has been widely embraced for

More information

Watson A System Designed for Answers

Watson A System Designed for Answers IBM Systems and Technology February 2011 An IBM White Paper Watson A System Designed for Answers The future of workload optimized systems design 2 Watson A System Designed for Answers Executive summary

More information

Impact of Big Data growth On Transparent Computing

Impact of Big Data growth On Transparent Computing Impact of Big Data growth On Transparent Computing Michael A. Greene Intel Vice President, Software and Services Group, General Manager, System Technologies and Optimization 1 Transparent Computing (TC)

More information

Dr. John E. Kelly III Senior Vice President, Director of Research. Differentiating IBM: Research

Dr. John E. Kelly III Senior Vice President, Director of Research. Differentiating IBM: Research Dr. John E. Kelly III Senior Vice President, Director of Research Differentiating IBM: Research IBM Research Priorities Impact on IBM and the Marketplace Globalization and Leverage Balanced Research Agenda

More information

SAP HANA Power of In-Memory Computing. Boonchu Chumsantivut Solution Advisor, SAP Thailand Ltd.

SAP HANA Power of In-Memory Computing. Boonchu Chumsantivut Solution Advisor, SAP Thailand Ltd. Power of In-Memory Computing Boonchu Chumsantivut Solution Advisor, SAP Thailand Ltd. Opportunities Speed GPS Emails Reality: Information Explosion Gartner - Enterprise data will grow 650% over the next

More information

A New Era Of Analytic

A New Era Of Analytic Penang egovernment Seminar 2014 A New Era Of Analytic Megat Anuar Idris Head, Project Delivery, Business Analytics & Big Data Agenda Overview of Big Data Case Studies on Big Data Big Data Technology Readiness

More information

Petascale Software Challenges. Piyush Chaudhary piyushc@us.ibm.com High Performance Computing

Petascale Software Challenges. Piyush Chaudhary piyushc@us.ibm.com High Performance Computing Petascale Software Challenges Piyush Chaudhary piyushc@us.ibm.com High Performance Computing Fundamental Observations Applications are struggling to realize growth in sustained performance at scale Reasons

More information

High Performance Computing. Course Notes 2007-2008. HPC Fundamentals

High Performance Computing. Course Notes 2007-2008. HPC Fundamentals High Performance Computing Course Notes 2007-2008 2008 HPC Fundamentals Introduction What is High Performance Computing (HPC)? Difficult to define - it s a moving target. Later 1980s, a supercomputer performs

More information

Big Data: Image & Video Analytics

Big Data: Image & Video Analytics Big Data: Image & Video Analytics How it could support Archiving & Indexing & Searching Dieter Haas, IBM Deutschland GmbH The Big Data Wave 60% of internet traffic is multimedia content (images and videos)

More information

Arif Goelmhd Goelammohamed Solutions Architect. @agoelammohamed. Hyperconverged Infrastructure: The How-To and Why Now?

Arif Goelmhd Goelammohamed Solutions Architect. @agoelammohamed. Hyperconverged Infrastructure: The How-To and Why Now? Arif Goelmhd Goelammohamed Solutions Architect @agoelammohamed Hyperconverged Infrastructure: The How-To and Why Now? Agenda: 1. SimpliVity Overview 2. The Problem 3. The Solution 4. Demo Simplify IT with

More information

IBM System x reference architecture solutions for big data

IBM System x reference architecture solutions for big data IBM System x reference architecture solutions for big data Easy-to-implement hardware, software and services for analyzing data at rest and data in motion Highlights Accelerates time-to-value with scalable,

More information

Massive Scale Analytics for a Smarter Planet

Massive Scale Analytics for a Smarter Planet David Konopnicki - Haifa Research Lab Massive Scale Analytics for a Smarter Planet The Big Data Challenge Manage and benefit from massive and growing amounts of data 44x growth in coming decade from 800,000

More information

SAP HANA SAP s In-Memory Database. Dr. Martin Kittel, SAP HANA Development January 16, 2013

SAP HANA SAP s In-Memory Database. Dr. Martin Kittel, SAP HANA Development January 16, 2013 SAP HANA SAP s In-Memory Database Dr. Martin Kittel, SAP HANA Development January 16, 2013 Disclaimer This presentation outlines our general product direction and should not be relied on in making a purchase

More information

Oracle Big Data SQL Technical Update

Oracle Big Data SQL Technical Update Oracle Big Data SQL Technical Update Jean-Pierre Dijcks Oracle Redwood City, CA, USA Keywords: Big Data, Hadoop, NoSQL Databases, Relational Databases, SQL, Security, Performance Introduction This technical

More information

IBM System x SAP HANA

IBM System x SAP HANA Place photo here IBM System x SAP HANA, IBM System X IBM SAP: 42 2012 Largest HANA implementation worldwide with 100 Terrabyte powered by IBM 2011 IBM Unveils Next Generation Smart Cloud Platform for Business

More information

Big Data on AWS. Services Overview. Bernie Nallamotu Principle Solutions Architect

Big Data on AWS. Services Overview. Bernie Nallamotu Principle Solutions Architect on AWS Services Overview Bernie Nallamotu Principle Solutions Architect \ So what is it? When your data sets become so large that you have to start innovating around how to collect, store, organize, analyze

More information

Introducing Oracle Exalytics In-Memory Machine

Introducing Oracle Exalytics In-Memory Machine Introducing Oracle Exalytics In-Memory Machine Jon Ainsworth Director of Business Development Oracle EMEA Business Analytics 1 Copyright 2011, Oracle and/or its affiliates. All rights Agenda Topics Oracle

More information

Trends in High-Performance Computing for Power Grid Applications

Trends in High-Performance Computing for Power Grid Applications Trends in High-Performance Computing for Power Grid Applications Franz Franchetti ECE, Carnegie Mellon University www.spiral.net Co-Founder, SpiralGen www.spiralgen.com This talk presents my personal views

More information

Big Data: A Storage Systems Perspective Muthukumar Murugan Ph.D. HP Storage Division

Big Data: A Storage Systems Perspective Muthukumar Murugan Ph.D. HP Storage Division Big Data: A Storage Systems Perspective Muthukumar Murugan Ph.D. HP Storage Division In this talk Big data storage: Current trends Issues with current storage options Evolution of storage to support big

More information

Maximizing Hadoop Performance with Hardware Compression

Maximizing Hadoop Performance with Hardware Compression Maximizing Hadoop Performance with Hardware Compression Robert Reiner Director of Marketing Compression and Security Exar Corporation November 2012 1 What is Big? sets whose size is beyond the ability

More information

Big Data Analytics. with EMC Greenplum and Hadoop. Big Data Analytics. Ofir Manor Pre Sales Technical Architect EMC Greenplum

Big Data Analytics. with EMC Greenplum and Hadoop. Big Data Analytics. Ofir Manor Pre Sales Technical Architect EMC Greenplum Big Data Analytics with EMC Greenplum and Hadoop Big Data Analytics with EMC Greenplum and Hadoop Ofir Manor Pre Sales Technical Architect EMC Greenplum 1 Big Data and the Data Warehouse Potential All

More information

2010 Ingres Corporation. Interactive BI for Large Data Volumes Silicon India BI Conference, 2011, Mumbai Vivek Bhatnagar, Ingres Corporation

2010 Ingres Corporation. Interactive BI for Large Data Volumes Silicon India BI Conference, 2011, Mumbai Vivek Bhatnagar, Ingres Corporation Interactive BI for Large Data Volumes Silicon India BI Conference, 2011, Mumbai Vivek Bhatnagar, Ingres Corporation Agenda Need for Fast Data Analysis & The Data Explosion Challenge Approaches Used Till

More information

Hur hanterar vi utmaningar inom området - Big Data. Jan Östling Enterprise Technologies Intel Corporation, NER

Hur hanterar vi utmaningar inom området - Big Data. Jan Östling Enterprise Technologies Intel Corporation, NER Hur hanterar vi utmaningar inom området - Big Data Jan Östling Enterprise Technologies Intel Corporation, NER Legal Disclaimers All products, computer systems, dates, and figures specified are preliminary

More information

Business opportunities from IOT and Big Data. Joachim Aertebjerg Director Enterprise Solution Sales Intel EMEA

Business opportunities from IOT and Big Data. Joachim Aertebjerg Director Enterprise Solution Sales Intel EMEA Business opportunities from IOT and Big Data Joachim Aertebjerg Director Enterprise Solution Sales Intel EMEA HOW INTEL IS TRANSFORMING COMPUTING? Smarter Devices Applications of Big Data Compute for Internet

More information

How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda

How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda 1 Outline Build a cost-efficient Swift cluster with expected performance Background & Problem Solution Experiments

More information

TUT NoSQL Seminar (Oracle) Big Data

TUT NoSQL Seminar (Oracle) Big Data Timo Raitalaakso +358 40 848 0148 rafu@solita.fi TUT NoSQL Seminar (Oracle) Big Data 11.12.2012 Timo Raitalaakso MSc 2000 Work: Solita since 2001 Senior Database Specialist Oracle ACE 2012 Blog: http://rafudb.blogspot.com

More information

Preview of Oracle Database 12c In-Memory Option. Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Preview of Oracle Database 12c In-Memory Option. Copyright 2013, Oracle and/or its affiliates. All rights reserved. Preview of Oracle Database 12c In-Memory Option 1 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any

More information

Poslovni slučajevi upotrebe IBM Netezze

Poslovni slučajevi upotrebe IBM Netezze Poslovni slučajevi upotrebe IBM Netezze data at the Speed and with Simplicity businesses need 25. ožujak 2015. vedran.travica@hr.ibm.com Agenda A. IBM PureData for Analytics Netezza B. Scenarij 1.: Novi

More information

Big Data Are You Ready? Thomas Kyte http://asktom.oracle.com

Big Data Are You Ready? Thomas Kyte http://asktom.oracle.com Big Data Are You Ready? Thomas Kyte http://asktom.oracle.com The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated

More information

Big Data Challenges in Bioinformatics

Big Data Challenges in Bioinformatics Big Data Challenges in Bioinformatics BARCELONA SUPERCOMPUTING CENTER COMPUTER SCIENCE DEPARTMENT Autonomic Systems and ebusiness Pla?orms Jordi Torres Jordi.Torres@bsc.es Talk outline! We talk about Petabyte?

More information

Global Technology Outlook 2011

Global Technology Outlook 2011 Global Technology Outlook 2011 Global Technology Outlook 2011 Since 1982, The Global Technology Outlook had identified significant technology trends five to even 10 years before they have come to realization.

More information

News and trends in Data Warehouse Automation, Big Data and BI. Johan Hendrickx & Dirk Vermeiren

News and trends in Data Warehouse Automation, Big Data and BI. Johan Hendrickx & Dirk Vermeiren News and trends in Data Warehouse Automation, Big Data and BI Johan Hendrickx & Dirk Vermeiren Extreme Agility from Source to Analysis DWH Appliances & DWH Automation Typical Architecture 3 What Business

More information

Hyper-converged IT drives: - TCO cost savings - data protection - amazing operational excellence

Hyper-converged IT drives: - TCO cost savings - data protection - amazing operational excellence Hyper-converged IT drives: - TCO cost savings - data protection - amazing operational excellence Sebastian Nowicki SimpliVity is one of the biggest innovations in enterprise computing since ware. ~John

More information

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Copyright 2012, Oracle and/or its affiliates. All rights reserved. 1 Oracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales Agenda Quick Overview on BDA and its Positioning Product Details and Updates Security and Encryption New Hadoop Versions

More information

Big Data Analytics - Accelerated. stream-horizon.com

Big Data Analytics - Accelerated. stream-horizon.com Big Data Analytics - Accelerated stream-horizon.com StreamHorizon & Big Data Integrates into your Data Processing Pipeline Seamlessly integrates at any point of your your data processing pipeline Implements

More information

Outline. High Performance Computing (HPC) Big Data meets HPC. Case Studies: Some facts about Big Data Technologies HPC and Big Data converging

Outline. High Performance Computing (HPC) Big Data meets HPC. Case Studies: Some facts about Big Data Technologies HPC and Big Data converging Outline High Performance Computing (HPC) Towards exascale computing: a brief history Challenges in the exascale era Big Data meets HPC Some facts about Big Data Technologies HPC and Big Data converging

More information

System Architecture. In-Memory Database

System Architecture. In-Memory Database System Architecture for Are SSDs Ready for Enterprise Storage Systems In-Memory Database Anil Vasudeva, President & Chief Analyst, Research 2007-13 Research All Rights Reserved Copying Prohibited Contact

More information

SCALABLE FILE SHARING AND DATA MANAGEMENT FOR INTERNET OF THINGS

SCALABLE FILE SHARING AND DATA MANAGEMENT FOR INTERNET OF THINGS Sean Lee Solution Architect, SDI, IBM Systems SCALABLE FILE SHARING AND DATA MANAGEMENT FOR INTERNET OF THINGS Agenda Converging Technology Forces New Generation Applications Data Management Challenges

More information

IBM Platform Computing Cloud Service Ready to use Platform LSF & Symphony clusters in the SoftLayer cloud

IBM Platform Computing Cloud Service Ready to use Platform LSF & Symphony clusters in the SoftLayer cloud IBM Platform Computing Cloud Service Ready to use Platform LSF & Symphony clusters in the SoftLayer cloud February 25, 2014 1 Agenda v Mapping clients needs to cloud technologies v Addressing your pain

More information

Sources: Summary Data is exploding in volume, variety and velocity timely

Sources: Summary Data is exploding in volume, variety and velocity timely 1 Sources: The Guardian, May 2010 IDC Digital Universe, 2010 IBM Institute for Business Value, 2009 IBM CIO Study 2010 TDWI: Next Generation Data Warehouse Platforms Q4 2009 Summary Data is exploding

More information

Integrating Apache Spark with an Enterprise Data Warehouse

Integrating Apache Spark with an Enterprise Data Warehouse Integrating Apache Spark with an Enterprise Warehouse Dr. Michael Wurst, IBM Corporation Architect Spark/R/Python base Integration, In-base Analytics Dr. Toni Bollinger, IBM Corporation Senior Software

More information

Enabling High performance Big Data platform with RDMA

Enabling High performance Big Data platform with RDMA Enabling High performance Big Data platform with RDMA Tong Liu HPC Advisory Council Oct 7 th, 2014 Shortcomings of Hadoop Administration tooling Performance Reliability SQL support Backup and recovery

More information

Five Best Practices for Maximizing Big Data ROI

Five Best Practices for Maximizing Big Data ROI E-PAPER FEBRUARY 2014 Five Best Practices for Maximizing Big Data ROI Lessons from early adopters show how IT can deliver better business results at less cost. TW_1401138 Organizations of all kinds have

More information

Big Data Management in the Clouds and HPC Systems

Big Data Management in the Clouds and HPC Systems Big Data Management in the Clouds and HPC Systems Hemera Final Evaluation Paris 17 th December 2014 Shadi Ibrahim Shadi.ibrahim@inria.fr Era of Big Data! Source: CNRS Magazine 2013 2 Era of Big Data! Source:

More information

NEXT GENERATION EMC: LEAD YOUR STORAGE TRANSFORMATION. Copyright 2013 EMC Corporation. All rights reserved.

NEXT GENERATION EMC: LEAD YOUR STORAGE TRANSFORMATION. Copyright 2013 EMC Corporation. All rights reserved. NEXT GENERATION EMC: LEAD YOUR STORAGE TRANSFORMATION 1 The Business Drivers Increase Revenue INCREASE AGILITY Lower Operational Costs Reduce Risk 2 CLOUD TRANSFORMS IT Lower Operational Costs 3 Disruptive

More information

Scaling Objectivity Database Performance with Panasas Scale-Out NAS Storage

Scaling Objectivity Database Performance with Panasas Scale-Out NAS Storage White Paper Scaling Objectivity Database Performance with Panasas Scale-Out NAS Storage A Benchmark Report August 211 Background Objectivity/DB uses a powerful distributed processing architecture to manage

More information

IBM Netezza High Capacity Appliance

IBM Netezza High Capacity Appliance IBM Netezza High Capacity Appliance Petascale Data Archival, Analysis and Disaster Recovery Solutions IBM Netezza High Capacity Appliance Highlights: Allows querying and analysis of deep archival data

More information

Powerful Duo: MapR Big Data Analytics with Cisco ACI Network Switches

Powerful Duo: MapR Big Data Analytics with Cisco ACI Network Switches Powerful Duo: MapR Big Data Analytics with Cisco ACI Network Switches Introduction For companies that want to quickly gain insights into or opportunities from big data - the dramatic volume growth in corporate

More information

PSAM, NEC PCIe SSD Appliance for Microsoft SQL Server (Reference Architecture) July 2014 NEC Corporation

PSAM, NEC PCIe SSD Appliance for Microsoft SQL Server (Reference Architecture) July 2014 NEC Corporation PSAM, NEC PCIe SSD Appliance for Microsoft SQL Server (Reference Architecture) July 2014 NEC Corporation 1. Overview of NEC PCIe SSD Appliance for Microsoft SQL Server Page 2 NEC Corporation 2014 Background

More information

Optimizing Storage for Better TCO in Oracle Environments. Part 1: Management INFOSTOR. Executive Brief

Optimizing Storage for Better TCO in Oracle Environments. Part 1: Management INFOSTOR. Executive Brief Optimizing Storage for Better TCO in Oracle Environments INFOSTOR Executive Brief a QuinStreet Excutive Brief. 2012 To the casual observer, and even to business decision makers who don t work in information

More information

Lecture 31: Introduction to Parallel I/O. William Gropp

Lecture 31: Introduction to Parallel I/O. William Gropp Lecture 31: Introduction to Parallel I/O William Gropp www.cs.illinois.edu/~wgropp I/O and File Systems Most applications need persistent storage Typical to store persistent data in files, accessed through

More information

In-Memory Computing Principles and Technology Overview

In-Memory Computing Principles and Technology Overview In-Memory Computing Principles and Technology Overview Agenda > Overview / Why In Memory" > Use Cases" > Concepts & Approaches" > In-Memory Computing Platform 6.1 In-Memory Data Grid In-Memory HPC In-Memory

More information

Enabling the Use of Data

Enabling the Use of Data Enabling the Use of Data Michael Kagan, CTO June 1, 2015 - Technion Computer Engineering Conference Safe Harbor Statement These slides and the accompanying oral presentation contain forward-looking statements

More information

ECMWF HPC Workshop: Accelerating Data Management

ECMWF HPC Workshop: Accelerating Data Management October 2012 ECMWF HPC Workshop: Accelerating Data Management Massively-Scalable Platforms and Solutions Engineered for the Big Data and Cloud Era Glenn Wright Systems Architect, DDN Data-Driven Paradigm

More information

Using Big Data for Smarter Decision Making. Colin White, BI Research July 2011 Sponsored by IBM

Using Big Data for Smarter Decision Making. Colin White, BI Research July 2011 Sponsored by IBM Using Big Data for Smarter Decision Making Colin White, BI Research July 2011 Sponsored by IBM USING BIG DATA FOR SMARTER DECISION MAKING To increase competitiveness, 83% of CIOs have visionary plans that

More information

A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM

A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM Sneha D.Borkar 1, Prof.Chaitali S.Surtakar 2 Student of B.E., Information Technology, J.D.I.E.T, sborkar95@gmail.com Assistant Professor, Information

More information

WATSON. Michael Dundek Industry Architect. Best Student Recognition Event July 6-8, 2011 EMEA IBM Innovation Center La Gaude, France

WATSON. Michael Dundek Industry Architect. Best Student Recognition Event July 6-8, 2011 EMEA IBM Innovation Center La Gaude, France WATSON Michael Dundek Industry Architect Best Student Recognition Event July 6-8, 2011 EMEA IBM Innovation Center La Gaude, France Want to Play Chess or Just Chat? Chess A finite, mathematically well-defined

More information

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or

More information

Big Data: Are You Ready? Kevin Lancaster

Big Data: Are You Ready? Kevin Lancaster Big Data: Are You Ready? Kevin Lancaster Director, Engineered Systems Oracle Europe, Middle East & Africa 1 A Data Explosion... Traditional Data Sources Billing engines Custom developed New, Non-Traditional

More information

Oracle Database - Engineered for Innovation. Sedat Zencirci Teknoloji Satış Danışmanlığı Direktörü Türkiye ve Orta Asya

Oracle Database - Engineered for Innovation. Sedat Zencirci Teknoloji Satış Danışmanlığı Direktörü Türkiye ve Orta Asya Oracle Database - Engineered for Innovation Sedat Zencirci Teknoloji Satış Danışmanlığı Direktörü Türkiye ve Orta Asya Oracle Database 11g Release 2 Shipping since September 2009 11.2.0.3 Patch Set now

More information

In-memory computing with SAP HANA

In-memory computing with SAP HANA In-memory computing with SAP HANA June 2015 Amit Satoor, SAP @asatoor 2015 SAP SE or an SAP affiliate company. All rights reserved. 1 Hyperconnectivity across people, business, and devices give rise to

More information

IBM System x GPFS Storage Server

IBM System x GPFS Storage Server IBM System x GPFS Storage Server Schöne Aussicht en für HPC Speicher ZKI-Arbeitskreis Paderborn, 15.03.2013 Karsten Kutzer Client Technical Architect Technical Computing IBM Systems & Technology Group

More information

High Performance Computing OpenStack Options. September 22, 2015

High Performance Computing OpenStack Options. September 22, 2015 High Performance Computing OpenStack PRESENTATION TITLE GOES HERE Options September 22, 2015 Today s Presenters Glyn Bowden, SNIA Cloud Storage Initiative Board HP Helion Professional Services Alex McDonald,

More information

Watson. An analytical computing system that specializes in natural human language and provides specific answers to complex questions at rapid speeds

Watson. An analytical computing system that specializes in natural human language and provides specific answers to complex questions at rapid speeds Watson An analytical computing system that specializes in natural human language and provides specific answers to complex questions at rapid speeds I.B.M. OHJ-2556 Artificial Intelligence Guest lecturing

More information

Unlocking Big Data: The Power of Cognitive Computing. James Kobielus, IBM

Unlocking Big Data: The Power of Cognitive Computing. James Kobielus, IBM Unlocking Big Data: The Power of Cognitive Computing James Kobielus, IBM James Kobielus IBM's big data evangelist IBM senior program director for product marketing in big data analytics Editor-in-chief

More information

LS-DYNA Performance Benchmark and Profiling on Windows. July 2009

LS-DYNA Performance Benchmark and Profiling on Windows. July 2009 LS-DYNA Performance Benchmark and Profiling on Windows July 2009 Note The following research was performed under the HPC Advisory Council activities AMD, Dell, Mellanox HPC Advisory Council Cluster Center

More information

David Rioja Redondo Telecommunication Engineer Englobe Technologies and Systems

David Rioja Redondo Telecommunication Engineer Englobe Technologies and Systems David Rioja Redondo Telecommunication Engineer Englobe Technologies and Systems About me David Rioja Redondo Telecommunication Engineer - Universidad de Alcalá >2 years building and managing clusters UPM

More information

Inge Os Sales Consulting Manager Oracle Norway

Inge Os Sales Consulting Manager Oracle Norway Inge Os Sales Consulting Manager Oracle Norway Agenda Oracle Fusion Middelware Oracle Database 11GR2 Oracle Database Machine Oracle & Sun Agenda Oracle Fusion Middelware Oracle Database 11GR2 Oracle Database

More information

Cost-Effective Business Intelligence with Red Hat and Open Source

Cost-Effective Business Intelligence with Red Hat and Open Source Cost-Effective Business Intelligence with Red Hat and Open Source Sherman Wood Director, Business Intelligence, Jaspersoft September 3, 2009 1 Agenda Introductions Quick survey What is BI?: reporting,

More information

IBM Cognos 10: Enhancing query processing performance for IBM Netezza appliances

IBM Cognos 10: Enhancing query processing performance for IBM Netezza appliances IBM Software Business Analytics Cognos Business Intelligence IBM Cognos 10: Enhancing query processing performance for IBM Netezza appliances 2 IBM Cognos 10: Enhancing query processing performance for

More information

The 4 Pillars of Technosoft s Big Data Practice

The 4 Pillars of Technosoft s Big Data Practice beyond possible Big Use End-user applications Big Analytics Visualisation tools Big Analytical tools Big management systems The 4 Pillars of Technosoft s Big Practice Overview Businesses have long managed

More information

Introducing the Singlechip Cloud Computer

Introducing the Singlechip Cloud Computer Introducing the Singlechip Cloud Computer Exploring the Future of Many-core Processors White Paper Intel Labs Jim Held Intel Fellow, Intel Labs Director, Tera-scale Computing Research Sean Koehl Technology

More information

How In-Memory Data Grids Can Analyze Fast-Changing Data in Real Time

How In-Memory Data Grids Can Analyze Fast-Changing Data in Real Time SCALEOUT SOFTWARE How In-Memory Data Grids Can Analyze Fast-Changing Data in Real Time by Dr. William Bain and Dr. Mikhail Sobolev, ScaleOut Software, Inc. 2012 ScaleOut Software, Inc. 12/27/2012 T wenty-first

More information

FLASH ARRAY MARKET TRENDS

FLASH ARRAY MARKET TRENDS 1 FLASH ARRAY MARKET TRENDS EHUD ROKACH, CO-FOUNDER, XTREMIO DAVID FLOYER, CTO & CO-FOUNDER, WIKIBON 2 >$1B ANNUALIZED Q4 RUN RATE Achieved in One Year Copyright 2015 2014 EMC Corporation. All rights reserved.

More information

LS DYNA Performance Benchmarks and Profiling. January 2009

LS DYNA Performance Benchmarks and Profiling. January 2009 LS DYNA Performance Benchmarks and Profiling January 2009 Note The following research was performed under the HPC Advisory Council activities AMD, Dell, Mellanox HPC Advisory Council Cluster Center The

More information

Unleashing the Performance Potential of GPUs for Atmospheric Dynamic Solvers

Unleashing the Performance Potential of GPUs for Atmospheric Dynamic Solvers Unleashing the Performance Potential of GPUs for Atmospheric Dynamic Solvers Haohuan Fu haohuan@tsinghua.edu.cn High Performance Geo-Computing (HPGC) Group Center for Earth System Science Tsinghua University

More information

HPC and Big Data. EPCC The University of Edinburgh. Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk

HPC and Big Data. EPCC The University of Edinburgh. Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk HPC and Big Data EPCC The University of Edinburgh Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk EPCC Facilities Technology Transfer European Projects HPC Research Visitor Programmes Training

More information

Dell* In-Memory Appliance for Cloudera* Enterprise

Dell* In-Memory Appliance for Cloudera* Enterprise Built with Intel Dell* In-Memory Appliance for Cloudera* Enterprise Find out what faster big data analytics can do for your business The need for speed in all things related to big data is an enormous

More information

White Paper. How Streaming Data Analytics Enables Real-Time Decisions

White Paper. How Streaming Data Analytics Enables Real-Time Decisions White Paper How Streaming Data Analytics Enables Real-Time Decisions Contents Introduction... 1 What Is Streaming Analytics?... 1 How Does SAS Event Stream Processing Work?... 2 Overview...2 Event Stream

More information

COSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters

COSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters COSC 6374 Parallel I/O (I) I/O basics Fall 2012 Concept of a clusters Processor 1 local disks Compute node message passing network administrative network Memory Processor 2 Network card 1 Network card

More information

How to Rev Up Your Data Analytics Programs

How to Rev Up Your Data Analytics Programs How to Rev Up Your Data Analytics Programs Three decades on since the advent of the x86 architecture, the computing world has steadily marched towards standard hardware regardless of the computing workload.

More information

Jean-Pierre Panziera Teratec 2011

Jean-Pierre Panziera Teratec 2011 Technologies for the future HPC systems Jean-Pierre Panziera Teratec 2011 3 petaflop systems : TERA 100, CURIE & IFERC Tera100 Curie IFERC 1.25 PetaFlops 256 TB ory 30 PB disk storage 140 000+ Xeon cores

More information

Hybrid Software Architectures for Big Data. Laurence.Hubert@hurence.com @hurence http://www.hurence.com

Hybrid Software Architectures for Big Data. Laurence.Hubert@hurence.com @hurence http://www.hurence.com Hybrid Software Architectures for Big Data Laurence.Hubert@hurence.com @hurence http://www.hurence.com Headquarters : Grenoble Pure player Expert level consulting Training R&D Big Data X-data hot-line

More information

EMC/Greenplum Driving the Future of Data Warehousing and Analytics

EMC/Greenplum Driving the Future of Data Warehousing and Analytics EMC/Greenplum Driving the Future of Data Warehousing and Analytics EMC 2010 Forum Series 1 Greenplum Becomes the Foundation of EMC s Data Computing Division E M C A CQ U I R E S G R E E N P L U M Greenplum,

More information

Proact whitepaper on Big Data

Proact whitepaper on Big Data Proact whitepaper on Big Data Summary Big Data is not a definite term. Even if it sounds like just another buzz word, it manifests some interesting opportunities for organisations with the skill, resources

More information

Klarna Tech Talk: Mind the Data! Jeff Pollock InfoSphere Information Integration & Governance

Klarna Tech Talk: Mind the Data! Jeff Pollock InfoSphere Information Integration & Governance Klarna Tech Talk: Mind the Data! Jeff Pollock InfoSphere Information Integration & Governance IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice

More information

Crossing the Performance Chasm with OpenPOWER

Crossing the Performance Chasm with OpenPOWER Crossing the Performance Chasm with OpenPOWER Dr. Srini Chari Cabot Partners/IBM chari@cabotpartners.com #OpenPOWERSummit Join the conversation at #OpenPOWERSummit 1 Disclosure Copyright 215. Cabot Partners

More information

Data-intensive HPC: opportunities and challenges. Patrick Valduriez

Data-intensive HPC: opportunities and challenges. Patrick Valduriez Data-intensive HPC: opportunities and challenges Patrick Valduriez Big Data Landscape Multi-$billion market! Big data = Hadoop = MapReduce? No one-size-fits-all solution: SQL, NoSQL, MapReduce, No standard,

More information

Oracle - Engineered for Innovation. Thomas Kyte http://asktom.oracle.com

Oracle - Engineered for Innovation. Thomas Kyte http://asktom.oracle.com Oracle - Engineered for Innovation Thomas Kyte http://asktom.oracle.com The Beginning... Data Model with Structure Data Independent of Code Set-oriented 1977 the work begins GPS 1978 First RDBMS: Version

More information

Using an In-Memory Data Grid for Near Real-Time Data Analysis

Using an In-Memory Data Grid for Near Real-Time Data Analysis SCALEOUT SOFTWARE Using an In-Memory Data Grid for Near Real-Time Data Analysis by Dr. William Bain, ScaleOut Software, Inc. 2012 ScaleOut Software, Inc. 12/27/2012 IN today s competitive world, businesses

More information

Big Systems, Big Data

Big Systems, Big Data Big Systems, Big Data When considering Big Distributed Systems, it can be noted that a major concern is dealing with data, and in particular, Big Data Have general data issues (such as latency, availability,

More information

IBM Announces Eight Universities Contributing to the Watson Computing System's Development

IBM Announces Eight Universities Contributing to the Watson Computing System's Development IBM Announces Eight Universities Contributing to the Watson Computing System's Development Press release Related XML feeds Contact(s) information Related resources ARMONK, N.Y. - 11 Feb 2011: IBM (NYSE:

More information

The Fusion of Supercomputing and Big Data. Peter Ungaro President & CEO

The Fusion of Supercomputing and Big Data. Peter Ungaro President & CEO The Fusion of Supercomputing and Big Data Peter Ungaro President & CEO The Supercomputing Company Supercomputing Big Data Because some great things never change One other thing that hasn t changed. Cray

More information