How E/EF-Series can help solve HPC Challenges MEW25 3 rd December 2014

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "How E/EF-Series can help solve HPC Challenges MEW25 3 rd December 2014"

Transcription

1 How E/EF-Series can help solve HPC Challenges MEW25 3 rd December 2014 Mohinder Toor Business Development Executive NEMEA E-Series Sales and Product Specialist Emerging Products Group NetApp, Inc. All rights reserved. NetApp Proprietary Limited Use Only

2 NetApp Storage Solutions Flash Arrays for ultra-high performance Clustered Data ONTAP for Shared Infrastructure E-Series for price and performance at scale StorageGRID for web scale object storage NetApp Proprietary - Limited Use Only

3 What is E-Series and EF-Series? Fast, performance efficient, reliable, block storage E2700 E5500 EF550 Superior price/performance for SAN, with application-driven data management Accelerate business operations; Improve customer experience NetApp Confidential - Limited Use Only

4 HPC File Systems (to name but a few ) CEPH BeeGFS Lustre GPFS PNFS CXFS PVFS OrangeFS A free software storage platform designed to provide a unified, distributed storage system, designed for excellent performance, reliability and scalability. Uses the CRUSH algorithm for data placement. Formerly known as FhgFS, a high performance parallel File System. Its distributed metadata architecture has been designed to provide the scalability and flexibility. Open Source, high performance, highly scalable file system, generally used for large scale cluster computing. A high-performance clustered file system, developed by IBM, used by many of the world s largest commercial companies, as well as some of the supercomputers on the TOP500 list. Parallel NFS is used in clustered server deployments, providing scalable parallel access to files distributed among multiple servers. Clustered XFS is a proprietary shared disk file systems designed by SGI, specifically used in Storage Area Network (SAN) environments. A parallel virtual filesystem that rearranges random, concurrent writes into sequential writes to unique files (i.e. N-1 into N-N). Designed to provide very high performance access to disk storage for parallel applications. A branch of Parallel Virtual Filesystem (PVFS), with roots all the way back to VAX Clusters. NetApp Proprietary Limited Use Only

5 Infrastructure Hot Spots System Management Storage Network Fabric High Availability & Reliability Performance Density Lustre Expertise Performance Analytics LSI Conf ident ial

6 6

7 HPSS for Lustre Easy Capacity and Performance Scaling Management 1G Switch Dell R320 Management Server MDS0 MDS1 E2724 MDT OSS0 OSS1 E5560 (OSTs) Management 1G Switch Dell R320 Management Server MDS0 MDS1 E2724 MDT OSS0 E5560 (OSTs) Bandwidth Growth DE6600 (OSTs) OSS1 E5560 (OSTs) DE6600 (OSTs) OSS2 E5560 (OSTs) Capacity Growth DE6600 (OSTs) OSS3 E5560 (OSTs) DE6600 (OSTs) ~ 5GB/Sec 360 TB (Raw) ~ 22GB/Sec 2.88 PB (Raw) ~ 40GB/Sec NetApp Proprietary Limited Use Only

8 8

9 Most Cost Efficient Way to High IOPs Compare the cost of SSD Performance to hard drive Performance SSD HDD Price $4,200* $750* IOPs Rating IOPs 200 IOPs GB 400 GB 600 GB $ per IOPs 25 cents per IOPs $ 3.75 per IOPs $ per GB $ 10.5 $ 1.25 Number to reach 200,000 IOPs $ to reach 200,000 IOPs 12 drives 1000 drives $50,000 (1 system) $750,000 (6 systems) Lowest $/IOPs on a reliable, proven storage platform *Example of a typical SAS drive price subject to change

10 Which protection scheme to pick Look at your requirements Think of the different protection schemes and performance metrics as zones. Customer Requirement - 8K 75% Reads - Latency <1.5ms - IOPS ~100,000 DDP will meet the requirement and provide best protection and ease of use 10

11 Latency EF550 Performance: 4K Random 100% Read / 0% Write 100% 4K Random Read DDP ART R5 4K ART R1 ART IOPS 11

12 Traditional HPC IO Each row of compute Nodes Communicates with the Scratch storage pool through an assigned IO Node. In the example provided here, there are 26 Compute Nodes assigned to one IO Node on each row. 12 Each IO Node contains one (or more) PCI-e slots, where Infiniband (HCAs) are installed. The HCAs allow the IO Nodes to communicate, through the Infiniband Fabric, with the OSS Nodes. OSS Nodes are normally located in a separate cluster, generally referred to as the filesystem cluster.

13 Checkpoint/Restart The primary function of the scratch filesystem (in most HPC environments) is to provide temporary storage space for periodic dumps of the Compute Nodes memory images for Checkpoint- Restart purposes. 13 The size and complexity of the Infiniband Fabric, as well as the size of the scratch filesystem, in terms of the number of paths, number of disks, disk capacity, number of controllers, etc., is a function of providing a target environment that is capable of quickly absorbing the Checkpoint images. The Compute Nodes remain idle during the Checkpoint process.

14 Burst Buffering The U.S. DoE has tested a plan that inserts one or more PCI-e-based SSD if each Compute Node is configured with 100GB of Main Memory, then the total memory footprint required for each Checkpoint would be 2600GB A three-times requirement would only require 7.8TB, and the five- times requirement would need 13TB of space to hold all of the Checkpoint files. Both are easily achievable given today s technology. 14

15 A Sharable, Scalable Burst Buffer By replacing the PCI-e-based SSDs with SAS HBAs, the IO Nodes could share a scalable Burst Buffer environment that is also able to be repaired without disruption. The PCI-e-based SSD solution would required the IO Nodes to be removed from service and opened to repair/replace a defective SSD. One or more EF-Series could be used to provide a highly available, sharable, external SSD configuration, providing a solution with NO single points of failure. 15

16 Lustre HSM Drains the Burst Buffer Compute Nodes back to work much quicker Checkpoint images moved to the spinning disks. The HSM is then used to move the memory images to Scratch before next Checkpoint. 16

17 17

18 Use Case: Large Scale Deployment, LLNL NetApp Proprietary Limited Use Only

19 Supported Lustre Based Appliance Each block is a pre-installed appliance Simple provisioning Entire storage cluster configuration is in one place: LxDaemon Setup, Hierarchy All configurable parameters Single data source. Deployment, Management, Monitoring Deploying the storage cluster is Simple: Auto-generate configuration of components from data in LxDaemon Storage devices setup, configuration Servers setup, configuration HA setup, services, dependencies, STONITH Formatting, Lustre mounting Managing & Monitoring Set of simple commands, hiding complexity of underlying setup Health monitoring: pro-active, sends messages to admins, reacts to issues Performance monitoring: time-history, GUI, discover issues and bottlenecks

20 Summary and Take Away Overall solid state drive (SSD) storage revenue is expected to grow to $5.6 billion by The market is changing from niche to mainstream. Pricing parity between SSDs and hard disk drives (HDDs) is no longer an imperative to drive adoption due to increasing use cases, which are turning end users' wants into needs. Burst buffering technology is coming to help reducing dump time

21 Thank You 21

DMF & Tiering Update. Kirill Malkin Director of Storage Engineering. September 2015

DMF & Tiering Update. Kirill Malkin Director of Storage Engineering. September 2015 DMF & Tiering Update Kirill Malkin Director of Storage Engineering September 2015 1 What s New in DMF? Data Migration Facility (DMF) Data management solution for HPC/HPDA Over 20 years of data management

More information

Lustre & Cluster. - monitoring the whole thing Erich Focht

Lustre & Cluster. - monitoring the whole thing Erich Focht Lustre & Cluster - monitoring the whole thing Erich Focht NEC HPC Europe LAD 2014, Reims, September 22-23, 2014 1 Overview Introduction LXFS Lustre in a Data Center IBviz: Infiniband Fabric visualization

More information

Netapp HPC Solution for Lustre. Rich Fenton (fenton@netapp.com) UK Solutions Architect

Netapp HPC Solution for Lustre. Rich Fenton (fenton@netapp.com) UK Solutions Architect Netapp HPC Solution for Lustre Rich Fenton (fenton@netapp.com) UK Solutions Architect Agenda NetApp Introduction Introducing the E-Series Platform Why E-Series for Lustre? Modular Scale-out Capacity Density

More information

An Alternative Storage Solution for MapReduce. Eric Lomascolo Director, Solutions Marketing

An Alternative Storage Solution for MapReduce. Eric Lomascolo Director, Solutions Marketing An Alternative Storage Solution for MapReduce Eric Lomascolo Director, Solutions Marketing MapReduce Breaks the Problem Down Data Analysis Distributes processing work (Map) across compute nodes and accumulates

More information

Worry-free Storage. E-Series Simple SAN Storage

Worry-free Storage. E-Series Simple SAN Storage Worry-free Storage E-Series Simple SAN Storage 1 What are Midsize Businesses Struggling with Today? #1 pain point: managing data growth (62%)* #2 pain point: managing costs (33%)* 52% view poor storage

More information

Lab Evaluation of NetApp Hybrid Array with Flash Pool Technology

Lab Evaluation of NetApp Hybrid Array with Flash Pool Technology Lab Evaluation of NetApp Hybrid Array with Flash Pool Technology Evaluation report prepared under contract with NetApp Introduction As flash storage options proliferate and become accepted in the enterprise,

More information

Storage Designed to Support an Oracle Database. White Paper

Storage Designed to Support an Oracle Database. White Paper Storage Designed to Support an Oracle Database White Paper Abstract Databases represent the backbone of most organizations. And Oracle databases in particular have become the mainstream data repository

More information

The Use of Flash in Large-Scale Storage Systems. Nathan.Rutman@Seagate.com

The Use of Flash in Large-Scale Storage Systems. Nathan.Rutman@Seagate.com The Use of Flash in Large-Scale Storage Systems Nathan.Rutman@Seagate.com 1 Seagate s Flash! Seagate acquired LSI s Flash Components division May 2014 Selling multiple formats / capacities today Nytro

More information

200 flash-related. petabytes of. flash shipped. patents. NetApp flash leadership. Metrics through October, 2015

200 flash-related. petabytes of. flash shipped. patents. NetApp flash leadership. Metrics through October, 2015 NetApp flash leadership Metrics through October, 2015 ~ 125 petabytes of 200 flash-related ~ flash shipped patents flash shipped in last 3 quarters hybrid systems shipped all-flash systems shipped 17 2014

More information

New Storage System Solutions

New Storage System Solutions New Storage System Solutions Craig Prescott Research Computing May 2, 2013 Outline } Existing storage systems } Requirements and Solutions } Lustre } /scratch/lfs } Questions? Existing Storage Systems

More information

Performance, Reliability, and Operational Issues for High Performance NAS Storage on Cray Platforms. Cray User Group Meeting June 2007

Performance, Reliability, and Operational Issues for High Performance NAS Storage on Cray Platforms. Cray User Group Meeting June 2007 Performance, Reliability, and Operational Issues for High Performance NAS Storage on Cray Platforms Cray User Group Meeting June 2007 Cray s Storage Strategy Background Broad range of HPC requirements

More information

Lustre* is designed to achieve the maximum performance and scalability for POSIX applications that need outstanding streamed I/O.

Lustre* is designed to achieve the maximum performance and scalability for POSIX applications that need outstanding streamed I/O. Reference Architecture Designing High-Performance Storage Tiers Designing High-Performance Storage Tiers Intel Enterprise Edition for Lustre* software and Intel Non-Volatile Memory Express (NVMe) Storage

More information

Hadoop on the Gordon Data Intensive Cluster

Hadoop on the Gordon Data Intensive Cluster Hadoop on the Gordon Data Intensive Cluster Amit Majumdar, Scientific Computing Applications Mahidhar Tatineni, HPC User Services San Diego Supercomputer Center University of California San Diego Dec 18,

More information

Solid State Storage in the Evolution of the Data Center

Solid State Storage in the Evolution of the Data Center Solid State Storage in the Evolution of the Data Center Trends and Opportunities Bruce Moxon CTO, Systems and Solutions stec Presented at the Lazard Capital Markets Solid State Storage Day New York, June

More information

Flash Memory Arrays Enabling the Virtualized Data Center. July 2010

Flash Memory Arrays Enabling the Virtualized Data Center. July 2010 Flash Memory Arrays Enabling the Virtualized Data Center July 2010 2 Flash Memory Arrays Enabling the Virtualized Data Center This White Paper describes a new product category, the flash Memory Array,

More information

Testing Performed at: Clemson Center of Excellence in Next Generation Computing Evaluation & Usability Labs October 2013

Testing Performed at: Clemson Center of Excellence in Next Generation Computing Evaluation & Usability Labs October 2013 OrangeFS DDN SFA12K Architecture Testing Performed at: Clemson Center of Excellence in Next Generation Computing Evaluation & Usability Labs October 2013 THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES

More information

Oracle Database-Optimized Flash

Oracle Database-Optimized Flash Oracle Database-Optimized Flash Setting New Standards for Database Performance Kothanda Umamageswaran Vice President, Exadata Development Gurmeet Goindi Technical Product Strategist Flash Has Transformed

More information

Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre

Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre University of Cambridge, UIS, HPC Service Authors: Wojciech Turek, Paul Calleja, John Taylor

More information

Intel Solid- State Drive Data Center P3700 Series NVMe Hybrid Storage Performance

Intel Solid- State Drive Data Center P3700 Series NVMe Hybrid Storage Performance Intel Solid- State Drive Data Center P3700 Series NVMe Hybrid Storage Performance Hybrid Storage Performance Gains for IOPS and Bandwidth Utilizing Colfax Servers and Enmotus FuzeDrive Software NVMe Hybrid

More information

Essentials Guide CONSIDERATIONS FOR SELECTING ALL-FLASH STORAGE ARRAYS

Essentials Guide CONSIDERATIONS FOR SELECTING ALL-FLASH STORAGE ARRAYS Essentials Guide CONSIDERATIONS FOR SELECTING ALL-FLASH STORAGE ARRAYS M ost storage vendors now offer all-flash storage arrays, and many modern organizations recognize the need for these highperformance

More information

Appro Supercomputer Solutions Best Practices Appro 2012 Deployment Successes. Anthony Kenisky, VP of North America Sales

Appro Supercomputer Solutions Best Practices Appro 2012 Deployment Successes. Anthony Kenisky, VP of North America Sales Appro Supercomputer Solutions Best Practices Appro 2012 Deployment Successes Anthony Kenisky, VP of North America Sales About Appro Over 20 Years of Experience 1991 2000 OEM Server Manufacturer 2001-2007

More information

Dynamic Disk Pools Delivering Worry-Free Storage

Dynamic Disk Pools Delivering Worry-Free Storage Dynamic Disk Pools Delivering Worry-Free Storage Dr. Didier Gava EMEA HPC Storage Architect MEW Workshop 2012 Liverpool, Germany Historic View Of RAID Advancement RAID 5 1987 Single disk failure protection

More information

NetApp High-Performance Computing Solution for Lustre: Solution Guide

NetApp High-Performance Computing Solution for Lustre: Solution Guide Technical Report NetApp High-Performance Computing Solution for Lustre: Solution Guide Robert Lai, NetApp August 2012 TR-3997 TABLE OF CONTENTS 1 Introduction... 5 1.1 NetApp HPC Solution for Lustre Introduction...5

More information

High Performance Computing OpenStack Options. September 22, 2015

High Performance Computing OpenStack Options. September 22, 2015 High Performance Computing OpenStack PRESENTATION TITLE GOES HERE Options September 22, 2015 Today s Presenters Glyn Bowden, SNIA Cloud Storage Initiative Board HP Helion Professional Services Alex McDonald,

More information

Flash Memory Technology in Enterprise Storage

Flash Memory Technology in Enterprise Storage NETAPP WHITE PAPER Flash Memory Technology in Enterprise Storage Flexible Choices to Optimize Performance Mark Woods and Amit Shah, NetApp November 2008 WP-7061-1008 EXECUTIVE SUMMARY Solid state drives

More information

Time Matters. K.D. Polzin FlashSystems Sales Specialist - South Central

Time Matters. K.D. Polzin FlashSystems Sales Specialist - South Central K.D. Polzin FlashSystems Sales Specialist - South Central 281-520-1307 kdpolzin@us.ibm.com Simon Cambridge FlashSystems Technical Specialist South Central Time Matters Think Microseconds, not Milliseconds

More information

MaxDeploy Ready. Hyper- Converged Virtualization Solution. With SanDisk Fusion iomemory products

MaxDeploy Ready. Hyper- Converged Virtualization Solution. With SanDisk Fusion iomemory products MaxDeploy Ready Hyper- Converged Virtualization Solution With SanDisk Fusion iomemory products MaxDeploy Ready products are configured and tested for support with Maxta software- defined storage and with

More information

Sun Storage Perspective & Lustre Architecture. Dr. Peter Braam VP Sun Microsystems

Sun Storage Perspective & Lustre Architecture. Dr. Peter Braam VP Sun Microsystems Sun Storage Perspective & Lustre Architecture Dr. Peter Braam VP Sun Microsystems Agenda Future of Storage Sun s vision Lustre - vendor neutral architecture roadmap Sun s view on storage introduction The

More information

HyperQ Hybrid Flash Storage Made Easy White Paper

HyperQ Hybrid Flash Storage Made Easy White Paper HyperQ Hybrid Flash Storage Made Easy White Paper Parsec Labs, LLC. 7101 Northland Circle North, Suite 105 Brooklyn Park, MN 55428 USA 1-763-219-8811 www.parseclabs.com info@parseclabs.com sales@parseclabs.com

More information

Accelerating Real Time Big Data Applications. PRESENTATION TITLE GOES HERE Bob Hansen

Accelerating Real Time Big Data Applications. PRESENTATION TITLE GOES HERE Bob Hansen Accelerating Real Time Big Data Applications PRESENTATION TITLE GOES HERE Bob Hansen Apeiron Data Systems Apeiron is developing a VERY high performance Flash storage system that alters the economics of

More information

Flash at the price of disk Redefining the Economics of Storage. Kris Van Haverbeke Enterprise Marketing Manager Dell Belux

Flash at the price of disk Redefining the Economics of Storage. Kris Van Haverbeke Enterprise Marketing Manager Dell Belux Flash at the price of disk Redefining the Economics of Storage Kris Van Haverbeke Enterprise Marketing Manager Dell Belux We believe that Redefining the Economics of Storage helps businesses achieve their

More information

A Case Study: Performance Analysis and Optimization of SAS Grid Computing Scaling on a Shared Storage

A Case Study: Performance Analysis and Optimization of SAS Grid Computing Scaling on a Shared Storage Paper 1815-2014 A Case Study: Performance Analysis and Optimization of SAS Grid Computing Scaling on a Shared Storage Suleyman Sair, Brett Lee, Ying M. Zhang, Intel Corporation ABSTRACT SAS Grid Computing

More information

The Data Placement Challenge

The Data Placement Challenge The Data Placement Challenge Entire Dataset Applications Active Data Lowest $/IOP Highest throughput Lowest latency 10-20% Right Place Right Cost Right Time 100% 2 2 What s Driving the AST Discussion?

More information

SOLID STATE STORAGE FOR DATA WAREHOUSING

SOLID STATE STORAGE FOR DATA WAREHOUSING Summary SOLID STATE STORAGE FOR DATA WAREHOUSING By Richard Winter Solid state devices (SSD) provide online data storage for computer systems without the mechanical moving parts of hard disk drives (HDD).

More information

Flash Storage Optimizing Virtual Desktop Deployments

Flash Storage Optimizing Virtual Desktop Deployments Flash Storage Optimizing Virtual Desktop Deployments Ashok Rajagopalan UCS Product Management May 2014 In Collaboration with Intel Old Fashioned VDI (circa 2012) was Financially Unattractive to Most Average

More information

Hitachi Unified Storage and Hitachi NAS Platform Performance Optimization with Flash Acceleration

Hitachi Unified Storage and Hitachi NAS Platform Performance Optimization with Flash Acceleration Hitachi Unified Storage and Hitachi NAS Platform Performance Optimization with Flash Acceleration Nick Jarvis Director, File, Content and Cloud Solutions August 14, 2013 1 WEBTECH EDUCATIONAL SERIES HITACHI

More information

Installation Requirements and Reference Guide

Installation Requirements and Reference Guide V-Series Systems Installation Requirements and Reference Guide NetApp, Inc. 495 East Java Drive Sunnyvale, CA 94089 U.S. Telephone: +1(408) 822-6000 Fax: +1(408) 822-4501 Support telephone: +1(888) 4-NETAPP

More information

Diablo and VMware TM powering SQL Server TM in Virtual SAN TM. A Diablo Technologies Whitepaper. May 2015

Diablo and VMware TM powering SQL Server TM in Virtual SAN TM. A Diablo Technologies Whitepaper. May 2015 A Diablo Technologies Whitepaper Diablo and VMware TM powering SQL Server TM in Virtual SAN TM May 2015 Ricky Trigalo, Director for Virtualization Solutions Architecture, Diablo Technologies Daniel Beveridge,

More information

Violin Memory Arrays With IBM System Storage SAN Volume Control

Violin Memory Arrays With IBM System Storage SAN Volume Control Technical White Paper Report Best Practices Guide: Violin Memory Arrays With IBM System Storage SAN Volume Control Implementation Best Practices and Performance Considerations Version 1.0 Abstract This

More information

Parallel IO. Single namespace. Performance. Disk locality awareness? Data integrity. Fault tolerance. Standard interface. Network of disks?

Parallel IO. Single namespace. Performance. Disk locality awareness? Data integrity. Fault tolerance. Standard interface. Network of disks? PARALLEL IO Parallel IO Single namespace Network of disks? Performance Data replication Multiple I/O paths Disk locality awareness? Data integrity Multiple writers Locking? Fault tolerance Hardware failure

More information

PowerVault MD3 SSD Cache Overview. White Paper

PowerVault MD3 SSD Cache Overview. White Paper PowerVault MD3 SSD Cache Overview White Paper 2012 Dell Inc. All Rights Reserved. PowerVault is a trademark of Dell Inc. 2 Dell PowerVault MD3 SSD Cache Overview Table of contents 1 Overview... 4 2 Architecture...

More information

Hitachi Flash Storage: Flash for Multiple Enterprise Data Center Deployment Strategies

Hitachi Flash Storage: Flash for Multiple Enterprise Data Center Deployment Strategies Technology Insight Paper Hitachi Flash Storage: Flash for Multiple Enterprise Data Center Deployment Strategies By John Webster January 2016 Enabling you to make the best technology decisions Hitachi Flash

More information

Clusters Your Way. TM SAN and #SANLess clusters for physical, virtual, and cloud environments.

Clusters Your Way. TM SAN and #SANLess clusters for physical, virtual, and cloud environments. Clusters Your Way. TM SAN and #SANLess clusters for physical, virtual, and cloud environments. High Performance and Highly Available SANless SQL clusters in Azure, AWS or On-Premise Mark L Round VCP :

More information

Exadata HW Overview. Marek Mintal

Exadata HW Overview. Marek Mintal Exadata HW Overview Marek Mintal marek.mintal@phaetech.com Oracle Day 2011 20.10.2011 Exadata Hardware Architecture Scalable Grid of industry standard servers for Compute and Storage Eliminates long-standing

More information

DELL SOLID STATE DISK (SSD) DRIVES

DELL SOLID STATE DISK (SSD) DRIVES DELL SOLID STATE DISK (SSD) DRIVES STORAGE SOLUTIONS FOR SELECT POWEREDGE SERVERS By Bryan Martin, Dell Product Marketing Manager for HDD & SSD delltechcenter.com TAB LE OF CONTENTS INTRODUCTION 3 DOWNFALLS

More information

Integrated Application and Data Protection. NEC ExpressCluster White Paper

Integrated Application and Data Protection. NEC ExpressCluster White Paper Integrated Application and Data Protection NEC ExpressCluster White Paper Introduction Critical business processes and operations depend on real-time access to IT systems that consist of applications and

More information

Shareable Storage with Switched SAS. Terry Gibbons, LSI Corporation

Shareable Storage with Switched SAS. Terry Gibbons, LSI Corporation Shareable Storage with Switched SAS Terry Gibbons, LSI Corporation SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless otherwise noted. Member companies and individual

More information

Building a cost-effective and high-performing public cloud. Sander Cruiming, founder Cloud Provider

Building a cost-effective and high-performing public cloud. Sander Cruiming, founder Cloud Provider Building a cost-effective and high-performing public cloud Sander Cruiming, founder Cloud Provider 1 Agenda Introduction of Cloud Provider Overview of our previous cloud architecture Challenges of that

More information

Correlating Multiple TB of Performance Data to User Jobs

Correlating Multiple TB of Performance Data to User Jobs Michael Kluge, ZIH Correlating Multiple TB of Performance Data to User Jobs Lustre User Group 2015, Denver, Colorado Zellescher Weg 12 Willers-Bau A 208 Tel. +49 351-463 34217 Michael Kluge (michael.kluge@tu-dresden.de)

More information

Direct NFS - Design considerations for next-gen NAS appliances optimized for database workloads Akshay Shah Gurmeet Goindi Oracle

Direct NFS - Design considerations for next-gen NAS appliances optimized for database workloads Akshay Shah Gurmeet Goindi Oracle Direct NFS - Design considerations for next-gen NAS appliances optimized for database workloads Akshay Shah Gurmeet Goindi Oracle Agenda Introduction Database Architecture Direct NFS Client NFS Server

More information

PSAM, NEC PCIe SSD Appliance for Microsoft SQL Server (Reference Architecture) July 2014 NEC Corporation

PSAM, NEC PCIe SSD Appliance for Microsoft SQL Server (Reference Architecture) July 2014 NEC Corporation PSAM, NEC PCIe SSD Appliance for Microsoft SQL Server (Reference Architecture) July 2014 NEC Corporation 1. Overview of NEC PCIe SSD Appliance for Microsoft SQL Server Page 2 NEC Corporation 2014 Background

More information

Lustre failover experience

Lustre failover experience Lustre failover experience Lustre Administrators and Developers Workshop Paris 1 September 25, 2012 TOC Who we are Our Lustre experience: the environment Deployment Benchmarks What's next 2 Who we are

More information

Accelerating Lustre! with Cray DataWarp Steve Woods, Solutions Architect

Accelerating Lustre! with Cray DataWarp Steve Woods, Solutions Architect Accelerating Lustre! with ray DataWarp Steve Woods, Solutions Architect Accelerate Your Storage! The Problem a new storage hierarchy DataWarp overview End user Perspectives Use cases Features Examples

More information

PARALLELS CLOUD STORAGE

PARALLELS CLOUD STORAGE PARALLELS CLOUD STORAGE Performance Benchmark Results 1 Table of Contents Executive Summary... Error! Bookmark not defined. Architecture Overview... 3 Key Features... 5 No Special Hardware Requirements...

More information

WITH A FUSION POWERED SQL SERVER 2014 IN-MEMORY OLTP DATABASE

WITH A FUSION POWERED SQL SERVER 2014 IN-MEMORY OLTP DATABASE WITH A FUSION POWERED SQL SERVER 2014 IN-MEMORY OLTP DATABASE 1 W W W. F U S I ON I O.COM Table of Contents Table of Contents... 2 Executive Summary... 3 Introduction: In-Memory Meets iomemory... 4 What

More information

How A V3 Appliance Employs Superior VDI Architecture to Reduce Latency and Increase Performance

How A V3 Appliance Employs Superior VDI Architecture to Reduce Latency and Increase Performance How A V3 Appliance Employs Superior VDI Architecture to Reduce Latency and Increase Performance www. ipro-com.com/i t Contents Overview...3 Introduction...3 Understanding Latency...3 Network Latency...3

More information

Mellanox Accelerated Storage Solutions

Mellanox Accelerated Storage Solutions Mellanox Accelerated Storage Solutions Moving Data Efficiently In an era of exponential data growth, storage infrastructures are being pushed to the limits of their capacity and data delivery capabilities.

More information

Data Center Performance Insurance

Data Center Performance Insurance Data Center Performance Insurance How NFS Caching Guarantees Rapid Response Times During Peak Workloads November 2010 2 Saving Millions By Making It Easier And Faster Every year slow data centers and application

More information

High Performance NAS for Hadoop

High Performance NAS for Hadoop High Performance NAS for Hadoop HPC ADVISORY COUNCIL, STANFORD FEB 8, 2013 DR. BRENT WELCH, CTO, PANASAS Panasas and Hadoop PANASAS TECHNICAL DIFFERENTIATION Scalable Performance Balanced object-storage

More information

Deploying Flash in the Enterprise Choices to Optimize Performance and Cost

Deploying Flash in the Enterprise Choices to Optimize Performance and Cost White Paper Deploying Flash in the Enterprise Choices to Optimize Performance and Cost Paul Feresten, Mohit Bhatnagar, Manish Agarwal, and Rip Wilson, NetApp April 2013 WP-7182 Executive Summary Flash

More information

Oracle on SSD for Performance FusionIO and EMC SSD performance for Oracle databases

Oracle on SSD for Performance FusionIO and EMC SSD performance for Oracle databases Oracle on SSD for Performance FusionIO and EMC SSD performance for Oracle databases Presenter: Steve Fluge Bank of America Background Steve Fluge Oracle Engineering team at Bank of America (formerly Merrill

More information

VIRTUOZZO STORAGE VS. CEPH

VIRTUOZZO STORAGE VS. CEPH VIRTUOZZO STORAGE VS. CEPH I/O Performance Comparison April 29, 2016 Executive Summary Software-defined storage (SDS) is one of the key technologies IT organizations are looking toward as they explore

More information

Hadoop: Embracing future hardware

Hadoop: Embracing future hardware Hadoop: Embracing future hardware Suresh Srinivas @suresh_m_s Page 1 About Me Architect & Founder at Hortonworks Long time Apache Hadoop committer and PMC member Designed and developed many key Hadoop

More information

Business-centric Storage FUJITSU Hyperscale Storage System ETERNUS CD10000

Business-centric Storage FUJITSU Hyperscale Storage System ETERNUS CD10000 Business-centric Storage FUJITSU Hyperscale Storage System ETERNUS CD10000 Clear the way for new business opportunities. Unlock the power of data. Overcoming storage limitations Unpredictable data growth

More information

Netapp @ 10th TF-Storage Meeting

Netapp @ 10th TF-Storage Meeting Netapp @ 10th TF-Storage Meeting Wojciech Janusz, Netapp Poland Bogusz Błaszkiewicz, Netapp Poland Ljubljana, 2012.02.20 Agenda Data Ontap Cluster-Mode pnfs E-Series NetApp Confidential - Internal Use

More information

Building All-Flash Software Defined Storages for Datacenters. Ji Hyuck Yun (dr.jhyun@sk.com) Storage Tech. Lab SK Telecom

Building All-Flash Software Defined Storages for Datacenters. Ji Hyuck Yun (dr.jhyun@sk.com) Storage Tech. Lab SK Telecom Building All-Flash Software Defined Storages for Datacenters Ji Hyuck Yun (dr.jhyun@sk.com) Storage Tech. Lab SK Telecom Introduction R&D Motivation Synergy between SK Telecom and SK Hynix Service & Solution

More information

Optimizing Dell PowerEdge Configurations for Hadoop

Optimizing Dell PowerEdge Configurations for Hadoop Optimizing Dell PowerEdge Configurations for Hadoop Understanding how to get the most out of Hadoop running on Dell hardware A Dell technical white paper July 2013 Michael Pittaro Principal Architect,

More information

Quick Reference Selling Guide for Intel Lustre Solutions Overview

Quick Reference Selling Guide for Intel Lustre Solutions Overview Overview The 30 Second Pitch Intel Solutions for Lustre* solutions Deliver sustained storage performance needed that accelerate breakthrough innovations and deliver smarter, data-driven decisions for enterprise

More information

FlexArray Virtualization

FlexArray Virtualization Updated for 8.2.1 FlexArray Virtualization Installation Requirements and Reference Guide NetApp, Inc. 495 East Java Drive Sunnyvale, CA 94089 U.S. Telephone: +1 (408) 822-6000 Fax: +1 (408) 822-4501 Support

More information

Virtualization of the MS Exchange Server Environment

Virtualization of the MS Exchange Server Environment MS Exchange Server Acceleration Maximizing Users in a Virtualized Environment with Flash-Powered Consolidation Allon Cohen, PhD OCZ Technology Group Introduction Microsoft (MS) Exchange Server is one of

More information

FLASH Implications in Enterprise Storage Designs

FLASH Implications in Enterprise Storage Designs 1 FLASH Implications in Enterprise Storage Designs Denis Vilfort Sr. Dir., Unified Storage Division 2 New Storage Reality 3000X Better Performance PCIe FLASH Capacity Cost 300X Better Performance FLASH

More information

# Fusion iocontrol n5 Hybrid Storage Architecture

# Fusion iocontrol n5 Hybrid Storage Architecture Fusion iocontrol n5 Hybrid Storage Architecture Performance Comparison versus Conventional NVRAM+SSD+Disk Architecture #213154 October 213 Commissioned by Fusion-io, Inc. EXECUTIVE SUMMARY Conventional

More information

Hyperscale. The new frontier for HPC. Philippe Trautmann. HPC/POD Sales Manager EMEA March 13th, 2011

Hyperscale. The new frontier for HPC. Philippe Trautmann. HPC/POD Sales Manager EMEA March 13th, 2011 Hyperscale The new frontier for HPC Philippe Trautmann HPC/POD Sales Manager EMEA March 13th, 2011 Hyperscale the new frontier for HPC New HPC customer requirements demand a shift in technology and market

More information

Maurice Askinazi Ofer Rind Tony Wong. HEPIX @ Cornell Nov. 2, 2010 Storage at BNL

Maurice Askinazi Ofer Rind Tony Wong. HEPIX @ Cornell Nov. 2, 2010 Storage at BNL Maurice Askinazi Ofer Rind Tony Wong HEPIX @ Cornell Nov. 2, 2010 Storage at BNL Traditional Storage Dedicated compute nodes and NFS SAN storage Simple and effective, but SAN storage became very expensive

More information

Data Center Storage Solutions

Data Center Storage Solutions Data Center Storage Solutions Enterprise software, appliance and hardware solutions you can trust When it comes to storage, most enterprises seek the same things: predictable performance, trusted reliability

More information

DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION

DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION A DIABLO WHITE PAPER AUGUST 2014 Ricky Trigalo Director of Business Development Virtualization, Diablo Technologies

More information

Unstructured Data Accelerator (UDA) Author: Motti Beck, Mellanox Technologies Date: March 27, 2012

Unstructured Data Accelerator (UDA) Author: Motti Beck, Mellanox Technologies Date: March 27, 2012 Unstructured Data Accelerator (UDA) Author: Motti Beck, Mellanox Technologies Date: March 27, 2012 1 Market Trends Big Data Growing technology deployments are creating an exponential increase in the volume

More information

Architecting a High Performance Storage System

Architecting a High Performance Storage System WHITE PAPER Intel Enterprise Edition for Lustre* Software High Performance Data Division Architecting a High Performance Storage System January 2014 Contents Introduction... 1 A Systematic Approach to

More information

Cray s Storage History and Outlook Lustre+ Jason Goodman, Cray LUG 2015 - Denver

Cray s Storage History and Outlook Lustre+ Jason Goodman, Cray LUG 2015 - Denver Cray s Storage History and Outlook Lustre+ Jason Goodman, Cray - Denver Agenda Cray History from Supercomputers to Lustre Where we are Today Cray Business OpenSFS Flashback to the Future SSDs, DVS, and

More information

Flash Performance in Storage Systems. Bill Moore Chief Engineer, Storage Systems Sun Microsystems

Flash Performance in Storage Systems. Bill Moore Chief Engineer, Storage Systems Sun Microsystems Flash Performance in Storage Systems Bill Moore Chief Engineer, Storage Systems Sun Microsystems 1 Disk to CPU Discontinuity Moore s Law is out-stripping disk drive performance (rotational speed) As a

More information

Moving Virtual Storage to the Cloud. Guidelines for Hosters Who Want to Enhance Their Cloud Offerings with Cloud Storage

Moving Virtual Storage to the Cloud. Guidelines for Hosters Who Want to Enhance Their Cloud Offerings with Cloud Storage Moving Virtual Storage to the Cloud Guidelines for Hosters Who Want to Enhance Their Cloud Offerings with Cloud Storage Table of Contents Overview... 1 Understanding the Storage Problem... 1 What Makes

More information

Accelerating I/O- Intensive Applications in IT Infrastructure with Innodisk FlexiArray Flash Appliance. Alex Ho, Product Manager Innodisk Corporation

Accelerating I/O- Intensive Applications in IT Infrastructure with Innodisk FlexiArray Flash Appliance. Alex Ho, Product Manager Innodisk Corporation Accelerating I/O- Intensive Applications in IT Infrastructure with Innodisk FlexiArray Flash Appliance Alex Ho, Product Manager Innodisk Corporation Outline Innodisk Introduction Industry Trend & Challenge

More information

IBM s System x with exflash: Workload Optimized, Pre-Integrated Server/Storage

IBM s System x with exflash: Workload Optimized, Pre-Integrated Server/Storage Advisory IBM s System x with exflash: Workload Optimized, Pre-Integrated Server/Storage Introduction In this Advisory, Clabby Analytics takes a closer look a class of servers known as preintegrated servers.

More information

Flash Accel, Flash Cache, Flash Pool, Flash Ray Was? Wann? Wie?

Flash Accel, Flash Cache, Flash Pool, Flash Ray Was? Wann? Wie? Flash Accel, Flash Cache, Flash Pool, Flash Ray Was? Wann? Wie? Systems Engineer Sven Willholz Performance Growth Performance Gap Performance Gap Challenge Server Huge gap between CPU and storage Relatively

More information

Best Practices for IBM Spectrum Scale: Announcing a new partnership

Best Practices for IBM Spectrum Scale: Announcing a new partnership Best Practices for IBM : Announcing a new partnership Bernard Shen, VP and Chief Engineer Re-Store, Infrastructure Engineering Tonya Witherspoon, Executive Director Ennovar, Institute of Emerging Technologies

More information

The Evolution of Microsoft SQL Server: The right time for Violin flash Memory Arrays

The Evolution of Microsoft SQL Server: The right time for Violin flash Memory Arrays The Evolution of Microsoft SQL Server: The right time for Violin flash Memory Arrays Executive Summary Microsoft SQL has evolved beyond serving simple workgroups to a platform delivering sophisticated

More information

MAKING THE BUSINESS CASE

MAKING THE BUSINESS CASE MAKING THE BUSINESS CASE LUSTRE FILE SYSTEMS ARE POISED TO PENETRATE COMMERCIAL MARKETS table of contents + Considerations in Building the.... 1... 3.... 4 A TechTarget White Paper by Long the de facto

More information

Building a Scalable Storage with InfiniBand

Building a Scalable Storage with InfiniBand WHITE PAPER Building a Scalable Storage with InfiniBand The Problem...1 Traditional Solutions and their Inherent Problems...2 InfiniBand as a Key Advantage...3 VSA Enables Solutions from a Core Technology...5

More information

www.thinkparq.com www.beegfs.com

www.thinkparq.com www.beegfs.com www.thinkparq.com www.beegfs.com KEY ASPECTS Maximum Flexibility Maximum Scalability BeeGFS supports a wide range of Linux distributions such as RHEL/Fedora, SLES/OpenSuse or Debian/Ubuntu as well as a

More information

IOmark- VDI. Nimbus Data Gemini Test Report: VDI- 130906- a Test Report Date: 6, September 2013. www.iomark.org

IOmark- VDI. Nimbus Data Gemini Test Report: VDI- 130906- a Test Report Date: 6, September 2013. www.iomark.org IOmark- VDI Nimbus Data Gemini Test Report: VDI- 130906- a Test Copyright 2010-2013 Evaluator Group, Inc. All rights reserved. IOmark- VDI, IOmark- VDI, VDI- IOmark, and IOmark are trademarks of Evaluator

More information

Database acceleration: 5x transactions per minute and 1/2 the cost of storage. Toby Coleridge, CTO

Database acceleration: 5x transactions per minute and 1/2 the cost of storage. Toby Coleridge, CTO Database acceleration: 5x transactions per minute and 1/2 the cost of storage Toby Coleridge, CTO Agenda: Database and USX in the real world Building database applications ontop of Atlantis USX. SQL Performance

More information

HPC Advisory Council

HPC Advisory Council HPC Advisory Council September 2012, Malaga CHRIS WEEDEN SYSTEMS ENGINEER WHO IS PANASAS? Panasas is a high performance storage vendor founded by Dr Garth Gibson Panasas delivers a fully supported, turnkey,

More information

21 st Century Storage What s New and What s Changing

21 st Century Storage What s New and What s Changing 21 st Century Storage What s New and What s Changing Randy Kerns Senior Strategist Evaluator Group Overview New technologies in storage - Continued evolution - Each has great economic value - Differing

More information

Cluster Implementation and Management; Scheduling

Cluster Implementation and Management; Scheduling Cluster Implementation and Management; Scheduling CPS343 Parallel and High Performance Computing Spring 2013 CPS343 (Parallel and HPC) Cluster Implementation and Management; Scheduling Spring 2013 1 /

More information

LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance

LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance 11 th International LS-DYNA Users Conference Session # LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance Gilad Shainer 1, Tong Liu 2, Jeff Layton 3, Onur Celebioglu

More information

Big Fast Data Hadoop acceleration with Flash. June 2013

Big Fast Data Hadoop acceleration with Flash. June 2013 Big Fast Data Hadoop acceleration with Flash June 2013 Agenda The Big Data Problem What is Hadoop Hadoop and Flash The Nytro Solution Test Results The Big Data Problem Big Data Output Facebook Traditional

More information

Solution Brief Network Design Considerations to Enable the Benefits of Flash Storage

Solution Brief Network Design Considerations to Enable the Benefits of Flash Storage Solution Brief Network Design Considerations to Enable the Benefits of Flash Storage Flash memory has been used to transform consumer devices such as smartphones, tablets, and ultranotebooks, and now it

More information

Fibre Channel Networked Flash Storage Fibre Channel Dominates Flash

Fibre Channel Networked Flash Storage Fibre Channel Dominates Flash Fibre Channel Networked Flash Storage Fibre Channel Dominates Flash Scott G. Kipp Director of Engineering Chair of FCIA Roadmap Subcommittee Chair of Ethernet Alliance Roadmap Subcommittee President of

More information

Busting Through I/O Bottlenecks with PCIe Flash Caching

Busting Through I/O Bottlenecks with PCIe Flash Caching Busting Through I/O Bottlenecks with PCIe Flash Caching Steve Fingerhut, VP of Marketing, Accelerated Solutions Division, LSI Corporation Santa Clara, CA November 2012 1 The Data Deluge Gap Network Traffic

More information

Seagate Lustre Update. Peter Bojanic 2015-04-13

Seagate Lustre Update. Peter Bojanic 2015-04-13 Seagate Lustre Update Peter Bojanic 2015-04-13 Seagate Cloud Systems and Solutions Delivering next-generation workloads with Intelligent Information Infrastructure tion OEM Cloud Services HPC HPC HPC Information

More information