February, 2015 Bill Loewe

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "February, 2015 Bill Loewe"

Transcription

1 February, 2015 Bill Loewe

2 Agenda System Metadata, a growing issue Parallel System - Lustre Overview Metadata and Distributed Namespace Test setup and implementation for metadata testing Scaling Metadata Servers High Availability Seagate Confidential

3 Metadata Performance System Performance typically viewed in Bandwidth Bandwidth problem largely addressed, but metadata is a growing issue. We see this in workloads with high numbers of files to access and process. Genome processing CPU Chip manufacturing Video compositing/rendering Seagate Confidential

4 Lustre Parallel System Lustre is an open source, distributed parallel file system Object-based design provides extreme scalability Compute clients interact directly with storage servers Comprised of: Clients Metadata Servers and Targets Storage Servers and Targets

5 Lustre Distributed NamespacE (DNE) Distributed NamespacE (DNE) is a new feature available in Lustre 2.5 that allows multiple MDS / MDT components to participate in a single file system. DNE allows the namespace to be divided across multiple metadata servers. Enables the size of the namespace and metadata throughput to be scaled with the number of servers. The Lustre DNE project is comprised of 2 phases. Seagate Confidential

6 Phase 1, Lustre 2.5 Release Remote Directories -- Lustre sub-directories are distributed over multiple metadata targets (MDTs). Sub-directory distribution is defined by an administrator. Remote Directories Root dir b dir c dir d dir e dir a dir b2 dir c2 dir d2 dir e2 Seagate Confidential

7 Phase 2, Lustre 2.7 Striped Directories -- The contents of a given directory are distributed over multiple MDTs. Striped Directories Striped Directory dir c2 dir e2 Seagate Confidential

8 Engineered Storage Solutions for HPC, Big Data & Cloud High speed networking (IB/40GB/e) Parallel file system/object Data protection High availability Flash optimization system (Ext4) Linux OS BIOS/IPMI ClusterStor GEM diagnostics Custom X86 embedded server Seagate storage platforms Seagate Storage Devices Architected Integrated Optimized Qualified Supported Seagate Confidential

9 Lustre Components Clients Directory Operations, open/close, metadata, and concurrency I/O and locking MDS creation, file status, and recovery OSS OSS OSS Seagate Confidential

10 ClusterStor Management Unit (CMU): Management and Metadata (MDS/MDT) CSM Manager and MDS/MGS Nodes 2RU 4-node Sandy Bridge Servers Server 1: CSM Mgmt Server 2: Boot Server 3: MGS Server 4: MDS Fault Tolerance (active/passive) Serviceability 2U24 JBOD MDT SAS JBOD for MDS/MGS/Management Disk Configuration Qty 4 Lustre Management (MGS) Qty 4 ClusterStor Management and NFS Qty 2 Global Hot spares Qty 14 Drives for MDT

11 Scalable Storage Unit (SSU) SSU 5U84 Enclosure 2 Object Storage Servers s per SSU Two (2) trays of 42 HDD s each for Object Storage Targets H/A on each SSU Infiniband QDR/FDR and 40Gb Ethernet data network connectivity

12 ClusterStor & Lustre 2.5 DNE Hardware DNE is available in ClusterStor v2.0 MDT0 is master and default in DNE environment DNE Servers are configured in active / active pairs Seagate 2U24 with 2 MDS embedded server modules Scale Metadata Capacity / Performance with DNE Server pairs Root dir b dir c dir d dir e Base MDS dir a dir b2 dir c2 dir d2 dir e2

13 ClusterStor Hardware and the Lustre System Meta Data and Management Servers 2U x 4 Servers Meta Data Target Seagate 2U24 JOBD 1) Where is file? 2) is at. Client 3) Single (3,072Kb) 4) is broken into block stripe segments (1,024Kb) Object Storage Server Seagate Embedded Application Server Object Storage Target Seagate 5U84 Storage Bay Bridge Enclosure 5a) block stripe 1 of 3 (1,024Kb) 5b) block stripe 2 of 3 (1,024Kb) 5c) block stripe 3 of 3 (1,024Kb)

14 Op/s Scaling MDS and DNEs MDS + 4 DNE Servers (2 ADUs) mdtest create/stat/del Mean of 5 iterations 600, , ,000 mdtest scaling MDS + 4 DNEs 300, , ,000 Mean Create Mean Stat Mean Remove 0 Seagate Confidential

15 Op/s Metadata High Availability MDT failover will ensure that the Lustre filesystem remains available in the face of MDS node failure Based on existing OSS pair failover model Failover is graceful, quick, and non-disruptive Failback is automatic and nondisruptive 140, , ,000 80,000 60,000 40,000 20,000 0 High Availability and Performance Before Failover Failed over After Failover Mean Create Mean Stat Mean Remove Seagate Confidential

16 Green Machine: Environmentally-Aware Cold Storage Solution Space Light weight Small foot print Cold storage optimized design Cooling Zero heat emission Ambient cooling/no fans High operating temp. tolerant HDDs Power Green Dynamic power management Low power servers Aggressive TCO goals Recyclable chassis Reduced metal Responsible disposal of old chassis Lowest Operating Cost Reduced Carbon footprint Best for the Planet 15

17 Typical Use cases Retrieve content, photographs etc. from deep archive while maintaining consistent user experience Online pictures/social media store use cases Pictures >45 days in cold storage Retrieve MRIs/X-rays of a patient Use cases leveraging Tape-based solutions 16

18 Thank you!

NetApp High-Performance Computing Solution for Lustre: Solution Guide

NetApp High-Performance Computing Solution for Lustre: Solution Guide Technical Report NetApp High-Performance Computing Solution for Lustre: Solution Guide Robert Lai, NetApp August 2012 TR-3997 TABLE OF CONTENTS 1 Introduction... 5 1.1 NetApp HPC Solution for Lustre Introduction...5

More information

An Alternative Storage Solution for MapReduce. Eric Lomascolo Director, Solutions Marketing

An Alternative Storage Solution for MapReduce. Eric Lomascolo Director, Solutions Marketing An Alternative Storage Solution for MapReduce Eric Lomascolo Director, Solutions Marketing MapReduce Breaks the Problem Down Data Analysis Distributes processing work (Map) across compute nodes and accumulates

More information

Seagate Lustre Update. Peter Bojanic 2015-04-13

Seagate Lustre Update. Peter Bojanic 2015-04-13 Seagate Lustre Update Peter Bojanic 2015-04-13 Seagate Cloud Systems and Solutions Delivering next-generation workloads with Intelligent Information Infrastructure tion OEM Cloud Services HPC HPC HPC Information

More information

Architecting a High Performance Storage System

Architecting a High Performance Storage System WHITE PAPER Intel Enterprise Edition for Lustre* Software High Performance Data Division Architecting a High Performance Storage System January 2014 Contents Introduction... 1 A Systematic Approach to

More information

New Storage System Solutions

New Storage System Solutions New Storage System Solutions Craig Prescott Research Computing May 2, 2013 Outline } Existing storage systems } Requirements and Solutions } Lustre } /scratch/lfs } Questions? Existing Storage Systems

More information

Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre

Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre University of Cambridge, UIS, HPC Service Authors: Wojciech Turek, Paul Calleja, John Taylor

More information

High Performance Computing OpenStack Options. September 22, 2015

High Performance Computing OpenStack Options. September 22, 2015 High Performance Computing OpenStack PRESENTATION TITLE GOES HERE Options September 22, 2015 Today s Presenters Glyn Bowden, SNIA Cloud Storage Initiative Board HP Helion Professional Services Alex McDonald,

More information

High-Availability and Scalable Cluster-in-a-Box HPC Storage Solution

High-Availability and Scalable Cluster-in-a-Box HPC Storage Solution Intel Solutions Reference Architecture High-Availability and Scalable Cluster-in-a-Box HPC Storage Solution Using RAIDIX Storage Software Integrated with Intel Enterprise Edition for Lustre* Audience and

More information

Data management challenges in todays Healthcare and Life Sciences ecosystems

Data management challenges in todays Healthcare and Life Sciences ecosystems Data management challenges in todays Healthcare and Life Sciences ecosystems Jose L. Alvarez Principal Engineer, WW Director Life Sciences jose.alvarez@seagate.com Evolution of Data Sets in Healthcare

More information

Cray Lustre File System Monitoring

Cray Lustre File System Monitoring Cray Lustre File System Monitoring esfsmon Jeff Keopp OSIO/ES Systems Cray Inc. St. Paul, MN USA keopp@cray.com Harold Longley OSIO Cray Inc. St. Paul, MN USA htg@cray.com Abstract The Cray Data Management

More information

Lustre * Filesystem for Cloud and Hadoop *

Lustre * Filesystem for Cloud and Hadoop * OpenFabrics Software User Group Workshop Lustre * Filesystem for Cloud and Hadoop * Robert Read, Intel Lustre * for Cloud and Hadoop * Brief Lustre History and Overview Using Lustre with Hadoop Intel Cloud

More information

Panasas at the RCF. Fall 2005 Robert Petkus RHIC/USATLAS Computing Facility Brookhaven National Laboratory. Robert Petkus Panasas at the RCF

Panasas at the RCF. Fall 2005 Robert Petkus RHIC/USATLAS Computing Facility Brookhaven National Laboratory. Robert Petkus Panasas at the RCF Panasas at the RCF HEPiX at SLAC Fall 2005 Robert Petkus RHIC/USATLAS Computing Facility Brookhaven National Laboratory Centralized File Service Single, facility-wide namespace for files. Uniform, facility-wide

More information

Lustre failover experience

Lustre failover experience Lustre failover experience Lustre Administrators and Developers Workshop Paris 1 September 25, 2012 TOC Who we are Our Lustre experience: the environment Deployment Benchmarks What's next 2 Who we are

More information

Reference Design: Scalable Object Storage with Seagate Kinetic, Supermicro, and SwiftStack

Reference Design: Scalable Object Storage with Seagate Kinetic, Supermicro, and SwiftStack Reference Design: Scalable Object Storage with Seagate Kinetic, Supermicro, and SwiftStack May 2015 Copyright 2015 SwiftStack, Inc. swiftstack.com Page 1 of 19 Table of Contents INTRODUCTION... 3 OpenStack

More information

GPFS Storage Server. Concepts and Setup in Lemanicus BG/Q system" Christian Clémençon (EPFL-DIT)" " 4 April 2013"

GPFS Storage Server. Concepts and Setup in Lemanicus BG/Q system Christian Clémençon (EPFL-DIT)  4 April 2013 GPFS Storage Server Concepts and Setup in Lemanicus BG/Q system" Christian Clémençon (EPFL-DIT)" " Agenda" GPFS Overview" Classical versus GSS I/O Solution" GPFS Storage Server (GSS)" GPFS Native RAID

More information

Xyratex Update. Michael K. Connolly. Partner and Alliances Development

Xyratex Update. Michael K. Connolly. Partner and Alliances Development Xyratex Update Michael K. Connolly Partner and Alliances Development Is Now 2 The Continued Power of Xyratex Global Solutions Provider of High Quality Data Storage Hardware, Software and Services Broad

More information

DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION

DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION A DIABLO WHITE PAPER AUGUST 2014 Ricky Trigalo Director of Business Development Virtualization, Diablo Technologies

More information

UCS M-Series Modular Servers

UCS M-Series Modular Servers UCS M-Series Modular Servers The Next Wave of UCS Innovation Marian Klas Cisco Systems June 2015 Cisco UCS - Powering Applications at Every Scale Edge-Scale Computing Cloud-Scale Computing Seamlessly Extend

More information

Hyper-V over SMB Remote File Storage support in Windows Server 8 Hyper-V. Jose Barreto Principal Program Manager Microsoft Corporation

Hyper-V over SMB Remote File Storage support in Windows Server 8 Hyper-V. Jose Barreto Principal Program Manager Microsoft Corporation Hyper-V over SMB Remote File Storage support in Windows Server 8 Hyper-V Jose Barreto Principal Program Manager Microsoft Corporation Agenda Hyper-V over SMB - Overview How to set it up Configuration Options

More information

Application Performance for High Performance Computing Environments

Application Performance for High Performance Computing Environments Application Performance for High Performance Computing Environments Leveraging the strengths of Computationally intensive applications With high performance scale out file serving In data storage modules

More information

Click on the diagram to see RAID 0 in action

Click on the diagram to see RAID 0 in action Click on the diagram to see RAID 0 in action RAID Level 0 requires a minimum of 2 drives to implement RAID 0 implements a striped disk array, the data is broken down into blocks and each block is written

More information

HPC Advisory Council

HPC Advisory Council HPC Advisory Council September 2012, Malaga CHRIS WEEDEN SYSTEMS ENGINEER WHO IS PANASAS? Panasas is a high performance storage vendor founded by Dr Garth Gibson Panasas delivers a fully supported, turnkey,

More information

www.thinkparq.com www.beegfs.com

www.thinkparq.com www.beegfs.com www.thinkparq.com www.beegfs.com KEY ASPECTS Maximum Flexibility Maximum Scalability BeeGFS supports a wide range of Linux distributions such as RHEL/Fedora, SLES/OpenSuse or Debian/Ubuntu as well as a

More information

IBM System x GPFS Storage Server

IBM System x GPFS Storage Server IBM System x GPFS Storage Server Schöne Aussicht en für HPC Speicher ZKI-Arbeitskreis Paderborn, 15.03.2013 Karsten Kutzer Client Technical Architect Technical Computing IBM Systems & Technology Group

More information

E4 UNIFIED STORAGE powered by Syneto

E4 UNIFIED STORAGE powered by Syneto E4 UNIFIED STORAGE powered by Syneto THE E4 UNIFIED STORAGE (US) SERIES POWERED BY SYNETO From working in the heart of IT environment and with our major customers coming from Research, Education and PA,

More information

How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda

How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda 1 Outline Build a cost-efficient Swift cluster with expected performance Background & Problem Solution Experiments

More information

THE SUN STORAGE AND ARCHIVE SOLUTION FOR HPC

THE SUN STORAGE AND ARCHIVE SOLUTION FOR HPC THE SUN STORAGE AND ARCHIVE SOLUTION FOR HPC The Right Data, in the Right Place, at the Right Time José Martins Storage Practice Sun Microsystems 1 Agenda Sun s strategy and commitment to the HPC or technical

More information

Stovepipes to Clouds. Rick Reid Principal Engineer SGI Federal. 2013 by SGI Federal. Published by The Aerospace Corporation with permission.

Stovepipes to Clouds. Rick Reid Principal Engineer SGI Federal. 2013 by SGI Federal. Published by The Aerospace Corporation with permission. Stovepipes to Clouds Rick Reid Principal Engineer SGI Federal 2013 by SGI Federal. Published by The Aerospace Corporation with permission. Agenda Stovepipe Characteristics Why we Built Stovepipes Cluster

More information

Current Status of FEFS for the K computer

Current Status of FEFS for the K computer Current Status of FEFS for the K computer Shinji Sumimoto Fujitsu Limited Apr.24 2012 LUG2012@Austin Outline RIKEN and Fujitsu are jointly developing the K computer * Development continues with system

More information

SMB Direct for SQL Server and Private Cloud

SMB Direct for SQL Server and Private Cloud SMB Direct for SQL Server and Private Cloud Increased Performance, Higher Scalability and Extreme Resiliency June, 2014 Mellanox Overview Ticker: MLNX Leading provider of high-throughput, low-latency server

More information

Investigation of Storage Systems for use in Grid Applications

Investigation of Storage Systems for use in Grid Applications Investigation of Storage Systems for use in Grid Applications Overview Test bed and testing methodology Tests of general performance Tests of the metadata servers Tests of root-based applications HEPiX

More information

LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance

LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance 11 th International LS-DYNA Users Conference Session # LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance Gilad Shainer 1, Tong Liu 2, Jeff Layton 3, Onur Celebioglu

More information

A Case Study: Performance Analysis and Optimization of SAS Grid Computing Scaling on a Shared Storage

A Case Study: Performance Analysis and Optimization of SAS Grid Computing Scaling on a Shared Storage Paper 1815-2014 A Case Study: Performance Analysis and Optimization of SAS Grid Computing Scaling on a Shared Storage Suleyman Sair, Brett Lee, Ying M. Zhang, Intel Corporation ABSTRACT SAS Grid Computing

More information

Improving Lustre OST Performance with ClusterStor GridRAID. John Fragalla Principal Architect High Performance Computing

Improving Lustre OST Performance with ClusterStor GridRAID. John Fragalla Principal Architect High Performance Computing Improving Lustre OST Performance with ClusterStor GridRAID John Fragalla Principal Architect High Performance Computing Legacy RAID 6 No Longer Sufficient 2013 RAID 6 data protection challenges Long rebuild

More information

NIVEO Network Attached Storage Series NNAS-D5 NNAS-R4. More information: WWW.NIVEOPROFESSIONAL.COM INFO@NIVEOPROFESSIONAL.COM

NIVEO Network Attached Storage Series NNAS-D5 NNAS-R4. More information: WWW.NIVEOPROFESSIONAL.COM INFO@NIVEOPROFESSIONAL.COM NIVEO Network Attached Storage Series NNAS-D5 NNAS-R4 More information: WWW.NIVEOPROFESSIONAL.COM INFO@NIVEOPROFESSIONAL.COM Product Specification Introduction The NIVEO NNAS series is specifically designed

More information

Lessons learned from parallel file system operation

Lessons learned from parallel file system operation Lessons learned from parallel file system operation Roland Laifer STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association

More information

Lustre SMB Gateway. Integrating Lustre with Windows

Lustre SMB Gateway. Integrating Lustre with Windows Lustre SMB Gateway Integrating Lustre with Windows Hardware: Old vs New Compute 60 x Dell PowerEdge 1950-8 x 2.6Ghz cores, 16GB, 500GB Sata, 1GBe - Win7 x64 Storage 1 x Dell R510-12 x 2TB Sata, RAID5,

More information

Performance Comparison of SQL based Big Data Analytics with Lustre and HDFS file systems

Performance Comparison of SQL based Big Data Analytics with Lustre and HDFS file systems Performance Comparison of SQL based Big Data Analytics with Lustre and HDFS file systems Rekha Singhal and Gabriele Pacciucci * Other names and brands may be claimed as the property of others. Lustre File

More information

The Use of Flash in Large-Scale Storage Systems. Nathan.Rutman@Seagate.com

The Use of Flash in Large-Scale Storage Systems. Nathan.Rutman@Seagate.com The Use of Flash in Large-Scale Storage Systems Nathan.Rutman@Seagate.com 1 Seagate s Flash! Seagate acquired LSI s Flash Components division May 2014 Selling multiple formats / capacities today Nytro

More information

HPC Update: Engagement Model

HPC Update: Engagement Model HPC Update: Engagement Model MIKE VILDIBILL Director, Strategic Engagements Sun Microsystems mikev@sun.com Our Strategy Building a Comprehensive HPC Portfolio that Delivers Differentiated Customer Value

More information

IBM System x GPFS Storage Server

IBM System x GPFS Storage Server IBM System x GPFS Storage Crispin Keable Technical Computing Architect 1 IBM Technical Computing comprehensive portfolio uniquely addresses supercomputing and mainstream client needs Technical Computing

More information

Evaluation of Dell PowerEdge VRTX Shared PERC8 in Failover Scenario

Evaluation of Dell PowerEdge VRTX Shared PERC8 in Failover Scenario Evaluation of Dell PowerEdge VRTX Shared PERC8 in Failover Scenario Evaluation report prepared under contract with Dell Introduction Dell introduced its PowerEdge VRTX integrated IT solution for remote-office

More information

CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT

CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT SS Data & Storage CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT HEPiX Fall 2012 Workshop October 15-19, 2012 Institute of High Energy Physics, Beijing, China SS Outline

More information

Introduction to NetApp Infinite Volume

Introduction to NetApp Infinite Volume Technical Report Introduction to NetApp Infinite Volume Sandra Moulton, Reena Gupta, NetApp April 2013 TR-4037 Summary This document provides an overview of NetApp Infinite Volume, a new innovation in

More information

Performance Comparison of Intel Enterprise Edition for Lustre* software and HDFS for MapReduce Applications

Performance Comparison of Intel Enterprise Edition for Lustre* software and HDFS for MapReduce Applications Performance Comparison of Intel Enterprise Edition for Lustre software and HDFS for MapReduce Applications Rekha Singhal, Gabriele Pacciucci and Mukesh Gangadhar 2 Hadoop Introduc-on Open source MapReduce

More information

Intel RAID SSD Cache Controller RCS25ZB040

Intel RAID SSD Cache Controller RCS25ZB040 SOLUTION Brief Intel RAID SSD Cache Controller RCS25ZB040 When Faster Matters Cost-Effective Intelligent RAID with Embedded High Performance Flash Intel RAID SSD Cache Controller RCS25ZB040 When Faster

More information

Quantum StorNext. Product Brief: Distributed LAN Client

Quantum StorNext. Product Brief: Distributed LAN Client Quantum StorNext Product Brief: Distributed LAN Client NOTICE This product brief may contain proprietary information protected by copyright. Information in this product brief is subject to change without

More information

Hyper-V over SMB: Remote File Storage Support in Windows Server 2012 Hyper-V. Jose Barreto Principal Program Manager Microsoft Corporation

Hyper-V over SMB: Remote File Storage Support in Windows Server 2012 Hyper-V. Jose Barreto Principal Program Manager Microsoft Corporation Hyper-V over SMB: Remote Storage Support in Windows Server 2012 Hyper-V Jose Barreto Principal Program Manager Microsoft Corporation Abstract In this session, we cover the Windows Server 2012 Hyper-V support

More information

VMware Virtual SAN Backup Using VMware vsphere Data Protection Advanced SEPTEMBER 2014

VMware Virtual SAN Backup Using VMware vsphere Data Protection Advanced SEPTEMBER 2014 VMware SAN Backup Using VMware vsphere Data Protection Advanced SEPTEMBER 2014 VMware SAN Backup Using VMware vsphere Table of Contents Introduction.... 3 vsphere Architectural Overview... 4 SAN Backup

More information

Sun Storage Perspective & Lustre Architecture. Dr. Peter Braam VP Sun Microsystems

Sun Storage Perspective & Lustre Architecture. Dr. Peter Braam VP Sun Microsystems Sun Storage Perspective & Lustre Architecture Dr. Peter Braam VP Sun Microsystems Agenda Future of Storage Sun s vision Lustre - vendor neutral architecture roadmap Sun s view on storage introduction The

More information

New and Improved Lustre Performance Monitoring Tool. Torben Kling Petersen, PhD Principal Engineer. Chris Bloxham Principal Architect

New and Improved Lustre Performance Monitoring Tool. Torben Kling Petersen, PhD Principal Engineer. Chris Bloxham Principal Architect New and Improved Lustre Performance Monitoring Tool Torben Kling Petersen, PhD Principal Engineer Chris Bloxham Principal Architect Lustre monitoring Performance Granular Aggregated Components Subsystem

More information

High Availability and Backup Strategies for the Lustre MDS Server

High Availability and Backup Strategies for the Lustre MDS Server HA and Backup Methods for Lustre Hepix May '08 K.Miers@gsi.de 1 High Availability and Backup Strategies for the Lustre MDS Server Spring 2008 Karin Miers / GSI HA and Backup Methods for Lustre Hepix May

More information

Netapp HPC Solution for Lustre. Rich Fenton (fenton@netapp.com) UK Solutions Architect

Netapp HPC Solution for Lustre. Rich Fenton (fenton@netapp.com) UK Solutions Architect Netapp HPC Solution for Lustre Rich Fenton (fenton@netapp.com) UK Solutions Architect Agenda NetApp Introduction Introducing the E-Series Platform Why E-Series for Lustre? Modular Scale-out Capacity Density

More information

Big data management with IBM General Parallel File System

Big data management with IBM General Parallel File System Big data management with IBM General Parallel File System Optimize storage management and boost your return on investment Highlights Handles the explosive growth of structured and unstructured data Offers

More information

BlueArc unified network storage systems 7th TF-Storage Meeting. Scale Bigger, Store Smarter, Accelerate Everything

BlueArc unified network storage systems 7th TF-Storage Meeting. Scale Bigger, Store Smarter, Accelerate Everything BlueArc unified network storage systems 7th TF-Storage Meeting Scale Bigger, Store Smarter, Accelerate Everything BlueArc s Heritage Private Company, founded in 1998 Headquarters in San Jose, CA Highest

More information

Seagate Kinetic Open Storage Platform Ecosystem. Build a more efficient cloud

Seagate Kinetic Open Storage Platform Ecosystem. Build a more efficient cloud Seagate Kinetic Open Storage Platform Ecosystem Build a more efficient cloud The Seagate Kinetic Open Storage Platform. It s a game-changer. The Seagate Kinetic Open Storage Platform is a revolutionary

More information

Investigation of storage options for scientific computing on Grid and Cloud facilities

Investigation of storage options for scientific computing on Grid and Cloud facilities Investigation of storage options for scientific computing on Grid and Cloud facilities Overview Context Test Bed Lustre Evaluation Standard benchmarks Application-based benchmark HEPiX Storage Group report

More information

Achieving Real-Time Business Solutions Using Graph Database Technology and High Performance Networks

Achieving Real-Time Business Solutions Using Graph Database Technology and High Performance Networks WHITE PAPER July 2014 Achieving Real-Time Business Solutions Using Graph Database Technology and High Performance Networks Contents Executive Summary...2 Background...3 InfiniteGraph...3 High Performance

More information

Increasing Storage Performance

Increasing Storage Performance Increasing Storage Performance High Performance MicroTiering for Server DAS Storage Andy Mills President/CEO, Enmotus andy.mills@enmotus.com Santa Clara, CA November 2011 Summary Review of challenges of

More information

POWER ALL GLOBAL FILE SYSTEM (PGFS)

POWER ALL GLOBAL FILE SYSTEM (PGFS) POWER ALL GLOBAL FILE SYSTEM (PGFS) Defining next generation of global storage grid Power All Networks Ltd. Technical Whitepaper April 2008, version 1.01 Table of Content 1. Introduction.. 3 2. Paradigm

More information

High Availability in Lustre. John Fragalla Principal Solutions Architect High Performance Computing

High Availability in Lustre. John Fragalla Principal Solutions Architect High Performance Computing High Availability in Lustre John Fragalla Principal Solutions Architect High Performance Computing Goal for Lustre HA Detect failures and provide architecture to deal with any level of Failure Provide

More information

Network Storage Appliance

Network Storage Appliance Enterprise File System HyperFS integrates high-performance SAN, NAS and data protection under one global namespace. No Per Seat Client Cost Based on concurrent RAID performance, connectivity, bandwidth,

More information

The last 18 months. AutoScale. IaaS. BizTalk Services Hyper-V Disaster Recovery Support. Multi-Factor Auth. Hyper-V Recovery.

The last 18 months. AutoScale. IaaS. BizTalk Services Hyper-V Disaster Recovery Support. Multi-Factor Auth. Hyper-V Recovery. Offline Operations Traffic ManagerLarge Memory SKU SQL, SharePoint, BizTalk Images HDInsight Windows Phone Support Per Minute Billing HTML 5/CORS Android Support Custom Mobile API AutoScale BizTalk Services

More information

Comparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet. September 2014

Comparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet. September 2014 Comparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet Anand Rangaswamy September 2014 Storage Developer Conference Mellanox Overview Ticker: MLNX Leading provider of high-throughput,

More information

Cluster Implementation and Management; Scheduling

Cluster Implementation and Management; Scheduling Cluster Implementation and Management; Scheduling CPS343 Parallel and High Performance Computing Spring 2013 CPS343 (Parallel and HPC) Cluster Implementation and Management; Scheduling Spring 2013 1 /

More information

The Panasas Parallel Storage Cluster. Acknowledgement: Some of the material presented is under copyright by Panasas Inc.

The Panasas Parallel Storage Cluster. Acknowledgement: Some of the material presented is under copyright by Panasas Inc. The Panasas Parallel Storage Cluster What Is It? What Is The Panasas ActiveScale Storage Cluster A complete hardware and software storage solution Implements An Asynchronous, Parallel, Object-based, POSIX

More information

Understanding Microsoft Storage Spaces

Understanding Microsoft Storage Spaces S T O R A G E Understanding Microsoft Storage Spaces A critical look at its key features and value proposition for storage administrators A Microsoft s Storage Spaces solution offers storage administrators

More information

Cisco Small Business NSS2000 Series Network Storage System

Cisco Small Business NSS2000 Series Network Storage System Cisco Small Business NSS2000 Series Network Storage System Cisco NSS2000 2-Bay Gigabit Storage System Chassis Cisco NSS2050 2-Bay Gigabit Storage System Chassis with Two 250-Gigabyte Hard Disk Drives Cisco

More information

Four Reasons To Start Working With NFSv4.1 Now

Four Reasons To Start Working With NFSv4.1 Now Four Reasons To Start Working With NFSv4.1 Now PRESENTATION TITLE GOES HERE Presented by: Alex McDonald Hosted by: Gilles Chekroun Ethernet Storage Forum Members The SNIA Ethernet Storage Forum (ESF) focuses

More information

Integrated Storage Solutions ISS Series

Integrated Storage Solutions ISS Series N E T W O R K e d s t o r a g e Integrated Storage Solutions ISS Series scalable NAS and iscsi SAN Product Family Affordable High Performance Enterprise Features Utmost Reliability and Flexibility NAS

More information

IBM General Parallel File System (GPFS ) 3.5 File Placement Optimizer (FPO)

IBM General Parallel File System (GPFS ) 3.5 File Placement Optimizer (FPO) IBM General Parallel File System (GPFS ) 3.5 File Placement Optimizer (FPO) Rick Koopman IBM Technical Computing Business Development Benelux Rick_koopman@nl.ibm.com Enterprise class replacement for HDFS

More information

OSDsim - a Simulation and Design Platform of an Object-based Storage Device

OSDsim - a Simulation and Design Platform of an Object-based Storage Device OSDsim - a Simulation and Design Platform of an Object-based Storage Device WeiYa Xi Wei-Khing For DongHong Wang Renuga Kanagavelu Wai-Kit Goh Data Storage Institute, Singapore xi weiya@dsi.a-star.edu.sg

More information

Appro Supercomputer Solutions Best Practices Appro 2012 Deployment Successes. Anthony Kenisky, VP of North America Sales

Appro Supercomputer Solutions Best Practices Appro 2012 Deployment Successes. Anthony Kenisky, VP of North America Sales Appro Supercomputer Solutions Best Practices Appro 2012 Deployment Successes Anthony Kenisky, VP of North America Sales About Appro Over 20 Years of Experience 1991 2000 OEM Server Manufacturer 2001-2007

More information

THE SUMMARY. ARKSERIES - pg. 3. ULTRASERIES - pg. 5. EXTREMESERIES - pg. 9

THE SUMMARY. ARKSERIES - pg. 3. ULTRASERIES - pg. 5. EXTREMESERIES - pg. 9 PRODUCT CATALOG THE SUMMARY ARKSERIES - pg. 3 ULTRASERIES - pg. 5 EXTREMESERIES - pg. 9 ARK SERIES THE HIGH DENSITY STORAGE FOR ARCHIVE AND BACKUP Unlimited scalability Painless Disaster Recovery The ARK

More information

Object storage in Cloud Computing and Embedded Processing

Object storage in Cloud Computing and Embedded Processing Object storage in Cloud Computing and Embedded Processing Jan Jitze Krol Systems Engineer DDN We Accelerate Information Insight DDN is a Leader in Massively Scalable Platforms and Solutions for Big Data

More information

Direct NFS - Design considerations for next-gen NAS appliances optimized for database workloads Akshay Shah Gurmeet Goindi Oracle

Direct NFS - Design considerations for next-gen NAS appliances optimized for database workloads Akshay Shah Gurmeet Goindi Oracle Direct NFS - Design considerations for next-gen NAS appliances optimized for database workloads Akshay Shah Gurmeet Goindi Oracle Agenda Introduction Database Architecture Direct NFS Client NFS Server

More information

Windows Server 2012 授 權 說 明

Windows Server 2012 授 權 說 明 Windows Server 2012 授 權 說 明 PROCESSOR + CAL HA 功 能 相 同 的 記 憶 體 及 處 理 器 容 量 虛 擬 化 Windows Server 2008 R2 Datacenter Price: NTD173,720 (2 CPU) Packaging All features Unlimited virtual instances Per processor

More information

modular Storage Solutions MSS Series

modular Storage Solutions MSS Series N E T W O R K e d s t o r a g e modular Storage Solutions MSS Series NAS and iscsi SAN Product Family High Performance Enterprise Features Easily Scalable Utmost Reliability and Flexibility NAS & iscsi

More information

Integrated Grid Solutions. and Greenplum

Integrated Grid Solutions. and Greenplum EMC Perspective Integrated Grid Solutions from SAS, EMC Isilon and Greenplum Introduction Intensifying competitive pressure and vast growth in the capabilities of analytic computing platforms are driving

More information

SMB Advanced Networking for Fault Tolerance and Performance. Jose Barreto Principal Program Managers Microsoft Corporation

SMB Advanced Networking for Fault Tolerance and Performance. Jose Barreto Principal Program Managers Microsoft Corporation SMB Advanced Networking for Fault Tolerance and Performance Jose Barreto Principal Program Managers Microsoft Corporation Agenda SMB Remote File Storage for Server Apps SMB Direct (SMB over RDMA) SMB Multichannel

More information

Cray DVS: Data Virtualization Service

Cray DVS: Data Virtualization Service Cray : Data Virtualization Service Stephen Sugiyama and David Wallace, Cray Inc. ABSTRACT: Cray, the Cray Data Virtualization Service, is a new capability being added to the XT software environment with

More information

Cisco Small Business NSS3000 Series Network Storage System

Cisco Small Business NSS3000 Series Network Storage System Cisco Small Business NSS3000 Series Network Storage System Cisco NSS3100 4-Bay Gigabit Storage System Chassis with Two 500-Gigabyte Hard Disk Drives Cisco NSS3200 4-Bay Gigabit Storage System Chassis with

More information

præsentation oktober 2011

præsentation oktober 2011 Johnny Olesen System X presale præsentation oktober 2011 2010 IBM Corporation 2 Hvem er jeg Dagens agenda Server overview System Director 3 4 Portfolio-wide Innovation with IBM System x and BladeCenter

More information

EXAScaler. Product Release Notes. Version 2.0.1. Revision A0

EXAScaler. Product Release Notes. Version 2.0.1. Revision A0 EXAScaler Version 2.0.1 Product Release Notes Revision A0 December 2013 Important Information Information in this document is subject to change without notice and does not represent a commitment on the

More information

WOS OBJECT STORAGE PRODUCT BROCHURE DDN.COM 1.800.837.2298. 360 Full Spectrum Object Storage

WOS OBJECT STORAGE PRODUCT BROCHURE DDN.COM 1.800.837.2298. 360 Full Spectrum Object Storage PRODUCT BROCHURE WOS OBJECT STORAGE 360 Full Spectrum Object Storage The promise of object storage is simple: to enable organizations to build highly Performance Scalability Reliability Efficiency Security

More information

Overview Context Test Bed Lustre Evaluation Standard benchmarks Fault tolerance Application-based benchmark HEPiX Storage Group report

Overview Context Test Bed Lustre Evaluation Standard benchmarks Fault tolerance Application-based benchmark HEPiX Storage Group report Storage Evaluation on FG, FC, and GPCF Overview Context Test Bed Lustre Evaluation Standard benchmarks Fault tolerance Application-based benchmark HEPiX Storage Group report Jan 18, 2011 Gabriele Garzoglio,

More information

EMC DATA DOMAIN OPERATING SYSTEM

EMC DATA DOMAIN OPERATING SYSTEM ESSENTIALS HIGH-SPEED, SCALABLE DEDUPLICATION Up to 58.7 TB/hr performance Reduces protection storage requirements by 10 to 30x CPU-centric scalability DATA INVULNERABILITY ARCHITECTURE Inline write/read

More information

Vess A2000 Series HA Surveillance with Milestone XProtect VMS Version 1.0

Vess A2000 Series HA Surveillance with Milestone XProtect VMS Version 1.0 Vess A2000 Series HA Surveillance with Milestone XProtect VMS Version 1.0 2014 PROMISE Technology, Inc. All Rights Reserved. Contents Introduction 1 Purpose 1 Scope 1 Audience 1 What is High Availability?

More information

Enabling High performance Big Data platform with RDMA

Enabling High performance Big Data platform with RDMA Enabling High performance Big Data platform with RDMA Tong Liu HPC Advisory Council Oct 7 th, 2014 Shortcomings of Hadoop Administration tooling Performance Reliability SQL support Backup and recovery

More information

Storage Architectures for Big Data in the Cloud

Storage Architectures for Big Data in the Cloud Storage Architectures for Big Data in the Cloud Sam Fineberg HP Storage CT Office/ May 2013 Overview Introduction What is big data? Big Data I/O Hadoop/HDFS SAN Distributed FS Cloud Summary Research Areas

More information

(Scale Out NAS System)

(Scale Out NAS System) For Unlimited Capacity & Performance Clustered NAS System (Scale Out NAS System) Copyright 2010 by Netclips, Ltd. All rights reserved -0- 1 2 3 4 5 NAS Storage Trend Scale-Out NAS Solution Scaleway Advantages

More information

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com Parallels Cloud Storage White Paper Performance Benchmark Results www.parallels.com Table of Contents Executive Summary... 3 Architecture Overview... 3 Key Features... 4 No Special Hardware Requirements...

More information

Highly-Available Distributed Storage. UF HPC Center Research Computing University of Florida

Highly-Available Distributed Storage. UF HPC Center Research Computing University of Florida Highly-Available Distributed Storage UF HPC Center Research Computing University of Florida Storage is Boring Slow, troublesome, albatross around the neck of high-performance computing UF Research Computing

More information

The Parallel File System HP SFS/Lustre on xc2

The Parallel File System HP SFS/Lustre on xc2 The Parallel File System HP SFS/Lustre on xc2 Computing Centre (SSCK) University of Karlsruhe Germany Laifer@rz.uni-karlsruhe.de page 1 Outline» What is Lustre?» What is HP SFS?» Overview of HP SFS on

More information

Eloquence Training What s new in Eloquence B.08.00

Eloquence Training What s new in Eloquence B.08.00 Eloquence Training What s new in Eloquence B.08.00 2010 Marxmeier Software AG Rev:100727 Overview Released December 2008 Supported until November 2013 Supports 32-bit and 64-bit platforms HP-UX Itanium

More information

Parallels Cloud Server 6.0

Parallels Cloud Server 6.0 Parallels Cloud Server 6.0 Parallels Cloud Storage I/O Benchmarking Guide September 05, 2014 Copyright 1999-2014 Parallels IP Holdings GmbH and its affiliates. All rights reserved. Parallels IP Holdings

More information

EMC AVAMAR. a reason for Cloud. Deduplication backup software Replication for Disaster Recovery

EMC AVAMAR. a reason for Cloud. Deduplication backup software Replication for Disaster Recovery EMC AVAMAR a reason for Cloud Deduplication backup software Replication for Disaster Recovery Bogdan Stefanescu (Bogs) EMC Data Protection Solutions bogdan.stefanescu@emc.com 1 BUSINESS DRIVERS Increase

More information

QoS-Aware Storage Virtualization for Cloud File Systems. Christoph Kleineweber (Speaker) Alexander Reinefeld Thorsten Schütt. Zuse Institute Berlin

QoS-Aware Storage Virtualization for Cloud File Systems. Christoph Kleineweber (Speaker) Alexander Reinefeld Thorsten Schütt. Zuse Institute Berlin QoS-Aware Storage Virtualization for Cloud File Systems Christoph Kleineweber (Speaker) Alexander Reinefeld Thorsten Schütt Zuse Institute Berlin 1 Outline Introduction Performance Models Reservation Scheduling

More information

Maginatics Cloud Storage Platform for Elastic NAS Workloads

Maginatics Cloud Storage Platform for Elastic NAS Workloads Maginatics Cloud Storage Platform for Elastic NAS Workloads Optimized for Cloud Maginatics Cloud Storage Platform () is the first solution optimized for the cloud. It provides lower cost, easier administration,

More information

VTrak 15200 SATA RAID Storage System

VTrak 15200 SATA RAID Storage System Page 1 15-Drive Supports over 5 TB of reliable, low-cost, high performance storage 15200 Product Highlights First to deliver a full HW iscsi solution with SATA drives - Lower CPU utilization - Higher data

More information