GPFS Storage Server. Concepts and Setup in Lemanicus BG/Q system" Christian Clémençon (EPFL-DIT)" " 4 April 2013"
|
|
- Gregory Robertson
- 2 years ago
- Views:
Transcription
1 GPFS Storage Server Concepts and Setup in Lemanicus BG/Q system" Christian Clémençon (EPFL-DIT)" "
2 Agenda" GPFS Overview" Classical versus GSS I/O Solution" GPFS Storage Server (GSS)" GPFS Native RAID (GNR)" Declustered RAID" Lemanicus Blue Gene/Q GSS Setup" Conclusion" Questions"
3 GPFS Overview" IBM General Parallel File System" Shared-disk (NSD), clustered file system" POSIX file system" High-performance scalable parallel file system" Concurrent file access to applications executing on multiple nodes of the cluster" File data striped across all disks in file system" Distributed locking mechanism" Distributed metadata" Fault tolerance and replication mechanism" Extensible: adding/removing disks on-the-fly "
4 Classical versus GSS I/O Solution" Classical GPFS solution BG/Q IB IB FC Meta data Controllers SAS Lemanicus BG/Q rack I/O Nodes QDR Infiniband switch GPFS NSD servers FC Data SAS SAN Storage Sub-Systems GPFS Storage Server (GSS) solution BG/Q IB IB SAS SAS Lemanicus BG/Q rack I/O Nodes QDR Infiniband switch GPFS GSS servers SAS
5 GFPS Storage Server (GSS)" GSS is a new IBM System-x solution based on the GPFS Native Raid (GNR) software" GSS was officially announced at SC 12" Lemanicus BG/Q is the first system in the world integrating this solution " GSS hardware consists of storage servers directly connected to enclosures with disks" " " "No storage controller needed" The GSS servers perform both" traditional NSD service, and" disk controller functions (GNR software layer)"
6 GSS Architecture" GPFS Clients GPFS G LAN (IB) NSD Servers NSD Servers Virtual disks GNR Virtual disks GSS SAS JBODs SAS
7 GPFS Native RAID (GNR)!
8 GNR Features" GPFS software layer enabling a GSS server to provide storage controller services" Software-based error correction codes (ECCs)" Multiple data redundancy schemes" Declustered RAID" Reduces disk rebuild overhead" Allows flexible virtual disk layout" End-to-end checksums" Prevents silent data corruption" Data caching"
9 GNR Features (cont d)" Disk hospital" Asynchronous disk fault diagnostics" Live disk replacement" Background disk scrubbing" Failover mechanisms" Seamlessly recovers from hardware failure" Uniform and familiar command interface" Standard GPFS command syntax, including commands for maintaining and replacing failed disks"
10 GNR Physical disks (pdisks)" JBOD disk enclosure" NL-SAS 3TB disks (in BG/Q Lemanicus config)" Dual-ported disks" Each pdisk is physically connected to two servers:" A primary GSS Server, and" A backup GSS server" GNR manages devices which map 1-to-1 to the SCSI disk devices" GNR provides disk failover" No multipathing drivers needed (no RDAC, no Device Mapper)"
11 Physical disks in GSS26 Building Block" Server JBOD Enclosures Server SSD SSD SSD SSD GSS Server N01 SSD SSD GSS Server N SAS 6Gb/s links pdisks (3 TB NL-SAS disks) SAS 6Gb/s links
12 GNR Recovery Groups (RGs)" Disks of a GSS building block are divided into two non-overlapping RGs" The RGs define the LUNs for primary and backup GSS servers" Accesses to disks are made through the active server of the RG" RG failover relies on inherent NSD failover capabilities of GPFS" when primary server fails, backup server automatically becomes the active server of the RG"
13 Recovery Groups in GSS26 Building Block" Primary server for RG01 Backup server for RG02 Two RGs containing 174 pdisks each RG01 RG02 Primary server for RG02 Backup server for RG01 SSD SSD SSD SSD GSS Server N01 SSD SSD GSS Server N
14 GNR Declustered Arrays (DAs)" A DA is a subset of pdisks within a RG" A DA can hold a number of virtual disks (vdisks) containing data and/or meta-data" Each vdisk has a specific RAID code and block size" Each vdisk is seen as a GPFS NSD (building block for File Systems)" Spare space for the DA is allocated within the DA"
15 Declustered Arrays in GSS26 Building Block" Six DAs containing 58 pdisks each (Lemanicus config) RG01 RG02 SSD SSD DA1 DA1 SSD SSD GSS Server N01 SSD SSD DA2 DA GSS Server N DA3 DA
16 GNR Fault Tolerant ECCs" 2-fault tolerant error correction codes (ECCs)" 8+2p Reed-Solomon code" 3-way replication" 8 data strips = GPFS Block 2 parity strips 3-fault tolerant error correction codes (ECCs)" 8+3p Reed-Solomon code" 4-way replication"
17 GNR Uniform and familiar interface" Command examples! List information about GNR Recovery Groups" mmlsrecoverygroup" List information for one or more GNR vdisks" mmlsvdisk" Check and replace a physical disk" mmlspdisk all --not-ok" mmchcarrier gss2_l --pdisk "e1d1s04" release" => Replace disk " mmchcarrier gss2_l --replace --pdisk e1d1s04 "
18 Declustered RAID!
19 Declustered RAID Motivation" Disk rebuilding becomes a critical issue in large scale storage systems" With disks and an MTBF disk of 600 Khrs, a disk rebuild is triggered every day" 24-hour disk rebuild time implies permanent rebuilding activity" As GPFS distributes (stripes) file data over multiple LUNs to increase parallel performance, a single slower LUN impedes performance of the entire file system" Rebuild time increases with larger disks" Risk of (too) many disk failures in same RAID increases"
20 Declustered RAID How it works" Example: conventional RAID 6 (3+P+Q)" data strip array (GPFS NSD) stripe 5 RAID stripes data plus parity 5 physical disks (pdisks)
21 Declustered RAID How it works" Example: conventional RAID 6" 21 disks, 4 x RAID 6 (3+P+Q), 1 x hot spare" array 1 array 2 array 3 array 4 hot spare rebuild time failed disk rebuild Only 5 disks participate to the rebuild operation
22 Declustered RAID How it works" How to improve (speed up) rebuild operation?" Distribute arrays across all pdisks" => Declustered Array" Data strips from any virtual vdisk (3+P+Q) are distributed across different pdisks (example too small!)"
23 Declustered RAID How it works" Rebuild" failed disk R R R R R rebuild time All disks participate to the rebuild operation
24 Declustered RAID Rebuild time" Conventional RAID" Typical disk rebuild time is 24 hours" Once a failed disk has been replaced, data has to be copied back from the hot spare to the new disk" GSS26 Building block" Assuming 58 x 3 TB disks DAs with 8+2p ECC: " Normal rebuild time for a 3 TB disk =~ 3 hours" Occurs as a background task not to interfere with regular disk access workload" Critical rebuild time =~ 15 minutes" A stripe is considered critical if one more disk failure would result in data loss"
25 Declustered RAID Flexibility" Conventional RAID RAID groups are mapped to specific disks Fixed number of disks in arrays Performance constrained by limited number of disks Unused slots hot spare 8+P+Q RAID 6 8+P+Q RAID 6 8+P+Q RAID 6 8+P+Q RAID 6 2 x RAID 10 Declustered Array Vdisks are uniformly distributed over all pdisks Vdisk0 (8+2p) - dataonly Vdisk1 (8+2p) - dataonly Vdisk2 (3-way replication) - metadataonly Spare space
26 Lemanicus Blue Gene/Q GSS Setup!
27 Lemanicus BG/Q GSS setup" GSS Server - Standard IBM System-x - x3650 M4 - E-2670, 2x8 cores, 2.6 GHz" - 64 GB memory x 2 x 3 x 6 JBOD disk enclosure 5 x drawers 58 x 3 TB NL-SAS disks GSS26 Building Block 2 x servers (x3650 M4) 6 x JBOD enclosures 348 x 3 TB NL-SAS raw capacity = 972 TiB usable capacity ~ 726 TiB Max streaming rate 10 GB/s Lemanicus BG/Q config 3 x GSS26 Building Blocks 1044 x 3 TB NL-SAS raw capacity = 2916 TiB usable capacity ~ 2178 TiB Max streaming rate 30 GB/s
28 Lemanicus BG/Q File Systems"
29 Lemanicus BG/Q I/O Performance" IOR Benchmark" Test cases" IOR running on 1024 nodes (on the entire BG/Q rack)" In each test case, the aggregate filesize is 2048 GiB" Transfer size is 16 MiB" IOR test cases 1024 MPI processes ranks-per-node 1 blocksize = 2 GiB MPI processes ranks-per-node 16 blocksize = 128 MiB IOR POSIX file-per-process / MiB/s / MiB/s IOR POSIX single-shared-file / MiB/s / MiB/s IOR MPIIO file-per-process / MiB/s / MiB/s IOR MPIIO single-shared-file / MiB/s / MiB/s
30 Conclusion" GSS/GNR advantages" Lower TCO" No external hardware controller required" One end-to-end technology IBM GPFS from the compute nodes to the disks" Easier maintenance" no training required for third-party storage system" GPFS standard command interface" standard RedHat Linux on GSS servers" Easier end-to-end I/O tuning" no Device-Mapper Multipath" nothing to tune at the back-end storage level"
31 Conclusion" GSS/GNR advantages (cont d)" Flexible vdisk and file system configuration" Any number of vdisks of different sizes, ECCs and block sizes can be defined, and integrated in FS" Improved data integrity" 2- and 3-fault tolerant ECCs" End-to-end checksum" Faster rebuild times using declustered RAIDs" GSS/GNR drawback! All file systems share disk bandwidth"
32 Conclusion" GSS is a cost-effective storage solution for largescale HPC systems" We are experiencing excellent performance for typical HPC I/O workload, such as massively parallel checkpoint/restart" The solution proved to be stable enough in a production environment " In our opinion, there is a bright future for this technology in the HPC world "
33 Questions?!
IBM System x GPFS Storage Server
IBM System x GPFS Storage Server Schöne Aussicht en für HPC Speicher ZKI-Arbeitskreis Paderborn, 15.03.2013 Karsten Kutzer Client Technical Architect Technical Computing IBM Systems & Technology Group
IBM System x GPFS Storage Server
IBM System x GPFS Storage Crispin Keable Technical Computing Architect 1 IBM Technical Computing comprehensive portfolio uniquely addresses supercomputing and mainstream client needs Technical Computing
IBM General Parallel File System (GPFS ) 3.5 File Placement Optimizer (FPO)
IBM General Parallel File System (GPFS ) 3.5 File Placement Optimizer (FPO) Rick Koopman IBM Technical Computing Business Development Benelux Rick_koopman@nl.ibm.com Enterprise class replacement for HDFS
General Parallel File System (GPFS) Native RAID For 100,000-Disk Petascale Systems
General Parallel File System (GPFS) Native RAID For 100,000-Disk Petascale Systems Veera Deenadhayalan IBM Almaden Research Center 2011 IBM Corporation Hard Disk Rates Are Lagging There have been recent
Comparing Dynamic Disk Pools (DDP) with RAID-6 using IOR
Comparing Dynamic Disk Pools (DDP) with RAID-6 using IOR December, 2012 Peter McGonigal petermc@sgi.com Abstract Dynamic Disk Pools (DDP) offer an exciting new approach to traditional RAID sets by substantially
IBM Spectrum Scale vs EMC Isilon for IBM Spectrum Protect Workloads
89 Fifth Avenue, 7th Floor New York, NY 10003 www.theedison.com @EdisonGroupInc 212.367.7400 IBM Spectrum Scale vs EMC Isilon for IBM Spectrum Protect Workloads A Competitive Test and Evaluation Report
File System & Device Drive. Overview of Mass Storage Structure. Moving head Disk Mechanism. HDD Pictures 11/13/2014. CS341: Operating System
CS341: Operating System Lect 36: 1 st Nov 2014 Dr. A. Sahu Dept of Comp. Sc. & Engg. Indian Institute of Technology Guwahati File System & Device Drive Mass Storage Disk Structure Disk Arm Scheduling RAID
PADS GPFS Filesystem: Crash Root Cause Analysis. Computation Institute
PADS GPFS Filesystem: Crash Root Cause Analysis Computation Institute Argonne National Laboratory Table of Contents Purpose 1 Terminology 2 Infrastructure 4 Timeline of Events 5 Background 5 Corruption
Current Status of FEFS for the K computer
Current Status of FEFS for the K computer Shinji Sumimoto Fujitsu Limited Apr.24 2012 LUG2012@Austin Outline RIKEN and Fujitsu are jointly developing the K computer * Development continues with system
HPC Advisory Council
HPC Advisory Council September 2012, Malaga CHRIS WEEDEN SYSTEMS ENGINEER WHO IS PANASAS? Panasas is a high performance storage vendor founded by Dr Garth Gibson Panasas delivers a fully supported, turnkey,
Data Protection Technologies: What comes after RAID? Vladimir Sapunenko, INFN-CNAF HEPiX Spring 2012 Workshop
Data Protection Technologies: What comes after RAID? Vladimir Sapunenko, INFN-CNAF HEPiX Spring 2012 Workshop Arguments to be discussed Scaling storage for clouds Is RAID dead? Erasure coding as RAID replacement
DELL RAID PRIMER DELL PERC RAID CONTROLLERS. Joe H. Trickey III. Dell Storage RAID Product Marketing. John Seward. Dell Storage RAID Engineering
DELL RAID PRIMER DELL PERC RAID CONTROLLERS Joe H. Trickey III Dell Storage RAID Product Marketing John Seward Dell Storage RAID Engineering http://www.dell.com/content/topics/topic.aspx/global/products/pvaul/top
Storage Architectures for Big Data in the Cloud
Storage Architectures for Big Data in the Cloud Sam Fineberg HP Storage CT Office/ May 2013 Overview Introduction What is big data? Big Data I/O Hadoop/HDFS SAN Distributed FS Cloud Summary Research Areas
NetApp High-Performance Computing Solution for Lustre: Solution Guide
Technical Report NetApp High-Performance Computing Solution for Lustre: Solution Guide Robert Lai, NetApp August 2012 TR-3997 TABLE OF CONTENTS 1 Introduction... 5 1.1 NetApp HPC Solution for Lustre Introduction...5
Pivot3 Desktop Virtualization Appliances. vstac VDI Technology Overview
Pivot3 Desktop Virtualization Appliances vstac VDI Technology Overview February 2012 Pivot3 Desktop Virtualization Technology Overview Table of Contents Executive Summary... 3 The Pivot3 VDI Appliance...
Infortrend ESVA Family Enterprise Scalable Virtualized Architecture
Infortrend ESVA Family Enterprise Scalable Virtualized Architecture R Optimized ROI Ensures the most efficient allocation of consolidated capacity and computing power, and meets wide array of service level
Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com
Parallels Cloud Storage White Paper Performance Benchmark Results www.parallels.com Table of Contents Executive Summary... 3 Architecture Overview... 3 Key Features... 4 No Special Hardware Requirements...
New Storage System Solutions
New Storage System Solutions Craig Prescott Research Computing May 2, 2013 Outline } Existing storage systems } Requirements and Solutions } Lustre } /scratch/lfs } Questions? Existing Storage Systems
Click on the diagram to see RAID 0 in action
Click on the diagram to see RAID 0 in action RAID Level 0 requires a minimum of 2 drives to implement RAID 0 implements a striped disk array, the data is broken down into blocks and each block is written
Driving IBM BigInsights Performance Over GPFS Using InfiniBand+RDMA
WHITE PAPER April 2014 Driving IBM BigInsights Performance Over GPFS Using InfiniBand+RDMA Executive Summary...1 Background...2 File Systems Architecture...2 Network Architecture...3 IBM BigInsights...5
Big data management with IBM General Parallel File System
Big data management with IBM General Parallel File System Optimize storage management and boost your return on investment Highlights Handles the explosive growth of structured and unstructured data Offers
Sonexion GridRAID Characteristics
Sonexion GridRAID Characteristics Mark Swan Performance Team Cray Inc. Saint Paul, Minnesota, USA mswan@cray.com Abstract This paper will present performance characteristics of the Sonexion declustered
Introduction to Gluster. Versions 3.0.x
Introduction to Gluster Versions 3.0.x Table of Contents Table of Contents... 2 Overview... 3 Gluster File System... 3 Gluster Storage Platform... 3 No metadata with the Elastic Hash Algorithm... 4 A Gluster
Enhancements of ETERNUS DX / SF
shaping tomorrow with you ETERNUS - Business-centric Storage Enhancements of ETERNUS DX / SF Global Product Marketing Storage ETERNUS Business-centric Storage Agenda: 1 Overview of the top 3 innovations
Architecting a High Performance Storage System
WHITE PAPER Intel Enterprise Edition for Lustre* Software High Performance Data Division Architecting a High Performance Storage System January 2014 Contents Introduction... 1 A Systematic Approach to
Getting performance & scalability on standard platforms, the Object vs Block storage debate. Copyright 2013 MPSTOR LTD. All rights reserved.
Getting performance & scalability on standard platforms, the Object vs Block storage debate 1 December Webinar Session Getting performance & scalability on standard platforms, the Object vs Block storage
Moving Virtual Storage to the Cloud. Guidelines for Hosters Who Want to Enhance Their Cloud Offerings with Cloud Storage
Moving Virtual Storage to the Cloud Guidelines for Hosters Who Want to Enhance Their Cloud Offerings with Cloud Storage Table of Contents Overview... 1 Understanding the Storage Problem... 1 What Makes
Clusters: Mainstream Technology for CAE
Clusters: Mainstream Technology for CAE Alanna Dwyer HPC Division, HP Linux and Clusters Sparked a Revolution in High Performance Computing! Supercomputing performance now affordable and accessible Linux
Violin Memory Arrays With IBM System Storage SAN Volume Control
Technical White Paper Report Best Practices Guide: Violin Memory Arrays With IBM System Storage SAN Volume Control Implementation Best Practices and Performance Considerations Version 1.0 Abstract This
Object storage in Cloud Computing and Embedded Processing
Object storage in Cloud Computing and Embedded Processing Jan Jitze Krol Systems Engineer DDN We Accelerate Information Insight DDN is a Leader in Massively Scalable Platforms and Solutions for Big Data
NetApp Software. SANtricity Storage Manager Concepts for Version 11.10. NetApp, Inc. 495 East Java Drive Sunnyvale, CA 94089 U.S.
NetApp Software SANtricity Storage Manager Concepts for Version 11.10 NetApp, Inc. 495 East Java Drive Sunnyvale, CA 94089 U.S. Telephone: +1 (408) 822-6000 Fax: +1 (408) 822-4501 Support telephone: +1
systems' resilience to disk failure through
BY Shashwath Veerappa Devaru CS615 Aspects of System Administration Using Multiple Hard Drives for Performance and Reliability RAID is the term used to describe a storage systems' resilience to disk failure
Using EonStor FC-host Storage Systems in VMware Infrastructure 3 and vsphere 4
Using EonStor FC-host Storage Systems in VMware Infrastructure 3 and vsphere 4 Application Note Abstract This application note explains the configure details of using Infortrend FC-host storage systems
Performance, Reliability, and Operational Issues for High Performance NAS Storage on Cray Platforms. Cray User Group Meeting June 2007
Performance, Reliability, and Operational Issues for High Performance NAS Storage on Cray Platforms Cray User Group Meeting June 2007 Cray s Storage Strategy Background Broad range of HPC requirements
SMB Direct for SQL Server and Private Cloud
SMB Direct for SQL Server and Private Cloud Increased Performance, Higher Scalability and Extreme Resiliency June, 2014 Mellanox Overview Ticker: MLNX Leading provider of high-throughput, low-latency server
Agenda. Enterprise Application Performance Factors. Current form of Enterprise Applications. Factors to Application Performance.
Agenda Enterprise Performance Factors Overall Enterprise Performance Factors Best Practice for generic Enterprise Best Practice for 3-tiers Enterprise Hardware Load Balancer Basic Unix Tuning Performance
PARALLELS CLOUD STORAGE
PARALLELS CLOUD STORAGE Performance Benchmark Results 1 Table of Contents Executive Summary... Error! Bookmark not defined. Architecture Overview... 3 Key Features... 5 No Special Hardware Requirements...
Cray DVS: Data Virtualization Service
Cray : Data Virtualization Service Stephen Sugiyama and David Wallace, Cray Inc. ABSTRACT: Cray, the Cray Data Virtualization Service, is a new capability being added to the XT software environment with
BlueArc unified network storage systems 7th TF-Storage Meeting. Scale Bigger, Store Smarter, Accelerate Everything
BlueArc unified network storage systems 7th TF-Storage Meeting Scale Bigger, Store Smarter, Accelerate Everything BlueArc s Heritage Private Company, founded in 1998 Headquarters in San Jose, CA Highest
Best Practices for Data Sharing in a Grid Distributed SAS Environment. Updated July 2010
Best Practices for Data Sharing in a Grid Distributed SAS Environment Updated July 2010 B E S T P R A C T I C E D O C U M E N T Table of Contents 1 Abstract... 2 1.1 Storage performance is critical...
Moving Virtual Storage to the Cloud
Moving Virtual Storage to the Cloud White Paper Guidelines for Hosters Who Want to Enhance Their Cloud Offerings with Cloud Storage www.parallels.com Table of Contents Overview... 3 Understanding the Storage
Comparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet. September 2014
Comparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet Anand Rangaswamy September 2014 Storage Developer Conference Mellanox Overview Ticker: MLNX Leading provider of high-throughput,
Performance Analysis of RAIDs in Storage Area Network
Performance Analysis of RAIDs in Storage Area Network Sneha M. Assistant Professor, Department of Computer Science and Engineering, R V College of Engineering Bengaluru-560059 ABSTRACT Direct Attached
SAN Conceptual and Design Basics
TECHNICAL NOTE VMware Infrastructure 3 SAN Conceptual and Design Basics VMware ESX Server can be used in conjunction with a SAN (storage area network), a specialized high speed network that connects computer
WHITEPAPER: Understanding Pillar Axiom Data Protection Options
WHITEPAPER: Understanding Pillar Axiom Data Protection Options Introduction This document gives an overview of the Pillar Data System Axiom RAID protection schemas. It does not delve into corner cases
VTrak 15200 SATA RAID Storage System
Page 1 15-Drive Supports over 5 TB of reliable, low-cost, high performance storage 15200 Product Highlights First to deliver a full HW iscsi solution with SATA drives - Lower CPU utilization - Higher data
Reliability and Fault Tolerance in Storage
Reliability and Fault Tolerance in Storage Dalit Naor/ Dima Sotnikov IBM Haifa Research Storage Systems 1 Advanced Topics on Storage Systems - Spring 2014, Tel-Aviv University http://www.eng.tau.ac.il/semcom
Netapp HPC Solution for Lustre. Rich Fenton (fenton@netapp.com) UK Solutions Architect
Netapp HPC Solution for Lustre Rich Fenton (fenton@netapp.com) UK Solutions Architect Agenda NetApp Introduction Introducing the E-Series Platform Why E-Series for Lustre? Modular Scale-out Capacity Density
HP Smart Array 5i Plus Controller and Battery Backed Write Cache (BBWC) Enabler
Overview HP Smart Array 5i Plus Controller and Battery Backed Write Cache (BBWC) Enabler Models Smart Array 5i Plus Controller and BBWC Enabler bundled Option Kit (for ProLiant DL380 G2, ProLiant DL380
Understanding Microsoft Storage Spaces
S T O R A G E Understanding Microsoft Storage Spaces A critical look at its key features and value proposition for storage administrators A Microsoft s Storage Spaces solution offers storage administrators
Large Scale Storage. Orlando Richards, Information Services orlando.richards@ed.ac.uk. LCFG Users Day, University of Edinburgh 18 th January 2013
Large Scale Storage Orlando Richards, Information Services orlando.richards@ed.ac.uk LCFG Users Day, University of Edinburgh 18 th January 2013 Overview My history of storage services What is (and is not)
QuickSpecs. HP Smart Array 5312 Controller. Overview
Overview Models 238633-B21 238633-291 (Japan) Feature List: High Performance PCI-X Architecture High Capacity Two Ultra 3 SCSI channels support up to 28 drives Modular battery-backed cache design 128 MB
Dynamically unify your data center Dell Compellent: Self-optimized, intelligently tiered storage
Dell Fluid Data architecture Dynamically unify your data center Dell Compellent: Self-optimized, intelligently tiered storage Dell believes that storage should help you spend less while giving you the
Testing Performed at: Clemson Center of Excellence in Next Generation Computing Evaluation & Usability Labs October 2013
OrangeFS DDN SFA12K Architecture Testing Performed at: Clemson Center of Excellence in Next Generation Computing Evaluation & Usability Labs October 2013 THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES
Blueprints for Scalable IBM Spectrum Protect (TSM) Disk-based Backup Solutions
Blueprints for Scalable IBM Spectrum Protect (TSM) Disk-based Backup Solutions Jason Basler Software Test Architect IBM Technical University/Symposia materials may not be reproduced in whole or in part
Highly-Available Distributed Storage. UF HPC Center Research Computing University of Florida
Highly-Available Distributed Storage UF HPC Center Research Computing University of Florida Storage is Boring Slow, troublesome, albatross around the neck of high-performance computing UF Research Computing
technology brief RAID Levels March 1997 Introduction Characteristics of RAID Levels
technology brief RAID Levels March 1997 Introduction RAID is an acronym for Redundant Array of Independent Disks (originally Redundant Array of Inexpensive Disks) coined in a 1987 University of California
Why disk arrays? CPUs improving faster than disks
Why disk arrays? CPUs improving faster than disks - disks will increasingly be bottleneck New applications (audio/video) require big files (motivation for XFS) Disk arrays - make one logical disk out of
Intel RAID Controllers
Intel RAID Controllers Best Practices White Paper April, 2008 Enterprise Platforms and Services Division - Marketing Revision History Date Revision Number April, 2008 1.0 Initial release. Modifications
The Panasas Parallel Storage Cluster. Acknowledgement: Some of the material presented is under copyright by Panasas Inc.
The Panasas Parallel Storage Cluster What Is It? What Is The Panasas ActiveScale Storage Cluster A complete hardware and software storage solution Implements An Asynchronous, Parallel, Object-based, POSIX
Software-defined Storage Architecture for Analytics Computing
Software-defined Storage Architecture for Analytics Computing Arati Joshi Performance Engineering Colin Eldridge File System Engineering Carlos Carrero Product Management June 2015 Reference Architecture
EMC Backup and Recovery for Microsoft SQL Server 2008 Enabled by EMC Celerra Unified Storage
EMC Backup and Recovery for Microsoft SQL Server 2008 Enabled by EMC Celerra Unified Storage Applied Technology Abstract This white paper describes various backup and recovery solutions available for SQL
IEEE Mass Storage Conference Vendor Reception Lake Tahoe, NV
IEEE Mass Storage Conference Vendor Reception Lake Tahoe, NV 11 Manager May 04, 2010 Joe Rotiroti Client Systems IBM, Federal 484 433 9756 cell 845 491 5227 fax rotiroti@us.ibm.com How is GPFS different?
Using HP StoreOnce Backup Systems for NDMP backups with Symantec NetBackup
Technical white paper Using HP StoreOnce Backup Systems for NDMP backups with Symantec NetBackup Table of contents Executive summary... 2 Introduction... 2 What is NDMP?... 2 Technology overview... 3 HP
IBM System Storage DS5020 Express
IBM DS5020 Express Manage growth, complexity, and risk with scalable, high-performance storage Highlights Mixed host interfaces support (Fibre Channel/iSCSI) enables SAN tiering Balanced performance well-suited
ntier Verde Simply Affordable File Storage
ntier Verde Simply Affordable File Storage Current Market Problems Data Growth Continues Data Retention Increases By 2020 the Digital Universe will hold 40 Zettabytes The Market is Missing: An easy to
How to Choose your Red Hat Enterprise Linux Filesystem
How to Choose your Red Hat Enterprise Linux Filesystem EXECUTIVE SUMMARY Choosing the Red Hat Enterprise Linux filesystem that is appropriate for your application is often a non-trivial decision due to
Picking the right number of targets per server for BeeGFS. Jan Heichler March 2015 v1.2
Picking the right number of targets per server for BeeGFS Jan Heichler March 2015 v1.2 Evaluating the MetaData Performance of BeeGFS 2 Abstract In this paper we will show the performance of two different
Scala Storage Scale-Out Clustered Storage White Paper
White Paper Scala Storage Scale-Out Clustered Storage White Paper Chapter 1 Introduction... 3 Capacity - Explosive Growth of Unstructured Data... 3 Performance - Cluster Computing... 3 Chapter 2 Current
EonStor DS 3000 Series
EonStor DS 000 Series High Speed Flexible Storage Delivers Robust Reliability and Fast Performance Up to.m IOPS throughput handles the most demanding applications Seamless SSD support The EonStor DS 000
Software-defined Storage at the Speed of Flash
TECHNICAL BRIEF: SOFTWARE-DEFINED STORAGE AT THE SPEED OF... FLASH..................................... Intel SSD Data Center P3700 Series and Symantec Storage Foundation with Flexible Storage Sharing
Direct NFS - Design considerations for next-gen NAS appliances optimized for database workloads Akshay Shah Gurmeet Goindi Oracle
Direct NFS - Design considerations for next-gen NAS appliances optimized for database workloads Akshay Shah Gurmeet Goindi Oracle Agenda Introduction Database Architecture Direct NFS Client NFS Server
The functionality and advantages of a high-availability file server system
The functionality and advantages of a high-availability file server system This paper discusses the benefits of deploying a JMR SHARE High-Availability File Server System. Hardware and performance considerations
Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre
Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre University of Cambridge, UIS, HPC Service Authors: Wojciech Turek, Paul Calleja, John Taylor
Worry-free Storage. E-Series Simple SAN Storage
Worry-free Storage E-Series Simple SAN Storage 1 What are Midsize Businesses Struggling with Today? #1 pain point: managing data growth (62%)* #2 pain point: managing costs (33%)* 52% view poor storage
EonStor DS 3000 Series
EonStor DS 000 Series High Speed Flexible Storage Delivers Robust Reliability and Fast Performance Up to.m IOPS throughput handles the most demanding applications Seamless SSD support Flexibility Choose
Hyper-V over SMB: Remote File Storage Support in Windows Server 2012 Hyper-V. Jose Barreto Principal Program Manager Microsoft Corporation
Hyper-V over SMB: Remote Storage Support in Windows Server 2012 Hyper-V Jose Barreto Principal Program Manager Microsoft Corporation Abstract In this session, we cover the Windows Server 2012 Hyper-V support
Panasas at the RCF. Fall 2005 Robert Petkus RHIC/USATLAS Computing Facility Brookhaven National Laboratory. Robert Petkus Panasas at the RCF
Panasas at the RCF HEPiX at SLAC Fall 2005 Robert Petkus RHIC/USATLAS Computing Facility Brookhaven National Laboratory Centralized File Service Single, facility-wide namespace for files. Uniform, facility-wide
An Alternative Storage Solution for MapReduce. Eric Lomascolo Director, Solutions Marketing
An Alternative Storage Solution for MapReduce Eric Lomascolo Director, Solutions Marketing MapReduce Breaks the Problem Down Data Analysis Distributes processing work (Map) across compute nodes and accumulates
OPTIMIZING EXCHANGE SERVER IN A TIERED STORAGE ENVIRONMENT WHITE PAPER NOVEMBER 2006
OPTIMIZING EXCHANGE SERVER IN A TIERED STORAGE ENVIRONMENT WHITE PAPER NOVEMBER 2006 EXECUTIVE SUMMARY Microsoft Exchange Server is a disk-intensive application that requires high speed storage to deliver
Overview of I/O Performance and RAID in an RDBMS Environment. By: Edward Whalen Performance Tuning Corporation
Overview of I/O Performance and RAID in an RDBMS Environment By: Edward Whalen Performance Tuning Corporation Abstract This paper covers the fundamentals of I/O topics and an overview of RAID levels commonly
Parallel IO. Single namespace. Performance. Disk locality awareness? Data integrity. Fault tolerance. Standard interface. Network of disks?
PARALLEL IO Parallel IO Single namespace Network of disks? Performance Data replication Multiple I/O paths Disk locality awareness? Data integrity Multiple writers Locking? Fault tolerance Hardware failure
FlexArray Virtualization
Updated for 8.2.1 FlexArray Virtualization Installation Requirements and Reference Guide NetApp, Inc. 495 East Java Drive Sunnyvale, CA 94089 U.S. Telephone: +1 (408) 822-6000 Fax: +1 (408) 822-4501 Support
SAS Analytics on IBM FlashSystem storage: Deployment scenarios and best practices
Paper 3290-2015 SAS Analytics on IBM FlashSystem storage: Deployment scenarios and best practices ABSTRACT Harry Seifert, IBM Corporation; Matt Key, IBM Corporation; Narayana Pattipati, IBM Corporation;
High Availability Databases based on Oracle 10g RAC on Linux
High Availability Databases based on Oracle 10g RAC on Linux WLCG Tier2 Tutorials, CERN, June 2006 Luca Canali, CERN IT Outline Goals Architecture of an HA DB Service Deployment at the CERN Physics Database
HADOOP ON ORACLE ZFS STORAGE A TECHNICAL OVERVIEW
HADOOP ON ORACLE ZFS STORAGE A TECHNICAL OVERVIEW 757 Maleta Lane, Suite 201 Castle Rock, CO 80108 Brett Weninger, Managing Director brett.weninger@adurant.com Dave Smelker, Managing Principal dave.smelker@adurant.com
Education. Servicing the IBM ServeRAID-8k Serial- Attached SCSI Controller. Study guide. XW5078 Release 1.00
Education Servicing the IBM ServeRAID-8k Serial- Attached SCSI Controller Study guide XW5078 Release 1.00 May 2006 Servicing the IBM ServeRAID-8k Serial- Attached SCSI (SAS) Controller Overview The ServeRAID
IBM ^ xseries ServeRAID Technology
IBM ^ xseries ServeRAID Technology Reliability through RAID technology Executive Summary: t long ago, business-critical computing on industry-standard platforms was unheard of. Proprietary systems were
June 2009. Blade.org 2009 ALL RIGHTS RESERVED
Contributions for this vendor neutral technology paper have been provided by Blade.org members including NetApp, BLADE Network Technologies, and Double-Take Software. June 2009 Blade.org 2009 ALL RIGHTS
The Revival of Direct Attached Storage for Oracle Databases
The Revival of Direct Attached Storage for Oracle Databases Revival of DAS in the IT Infrastructure Introduction Why is it that the industry needed SANs to get more than a few hundred disks attached to
Mass-Storage Devices: Disks. CSCI 5103 Operating Systems. Basic Disk Functionality. Magnetic Disks
Mass-Storage Devices: Disks CSCI 5103 Operating Systems Instructor: Abhishek Chandra Disk Structure and Attachment Disk Scheduling Disk Management RAID Structure Stable Storage 2 Magnetic Disks Most common
LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance
11 th International LS-DYNA Users Conference Session # LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance Gilad Shainer 1, Tong Liu 2, Jeff Layton 3, Onur Celebioglu
High Performance NAS for Hadoop
High Performance NAS for Hadoop HPC ADVISORY COUNCIL, STANFORD FEB 8, 2013 DR. BRENT WELCH, CTO, PANASAS Panasas and Hadoop PANASAS TECHNICAL DIFFERENTIATION Scalable Performance Balanced object-storage
3 Red Hat Enterprise Linux 6 Consolidation
Whitepaper Consolidation EXECUTIVE SUMMARY At this time of massive and disruptive technological changes where applications must be nimbly deployed on physical, virtual, and cloud infrastructure, Red Hat
Netapp @ 10th TF-Storage Meeting
Netapp @ 10th TF-Storage Meeting Wojciech Janusz, Netapp Poland Bogusz Błaszkiewicz, Netapp Poland Ljubljana, 2012.02.20 Agenda Data Ontap Cluster-Mode pnfs E-Series NetApp Confidential - Internal Use
IBM ELASTIC STORAGE SEAN LEE
IBM ELASTIC STORAGE SEAN LEE Solution Architect Platform Computing Division IBM Greater China Group Agenda Challenges in Data Management What is IBM Elastic Storage Key Features Elastic Storage Server
The safer, easier way to help you pass any IT exams. Exam : 000-115. Storage Sales V2. Title : Version : Demo 1 / 5
Exam : 000-115 Title : Storage Sales V2 Version : Demo 1 / 5 1.The IBM TS7680 ProtecTIER Deduplication Gateway for System z solution is designed to provide all of the following EXCEPT: A. ESCON attach
Lessons learned from parallel file system operation
Lessons learned from parallel file system operation Roland Laifer STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association
Module 6. RAID and Expansion Devices
Module 6 RAID and Expansion Devices Objectives 1. PC Hardware A.1.5 Compare and contrast RAID types B.1.8 Compare expansion devices 2 RAID 3 RAID 1. Redundant Array of Independent (or Inexpensive) Disks
Why disk arrays? CPUs speeds increase faster than disks. - Time won t really help workloads where disk in bottleneck
1/19 Why disk arrays? CPUs speeds increase faster than disks - Time won t really help workloads where disk in bottleneck Some applications (audio/video) require big files Disk arrays - make one logical