Distributed RAID Architectures for Cluster I/O Computing. Kai Hwang
|
|
- Ashley Griffith
- 8 years ago
- Views:
Transcription
1 Distributed RAID Architectures for Cluster I/O Computing Kai Hwang Internet and Cluster Computing Lab. University of Southern California 1
2 Presentation Outline : Scalable Cluster I/O The RAID-x Architecture Cooperative disk drivers Benchmark Experiments Security and Fault Tolerance Conclusions
3 Scalable clusters providing SSI services are gradually replacing the SMP, cc-numa, and MPP in Servers, Web Sites, and Database Centers K. Hwang, March 15,2001 in Beijing 3
4 g Issues in Cluster Design Size Scalability (physical & application) g Enhanced Availability (failure management) g Single System Image (Middleware,OS extensions) g Fast Communication (networks & protocols) g Load Balancing (CPU, Net, Memory, Disk) g Security and Encryption (clusters of clusters) g Distributed Environment (User friendly) g Manageability (Jobs and resources ) g Programmability (simple API required) g Applicability (cluster- and grid-awareness) K. Hwang, March 15,2001 in Beijing 4
5 The USC Trojans Cluster Project Internet and Cluster Computing Lab. EEB Rm.104 g Sixteen Pentinum PCs are housed in two 9-ft computer racks. g All PCs run with the RedHat Linux v. 6.0 (Kernel v ) g All nodes are connected by a 100 Mbps Fast Ethernet g g The cluster is ported with DQS, LSF, MPI, PVM, TreadMarks, Elias, and NAS benchmarks, etc. Scalable to a future system with 100 s of future PC nodes interconnected by Gigabit networks K. Hwang, March 15,2001 in Beijing 5
6 Trojans Linux Cluster with Middleware for Security and Checkpoint Recovery Programming Environments (Java, EDI, HTML, XML) Web Windows User Interface Other Subsystems (Database, OLTP, etc.) Single-System Image and Availability Infrastructure Security and Checkpointing Middleware Linux Linux Linux Pentium PC Pentium PC Pentium PC Gigabit Network Interconnect K. Hwang, March 15,2001 in Beijing 6
7 An I/O-centric cluster architecture Entry Partition Client Internet/Intranet Database Partition Fast Ethernet Service Partition Service Flow Data Flow 4
8 Distributed RAID Embedded in Clusters or Storage-Area Networks: g I/O Bottleneck in Scalable Cluster Computing n The gap between CPU/Memory and disk-io widens as the mp doubles in speed every year n Cluster applications are often I/O-bound g Disks connected to hosts are often subject to failure by hosts themselves. Distributed RAID has much higher availability by fault isolation, rollback recovery, and automatic file migration. K. Hwang, March 15,2001 in Beijing 8
9 Distributed RAID with a single I/O space embedded in a cluster Cluster Network (SAN or LAN) Workstations or PCs ds-raid
10 Research Projects on Parallel and Distributed RAID System Attributes RAID Architecture environment Enabling Mechanism for SIOS Data Consistency Checking Reliability and Fault Tolerance USC Trojans RAID-x Orthogonal striping and mirroring in a Linux cluster Cooperative device drivers in Linux kernel Locks at device driver level Orthogonal striping and mirroring Princeton TickerTAIP RAID-5 with multiple controllers Single RAID server implementation Sequencing of user requests Parity checks in RAID-5 Digital Petal Chained Declustering in Unix cluster Petal device drivers at user level Lamport s Paxos algorithm Chained Declustering Berkeley Tertiary Disk RAID-5 built with a PC cluster xfs storage servers at file level Modified DASH protocol in the xfs file system SCSI disks with parity in RAID-5 HP AutoRAID Hierarchical with RAID-1 and RAID-5 Disk array within single controller Use mark to update the parity disk Mirroring and parity checks
11 Four RAID architectures using different mirroring and parity checking schemes Disk 1 Disk 2 Disk 3 Disk 4 Disk 1 Disk 2 Disk 3 Disk 4 B 0 B 1 M 0 M 1 B 0 B 1 B 2 P 1 B 2 B 3 M 2 M 3 B 4 B 5 P 2 B 3 B 4 B 5 M 4 M 5 B 8 P 3 B 6 B 7 B 6 B 7 M 6 M 7 P 4 B 9 B 10 B 11 B 8 B 10 B 9 B 11 M 8 M 10 M 9 M 11 B 12 B 16 B 13 B 14 P 5 (a) Striped mirroring in RAID-10 Disk 0 Disk 1 Disk 2 Disk 3 B 17 P 6 B 15 (b) Parity checking in RAID-5 Disk 0 Disk 1 Disk 2 Disk 3 Data blocks B 0 B 1 B 2 B 3 B 4 B 5 B 6 B 7 B 8 B 9 M 6 B 10 B 11 B 0 M 3 B 4 B 1 M 0 B 5 B 2 M 1 B 6 B 3 M 2 B 7 Mirrored blocks M 9 M 10 M 11 M 7 M 8 M 3 M 4 M 5 M 0 M 1 M 2 (c) Orthogonal striping and mirroring (OSM) in the RAID-x M 7 M 4 M 5 M 6 7B 8 B 9 B 10 B 11 M 11 M 8 M 9 M 10 (d) Skewed striping in a chained declustering RAID 6
12 Theoretical Peak Performance of Four RAID Architectures Performance Chained RAID-10 RAID-5 Indicators Declustering RAID-x Read n B n B n B n B Max. I/O Large Write n B (n-1) B n B n B Bandwidth Small Write n B nb / 2 n B n B Large Read mr / n mr / n mr / n mr / n Parallel Small Read R R R R Read or Parallel Large Write 2 mw / n mw / (n-1) 2 mw / n mw / n + Write Time mw / n(n-1) Max. Fault Coverage Small Write 2W R+W 2W W n/2 disk Single disk n/2 disk failures failure failures Single disk failure 7
13 P/M Distributed RAID-x architecture CDD P/M CDD Cluster Network Node 0 Node 1 Node 2 Node 3 B 0 B 12 B 24 M 25 M 26 M 27 B 4 B 16 B 28 M 29 M 30 M 31 B 8 B 20 B 32 M 33 M 34 M 35 D0 B 1 B 13 B 25 M 14 M 15 M 24 P/M CDD P/M CDD D1 D2 D3 B 2 B 14 B 26 M 3 M 12 M 13 D4 D5 D6 D7 B 5 B 17 B 29 M 18 M 19 M 28 B 6 B 18 B 30 M 7 M 16 M 17 D8 D9 D10 D11 B 9 B 21 B 33 M 22 M 23 M 32 B 10 B 22 B 34 M 11 M 20 M 21 B 3 B 15 B 27 M 0 M 1 M 2 B 7 B 19 B 31 M 4 M 5 M 6 B 11 B 23 B 35 M 8 M 9 M 10 8
14 Single I/O space in a Distributed RAID enabled by CDDs at Linux kernel level Interconnection Network Central NFS Server IDD Interconnection Network Cluster node IDD Cluster node IDD Cluster node Cluster node Cluster node CDD CDD CDD (a) Separate disks driven by independent disk drivers (IDDs) (b) A global virtual disk with a SIOS formed by cooperative disks K. Hwang, March 15,2001 in Beijing 14
15 Remote disk access using central NFS server versus using cooperative disk drivers in the RAID-x cluster User Level User Application NFS Server User Level Kernel Level 1 NFS Client Traditional Device Driver Kernel Level Client side Server side (a) Parallel disk I/O using the NFS in a server/client cluster. User Level User Application NFS Server is bypassed User Level Kernel Level 1 CDD CDD 4 3 Kernel Level Client side Server side (b) Using CDDs to achieve a SIOS in a serverless cluster. K. Hwang, March 15,2001 in Beijing 15
16 Architectural design of cooperative device drivers Node 1 CDD Communications through the network Node 2 CDD Cooperative Disk Driver (CDD) Data Consistency Module Storage Manager CDD Client Module Physical disks Virtual disks Communications through the network (a) Device masquerading (b) CDD architecture K. Hwang, March 15,2001 in Beijing 16
17 Maintaining consistency of the global directory /sios by all CDDs in the distributed RAID-x Cluster node 1 Cluster node 2 Cluster node 3 Cluster node 4 Application Application Application Application /sios /sios /sios /sios dir1 dir2 dir1 dir2 dir1 dir2 dir1 dir2 file1 file2 file3 file1 file2 file3 file1 file2 file3 file1 file2 file3 CDD CDD CDD CDD Cluster Network K. Hwang, March 15,2001 in Beijing 17
18 Elapsed Time in Executing the Andrew Benchmark on the Linux Cluster at USC Elapsed Time (sec) Compile Read File Scan Dir Copy Files Make Dir Elapsed Time (sec) Number of Clients Number of Clients NFS results RAID-x results
19 Parallel Write Performance of four RAID Architectures against Traffic Rate 18 Aggregate Bandwidth (MB/s) RAID-x Chained Declustering RAID-10 RAID-5 NFS Number of Clients Parallel writes (20MB per client) 13
20 Parallel Write Performance of four RAID architectures vs. Disk Array Size 18 Aggregate Bandwidth (MB/s) RAID-x Chained Declustering RAID-10 RAID Disk Numbers Parallel write 13
21 Achievable I/O Bandwidth and Improvement Factor on Trojans Cluster I/O NFS RAID-x Operations 1 Client 16 Clients Improve 1 Client 16 Clients Improve Large Read 2.58 MB/s 2.3 MB/s MB/s MB/s 6.03 Large Write 2.11 MB/s 2.77 MB/s MB/s MB/s 5.24 Small Write 2.47 MB/s 2.81 MB/s MB/s 15.1 MB/s 6.43 Operations Chained Declustering RAID-10 1 Client 16 Clients Improve 1 Client 16 Clients Improve Large Read 2.46 MB/s 15.8 MB/s MB/s MB/s 4.54 Large Write 2.62 MB/s MB/s MB/s 9.96 MB/s 4.31 Small Write 2.31 MB/s MB/s MB/s 9.98 MB/s 4.39 K. Hwang, March 15,2001 in Beijing 21
22 Effects of Stripe Unit Size on I/O Bandwidth of RAID Architectures Aggregate Bandwidth (MB/s) RAID-x Chained Declustering RAID-10 RAID Stripe Unit Size (KB) Large write (320MB for 16 clients) 14
23 Bonnie Benchmark Results on Trojans Cluster Output Rate (MB/s) Number of Disks File rewrite RAID-x chained declustering RAID-10 RAID-5
24 Fault tolerance Increasing Reliability No data protection Securing Networks, Intranets, Clusters, or Grid Resources with intrusion control and automatic recovery from malicious attacks Gateway firewall to screen Cluster with no security protection Highly secured Intranet with intrusion detection and response, automatic traffic flow between networks recovery from malicious attacks, and fault-tolerance with distributed storage for reliable I/O SMP cluster Intranet Grid Increasing scalability
25 Distributed Micro-Firewalls K. Hwang, March 15,2001 Source: in BeijingMurali and Hwang, USC 25
26 Distributed Checkpointing on The RAID-x x in Trojans Cluster Time Process 0 Process 1 Process 2 Stripe0 Process 3 Process 4 Process 5 Process 6 Stripe1 Process 7 Process 8 Process 9 Process10 Stripe2 Process11 C: Checkpointing overhead C S S: Synchronization overhead 18
27 Security Component Technologies Firewalls and Cryptography Cluster Middleware for Security Anti-virus and Immune Systems Intrusion Detection and Response Distributed Software RAIDs Security & Assurance Policies K. Hwang, March 15,2001 in Beijing 27
28 Distributed Intrusion Detection and Responses Security Threats Insider attacks Denial-of- Service attacks Trojan Program IP Address Spoofing Probes and Scans Unauthorized External access Attacks on Intranet Infrastructure Effectiveness in using Micro-Firewalls Protect hosts against attack from insiders Protect against denial-of-service attacks from any source Protect hosts from trapdoors by any source Can be reconfigured to prevent IP spoofing at the client host level Use with IDS to block the probes and scans close to their sources Can prevent unauthorized access to the external networks at the source Resist both internal and external attacks and provide fine-grained access control
29 checkpoint overhead (sec) Checkpointing overhead on distributed RAIDs NFS Vaidya Striped checkpoint file size (MB) K. Hwang, March 15,2001 in Beijing 29
30 Advantages and Shortcomings of Distributed Checkpointing Checkpointing Scheme Advantages Shortcomings Suitable applications Simultaneous writing to a central storage (The NFS scheme) Staggered writing to a central storage (Vaidya scheme) Striped staggering checkpointing on any distributed RAID (Our scheme) Simple, no inconsistent state Eliminate the network and I/O contention Eliminate network and I/O contentions, low checkpoint overhead, fully utilize network bandwidth, tolerate multiple failures among stripe groups Has network and I/O contentions, NFS is single point of failure Network bandwidth is wasted, NFS is a single point of failure Can not tolerate more node failures within each stripe group Small size of checkpoint, small number of nodes, low I/O operation Small size of checkpointers, small number of nodes, low I/O operations Large size of checkpointers, large number of nodes, low communication, I/O intensive applications K. Hwang, March 15,2001 in Beijing 30
31 Conclusions : Distributed storage-area networks demands hardware or software support of a single I/O space not only in clusters but also in pervasive information grids. Hierarchical checkpointing with striping and staggered mirroring for building fault-tolerant clusters to provide continuous network services Hacker-proof clusters are in great demand for securing E-business, distributed computing, and metacomputing grid applications. Exploring new applications in multiserver consolidation, collaborative design, and pervasive network services. K. Hwang, March 15,2001 in Beijing 31
32 Call for Participation IEEE Third International Conference on Cluster Computing CLUSTER 2001 Sutton Place Hotel, Newport Beach, California October 8-11, 2001
RAID-x: A New Distributed Disk Array for I/O-Centric Cluster Computing
: A New Distributed Disk Array for I/O-Centric Cluster Computing Kai Hwang 1, Hai Jin 1,, and Roy Ho University of Southern California 1 The University of Hong Kong Email: {kaihwang, hjin}@ceng.usc.edu
More informationModule 6. RAID and Expansion Devices
Module 6 RAID and Expansion Devices Objectives 1. PC Hardware A.1.5 Compare and contrast RAID types B.1.8 Compare expansion devices 2 RAID 3 RAID 1. Redundant Array of Independent (or Inexpensive) Disks
More informationA Comparison on Current Distributed File Systems for Beowulf Clusters
A Comparison on Current Distributed File Systems for Beowulf Clusters Rafael Bohrer Ávila 1 Philippe Olivier Alexandre Navaux 2 Yves Denneulin 3 Abstract This paper presents a comparison on current file
More informationNetwork Attached Storage. Jinfeng Yang Oct/19/2015
Network Attached Storage Jinfeng Yang Oct/19/2015 Outline Part A 1. What is the Network Attached Storage (NAS)? 2. What are the applications of NAS? 3. The benefits of NAS. 4. NAS s performance (Reliability
More informationHigh Performance Computing. Course Notes 2007-2008. High Performance Storage
High Performance Computing Course Notes 2007-2008 2008 High Performance Storage Storage devices Primary storage: register (1 CPU cycle, a few ns) Cache (10-200 cycles, 0.02-0.5us) Main memory Local main
More informationCSAR: Cluster Storage with Adaptive Redundancy
CSAR: Cluster Storage with Adaptive Redundancy Manoj Pillai, Mario Lauria Department of Computer and Information Science The Ohio State University Columbus, OH, 4321 Email: pillai,lauria@cis.ohio-state.edu
More informationEnabling Technologies for Distributed Computing
Enabling Technologies for Distributed Computing Dr. Sanjay P. Ahuja, Ph.D. Fidelity National Financial Distinguished Professor of CIS School of Computing, UNF Multi-core CPUs and Multithreading Technologies
More informationCOM 444 Cloud Computing
COM 444 Cloud Computing Lec 2: Computer Clusters for Scalable Parallel Computing Computer Clusters for Scalable Parallel Computing 1. Clustering for Massive Parallelism 2. Computer Clusters and MPP Architectures
More informationM.Sc. IT Semester III VIRTUALIZATION QUESTION BANK 2014 2015 Unit 1 1. What is virtualization? Explain the five stage virtualization process. 2.
M.Sc. IT Semester III VIRTUALIZATION QUESTION BANK 2014 2015 Unit 1 1. What is virtualization? Explain the five stage virtualization process. 2. What are the different types of virtualization? Explain
More informationTHE EXPAND PARALLEL FILE SYSTEM A FILE SYSTEM FOR CLUSTER AND GRID COMPUTING. José Daniel García Sánchez ARCOS Group University Carlos III of Madrid
THE EXPAND PARALLEL FILE SYSTEM A FILE SYSTEM FOR CLUSTER AND GRID COMPUTING José Daniel García Sánchez ARCOS Group University Carlos III of Madrid Contents 2 The ARCOS Group. Expand motivation. Expand
More informationHow To Create A Multi Disk Raid
Click on the diagram to see RAID 0 in action RAID Level 0 requires a minimum of 2 drives to implement RAID 0 implements a striped disk array, the data is broken down into blocks and each block is written
More informationStorage Virtualization from clusters to grid
Seanodes presents Storage Virtualization from clusters to grid Rennes 4th october 2007 Agenda Seanodes Presentation Overview of storage virtualization in clusters Seanodes cluster virtualization, with
More informationHow To Build A Clustered Storage Area Network (Csan) From Power All Networks
Power-All Networks Clustered Storage Area Network: A scalable, fault-tolerant, high-performance storage system. Power-All Networks Ltd Abstract: Today's network-oriented computing environments require
More informationtechnology brief RAID Levels March 1997 Introduction Characteristics of RAID Levels
technology brief RAID Levels March 1997 Introduction RAID is an acronym for Redundant Array of Independent Disks (originally Redundant Array of Inexpensive Disks) coined in a 1987 University of California
More informationViolin: A Framework for Extensible Block-level Storage
Violin: A Framework for Extensible Block-level Storage Michail Flouris Dept. of Computer Science, University of Toronto, Canada flouris@cs.toronto.edu Angelos Bilas ICS-FORTH & University of Crete, Greece
More informationCray DVS: Data Virtualization Service
Cray : Data Virtualization Service Stephen Sugiyama and David Wallace, Cray Inc. ABSTRACT: Cray, the Cray Data Virtualization Service, is a new capability being added to the XT software environment with
More informationAgenda. Enterprise Application Performance Factors. Current form of Enterprise Applications. Factors to Application Performance.
Agenda Enterprise Performance Factors Overall Enterprise Performance Factors Best Practice for generic Enterprise Best Practice for 3-tiers Enterprise Hardware Load Balancer Basic Unix Tuning Performance
More informationEnabling Technologies for Distributed and Cloud Computing
Enabling Technologies for Distributed and Cloud Computing Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF Multi-core CPUs and Multithreading
More informationPerformance Analysis of RAIDs in Storage Area Network
Performance Analysis of RAIDs in Storage Area Network Sneha M. Assistant Professor, Department of Computer Science and Engineering, R V College of Engineering Bengaluru-560059 ABSTRACT Direct Attached
More informationCOSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters
COSC 6374 Parallel Computation Parallel I/O (I) I/O basics Spring 2008 Concept of a clusters Processor 1 local disks Compute node message passing network administrative network Memory Processor 2 Network
More informationPerformance, Reliability, and Operational Issues for High Performance NAS Storage on Cray Platforms. Cray User Group Meeting June 2007
Performance, Reliability, and Operational Issues for High Performance NAS Storage on Cray Platforms Cray User Group Meeting June 2007 Cray s Storage Strategy Background Broad range of HPC requirements
More informationBuilding scalable and reliable systems
Lectures on distributed systems Building scalable and reliable systems Paul Krzyzanowski Background The traditional approach to designing highly available systems was to incorporate elements of fault-tolerant
More informationBest Practices for Data Sharing in a Grid Distributed SAS Environment. Updated July 2010
Best Practices for Data Sharing in a Grid Distributed SAS Environment Updated July 2010 B E S T P R A C T I C E D O C U M E N T Table of Contents 1 Abstract... 2 1.1 Storage performance is critical...
More informationWrite a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical
Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or
More informationHow To Improve Performance On A Single Chip Computer
: Redundant Arrays of Inexpensive Disks this discussion is based on the paper:» A Case for Redundant Arrays of Inexpensive Disks (),» David A Patterson, Garth Gibson, and Randy H Katz,» In Proceedings
More informationOracle BI EE Implementation on Netezza. Prepared by SureShot Strategies, Inc.
Oracle BI EE Implementation on Netezza Prepared by SureShot Strategies, Inc. The goal of this paper is to give an insight to Netezza architecture and implementation experience to strategize Oracle BI EE
More informationMOSIX: High performance Linux farm
MOSIX: High performance Linux farm Paolo Mastroserio [mastroserio@na.infn.it] Francesco Maria Taurino [taurino@na.infn.it] Gennaro Tortone [tortone@na.infn.it] Napoli Index overview on Linux farm farm
More informationA NOVEL APPROACH FOR PROTECTING EXPOSED INTRANET FROM INTRUSIONS
A NOVEL APPROACH FOR PROTECTING EXPOSED INTRANET FROM INTRUSIONS K.B.Chandradeep Department of Centre for Educational Technology, IIT Kharagpur, Kharagpur, India kbchandradeep@gmail.com ABSTRACT This paper
More informationVirtualised MikroTik
Virtualised MikroTik MikroTik in a Virtualised Hardware Environment Speaker: Tom Smyth CTO Wireless Connect Ltd. Event: MUM Krackow Feb 2008 http://wirelessconnect.eu/ Copyright 2008 1 Objectives Understand
More informationan introduction to networked storage
an introduction to networked storage How networked storage can simplify your data management The key differences between SAN, DAS, and NAS The business benefits of networked storage Introduction Historical
More informationOVERVIEW. CEP Cluster Server is Ideal For: First-time users who want to make applications highly available
Phone: (603)883-7979 sales@cepoint.com Cepoint Cluster Server CEP Cluster Server turnkey system. ENTERPRISE HIGH AVAILABILITY, High performance and very reliable Super Computing Solution for heterogeneous
More informationThe team that wrote this redbook Comments welcome Introduction p. 1 Three phases p. 1 Netfinity Performance Lab p. 2 IBM Center for Microsoft
Foreword p. xv Preface p. xvii The team that wrote this redbook p. xviii Comments welcome p. xx Introduction p. 1 Three phases p. 1 Netfinity Performance Lab p. 2 IBM Center for Microsoft Technologies
More informationSMB Direct for SQL Server and Private Cloud
SMB Direct for SQL Server and Private Cloud Increased Performance, Higher Scalability and Extreme Resiliency June, 2014 Mellanox Overview Ticker: MLNX Leading provider of high-throughput, low-latency server
More informationClusters: Mainstream Technology for CAE
Clusters: Mainstream Technology for CAE Alanna Dwyer HPC Division, HP Linux and Clusters Sparked a Revolution in High Performance Computing! Supercomputing performance now affordable and accessible Linux
More informationVMware vsphere 5.1 Advanced Administration
Course ID VMW200 VMware vsphere 5.1 Advanced Administration Course Description This powerful 5-day 10hr/day class is an intensive introduction to VMware vsphere 5.0 including VMware ESX 5.0 and vcenter.
More informationStorage Networking Overview
Networking Overview iscsi Attached LAN Networking SAN NAS Gateway NAS Attached SAN Attached IBM Total Module Flow Business Challenges Networking Trends and Directions What is Networking? Technological
More informationClient/Server Computing Distributed Processing, Client/Server, and Clusters
Client/Server Computing Distributed Processing, Client/Server, and Clusters Chapter 13 Client machines are generally single-user PCs or workstations that provide a highly userfriendly interface to the
More informationApplications of Passive Message Logging and TCP Stream Reconstruction to Provide Application-Level Fault Tolerance. Sunny Gleason COM S 717
Applications of Passive Message Logging and TCP Stream Reconstruction to Provide Application-Level Fault Tolerance Sunny Gleason COM S 717 December 17, 2001 0.1 Introduction The proliferation of large-scale
More informationDependable Systems. 9. Redundant arrays of. Prof. Dr. Miroslaw Malek. Wintersemester 2004/05 www.informatik.hu-berlin.de/rok/zs
Dependable Systems 9. Redundant arrays of inexpensive disks (RAID) Prof. Dr. Miroslaw Malek Wintersemester 2004/05 www.informatik.hu-berlin.de/rok/zs Redundant Arrays of Inexpensive Disks (RAID) RAID is
More informationStorage Architectures for Big Data in the Cloud
Storage Architectures for Big Data in the Cloud Sam Fineberg HP Storage CT Office/ May 2013 Overview Introduction What is big data? Big Data I/O Hadoop/HDFS SAN Distributed FS Cloud Summary Research Areas
More informationDistributed File System Performance. Milind Saraph / Rich Sudlow Office of Information Technologies University of Notre Dame
Distributed File System Performance Milind Saraph / Rich Sudlow Office of Information Technologies University of Notre Dame Questions to answer: Why can t you locate an AFS file server in my lab to improve
More informationPetascale Software Challenges. Piyush Chaudhary piyushc@us.ibm.com High Performance Computing
Petascale Software Challenges Piyush Chaudhary piyushc@us.ibm.com High Performance Computing Fundamental Observations Applications are struggling to realize growth in sustained performance at scale Reasons
More informationScalable NAS for Oracle: Gateway to the (NFS) future
Scalable NAS for Oracle: Gateway to the (NFS) future Dr. Draško Tomić ESS technical consultant, HP EEM 2006 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change
More informationCloud Computing through Virtualization and HPC technologies
Cloud Computing through Virtualization and HPC technologies William Lu, Ph.D. 1 Agenda Cloud Computing & HPC A Case of HPC Implementation Application Performance in VM Summary 2 Cloud Computing & HPC HPC
More informationAn On-line Backup Function for a Clustered NAS System (X-NAS)
_ An On-line Backup Function for a Clustered NAS System (X-NAS) Yoshiko Yasuda, Shinichi Kawamoto, Atsushi Ebata, Jun Okitsu, and Tatsuo Higuchi Hitachi, Ltd., Central Research Laboratory 1-28 Higashi-koigakubo,
More informationInstallation Guide July 2009
July 2009 About this guide Edition notice This edition applies to Version 4.0 of the Pivot3 RAIGE Operating System and to any subsequent releases until otherwise indicated in new editions. Notification
More informationOperating Systems. RAID Redundant Array of Independent Disks. Submitted by Ankur Niyogi 2003EE20367
Operating Systems RAID Redundant Array of Independent Disks Submitted by Ankur Niyogi 2003EE20367 YOUR DATA IS LOST@#!! Do we have backups of all our data???? - The stuff we cannot afford to lose?? How
More informationW H I T E P A P E R. VMware Infrastructure Architecture Overview
W H I T E P A P E R ware Infrastructure Architecture Overview ware white paper Table of Contents Physical Topology of the ware Infrastructure Data Center............................... 4 Virtual Data Center
More informationTechnical White Paper. Symantec Backup Exec 10d System Sizing. Best Practices For Optimizing Performance of the Continuous Protection Server
Symantec Backup Exec 10d System Sizing Best Practices For Optimizing Performance of the Continuous Protection Server Table of Contents Table of Contents...2 Executive Summary...3 System Sizing and Performance
More informationBlueArc unified network storage systems 7th TF-Storage Meeting. Scale Bigger, Store Smarter, Accelerate Everything
BlueArc unified network storage systems 7th TF-Storage Meeting Scale Bigger, Store Smarter, Accelerate Everything BlueArc s Heritage Private Company, founded in 1998 Headquarters in San Jose, CA Highest
More informationGPFS Storage Server. Concepts and Setup in Lemanicus BG/Q system" Christian Clémençon (EPFL-DIT)" " 4 April 2013"
GPFS Storage Server Concepts and Setup in Lemanicus BG/Q system" Christian Clémençon (EPFL-DIT)" " Agenda" GPFS Overview" Classical versus GSS I/O Solution" GPFS Storage Server (GSS)" GPFS Native RAID
More information760 Veterans Circle, Warminster, PA 18974 215-956-1200. Technical Proposal. Submitted by: ACT/Technico 760 Veterans Circle Warminster, PA 18974.
760 Veterans Circle, Warminster, PA 18974 215-956-1200 Technical Proposal Submitted by: ACT/Technico 760 Veterans Circle Warminster, PA 18974 for Conduction Cooled NAS Revision 4/3/07 CC/RAIDStor: Conduction
More information(Scale Out NAS System)
For Unlimited Capacity & Performance Clustered NAS System (Scale Out NAS System) Copyright 2010 by Netclips, Ltd. All rights reserved -0- 1 2 3 4 5 NAS Storage Trend Scale-Out NAS Solution Scaleway Advantages
More informationDistributed File System. MCSN N. Tonellotto Complements of Distributed Enabling Platforms
Distributed File System 1 How do we get data to the workers? NAS Compute Nodes SAN 2 Distributed File System Don t move data to workers move workers to the data! Store data on the local disks of nodes
More informationVMware vsphere 5.0 Boot Camp
VMware vsphere 5.0 Boot Camp This powerful 5-day 10hr/day class is an intensive introduction to VMware vsphere 5.0 including VMware ESX 5.0 and vcenter. Assuming no prior virtualization experience, this
More informationChapter 6. 6.1 Introduction. Storage and Other I/O Topics. p. 570( 頁 585) Fig. 6.1. I/O devices can be characterized by. I/O bus connections
Chapter 6 Storage and Other I/O Topics 6.1 Introduction I/O devices can be characterized by Behavior: input, output, storage Partner: human or machine Data rate: bytes/sec, transfers/sec I/O bus connections
More informationVIA COLLAGE Deployment Guide
VIA COLLAGE Deployment Guide www.true-collaboration.com Infinite Ways to Collaborate CONTENTS Introduction... 3 User Experience... 3 Pre-Deployment Planning... 3 Connectivity... 3 Network Addressing...
More informationOverview of I/O Performance and RAID in an RDBMS Environment. By: Edward Whalen Performance Tuning Corporation
Overview of I/O Performance and RAID in an RDBMS Environment By: Edward Whalen Performance Tuning Corporation Abstract This paper covers the fundamentals of I/O topics and an overview of RAID levels commonly
More informationUsing Linux Clusters as VoD Servers
HAC LUCE Using Linux Clusters as VoD Servers Víctor M. Guĺıas Fernández gulias@lfcia.org Computer Science Department University of A Corunha funded by: Outline Background: The Borg Cluster Video on Demand.
More informationKFUPM Enterprise Network. Sadiq M. Sait sadiq@kfupm.edu.sa
KFUPM Enterprise Network Sadiq M. Sait sadiq@kfupm.edu.sa 1 Outline KFUPM Enterprise Network Diagram KFUPM Network Model KFUPM Backbone Network Connectivity: Academic Buildings, SDN, RAS Major Acheivements
More informationOnline Remote Data Backup for iscsi-based Storage Systems
Online Remote Data Backup for iscsi-based Storage Systems Dan Zhou, Li Ou, Xubin (Ben) He Department of Electrical and Computer Engineering Tennessee Technological University Cookeville, TN 38505, USA
More informationBuilding a Linux Cluster
Building a Linux Cluster CUG Conference May 21-25, 2001 by Cary Whitney Clwhitney@lbl.gov Outline What is PDSF and a little about its history. Growth problems and solutions. Storage Network Hardware Administration
More informationSAN Conceptual and Design Basics
TECHNICAL NOTE VMware Infrastructure 3 SAN Conceptual and Design Basics VMware ESX Server can be used in conjunction with a SAN (storage area network), a specialized high speed network that connects computer
More informationDiscover Smart Storage Server Solutions
Accelerating Your Success Discover Smart Server Solutions Performance, efficiency and value STORAGE Block and file explained Block explained IP-SAN, also known as Internet Small Computer System Interface
More informationCOSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters
COSC 6374 Parallel I/O (I) I/O basics Fall 2012 Concept of a clusters Processor 1 local disks Compute node message passing network administrative network Memory Processor 2 Network card 1 Network card
More informationPOWER ALL GLOBAL FILE SYSTEM (PGFS)
POWER ALL GLOBAL FILE SYSTEM (PGFS) Defining next generation of global storage grid Power All Networks Ltd. Technical Whitepaper April 2008, version 1.01 Table of Content 1. Introduction.. 3 2. Paradigm
More informationHow To Virtualize A Storage Area Network (San) With Virtualization
A New Method of SAN Storage Virtualization Table of Contents 1 - ABSTRACT 2 - THE NEED FOR STORAGE VIRTUALIZATION 3 - EXISTING STORAGE VIRTUALIZATION METHODS 4 - A NEW METHOD OF VIRTUALIZATION: Storage
More informationComparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet. September 2014
Comparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet Anand Rangaswamy September 2014 Storage Developer Conference Mellanox Overview Ticker: MLNX Leading provider of high-throughput,
More informationLessons learned from parallel file system operation
Lessons learned from parallel file system operation Roland Laifer STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association
More informationCOMP 7970 Storage Systems
COMP 797 Storage Systems Dr. Xiao Qin Department of Computer Science and Software Engineering Auburn University http://www.eng.auburn.edu/~xqin xqin@auburn.edu COMP 797, Auburn University Slide 3b- Problems
More informationHigh Availability Solutions with MySQL
High Availability Solutions with MySQL best OpenSystems Day Fall 2008 Ralf Gebhardt Senior Systems Engineer MySQL Global Software Practice ralf.gebhardt@sun.com 1 HA Requirements and Considerations HA
More informationMicrosoft Exchange Server 2003 Deployment Considerations
Microsoft Exchange Server 3 Deployment Considerations for Small and Medium Businesses A Dell PowerEdge server can provide an effective platform for Microsoft Exchange Server 3. A team of Dell engineers
More informationLecture 23: Multiprocessors
Lecture 23: Multiprocessors Today s topics: RAID Multiprocessor taxonomy Snooping-based cache coherence protocol 1 RAID 0 and RAID 1 RAID 0 has no additional redundancy (misnomer) it uses an array of disks
More informationIntroduction to Gluster. Versions 3.0.x
Introduction to Gluster Versions 3.0.x Table of Contents Table of Contents... 2 Overview... 3 Gluster File System... 3 Gluster Storage Platform... 3 No metadata with the Elastic Hash Algorithm... 4 A Gluster
More informationLS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance
11 th International LS-DYNA Users Conference Session # LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance Gilad Shainer 1, Tong Liu 2, Jeff Layton 3, Onur Celebioglu
More informationES-1 Elettronica dei Sistemi 1 Computer Architecture
ES- Elettronica dei Sistemi Computer Architecture Lesson 7 Disk Arrays Network Attached Storage 4"» "» 8"» 525"» 35"» 25"» 8"» 3"» high bandwidth disk systems based on arrays of disks Decreasing Disk Diameters
More informationLecture 36: Chapter 6
Lecture 36: Chapter 6 Today s topic RAID 1 RAID Redundant Array of Inexpensive (Independent) Disks Use multiple smaller disks (c.f. one large disk) Parallelism improves performance Plus extra disk(s) for
More informationORACLE DATABASE 10G ENTERPRISE EDITION
ORACLE DATABASE 10G ENTERPRISE EDITION OVERVIEW Oracle Database 10g Enterprise Edition is ideal for enterprises that ENTERPRISE EDITION For enterprises of any size For databases up to 8 Exabytes in size.
More informationScala Storage Scale-Out Clustered Storage White Paper
White Paper Scala Storage Scale-Out Clustered Storage White Paper Chapter 1 Introduction... 3 Capacity - Explosive Growth of Unstructured Data... 3 Performance - Cluster Computing... 3 Chapter 2 Current
More informationCisco Small Business NAS Storage
Cisco Small Business NAS Storage Stanislav.Hrda@alefnula.sk Session number:208 070 400 Bezplatné číslo (volanie cez Skype): +1 866 432 9903 Lokálny tel. pre Česko: +420 221 435 100 Lokálny tel. pre Slovensko:
More informationQuantum StorNext. Product Brief: Distributed LAN Client
Quantum StorNext Product Brief: Distributed LAN Client NOTICE This product brief may contain proprietary information protected by copyright. Information in this product brief is subject to change without
More informationA Performance Monitor based on Virtual Global Time for Clusters of PCs
A Performance Monitor based on Virtual Global Time for Clusters of PCs Michela Taufer Scripps Institute & UCSD Dept. of CS San Diego, USA Thomas Stricker Cluster 2003, 12/2/2003 Hong Kong, SAR, China Lab.
More informationRemote PC Guide Series - Volume 1
Introduction and Planning for Remote PC Implementation with NETLAB+ Document Version: 2016-02-01 What is a remote PC and how does it work with NETLAB+? This educational guide will introduce the concepts
More informationVIA CONNECT PRO Deployment Guide
VIA CONNECT PRO Deployment Guide www.true-collaboration.com Infinite Ways to Collaborate CONTENTS Introduction... 3 User Experience... 3 Pre-Deployment Planning... 3 Connectivity... 3 Network Addressing...
More informationHP recommended configuration for Microsoft Exchange Server 2010: HP LeftHand P4000 SAN
HP recommended configuration for Microsoft Exchange Server 2010: HP LeftHand P4000 SAN Table of contents Executive summary... 2 Introduction... 2 Solution criteria... 3 Hyper-V guest machine configurations...
More informationDistributed Operating Systems. Cluster Systems
Distributed Operating Systems Cluster Systems Ewa Niewiadomska-Szynkiewicz ens@ia.pw.edu.pl Institute of Control and Computation Engineering Warsaw University of Technology E&IT Department, WUT 1 1. Cluster
More informationDAS, NAS or SAN: Choosing the Right Storage Technology for Your Organization
DAS, NAS or SAN: Choosing the Right Storage Technology for Your Organization New Drivers in Information Storage Data is unquestionably the lifeblood of today s digital organization. Storage solutions remain
More informationGeneral Parallel File System (GPFS) Native RAID For 100,000-Disk Petascale Systems
General Parallel File System (GPFS) Native RAID For 100,000-Disk Petascale Systems Veera Deenadhayalan IBM Almaden Research Center 2011 IBM Corporation Hard Disk Rates Are Lagging There have been recent
More informationContent Distribution Management
Digitizing the Olympics was truly one of the most ambitious media projects in history, and we could not have done it without Signiant. We used Signiant CDM to automate 54 different workflows between 11
More informationSCSI vs. Fibre Channel White Paper
SCSI vs. Fibre Channel White Paper 08/27/99 SCSI vs. Fibre Channel Over the past decades, computer s industry has seen radical change in key components. Limitations in speed, bandwidth, and distance have
More informationCourse Description and Outline. IT Essential II: Network Operating Systems V2.0
Course Description and Outline IT Essential II: Network Operating Systems V2.0 Course Outline 1. Operating System Fundamentals 1.1 Operating System Basics 1.1.1 Overview of PC operating systems 1.1.2 PCs
More informationHow To Install Linux Titan
Linux Titan Distribution Presented By: Adham Helal Amgad Madkour Ayman El Sayed Emad Zakaria What Is a Linux Distribution? What is a Linux Distribution? The distribution contains groups of packages and
More informationMellanox Cloud and Database Acceleration Solution over Windows Server 2012 SMB Direct
Mellanox Cloud and Database Acceleration Solution over Windows Server 2012 Direct Increased Performance, Scaling and Resiliency July 2012 Motti Beck, Director, Enterprise Market Development Motti@mellanox.com
More informationIBM ^ xseries ServeRAID Technology
IBM ^ xseries ServeRAID Technology Reliability through RAID technology Executive Summary: t long ago, business-critical computing on industry-standard platforms was unheard of. Proprietary systems were
More informationIBM System x GPFS Storage Server
IBM System x GPFS Storage Server Schöne Aussicht en für HPC Speicher ZKI-Arbeitskreis Paderborn, 15.03.2013 Karsten Kutzer Client Technical Architect Technical Computing IBM Systems & Technology Group
More informationTotalStorage Network Attached Storage 300G Cost effective integration of NAS and LAN solutions
TotalStorage Network Attached Storage 300G Cost effective integration of NAS and LAN solutions Overview The new IBM TotalStorage Network Attached Storage 300G series is part of the overall IBM Storage
More informationVMWARE VSPHERE 5.0 WITH ESXI AND VCENTER
VMWARE VSPHERE 5.0 WITH ESXI AND VCENTER CORPORATE COLLEGE SEMINAR SERIES Date: April 15-19 Presented by: Lone Star Corporate College Format: Location: Classroom instruction 8 a.m.-5 p.m. (five-day session)
More informationSystems Infrastructure for Data Science. Web Science Group Uni Freiburg WS 2014/15
Systems Infrastructure for Data Science Web Science Group Uni Freiburg WS 2014/15 Lecture I: Storage Storage Part I of this course Uni Freiburg, WS 2014/15 Systems Infrastructure for Data Science 3 The
More informationMilestone Solution Partner IT Infrastructure Components Certification Summary
Milestone Solution Partner IT Infrastructure Components Certification Summary Dell FS8600 NAS Storage 12-1-2014 Table of Contents Introduction:... 2 Dell Storage Architecture:... 3 Certified Products:...
More informationList of Figures and Tables
List of Figures and Tables FIGURES 1.1 Server-Centric IT architecture 2 1.2 Inflexible allocation of free storage capacity 3 1.3 Storage-Centric IT architecture 4 1.4 Server upgrade: preparation of a new
More information