Summer Student Project Report

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Summer Student Project Report"

Transcription

1 Summer Student Project Report Dimitris Kalimeris National and Kapodistrian University of Athens June September 2014 Abstract This report will outline two projects that were done as part of a three months long summer internship at CERN. In the first project we dealt with Worldwide LHC Computing Grid (WLCG) and its information system. The information system currently conforms to a schema called GLUE and it is evolving towards a new version: GLUE2. The aim of the project was to develop and adapt the current information system of the WLCG, used by the Large Scale Storage Systems at CERN (CASTOR and EOS), to the new GLUE2 schema. During the second project we investigated different RAID configurations so that we can get performance boost from CERN s disk systems in the future. RAID 1 that is currently in use is not an option anymore because of limited performance and high cost. We tried to discover RAID configurations that will improve the performance and simultaneously decrease the cost. 1 Information-provider scripts for GLUE2 1.1 Introduction The Worldwide LHC Computing Grid (WLCG, see also 1) is an international collaboration consisting of a grid-based computer network infrastructure incorporating over 170 computing centres in 36 countries. It was originally designed by CERN to handle the large data volume produced by the Large Hadron Collider (LHC) experiments. This data is stored at CERN Storage Systems which are responsible for keeping and making available more than 100 Petabytes (105 Terabytes) of data to the physics community. The data is also replicated from CERN to the main computing centres within the WLCG. With such a diversity of sites and different storage systems having a common information system is crucial to guarantee interoperation among the different computing centres. The current WLCG information system conforms to a schema named GLUE which evolving towards a new version: GLUE2. The specific work we had to do was to develop and adapt the current information system used by the Large Scale Storage Systems at CERN (CASTOR and EOS) to the new GLUE2 schema. Reader can check references 2, 3 and 4 for more information about GLUE/GLUE2. 1

2 1.2 Information Providers for CASTOR and EOS The information system of CERN s Storage Systems is expected to provide the user with data i.e. total storage space used by some Virtual Organization (VO), how much of this storage space is online and how much is nearline, the protocols the user can use to access the data, etc. The user can ask for this information through LDAP (standard application protocol for accessing and maintaining distributed directory information services over an Internet Protocol (IP) network) search. There are two monitoring boxes for CASTOR and two for EOS. The information providers are perl scripts running inside those boxes and they are responsible for fetching all the required information from web pages and Service Level Status (SLS). As long as they gather the required data they print an LDIF (a standard plain text data interchange format for representing LDAP) file in a way compatible with GLUE schema. What we had to do was update the perl scripts so that they would display the information in a way that will be compatible with GLUE2 schema. Information providers work in two levels: 1. Gather information for CASTOR and EOS services in the way mentioned before. 2. Arrange this information in classes and print one LDIF file with the attributes of every class. There was no need to change the way that we collect the information. What we did was organizing it under a different structure. By that we mean that some classes that existed in GLUE do not exist anymore in GLUE2 (e.g. GlueSEControlProtocol), some were splitted in several others (e.g. GlueSE is splitted in GLUE2StorageService and GLUE2StorageServiceCapacity), GLUE2 has new classes as well (e.g. GLUE2StorageManager), etc. We firstly designed and implemented the architecture of the new classes. After that we pass the information that we collect in this structure and we print the LDIF file which is compatible with GLUE2 schema. Before we put the new information providers into production we tested that the LDIF file was correct under GLUE2, using a tool called GLUE validator, which confirmed the correctness of the output of the scripts. Next step was the integration of the information providers in production. We deployed the new scripts (GLUE2) in the monitoring boxes and run them in parallel with the old ones (GLUE), because we want be able to publish both GLUE and GLUE2. After some finals tests about the correctness of the LDIF files in production CERN s Storage Systems can now publish their information also in GLUE2. The copmlete transition from GLUE to GLUE2 is expected to happen in the future. 1.3 Reuse of this work As a conclusion we would like to add that there are more sites outside CERN running CASTOR/EOS (e.g. RAL, Taiwan). Despite the fact that the scripts that we developed for this project are CERN specific, with a little effort we can 2

3 adapt them to the singularities of every site and the transition from GLUE to GLUE2 for these sites can be done easily. 2 RAID Configurations 2.1 Introduction RAID (Redundant Array of Inexpensive Disks, see also ref 1) is a concept that was developed in 1977 by David Patterson, Garth Gibson, and Randy Katz as a way to use several inexpensive disks to create a single disk from the perspective of the OS while also achieving enhanced reliability or performance or both. As the disks of the array are independent you can read/write data from/to several disks at a time and this improves the performance. Apart from that in almost all RAID levels (except from RAID 0) there is a way to recover the data after a disk failure and avoid an array failure. This offers data reliability. There are 7 different standard levels of RAID: 1. RAID 0: splits data across drives, resulting in higher data throughput. The performance of this configuration is extremely high, but a loss of any drive in the array will result in data loss. This level is commonly referred to as striping. 2. RAID 1: writes all data to two or more drives for 100% redundancy: if either drive fails, no data is lost. Compared to a single drive, RAID 1 tends to be faster on reads, slower on writes. This is a good entry-level redundant configuration. However, since an entire drive is a duplicate, the cost per megabyte is high. This is commonly referred to as mirroring. 3. RAID 2: stripes data at the bit level instead of the block level (remember that RAID-0 stripes at the block level) and uses a Hamming Coding for parity computations. In RAID-2, the first bit is written on the first drive, the second bit is written on the second drive, and so on. Then a Hammingcode parity is computed and either stored on the disks or on a separate disk. 4. RAID 3: uses data striping at the byte level and also adds parity computations and stores them on a dedicated parity disk. 5. RAID 4: uses data striping at the block level and also adds parity computations and stores them on a dedicated parity disk. In a similar fashion, RAID-4 builds on RAID-0 by adding a parity disk to block-level striping. 6. RAID 5: stripes data at a block level across several drives, with parity equality distributed among the drives. The parity information allows recovery from the failure of any single drive. Write performance is rather quick, but because parity data must be skipped on each drive during reads, reads are slower. The low ratio of parity to data means low redundancy overhead. 3

4 (a) RAID 10 array. (b) RAID 60 array. Figure 1: Hybrid RAID arrays. 7. RAID 6: is an upgrade from RAID-5: data is striped at a block level across several drives with double parity distributed among the drives. As in RAID-5, parity information allows recovery from the failure of any single drive. The double parity gives RAID-6 additional redundancy at the cost of lower write performance (read performance is the same), and redundancy overhead remains low. Note that RAID 2,3,4 are not in use anymore. There are nested RAID levels too, also known as hybrid RAID, which combine two or more of the standard levels of RAID to gain even better performance, additional redundancy, or both. Examples: RAID 1+0 or 10, 5+0 or 50, 6+0 or Hardware and Software RAID arrays There are several operations that take place in a RAID array, such as sending chunks of data to the appropriate disks, computing parity, hot-swapping, disk fail-over, checking read transactions to determine if the read was successfully and if not, declaring that disk as down and more. All of these tasks require some sort of computation and have to be performed by a RAID controller. There are two options for RAID controllers: 1. hardware RAID that has a dedicated RAID controller to run the RAID application. Hardware RAID has very good performance but it is expensive and the RAID controller is a single point of failure. If it fails the whole RAID array fails. 2. software RAID that uses the CPU for RAID chores without need for any additional hardware. Software RAID is cheaper but the performance is worse and processing load is added on the CPU. During the last years the gap in performance between hardware and software RAID is closing and the flexibility and reliability offered by software RAID made it the first choice for many systems. 2.3 Creating and testing software RAID arrays During our performance tests we focused on two RAID levels that fitted our needs, i.e. high performance, good performance/cost ratio, high fault tolerance and good rebuild performance. Those were RAID 10 and RAID 60 (figure 1). The basic reasons that led into excluding other RAID levels were: 4

5 RAID 0: no fault tolerance at all. RAID 1: same cost as RAID 10 but much worse performance. RAID 5: if one of the HDD of the array fails then you have to read all the rest of the HDDs without any error to rebuild it. Otherwise the whole array will fail. RAID 50: it connects several RAID 5s and that makes it vulnerable in the way that we mentioned above for RAID 5. RAID 6: you need many HDDs to get a big performance boost and that limits data security. Let us symbolize RAID x/y(/z): x: RAID level. y: total number of disks in the array. z: number of standard RAID level arrays that we have if the array is nested. For example RAID 60/12/2 is a RAID 60 array with 12 disks in total and 2 RAID 6 arrays (of 6 disks each). We tested the following RAID 10 and RAID 60 configurations (inside the brackets is the redundancy of the configuration): RAID arrays: 10/12/6 (50%), 60/12/2 (33.3%), 60/16/2(25%), 60/24/3 (25%), 60/24/2 (25%). In all the arrays we used software RAID and were created with mdadm (see ref 2) and benchmarked with IOzone (see ref 3). The tests that we performed with IOzone measured: Write/Re-write performance. Read/Re-read performance. CPU utilization. of the RAID array for files of size: 1, 2, 4, 8, 16, 32, 64, 128GB. We took measurements when we had only 1 thread and when we had 10 threads running in the array. Furthermore, we measured the degradation in the array s performance while it was in rebuild mode. The machines that we performed the the tests in have the following characteristics: 64GB RAM 48 HDDs 4TB/HDD 5

6 Figure 2: Comparison for RAID 60s, 1 thread. 2.4 Results For small file sizes we notice very high performance due to buffer caching of the system (figures 2, 3). The real disk performance is much lower. After the caching effects are over we can notice that bigger arrays have greater speedup because writing/reading happens in many disks simultaneously. For 10 threads the ranking of the RAID arrays remains the same in terms of performance but the actual performance is degraded a lot because of many clients asking blocks from the array simultaneously. When we compare RAID 10 with RAID 60 we see that in reading there is no diference and the performance of the arrays is simply proportional to the number of disks they contain (see figure 4, for RAID 10 and 60 with 12 HDD each). In writing though there is a big gap in performance because RAID 10 does not need to compute any parity and just writes blocks in all disks simultaneously while writing in RAID 60 is more time consuming. 6

7 Figure 3: Comparison for RAID 60s, 10 threads. 7

8 Figure 4: Comparison for RAID 10 and 60, 1 thread. 8

9 2.5 Results for RAID 60 during rebuilding In RAID 60 a single HDD failure leads to reading the whole RAID 6 array where this disk belongs to. So, due to rapid increases in disk capacity and the slow improvement in disk access speed, RAID system are facing much longer disk reconstruction times than before, which can also reduce system reliability. Figure 5: Normal vs rebuilding for RAID 60, 12 disks, 2 RAID 6s. On the other hand the performance is not that much degraded during the rebuilding of the array, as we can see in figure 5. We notice 50% decreasing in performance in reading. During writing the performance is decreased by slightly less, around 20-30%. Despite the fact that the performance is still in accepted levels the process of rebuilding lasts for a long time. With very intensive requests to the server that happen when we run IOzone tests we had a rebuild time of days after only one single disk failure in RAID 60/24/3. In reality the load of the requests from clients in the server is never expected to be that high as with IOzone requests but still the rebuilding period can be several days for only one disk. 9

10 (a) Writing performance while rebuilding (b) Reading performance while rebuilding Figure 6: RAID configurations are ordered in the same way while rebuilding. 2.6 Ways to improve rebuild times Storage systems in CERN are about to use really big arrays so the reconstruction process is expected to take long. This is obviously a problem. If reconstruction takes too long then the probability of having a second disk failure increases because the load on the disks is much higher than usually due to the rebuilding process. There are two major ways to deal with it if we notice that the array is at risk. 1. Remove the machine from production. This will lead to 0 client requests and the array will be fully reconstructed in only some hours. 2. Linux kernel limits the speed imposed on the RAID reconstruction. In urgent cases you can speed up the process of reconstruction by increasing the minimum guaranteed speed of the rebuild of the array. This method 10

11 though, will place a much higher load on the system and will degrade the performance so it should be used with care. Code for decreasing the rebuilding time: Display maximum and minimum speed guaranteed for your system: > cat /proc/sys/dev/raid/speed limit max > cat /proc/sys/dev/raid/speed limit min 1000 Increase minimum speed guaranteed: > echo > /proc/sys/dev/raid/speed limit min I would like to add that from the results that we got we can see that we cannot continue increasing the size of the arrays otherwise we will face reconstruction times of weeks as disks will go larger. We have to find alternative solutions if we want extra performance for our systems. 2.7 Conclusion From the tests that we performed we got results that are compatible with theory. The most obvious is that the bigger the speedup we want, the bigger the array that we have to use. As for the configurations we got that RAID 10 is the best in terms of performance and rebuild times. RAID 60/24/3 seems to be the best among the RAID 60 configurations, and it is a good compromise between cost and performance. There are of course more tests to be done like: Rebuild tests: Rebuild time and performance with more than one failed disks and in several of the RAID 6 arrays that consist the RAID 60. Rebuild time and performance when we have less intensive requests than with IOzone. Rebuild time, performance and CPU utillization when you increase the default speed imposed on the RAID reconstruction. Check how the machines behave as a whole with many RAID arrays, and not only how a specific array performs. Finally I would like to say that before we apply a RAID configuration into production we have to think if we need all the speed-up that it can offer. Depending on the server we can have many small arrays (if we cannot exploit huge boost in performance for reasons like net bandwidth, or simply we do not need that great speed), or fewer bigger arrays for great performance guarantees. In the near future it is possible that many clients of CERN s servers (tapes, etc.) will start performing in much higher speed than they do now so the extra boost from the RAID arrays that today seems meaningless would be a must. 11

12 3 References Information providers for GLUE and GLUE2: 1. WLCG. 2. Glue Schema Specification. 3. Glue2.0 Schema Specification. 4. GLUE2 Template for DPM Storage Service RAID arrays: 1. RAID levels. 2. Software RAID with mdadm. 3. IOzone documentation. 4. Increase the speed of rebuilding. 5. Large disks, the end of RAID? 12

PIONEER RESEARCH & DEVELOPMENT GROUP

PIONEER RESEARCH & DEVELOPMENT GROUP SURVEY ON RAID Aishwarya Airen 1, Aarsh Pandit 2, Anshul Sogani 3 1,2,3 A.I.T.R, Indore. Abstract RAID stands for Redundant Array of Independent Disk that is a concept which provides an efficient way for

More information

RAID Made Easy By Jon L. Jacobi, PCWorld

RAID Made Easy By Jon L. Jacobi, PCWorld 9916 Brooklet Drive Houston, Texas 77099 Phone 832-327-0316 www.safinatechnolgies.com RAID Made Easy By Jon L. Jacobi, PCWorld What is RAID, why do you need it, and what are all those mode numbers that

More information

CS161: Operating Systems

CS161: Operating Systems CS161: Operating Systems Matt Welsh mdw@eecs.harvard.edu Lecture 18: RAID April 19, 2007 2007 Matt Welsh Harvard University 1 RAID Redundant Arrays of Inexpensive Disks Invented in 1986-1987 by David Patterson

More information

RAID Basics Training Guide

RAID Basics Training Guide RAID Basics Training Guide Discover a Higher Level of Performance RAID matters. Rely on Intel RAID. Table of Contents 1. What is RAID? 2. RAID Levels RAID 0 RAID 1 RAID 5 RAID 6 RAID 10 RAID 0+1 RAID 1E

More information

Reliability and Fault Tolerance in Storage

Reliability and Fault Tolerance in Storage Reliability and Fault Tolerance in Storage Dalit Naor/ Dima Sotnikov IBM Haifa Research Storage Systems 1 Advanced Topics on Storage Systems - Spring 2014, Tel-Aviv University http://www.eng.tau.ac.il/semcom

More information

Operating Systems. RAID Redundant Array of Independent Disks. Submitted by Ankur Niyogi 2003EE20367

Operating Systems. RAID Redundant Array of Independent Disks. Submitted by Ankur Niyogi 2003EE20367 Operating Systems RAID Redundant Array of Independent Disks Submitted by Ankur Niyogi 2003EE20367 YOUR DATA IS LOST@#!! Do we have backups of all our data???? - The stuff we cannot afford to lose?? How

More information

Click on the diagram to see RAID 0 in action

Click on the diagram to see RAID 0 in action Click on the diagram to see RAID 0 in action RAID Level 0 requires a minimum of 2 drives to implement RAID 0 implements a striped disk array, the data is broken down into blocks and each block is written

More information

Lecture 36: Chapter 6

Lecture 36: Chapter 6 Lecture 36: Chapter 6 Today s topic RAID 1 RAID Redundant Array of Inexpensive (Independent) Disks Use multiple smaller disks (c.f. one large disk) Parallelism improves performance Plus extra disk(s) for

More information

technology brief RAID Levels March 1997 Introduction Characteristics of RAID Levels

technology brief RAID Levels March 1997 Introduction Characteristics of RAID Levels technology brief RAID Levels March 1997 Introduction RAID is an acronym for Redundant Array of Independent Disks (originally Redundant Array of Inexpensive Disks) coined in a 1987 University of California

More information

Distribution One Server Requirements

Distribution One Server Requirements Distribution One Server Requirements Introduction Welcome to the Hardware Configuration Guide. The goal of this guide is to provide a practical approach to sizing your Distribution One application and

More information

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com Parallels Cloud Storage White Paper Performance Benchmark Results www.parallels.com Table of Contents Executive Summary... 3 Architecture Overview... 3 Key Features... 4 No Special Hardware Requirements...

More information

Dependable Systems. 9. Redundant arrays of. Prof. Dr. Miroslaw Malek. Wintersemester 2004/05 www.informatik.hu-berlin.de/rok/zs

Dependable Systems. 9. Redundant arrays of. Prof. Dr. Miroslaw Malek. Wintersemester 2004/05 www.informatik.hu-berlin.de/rok/zs Dependable Systems 9. Redundant arrays of inexpensive disks (RAID) Prof. Dr. Miroslaw Malek Wintersemester 2004/05 www.informatik.hu-berlin.de/rok/zs Redundant Arrays of Inexpensive Disks (RAID) RAID is

More information

High Performance Computing. Course Notes 2007-2008. High Performance Storage

High Performance Computing. Course Notes 2007-2008. High Performance Storage High Performance Computing Course Notes 2007-2008 2008 High Performance Storage Storage devices Primary storage: register (1 CPU cycle, a few ns) Cache (10-200 cycles, 0.02-0.5us) Main memory Local main

More information

Definition of RAID Levels

Definition of RAID Levels RAID The basic idea of RAID (Redundant Array of Independent Disks) is to combine multiple inexpensive disk drives into an array of disk drives to obtain performance, capacity and reliability that exceeds

More information

PARALLELS CLOUD STORAGE

PARALLELS CLOUD STORAGE PARALLELS CLOUD STORAGE Performance Benchmark Results 1 Table of Contents Executive Summary... Error! Bookmark not defined. Architecture Overview... 3 Key Features... 5 No Special Hardware Requirements...

More information

Chapter 6. 6.1 Introduction. Storage and Other I/O Topics. p. 570( 頁 585) Fig. 6.1. I/O devices can be characterized by. I/O bus connections

Chapter 6. 6.1 Introduction. Storage and Other I/O Topics. p. 570( 頁 585) Fig. 6.1. I/O devices can be characterized by. I/O bus connections Chapter 6 Storage and Other I/O Topics 6.1 Introduction I/O devices can be characterized by Behavior: input, output, storage Partner: human or machine Data rate: bytes/sec, transfers/sec I/O bus connections

More information

Storage Technologies - 2

Storage Technologies - 2 Antoniu Pop 1 The University of Manchester COMP 25212 Storage Technologies - 2 Antoniu Pop antoniu.pop@manchester.ac.uk 16 April 2015 Antoniu Pop 2 The University of Manchester COMP 25212 Storage Technologies

More information

RAID Overview: Identifying What RAID Levels Best Meet Customer Needs. Diamond Series RAID Storage Array

RAID Overview: Identifying What RAID Levels Best Meet Customer Needs. Diamond Series RAID Storage Array ATTO Technology, Inc. Corporate Headquarters 155 Crosspoint Parkway Amherst, NY 14068 Phone: 716-691-1999 Fax: 716-691-9353 www.attotech.com sales@attotech.com RAID Overview: Identifying What RAID Levels

More information

Guide to SATA Hard Disks Installation and RAID Configuration

Guide to SATA Hard Disks Installation and RAID Configuration Guide to SATA Hard Disks Installation and RAID Configuration 1. Guide to SATA Hard Disks Installation...2 1.1 Serial ATA (SATA) Hard Disks Installation...2 2. Guide to RAID Configurations...3 2.1 Introduction

More information

RAID Technology Overview

RAID Technology Overview RAID Technology Overview HP Smart Array RAID Controllers HP Part Number: J6369-90050 Published: September 2007 Edition: 1 Copyright 2007 Hewlett-Packard Development Company L.P. Legal Notices Copyright

More information

1 Storage Devices Summary

1 Storage Devices Summary Chapter 1 Storage Devices Summary Dependability is vital Suitable measures Latency how long to the first bit arrives Bandwidth/throughput how fast does stuff come through after the latency period Obvious

More information

RAID: Redundant Arrays of Inexpensive Disks this discussion is based on the paper: on Management of Data (Chicago, IL), pp.109--116, 1988.

RAID: Redundant Arrays of Inexpensive Disks this discussion is based on the paper: on Management of Data (Chicago, IL), pp.109--116, 1988. : Redundant Arrays of Inexpensive Disks this discussion is based on the paper:» A Case for Redundant Arrays of Inexpensive Disks (),» David A Patterson, Garth Gibson, and Randy H Katz,» In Proceedings

More information

CS420: Operating Systems

CS420: Operating Systems NK YORK COLLEGE OF PENNSYLVANIA HG OK 2 RAID YORK COLLEGE OF PENNSYLVAN James Moscola Department of Physical Sciences York College of Pennsylvania Based on Operating System Concepts, 9th Edition by Silberschatz,

More information

What is RAID and how does it work?

What is RAID and how does it work? What is RAID and how does it work? What is RAID? RAID is the acronym for either redundant array of inexpensive disks or redundant array of independent disks. When first conceived at UC Berkley the former

More information

RAID. RAID 0 No redundancy ( AID?) Just stripe data over multiple disks But it does improve performance. Chapter 6 Storage and Other I/O Topics 29

RAID. RAID 0 No redundancy ( AID?) Just stripe data over multiple disks But it does improve performance. Chapter 6 Storage and Other I/O Topics 29 RAID Redundant Array of Inexpensive (Independent) Disks Use multiple smaller disks (c.f. one large disk) Parallelism improves performance Plus extra disk(s) for redundant data storage Provides fault tolerant

More information

IncidentMonitor Server Specification Datasheet

IncidentMonitor Server Specification Datasheet IncidentMonitor Server Specification Datasheet Prepared by Monitor 24-7 Inc October 1, 2015 Contact details: sales@monitor24-7.com North America: +1 416 410.2716 / +1 866 364.2757 Europe: +31 088 008.4600

More information

How to recover a failed Storage Spaces

How to recover a failed Storage Spaces www.storage-spaces-recovery.com How to recover a failed Storage Spaces ReclaiMe Storage Spaces Recovery User Manual 2013 www.storage-spaces-recovery.com Contents Overview... 4 Storage Spaces concepts and

More information

Price/performance Modern Memory Hierarchy

Price/performance Modern Memory Hierarchy Lecture 21: Storage Administration Take QUIZ 15 over P&H 6.1-4, 6.8-9 before 11:59pm today Project: Cache Simulator, Due April 29, 2010 NEW OFFICE HOUR TIME: Tuesday 1-2, McKinley Last Time Exam discussion

More information

DELL RAID PRIMER DELL PERC RAID CONTROLLERS. Joe H. Trickey III. Dell Storage RAID Product Marketing. John Seward. Dell Storage RAID Engineering

DELL RAID PRIMER DELL PERC RAID CONTROLLERS. Joe H. Trickey III. Dell Storage RAID Product Marketing. John Seward. Dell Storage RAID Engineering DELL RAID PRIMER DELL PERC RAID CONTROLLERS Joe H. Trickey III Dell Storage RAID Product Marketing John Seward Dell Storage RAID Engineering http://www.dell.com/content/topics/topic.aspx/global/products/pvaul/top

More information

RAID Implementation for StorSimple Storage Management Appliance

RAID Implementation for StorSimple Storage Management Appliance RAID Implementation for StorSimple Storage Management Appliance Alpa Kohli June, 2012 KB-00008 Document Revision 1 StorSimple knowledge base articles are intended to provide customers with the information

More information

Introduction. What is RAID? The Array and RAID Controller Concept. Click here to print this article. Re-Printed From SLCentral

Introduction. What is RAID? The Array and RAID Controller Concept. Click here to print this article. Re-Printed From SLCentral Click here to print this article. Re-Printed From SLCentral RAID: An In-Depth Guide To RAID Technology Author: Tom Solinap Date Posted: January 24th, 2001 URL: http://www.slcentral.com/articles/01/1/raid

More information

Online Remote Data Backup for iscsi-based Storage Systems

Online Remote Data Backup for iscsi-based Storage Systems Online Remote Data Backup for iscsi-based Storage Systems Dan Zhou, Li Ou, Xubin (Ben) He Department of Electrical and Computer Engineering Tennessee Technological University Cookeville, TN 38505, USA

More information

Storage node capacity in RAID0 is equal to the sum total capacity of all disks in the storage node.

Storage node capacity in RAID0 is equal to the sum total capacity of all disks in the storage node. RAID configurations defined 1/7 Storage Configuration: Disk RAID and Disk Management > RAID configurations defined Next RAID configurations defined The RAID configuration you choose depends upon how you

More information

RAID: Redundant Arrays of Independent Disks

RAID: Redundant Arrays of Independent Disks RAID: Redundant Arrays of Independent Disks Dependable Systems Dr.-Ing. Jan Richling Kommunikations- und Betriebssysteme TU Berlin Winter 2012/2013 RAID: Introduction Redundant array of inexpensive disks

More information

Identifying the Hidden Risk of Data De-duplication: How the HYDRAstor Solution Proactively Solves the Problem

Identifying the Hidden Risk of Data De-duplication: How the HYDRAstor Solution Proactively Solves the Problem Identifying the Hidden Risk of Data De-duplication: How the HYDRAstor Solution Proactively Solves the Problem October, 2006 Introduction Data de-duplication has recently gained significant industry attention,

More information

RAID Storage, Network File Systems, and DropBox

RAID Storage, Network File Systems, and DropBox RAID Storage, Network File Systems, and DropBox George Porter CSE 124 February 24, 2015 * Thanks to Dave Patterson and Hong Jiang Announcements Project 2 due by end of today Office hour today 2-3pm in

More information

Guide to SATA Hard Disks Installation and RAID Configuration

Guide to SATA Hard Disks Installation and RAID Configuration Guide to SATA Hard Disks Installation and RAID Configuration 1. Guide to SATA Hard Disks Installation... 2 1.1 Serial ATA (SATA) Hard Disks Installation... 2 2. Guide to RAID Configurations... 3 2.1 Introduction

More information

ISTANBUL AYDIN UNIVERSITY

ISTANBUL AYDIN UNIVERSITY ISTANBUL AYDIN UNIVERSITY 2013-2014 Academic Year Fall Semester Department of Software Engineering SEN361 COMPUTER ORGANIZATION HOMEWORK REPORT STUDENT S NAME : GÖKHAN TAYMAZ STUDENT S NUMBER : B1105.090068

More information

Implementation of Reliable Fault Tolerant Data Storage System over Cloud using Raid 60

Implementation of Reliable Fault Tolerant Data Storage System over Cloud using Raid 60 International Journal of Computer Sciences and Engineering Open Access Review Paper Volume-4, Issue-2 E-ISSN: 2347-2693 Implementation of Reliable Fault Tolerant Data Storage System over Cloud using Raid

More information

Input / Ouput devices. I/O Chapter 8. Goals & Constraints. Measures of Performance. Anatomy of a Disk Drive. Introduction - 8.1

Input / Ouput devices. I/O Chapter 8. Goals & Constraints. Measures of Performance. Anatomy of a Disk Drive. Introduction - 8.1 Introduction - 8.1 I/O Chapter 8 Disk Storage and Dependability 8.2 Buses and other connectors 8.4 I/O performance measures 8.6 Input / Ouput devices keyboard, mouse, printer, game controllers, hard drive,

More information

COSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters

COSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters COSC 6374 Parallel I/O (I) I/O basics Fall 2012 Concept of a clusters Processor 1 local disks Compute node message passing network administrative network Memory Processor 2 Network card 1 Network card

More information

CS 153 Design of Operating Systems Spring 2015

CS 153 Design of Operating Systems Spring 2015 CS 153 Design of Operating Systems Spring 2015 Lecture 22: File system optimizations Physical Disk Structure Disk components Platters Surfaces Tracks Arm Track Sector Surface Sectors Cylinders Arm Heads

More information

SSDs and RAID: What s the right strategy. Paul Goodwin VP Product Development Avant Technology

SSDs and RAID: What s the right strategy. Paul Goodwin VP Product Development Avant Technology SSDs and RAID: What s the right strategy Paul Goodwin VP Product Development Avant Technology SSDs and RAID: What s the right strategy Flash Overview SSD Overview RAID overview Thoughts about Raid Strategies

More information

VERY IMPORTANT NOTE! - RAID

VERY IMPORTANT NOTE! - RAID Disk drives are an integral part of any computing system. Disk drives are usually where the operating system and all of an enterprise or individual s data are stored. They are also one of the weakest links

More information

The idea behind RAID is to have a number of disks co-operate in such a way that it looks like one big disk.

The idea behind RAID is to have a number of disks co-operate in such a way that it looks like one big disk. People often ask: Should I RAID my disks? The question is simple, unfortunately the answer is not. So here is a guide to help you decide when a RAID array is advantageous and how to go about it. This guide

More information

CHAPTER 4 RAID. Section Goals. Upon completion of this section you should be able to:

CHAPTER 4 RAID. Section Goals. Upon completion of this section you should be able to: HPTER 4 RI s it was originally proposed, the acronym RI stood for Redundant rray of Inexpensive isks. However, it has since come to be known as Redundant rray of Independent isks. RI was originally described

More information

RAID technology and IBM TotalStorage NAS products

RAID technology and IBM TotalStorage NAS products IBM TotalStorage Network Attached Storage October 2001 RAID technology and IBM TotalStorage NAS products By Janet Anglin and Chris Durham Storage Networking Architecture, SSG Page No.1 Contents 2 RAID

More information

The Pros and Cons of Erasure Coding & Replication vs. RAID in Next-Gen Storage Platforms. Abhijith Shenoy Engineer, Hedvig Inc.

The Pros and Cons of Erasure Coding & Replication vs. RAID in Next-Gen Storage Platforms. Abhijith Shenoy Engineer, Hedvig Inc. The Pros and Cons of Erasure Coding & Replication vs. RAID in Next-Gen Storage Platforms Abhijith Shenoy Engineer, Hedvig Inc. @hedviginc The need for new architectures Business innovation Time-to-market

More information

An Introduction to RAID. Giovanni Stracquadanio stracquadanio@dmi.unict.it www.dmi.unict.it/~stracquadanio

An Introduction to RAID. Giovanni Stracquadanio stracquadanio@dmi.unict.it www.dmi.unict.it/~stracquadanio An Introduction to RAID Giovanni Stracquadanio stracquadanio@dmi.unict.it www.dmi.unict.it/~stracquadanio Outline A definition of RAID An ensemble of RAIDs JBOD RAID 0...5 Configuring and testing a Linux

More information

VMware Virtual SAN Backup Using VMware vsphere Data Protection Advanced SEPTEMBER 2014

VMware Virtual SAN Backup Using VMware vsphere Data Protection Advanced SEPTEMBER 2014 VMware SAN Backup Using VMware vsphere Data Protection Advanced SEPTEMBER 2014 VMware SAN Backup Using VMware vsphere Table of Contents Introduction.... 3 vsphere Architectural Overview... 4 SAN Backup

More information

Introduction Disks RAID Tertiary storage. Mass Storage. CMSC 412, University of Maryland. Guest lecturer: David Hovemeyer.

Introduction Disks RAID Tertiary storage. Mass Storage. CMSC 412, University of Maryland. Guest lecturer: David Hovemeyer. Guest lecturer: David Hovemeyer November 15, 2004 The memory hierarchy Red = Level Access time Capacity Features Registers nanoseconds 100s of bytes fixed Cache nanoseconds 1-2 MB fixed RAM nanoseconds

More information

Physical Data Organization

Physical Data Organization Physical Data Organization Database design using logical model of the database - appropriate level for users to focus on - user independence from implementation details Performance - other major factor

More information

RAID. Storage-centric computing, cloud computing. Benefits:

RAID. Storage-centric computing, cloud computing. Benefits: RAID Storage-centric computing, cloud computing. Benefits: Improved reliability (via error correcting code, redundancy). Improved performance (via redundancy). Independent disks. RAID Level 0 Provides

More information

Best Practices RAID Implementations for Snap Servers and JBOD Expansion

Best Practices RAID Implementations for Snap Servers and JBOD Expansion STORAGE SOLUTIONS WHITE PAPER Best Practices RAID Implementations for Snap Servers and JBOD Expansion Contents Introduction...1 Planning for the End Result...1 Availability Considerations...1 Drive Reliability...2

More information

Identifying the Hidden Risk of Data Deduplication: How the HYDRAstor TM Solution Proactively Solves the Problem

Identifying the Hidden Risk of Data Deduplication: How the HYDRAstor TM Solution Proactively Solves the Problem Identifying the Hidden Risk of Data Deduplication: How the HYDRAstor TM Solution Proactively Solves the Problem Advanced Storage Products Group Table of Contents 1 - Introduction 2 Data Deduplication 3

More information

Assessing RAID ADG vs. RAID 5 vs. RAID 1+0

Assessing RAID ADG vs. RAID 5 vs. RAID 1+0 White Paper October 2001 Prepared by Industry Standard Storage Group Compaq Computer Corporation Contents Overview...3 Defining RAID levels...3 Evaluating RAID levels...3 Choosing a RAID level...4 Assessing

More information

HP Smart Array Controllers and basic RAID performance factors

HP Smart Array Controllers and basic RAID performance factors Technical white paper HP Smart Array Controllers and basic RAID performance factors Technology brief Table of contents Abstract 2 Benefits of drive arrays 2 Factors that affect performance 2 HP Smart Array

More information

Sistemas Operativos: Input/Output Disks

Sistemas Operativos: Input/Output Disks Sistemas Operativos: Input/Output Disks Pedro F. Souto (pfs@fe.up.pt) April 28, 2012 Topics Magnetic Disks RAID Solid State Disks Topics Magnetic Disks RAID Solid State Disks Magnetic Disk Construction

More information

CS 6290 I/O and Storage. Milos Prvulovic

CS 6290 I/O and Storage. Milos Prvulovic CS 6290 I/O and Storage Milos Prvulovic Storage Systems I/O performance (bandwidth, latency) Bandwidth improving, but not as fast as CPU Latency improving very slowly Consequently, by Amdahl s Law: fraction

More information

RAID Levels and Components Explained Page 1 of 23

RAID Levels and Components Explained Page 1 of 23 RAID Levels and Components Explained Page 1 of 23 What's RAID? The purpose of this document is to explain the many forms or RAID systems, and why they are useful, and their disadvantages. RAID - Redundant

More information

Planning Domain Controller Capacity

Planning Domain Controller Capacity C H A P T E R 4 Planning Domain Controller Capacity Planning domain controller capacity helps you determine the appropriate number of domain controllers to place in each domain that is represented in a

More information

Filing Systems. Filing Systems

Filing Systems. Filing Systems Filing Systems At the outset we identified long-term storage as desirable characteristic of an OS. EG: On-line storage for an MIS. Convenience of not having to re-write programs. Sharing of data in an

More information

RAID HARDWARE. On board SATA RAID controller. RAID drive caddy (hot swappable) SATA RAID controller card. Anne Watson 1

RAID HARDWARE. On board SATA RAID controller. RAID drive caddy (hot swappable) SATA RAID controller card. Anne Watson 1 RAID HARDWARE On board SATA RAID controller SATA RAID controller card RAID drive caddy (hot swappable) Anne Watson 1 RAID The word redundant means an unnecessary repetition. The word array means a lineup.

More information

BrightStor ARCserve Backup for Windows

BrightStor ARCserve Backup for Windows BrightStor ARCserve Backup for Windows Tape RAID Option Guide r11.5 D01183-1E This documentation and related computer software program (hereinafter referred to as the "Documentation") is for the end user's

More information

Design and Implementation of a Storage Repository Using Commonality Factoring. IEEE/NASA MSST2003 April 7-10, 2003 Eric W. Olsen

Design and Implementation of a Storage Repository Using Commonality Factoring. IEEE/NASA MSST2003 April 7-10, 2003 Eric W. Olsen Design and Implementation of a Storage Repository Using Commonality Factoring IEEE/NASA MSST2003 April 7-10, 2003 Eric W. Olsen Axion Overview Potentially infinite historic versioning for rollback and

More information

Version : 1.0. SR3620-2S-SB2 User Manual. SOHORAID Series

Version : 1.0. SR3620-2S-SB2 User Manual. SOHORAID Series Version : 1.0 SR3620-2S-SB2 User Manual SOHORAID Series Introduction About this Manual Thank you for using the product of RAIDON Technology Inc. This user manual will introduce the STARDOM SR3620-2S-SB2

More information

Using Synology SSD Technology to Enhance System Performance Synology Inc.

Using Synology SSD Technology to Enhance System Performance Synology Inc. Using Synology SSD Technology to Enhance System Performance Synology Inc. Synology_SSD_Cache_WP_ 20140512 Table of Contents Chapter 1: Enterprise Challenges and SSD Cache as Solution Enterprise Challenges...

More information

IBM ^ xseries ServeRAID Technology

IBM ^ xseries ServeRAID Technology IBM ^ xseries ServeRAID Technology Reliability through RAID technology Executive Summary: t long ago, business-critical computing on industry-standard platforms was unheard of. Proprietary systems were

More information

Outline. Database Management and Tuning. Overview. Hardware Tuning. Johann Gamper. Unit 12

Outline. Database Management and Tuning. Overview. Hardware Tuning. Johann Gamper. Unit 12 Outline Database Management and Tuning Hardware Tuning Johann Gamper 1 Free University of Bozen-Bolzano Faculty of Computer Science IDSE Unit 12 2 3 Conclusion Acknowledgements: The slides are provided

More information

How to choose the right RAID for your Dedicated Server

How to choose the right RAID for your Dedicated Server Overview of RAID Let's first address, "What is RAID and what does RAID stand for?" RAID, an acronym for "Redundant Array of Independent Disks, is a storage technology that links or combines multiple hard

More information

5-BAY RAID STATION. Manual

5-BAY RAID STATION. Manual 5-BAY RAID STATION Manual Content 1. Features 3 1.1 Overview 3 1.2 SATA features 3 1.3 USB features 4 2. Specifications 4 3. System requirements 4 4. The device at a glance 4 4.1 Rear panel 5 4.2 The LED

More information

SAN Conceptual and Design Basics

SAN Conceptual and Design Basics TECHNICAL NOTE VMware Infrastructure 3 SAN Conceptual and Design Basics VMware ESX Server can be used in conjunction with a SAN (storage area network), a specialized high speed network that connects computer

More information

Data Integrity: Backups and RAID

Data Integrity: Backups and RAID Data Integrity: Backups and RAID Introduction Keeping your data safe and reliable TM Backups Types of backups Strategy Tools RAID (Redundant Array of Independent Disks) Types of RAID What type to Use Disk

More information

Disk Storage & Dependability

Disk Storage & Dependability Disk Storage & Dependability Computer Organization Architectures for Embedded Computing Wednesday 19 November 14 Many slides adapted from: Computer Organization and Design, Patterson & Hennessy 4th Edition,

More information

An Introduction to RAID 6 ULTAMUS TM RAID

An Introduction to RAID 6 ULTAMUS TM RAID An Introduction to RAID 6 ULTAMUS TM RAID The highly attractive cost per GB of SATA storage capacity is causing RAID products based on the technology to grow in popularity. SATA RAID is now being used

More information

Hard Disk Drives and RAID

Hard Disk Drives and RAID Hard Disk Drives and RAID Janaka Harambearachchi (Engineer/Systems Development) INTERFACES FOR HDD A computer interfaces is what allows a computer to send and retrieve information for storage devices such

More information

Enterprise Architectures for Large Tiled Basemap Projects. Tommy Fauvell

Enterprise Architectures for Large Tiled Basemap Projects. Tommy Fauvell Enterprise Architectures for Large Tiled Basemap Projects Tommy Fauvell Tommy Fauvell Senior Technical Analyst Esri Professional Services Washington D.C Regional Office Project Technical Lead: - Responsible

More information

Comprehending the Tradeoffs between Deploying Oracle Database on RAID 5 and RAID 10 Storage Configurations. Database Solutions Engineering

Comprehending the Tradeoffs between Deploying Oracle Database on RAID 5 and RAID 10 Storage Configurations. Database Solutions Engineering Comprehending the Tradeoffs between Deploying Oracle Database on RAID 5 and RAID 10 Storage Configurations A Dell Technical White Paper Database Solutions Engineering By Sudhansu Sekhar and Raghunatha

More information

Algorithms and Methods for Distributed Storage Networks 4: Volume Manager and RAID Christian Schindelhauer

Algorithms and Methods for Distributed Storage Networks 4: Volume Manager and RAID Christian Schindelhauer Algorithms and Methods for Distributed Storage Networks 4: Volume Manager and RAID Institut für Informatik Wintersemester 2007/08 RAID Redundant Array of Independent Disks Patterson, Gibson, Katz, A Case

More information

NVIDIA RAID Installation Guide

NVIDIA RAID Installation Guide NVIDIA RAID Installation Guide 1. NVIDIA BIOS RAID Installation Guide.. 2 1.1 Introduction to RAID. 2 1.2 RAID Configurations Precautions.. 3 1.3 Create Disk Array. 4 2. NVIDIA Windows RAID Installation

More information

Parallels Cloud Server 6.0

Parallels Cloud Server 6.0 Parallels Cloud Server 6.0 Parallels Cloud Storage I/O Benchmarking Guide September 05, 2014 Copyright 1999-2014 Parallels IP Holdings GmbH and its affiliates. All rights reserved. Parallels IP Holdings

More information

File System & Device Drive. Overview of Mass Storage Structure. Moving head Disk Mechanism. HDD Pictures 11/13/2014. CS341: Operating System

File System & Device Drive. Overview of Mass Storage Structure. Moving head Disk Mechanism. HDD Pictures 11/13/2014. CS341: Operating System CS341: Operating System Lect 36: 1 st Nov 2014 Dr. A. Sahu Dept of Comp. Sc. & Engg. Indian Institute of Technology Guwahati File System & Device Drive Mass Storage Disk Structure Disk Arm Scheduling RAID

More information

Q & A From Hitachi Data Systems WebTech Presentation:

Q & A From Hitachi Data Systems WebTech Presentation: Q & A From Hitachi Data Systems WebTech Presentation: RAID Concepts 1. Is the chunk size the same for all Hitachi Data Systems storage systems, i.e., Adaptable Modular Systems, Network Storage Controller,

More information

Guide to SATA Hard Disks Installation and RAID Configuration

Guide to SATA Hard Disks Installation and RAID Configuration Guide to SATA Hard Disks Installation and RAID Configuration 1. Guide to SATA Hard Disks Installation... 2 1.1 Serial ATA (SATA) Hard Disks Installation... 2 2. Guide to RAID Configurations... 3 2.1 Introduction

More information

Block1. Block2. Block3. Block3 Striping

Block1. Block2. Block3. Block3 Striping Introduction to RI Team members: 電 機 一 94901150 王 麒 鈞, 電 機 一 94901151 吳 炫 逸, 電 機 一 94901154 孫 維 隆. Motivation Gosh, my hard disk is broken again, and my computer can t boot normally. I even have no chance

More information

Performance Analysis of RAIDs in Storage Area Network

Performance Analysis of RAIDs in Storage Area Network Performance Analysis of RAIDs in Storage Area Network Sneha M. Assistant Professor, Department of Computer Science and Engineering, R V College of Engineering Bengaluru-560059 ABSTRACT Direct Attached

More information

What is RAID? data reliability with performance

What is RAID? data reliability with performance What is RAID? RAID is the use of multiple disks and data distribution techniques to get better Resilience and/or Performance RAID stands for: Redundant Array of Inexpensive / Independent Disks RAID can

More information

RAID Storage System of Standalone NVR

RAID Storage System of Standalone NVR ACTi Knowledge Base Category: Installation & Configuration Note Sub-category: Hardware; Recording & Storage Model: XNR-4200, GNR-2000 Firmware: Software: Author: Ando.Meritee Published: 2011/05/04 Reviewed:

More information

Exercise 2 : checksums, RAID and erasure coding

Exercise 2 : checksums, RAID and erasure coding Exercise 2 : checksums, RAID and erasure coding Sébastien Ponce May 22, 2015 1 Goals of the exercise Play with checksums and compare efficiency and robustness Use hand written version of RAID systems and

More information

REMOTE OFFICE BACKUP

REMOTE OFFICE BACKUP REMOTE OFFICE BACKUP Remote offices present IT managers with a number of technical challenges. Often businesses use remote offices to expand into either new markets, or into new geographical areas. Many

More information

RAID. Contents. Definition and Use of the Different RAID Levels. The different RAID levels: Definition Cost / Efficiency Reliability Performance

RAID. Contents. Definition and Use of the Different RAID Levels. The different RAID levels: Definition Cost / Efficiency Reliability Performance RAID Definition and Use of the Different RAID Levels Contents The different RAID levels: Definition Cost / Efficiency Reliability Performance Further High Availability Aspects Performance Optimization

More information

Overview of I/O Performance and RAID in an RDBMS Environment. By: Edward Whalen Performance Tuning Corporation

Overview of I/O Performance and RAID in an RDBMS Environment. By: Edward Whalen Performance Tuning Corporation Overview of I/O Performance and RAID in an RDBMS Environment By: Edward Whalen Performance Tuning Corporation Abstract This paper covers the fundamentals of I/O topics and an overview of RAID levels commonly

More information

I-Motion SQL Server admin concerns

I-Motion SQL Server admin concerns I-Motion SQL Server admin concerns I-Motion SQL Server admin concerns Version Date Author Comments 4 2014-04-29 Rebrand 3 2011-07-12 Vincent MORIAUX Add Maintenance Plan tutorial appendix Add Recommended

More information

Architecting High-Speed Data Streaming Systems. Sujit Basu

Architecting High-Speed Data Streaming Systems. Sujit Basu Architecting High-Speed Data Streaming Systems Sujit Basu stream ing [stree-ming] verb 1. The act of transferring data to or from an instrument at a rate high enough to sustain continuous acquisition or

More information

an analysis of RAID 5DP

an analysis of RAID 5DP an analysis of RAID 5DP a qualitative and quantitative comparison of RAID levels and data protection hp white paper for information about the va 7000 series and periodic updates to this white paper see

More information

EMC Backup and Recovery for Microsoft SQL Server 2008 Enabled by EMC Celerra Unified Storage

EMC Backup and Recovery for Microsoft SQL Server 2008 Enabled by EMC Celerra Unified Storage EMC Backup and Recovery for Microsoft SQL Server 2008 Enabled by EMC Celerra Unified Storage Applied Technology Abstract This white paper describes various backup and recovery solutions available for SQL

More information

RAID Technology. RAID Overview

RAID Technology. RAID Overview Technology In the 1980s, hard-disk drive capacities were limited and large drives commanded a premium price. As an alternative to costly, high-capacity individual drives, storage system developers began

More information

PARALLEL I/O FOR HIGH PERFORMANCE COMPUTING

PARALLEL I/O FOR HIGH PERFORMANCE COMPUTING o. rof. Dr. eter Brezany arallele and Verteilte Datenbanksysteme 1 ARALLEL I/O FOR HIGH ERFORMANCE COMUTING Skriptum zur Vorlesung eter Brezany Institut für Scientific Computing Universität Wien E-Mail:

More information

Linux Software Raid. Aug 2010. Mark A. Davis

Linux Software Raid. Aug 2010. Mark A. Davis Linux Software Raid Aug 2010 Mark A. Davis a What is RAID? Redundant Array of Inexpensive/Independent Drives It is a method of combining more than one hard drive into a logic unit for the purpose of: Increasing

More information