Storage Virtualization in Cloud
|
|
- Olivia Short
- 7 years ago
- Views:
Transcription
1 Storage Virtualization in Cloud Cloud Strategy Partners, LLC Sponsored by: IEEE Educational Activities and IEEE Cloud Computing
2 Course Presenter s Biography This IEEE Cloud Computing tutorial has been developed by Cloud Strategy Partners, LLC. Cloud Strategy Partners, LLC is an expert consultancy firm that specializes in Technology and Strategy relating to Cloud Computing. IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 2 / 16
3 Course Summary In this tutorial, we will review the various ways that Storage is virtualized and implemented for large scale, distributed systems, including Cloud. We will also discuss the storage primitive which was virtualized, and see that some systems concentrate on virtualizing files, and some systems concentrate on virtualizing blocks. Next we will review how the virtualization function can run in a variety of places in the architecture. It can run in the host, in the network, or all the way back where the drives are. We also will discuss how virtualization can be placed in band of the storage operations, and for scale, is usually placed out of band. Finally, we will look at several of the New file systems optimized for managing heterogeneous cloud storage farms. IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 3 / 16
4 Outline As you can see from the slide, we will be covering several areas relating to Storage virtualization in Cloud IaaS; Storage virtualization techniques Storage virtualization layers; Approaches to storage virtualization; Storage types in cloud; Virtualized file systems for cloud. Storage Virtualization This slide illustrates Virtualization techniques in Cloud IaaS. At the top the General Virtualization Technique is illustrated. Underneath, one can visualize how the general technique is applied to several virtualization problems. For Server virtualization, hypervisors are extensively used. For storage virtualization, many software techniques including volume management, file systems, and replication are applied. For network virtualization, there are many different features including link aggregation, VPN, and also firewall, switching, routing, and application filtering and load balancing have virtual capability in many cloud implementations today. Outline As we have seen from previous lessons, virtualization of any kind of resource follows a common blueprint. The physical hardware is put under a special kind of software control which abstracts the physical layer, and presents to the user what looks like the actual resource but is actually a virtual instance of that resource. Storage follows this blueprint. The hardware and the hardware interfaces are virtualized by a software layer as shown in the illustration, thus presenting virtual storage primitives (disks, file systems, etc.) to the consumer. Common Storage Architectures In clouds, there are many options for implementing physical storage, because, the virtual storage interfaces can be kept to a small set, with the software providing common interfaces. The illustration shows three kinds of physical storage: DAS -Direct Attached Storage NAS - Network Attached Storage SAN -Storage Area Network. IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 4 / 16
5 Once can see by the diagram, that the underlying architectures of these different schemes are quite different. The applications interface (legacy, non-virtual) all look the same basically an application accesses a file system. Under the hood, the connectivity of the components and where the file system code actually runs can be at any number of locations (as shown) The software layer which is implementing the virtualized storage, can also enhance the storage model offered to beyond that which physical storage can accomplish. Manageability Virtualized storage resource are easier to configure and manage Scalability virtualization simplifies storage resources scalability. Availability virtualization simplifies protecting against storage hardware failures and overloading Storage redundancy, backup and load balancing are part of the distributed cloud storage. Security Virtualized storage instances provide additional security by storage segments isolation The illustration on this slide goes into more depth as to how the virtualization layers for storage work, and what kind of storage model they present. Properties of Storage Virtualization The software layer for storage is in kernel space in the operating system, where it can intercept the disk and file system primitives and insert the capability of utilizing external, networked storage, and storage built from replicating, distributed drives. The most common storage models are file system and block device. While the names imply the capability of the models, the Cloud OS may not provide the exact same capabilities as a standard let s say Linux file system or block device. We will discuss more on this later. File System Level Virtualization What is file system A file system is a software layer responsible for organizing and policing the creation, modification, and deletion of files File systems provide a hierarchical organization of files into directories and subdirectories The B-tree algorithm facilitates more rapid search and retrieval of files by name File system integrity is maintained through duplication of master tables, change logs, and immediate writes off file changes IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 5 / 16
6 Different file systems In Unix, the super block contains information on the current state of the file system and its resources. In Windows NTFS, the master file table contains information on all file entries and status. File Metadata The control information for file management is known as metadata. File metadata includes file attributes and pointers to the location of file data content. File metadata may be segregated from a file's data content. Metadata on file ownership and permissions is used in file access. File timestamp metadata facilitates automated processes such as backup and life cycle management. Different file systems In Unix systems, file metadata is contained in the i-node structure. In Windows systems, file metadata is contained in records of file attributes. Block Device Level Virtualization Block Device Level virtualization is a low level technique which creates a volume pool from a collection of drives. It presents virtualized storage primitives called LUN for Logical Unit Identifier, and an offset within that LUN, which known as a Logical Block Address (LBA) This is illustrated in the slide Block Device Level: Logical Unit and Logical Volume Block level data The file system block The atomic unit of file system management is the file system block. A file's data may span multiple file system blocks. IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 6 / 16
7 A file system block is composed of a consecutive range of disk block addresses. Data in disk Disk drives read and write data to media through cylinder, head, and sector geometry. Microcode on a disk translates between disk block numbers and cylinder/head/sector locations. This translation is an elementary form of virtualization. Block device level interface: SCSI (Small Computer System Interface) The exchange of data blocks between the host system and storage is governed by the SCSI protocol. Storage Interconnection Drives are not always local to the server, and therefore astorage Interconnection is utilized. This illustration shows that the path to storage includes multiple layers of physical and logical data transformation The storage interconnection provides the data path between servers and storage The storage interconnection is composed of both hardware and software components Approaches to Storage Virtualization Abstracting physical storage Physical to virtual The cylinder, head and sector geometry of individual disks is virtualized into logical block addresses (LBAs).For storage networks, the physical storage system is identified by a network address / LUN pair.combining RAID and JBOD assets to create a virtualized mirror must accommodate performance differences. Metadata integrity Storage metadata integrity requires redundancy for failover or load balancing. Virtualization intelligence may need to interface with upper layer applications to ensure data consistency. Host-based Virtualization Important issues IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 7 / 16
8 Storage metadata servers Storage metadata may be shared by multiple servers. Shared metadata enables a SAN file system view for multiple servers. Provides virtual to real logical block address mapping for client. A distributed SAN file system requires file locking mechanisms to preserve data integrity. Host-based storage APIs May be implemented by the operating system to provide a common interface to disparate virtualized resources. Microsoft's virtual disk service (VDS) provides a management interface for dynamic generation of virtualized storage. Host-based Virtualization: Example An additional layer of abstraction and control can be run on each host, and is called Logical Volume Manager (LVM). This code runs on the host and front-ends all kinds of back end storage resources. The use cases and the architecture are shown on the slide. Host-based Virtualization: Pros and Cons Host based storage virtualization has gotten very popular in server machines because no additional hardware or infrastructure requirements Simple to design and implement Improve storage utilization However Storage utilization optimized only on a per host base Software implementation is depending on each operating system Consume CPU cycles for virtualization As we all know, NFS is very popular, and it is a form of Host based storage virtualization Network-based Virtualization Animation illustrates Fabric switch should provide Connectivity for all storage transactions Interoperability between disparate servers, operating systems, and target devices FAIS ( Fabric Application Interface Standard ) Define a set of standard APIs to integrate applications and switches. FAIS separates control information and data paths. The control path processor (CPP) supports the FAIS APIs and upper layer storage virtualization application. The data path controller (DPC) executes the virtualized SCSI I/Os under the management of one or more CPPs Network-based Virtualization: Pros and Cons Network-based Virtualization come with plus and minus factors as well True heterogeneous storage virtualization No need for modification of host or storage system Multi-path technique improve the access performance However Complex interoperability matrices -limited by vendors support Difficult to implement fast metadata updates in switch device Usually require IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 8 / 16
9 to build specific network equipment (e.g., Fibre Channel) IBM SVC ( SAN Volume Controller )is an example Storage-based Virtualization This animation illustrates how the different layers in a storage system perform their function. In the first part of the animation one can see a mode where the underlying virtualized storage provides a virtual filesystem interface; the connected Operating Systems send the files there to get saved. The virtualized storage takes care of replicating the file across actual drives in the cloud for high durability. In the second part of the animation, on can see the mode where the underlying virtualized storage presents and block level interface. Here, the application is running on an OS, which presents a local file system interface. The operating system deconstructs, through the file system code, the save into a series of blocks which need to be written. The blocks go to the virtualization layer in this case, which stores and replicates at the block level. Storage-based Virtualization: Pros and Cons Storage virtualization is extremely useful On the one hand Provide most of the benefits of storage virtualization Reduce additional latency to individual IO However Storage utilization optimized only across the connected controllers Replication and data migration only possible across the connected controllers and the same vendors devices In-band Virtualization Storage virtualization can be implemented in a number of ways. The animation in this slide shows what is called In-Band virtualization, Also known as symmetric, virtualization devices actually sit in the data path between the host and storage. Hosts perform IO to the virtualized device and never interact with the actual storage device. While Easy to implement it has Bad scalability & Bottle neck characteristics IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 9 / 16
10 Out-of-band Virtualization The animation in this illustration shows out of band virtualization Also known as asymmetric, virtualization devices are sometimes called metadata servers. It Requires additional software in the host which knows the first request location of the actual data. While a good architecture for Scalability & Performance It is Hard to implement Storage Types in Cloud (1) Block storage Block storage is a type of data storage where data is stored in blocks, also referred to as volumes. Each block is treated as individual disk drive and can contain multiple files. In this way, block storage provides a good abstraction for physical storage devices and well suited for most of file systems. In cloud, VM instance is often provisioned with the attached block storage of the configured or requested size. Object storage Storage architecture that manages data as objects. Each object contains data, metadata and accessed via a globally unique identifier, typically in a form of URI or URL. Object storage systems use namespace that is consistent across multiple physical devices. Object storage systems usually include such additional services as data replication and distribution, and may also support application specific access protocols and data management. As an example, object storage infrastructure is used by Dropbox for storing files and Facebook for storing photos. Storage Types in Cloud (2) Bucket storage Bucket storage is a storage organisation where data objects are stored in the basic containers and using single global namespace, where data can be accessed with their own methods. Bucket storage type is used by Amazon S3 and Google. Blob storage Blob storage represents a generic key-value data store, often designed for storing large data objects. A blob (short for binary large object) is a collection of binary data stored as a single entity in a database management system. Blob storage is used in Microsoft Azure cloud. Virtualized File Systems for Cloud We will look closer at some popular file systems for cloud. IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 10 / 16
11 Each of them has a number of benefits when implemented in specific cloud environment. Most applications need filesystems because servers have filesystems. Most filesystems need block storage upon which to mount. Therefore it is no surprise that filesystems with underlying block storage are popular cloud storage options. A simple approach is to use server or NAS based technologies extended directly to the cloud for the block filesystems. That is tie a number of drives together, and maybe access across the network. As th slide lists, LVM, RAID, and NFS are all examples of virtualized filesystems on block storage commonly found in smaller clouds. Larger clouds utilize distributed file systems, which have a high degree of redundancy across the cloud they are serving. As the slide shows, they can be file or object based and there are many examples of both popular in cloud implementations. HDFS (Hadoop Distributed File System) is specifically designed for large scale data processing on massively parallel clusters. Can be used in cloud for high performance data input/output, in particular for CDN (Content Distribution Network) Logical Volume Management (LVM) Logical Volume Manager is very popular because it is commonly found in Linux. It implements block level host-based virtualization approach Allows disks to be added or replaced without downtime and service disruption. Supports file systems extension and dynamic re-sizing, data backup, creation and dynamic resizing of logical volumes Suitable for managing large disk farms Logical Volume Management Architecture This slide illustrates the Logical Volume Management Architecture Tools and utilities are in user space Device mapper framework implements a Linux kernel driver for different mappings Logical Volume Management Implementation Logical Volume Management Implementation LVM project is implemented in two components: In user space Based on FUSE (Filesystem in Userspace) In kernel space which Implements device mapper framework IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 11 / 16
12 LVM implementation using FUSE This slide diagrams the Logical Volume Manager using Filesystem in User Space As can be seen by the diagram, the key is a that the Loadable kernel module FUSE provides a bridge to actual kernel interfaces LVM Implementation in Kernel Space For performance reasons, there is an implementation available of LVM in kernel space This slide speaks to the system calls when implemented this way Redundant Array of Independent Disks (RAID) Another common scheme for virtualizing storage is RAID (Redundant Array of Independent Disks) RAID is a software layer which groups together disks and implements various levels of replication and distribution of data across the drives. RAID schemes provide different balance between the key goals: Reliability, Availability, Performance, Capacity The slide speaks to the Difference in common RAID schemes RAID0, RAID1, RAID1+0, RAID5, and RAID5+0 Network File System (NFS) Filesystems have characteristic which application developers have come to depend on. For example, when the write call returns from the kernel to the user application, the user application assumes that the data is actually written, or at least that a subsequent read of the same data will return what was just written. These behavioral assumptions are part of the POSIX specification. When clusters of disks are used, and when filesystems ae exported across the network, it becomes challenging to live up to all of the POSIX filesystem requirements. Network filesystems which had correct behavior became very popular. The SUN Network File System (NFS) was one of the first and most reliable POSIX-compliant distributed file systems. As the slide lists, NFS is specified by a number of RFCs and the protocols use to implement NFS are well known and understood. Ove the years NFS has become a go to network filesystem. IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 12 / 16
13 Notably, NFS has been extended to support clustered server deployments including scalable parallel access to files distributed among multiple servers (pnfs extension). This technology is a key part of for example the IBM private cloud implementation. Lustre Lustre is a type of parallel distributed file system used for large amount of data and can work with large computer clusters. Lustre name is derived from two words "Linux" and "cluster". Lustre is often used as a file system for supercomputers and multi-site computer clusters. Lustre storage cluster may contain thousands of nodes and Petabytes of storage volume. Lustre architecture includes three main components: metadata servers that stores filesystem information (files and directories) as well as access rights, object storage servers, and clients that access and use data. Lustre uses unified namespace compatible with POSIX semantics. Lustre File System Architecture Components The main file system components inside of Lustre are described in this slide. Note one of the most significant elements of the design is the notion of many Object Storage Servers. This lends to the scalability of the design. Lustre Cluster at Scale This slide illustrates the scalability design introduced in the previous slide. In general, highly available and high scalability concepts are both used in large deployments. Here a Lustre deployment at scale is illustrated showing multiple networks between clients and the Lustre cluster, and also the number of I/O Servers (paired as fail over groups). Lustre File System in HPC: Examples The high performance computing community has been working on pushing the limits of performance and scalability and Lustre has achieved popularity within that community. Lustre has significant momentum in the HPC community and is actually the leading distributed filesystem for those systems, as the slide details. You can see impressive scale-out and high performance numbers achieved. IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 13 / 16
14 Ceph Ceph is an example of fully distributed storage architecture (not having central management) and the file system designed to integrate object, block and file storage servers from a single distributed computer cluster. Ceph distributed object storage is built around the Reliable Autonomic Distributed Object Store (RADOS) that support data replication. Ceph block storage can be directly mounted to a VM and provides automatic data replication across the storage cluster. Ceph file system runs on top of the object or block storage and maps file names and directories across RADOS cluster. Ceph Architecture and Design Ceph has three components Clients: Near-POSIX file system interface Cluster of OSDs: Store all data and metadata Metadata server (MDS) cluster : Manage namespace (file names) It is designed for high availability and scalability using key design patterns: Separating data and metadata Dynamic distributed metadata management Reliable Autonomic Distributed Object Storage Ceph Architecture The illustration in this slide shows the Ceph architecture. Ceph separates data and metadata operations Data/file request includes request to MDS to obtain file components/inodes location and metadata Ceph Operation on Request Ceph uses an effective client synchronization model. The client makes a request to the Metadate Server which translates the file name into inode (inode number, file owner, mode, size, ) Then the CRUSH (Controlled Replication Under Scalable Hashing) module goes to work. CRUSH is A scalable pseudo-random data distribution function designed for distributed object-based storage systems Maps objects to Placement groups (PGs) using a simple hash function It returns inode number, map file data into objects. The client then accesses the Object Storage Device, as can be seen by the illustration. IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 14 / 16
15 Gluster Gluster storage and files system is an Open Source platform for scale-out public and private cloud storage. Similar to Lustre, the Gluster name is derived from two words GNU and cluster. Gluster aggregates heterogeneous storage server connected over Ethernet or Infiniband network. The Gluster file system provides simple functionality and leave all file management functionality to clients. Gluster Architecture This slide illustrates the Gluster architecture. Gluster accesses a variety of physical storage devices, from Direct Attached, to JBOD, to SAN, and creates a global namespace across them. It also puts a virtualization layer across them, providing a variety of filesystem models, such as NFS or CIFS or WebDAV up to the clients. Gluster has been widely used in many cloud distributions. The slide lists many of them. Gluster is the standard storage system used in Red Hat s OpenStack distribution One can also use Gluster easily in Amazon with the available AMI Hadoop Distributed File System (HDFS) The Hadoop Distributed File System (HDFS) is a very different sort of filesystem optimized for a specific class of applications, those are Map Reduce and similar Big Data systems like no-sql databases. It is a scalable distributed file system for large scale data analysis A part of the Open Source Apache Hadoop suite The primary storage used by Hadoop MapReduce applications Can run on commodity hardware assuring high fault-tolerant HDFS Architecture HDFS cluster consists of a single master node/server that runs NameNode and multiple DataNodes, usually one per physical node in the Hadoop cluster. User data are stored in the files, externally they are exposed through namespace managed by the NameNode. To access a file, a user client needs to request a file location or metadata from the NameNode, and after that it can send read or write request to the DataNode directly. DataNodes create data blocks and do replication based on instructions from NameNode. IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 15 / 16
16 Summary and Take Away This tutorial has explored the various ways that Storage is virtualized and implemented for large scale, distributed systems, including Cloud. We explored the storage primitive which was virtualized, and saw that some systems concentrate on virtualizing files, and some systems concentrate on virtualizing blocks. We saw that the virtualization function can run in a variety of places in the architecture. It can run in the host, in the network, or all the way back where the drives are. We saw that virtualization can be placed in band of the storage operations, and for scale, is usually placed out of band. There are many types of storage primitives which, after all the virtualization has occurred, end up getting exposed to applications. Objects (buckets, blobs), blocks, or file systems There are many ways to layer the filesystem in leveraging the virtualization and replication in a cluster. The capability can be close to the operating system such as LVM or across the network like NFS. Finally, we took a hard look at several of the New file systems optimized for managing heterogeneous cloud storage farms IEEE elearning Library Storage Virtualization in Cloud Transcript pg. 16 / 16
Introduction to Gluster. Versions 3.0.x
Introduction to Gluster Versions 3.0.x Table of Contents Table of Contents... 2 Overview... 3 Gluster File System... 3 Gluster Storage Platform... 3 No metadata with the Elastic Hash Algorithm... 4 A Gluster
More informationEnterprise Storage Solution for Hyper-V Private Cloud and VDI Deployments using Sanbolic s Melio Cloud Software Suite April 2011
Enterprise Storage Solution for Hyper-V Private Cloud and VDI Deployments using Sanbolic s Melio Cloud Software Suite April 2011 Executive Summary Large enterprise Hyper-V deployments with a large number
More informationDesigning a Cloud Storage System
Designing a Cloud Storage System End to End Cloud Storage When designing a cloud storage system, there is value in decoupling the system s archival capacity (its ability to persistently store large volumes
More informationPolyServe Matrix Server for Linux
PolyServe Matrix Server for Linux Highly Available, Shared Data Clustering Software PolyServe Matrix Server for Linux is shared data clustering software that allows customers to replace UNIX SMP servers
More informationRed Hat Storage Server Administration Deep Dive
Red Hat Storage Server Administration Deep Dive Dustin L. Black, RHCA Sr. Technical Account Manager Red Hat Global Support Services ** This session will include a live demo from 6-7pm ** Dustin L. Black,
More informationRed Hat Storage Server
Red Hat Storage Server Marcel Hergaarden Solution Architect, Red Hat marcel.hergaarden@redhat.com May 23, 2013 Unstoppable, OpenSource Software-based Storage Solution The Foundation for the Modern Hybrid
More informationBig data management with IBM General Parallel File System
Big data management with IBM General Parallel File System Optimize storage management and boost your return on investment Highlights Handles the explosive growth of structured and unstructured data Offers
More informationStorage Architectures for Big Data in the Cloud
Storage Architectures for Big Data in the Cloud Sam Fineberg HP Storage CT Office/ May 2013 Overview Introduction What is big data? Big Data I/O Hadoop/HDFS SAN Distributed FS Cloud Summary Research Areas
More informationBuilding Storage Service in a Private Cloud
Building Storage Service in a Private Cloud Sateesh Potturu & Deepak Vasudevan Wipro Technologies Abstract Storage in a private cloud is the storage that sits within a particular enterprise security domain
More informationIntroduction to Cloud Computing
Introduction to Cloud Computing Cloud Computing I (intro) 15 319, spring 2010 2 nd Lecture, Jan 14 th Majd F. Sakr Lecture Motivation General overview on cloud computing What is cloud computing Services
More informationHypertable Architecture Overview
WHITE PAPER - MARCH 2012 Hypertable Architecture Overview Hypertable is an open source, scalable NoSQL database modeled after Bigtable, Google s proprietary scalable database. It is written in C++ for
More informationDistributed File Systems
Distributed File Systems Paul Krzyzanowski Rutgers University October 28, 2012 1 Introduction The classic network file systems we examined, NFS, CIFS, AFS, Coda, were designed as client-server applications.
More informationSep 23, 2014. OSBCONF 2014 Cloud backup with Bareos
Sep 23, 2014 OSBCONF 2014 Cloud backup with Bareos OSBCONF 23/09/2014 Content: Who am I Quick overview of Cloud solutions Bareos and Backup/Restore using Cloud Storage Bareos and Backup/Restore of Cloud
More informationSAN Conceptual and Design Basics
TECHNICAL NOTE VMware Infrastructure 3 SAN Conceptual and Design Basics VMware ESX Server can be used in conjunction with a SAN (storage area network), a specialized high speed network that connects computer
More informationHigh Performance Computing OpenStack Options. September 22, 2015
High Performance Computing OpenStack PRESENTATION TITLE GOES HERE Options September 22, 2015 Today s Presenters Glyn Bowden, SNIA Cloud Storage Initiative Board HP Helion Professional Services Alex McDonald,
More informationIP SAN Fundamentals: An Introduction to IP SANs and iscsi
IP SAN Fundamentals: An Introduction to IP SANs and iscsi Updated April 2007 Sun Microsystems, Inc. 2007 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, CA 95054 USA All rights reserved. This
More informationRED HAT STORAGE SERVER TECHNICAL OVERVIEW
RED HAT STORAGE SERVER TECHNICAL OVERVIEW Ingo Börnig Solution Architect, Red Hat 24.10.2013 NEW STORAGE REQUIREMENTS FOR THE MODERN HYBRID DATACENTER DESIGNED FOR THE NEW DATA LANDSCAPE PETABYTE SCALE
More informationSUSE Linux uutuudet - kuulumiset SUSECon:sta
SUSE Linux uutuudet - kuulumiset SUSECon:sta Olli Tuominen Technology Specialist olli.tuominen@suse.com 2 SUSECon 13 4 days, 95 Sessions Keynotes, Breakout Sessions,Technology Showcase Case Studies, Technical
More informationBig Data Storage Options for Hadoop Sam Fineberg, HP Storage
Sam Fineberg, HP Storage SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless otherwise noted. Member companies and individual members may use this material in presentations
More informationProduct Spotlight. A Look at the Future of Storage. Featuring SUSE Enterprise Storage. Where IT perceptions are reality
Where IT perceptions are reality Product Spotlight A Look at the Future of Storage Featuring SUSE Enterprise Storage Document # SPOTLIGHT2013001 v5, January 2015 Copyright 2015 IT Brand Pulse. All rights
More informationUnderstanding Storage Virtualization of Infortrend ESVA
Understanding Storage Virtualization of Infortrend ESVA White paper Abstract This white paper introduces different ways of implementing storage virtualization and illustrates how the virtualization technology
More informationBuilding Storage as a Service with OpenStack. Greg Elkinbard Senior Technical Director
Building Storage as a Service with OpenStack Greg Elkinbard Senior Technical Director MIRANTIS 2012 PAGE 1 About the Presenter Greg Elkinbard Senior Technical Director at Mirantis Builds on demand IaaS
More informationAccelerating and Simplifying Apache
Accelerating and Simplifying Apache Hadoop with Panasas ActiveStor White paper NOvember 2012 1.888.PANASAS www.panasas.com Executive Overview The technology requirements for big data vary significantly
More information70-414: Implementing a Cloud Based Infrastructure. Course Overview
70-414: Implementing a Cloud Based Infrastructure Course Overview This course covers will prepare the student for Exam 70-414: Implementing a Cloud Based Infrastructure. Students will learn how to create
More informationovirt and Gluster Hyperconvergence
ovirt and Gluster Hyperconvergence January 2015 Federico Simoncelli Principal Software Engineer Red Hat ovirt and GlusterFS Hyperconvergence, Jan 2015 1 Agenda ovirt Architecture and Software-defined Data
More informationIBM Global Technology Services September 2007. NAS systems scale out to meet growing storage demand.
IBM Global Technology Services September 2007 NAS systems scale out to meet Page 2 Contents 2 Introduction 2 Understanding the traditional NAS role 3 Gaining NAS benefits 4 NAS shortcomings in enterprise
More informationSwiftStack Filesystem Gateway Architecture
WHITEPAPER SwiftStack Filesystem Gateway Architecture March 2015 by Amanda Plimpton Executive Summary SwiftStack s Filesystem Gateway expands the functionality of an organization s SwiftStack deployment
More informationData-Intensive Programming. Timo Aaltonen Department of Pervasive Computing
Data-Intensive Programming Timo Aaltonen Department of Pervasive Computing Data-Intensive Programming Lecturer: Timo Aaltonen University Lecturer timo.aaltonen@tut.fi Assistants: Henri Terho and Antti
More informationEMC Virtual Infrastructure for Microsoft Applications Data Center Solution
EMC Virtual Infrastructure for Microsoft Applications Data Center Solution Enabled by EMC Symmetrix V-Max and Reference Architecture EMC Global Solutions Copyright and Trademark Information Copyright 2009
More informationApache HBase. Crazy dances on the elephant back
Apache HBase Crazy dances on the elephant back Roman Nikitchenko, 16.10.2014 YARN 2 FIRST EVER DATA OS 10.000 nodes computer Recent technology changes are focused on higher scale. Better resource usage
More informationScientific Computing Data Management Visions
Scientific Computing Data Management Visions ELI-Tango Workshop Szeged, 24-25 February 2015 Péter Szász Group Leader Scientific Computing Group ELI-ALPS Scientific Computing Group Responsibilities Data
More informationIBM Global Technology Services November 2009. Successfully implementing a private storage cloud to help reduce total cost of ownership
IBM Global Technology Services November 2009 Successfully implementing a private storage cloud to help reduce total cost of ownership Page 2 Contents 2 Executive summary 3 What is a storage cloud? 3 A
More informationNetwork Attached Storage. Jinfeng Yang Oct/19/2015
Network Attached Storage Jinfeng Yang Oct/19/2015 Outline Part A 1. What is the Network Attached Storage (NAS)? 2. What are the applications of NAS? 3. The benefits of NAS. 4. NAS s performance (Reliability
More informationIntegrated Application and Data Protection. NEC ExpressCluster White Paper
Integrated Application and Data Protection NEC ExpressCluster White Paper Introduction Critical business processes and operations depend on real-time access to IT systems that consist of applications and
More informationDeployment Guide. How to prepare your environment for an OnApp Cloud deployment.
Deployment Guide How to prepare your environment for an OnApp Cloud deployment. Document version 1.07 Document release date 28 th November 2011 document revisions 1 Contents 1. Overview... 3 2. Network
More informationImplementing the Hadoop Distributed File System Protocol on OneFS Jeff Hughes EMC Isilon
Implementing the Hadoop Distributed File System Protocol on OneFS Jeff Hughes EMC Isilon Outline Hadoop Overview OneFS Overview MapReduce + OneFS Details of isi_hdfs_d Wrap up & Questions 2 Hadoop Overview
More informationMoving Virtual Storage to the Cloud. Guidelines for Hosters Who Want to Enhance Their Cloud Offerings with Cloud Storage
Moving Virtual Storage to the Cloud Guidelines for Hosters Who Want to Enhance Their Cloud Offerings with Cloud Storage Table of Contents Overview... 1 Understanding the Storage Problem... 1 What Makes
More informationOptimizing Large Arrays with StoneFly Storage Concentrators
Optimizing Large Arrays with StoneFly Storage Concentrators All trademark names are the property of their respective companies. This publication contains opinions of which are subject to change from time
More informationSUSE Enterprise Storage Highly Scalable Software Defined Storage. Gábor Nyers Sales Engineer @SUSE gnyers@suse.com
SUSE Enterprise Storage Highly Scalable Software Defined Storage Gábor Nyers Sales Engineer @SUSE gnyers@suse.com Setting the Stage Enterprise Data Capacity Utilization 1-3% 15-20% 20-25% Tier 0 Ultra
More informationMajor Commercial and Research Cloud Service Providers
Major Commercial and Research Cloud Service Providers Cloud Strategy Partners, LLC Sponsored by: IEEE Educational Activities and IEEE Cloud Computing Course Presenter s Biography This IEEE Cloud Computing
More informationChapter 3: Operating-System Structures. System Components Operating System Services System Calls System Programs System Structure Virtual Machines
Chapter 3: Operating-System Structures System Components Operating System Services System Calls System Programs System Structure Virtual Machines Operating System Concepts 3.1 Common System Components
More informationAccelerating Applications and File Systems with Solid State Storage. Jacob Farmer, Cambridge Computer
Accelerating Applications and File Systems with Solid State Storage Jacob Farmer, Cambridge Computer SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless otherwise
More informationA Crash Course in Wide Area Data Replication. Jacob Farmer, CTO, Cambridge Computer
A Crash Course in Wide Area Data Replication Jacob Farmer, CTO, Cambridge Computer SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA. Member companies and individual
More informationAn Alternative Storage Solution for MapReduce. Eric Lomascolo Director, Solutions Marketing
An Alternative Storage Solution for MapReduce Eric Lomascolo Director, Solutions Marketing MapReduce Breaks the Problem Down Data Analysis Distributes processing work (Map) across compute nodes and accumulates
More informationStorage Virtualization
Section 2 : Storage Networking Technologies and Virtualization Storage Virtualization Chapter 10 EMC Proven Professional The #1 Certification Program in the information storage and management industry
More informationHow To Virtualize A Storage Area Network (San) With Virtualization
A New Method of SAN Storage Virtualization Table of Contents 1 - ABSTRACT 2 - THE NEED FOR STORAGE VIRTUALIZATION 3 - EXISTING STORAGE VIRTUALIZATION METHODS 4 - A NEW METHOD OF VIRTUALIZATION: Storage
More informationHBA Virtualization Technologies for Windows OS Environments
HBA Virtualization Technologies for Windows OS Environments FC HBA Virtualization Keeping Pace with Virtualized Data Centers Executive Summary Today, Microsoft offers Virtual Server 2005 R2, a software
More informationStorPool Distributed Storage Software Technical Overview
StorPool Distributed Storage Software Technical Overview StorPool 2015 Page 1 of 8 StorPool Overview StorPool is distributed storage software. It pools the attached storage (hard disks or SSDs) of standard
More informationVirtualizing SQL Server 2008 Using EMC VNX Series and Microsoft Windows Server 2008 R2 Hyper-V. Reference Architecture
Virtualizing SQL Server 2008 Using EMC VNX Series and Microsoft Windows Server 2008 R2 Hyper-V Copyright 2011 EMC Corporation. All rights reserved. Published February, 2011 EMC believes the information
More informationA very short Intro to Hadoop
4 Overview A very short Intro to Hadoop photo by: exfordy, flickr 5 How to Crunch a Petabyte? Lots of disks, spinning all the time Redundancy, since disks die Lots of CPU cores, working all the time Retry,
More informationUsing Multipathing Technology to Achieve a High Availability Solution
Using Multipathing Technology to Achieve a High Availability Solution Table of Contents Introduction...3 Multipathing Technology...3 Multipathing I/O Implementations...5 Storage Redundancy...5 Infortrend
More informationVicom Storage Virtualization Engine. Simple, scalable, cost-effective storage virtualization for the enterprise
Vicom Storage Virtualization Engine Simple, scalable, cost-effective storage virtualization for the enterprise Vicom Storage Virtualization Engine (SVE) enables centralized administration of multi-platform,
More informationHigh Performance Computing (HPC)
High Performance Computing (HPC) High Performance Computing (HPC) White Paper Attn: Name, Title Phone: xxx.xxx.xxxx Fax: xxx.xxx.xxxx 1.0 OVERVIEW When heterogeneous enterprise environments are involved,
More informationPARALLELS CLOUD STORAGE
PARALLELS CLOUD STORAGE Performance Benchmark Results 1 Table of Contents Executive Summary... Error! Bookmark not defined. Architecture Overview... 3 Key Features... 5 No Special Hardware Requirements...
More informationClustering Windows File Servers for Enterprise Scale and High Availability
Enabling the Always-On Enterprise Clustering Windows File Servers for Enterprise Scale and High Availability By Andrew Melmed Director of Enterprise Solutions, Sanbolic, Inc. April 2012 Introduction Microsoft
More informationMoving Virtual Storage to the Cloud
Moving Virtual Storage to the Cloud White Paper Guidelines for Hosters Who Want to Enhance Their Cloud Offerings with Cloud Storage www.parallels.com Table of Contents Overview... 3 Understanding the Storage
More informationCloud Computing Trends
UT DALLAS Erik Jonsson School of Engineering & Computer Science Cloud Computing Trends What is cloud computing? Cloud computing refers to the apps and services delivered over the internet. Software delivered
More informationEMC Virtual Infrastructure for Microsoft SQL Server
Microsoft SQL Server Enabled by EMC Celerra and Microsoft Hyper-V Copyright 2010 EMC Corporation. All rights reserved. Published February, 2010 EMC believes the information in this publication is accurate
More informationTesting of several distributed file-system (HadoopFS, CEPH and GlusterFS) for supporting the HEP experiments analisys. Giacinto DONVITO INFN-Bari
Testing of several distributed file-system (HadoopFS, CEPH and GlusterFS) for supporting the HEP experiments analisys. Giacinto DONVITO INFN-Bari 1 Agenda Introduction on the objective of the test activities
More informationHadoop: Embracing future hardware
Hadoop: Embracing future hardware Suresh Srinivas @suresh_m_s Page 1 About Me Architect & Founder at Hortonworks Long time Apache Hadoop committer and PMC member Designed and developed many key Hadoop
More informationHadoop Architecture. Part 1
Hadoop Architecture Part 1 Node, Rack and Cluster: A node is simply a computer, typically non-enterprise, commodity hardware for nodes that contain data. Consider we have Node 1.Then we can add more nodes,
More informationThe functionality and advantages of a high-availability file server system
The functionality and advantages of a high-availability file server system This paper discusses the benefits of deploying a JMR SHARE High-Availability File Server System. Hardware and performance considerations
More informationWelcome to the unit of Hadoop Fundamentals on Hadoop architecture. I will begin with a terminology review and then cover the major components
Welcome to the unit of Hadoop Fundamentals on Hadoop architecture. I will begin with a terminology review and then cover the major components of Hadoop. We will see what types of nodes can exist in a Hadoop
More informationRADOS: A Scalable, Reliable Storage Service for Petabyte- scale Storage Clusters
RADOS: A Scalable, Reliable Storage Service for Petabyte- scale Storage Clusters Sage Weil, Andrew Leung, Scott Brandt, Carlos Maltzahn {sage,aleung,scott,carlosm}@cs.ucsc.edu University of California,
More informationStorReduce Technical White Paper Cloud-based Data Deduplication
StorReduce Technical White Paper Cloud-based Data Deduplication See also at storreduce.com/docs StorReduce Quick Start Guide StorReduce FAQ StorReduce Solution Brief, and StorReduce Blog at storreduce.com/blog
More informationVirtualizing Microsoft Exchange Server 2010 with NetApp and VMware
Virtualizing Microsoft Exchange Server 2010 with NetApp and VMware Deploying Microsoft Exchange Server 2010 in a virtualized environment that leverages VMware virtualization and NetApp unified storage
More informationLecture 02a Cloud Computing I
Mobile Cloud Computing Lecture 02a Cloud Computing I 吳 秀 陽 Shiow-yang Wu What is Cloud Computing? Computing with cloud? Mobile Cloud Computing Cloud Computing I 2 Note 1 What is Cloud Computing? Walking
More informationRedbooks Redpaper. IBM TotalStorage NAS Advantages of the Windows Powered OS. Roland Tretau
Redbooks Redpaper Roland Tretau IBM TotalStorage NAS Advantages of the Windows Powered OS Copyright IBM Corp. 2002. All rights reserved. ibm.com/redbooks 1 What is Network Attached Storage (NAS) Storage
More informationOperating System Components
Lecture Overview Operating system software introduction OS components OS services OS structure Operating Systems - April 24, 2001 Operating System Components Process management Memory management Secondary
More informationOverview of I/O Performance and RAID in an RDBMS Environment. By: Edward Whalen Performance Tuning Corporation
Overview of I/O Performance and RAID in an RDBMS Environment By: Edward Whalen Performance Tuning Corporation Abstract This paper covers the fundamentals of I/O topics and an overview of RAID levels commonly
More informationWOS Cloud. ddn.com. Personal Storage for the Enterprise. DDN Solution Brief
DDN Solution Brief Personal Storage for the Enterprise WOS Cloud Secure, Shared Drop-in File Access for Enterprise Users, Anytime and Anywhere 2011 DataDirect Networks. All Rights Reserved DDN WOS Cloud
More informationSun Storage Perspective & Lustre Architecture. Dr. Peter Braam VP Sun Microsystems
Sun Storage Perspective & Lustre Architecture Dr. Peter Braam VP Sun Microsystems Agenda Future of Storage Sun s vision Lustre - vendor neutral architecture roadmap Sun s view on storage introduction The
More informationCeph. A file system a little bit different. Udo Seidel
Ceph A file system a little bit different Udo Seidel Ceph what? So-called parallel distributed cluster file system Started as part of PhD studies at UCSC Public announcement in 2006 at 7 th OSDI File system
More informationAnalysis and Research of Cloud Computing System to Comparison of Several Cloud Computing Platforms
Volume 1, Issue 1 ISSN: 2320-5288 International Journal of Engineering Technology & Management Research Journal homepage: www.ijetmr.org Analysis and Research of Cloud Computing System to Comparison of
More informationEMC Celerra Unified Storage Platforms
EMC Solutions for Microsoft SQL Server EMC Celerra Unified Storage Platforms EMC NAS Product Validation Corporate Headquarters Hopkinton, MA 01748-9103 1-508-435-1000 www.emc.com Copyright 2008, 2009 EMC
More informationScala Storage Scale-Out Clustered Storage White Paper
White Paper Scala Storage Scale-Out Clustered Storage White Paper Chapter 1 Introduction... 3 Capacity - Explosive Growth of Unstructured Data... 3 Performance - Cluster Computing... 3 Chapter 2 Current
More informationCloud and Big Data initiatives. Mark O Connell, EMC
Object storage PRESENTATION systems: TITLE GOES the underpinning HERE of Cloud and Big Data initiatives Mark O Connell, EMC SNIA Legal Notice The material contained in this tutorial is copyrighted by the
More informationPART 1: Breaking the Connections
STORAGE VIRTUALIZATION SEMINAR PRESENTATION DOWNLOAD presented by PART 1: Breaking the Connections Storage virtualization is here, breaking the connection between physical storage infrastructure and the
More informationHDFS Under the Hood. Sanjay Radia. Sradia@yahoo-inc.com Grid Computing, Hadoop Yahoo Inc.
HDFS Under the Hood Sanjay Radia Sradia@yahoo-inc.com Grid Computing, Hadoop Yahoo Inc. 1 Outline Overview of Hadoop, an open source project Design of HDFS On going work 2 Hadoop Hadoop provides a framework
More informationObject Storage: A Growing Opportunity for Service Providers. White Paper. Prepared for: 2012 Neovise, LLC. All Rights Reserved.
Object Storage: A Growing Opportunity for Service Providers Prepared for: White Paper 2012 Neovise, LLC. All Rights Reserved. Introduction For service providers, the rise of cloud computing is both a threat
More informationUsing EonStor FC-host Storage Systems in VMware Infrastructure 3 and vsphere 4
Using EonStor FC-host Storage Systems in VMware Infrastructure 3 and vsphere 4 Application Note Abstract This application note explains the configure details of using Infortrend FC-host storage systems
More informationIBM Tivoli Storage Manager Version 7.1.4. Introduction to Data Protection Solutions IBM
IBM Tivoli Storage Manager Version 7.1.4 Introduction to Data Protection Solutions IBM IBM Tivoli Storage Manager Version 7.1.4 Introduction to Data Protection Solutions IBM Note: Before you use this
More informationZadara Storage Cloud A whitepaper. @ZadaraStorage
Zadara Storage Cloud A whitepaper @ZadaraStorage Zadara delivers two solutions to its customers: On- premises storage arrays Storage as a service from 31 locations globally (and counting) Some Zadara customers
More informationSnapshots in Hadoop Distributed File System
Snapshots in Hadoop Distributed File System Sameer Agarwal UC Berkeley Dhruba Borthakur Facebook Inc. Ion Stoica UC Berkeley Abstract The ability to take snapshots is an essential functionality of any
More informationPrivate cloud computing advances
Building robust private cloud services infrastructures By Brian Gautreau and Gong Wang Private clouds optimize utilization and management of IT resources to heighten availability. Microsoft Private Cloud
More informationHGST Virident Solutions 2.0
Brochure HGST Virident Solutions 2.0 Software Modules HGST Virident Share: Shared access from multiple servers HGST Virident HA: Synchronous replication between servers HGST Virident ClusterCache: Clustered
More informationCSE-E5430 Scalable Cloud Computing Lecture 2
CSE-E5430 Scalable Cloud Computing Lecture 2 Keijo Heljanko Department of Computer Science School of Science Aalto University keijo.heljanko@aalto.fi 14.9-2015 1/36 Google MapReduce A scalable batch processing
More informationDevelop a process for applying updates to systems, including verifying properties of the update. Create File Systems
RH413 Manage Software Updates Develop a process for applying updates to systems, including verifying properties of the update. Create File Systems Allocate an advanced file system layout, and use file
More informationHIGHLY AVAILABLE MULTI-DATA CENTER WINDOWS SERVER SOLUTIONS USING EMC VPLEX METRO AND SANBOLIC MELIO 2010
White Paper HIGHLY AVAILABLE MULTI-DATA CENTER WINDOWS SERVER SOLUTIONS USING EMC VPLEX METRO AND SANBOLIC MELIO 2010 Abstract This white paper demonstrates key functionality demonstrated in a lab environment
More informationWHITE PAPER. Permabit Albireo Data Optimization Software. Benefits of Albireo for Virtual Servers. January 2012. Permabit Technology Corporation
WHITE PAPER Permabit Albireo Data Optimization Software Benefits of Albireo for Virtual Servers January 2012 Permabit Technology Corporation Ten Canal Park Cambridge, MA 02141 USA Phone: 617.252.9600 FAX:
More informationOVERVIEW. CEP Cluster Server is Ideal For: First-time users who want to make applications highly available
Phone: (603)883-7979 sales@cepoint.com Cepoint Cluster Server CEP Cluster Server turnkey system. ENTERPRISE HIGH AVAILABILITY, High performance and very reliable Super Computing Solution for heterogeneous
More informationHadoop Distributed File System. T-111.5550 Seminar On Multimedia 2009-11-11 Eero Kurkela
Hadoop Distributed File System T-111.5550 Seminar On Multimedia 2009-11-11 Eero Kurkela Agenda Introduction Flesh and bones of HDFS Architecture Accessing data Data replication strategy Fault tolerance
More informationDeveloping a dynamic, real-time IT infrastructure with Red Hat integrated virtualization
Developing a dynamic, real-time IT infrastructure with Red Hat integrated virtualization www.redhat.com Table of contents Introduction Page 3 Benefits of virtualization Page 3 Virtualization challenges
More informationWeekly Report. Hadoop Introduction. submitted By Anurag Sharma. Department of Computer Science and Engineering. Indian Institute of Technology Bombay
Weekly Report Hadoop Introduction submitted By Anurag Sharma Department of Computer Science and Engineering Indian Institute of Technology Bombay Chapter 1 What is Hadoop? Apache Hadoop (High-availability
More informationSolution Guide Parallels Virtualization for Linux
Solution Guide Parallels Virtualization for Linux Overview Created in 1991, Linux was designed to be UNIX-compatible software that was composed entirely of open source or free software components. Linux
More informationHP StorageWorks MPX200 Simplified Cost-Effective Virtualization Deployment
HP StorageWorks MPX200 Simplified Cost-Effective Virtualization Deployment Executive Summary... 2 HP StorageWorks MPX200 Architecture... 2 Server Virtualization and SAN based Storage... 3 VMware Architecture...
More informationCloud Based Distributed Databases: The Future Ahead
Cloud Based Distributed Databases: The Future Ahead Arpita Mathur Mridul Mathur Pallavi Upadhyay Abstract Fault tolerant systems are necessary to be there for distributed databases for data centers or
More informationTake An Internal Look at Hadoop. Hairong Kuang Grid Team, Yahoo! Inc hairong@yahoo-inc.com
Take An Internal Look at Hadoop Hairong Kuang Grid Team, Yahoo! Inc hairong@yahoo-inc.com What s Hadoop Framework for running applications on large clusters of commodity hardware Scale: petabytes of data
More informationA Review on Cloud Data Storage in Virtual Perspective
A Review on Cloud Data Storage in Virtual Perspective B.Nagarajan #1, Dr.J.Suguna *2 # Ph.D., Research Scholar, Research and Development Centre, Bharathiar University, Coimbatore-621046, India. * Associate
More informationEnabling High performance Big Data platform with RDMA
Enabling High performance Big Data platform with RDMA Tong Liu HPC Advisory Council Oct 7 th, 2014 Shortcomings of Hadoop Administration tooling Performance Reliability SQL support Backup and recovery
More information