Introduction to OpenStack Swift CloudOpen Japan 2014
|
|
- Nicholas Bradley
- 8 years ago
- Views:
Transcription
1 Introduction to OpenStack Swift CloudOpen Japan 2014 Yuji Hagiwara Platform Engineer, NTT DATA Corp. Copyright 2014 NTT DATA Corporation
2 2 Agenda 1.What is Swift? 2.Swift s Latest Information 3.Swift s Future
3 3 Who am I Yuji Hagiwara Platform Engineer, NTT DATA Corp. Since Using OpenStack Since Developing Searching on Swift Demo App for Searching on Swift
4 Background Data Explosion on Enterprise Amount of Unstructured Data has been growing. We need storage with Scalability, Durability, Availability. Amount of Unstructured Data EB or PB scale Growing exponentially Examples of Unstructured Data Media (Images, Videos, Audios) Web Contents Documents Backups/Archives Where should we store these data? One of the Solutions is Swift. Copyright 2014 NTT DATA Corporation 4
5 What is Swift? Swift is... A storage system with Scalability, Durability, Availability. The REST-ful Distributed Object Storage likely Amazon S3. One of OpenStack Core Components. Implemented by Python. A Open Source Software. 1 Block Storage (Cinder) 2 Object Storage (Swift) Copyright 2014 NTT DATA Corporation 5
6 6 Usage so simple. $ curl -XPUT $ curl XGET $ curl XDELETE
7 7 Use cases of Swift
8 8 Swift as a storage for a variety of applications System Backup CMS FTP-like use Digital Distribution Web Apps Cyber Duck REST API Swift
9 9 OpenStack Swift deployments and use cases Name of enterprise Product/ service Description Rackspace(USA) Korean Telecom (Sourth Korea) Cloud Files ucloud storage service Cloud file share service by Rackspace itself. They use same code as OSS except for features such as authentication, Accounting and CDN (<500PB) Object storage service using OpenStack/Swift (16PB+ size) Sina (Republic of China) Sina App Engine(SAE) Public storage service. They moved to OpenStack from another technology MongoDB in San Diego Supercomputer Center (USA) SDSC Cloud Storage Services Cloud storage service on SDSC. Users can select Amazon/S3 or Rackspace Swift. SME Storage (USA) SMEStorage Open Cloud Platform Cloud storage service based on Rackspace Cloud File SoftLayer (USA) SoftLayer Object Storage Public object storage service. Acquisition by IBM SwiftStack(USA) Swift Stack Provide professional service and Operation and management product HP(USA) HP Cloud Private cloud storage service uses OpenStack. Wikimedia(USA) Wikimedia storage Media files store for Wikipedia. NII(JAPAN) Academic Cloud service Academic cloud service by National Institute of Informatics in Japan (NII)(Integrated and supported by NTT Data)
10 10 Inside Swift
11 11 Architecture: Nodes Swift consist of 2-type Nodes: Proxy Node and Storage Node. Application Forward Data to node HTTP Load balancer Proxy Node Proxy Node Proxy Node Store data Storage Node Storage Node Storage Node Storage Node
12 12 Architecture: The Ring The Ring (static table for data allocation on storage node) decide the optimal Storage Node by Name. Application HTTP Load balancer Proxy Node Proxy Node Proxy Node Ring Ring Ring Storage Node Storage Node Storage Node Storage Node Ring Ring Ring Ring
13 13 Architecture: The role of Ring If you requested to Store the data A, 3 Replica nodes store the data A. data Application HTTP Load balancer Proxy Node Proxy Node Proxy Node A must be located at 1, 2, 4 Ring Ring Ring Storage Node 1 Storage Node 2 Storage Node 3 Storage Node 4 Data A Data A Data B Data A Data B Data B Ring Ring Ring Ring
14 14 Architecture: The role of Ring If you requested to Get the data A, One of Nodes reply the data A. data Application data HTTP Load balancer Proxy Node Proxy Node Proxy Node Ring Ring Ring A must be located at 1, 2, 4 Storage Node 1 Storage Node 2 Storage Node 3 Storage Node 4 Data A Data A Data B Data A Data B Data B Ring Ring Ring Ring
15 15 Scalability (1) Expand proxy server Throughput (2)Expand Storage servers or disks volume More Throughput Proxy Proxy (expand) Proxy Proxy Storage Storage Storage Storage Storage Storage Storage (Expand) More Volume
16 16 Many processes working together containersync proxyserver Swift objectreplicator accountserver objectauditor objectupdater objectserver objectexpirer accountreplicator accountauditor accountreaper containerserver containerreplicator containerauditor containerupdater
17 17 Disk Nor mal Node 1 Node 2 Node 3 Node 4 (1) Each nodes checks data in others Node 1 Node 2 Node 3 Node 4 (2) Disk defeat (3) Detect disk trouble Defe at Node 1 Node 2 Node 3 Node 4 (4) Copy data to another node Node 1 Node 2 Node 3 Node 4 (5) Recover disk (6) recover data to original node Reco very Node 1 Node 2 Node 3 Node 4 (7)Delete temporal data
18 18 Disk Nor mal Node 1 Node 2 Node 3 Node 4 (1) Each nodes checks data in others Node 1 Node 2 Node 3 Node 4 (2) Disk defeat (3) Detect disk trouble Defe at Node 1 Node 2 Node 3 Node 4 (4) Copy data to another node Node 1 Node 2 Node 3 Node 4 (5) Recover disk (6) recover data to original node Reco very Node 1 Node 2 Node 3 Node 4 (7)Delete temporal data
19 19 Normal state Each Data has replicated. Application HTTP Load balancer Proxy Node Proxy Node Proxy Node Ring Ring Ring Ring Storage Node Storage Node Storage Node Storage Node Server Server Server Server Data A Data A Data B Data A Data B Data B Auditor Auditor Auditor Auditor Ring Ring Ring
20 20 Disk Nor mal Node 1 Node 2 Node 3 Node 4 (1) Each nodes checks data in others Node 1 Node 2 Node 3 Node 4 (2) Disk defeat (3) Detect disk trouble Defe at Node 1 Node 2 Node 3 Node 4 (4) Copy data to another node Node 1 Node 2 Node 3 Node 4 (5) Recover disk (6) recover data to original node Reco very Node 1 Node 2 Node 3 Node 4 (7)Delete temporal data
21 21 Defeat state If a disk is broken... Application HTTP Load balancer Proxy Node Proxy Node Proxy Node Ring Ring Ring Ring Storage Node Storage Node Storage Node Storage Node Server Server Server Server Data A Data A Data B Data A Broken Data B Data B Auditor Auditor Auditor Auditor Ring Ring Ring
22 Defeat state detects the lost data and replicates the data to another node for temporary. Application HTTP Load balancer Proxy Node Proxy Node Proxy Node Ring Ring Ring Ring Storage Node Storage Node Storage Node Storage Node Server Server Server Server Data A Data B Data A Data B Data A Broken Data B Auditor Auditor Auditor Auditor Ring Ring Ring Copyright 2014 NTT DATA Corporation Temporary data When detect a lost data, Replicate the data. 22
23 23 Disk Nor mal Node 1 Node 2 Node 3 Node 4 (1) Each nodes checks data in others Node 1 Node 2 Node 3 Node 4 (2) Disk defeat (3) Detect disk trouble Defe at Node 1 Node 2 Node 3 Node 4 (4) Copy data to another node Node 1 Node 2 Node 3 Node 4 (5) Recover disk (6) recover data to original node Reco very Node 1 Node 2 Node 3 Node 4 (7)Delete temporal data
24 24 Recovery state When the broken disk is replaced to a fresh disk... Application HTTP Load balancer Proxy Node Proxy Node Proxy Node Ring Ring Ring Ring Storage Node Storage Node Storage Node Storage Node Server Server Server Server Data A Data B Data A Data B Data A Data B Auditor Auditor Auditor Auditor Ring Ring Ring
25 Recovery state replicates the data and removes the temporary data. Application HTTP Load balancer Proxy Node Proxy Node Proxy Node Ring Ring Ring Ring Storage Node Storage Node Storage Node Storage Node Server Server Server Server Data A Data B Data A Data B Data A Data B Auditor Auditor Auditor Auditor Ring Ring Ring Copyright 2014 NTT DATA Corporation Removed Replicate the data to the correct node. 25
26 26 Disk Nor mal Node 1 Node 2 Node 3 Node 4 (1) Each nodes checks data in others Node 1 Node 2 Node 3 Node 4 (2) Disk defeat (3) Detect disk trouble Defe at Node 1 Node 2 Node 3 Node 4 (4) Copy data to another node Node 1 Node 2 Node 3 Node 4 (5) Recover disk (6) recover data to original node Reco very Node 1 Node 2 Node 3 Node 4 (7)Delete temporal data
27 27 Latest Information
28 28 History and Trend of Community Start OpenStack Project Grizzly Icehouse st release "Austin" History of OpenStack Havana Juno Now Hot Topics on Now Erasure Coding Storage Policy Fundamental Global Cluster Development Trend in Swift Timeline in each functions Developing Supported
29 29 Latest info: Erasure Coding Replication Erasure Coding Data Data Data distribute Data Data Partial Data Partial Data distribute Parity 1 Parity 2 Size 3x original 2x original
30 30 Latest info: Storage Policy Before Data Data Data After Data Data Data Data Data Data Partial Data Partial Data Parity 1 Parity 2 Data Data Data Data Data Same Policy on cluster Variety Policy on cluster
31 31 2 concepts: Integrated Searchable Storage Intelligent Resource Management Future Direction of Swift
32 32 Integrated Searchable Storage Swift should be integrated with Searching. It means to need searching as Scalable, Durable, Available as Swift. Users Operators Store Get Swift Storage System Managers Search
33 Use cases of Search 1.Content Search 2.Detection for de-duplication 3.Tiered storage Data Hash Already Stored? Data Data Hash Hash Data Hash Copyright 2014 NTT DATA Corporation Hot Content More Modified Cheap Storage Cold Content Less Modified Modifed Date is older than 05/20/2014? 33
34 34 Future: Integrated Searchable Storage How do we implement? Internal External Search Swift Storage feature Store Index Search feature Search Hook Swift Storage feature Store Index Search preexist Search Engine Where do search Internal Swift with search library (such as Lucene) External Search Engine (such as Solr) Redundancy High Depend on Search engine Availability High Depend on Search engine Scalability High Depend on Search engine Difficulty of implementation Hard Easy
35 35 Our Implementation Internal Approach Hack Swift to embed the search library. New Search API Application Proxy Node Swift s Ordinary API Indexing Searching Query Distributed by the Ring Data Metadata objectserver accountserver container-server SQLite Search ContainerA DB ContainerB DB Lucene Container A Index Container B Index Indexing
36 Future: Intelligent Resource Management Swift has more and more different functions. Other arbitrary processes Search...? Compression...? Encryption...? objectexpirer accountreplicator Multi-ring support Swift Storage Policy Erasure Coding objectupdater objectserver objectreplicator accountserver objectauditor proxyserver ecreconstructor ecauditor ec-stripeauditor accountauditor accountreaper containerserver containerreplicator containerauditor containerupdater containersync Copyright 2014 NTT DATA Corporation 36
37 37 Future: Intelligent Resource Management Resources are drained! IOPS, CPU, Network, Memory Performance Priorities of these functions are different by the Requirement. Ex1) Store performance VS Search performance Ex2) Service Level on Business Hour VS on Outside Hour 0 Outside (High-prio to check durability) 18 Business Hour (High-prio to process requests) 12 6 More Intelligent Resource Management is necessary. with cgroups
38 38 Summary 1.What is Swift? Swift is a Great OSS, for storing unstructured data. 2.Swift s Latest Information Erasure Coding Storage Policy 3.Swift s Future Integrated Searchable Storage Intelligent Resource management
39 PR: Demonstration is Now Available! We exhibit the Demo Application(Contents delivery system) built with Swift. On-demand Delivery a lot of contents(pictures or movies) stored at Swift. Implemented Searching on Swift. (Our original implementation) (map for demo booth) Copyright 2014 NTT DATA Corporation 39
40 Q&A: Do you have any question? Copyright 2011 NTT DATA Corporation Thank you for your attention! Please contact to if you have any questions or comments. Copyright 2014 NTT DATA Corporation
41 Challenges and Questions How to integrate Swift with cgroups? How to use cgroups? What is the best toolset for cgroups? VFS? libcgroup? systemd? How to control multiple hosts with cgroups dynamically? How to integrate Swift with search? What is the best implementation way? What is the best search middleware? How to search Multilingual? Copyright 2014 NTT DATA Corporation 41
SWIFT. Page:1. Openstack Swift. Object Store Cloud built from the grounds up. David Hadas Swift ATC. HRL davidh@il.ibm.com 2012 IBM Corporation
Page:1 Openstack Swift Object Store Cloud built from the grounds up David Hadas Swift ATC HRL davidh@il.ibm.com Page:2 Object Store Cloud Services Expectations: PUT/GET/DELETE Huge Capacity (Scale) Always
More informationHiroshi Miura. Scalable Private Cloud Storage with Full OSS Stack. CloudOpen Japan 5, June 2015. System Platform Sector, NTT DATA Corporation 年 月 日
CloudOpen Japan 5, June 2015 Scalable Private Cloud Storage with Full OSS Stack Hiroshi Miura 年 月 日 株 式 会 社 NTTデータ System Platform Sector, NTT DATA Corporation Copyright 2015 NTT DATA Corporation 2 Disclaimer
More informationHow to manage your OpenStack Swift Cluster using Swift Metrics Sreedhar Varma Vedams Inc.
How to manage your OpenStack Swift Cluster using Swift Metrics Sreedhar Varma Vedams Inc. What is OpenStack Swift Cluster? Cluster of Storage Server Nodes, Proxy Server Nodes and Storage Devices 2 Data
More informationwww.basho.com Technical Overview Simple, Scalable, Object Storage Software
www.basho.com Technical Overview Simple, Scalable, Object Storage Software Table of Contents Table of Contents... 1 Introduction & Overview... 1 Architecture... 2 How it Works... 2 APIs and Interfaces...
More informationIBM Spectrum Protect in the Cloud
IBM Spectrum Protect in the Cloud. Disclaimer IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM s sole discretion. Information regarding
More informationService Description Cloud Storage Openstack Swift
Service Description Cloud Storage Openstack Swift Table of Contents Overview iomart Cloud Storage... 3 iomart Cloud Storage Features... 3 Technical Features... 3 Proxy... 3 Storage Servers... 4 Consistency
More informationKT ucloud storage. Two Years of Life with OpenStack Swift. 2012. 08. 11 / Jaesuk Ahn, Cloud OS Dev. Team, Korea Telecom
KT ucloud storage Two Years of Life with OpenStack Swift 2012. 08. 11 / Jaesuk Ahn, Cloud OS Dev. Team, Korea Telecom Who I am. Jaesuk Ahn ( 安 宰 奭 ) 2012 ~ : Cloud OS Dev. Team Lead, KT 2009 ~ 2011: KT
More informationIntroduction to Cloud : Cloud and Cloud Storage. Lecture 2. Dr. Dalit Naor IBM Haifa Research Storage Systems. Dalit Naor, IBM Haifa Research
Introduction to Cloud : Cloud and Cloud Storage Lecture 2 Dr. Dalit Naor IBM Haifa Research Storage Systems 1 Advanced Topics in Storage Systems for Big Data - Spring 2014, Tel-Aviv University http://www.eng.tau.ac.il/semcom
More informationCreating a Cloud Backup Service. Deon George
Creating a Cloud Backup Service Deon George Agenda TSM Cloud Service features Cloud Service Customer, providing a internal backup service Internal Backup Cloud Service Service Provider, providing a backup
More informationIntroduction to Red Hat Storage. January, 2012
Introduction to Red Hat Storage January, 2012 1 Today s Speakers 2 Heather Wellington Tom Trainer Storage Program Marketing Manager Storage Product Marketing Manager Red Hat Acquisition of Gluster What
More informationBuilding Storage-as-a-Service Businesses
White Paper Service Providers Greatest New Growth Opportunity: Building Storage-as-a-Service Businesses According to 451 Research, Storage as a Service represents a large and rapidly growing market with
More informationHigh Performance Computing OpenStack Options. September 22, 2015
High Performance Computing OpenStack PRESENTATION TITLE GOES HERE Options September 22, 2015 Today s Presenters Glyn Bowden, SNIA Cloud Storage Initiative Board HP Helion Professional Services Alex McDonald,
More informationOpenStack IaaS. Rhys Oxenham OSEC.pl BarCamp, Warsaw, Poland November 2013
OpenStack IaaS 1 Rhys Oxenham OSEC.pl BarCamp, Warsaw, Poland November 2013 Disclaimer The information provided within this presentation is for educational purposes only and was prepared for a community
More informationAlexandria Overview. Sept 4, 2015
Alexandria Overview Sept 4, 2015 Alexandria 1U System Block Diagram SAS Interface Board Zoneboard Zoneboard I2C UART SAS to SATA I2C 12V AC Power Supply Power 60w Supply Seagate Confidential Alexandria
More informationUtilizing the SDSC Cloud Storage Service
Utilizing the SDSC Cloud Storage Service PASIG Conference January 13, 2012 Richard L. Moore rlm@sdsc.edu San Diego Supercomputer Center University of California San Diego Traditional supercomputer center
More informationBuilding low cost disk storage with Ceph and OpenStack Swift
Background photo from: http://edelomahony.com/2011/07/25/loving-money-doesnt-bring-you-more/ Building low cost disk storage with Ceph and OpenStack Swift Paweł Woszuk, Maciej Brzeźniak TERENA TF-Storage
More informationClodoaldo Barrera Chief Technical Strategist IBM System Storage. Making a successful transition to Software Defined Storage
Clodoaldo Barrera Chief Technical Strategist IBM System Storage Making a successful transition to Software Defined Storage Open Server Summit Santa Clara Nov 2014 Data at the core of everything Data is
More informationOn- Prem MongoDB- as- a- Service Powered by the CumuLogic DBaaS Platform
On- Prem MongoDB- as- a- Service Powered by the CumuLogic DBaaS Platform Page 1 of 16 Table of Contents Table of Contents... 2 Introduction... 3 NoSQL Databases... 3 CumuLogic NoSQL Database Service...
More informationIn Memory Accelerator for MongoDB
In Memory Accelerator for MongoDB Yakov Zhdanov, Director R&D GridGain Systems GridGain: In Memory Computing Leader 5 years in production 100s of customers & users Starts every 10 secs worldwide Over 15,000,000
More informationWHITE PAPER. Software Defined Storage Hydrates the Cloud
WHITE PAPER Software Defined Storage Hydrates the Cloud Table of Contents Overview... 2 NexentaStor (Block & File Storage)... 4 Software Defined Data Centers (SDDC)... 5 OpenStack... 5 CloudStack... 6
More informationHitachi Content Platform. Andrej Gursky, Solutions Consultant May 2015
Hitachi Content Platform Andrej Gursky, Solutions Consultant May 2015 What Is Object Storage? Aggregate, manage, protect and use content Just like we move photos from devices to a PC Hard to use on the
More informationKVM, OpenStack, and the Open Cloud
KVM, OpenStack, and the Open Cloud Adam Jollans, IBM & Mike Kadera, Intel CloudOpen Europe - October 13, 2014 13Oct14 Open VirtualizaGon Alliance 1 Agenda A Brief History of VirtualizaGon KVM Architecture
More informationHadoop on OpenStack Cloud. Dmitry Mescheryakov Software Engineer, @MirantisIT
Hadoop on OpenStack Cloud Dmitry Mescheryakov Software Engineer, @MirantisIT Agenda OpenStack Sahara Demo Hadoop Performance on Cloud Conclusion OpenStack Open source cloud computing platform 17,209 commits
More informationNetApp Data Fabric: Secured Backup to Public Cloud. Sonny Afen Senior Technical Consultant NetApp Indonesia
NetApp Data Fabric: Secured Backup to Public Cloud Sonny Afen Senior Technical Consultant NetApp Indonesia Agenda Introduction Solution Overview Solution Technical Overview 2 Introduction 3 Hybrid cloud:
More informationThe OpenStack TM Object Storage system
The OpenStack TM Object Storage system Deploying and managing a scalable, open- source cloud storage system with the SwiftStack Platform By SwiftStack, Inc. contact@swiftstack.com Contents Introduction...
More informationSetting Up SQL Server on Windows Azure Understanding Options and Differences
Setting Up SQL Server on Windows Azure Understanding Options and Differences Stephen Wynkoop, Microsoft SQL Server MVP Founder, SSWUG.ORG Session Q&A Housekeeping Questions after the fact: swynk@sswug.org
More informationHadoop & its Usage at Facebook
Hadoop & its Usage at Facebook Dhruba Borthakur Project Lead, Hadoop Distributed File System dhruba@apache.org Presented at the Storage Developer Conference, Santa Clara September 15, 2009 Outline Introduction
More informationImproving Scalability Of Storage System:Object Storage Using Open Stack Swift
Improving Scalability Of Storage System:Object Storage Using Open Stack Swift G.Kathirvel Karthika 1,R.C.Malathy 2,M.Keerthana 3 1,2,3 Student of Computer Science and Engineering, R.M.K Engineering College,Kavaraipettai.
More informationGPFS Cloud ILM. IBM Research - Zurich. Storage Research Technology Outlook
IBM Research - Zurich GPFS Cloud ILM Storage Research Technology Outlook Dr. Thomas Weigold (twe@zurich.ibm.com) Manager Cloud Storage & Security IBM Research Zurich Why Cloud Storage? Economics! Lower
More informationSystem Administrators, engineers and consultants who will plan and manage OpenStack-based environments.
OpenStack Foundations (HP-H6C68) Course Overview This three day course assists administrators and users to configure, manage, and use the OpenStack cloud services platform. An architectural overview ensures
More informationPrepared for: How to Become Cloud Backup Provider
Prepared for: How to Become Cloud Backup Provider Contents Abstract... 3 Introduction... 3 Purpose... 3 Architecture... 4 Result... 4 Requirements... 4 OS... 5 Networking... 5 Database... 5 Permissions...
More informationStorage and Disaster Recovery
Storage and Disaster Recovery Matt Tavis Principal Solutions Architect The Business Continuity Continuum High Data Backup Disaster Recovery High, Storage Backup and Disaster Recovery form a continuum of
More informationDesigning a Cloud Storage System
Designing a Cloud Storage System End to End Cloud Storage When designing a cloud storage system, there is value in decoupling the system s archival capacity (its ability to persistently store large volumes
More informationAdrian Otto, Rackspace @adrian_otto
Adrian Otto, Rackspace @adrian_otto Ancient History RackSpace Cloud Files Swift NASA Nova OpenStack born with 25 participating companies! Mission Statement "To produce the ubiquitous Open Source cloud
More informationCloud Models and Platforms
Cloud Models and Platforms Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF A Working Definition of Cloud Computing Cloud computing is a model
More informationMulti Provider Cloud. Srinivasa Acharya, Engineering Manager, Hewlett-Packard rsacharya@hp.com
Multi Provider Cloud Srinivasa Acharya, Engineering Manager, Hewlett-Packard rsacharya@hp.com Agenda Introduction to OpenStack Multi Hypervisor Architecture Use cases for Multi Hypervisor cloud Ironic
More informationWikimedia architecture. Mark Bergsma <mark@wikimedia.org> Wikimedia Foundation Inc.
Mark Bergsma Wikimedia Foundation Inc. Overview Intro Global architecture Content Delivery Network (CDN) Application servers Persistent storage Focus on architecture, not so much on
More informationArchiving On-Premise and in the Cloud. March 2015
Archiving On-Premise and in the Cloud March 2015 Cloud Storage Storage accessed over a network via web services APIs. http://swift.example.com/v1/account/container/object Source: http://docs.openstack.org/admin-guide-cloud/content/objectstorage_characteristics.html
More informationOpenStack Introduction. November 4, 2015
OpenStack Introduction November 4, 2015 Application Platforms Undergoing A Major Shift What is OpenStack Open Source Cloud Software Launched by NASA and Rackspace in 2010 Massively scalable Managed by
More informationProduct Spotlight. A Look at the Future of Storage. Featuring SUSE Enterprise Storage. Where IT perceptions are reality
Where IT perceptions are reality Product Spotlight A Look at the Future of Storage Featuring SUSE Enterprise Storage Document # SPOTLIGHT2013001 v5, January 2015 Copyright 2015 IT Brand Pulse. All rights
More informationSurviving the Worst: Disaster Recovery for OpenStack
Surviving the Worst: Disaster Recovery for OpenStack Ronen Kat, IBM Research Haifa Ayal Baron, Red Hat Partially sponsored by Agenda What is disaster recovery Disaster Recovery trailer Volume storage replication
More informationSheepdog: distributed storage system for QEMU
Sheepdog: distributed storage system for QEMU Kazutaka Morita NTT Cyber Space Labs. 9 August, 2010 Motivation There is no open source storage system which fits for IaaS environment like Amazon EBS IaaS
More informationExposing the Cloud: It It s More than a Buzzword Tim Connors, Director, AT&T AT&T
Exposing the Cloud: It s More than a Buzzword Tim Connors, Director, AT&T Exposing the Cloud: It s More than a Buzzword Discussion Agenda Cloud Storage Target Uses When is it right? When is it wrong? Specific
More informationScalable Architecture on Amazon AWS Cloud
Scalable Architecture on Amazon AWS Cloud Kalpak Shah Founder & CEO, Clogeny Technologies kalpak@clogeny.com 1 * http://www.rightscale.com/products/cloud-computing-uses/scalable-website.php 2 Architect
More informationRed Hat Storage Server
Red Hat Storage Server Marcel Hergaarden Solution Architect, Red Hat marcel.hergaarden@redhat.com May 23, 2013 Unstoppable, OpenSource Software-based Storage Solution The Foundation for the Modern Hybrid
More informationcloud functionality: advantages and Disadvantages
Whitepaper RED HAT JOINS THE OPENSTACK COMMUNITY IN DEVELOPING AN OPEN SOURCE, PRIVATE CLOUD PLATFORM Introduction: CLOUD COMPUTING AND The Private Cloud cloud functionality: advantages and Disadvantages
More informationBusiness Intelligence Competency Partners
Business Intelligence Competency Partners BI 4.1 Installation Best Practices Presenter: Rich Chlebek May 15, 2014 What We ll Cover Webinar Protocol Introductions Architecture Server Clustering High Availability
More informationBuilding Cost-Effective Storage Clouds A Metrics-based Approach
Building Cost-Effective Storage Clouds A Metrics-based Approach Ning Zhang #1, Chander Kant 2 # Computer Sciences Department, University of Wisconsin Madison Madison, WI, USA 1 nzhang@cs.wisc.edu Zmanda
More informationSMART SCALE YOUR STORAGE - Object "Forever Live" Storage - Roberto Castelli EVP Sales & Marketing BCLOUD
SMART SCALE YOUR STORAGE - Object "Forever Live" Storage - Roberto Castelli EVP Sales & Marketing BCLOUD 1 BCLOUD at a Glance 4 years constantly growing + 3PBs protected and distributed every day from
More informationAspera Direct-to-Cloud Storage WHITE PAPER
Transport Direct-to-Cloud Storage and Support for Third Party April 2014 WHITE PAPER TABLE OF CONTENTS OVERVIEW 3 1 - THE PROBLEM 3 2 - A FUNDAMENTAL SOLUTION - ASPERA DIRECT-TO-CLOUD TRANSPORT 5 3 - VALIDATION
More informationACCELERATING SQL SERVER WITH XTREMIO
1 ACCELERATING SQL SERVER WITH XTREMIO WANDA HE PRINCIPAL SOLUTIONS ARCHITECT EMC XTREMIO KAREN WALLACE DIR. OF ARCHITECTURE CONNECTIONS EDUCATION GORDON LAQUE INFRASTRUCTURE ARCHITECT/DATABASE MANAGER
More informationOpenStack. Orgad Kimchi. Principal Software Engineer. Oracle ISV Engineering. 1 Copyright 2013, Oracle and/or its affiliates. All rights reserved.
OpenStack Orgad Kimchi Principal Software Engineer Oracle ISV Engineering 1 Copyright 2013, Oracle and/or its affiliates. All rights reserved. Safe Harbor Statement The following is intended to outline
More informationWikimedia Architecture Doing More With Less. Asher Feldman <asher@wikimedia.org> Ryan Lane <ryan@wikimedia.org> Wikimedia Foundation Inc.
Wikimedia Architecture Doing More With Less Asher Feldman Ryan Lane Wikimedia Foundation Inc. Overview Intro Scale at WMF How We Work Architecture Dive Top Five
More informationOpenStack Object Storage Administrator Guide
OpenStack Object Storage (02/03/11) Copyright 2010, 2011 OpenStack LLC All rights reserved. OpenStack Object Storage offers open source software for cloud-based object storage for any organization. This
More informationBDR TM for VMware. VMware BACKUP WITH VEMBU. VEMBU TECHNOLOGIES www.vembu.com TRUSTED BY OVER 25,000 BUSINESSES
BDR TM for VMware VMware BACKUP WITH VEMBU VEMBU TECHNOLOGIES www.vembu.com Copyright Information Information in this document is subject to change without notice. The entire risk of the use or the results
More informationIntroduction. Examples of use cases:
Introduction An object store is a distributed storage platform were objects (files) can be stored, managed and queried by using simple API-calls. This is the ideal cloud storage solution for larger pieces
More informationA 5 Year Total Cost of Ownership Study on the Economics of Cloud Storage
2016 Industry Report Cloud Storage Rent or Buy? A 5 Year Total Cost of Ownership Study on the Economics of Cloud Storage Sponsored by StrongBox Data Solutions, Inc. Copyright 2016 Brad Johns Consulting
More informationKVM, OpenStack, and the Open Cloud
KVM, OpenStack, and the Open Cloud Adam Jollans, IBM Southern California Linux Expo February 2015 1 Agenda A Brief History of VirtualizaJon KVM Architecture OpenStack Architecture KVM and OpenStack Case
More informationBacking up to the Cloud
Backing up to the Cloud Sean Lim, Director, Cloud Service Providers, APAC sean.lim@netapp.com @rhipecloud #RCCS15 Agenda Backup & Archive to the Cloud Veeam & Steelstore Cloud Service Provider Partners
More informationPetabyte Scale Data at Facebook. Dhruba Borthakur, Engineer at Facebook, SIGMOD, New York, June 2013
Petabyte Scale Data at Facebook Dhruba Borthakur, Engineer at Facebook, SIGMOD, New York, June 2013 Agenda 1 Types of Data 2 Data Model and API for Facebook Graph Data 3 SLTP (Semi-OLTP) and Analytics
More informationBig Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013
Big Data Use Case How Rackspace is using Private Cloud for Big Data Bryan Thompson May 8th, 2013 Our Big Data Problem Consolidate all monitoring data for reporting and analytical purposes. Every device
More informationCloud Computing: Making the right choices
Cloud Computing: Making the right choices Kalpak Shah Clogeny Technologies Pvt Ltd 1 About Me Kalpak Shah Founder & CEO, Clogeny Technologies Passionate about economics and technology evolving through
More informationProtecting the Microsoft Data Center with NetBackup 7.6
Protecting the Microsoft Data Center with NetBackup 7.6 Amit Sinha NetBackup Product Management 1 Major Components of a Microsoft Data Center Software Hardware Servers Disk Tape Networking Server OS Applications
More informationHow To Choose Cloud Computing
IJSRD - International Journal for Scientific Research & Development Vol. 2, Issue 09, 2014 ISSN (online): 2321-0613 Comparison of Several IaaS Cloud Computing Platforms Amar Deep Gorai 1 Dr. Birendra Goswami
More informationMark Bennett. Search and the Virtual Machine
Mark Bennett Search and the Virtual Machine Agenda Intro / Business Drivers What to do with Search + Virtual What Makes Search Fast (or Slow!) Virtual Platforms Test Results Trends / Wrap Up / Q & A Business
More informationDistributed Block-level Storage Management for OpenStack
Distributed Block-level Storage Management for OpenStack OpenStack APAC Conference Daniel Lee CCMA/ITRI Cloud Computing Center for Mobile Applications Industrial Technology Research Institute ( 雲 端 運 算
More informationSUSE Storage. FUT7537 Software Defined Storage Introduction and Roadmap: Getting your tentacles around data growth. Larry Morris
SUSE FUT7537 Software Defined Introduction and Roadmap: Getting your tentacles around data growth Larry Morris Sr. Product Manager lmorris@suse.com AGENDA Enterprise Market SUSE Product SUSE Solutions
More informationLambert: Achieve High Durability, Low Cost & Flexibility at Same Time
Lambert: Achieve High Durability, Low Cost & Flexibility at Same Time Open source storage engine for exabyte data in Alibaba Coly Li Robin Dong About
More informationSOFTWARE-DEFINED STORAGE IN ACTION
SOFTWARE-DEFINED STORAGE IN ACTION Anne Tarantino VP Sales- Velocity Tech Solutions Copyright 2014 DataCore Software Corp. All Rights Reserved. Copyright 2014 DataCore Software Corp. All Rights Reserved.
More informationThe Pros and Cons of Erasure Coding & Replication vs. RAID in Next-Gen Storage Platforms. Abhijith Shenoy Engineer, Hedvig Inc.
The Pros and Cons of Erasure Coding & Replication vs. RAID in Next-Gen Storage Platforms Abhijith Shenoy Engineer, Hedvig Inc. @hedviginc The need for new architectures Business innovation Time-to-market
More informationHow To Build A Cloud Storage System
Reference Architectures for Digital Libraries Keith Rajecki Education Solutions Architect Sun Microsystems, Inc. 1 Agenda Challenges Digital Library Solution Architectures > Open Storage/Open Archive >
More informationGPFS-OpenStack Integration. Dinesh Subhraveti IBM Research
GPFS-OpenStack Integration Dinesh Subhraveti IBM Research GPFS File Placement Optimization Tradi5onal shared architecture Shared nothing architecture SAN I/O bo*leneck Scale out performance GPFS cluster
More informationHadoop & its Usage at Facebook
Hadoop & its Usage at Facebook Dhruba Borthakur Project Lead, Hadoop Distributed File System dhruba@apache.org Presented at the The Israeli Association of Grid Technologies July 15, 2009 Outline Architecture
More informationHadoop: Embracing future hardware
Hadoop: Embracing future hardware Suresh Srinivas @suresh_m_s Page 1 About Me Architect & Founder at Hortonworks Long time Apache Hadoop committer and PMC member Designed and developed many key Hadoop
More informationClient-aware Cloud Storage
Client-aware Cloud Storage Feng Chen Computer Science & Engineering Louisiana State University Michael Mesnier Circuits & Systems Research Intel Labs Scott Hahn Circuits & Systems Research Intel Labs Cloud
More informationSoftware Defined Storage @ Microsoft. PRESENTATION TITLE GOES HERE Siddhartha Roy Cloud + Enterprise Division Microsoft Corporation
Software Defined @ Microsoft PRESENTATION TITLE GOES HERE Siddhartha Roy Cloud + Enterprise Division Microsoft Corporation Lessons Learned operating large cloud properties Industry trends Cloud scale services
More informationInvestigating Private Cloud Storage Deployment using Cumulus, Walrus, and OpenStack/Swift
Investigating Private Cloud Storage Deployment using Cumulus, Walrus, and OpenStack/Swift Prakashan Korambath Institute for Digital Research and Education (IDRE) 5308 Math Sciences University of California,
More informationAn Intro to OpenStack. Ian Lawson Senior Solution Architect, Red Hat ilawson@redhat.com
An Intro to OpenStack Ian Lawson Senior Solution Architect, Red Hat ilawson@redhat.com What is OpenStack? What is OpenStack? Fully open source cloud operating system Comprised of several open source sub-projects
More informationHadoop Distributed File System. Dhruba Borthakur Apache Hadoop Project Management Committee dhruba@apache.org dhruba@facebook.com
Hadoop Distributed File System Dhruba Borthakur Apache Hadoop Project Management Committee dhruba@apache.org dhruba@facebook.com Hadoop, Why? Need to process huge datasets on large clusters of computers
More information922-280 Avamar Backup and Data De-duplication Exam
922-280 Avamar Backup and Data De-duplication Exam Q&A DEMO Version Copyright (c) 2007 Chinatag LLC. All rights reserved. Important Note Please Read Carefully For demonstration purpose only, this free
More informationZingMe Practice For Building Scalable PHP Website. By Chau Nguyen Nhat Thanh ZingMe Technical Manager Web Technical - VNG
ZingMe Practice For Building Scalable PHP Website By Chau Nguyen Nhat Thanh ZingMe Technical Manager Web Technical - VNG Agenda About ZingMe Scaling PHP application Scalability definition Scaling up vs
More informationDISTRIBUTED SYSTEMS [COMP9243] Lecture 9a: Cloud Computing WHAT IS CLOUD COMPUTING? 2
DISTRIBUTED SYSTEMS [COMP9243] Lecture 9a: Cloud Computing Slide 1 Slide 3 A style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet.
More informationHow swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda
How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda 1 Outline Build a cost-efficient Swift cluster with expected performance Background & Problem Solution Experiments
More informationTECHNOLOGY WHITE PAPER Jun 2012
TECHNOLOGY WHITE PAPER Jun 2012 Technology Stack C# Windows Server 2008 PHP Amazon Web Services (AWS) Route 53 Elastic Load Balancing (ELB) Elastic Compute Cloud (EC2) Amazon RDS Amazon S3 Elasticache
More informationApache Hadoop FileSystem and its Usage in Facebook
Apache Hadoop FileSystem and its Usage in Facebook Dhruba Borthakur Project Lead, Apache Hadoop Distributed File System dhruba@apache.org Presented at Indian Institute of Technology November, 2010 http://www.facebook.com/hadoopfs
More informationHP OpenStack & Automation
HP OpenStack & Automation Where we are heading Thomas Goh Cloud Computing Cloud Computing Cloud computing is a model for enabling ubiquitous network access to a shared pool of configurable computing resources.
More informationVMware Software-Defined Storage Vision
VMware Software-Defined Storage Vision Lee Dilworth (@leedilworth) Principal Systems Engineer 2014 VMware Inc. All rights reserved. The Software-Defined Data Center Expand virtual compute to all applications
More informationStorage Made Easy Enterprise File Share and Sync (EFSS) Cloud Control Gateway Architecture
Storage Made Easy Enterprise File Share and Sync (EFSS) Architecture Software Stack The SME platform is built using open Internet technologies. The base operating system used s hardened Linux CentOS. HTTPD
More informationBig data Devices Apps
Trends driving IT pressures Cloud Big data Devices Apps Devices: Info Workers Will Erase Boundary Between enterprise And Consumer Technologies. Forrester Research. August 30, 2012 Apps: Gartner: Predicts
More informationBuilding a Cloud Computing Platform based on Open Source Software. 10. 18. 2011. Donghoon Kim ( donghoon.kim@kt.com ) Yoonbum Huh ( huhbum@kt.
Building a Cloud Computing Platform based on Open Source Software 10. 18. 2011. Donghoon Kim ( donghoon.kim@kt.com ) Yoonbum Huh ( huhbum@kt.com) Topics I.Open Source SW and Cloud Computing II. About OpenStack
More informationUsing object storage as a target for backup, disaster recovery, archiving
Using object storage as a target for backup, disaster recovery, archiving By Deni Connor, founding analyst Earl Follis, senior analyst SSG-NOW June 2015 I n many organizations, backup data accounts for
More informationClusters in the Cloud
Clusters in the Cloud Dr. Paul Coddington, Deputy Director Dr. Shunde Zhang, Compu:ng Specialist eresearch SA October 2014 Use Cases Make the cloud easier to use for compute jobs Par:cularly for users
More informationCloud Computing using
Cloud Computing using Summary of Content Introduction of Cloud Computing Cloud Computing vs. Server Virtualization Cloud Computing Components Stack Public vs. Private Clouds Open Source Software for Private
More informationThe Design and Implementation of the Zetta Storage Service. October 27, 2009
The Design and Implementation of the Zetta Storage Service October 27, 2009 Zetta s Mission Simplify Enterprise Storage Zetta delivers enterprise-grade storage as a service for IT professionals needing
More informationTrends in Application Recovery. Andreas Schwegmann, HP
Andreas Schwegmann, HP SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless otherwise noted. Member companies and individual members may use this material in presentations
More informationDiagram 1: Islands of storage across a digital broadcast workflow
XOR MEDIA CLOUD AQUA Big Data and Traditional Storage The era of big data imposes new challenges on the storage technology industry. As companies accumulate massive amounts of data from video, sound, database,
More informationCloud Storage and Backup
Cloud Storage and Backup Cloud Storage and Backup Cloud Storage and Backup services from iomartcloud have been designed to deliver the performance, capacity, security and flexibility needed to address
More informationCLOUD BASED SERVICE (CBS STORAGE)
CLOUD BASED SERVICE (CBS STORAGE) Defining next generation of cloud based grid Power All Networks Ltd. Technical Whitepaper September 2008, version 1.04 PAGE 1 Table of Content Cloud Based Services (CBS
More informationWOS OBJECT STORAGE PRODUCT BROCHURE DDN.COM 1.800.837.2298. 360 Full Spectrum Object Storage
PRODUCT BROCHURE WOS OBJECT STORAGE 360 Full Spectrum Object Storage The promise of object storage is simple: to enable organizations to build highly Performance Scalability Reliability Efficiency Security
More informationPROPOSAL To Develop an Enterprise Scale Disease Modeling Web Portal For Ascel Bio Updated March 2015
Enterprise Scale Disease Modeling Web Portal PROPOSAL To Develop an Enterprise Scale Disease Modeling Web Portal For Ascel Bio Updated March 2015 i Last Updated: 5/8/2015 4:13 PM3/5/2015 10:00 AM Enterprise
More information