DSS. Data & Storage Services. Huawei Cloud Storage Evaluation and Testing with Prototype Services. Seppo S. Heikkila CERN IT
|
|
|
- Preston Moore
- 7 years ago
- Views:
Transcription
1 Data & Storage Huawei Cloud Storage Evaluation and Testing with Prototype Seppo S. Heikkila CERN IT Openlab Minor Review 29th of October 2013 CERN, Geneva
2 Introduction Motivation Cloud storage market is growing fast CERN uses custom made storage solutions Question Are cloud storages able to meet the High Energy Physics (HEP) data storage requirements? Method Evaluate scalability and fault-tolerance Test with real applications 2
3 Huawei cloud storage Location: CERN Computer Center We are now here Cloud storage 3
4 Huawei cloud storage setup Three racks Storage nodes Front-end nodes 4 Storage nodes
5 Huawei cloud storage setup 384 disks 768 TB Storage nodes S3 compatible 5 Front-end nodes Buckets divide the name space Storage nodes
6 Storage nodes Each blade has eight storage nodes One chassis has two blades (16 disks) 6
7 Timeline 1.5 years of Huawei... Major Review Major Major Review Review Board of Minor Minor Board of Minor Sponsors Review Review Sponsors Review 01/2012 Major Review 01/ /2013 Project First Upgrade Stress File-system of the tests starts testing integration system Commissioning Failure Full-scale of the system recovery stress testing testing 7
8 Benchmark setup Distributed C++ benchmark Integrated with ROOT Client nodes connected with ssh S3 Python library to read and write files Histograms about specific metrics Operation time, read/write speed, CPU/memory utilisation Benchmark h ss Master: C++ ssh ss Client: C++ & Python 8 S3 S3... Client: C++ & Python h Client: C++ & Python Cloud storage S3
9 Benchmarking scalability Speed vs concurrent clients 9
10 Benchmarking scalability Small (4kB) & big (100MB) files Speed vs concurrent clients Enough network capacity & client machines 10
11 Download throughput scalability clients 100MB cloud storage Scales up to bandwidth limit 11
12 Download metadata scalability 4kB clients a e Lin 12 c s r ng i l a cloud storage Front-end performance limit reached
13 Upload metadata scalability clients 4kB More buckets required, as expected 13 cloud storage Scales: up to 2,500 files/s
14 Number of buckets vs upload speed clients 4kB cloud storage At least 200 buckets needed with 1200 concurrent clients 14
15 Front-end scalability 100MB 1 cloud storage clients MB... 2 cloud storage clients 100MB 7 cloud storage We use different numbers of front-ends: from 1 to 7 clients 15
16 Front-end scalability 100MB 1 cloud storage clients MB... 2 cloud storage clients 100MB 7 cloud storage Small (4kB) & big (100MB) files Uploads & downloads clients 16 We use different numbers of front-ends: from 1 to 7
17 Front-end scalability clients 100MB cloud storage Three front-ends is enough in 20Gbit network
18 Front-end scalability clients 100MB cloud storage Four front-ends is enough in 20Gbit network
19 Front-end scalability clients 4kB cloud storage Linear scaling
20 Front-end scalability clients 4kB Two front-ends would be enough 20 cloud storage
21 Raw performance summary Metadata (4kB) performance 2,500 files/second upload 25,000 files/second download Throughput (100MB) performance 20Gbit network fully utilized Front-end scalability Each front-end can download 3500 files/s Each front-end can upload 550 MB/s 21
22 Recovery after powering off a chassis Two blades are powered off: 16 disks down 22
23 Recovery after powering off a chassis Two blades are powered off: 16 disks down 23 Uploads and downloads continue normally?
24 Recovery after powering off a chassis Chassis down Chassis up 16 disks down 24 Time (seconds)
25 Recovery after powering off a chassis Chassis down No failures, only delays Chassis up First storage nodes respond to ping All storage nodes in use 25 Time (seconds)
26 CVMFS introduction What is CVMFS (CernVM File System) Read only cached file system to deliver software Widely used in WLCG (Worldwide LHC Computing Grid) Mounted by users and files are downloaded on demand 26
27 CVMFS introduction What is CVMFS (CernVM File System) Read only cached file system to deliver software Widely used in WLCG (Worldwide LHC Computing Grid) Mounted by users and files are downloaded on demand CVMFS challenges Publishing new software should be fast (upload tens of thousands of files) Files should be accessed with HTTP protocol 27
28 File system with Huawei back-end Implementation Files are uploaded to multiple buckets in the cloud storage Files are downloaded with unified name space
29 File system with Huawei back-end Implementation Files are uploaded to multiple buckets in the cloud storage Files are downloaded with unified name space Result Full publish procedure tested to work using 30,000 small files Upload speed 1200 files/second (with 240 threads) 29
30 CVMFS incremental uploads Uploading 30,000 files (of average size 10kB) to Huawei UDS Motivation: only part of the files change between releases 30
31 CVMFS incremental uploads Uploading 30,000 files (of average size 10kB) to OpenStack SWIFT SWIFT: 2 front-ends Huawei: 7 front-ends Not comparable! 31
32 Conclusion Raw performance Upload and download scalability demonstrated Additional front-end nodes increased linearly the performance Fault tolerance: powering off a chassis Transparent disk failure recovery demonstrated File system with cloud storage back-end Full publishing procedure tested Uploading of only new files feature tested 32
33 Future plans Short term Benchmark CVMFS with real release data Test ROOT's new S3 plugin performance Long term Second petabyte system with enterprise disks expected to arrive soon Upgrade old Huawei cloud storage software version Replication tests between two cloud storages Prove total cost of ownership (TCO) gains of the system as part of a production service 33
34 Future plans Short term Benchmark CVMFS with real release data Test ROOT's new S3 plugin performance Long term Second petabyte system with enterprise disks expected to arrive soon Upgrade old Huawei cloud storage software version Replication tests between two cloud storages Prove total cost of ownership (TCO) gains of the system as part of a production service Thank you! seppo.heikkila@cern.ch 34
DSS. Data & Storage Services. Cloud storage performance and first experience from prototype services at CERN
Data & Storage Cloud storage performance and first experience from prototype services at CERN Maitane Zotes Resines, Seppo S. Heikkila, Dirk Duellmann, Geoffray Adde, Rainer Toebbicke, CERN James Hughes,
Using S3 cloud storage with ROOT and CernVMFS. Maria Arsuaga-Rios Seppo Heikkila Dirk Duellmann Rene Meusel Jakob Blomer Ben Couturier
Using S3 cloud storage with ROOT and CernVMFS Maria Arsuaga-Rios Seppo Heikkila Dirk Duellmann Rene Meusel Jakob Blomer Ben Couturier INDEX Huawei cloud storages at CERN Old vs. new Huawei UDS comparative
CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT
SS Data & Storage CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT HEPiX Fall 2012 Workshop October 15-19, 2012 Institute of High Energy Physics, Beijing, China SS Outline
Development of Monitoring and Analysis Tools for the Huawei Cloud Storage
Development of Monitoring and Analysis Tools for the Huawei Cloud Storage September 2014 Author: Veronia Bahaa Supervisors: Maria Arsuaga-Rios Seppo S. Heikkila CERN openlab Summer Student Report 2014
Improved metrics collection and correlation for the CERN cloud storage test framework
Improved metrics collection and correlation for the CERN cloud storage test framework September 2013 Author: Carolina Lindqvist Supervisors: Maitane Zotes Seppo Heikkila CERN openlab Summer Student Report
DSS. Diskpool and cloud storage benchmarks used in IT-DSS. Data & Storage Services. Geoffray ADDE
DSS Data & Diskpool and cloud storage benchmarks used in IT-DSS CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/it Geoffray ADDE DSS Outline I- A rational approach to storage systems evaluation
files without borders
files without borders exploring Internet-connected storage for research Fabio Hernandez fabio@in2p3.fr IN2P3 / CNRS computing center, Lyon, France FJPPL compu+ng workshop, Lyon, March 11th 2015 2 Preamble
Storage strategy and cloud storage evaluations at CERN Dirk Duellmann, CERN IT
SS Data & Storage Storage strategy and cloud storage evaluations at CERN Dirk Duellmann, CERN IT (with slides from Andreas Peters and Jan Iven) 5th International Conference "Distributed Computing and Grid-technologies
Reference Design: Scalable Object Storage with Seagate Kinetic, Supermicro, and SwiftStack
Reference Design: Scalable Object Storage with Seagate Kinetic, Supermicro, and SwiftStack May 2015 Copyright 2015 SwiftStack, Inc. swiftstack.com Page 1 of 19 Table of Contents INTRODUCTION... 3 OpenStack
Building Storage Clouds for Online Applications A Case for Optimized Object Storage
Building Storage Clouds for Online Applications A Case for Optimized Object Storage Agenda Introduction: storage facts and trends Call for more online storage! AmpliStor: Optimized Object Storage Cost
Business-centric Storage FUJITSU Hyperscale Storage System ETERNUS CD10000
Business-centric Storage FUJITSU Hyperscale Storage System ETERNUS CD10000 Clear the way for new business opportunities. Unlock the power of data. Overcoming storage limitations Unpredictable data growth
www.basho.com Technical Overview Simple, Scalable, Object Storage Software
www.basho.com Technical Overview Simple, Scalable, Object Storage Software Table of Contents Table of Contents... 1 Introduction & Overview... 1 Architecture... 2 How it Works... 2 APIs and Interfaces...
How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda
How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda 1 Outline Build a cost-efficient Swift cluster with expected performance Background & Problem Solution Experiments
Distributed File System. MCSN N. Tonellotto Complements of Distributed Enabling Platforms
Distributed File System 1 How do we get data to the workers? NAS Compute Nodes SAN 2 Distributed File System Don t move data to workers move workers to the data! Store data on the local disks of nodes
Testing of several distributed file-system (HadoopFS, CEPH and GlusterFS) for supporting the HEP experiments analisys. Giacinto DONVITO INFN-Bari
Testing of several distributed file-system (HadoopFS, CEPH and GlusterFS) for supporting the HEP experiments analisys. Giacinto DONVITO INFN-Bari 1 Agenda Introduction on the objective of the test activities
Introduction to HDFS. Prasanth Kothuri, CERN
Prasanth Kothuri, CERN 2 What s HDFS HDFS is a distributed file system that is fault tolerant, scalable and extremely easy to expand. HDFS is the primary distributed storage for Hadoop applications. Hadoop
Building low cost disk storage with Ceph and OpenStack Swift
Background photo from: http://edelomahony.com/2011/07/25/loving-money-doesnt-bring-you-more/ Building low cost disk storage with Ceph and OpenStack Swift Paweł Woszuk, Maciej Brzeźniak TERENA TF-Storage
The OpenStack TM Object Storage system
The OpenStack TM Object Storage system Deploying and managing a scalable, open- source cloud storage system with the SwiftStack Platform By SwiftStack, Inc. contact@swiftstack.com Contents Introduction...
Summer Student Project Report
Summer Student Project Report Dimitris Kalimeris National and Kapodistrian University of Athens June September 2014 Abstract This report will outline two projects that were done as part of a three months
Shoal: IaaS Cloud Cache Publisher
University of Victoria Faculty of Engineering Winter 2013 Work Term Report Shoal: IaaS Cloud Cache Publisher Department of Physics University of Victoria Victoria, BC Mike Chester V00711672 Work Term 3
A programming model in Cloud: MapReduce
A programming model in Cloud: MapReduce Programming model and implementation developed by Google for processing large data sets Users specify a map function to generate a set of intermediate key/value
DSS. High performance storage pools for LHC. Data & Storage Services. Łukasz Janyst. on behalf of the CERN IT-DSS group
DSS High performance storage pools for LHC Łukasz Janyst on behalf of the CERN IT-DSS group CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it Introduction The goal of EOS is to provide a
Status and Evolution of ATLAS Workload Management System PanDA
Status and Evolution of ATLAS Workload Management System PanDA Univ. of Texas at Arlington GRID 2012, Dubna Outline Overview PanDA design PanDA performance Recent Improvements Future Plans Why PanDA The
Hadoop Architecture. Part 1
Hadoop Architecture Part 1 Node, Rack and Cluster: A node is simply a computer, typically non-enterprise, commodity hardware for nodes that contain data. Consider we have Node 1.Then we can add more nodes,
Testing & Assuring Mobile End User Experience Before Production. Neotys
Testing & Assuring Mobile End User Experience Before Production Neotys Agenda Introduction The challenges Best practices NeoLoad mobile capabilities Mobile devices are used more and more At Home In 2014,
HEP Compu*ng in a Context- Aware Cloud Environment
HEP Compu*ng in a Context- Aware Cloud Environment Randall Sobie A.Charbonneau F.Berghaus R.Desmarais I.Gable C.LeaveC- Brown M.Paterson R.Taylor InsItute of ParIcle Physics University of Victoria and
<Insert Picture Here> Refreshing Your Data Protection Environment with Next-Generation Architectures
1 Refreshing Your Data Protection Environment with Next-Generation Architectures Dale Rhine, Principal Sales Consultant Kelly Boeckman, Product Marketing Analyst Program Agenda Storage
Integration of Network Performance Monitoring Data at FTS3
Integration of Network Performance Monitoring Data at FTS3 July-August 2013 Author: Rocío Rama Ballesteros Supervisor(s): Michail Salichos Alejandro Álvarez CERN openlab Summer Student Report 2013 Project
Diagram 1: Islands of storage across a digital broadcast workflow
XOR MEDIA CLOUD AQUA Big Data and Traditional Storage The era of big data imposes new challenges on the storage technology industry. As companies accumulate massive amounts of data from video, sound, database,
Key Messages of Enterprise Cluster NAS Huawei OceanStor N8500
Messages of Enterprise Cluster NAS Huawei OceanStor Messages of Enterprise Cluster NAS 1. High performance and high reliability, addressing bid data challenges High performance: In the SPEC benchmark test,
Introduction to Windows Azure Cloud Computing Futures Group, Microsoft Research Roger Barga, Jared Jackson,Nelson Araujo, Dennis Gannon, Wei Lu, and
Introduction to Windows Azure Cloud Computing Futures Group, Microsoft Research Roger Barga, Jared Jackson,Nelson Araujo, Dennis Gannon, Wei Lu, and Jaliya Ekanayake Range in size from edge facilities
Comparison of the Frontier Distributed Database Caching System with NoSQL Databases
Comparison of the Frontier Distributed Database Caching System with NoSQL Databases Dave Dykstra dwd@fnal.gov Fermilab is operated by the Fermi Research Alliance, LLC under contract No. DE-AC02-07CH11359
BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB
BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB Planet Size Data!? Gartner s 10 key IT trends for 2012 unstructured data will grow some 80% over the course of the next
Maximizing Data Value with HUAWEI OceanStor. Success Cases
Maximizing Data Value with HUAWEI OceanStor Success Cases Operator Vodafone Hybrid IaaS Cloud Storage Selects Huawei OceanStor Storage Vodaplex hybrid IaaS cloud broker enables resources to be provisioned
Cloud Computing for Control Systems CERN Openlab Summer Student Program 9/9/2011 ARSALAAN AHMED SHAIKH
Cloud Computing for Control Systems CERN Openlab Summer Student Program 9/9/2011 ARSALAAN AHMED SHAIKH CONTENTS Introduction... 4 System Components... 4 OpenNebula Cloud Management Toolkit... 4 VMware
Instant-On Enterprise
Instant-On Enterprise Winning with NonStop SQL 2011Hewlett-Packard Dev elopment Company,, L.P. The inf ormation contained herein is subject to change without notice LIBERATE Your infrastructure with HP
Alternative models to distribute VO specific software to WLCG sites: a prototype set up at PIC
EGEE and glite are registered trademarks Enabling Grids for E-sciencE Alternative models to distribute VO specific software to WLCG sites: a prototype set up at PIC Elisa Lanciotti, Arnau Bria, Gonzalo
IPv6 Traffic Analysis and Storage
Report from HEPiX 2012: Network, Security and Storage david.gutierrez@cern.ch Geneva, November 16th Network and Security Network traffic analysis Updates on DC Networks IPv6 Ciber-security updates Federated
Use of Hadoop File System for Nuclear Physics Analyses in STAR
1 Use of Hadoop File System for Nuclear Physics Analyses in STAR EVAN SANGALINE UC DAVIS Motivations 2 Data storage a key component of analysis requirements Transmission and storage across diverse resources
Analisi di un servizio SRM: StoRM
27 November 2007 General Parallel File System (GPFS) The StoRM service Deployment configuration Authorization and ACLs Conclusions. Definition of terms Definition of terms 1/2 Distributed File System The
VIPERVAULT APPASSURE REPLICATION SETUP GUIDE
Solution Overview VIPERVAULT APPASSURE REPLICATION SETUP GUIDE Thank you for choosing the ViperVault AppAssure cloud replication and disaster recovery service. Using this service you can replicate your
Distributed Block-level Storage Management for OpenStack
Distributed Block-level Storage Management for OpenStack OpenStack APAC Conference Daniel Lee CCMA/ITRI Cloud Computing Center for Mobile Applications Industrial Technology Research Institute ( 雲 端 運 算
Client-aware Cloud Storage
Client-aware Cloud Storage Feng Chen Computer Science & Engineering Louisiana State University Michael Mesnier Circuits & Systems Research Intel Labs Scott Hahn Circuits & Systems Research Intel Labs Cloud
Big Data Processing Experience in the ATLAS Experiment
Big Data Processing Experience in the ATLAS Experiment A. on behalf of the ATLAS Collabora5on Interna5onal Symposium on Grids and Clouds (ISGC) 2014 March 23-28, 2014 Academia Sinica, Taipei, Taiwan Introduction
Big data management with IBM General Parallel File System
Big data management with IBM General Parallel File System Optimize storage management and boost your return on investment Highlights Handles the explosive growth of structured and unstructured data Offers
Hadoop IST 734 SS CHUNG
Hadoop IST 734 SS CHUNG Introduction What is Big Data?? Bulk Amount Unstructured Lots of Applications which need to handle huge amount of data (in terms of 500+ TB per day) If a regular machine need to
Chapter 7. Using Hadoop Cluster and MapReduce
Chapter 7 Using Hadoop Cluster and MapReduce Modeling and Prototyping of RMS for QoS Oriented Grid Page 152 7. Using Hadoop Cluster and MapReduce for Big Data Problems The size of the databases used in
SwiftStack Filesystem Gateway Architecture
WHITEPAPER SwiftStack Filesystem Gateway Architecture March 2015 by Amanda Plimpton Executive Summary SwiftStack s Filesystem Gateway expands the functionality of an organization s SwiftStack deployment
Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com
Parallels Cloud Storage White Paper Performance Benchmark Results www.parallels.com Table of Contents Executive Summary... 3 Architecture Overview... 3 Key Features... 4 No Special Hardware Requirements...
Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000
Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000 Alexandra Carpen-Amarie Diana Moise Bogdan Nicolae KerData Team, INRIA Outline
Archiving On-Premise and in the Cloud. March 2015
Archiving On-Premise and in the Cloud March 2015 Cloud Storage Storage accessed over a network via web services APIs. http://swift.example.com/v1/account/container/object Source: http://docs.openstack.org/admin-guide-cloud/content/objectstorage_characteristics.html
Aspera Direct-to-Cloud Storage WHITE PAPER
Transport Direct-to-Cloud Storage and Support for Third Party April 2014 WHITE PAPER TABLE OF CONTENTS OVERVIEW 3 1 - THE PROBLEM 3 2 - A FUNDAMENTAL SOLUTION - ASPERA DIRECT-TO-CLOUD TRANSPORT 5 3 - VALIDATION
OSG Hadoop is packaged into rpms for SL4, SL5 by Caltech BeStMan, gridftp backend
Hadoop on HEPiX storage test bed at FZK Artem Trunov Karlsruhe Institute of Technology Karlsruhe, Germany KIT The cooperation of Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) www.kit.edu
Protect Data... in the Cloud
QUASICOM Private Cloud Backups with ExaGrid Deduplication Disk Arrays Martin Lui Senior Solution Consultant Quasicom Systems Limited Protect Data...... in the Cloud 1 Mobile Computing Users work with their
POWER ALL GLOBAL FILE SYSTEM (PGFS)
POWER ALL GLOBAL FILE SYSTEM (PGFS) Defining next generation of global storage grid Power All Networks Ltd. Technical Whitepaper April 2008, version 1.01 Table of Content 1. Introduction.. 3 2. Paradigm
Introduction to HDFS. Prasanth Kothuri, CERN
Prasanth Kothuri, CERN 2 What s HDFS HDFS is a distributed file system that is fault tolerant, scalable and extremely easy to expand. HDFS is the primary distributed storage for Hadoop applications. HDFS
Scala Storage Scale-Out Clustered Storage White Paper
White Paper Scala Storage Scale-Out Clustered Storage White Paper Chapter 1 Introduction... 3 Capacity - Explosive Growth of Unstructured Data... 3 Performance - Cluster Computing... 3 Chapter 2 Current
ViSION Status Update. Dan Savu Stefan Stancu. D. Savu - CERN openlab
ViSION Status Update Dan Savu Stefan Stancu D. Savu - CERN openlab 1 Overview Introduction Update on Software Defined Networking ViSION Software Stack HP SDN Controller ViSION Core Framework Load Balancer
Computational infrastructure for NGS data analysis. José Carbonell Caballero Pablo Escobar
Computational infrastructure for NGS data analysis José Carbonell Caballero Pablo Escobar Computational infrastructure for NGS Cluster definition: A computer cluster is a group of linked computers, working
Scaling Objectivity Database Performance with Panasas Scale-Out NAS Storage
White Paper Scaling Objectivity Database Performance with Panasas Scale-Out NAS Storage A Benchmark Report August 211 Background Objectivity/DB uses a powerful distributed processing architecture to manage
In Memory Accelerator for MongoDB
In Memory Accelerator for MongoDB Yakov Zhdanov, Director R&D GridGain Systems GridGain: In Memory Computing Leader 5 years in production 100s of customers & users Starts every 10 secs worldwide Over 15,000,000
Investigation of storage options for scientific computing on Grid and Cloud facilities
Investigation of storage options for scientific computing on Grid and Cloud facilities Overview Context Test Bed Lustre Evaluation Standard benchmarks Application-based benchmark HEPiX Storage Group report
SPC BENCHMARK 1 EXECUTIVE SUMMARY 3PAR INC. 3PARINSERV T800 STORAGE SERVER SPC-1 V1.10.1
SPC BENCHMARK 1 EXECUTIVE SUMMARY 3PAR INC. 3PARINSERV T800 STORAGE SERVER SPC-1 V1.10.1 Submitted for Review: September 2, 2008 Submission Identifier: A00069 EXECUTIVE SUMMARY Page 2 of 7 EXECUTIVE SUMMARY
PARALLELS CLOUD STORAGE
PARALLELS CLOUD STORAGE Performance Benchmark Results 1 Table of Contents Executive Summary... Error! Bookmark not defined. Architecture Overview... 3 Key Features... 5 No Special Hardware Requirements...
Understanding Enterprise NAS
Anjan Dave, Principal Storage Engineer LSI Corporation Author: Anjan Dave, Principal Storage Engineer, LSI Corporation SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA
HSS: A simple file storage system for web applications
HSS: A simple file storage system for web applications Abstract AOL Technologies has created a scalable object store for web applications. The goal of the object store was to eliminate the creation of
High Availability Databases based on Oracle 10g RAC on Linux
High Availability Databases based on Oracle 10g RAC on Linux WLCG Tier2 Tutorials, CERN, June 2006 Luca Canali, CERN IT Outline Goals Architecture of an HA DB Service Deployment at the CERN Physics Database
Panasas at the RCF. Fall 2005 Robert Petkus RHIC/USATLAS Computing Facility Brookhaven National Laboratory. Robert Petkus Panasas at the RCF
Panasas at the RCF HEPiX at SLAC Fall 2005 Robert Petkus RHIC/USATLAS Computing Facility Brookhaven National Laboratory Centralized File Service Single, facility-wide namespace for files. Uniform, facility-wide
StorReduce Technical White Paper Cloud-based Data Deduplication
StorReduce Technical White Paper Cloud-based Data Deduplication See also at storreduce.com/docs StorReduce Quick Start Guide StorReduce FAQ StorReduce Solution Brief, and StorReduce Blog at storreduce.com/blog
Datto Whitepaper: Business Continuity Built from the Ground Up. Business Continuity Built from the Ground Up THE ONLY CONTINUITY
Meet the Next Generation of Datto THE ONLY CONTINUITY SOLUTION DESIGNED FOR SMALL BUSINESS SMART SCALABLE STORAGE CONNECTED TO THE CLOUD ENTERPRISE BUSINESS CONTINUITY BUILT FROM THE GROUND UP Meet the
Object storage in Cloud Computing and Embedded Processing
Object storage in Cloud Computing and Embedded Processing Jan Jitze Krol Systems Engineer DDN We Accelerate Information Insight DDN is a Leader in Massively Scalable Platforms and Solutions for Big Data
Creating a Cloud Backup Service. Deon George
Creating a Cloud Backup Service Deon George Agenda TSM Cloud Service features Cloud Service Customer, providing a internal backup service Internal Backup Cloud Service Service Provider, providing a backup
Cloud Storage and Backup
Cloud Storage and Backup Cloud Storage and Backup Cloud Storage and Backup services from iomartcloud have been designed to deliver the performance, capacity, security and flexibility needed to address
Accelerating and Simplifying Apache
Accelerating and Simplifying Apache Hadoop with Panasas ActiveStor White paper NOvember 2012 1.888.PANASAS www.panasas.com Executive Overview The technology requirements for big data vary significantly
The dcache Storage Element
16. Juni 2008 Hamburg The dcache Storage Element and it's role in the LHC era for the dcache team Topics for today Storage elements (SEs) in the grid Introduction to the dcache SE Usage of dcache in LCG
Development of nosql data storage for the ATLAS PanDA Monitoring System
Development of nosql data storage for the ATLAS PanDA Monitoring System M.Potekhin Brookhaven National Laboratory, Upton, NY11973, USA E-mail: potekhin@bnl.gov Abstract. For several years the PanDA Workload
Data Sheet FUJITSU Storage ETERNUS CD10000
Data Sheet FUJITSU Storage ETERNUS CD10000 Data Sheet FUJITSU Storage ETERNUS CD10000 The ultimate hyperscale storage system for large enterprises ETERNUS CD10000 The ETERNUS CD10000 provides unlimited,
Evolution of Database Replication Technologies for WLCG
Home Search Collections Journals About Contact us My IOPscience Evolution of Database Replication Technologies for WLCG This content has been downloaded from IOPscience. Please scroll down to see the full
A Survey on Cloud Storage Systems
A Survey on Cloud Storage Systems Team : Xiaoming Xiaogang Adarsh Abhijeet Pranav Motivations No Taxonomy Detailed Survey for users Starting point for researchers Taxonomy Category Definition Example Instance
Service Overview CloudCare Online Backup
Service Overview CloudCare Online Backup CloudCare s Online Backup service is a secure, fully automated set and forget solution, powered by Attix5, and is ideal for organisations with limited in-house
NoSQL Data Base Basics
NoSQL Data Base Basics Course Notes in Transparency Format Cloud Computing MIRI (CLC-MIRI) UPC Master in Innovation & Research in Informatics Spring- 2013 Jordi Torres, UPC - BSC www.jorditorres.eu HDFS
The State of Cloud Storage
205 Industry Report A Benchmark Comparison of Speed, Availability and Scalability Executive Summary Both 203 and 204 were record-setting years for adoption of cloud services in the enterprise. More than
Big Science and Big Data Dirk Duellmann, CERN Apache Big Data Europe 28 Sep 2015, Budapest, Hungary
Big Science and Big Data Dirk Duellmann, CERN Apache Big Data Europe 28 Sep 2015, Budapest, Hungary 16/02/2015 Real-Time Analytics: Making better and faster business decisions 8 The ATLAS experiment
Data sharing in the Big Data era
www.bsc.es Data sharing in the Big Data era Anna Queralt and Toni Cortes Storage System Research Group Introduction What ignited our research Different data models: persistent vs. non persistent New storage
Drupal in the Cloud. Scaling with Drupal and Amazon Web Services. Northern Virginia Drupal Meetup
Drupal in the Cloud Scaling with Drupal and Amazon Web Services Northern Virginia Drupal Meetup 3 Dec 2008 Cast of Characters Eric at The Case Foundation: The Client With typical client challenges Cost:
AIX NFS Client Performance Improvements for Databases on NAS
AIX NFS Client Performance Improvements for Databases on NAS October 20, 2005 Sanjay Gulabani Sr. Performance Engineer Network Appliance, Inc. gulabani@netapp.com Diane Flemming Advisory Software Engineer
Avid ISIS 7000. www.avid.com
Avid ISIS 7000 www.avid.com Table of Contents Overview... 3 Avid ISIS Technology Overview... 6 ISIS Storage Blade... 6 ISIS Switch Blade... 7 ISIS System Director... 7 ISIS Client Software... 8 ISIS Redundant
Understanding AWS Storage Options
Understanding AWS Storage Options Ian Massingham, Technical Evangelist @IanMmmm ianmas@amazon.com 30 April 2014 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified,
SQL Server Consolidation Using Cisco Unified Computing System and Microsoft Hyper-V
SQL Server Consolidation Using Cisco Unified Computing System and Microsoft Hyper-V White Paper July 2011 Contents Executive Summary... 3 Introduction... 3 Audience and Scope... 4 Today s Challenges...
Product Innovation Award - 2013: Frost & Sullivan
2013 PRODUCT INNOVATION AWARD IN CLOUD INFRASTRUCTURE RECIPIENT HUAWEI TECHNOLOGIES CO., LTD Frost & Sullivan s Global Research Platform Frost & Sullivan is in its 51th year in business with a global research
Scalable NAS for Oracle: Gateway to the (NFS) future
Scalable NAS for Oracle: Gateway to the (NFS) future Dr. Draško Tomić ESS technical consultant, HP EEM 2006 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change
Introduction. Need for ever-increasing storage scalability. Arista and Panasas provide a unique Cloud Storage solution
Arista 10 Gigabit Ethernet Switch Lab-Tested with Panasas ActiveStor Parallel Storage System Delivers Best Results for High-Performance and Low Latency for Scale-Out Cloud Storage Applications Introduction
Running a Workflow on a PowerCenter Grid
Running a Workflow on a PowerCenter Grid 2010-2014 Informatica Corporation. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise)
Quantcast Petabyte Storage at Half Price with QFS!
9-131 Quantcast Petabyte Storage at Half Price with QFS Presented by Silvius Rus, Director, Big Data Platforms September 2013 Quantcast File System (QFS) A high performance alternative to the Hadoop Distributed
Cisco Wide Area Application Services Optimizes Application Delivery from the Cloud
Cisco Wide Area Application Services Optimizes Application Delivery from the Cloud What You Will Learn The adoption of cloud-based computing and applications promises to improve the agility, efficiency,
Hadoop: A Framework for Data- Intensive Distributed Computing. CS561-Spring 2012 WPI, Mohamed Y. Eltabakh
1 Hadoop: A Framework for Data- Intensive Distributed Computing CS561-Spring 2012 WPI, Mohamed Y. Eltabakh 2 What is Hadoop? Hadoop is a software framework for distributed processing of large datasets
Amazon Cloud Storage Options
Amazon Cloud Storage Options Table of Contents 1. Overview of AWS Storage Options 02 2. Why you should use the AWS Storage 02 3. How to get Data into the AWS.03 4. Types of AWS Storage Options.03 5. Object
February, 2015 Bill Loewe
February, 2015 Bill Loewe Agenda System Metadata, a growing issue Parallel System - Lustre Overview Metadata and Distributed Namespace Test setup and implementation for metadata testing Scaling Metadata
Scalable stochastic tracing of distributed data management events
Scalable stochastic tracing of distributed data management events Mario Lassnig mario.lassnig@cern.ch ATLAS Data Processing CERN Physics Department Distributed and Parallel Systems University of Innsbruck