1 Panasas at the RCF HEPiX at SLAC Fall 2005 Robert Petkus RHIC/USATLAS Computing Facility Brookhaven National Laboratory
2 Centralized File Service Single, facility-wide namespace for files. Uniform, facility-wide POSIX-like access to data. Transparent to end-user.
3 Current Storage Implementation Traditional SAN ~220 TB fibre-channel disks in RAID 5 arrays 13 TB IDE storage 24 Brocade fibre-channel switches 37 Sun Solaris 9 servers (E450, V480, V240) using Veritas 4.0 (VxVM, VxFS) NFS transfer rate: 70MB/sec/server I/O throughput to disks: MB/sec writes, 75 MB/sec reads
4 Current Storage Implementation Panasas 100 TB on 20 shelves. 10 Realms. 2 trunked 1000BaseT network ports (out of 4) enabled. Data transfer rate from server ~250 MB/sec max per shelf. DirectFLOW client deployed on ~2000 node RHIC analysis/reconstruction farm.
7 VXFS and VXVM Pros Veritas Issues Ability to shrink file systems. Easy to import/deport volumes among servers. Dynamic multipathing Cons Quotas broken on file systems > 1 TB Expensive Poor customer support
8 Panasas Features Fast, scalable, reliable and fault tolerant Direct and parallel data access Load balancing (Efficient resource utilization) Incremental growth Integrated hardware / software solution Economic benefits of IDE disk Security and centralized management Global namespace Distributed metadata Active development Excellent customer support
9 Spectrum of Solutions GPFS: data block striping across disks. Parallel reads/writes. Fail-over, replication, distributed metadata. Xrootd: Load balancing, single namespace, proxy servers. Client access through TXNetFile (ROOT framework) BlueArc: ASICs dedicated to NFS, network, and filesystem. Feature-rich, fast and robust NFS solution. Ibrix: Meta servers assigned to segments in a disk pool. Traditional NFS or Fusion. Linux LVS model. Dcache: Management of heterogeneous storage repositories. Hot spot detection. Lustre: Object-based storage solution. Software only. Active development. Older versions are free.
10 Various Models POSIX-like vs Non POSIX-like Hardware vs Software Existing Protocol vs New Protocol Ethernet vs Fibre Channel Unlimited Scalability vs Just Fast Enough Dedicated vs Non-Dedicated Resources Proprietary vs Open
11 Panasas Architecture Components of each shelf at the RCF 1 Director Blade 10 Storage Blades 1 Gigabit internal ethernet ports per blade (11 total) 4 Gigabit links to external network 5 or 8 TB raw disk/shelf
12 Panasas Architecture Director Blade Director Blades (The brains) File Namespace server(s) Manages Metadata object map Coordinates between clients and Storage Blades Determines RAID characteristics of a file. Determines distribution of file objects over OSD's Specifications 2.4 GHz Intel Xeon 4 GB RAM BaseT ports FreeBSD
13 Panasas Architecture Storage Blade Storage Blades (Object Storage Devices or OSD's) Store and retrieve data objects Handles I/O to client Specifications Each Storage Blade (10) 500GB or 800GB storage (2 SATA 250GB or 400GB HDD) 1.2 GHz Intel Celeron 512 MB RAM 100/1000BaseT FreeBSD
14 Panasas Architecture ActiveScale Operating System Runs on Director Blade Divides files into data objects, which are arbitrary in size, and stripes them across storage blades Dynamically distributes workload across storage blades Each storage blades is only filled to 90% capacity. The remaining 10% is reserved for rebuilding parity. Direct Flow Software Installed on the Linux compute node Direct data path from client to storage blades Optimizes data layout, caching and prefetching File is reconstructed at the compute node
15 Panasas performance snapshots
16 Panasas performance snapshots
17 Issues No LDAP support No user or group quotas volume quotas only A new command pan_du is in development to list the amount of storage used per user but no enforcement. No on-line director blade failover Can't downgrade ActiveScale DirectFLOW memory leaks fixed? DHCP server stubborn to release addresses need an excess pool of IP addresses in case a blade crashes.
18 Issues Maxed-out volumes are unstable. Hard volume quotas need to be set at 95% of total usable capacity for protection. Atime records written to disk on writes not reads (JLAB) Hard limit on files per directory (350k) Mass-deletes are not handled gracefully.
19 Conclusions Panasas scalability and performance much better than NFS. Easy to administrate and reliable. Responsive, knowledgeable support staff. Intermittent client-side DirectFLOW crashes are frustrating on a very large cluster. Possibility of separating software from hardware? Emergence of strong competitors. NFSv4.1? What role will centralized storage play in a grid environment?
NDMP Backup of Dell EqualLogic FS Series NAS using CommVault Simpana A Dell EqualLogic Reference Architecture Dell Storage Engineering June 2013 Revisions Date January 2013 June 2013 Description Initial
TECHNICAL BRIEF:........................................ Running Highly Available, High Performance Databases in a SAN-Free Environment Who should read this paper Architects, application owners and database
HP StoreOnce Catalyst and HP Data Protector 7 Implementation and Best Practice Guide Release 1 Executive Summary This guide is intended to enable the reader to understand the basic technology of HP StoreOnce
UNIVERSITY OF CALIFORNIA SANTA CRUZ CEPH: RELIABLE, SCALABLE, AND HIGH-PERFORMANCE DISTRIBUTED STORAGE A dissertation submitted in partial satisfaction of the requirements for the degree of DOCTOR OF PHILOSOPHY
ware vcloud Implementation Example Public vcloud Service Provider TECHNICAL WHITE PAPER Document Title Table of Contents 1. Purpose and Overview... 4 1.1 Executive Summary... 4 1.2 Business Requirements...
Proven Infrastructure Guide EMC VSPEX PRIVATE CLOUD VMware vsphere 5.5 for up to 1,000 Virtual Machines Enabled by Microsoft Windows Server 2012 R2, EMC VNX Series, and EMC Powered Backup EMC VSPEX Abstract
EonNAS 000 Series Fully optimized for the applications that power your business Why EonNAS 000 Infortrend s EonNAS 000 Series offers rackmount NAS solutions designed to help SMBs users optimize file sharing,
Microsoft System Center 2012 R2 Why Microsoft? For Virtualizing & Managing SharePoint July 2014 v1.0 2014 Microsoft Corporation. All rights reserved. This document is provided as-is. Information and views
Network Data Management Protocol (NDMP) White Paper Paul Mayer Product Manager Revision 2 April 2002 Table of Contents Introduction...1 Network data management protocol (NDMP)...3 NAS backup configurations...6
Microsoft Corporation and HP Using Network Attached Storage for Reliable Backup and Recovery Microsoft Corporation Published: March 2010 Abstract Tape-based backup and restore technology has for decades
What s New in the VMware vsphere 6.0 Platform VERSION 1.1/TECHNICAL WHITE PAPER MARCH 2015 Table of Contents Introduction.... 3 vsphere Hypervisor Enhancements.... 3 Scalability Improvements.... 3 ESXi
Data Protection for Isilon Scale-Out NAS A Data Protection Best Practices Guide for Isilon IQ and OneFS By David Thomas, Solutions Architect An Isilon Systems Best Practices Guide May 2009 ISILON SYSTEMS
WHITE PAPER CONNECTING SANS OVER METROPOLITAN AND WIDE AREA NETWORKS Internetworking technologies help link Fibre Channel SANs over existing long-distance connections As Storage Area Network (SAN) infrastructures
Database Configuration: SAN or NAS Discussion of Fibre Channel SAN and NAS Version:.0 Greg Schulz DBSANNAS.DOC Preface This publication is for general guidance and informational purposes only. The information
Best Practices Guide McAfee epolicy Orchestrator for use with epolicy Orchestrator versions 4.5.0 and 4.0.0 COPYRIGHT Copyright 2011 McAfee, Inc. All Rights Reserved. No part of this publication may be
Hyper-V Live Migration over Distance Reference Architecture Guide By Hitachi Data Systems in collaboration with Microsoft, Brocade and Ciena June 2010 Summary Hitachi Data Systems, Microsoft, Brocade and
EMC Backup and Recovery Solutions Use Cases and Quick Selling Guides Leveraging g best-in-class deduplication solutions for next-generation backup May 2011 Copyright 2011 EMC Corporation. All rights reserved.
Best Practices and Recommendations for Scale-up Deployments of SAP HANA on VMware vsphere DEPLOYMENT AND TECHNICAL CONSIDERATIONS GUIDE Table of Contents Introduction...................................................................
OPEN DATA CENTER ALLIANCE MASTER USAGE Model: Scale-Out Storage Rev. 1.0 Table of Contents Legal Notice...3 Executive Summary...4 Purpose...5 Understanding the Scale-Out Storage Model...6 Common Scale-Out
NEC Storage NV Series Guide to Oracle Storage Compatibility Program Network Attached File Server Snapshot Technologies is-wp-04-001 Rev-1.00 February, 2005 NEC Corporation. - 1 - Copyright 2005 NEC Corporation.
Kingston Communications Virtualisation Platforms The KC Managed Server is architected to provide a dedicated server based on the hugely successful VMWare platform. By using the Enterprise version of VMware
GamCo Managed Dedicated Virtual Hosting Business growth in the number of users, application and data has left many organisations with sprawling servers, storage and supporting infrastructure crammed into
4-Bay SATA NAS RAID Server with iscsi Hardware High performance Intel Dual Core 1.8GHz processor Provides scalability up to 16TB (with 4TB per Hard Drive) Tray-less design for genuine plug & play and hot
ES 2 : A Cloud Data Storage System for Supporting Both OLTP and OLAP Yu Cao, Chun Chen,FeiGuo, Dawei Jiang,YutingLin, Beng Chin Ooi, Hoang Tam Vo,SaiWu, Quanqing Xu School of Computing, National University
Overview Hewlett Packard's MSA P2000 Family of storage arrays features P2000 G3 arrays with the latest 8 Gb Fibre Channel, 6 Gb SAS, 10GbE iscsi connected models, and an iscsi model with four 1 Gb iscsi
Plug Into The Cloud with Oracle Database 12c ORACLE WHITE PAPER DECEMBER 2014 Disclaimer The following is intended to outline our general product direction. It is intended for information purposes only,