HP + HPC Storage. Gavin Brebner USD Engineering

Transcription

1 HP + HPC Storage Gavin Brebner USD Engineering 2009 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

2 Where storage is used in HPC. Scratch 2 11 March 2010

3 HP covers HPC space with Ibrix + Lustre Lustre high end streaming IO and shared files Ibrix small and large files, data services. Both are scalable solutions performance & capacity can go beyond a single server, network connection, storage array. Both benefit from a fast network i.e. IB HPC Storage Ibrix Overlap Lustre 3 11 March

4 HP and Ibrix is now part of HP 4 11 March 2010

5 Everyone is a content producer Data Analytics Financial Services Insurance Customer Experience Digital Entertainment Web 2.0 Scientific Research Genetic sequencing Oil & Gas Exploration Requires a new way to store, manage, and protect information 5 11 March 2010

6 HPC storage trends Clusters are scalable, distributed, multi-use and commoditized Storage needs to mirror this. Need connection to fast cluster interconnect High throughput / high capacity = lots of hardware Cost is important. Typically want a global shared name space. But not necessarily a single entity underneath Management needs to be simple Admin costs must be kept down. Storage is more complex than just a scratch file system. Multi-location / grid / cloud is common March 2010

7 We can bind different types of storage into one storage solution With Ibrix technology and the X9000 series you can combine : Fast performance storage for scratch FC/SAN storage for capacity pools. Highly dense online archive storage. Tape (2010) In a single storage system. And if appropriate in a single file system March 2010 HP Restricted

8 X9000 solution as seen from the outside PB sized pool of easy-to-manage file storage. High availability access via standard protocols NFS, CIFS, and native Ibrix 8 11 March 2010

9 Introduction to the Ibrix file system 2009 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

10 Scalable NAS Ibrix Fusion is a highly scalable NAS shared file system Up to ~100 servers, ~1000s of clients Petabytes of data in a single namespace, (mount pt.) Files are distributed over many Linux servers on a fast network, without the need for a SAN C C C C C C C C C C C C C C C S S S S S S S S S S S S Scale-out capacity and performance by adding servers Native client (Linux/Windows) for best performance NFS and CIFS support for connectivity to vanilla clients March 2010

11 Previous Solutions Have Bottlenecks Centralized Architectures Metadata Server Namespace Server Lock Manager Token Manager March 2010 Data Data Doesn t Scale Most enterprise application workloads are metadata intensive Small files, small I/Os, random access, metadata operations Central component is a bottleneck Distributed Lock Managers Any server can modify any portion of the file system Doesn t Scale Inter-server messaging grows as (number of servers) 2 Messaging overhead too high beyond few (10) servers Complex failure modes

12 Challenges of Scaling Storage: What Customers Need: Best of Both Many Small File Systems Scalable Single Name Space File System with Resiliency and Fault Containment Advantages: All or Nothing File Systems Independent Performance and Capacity Scaling Manageability Availability March 2010

13 Non-hierarchical Architecture Dir F1 F2 F3 Fn Subdir US Patent # 6,782,389 S1 S2 S3 Sn S1 1 Dir S2 4 F1 Extremely High Aggregate Performance S2 Sn 2 3 F2 F3 S1 Subdir Sn 100 Fn S March 2010 Segment Servers Segments

14 The IBRIX Distribution Engine (IDE) Standard OS Stack OS Stack w/ide Clients Clients Network Drivers Network Drivers TCP / IP NFS / CIFS VFS API Layer PFS Device Drivers Storage IBRIX didn t reinvent the physical file system. TCP / IP NFS / CIFS VFS API Layer IBRIX Distribution Engine API Layer PFS Device Drivers March Storage

15 IBRIX Routing at the Client Application IBRIX Distribution Engine S File Handle Transform Seg # Segment 1 Server S1 IBRIX Distribution Engine IBRIX Physical Filesystem S S2 S3 S1 S2 S3 S1 S3 S3 IBRIX Management Station S 3 Segment Servers Segments March 2010

16 Ibrix networking Ibrix runs on top of TCP/IP Will work with any TCP/IP compatible interconnect. Networking is key for Inter-server communication Critical for NFS/CIFS cases where we re-route messages. Client-server communication Bandwidth to servers will determine system aggregate performance Fast switch, fast interconnect please For benchmarking we use IPoIB on QDR-IB March 2010 HP Restricted

17 High Availability Resilient & Modular Application IBRIX Distribution Engine S High Availability Features: Server Failover 1:1, N:1, 1:N Active/Active or Active/Passive Redundant Component Failover GbE, FC Ports HBAs IBRIX Management Station S NFS Client Table Segment Servers S Segments March 2010 IBRIX Resiliency Model is basis for rolling software upgrade

18 Clients and Servers Ibrix has clients for Linux, Windows And easy re-export over NFS & CIFS Ibrix segment servers are also clients See the entire file system Segment servers can act as NFS or CIFS export servers, without a middle tier (This also enables server-less clusters, with nodes sharing their local disks) Helps enable data services March 2010 HP Restricted - HP Internal Use Only

19 Servers with no storage Ibrix servers cache aggressively (LRU). A server with no storage can still be used as an extra NFS node Can be positioned far from the main cluster to give good local NFS performance for a remote cluster. X9300 variant will permit large memory configurations March 2010

20 Misc File system features Posix Quotas (user, group, hard and soft) including dir tree quotas. Re-export NFS, CIFS AD for CIFS RHEL5 based. Clients can be 64b Linux, Windows March 2010

21 Data Services 2009 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

22 Continuous Remote Replication New York London Site A Data Replication Site B March 2010

23 What is Continuous Remote Replication? Continuous replication of changes to the file system from one cluster to another Each cluster has its own Fusion Manager management console Replicates updates to both files and directories Replication can be done over the User or Cluster network Stats on processed files and backlog queue Displays the number of files processed and remaining in the queue Includes initial synchronization of the two file systems Permits special run-once jobs to synchronize specific directories March 2010

24 Continuous and Parallel Tracks changes on the source file system and continuously replicates these changes to the target file system ASAP Each segment server replicates the changes in parallel and independently Site A + Parallel Continuous Site B March 2010

25 IBRIX Fusion File System - Rebalancer Functionality Dynamic, non-disruptive redistribution of data across physical storage elements to provide balanced storage utilization 1. Existing filesystem approaching capacity Business Benefit Improves performance by avoiding hot-spots on existing storage or when new storage capacity is added Improves storage procurement process through efficient utilization for existing storage Lowers administrative overhead and operational costs by automating rebalancing March Add storage to existing filesystem 3. Distribute active filesystem across all available storage to insure efficient utilization

26 Storage Tier Levels Fibre Channel Storage (Tier 1) Flash/SSD Storage (Tier 0) SATA Storage (Tier 2) March 2010

27 Data Tiering Ability to migrate files from one tier of storage to another within the same namespace based on file characteristics or attributes The policies for triggering file movement can be based on several different attributes This is a scheduled activity performed in the background by the system Tier 0 Flash/SSD; Tier-1 FC; Tier-2 SATA SS1 SS Tier Tier March 2010 SINGLE NAMESPACE

28 Policy for File Movement Access time File was last accessed X or more days ago File was accessed in the last Y days Modification time File was last modified X or more days ago File size greater than Z KB File Name or File type - jpg, wav, exe, etc. (Inclusion/Exclusion) File ownership owned by user(s) (Inclusion/Exclusion) Combinations can be created using the AND or OR operators March 2010

29 Data Tiering Release Features Ability to assign segments to Tiers Ability to prefer a Tier for initial placement of all files Ability to move files between any two Tiers Define policy for file movement Ability to schedule or manually run the task Use the cron job to schedule the task March 2010

30 HP X9000 composite Scale-out storage solutions Existing Tape Libraries An X9000 Single Namespace grows up to 16 PB X9300 X9300 X9320 X9320 X9720 X TB Performance 192TB Capacity Existing SAN and Legacy Storage March 2010 X PB Management Server Extreme density

31 HP StorageWorks X9720 X9720cx: cascaded dense storage Dual drawer Fully connected and operational when pulled out 70x1TB* SAS midline disks drives in 5U All hot swappable 4 SAS expansion ports 2 per drawer Dual redundant power supplies and fans per drawer All hot swappable Front display showing drive status (green, orange, ) Power supply Front Display Fan * 2TB from 03/ March 2010 HP Restricted - HP and HP channel partners only IO Module

32 GUI File System View (Default screen) March 2010

33 No add-on software costs Avoid controller bottlenecks X9000 series Scalable performance and capacity Automated rebalancing Maintain consistent system performance over time Data tiering Optimize storage efficiency Continuous remote replication ❺ Integrated capabilities for disaster recovery Single management UI for all X9000 systems Simplify and consolidate NAS management March 2010

34 Example Configurations 2009 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

35 Maximum performance Scale out by adding fast appliances X9320 SAS Appliance X9320 SAS Appliance X9320 SAS Appliance X9320 SAS Appliance X9320 SAS Appliance X9320 SAS Appliance March 2010 X9000 Mgmt UI Single Namespace

36 Maximum capacity Scale out by adding capacity appliances March 2010

37 Good enough perf and massive capacity Combine performance and capacity X9320 SAS Appliance X9000 Mgmt UI X9320 SAS Appliance March 2010 HP Restricted

38 Single file system split across 2 sites Local tiers defined for each site SiteAFastTier SiteBFastTier X9000 Mgmt UI SiteACapacityTier SiteBCapacityTier March 2010 HP Restricted

39 Follow the work flow. Single global name space multiple tasks and locations Post-processing Pre-processing storage X9000 Mgmt UI March 2010 HP Restricted

40 The X9000 Summary Key characteristics Consolidated Namespace up to 16PB 255 Separate namespaces (file systems) Scales to 1000s of nodes Broad use for applications Multiple NAS protocols Highly available Scales non-disruptively 3 hardware form factors SINGLE NAME SPACE Built-in functionality Data tiering Online Rebalancer Continuous, parallel, remote replication Simple Management March 2010

41 For More Information age/default.aspx twitter.com/scalable_nas twitter.com/hpstorageguy March 2010