Lustre SMB Gateway. Integrating Lustre with Windows

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Lustre SMB Gateway. Integrating Lustre with Windows"

Transcription

1 Lustre SMB Gateway Integrating Lustre with Windows

2 Hardware: Old vs New Compute 60 x Dell PowerEdge x 2.6Ghz cores, 16GB, 500GB Sata, 1GBe - Win7 x64 Storage 1 x Dell R x 2TB Sata, RAID5, 1GBe - Centos 5 Compute 63 x Dell 7910 rack workstation - 24 x 2.5Ghz cores, 64GB, 4 x 900GB 10k SAS, 256GB SSD, 2 x 10Gbe - ESXi 5.5u2 Hypervisor & Win7 VM Storage 4 x Dell R630-8 x 2.4Ghz cores, 64GB, 4 x 600GB 10k SAS, 2 x 10Gbe, 1 x dualport Mellanox ConnectX3-1 x MD3460 Array, 42 x 600GB 10k, 1TB flash - Red Hat EL6 Network Cisco Nexus 2232TM-E Fabric Extender Cisco Nexus 6K 6001 Switches Qlogic QDR Infiniband switches 2

3 3

4 What is CTDB? Clustered implementation of Trivial Database system High Availability service for clustered file-system 4

5 Why use CTDB? Compute nodes had to be Windows 7 x64 but no native Windows Lustre client existed Save costs on Infiniband network hardware NFS client in Windows is average Opportunity to leverage existing network infrastructure in the Datacentre If Windows could do something well.. CIFS/SMB access would be one of them 5

6 Differences in CTDB vs SMB CTDB Many hosts for single file-system CTDB service can manage SMB NMB Winbind Host resiliency inbuilt Recovery file lock between CTDB hosts Shared password db SMB Single SMB host per file-system No failover Less potential bandwidth 6

7 How we implemented CTDB 2 x Physical nodes Simple tdbsam password database Bonded 2 x 10Gbe per host Single QDR link per host to Lustre Local config files [smb.conf, public_addresses,nodes, etc] Shared config / working files [*.tdb, recovery_lock, etc] Round Robin DNS for public IPs 7

8 HPC Render Cluster Lustre Storage 12 x Object Servers 2 x Metadata Servers QDR Infiniband Network VMware Compute Cluster 63 x Dell 7910 rack workstation 10GBe connectivity per host 1:1 VM to Host mapping CTDB Hosts 4 x Dell R630 2 x 10Gbe in 802.3ad for data 2 x 1Gbe in 802.3ad for heartbeat 1 x Mellanox 2-port ConnectX3 IB ctdb-01 ctdb-02 ctdb-03 ctdb-04 CTDB heartbeat Data Network Networking In rack Cisco Nexus 2232TM-E FEX 4 x 10 Gbit uplink per FEX Pod Cisco Nexus 6001 Switch 6 x 10 Gbit to core network

9 HPC Render Cluster Lustre Storage 12 x Object Servers 2 x Metadata Servers QDR Infiniband Network VMware Compute Cluster 63 x Dell 7910 rack workstation 10GBe connectivity per host 1:1 VM to Host mapping CTDB Hosts 2 x Dell R630 2 x 10Gbe 802.3ad for data 1 x 1Gbe heartbeat (crossover) 1 x Mellanox 2-port IB HA-SMB Hosts 2 x Dell R630 2 x 10Gbe in 802.3ad for data 1 x 1Gbe heartbeat (crossover) ctdb-01 ctdb-02 HA-smb-01 HA-smb-02 CTDB heartbeat CCS heartbeat Data Network Networking In rack Cisco Nexus 2232TM-E FEX 4 x 10 Gbit uplink per FEX Pod Cisco Nexus 6001 Switch 6 x 10 Gbit to core network

10 Storage Services Scratch volume Red Hat EL6 host 17 TB Direct attached storage Two node cluster providing an SMB service with HA failover Clustered LVM with EXT4 FS Bandwidth 20Gbit 42 x 600GB DDP, 1TB Flash Archive volume Red Hat EL6 host Samba gateway to 3.0 PB Lustre file-system Clustered TDB (CTDB) used to provide SMB service Lustre FS direct mounted on host CTDB shares a lustre directory Bandwidth up to 40Gbit 10

11 How the storage is used Storing 8k IMAX film renders Min. 5MB image frame 50,000 frames ~275MB/s stream rate Around GB of renders for 35 mins footage Streaming from Lustre archive back into final film cuts 11

12 Sample 8k render RGB 12

13 How does CTDB perform in practice? Approximately 30 seconds for complete takeover Tools and documentation are helpful Measured throughput is good ~600MB/s for archive sync from scratch disk over ethernet Normal system load on CTDB hosts with many connections Basic config is reliable 13

14 What did not work well? Load balancing: TCP connections killed as part of failover Round Robin DNS effect in practice Some notify.sh script handling for monitoring Limited set of default handlers for alerting of cluster state Integrating with existing SMB services: Internal CTDB service scripts modify eth adapter state Splitting out NMB & SMB pids, configs, init scripts was problematic 14

15 Ideas to improve our CTDB Lustre service Load balancing: Hardware load balancer for initial connection but not transfers Write a software service to manage connections Red Hat EL6 Load Balancer Add-on? Write more comprehensive scripts to alert on cluster state 15

16 Test results: Lustre disk 2 x CTDB host 12 x clients 4GB data size Sequential read Sequential write Random 512k read Random 512k write 16

17 Test results: Render Node FEXs 17

18 Test results: CTDB Node FEXs 18

19 Conclusions Reasonable performance Reliable and manageable storage service Scalable [as a gateway service] Provides greater accessibility to lustre for non-intensive data applications 19

20 Basic CTDB Config Mounted Lustre with -o flock ~]# mount grep lustre on /lustre type lustre (rw,flock) ~]#

21 Basic CTDB Config Created data, and private directory for ctdb operations ~]# ls -la /lustre/ctdb drwx root root 4096 Jan ctdb drwxr-xr-x 3 root root 4096 Feb data ~]#

22 Basic CTDB Config Configured /etc/sysconfig/ctdb ~]# egrep -v '#' /etc/sysconfig/ctdb CTDB_RECOVERY_LOCK=/lustre/ctdb/.ctdb/recovery_lock CTDB_NODES=/etc/ctdb/nodes CTDB_PUBLIC_ADDRESSES=/etc/ctdb/public_addresses CTDB_MANAGES_SAMBA=yes ulimit -n CTDB_LOGFILE=/var/log/log.ctdb CTDB_DEBUGLEVEL=NOTICE CTDB_NOTIFY_SCRIPT=/etc/ctdb/notify.sh ~]#

23 Basic CTDB Config Configured node file (Heartbeat network) ~]# cat /etc/ctdb/nodes ~]#

24 Basic CTDB Config Configured public_addresses file (service IP s) ~]# cat /etc/ctdb/public_addresses /24 intelbond /24 intelbond0 ~]#

25 Basic CTDB Config Configured public_addresses file (service IP s) ~]# cat /etc/ctdb/public_addresses /24 intelbond /24 intelbond0 ~]#

26 Setup smb.conf Basic CTDB Config ~]# egrep -v '#' /etc/samba/smb.conf [global] workgroup = render server string = Samba Server Version %v netbios name = render-archive interfaces = intelbond0 hosts allow = clustering = yes ctdbd socket = /var/run/ctdb/ctdbd.socket cluster addresses = idmap backend = tdb2 bind interfaces only = no pid directory = /var/run/samba/ctdb private dir = /lustre/ctdb/.ctdb/privdir fileid:mapping = fsname use mmap = no nt acl support = yes ea support = yes log file = /var/log/samba/ctdb/log.%m max log size = 50 security = user passdb backend = tdbsam load printers = no printing = bsd printcap name = /dev/null [archive] comment = Lustre Archive browseable = no writable = yes path = /lustre/ctdb/data/archive ~]#

Clustered CIFS For Everybody Clustering Samba With CTDB. LinuxTag 2009

Clustered CIFS For Everybody Clustering Samba With CTDB. LinuxTag 2009 Clustered CIFS For Everybody Clustering Samba With CTDB LinuxTag 2009 Michael Adam obnox@samba.org 2009-06-24 Contents 1 Cluster Challenges 2 1.1 The Ideas............................... 2 1.2 Challenges

More information

# Windows Internet Name Serving Support Section: # WINS Support - Tells the NMBD component of Samba to enable its WINS Server ; wins support = no

# Windows Internet Name Serving Support Section: # WINS Support - Tells the NMBD component of Samba to enable its WINS Server ; wins support = no Sample configuration file for the Samba suite for Debian GNU/Linux. This is the main Samba configuration file. You should read the smb.conf(5) manual page in order to understand the options listed here.

More information

Load Balancing and High availability using CTDB + DNS round robin

Load Balancing and High availability using CTDB + DNS round robin Introduction As you may already know, GlusterFS provides several methods for storage access from clients. However, only the native FUSE GlusterFS client has built-in failover and high availability features.

More information

The Simple High Available Linux File Server. Schlomo Schapiro Principal Consultant Leitung Virtualisierung & Open Source

The Simple High Available Linux File Server. Schlomo Schapiro Principal Consultant Leitung Virtualisierung & Open Source The Simple High Available Linux File Server Schlomo Schapiro Principal Consultant Leitung Virtualisierung & Open Source 18.04.2008 Agenda Background Information Possible Solutions The Simple High Available

More information

Large Scale Storage. Orlando Richards, Information Services orlando.richards@ed.ac.uk. LCFG Users Day, University of Edinburgh 18 th January 2013

Large Scale Storage. Orlando Richards, Information Services orlando.richards@ed.ac.uk. LCFG Users Day, University of Edinburgh 18 th January 2013 Large Scale Storage Orlando Richards, Information Services orlando.richards@ed.ac.uk LCFG Users Day, University of Edinburgh 18 th January 2013 Overview My history of storage services What is (and is not)

More information

Homework 5b: Homework 5b: Samba

Homework 5b: Homework 5b: Samba Homework 5b: Samba Network-based File Sharing (1) NFS (UNIX-based) mountd is responsible for mount request nfsd and nfsiod CIFS (Microsoft) Common Internet File System 網 路 芳 鄰 2 Network-based File Sharing

More information

SMB Direct for SQL Server and Private Cloud

SMB Direct for SQL Server and Private Cloud SMB Direct for SQL Server and Private Cloud Increased Performance, Higher Scalability and Extreme Resiliency June, 2014 Mellanox Overview Ticker: MLNX Leading provider of high-throughput, low-latency server

More information

VIDEO SURVEILLANCE WITH SURVEILLUS VMS AND EMC ISILON STORAGE ARRAYS

VIDEO SURVEILLANCE WITH SURVEILLUS VMS AND EMC ISILON STORAGE ARRAYS VIDEO SURVEILLANCE WITH SURVEILLUS VMS AND EMC ISILON STORAGE ARRAYS Successfully configure all solution components Use VMS at the required bandwidth for NAS storage Meet the bandwidth demands of a 2,200

More information

Preparation Guide. How to prepare your environment for an OnApp Cloud v3.0 (beta) deployment.

Preparation Guide. How to prepare your environment for an OnApp Cloud v3.0 (beta) deployment. Preparation Guide v3.0 BETA How to prepare your environment for an OnApp Cloud v3.0 (beta) deployment. Document version 1.0 Document release date 25 th September 2012 document revisions 1 Contents 1. Overview...

More information

Step-by-Step Guide to Open-E DSS V7 Active-Active Load Balanced iscsi HA Cluster

Step-by-Step Guide to Open-E DSS V7 Active-Active Load Balanced iscsi HA Cluster www.open-e.com 1 Step-by-Step Guide to Open-E DSS V7 Active-Active Load Balanced iscsi HA Cluster (without bonding) Software Version: DSS ver. 7.00 up10 Presentation updated: May 2013 www.open-e.com 2

More information

Post-production Video Editing Solution Guide with Microsoft SMB 3 File Serving AssuredSAN 4000

Post-production Video Editing Solution Guide with Microsoft SMB 3 File Serving AssuredSAN 4000 Post-production Video Editing Solution Guide with Microsoft SMB 3 File Serving AssuredSAN 4000 Dot Hill Systems introduction 1 INTRODUCTION Dot Hill Systems offers high performance network storage products

More information

Configure Samba with ACL and Active Directory integration Robert LeBlanc (leblanc@byu.edu) BioAg Computer Support, Brigham Young University

Configure Samba with ACL and Active Directory integration Robert LeBlanc (leblanc@byu.edu) BioAg Computer Support, Brigham Young University Configure Samba with ACL and Active Directory integration Robert LeBlanc (leblanc@byu.edu) BioAg Computer Support, Brigham Young University This document uses Debain Linux 3.1 (Sarge) on x86 hardware.

More information

Quantum StorNext. Product Brief: Distributed LAN Client

Quantum StorNext. Product Brief: Distributed LAN Client Quantum StorNext Product Brief: Distributed LAN Client NOTICE This product brief may contain proprietary information protected by copyright. Information in this product brief is subject to change without

More information

BRIDGING EMC ISILON NAS ON IP TO INFINIBAND NETWORKS WITH MELLANOX SWITCHX

BRIDGING EMC ISILON NAS ON IP TO INFINIBAND NETWORKS WITH MELLANOX SWITCHX White Paper BRIDGING EMC ISILON NAS ON IP TO INFINIBAND NETWORKS WITH Abstract This white paper explains how to configure a Mellanox SwitchX Series switch to bridge the external network of an EMC Isilon

More information

Multiple Public IPs (virtual service IPs) are supported either to cover multiple network segments or to increase network performance.

Multiple Public IPs (virtual service IPs) are supported either to cover multiple network segments or to increase network performance. EliteNAS Cluster Mirroring Option - Introduction Real Time NAS-to-NAS Mirroring & Auto-Failover Cluster Mirroring High-Availability & Data Redundancy Option for Business Continueity Typical Cluster Mirroring

More information

Step-by-Step Guide to Open-E DSS V7 Active-Active iscsi Failover

Step-by-Step Guide to Open-E DSS V7 Active-Active iscsi Failover www.open-e.com 1 Step-by-Step Guide to Software Version: DSS ver. 7.00 up10 Presentation updated: June 2013 www.open-e.com 2 TO SET UP ACTIVE-ACTIVE ISCSI FAILOVER, PERFORM THE FOLLOWING STEPS: 1. Hardware

More information

Introduction to Gluster. Versions 3.0.x

Introduction to Gluster. Versions 3.0.x Introduction to Gluster Versions 3.0.x Table of Contents Table of Contents... 2 Overview... 3 Gluster File System... 3 Gluster Storage Platform... 3 No metadata with the Elastic Hash Algorithm... 4 A Gluster

More information

HPC Update: Engagement Model

HPC Update: Engagement Model HPC Update: Engagement Model MIKE VILDIBILL Director, Strategic Engagements Sun Microsystems mikev@sun.com Our Strategy Building a Comprehensive HPC Portfolio that Delivers Differentiated Customer Value

More information

New Storage System Solutions

New Storage System Solutions New Storage System Solutions Craig Prescott Research Computing May 2, 2013 Outline } Existing storage systems } Requirements and Solutions } Lustre } /scratch/lfs } Questions? Existing Storage Systems

More information

Highly-Available Distributed Storage. UF HPC Center Research Computing University of Florida

Highly-Available Distributed Storage. UF HPC Center Research Computing University of Florida Highly-Available Distributed Storage UF HPC Center Research Computing University of Florida Storage is Boring Slow, troublesome, albatross around the neck of high-performance computing UF Research Computing

More information

Fakultät für Physik Ludwig Maximilians Universität Klaus Steinberger Ralph Simmler Alexander Thomas. HA Cluster using Open Shared Root

Fakultät für Physik Ludwig Maximilians Universität Klaus Steinberger Ralph Simmler Alexander Thomas. HA Cluster using Open Shared Root Ludwig Maximilians Universität Klaus Steinberger Ralph Simmler Alexander Thomas HA Cluster using Open Shared Root Agenda Our Requirements The Hardware Why Openshared Root Technical Details of Openshared

More information

Step-by-Step Guide. to configure Open-E DSS V7 Active-Active iscsi Failover on Intel Server Systems R2224GZ4GC4. Software Version: DSS ver. 7.

Step-by-Step Guide. to configure Open-E DSS V7 Active-Active iscsi Failover on Intel Server Systems R2224GZ4GC4. Software Version: DSS ver. 7. Step-by-Step Guide to configure on Intel Server Systems R2224GZ4GC4 Software Version: DSS ver. 7.00 up01 Presentation updated: April 2013 www.open-e.com 1 www.open-e.com 2 TECHNICAL SPECIFICATIONS OF THE

More information

Hyper-V over SMB Remote File Storage support in Windows Server 8 Hyper-V. Jose Barreto Principal Program Manager Microsoft Corporation

Hyper-V over SMB Remote File Storage support in Windows Server 8 Hyper-V. Jose Barreto Principal Program Manager Microsoft Corporation Hyper-V over SMB Remote File Storage support in Windows Server 8 Hyper-V Jose Barreto Principal Program Manager Microsoft Corporation Agenda Hyper-V over SMB - Overview How to set it up Configuration Options

More information

SMB Advanced Networking for Fault Tolerance and Performance. Jose Barreto Principal Program Managers Microsoft Corporation

SMB Advanced Networking for Fault Tolerance and Performance. Jose Barreto Principal Program Managers Microsoft Corporation SMB Advanced Networking for Fault Tolerance and Performance Jose Barreto Principal Program Managers Microsoft Corporation Agenda SMB Remote File Storage for Server Apps SMB Direct (SMB over RDMA) SMB Multichannel

More information

Lustre failover experience

Lustre failover experience Lustre failover experience Lustre Administrators and Developers Workshop Paris 1 September 25, 2012 TOC Who we are Our Lustre experience: the environment Deployment Benchmarks What's next 2 Who we are

More information

www.thinkparq.com www.beegfs.com

www.thinkparq.com www.beegfs.com www.thinkparq.com www.beegfs.com KEY ASPECTS Maximum Flexibility Maximum Scalability BeeGFS supports a wide range of Linux distributions such as RHEL/Fedora, SLES/OpenSuse or Debian/Ubuntu as well as a

More information

NexentaStor Enterprise Backend for CLOUD. Marek Lubinski Marek Lubinski Sr VMware/Storage Engineer, LeaseWeb B.V.

NexentaStor Enterprise Backend for CLOUD. Marek Lubinski Marek Lubinski Sr VMware/Storage Engineer, LeaseWeb B.V. NexentaStor Enterprise Backend for CLOUD Marek Lubinski Marek Lubinski Sr VMware/Storage Engineer, LeaseWeb B.V. AGENDA LeaseWeb overview Express Cloud platform Initial storage build Why NexentaStor NexentaStor

More information

Comparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet. September 2014

Comparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet. September 2014 Comparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet Anand Rangaswamy September 2014 Storage Developer Conference Mellanox Overview Ticker: MLNX Leading provider of high-throughput,

More information

RFP-MM-1213-11067 Enterprise Storage Addendum 1

RFP-MM-1213-11067 Enterprise Storage Addendum 1 Purchasing Department August 16, 2012 RFP-MM-1213-11067 Enterprise Storage Addendum 1 A. SPECIFICATION CLARIFICATIONS / REVISIONS NONE B. REQUESTS FOR INFORMATION Oracle: 1) What version of Oracle is in

More information

SuSE File and Print Services with

SuSE File and Print Services with SuSE File and with SuSE Linux Enterprise Server 8 SuSE Linux AG Whitepaper SuSE File and with SuSE Linux Enterprise Server 8 Overview. 3 File Services The User View 3 The Administrator View 3 Samba Web

More information

Samba. Samba. Samba 2.2.x. Limitations of Samba 2.2.x 1. Interoperating with Windows. Implements Microsoft s SMB protocol

Samba. Samba. Samba 2.2.x. Limitations of Samba 2.2.x 1. Interoperating with Windows. Implements Microsoft s SMB protocol Samba Samba Interoperating with Windows Nick Urbanik Copyright Conditions: GNU FDL (seehttp://www.gnu.org/licenses/fdl.html) A computing department Implements Microsoft s SMB protocol

More information

IT of SPIM Data Storage and Compression. EMBO Course - August 27th! Jeff Oegema, Peter Steinbach, Oscar Gonzalez

IT of SPIM Data Storage and Compression. EMBO Course - August 27th! Jeff Oegema, Peter Steinbach, Oscar Gonzalez IT of SPIM Data Storage and Compression EMBO Course - August 27th Jeff Oegema, Peter Steinbach, Oscar Gonzalez 1 Talk Outline Introduction and the IT Team SPIM Data Flow Capture, Compression, and the Data

More information

Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre

Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre University of Cambridge, UIS, HPC Service Authors: Wojciech Turek, Paul Calleja, John Taylor

More information

Lessons learned from parallel file system operation

Lessons learned from parallel file system operation Lessons learned from parallel file system operation Roland Laifer STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association

More information

FLOW-3D Performance Benchmark and Profiling. September 2012

FLOW-3D Performance Benchmark and Profiling. September 2012 FLOW-3D Performance Benchmark and Profiling September 2012 Note The following research was performed under the HPC Advisory Council activities Participating vendors: FLOW-3D, Dell, Intel, Mellanox Compute

More information

Linux Administrator (Advance)

Linux Administrator (Advance) Linux Administrator (Advance) Mr.Kriangsak Namkot Trainer & Director Jodoi IT&Service Co.,Ltd. jodoi@jodoi.com jodoi1819@hotmail.com http://www.jodoi.com Linux Administrator I Day 1 9.00 10.30 - Samba

More information

Replacing a Windows NT/2000 server using Linux and SAMBA

Replacing a Windows NT/2000 server using Linux and SAMBA LinuxFocus article number 247 http://linuxfocus.org Replacing a Windows NT/2000 server using Linux and SAMBA by Sebastian Sasías About the author: He has used Linux for several

More information

Hyper-V over SMB: Remote File Storage Support in Windows Server 2012 Hyper-V. Jose Barreto Principal Program Manager Microsoft Corporation

Hyper-V over SMB: Remote File Storage Support in Windows Server 2012 Hyper-V. Jose Barreto Principal Program Manager Microsoft Corporation Hyper-V over SMB: Remote Storage Support in Windows Server 2012 Hyper-V Jose Barreto Principal Program Manager Microsoft Corporation Abstract In this session, we cover the Windows Server 2012 Hyper-V support

More information

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com Parallels Cloud Storage White Paper Performance Benchmark Results www.parallels.com Table of Contents Executive Summary... 3 Architecture Overview... 3 Key Features... 4 No Special Hardware Requirements...

More information

LBNC and IBM Corporation 2009. Document: LBNC-Install.doc Date: 06.03.2009 Path: D:\Doc\EPFL\LNBC\LBNC-Install.doc Version: V1.0

LBNC and IBM Corporation 2009. Document: LBNC-Install.doc Date: 06.03.2009 Path: D:\Doc\EPFL\LNBC\LBNC-Install.doc Version: V1.0 LBNC Compute Cluster Installation and Configuration Author: Markus Baertschi Owner: Markus Baertschi Customer: LBNC Subject: LBNC Compute Cluster Installation and Configuration Page 1 of 14 Contents 1.

More information

GPFS Storage Server. Concepts and Setup in Lemanicus BG/Q system" Christian Clémençon (EPFL-DIT)" " 4 April 2013"

GPFS Storage Server. Concepts and Setup in Lemanicus BG/Q system Christian Clémençon (EPFL-DIT)  4 April 2013 GPFS Storage Server Concepts and Setup in Lemanicus BG/Q system" Christian Clémençon (EPFL-DIT)" " Agenda" GPFS Overview" Classical versus GSS I/O Solution" GPFS Storage Server (GSS)" GPFS Native RAID

More information

Linux Development Environment Description Based on VirtualBox Structure

Linux Development Environment Description Based on VirtualBox Structure Linux Development Environment Description Based on VirtualBox Structure V1.0 1 VirtualBox is open source virtual machine software. It mainly has three advantages: (1) Free (2) compact (3) powerful. At

More information

Moving Virtual Storage to the Cloud

Moving Virtual Storage to the Cloud Moving Virtual Storage to the Cloud White Paper Guidelines for Hosters Who Want to Enhance Their Cloud Offerings with Cloud Storage www.parallels.com Table of Contents Overview... 3 Understanding the Storage

More information

Globus and the Centralized Research Data Infrastructure at CU Boulder

Globus and the Centralized Research Data Infrastructure at CU Boulder Globus and the Centralized Research Data Infrastructure at CU Boulder Daniel Milroy, daniel.milroy@colorado.edu Conan Moore, conan.moore@colorado.edu Thomas Hauser, thomas.hauser@colorado.edu Peter Ruprecht,

More information

Investigation of storage options for scientific computing on Grid and Cloud facilities

Investigation of storage options for scientific computing on Grid and Cloud facilities Investigation of storage options for scientific computing on Grid and Cloud facilities Overview Hadoop Test Bed Hadoop Evaluation Standard benchmarks Application-based benchmark Blue Arc Evaluation Standard

More information

IOmark- VDI. HP HP ConvergedSystem 242- HC StoreVirtual Test Report: VDI- HC- 150427- b Test Report Date: 27, April 2015. www.iomark.

IOmark- VDI. HP HP ConvergedSystem 242- HC StoreVirtual Test Report: VDI- HC- 150427- b Test Report Date: 27, April 2015. www.iomark. IOmark- VDI HP HP ConvergedSystem 242- HC StoreVirtual Test Report: VDI- HC- 150427- b Test Copyright 2010-2014 Evaluator Group, Inc. All rights reserved. IOmark- VDI, IOmark- VM, VDI- IOmark, and IOmark

More information

Microsoft SQL Server 2012 on Cisco UCS with iscsi-based Storage Access in VMware ESX Virtualization Environment: Performance Study

Microsoft SQL Server 2012 on Cisco UCS with iscsi-based Storage Access in VMware ESX Virtualization Environment: Performance Study White Paper Microsoft SQL Server 2012 on Cisco UCS with iscsi-based Storage Access in VMware ESX Virtualization Environment: Performance Study 2012 Cisco and/or its affiliates. All rights reserved. This

More information

Springpath Data Platform with Cisco UCS Servers

Springpath Data Platform with Cisco UCS Servers Springpath Data Platform with Cisco UCS Servers Reference Architecture March 2015 SPRINGPATH DATA PLATFORM WITH CISCO UCS SERVERS Reference Architecture 1.0 Introduction to Springpath Data Platform 1 2.0

More information

An Alternative Storage Solution for MapReduce. Eric Lomascolo Director, Solutions Marketing

An Alternative Storage Solution for MapReduce. Eric Lomascolo Director, Solutions Marketing An Alternative Storage Solution for MapReduce Eric Lomascolo Director, Solutions Marketing MapReduce Breaks the Problem Down Data Analysis Distributes processing work (Map) across compute nodes and accumulates

More information

Linux Powered Storage:

Linux Powered Storage: Linux Powered Storage: Building a Storage Server with Linux Architect & Senior Manager rwheeler@redhat.com June 6, 2012 1 Linux Based Systems are Everywhere Used as the base for commercial appliances Enterprise

More information

Scientific Computing Data Management Visions

Scientific Computing Data Management Visions Scientific Computing Data Management Visions ELI-Tango Workshop Szeged, 24-25 February 2015 Péter Szász Group Leader Scientific Computing Group ELI-ALPS Scientific Computing Group Responsibilities Data

More information

SLIDE 1 www.bitmicro.com. Previous Next Exit

SLIDE 1 www.bitmicro.com. Previous Next Exit SLIDE 1 MAXio All Flash Storage Array Popular Applications MAXio N1A6 SLIDE 2 MAXio All Flash Storage Array Use Cases High speed centralized storage for IO intensive applications email, OLTP, databases

More information

Syncplicity On-Premise Storage Connector

Syncplicity On-Premise Storage Connector Syncplicity On-Premise Storage Connector Implementation Guide Abstract This document explains how to install and configure the Syncplicity On-Premise Storage Connector. In addition, it also describes how

More information

Hadoop on the Gordon Data Intensive Cluster

Hadoop on the Gordon Data Intensive Cluster Hadoop on the Gordon Data Intensive Cluster Amit Majumdar, Scientific Computing Applications Mahidhar Tatineni, HPC User Services San Diego Supercomputer Center University of California San Diego Dec 18,

More information

Cloud Optimize Your IT

Cloud Optimize Your IT Cloud Optimize Your IT Windows Server 2012 The information contained in this presentation relates to a pre-release product which may be substantially modified before it is commercially released. This pre-release

More information

Deployment Guide. How to prepare your environment for an OnApp Cloud deployment.

Deployment Guide. How to prepare your environment for an OnApp Cloud deployment. Deployment Guide How to prepare your environment for an OnApp Cloud deployment. Document version 1.07 Document release date 28 th November 2011 document revisions 1 Contents 1. Overview... 3 2. Network

More information

Configuration Maximums

Configuration Maximums Topic Configuration s VMware vsphere 5.1 When you select and configure your virtual and physical equipment, you must stay at or below the maximums supported by vsphere 5.1. The limits presented in the

More information

Netapp @ 10th TF-Storage Meeting

Netapp @ 10th TF-Storage Meeting Netapp @ 10th TF-Storage Meeting Wojciech Janusz, Netapp Poland Bogusz Błaszkiewicz, Netapp Poland Ljubljana, 2012.02.20 Agenda Data Ontap Cluster-Mode pnfs E-Series NetApp Confidential - Internal Use

More information

Moving Virtual Storage to the Cloud. Guidelines for Hosters Who Want to Enhance Their Cloud Offerings with Cloud Storage

Moving Virtual Storage to the Cloud. Guidelines for Hosters Who Want to Enhance Their Cloud Offerings with Cloud Storage Moving Virtual Storage to the Cloud Guidelines for Hosters Who Want to Enhance Their Cloud Offerings with Cloud Storage Table of Contents Overview... 1 Understanding the Storage Problem... 1 What Makes

More information

IOmark- VDI. Nimbus Data Gemini Test Report: VDI- 130906- a Test Report Date: 6, September 2013. www.iomark.org

IOmark- VDI. Nimbus Data Gemini Test Report: VDI- 130906- a Test Report Date: 6, September 2013. www.iomark.org IOmark- VDI Nimbus Data Gemini Test Report: VDI- 130906- a Test Copyright 2010-2013 Evaluator Group, Inc. All rights reserved. IOmark- VDI, IOmark- VDI, VDI- IOmark, and IOmark are trademarks of Evaluator

More information

Data storage considerations for HTS platforms. George Magklaras -- node manager http://www.no.embnet.org http://www.biotek.uio.no admin@embnet.uio.

Data storage considerations for HTS platforms. George Magklaras -- node manager http://www.no.embnet.org http://www.biotek.uio.no admin@embnet.uio. Data storage considerations for HTS platforms George Magklaras -- node manager http://www.no.embnet.org http://www.biotek.uio.no admin@embnet.uio.no Overview: The need for data storage Volume dimensioning

More information

Lustre on Hyperion. Marc Stearman. April, 2009

Lustre on Hyperion. Marc Stearman. April, 2009 Lustre on Hyperion Marc tearman marc@llnl.gov April, 2009 This work performed under the auspices of the U.. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.

More information

Remote PC Guide Series - Volume 1

Remote PC Guide Series - Volume 1 Introduction and Planning for Remote PC Implementation with NETLAB+ Document Version: 2016-02-01 What is a remote PC and how does it work with NETLAB+? This educational guide will introduce the concepts

More information

Handling Multimedia Under Desktop Virtualization for Knowledge Workers

Handling Multimedia Under Desktop Virtualization for Knowledge Workers Handling Multimedia Under Desktop Virtualization for Knowledge Workers Wyse TCX Multimedia capabilities deliver the applications and performance required, for less A white paper by Wyse Technology Inc.

More information

Transforming the UL into a Big Data University. Current status and planned evolutions

Transforming the UL into a Big Data University. Current status and planned evolutions Transforming the UL into a Big Data University Current status and planned evolutions December 6th, 2013 UniGR Workshop - Big Data Sébastien Varrette, PhD Prof. Pascal Bouvry Prof. Volker Müller http://hpc.uni.lu

More information

February, 2015 Bill Loewe

February, 2015 Bill Loewe February, 2015 Bill Loewe Agenda System Metadata, a growing issue Parallel System - Lustre Overview Metadata and Distributed Namespace Test setup and implementation for metadata testing Scaling Metadata

More information

WASHTENAW COUNTY FINANCE DEPARTMENT Purchasing Division

WASHTENAW COUNTY FINANCE DEPARTMENT Purchasing Division WASHTENAW COUNTY FINANCE DEPARTMENT Purchasing Division P. O. Box 8645, 220 N. Main, Ann Arbor, MI 48107-8645 Phone (734) 222-6760, Fax (734) 222-6764 www.purchasing.ewashtenaw.org May 22, 2014 RFP 6783

More information

Deploying Ceph with High Performance Networks, Architectures and benchmarks for Block Storage Solutions

Deploying Ceph with High Performance Networks, Architectures and benchmarks for Block Storage Solutions WHITE PAPER May 2014 Deploying Ceph with High Performance Networks, Architectures and benchmarks for Block Storage Solutions Contents Executive Summary...2 Background...2 Network Configuration...3 Test

More information

Windows Server 2012 R2 Hyper-V: Designing for the Real World

Windows Server 2012 R2 Hyper-V: Designing for the Real World Windows Server 2012 R2 Hyper-V: Designing for the Real World Steve Evans @scevans www.loudsteve.com Nick Hawkins @nhawkins www.nickahawkins.com Is Hyper-V for real? Microsoft Fan Boys Reality VMware Hyper-V

More information

Optimizing Dell PowerEdge Configurations for Hadoop

Optimizing Dell PowerEdge Configurations for Hadoop Optimizing Dell PowerEdge Configurations for Hadoop Understanding how to get the most out of Hadoop running on Dell hardware A Dell technical white paper July 2013 Michael Pittaro Principal Architect,

More information

DELL. Dell Microsoft Windows Server 2008 Hyper-V TM Reference Architecture VIRTUALIZATION SOLUTIONS ENGINEERING

DELL. Dell Microsoft Windows Server 2008 Hyper-V TM Reference Architecture VIRTUALIZATION SOLUTIONS ENGINEERING DELL Dell Microsoft Windows Server 2008 Hyper-V TM Reference Architecture VIRTUALIZATION SOLUTIONS ENGINEERING September 2008 1 THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL

More information

High Availability with DRBD & Heartbeat. Chris Barber http://www.cb1inc.com

High Availability with DRBD & Heartbeat. Chris Barber http://www.cb1inc.com High Availability with DRBD & Heartbeat Chris Barber http://www.cb1inc.com What is DRBD? Distributed Replicated Block Device RAID 1 mirror across network Realtime replications Linux-only kernel module

More information

Deep Dive on SimpliVity s OmniStack A Technical Whitepaper

Deep Dive on SimpliVity s OmniStack A Technical Whitepaper Deep Dive on SimpliVity s OmniStack A Technical Whitepaper By Hans De Leenheer and Stephen Foskett August 2013 1 Introduction This paper is an in-depth look at OmniStack, the technology that powers SimpliVity

More information

Unitrends Virtual Backup Installation Guide Version 8.0

Unitrends Virtual Backup Installation Guide Version 8.0 Unitrends Virtual Backup Installation Guide Version 8.0 Release June 2014 7 Technology Circle, Suite 100 Columbia, SC 29203 Phone: 803.454.0300 Contents Chapter 1 Getting Started... 1 Version 8 Architecture...

More information

Enhancing the Dell iscsi SAN with Dell PowerVault TM Tape Libraries and Chelsio Unified Storage Router iscsi Appliance

Enhancing the Dell iscsi SAN with Dell PowerVault TM Tape Libraries and Chelsio Unified Storage Router iscsi Appliance Enhancing the Dell iscsi SAN with Dell PowerVault TM Tape Libraries and Chelsio Unified Storage Router April 12, 2012 Table of Contents 1.0 Introduction... 4 2.0 Chelsio Unified Storage Router (USR) SAS-to-...

More information

.NET UI Load Balancing & Clustering

.NET UI Load Balancing & Clustering QAD Load Balancing & Clustering Nectarios Daloglou President & Principal Consultant Dalo Consulting Inc. 1 2014 Dalo Consulting Inc. Agenda Introduction Manual Load Balancing DNS Round Robin Other Solutions

More information

Active Fabric Manager (AFM) Plug-in for VMware vcenter Virtual Distributed Switch (VDS) CLI Guide

Active Fabric Manager (AFM) Plug-in for VMware vcenter Virtual Distributed Switch (VDS) CLI Guide Active Fabric Manager (AFM) Plug-in for VMware vcenter Virtual Distributed Switch (VDS) CLI Guide Notes, Cautions, and Warnings NOTE: A NOTE indicates important information that helps you make better use

More information

HITACHI VIRTUAL STORAGE PLATFORM FAMILY MATRIX

HITACHI VIRTUAL STORAGE PLATFORM FAMILY MATRIX HITACHI VIRTUAL STORAGE PLATFORM FAMILY MATRIX 1 G1000 Capacity Specifications Maximum (Max.) Number of Hard Drives, Including Spares 264 SFF 264 LFF 480 SFF 480 LFF 720 SFF 720 LFF 1,440 SFF 1,440 LFF

More information

Managing Live Migrations

Managing Live Migrations Managing Live Migrations from Novell NetWare to SUSE Linux Enterprise Server For many IT departments, avoiding downtime for mission-critical systems during migrations and upgrades is a top priority. The

More information

Samba on HP StorageWorks Enterprise File Services (EFS) Clustered File System Software

Samba on HP StorageWorks Enterprise File Services (EFS) Clustered File System Software Samba on HP StorageWorks Enterprise File Services (EFS) Clustered File System Software Installation and integration guide Abstract... 2 Introduction... 2 Application overview... 2 Application configuration...

More information

VMware for Bosch VMS. en Software Manual

VMware for Bosch VMS. en Software Manual VMware for Bosch VMS en Software Manual VMware for Bosch VMS Table of Contents en 3 Table of contents 1 Introduction 4 1.1 Restrictions 4 2 Overview 5 3 Installing and configuring ESXi server 6 3.1 Installing

More information

Dell Virtual Remote Desktop Reference Architecture. Technical White Paper Version 1.0

Dell Virtual Remote Desktop Reference Architecture. Technical White Paper Version 1.0 Dell Virtual Remote Desktop Reference Architecture Technical White Paper Version 1.0 July 2010 THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES.

More information

NERSC File Systems and How to Use Them

NERSC File Systems and How to Use Them NERSC File Systems and How to Use Them David Turner! NERSC User Services Group! Joint Facilities User Forum on Data- Intensive Computing! June 18, 2014 The compute and storage systems 2014 Hopper: 1.3PF,

More information

Reference Design: Scalable Object Storage with Seagate Kinetic, Supermicro, and SwiftStack

Reference Design: Scalable Object Storage with Seagate Kinetic, Supermicro, and SwiftStack Reference Design: Scalable Object Storage with Seagate Kinetic, Supermicro, and SwiftStack May 2015 Copyright 2015 SwiftStack, Inc. swiftstack.com Page 1 of 19 Table of Contents INTRODUCTION... 3 OpenStack

More information

HPC @ CRIBI. Calcolo Scientifico e Bioinformatica oggi Università di Padova 13 gennaio 2012

HPC @ CRIBI. Calcolo Scientifico e Bioinformatica oggi Università di Padova 13 gennaio 2012 HPC @ CRIBI Calcolo Scientifico e Bioinformatica oggi Università di Padova 13 gennaio 2012 what is exact? experience on advanced computational technologies a company lead by IT experts with a strong background

More information

Four Reasons To Start Working With NFSv4.1 Now

Four Reasons To Start Working With NFSv4.1 Now Four Reasons To Start Working With NFSv4.1 Now PRESENTATION TITLE GOES HERE Presented by: Alex McDonald Hosted by: Gilles Chekroun Ethernet Storage Forum Members The SNIA Ethernet Storage Forum (ESF) focuses

More information

Configuration Maximums

Configuration Maximums Configuration s vsphere 6.0 This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new edition. To check for more recent editions

More information

IRODS use case : Ciment, the Univ. Grenoble-Alpes HPC center. B.Bzeznik / X.Briand Irods users group meeting 11/06/2015

IRODS use case : Ciment, the Univ. Grenoble-Alpes HPC center. B.Bzeznik / X.Briand Irods users group meeting 11/06/2015 IRODS use case : Ciment, the Univ. Grenoble-Alpes HPC center B.Bzeznik / X.Briand Irods users group meeting 11/06/2015 IRODS rocks! We like rocks here... Irods is used (famous) in the French Alps since

More information

VMware Virtual SAN Hardware Guidance. TECHNICAL MARKETING DOCUMENTATION v 1.0

VMware Virtual SAN Hardware Guidance. TECHNICAL MARKETING DOCUMENTATION v 1.0 VMware Virtual SAN Hardware Guidance TECHNICAL MARKETING DOCUMENTATION v 1.0 Table of Contents Introduction.... 3 Server Form Factors... 3 Rackmount.... 3 Blade.........................................................................3

More information

Driving IBM BigInsights Performance Over GPFS Using InfiniBand+RDMA

Driving IBM BigInsights Performance Over GPFS Using InfiniBand+RDMA WHITE PAPER April 2014 Driving IBM BigInsights Performance Over GPFS Using InfiniBand+RDMA Executive Summary...1 Background...2 File Systems Architecture...2 Network Architecture...3 IBM BigInsights...5

More information

DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION

DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION A DIABLO WHITE PAPER AUGUST 2014 Ricky Trigalo Director of Business Development Virtualization, Diablo Technologies

More information

Object storage in Cloud Computing and Embedded Processing

Object storage in Cloud Computing and Embedded Processing Object storage in Cloud Computing and Embedded Processing Jan Jitze Krol Systems Engineer DDN We Accelerate Information Insight DDN is a Leader in Massively Scalable Platforms and Solutions for Big Data

More information

Clustered NAS meets GPFS. Andrew Tridgell LTC ALRT Team

Clustered NAS meets GPFS. Andrew Tridgell LTC ALRT Team Clustered NAS meets GPFS Andrew Tridgell LTC ALRT Team Scaling NAS What if? you have 30,000 NAS users you have 100 NAS servers every day you run out of space on one of them What can you do? Get a really

More information

File server infrastructure @NIKHEF

File server infrastructure @NIKHEF File server infrastructure @NIKHEF CT system support June 2003 1 CT NIKHEF Outline Protocols Naming scheme (Unix, Windows) Backup and archiving Server systems Disk quota policy AFS June 2003 2 CT NIKHEF

More information

Fujitsu PRIMEFLEX reference architectures

Fujitsu PRIMEFLEX reference architectures SharePoint, Exchange, Lync, OfficeMaster Gate Fujitsu PRIMEFLEX reference architectures 0 Fujitsu PRIMEFLEX for SharePoint based on Microsoft SharePoint 2013 SP1 on-premise solution designed for 250 SharePoint

More information

NetApp High-Performance Computing Solution for Lustre: Solution Guide

NetApp High-Performance Computing Solution for Lustre: Solution Guide Technical Report NetApp High-Performance Computing Solution for Lustre: Solution Guide Robert Lai, NetApp August 2012 TR-3997 TABLE OF CONTENTS 1 Introduction... 5 1.1 NetApp HPC Solution for Lustre Introduction...5

More information

Virtual Desktop Infrastructure (VDI) made Easy

Virtual Desktop Infrastructure (VDI) made Easy Virtual Desktop Infrastructure (VDI) made Easy HOW-TO Preface: Desktop virtualization can increase the effectiveness of information technology (IT) teams by simplifying how they configure and deploy endpoint

More information

QuickBooks Enterprise Solutions. Linux Database Server Manager Installation and Configuration Guide

QuickBooks Enterprise Solutions. Linux Database Server Manager Installation and Configuration Guide QuickBooks Enterprise Solutions Linux Database Server Manager Installation and Configuration Guide Copyright Copyright 2007 Intuit Inc. All rights reserved. STATEMENTS IN THIS DOCUMENT REGARDING THIRD-PARTY

More information

Current Status of FEFS for the K computer

Current Status of FEFS for the K computer Current Status of FEFS for the K computer Shinji Sumimoto Fujitsu Limited Apr.24 2012 LUG2012@Austin Outline RIKEN and Fujitsu are jointly developing the K computer * Development continues with system

More information

Cluster Implementation and Management; Scheduling

Cluster Implementation and Management; Scheduling Cluster Implementation and Management; Scheduling CPS343 Parallel and High Performance Computing Spring 2013 CPS343 (Parallel and HPC) Cluster Implementation and Management; Scheduling Spring 2013 1 /

More information