CRIBI. Calcolo Scientifico e Bioinformatica oggi Università di Padova 13 gennaio 2012

Size: px
Start display at page:

Download "HPC @ CRIBI. Calcolo Scientifico e Bioinformatica oggi Università di Padova 13 gennaio 2012"

Transcription

1 CRIBI Calcolo Scientifico e Bioinformatica oggi Università di Padova 13 gennaio 2012

2 what is exact? experience on advanced computational technologies a company lead by IT experts with a strong background in physics and computer science, who have experience in everyday use, maintenance and configuration of high performance computing systems for the scientific community (CNR-IOM DEMOCRITOS, SISSA) international collaborations for training activities and HPC/GRID infrastructure deployments (ICTP) Calcolo Scientifico e Bioinformatica oggi 13 gennaio

3 services for the HPC cluster deployment installation and configuration of HPC and GRID infrastructures high performance storage solution lustre expertise training sys admin and user oriented training programs application tuning optimization, parallelization, porting, development on-demand HPC from raw ssh to highly customized web portals Calcolo Scientifico e Bioinformatica oggi 13 gennaio

4 fill the gap collaboration with Aldebra/HP scientific SW mgmt SW HW END USER optimization / parallelization compilers / scientific libraries deployment / administration / management hardware tuning Aldebra / HP exact Calcolo Scientifico e Bioinformatica oggi 13 gennaio

5 HPC CRIBI masternode 32 blades (2x6 cores, 24,48,96GB RAM) GPU node GPU node STORAGE 12x600GB 36x2TB STORAGE 12x600GB 36x2TB FAT node (2TB RAM) 1 GB Ethernet (SP/iLO/mgmt) 1 GB Ethernet (NFS) 40 GB Infiniband (LUSTRE/MPI) 10 GB Ethernet (iscsi) 1 GB (LAN) Calcolo Scientifico e Bioinformatica oggi 13 gennaio

6 HPC CRIBI masternode 32 blades (2x6 cores, 24,48,96GB RAM) GPU node GPU node STORAGE 12x600GB 36x2TB STORAGE 12x600GB 36x2TB FAT node (2TB RAM) 1 GB Ethernet (SP/iLO/mgmt) 1 GB Ethernet (NFS) 40 GB Infiniband (LUSTRE/MPI) 10 GB Ethernet (iscsi) 1 GB (LAN) Calcolo Scientifico e Bioinformatica oggi 13 gennaio

7 HPC CRIBI masternode 32 blades (2x6 cores, 24,48,96GB RAM) GPU node GPU node STORAGE 12x600GB 36x2TB STORAGE 12x600GB 36x2TB FAT node (2TB RAM) 1 GB Ethernet (SP/iLO/mgmt) 1 GB Ethernet (NFS) 40 GB Infiniband (LUSTRE/MPI) 10 GB Ethernet (iscsi) 1 GB (LAN) Calcolo Scientifico e Bioinformatica oggi 13 gennaio

8 HPC CRIBI masternode 32 blades (2x6 cores, 24,48,96GB RAM) GPU node GPU node STORAGE 12x600GB 36x2TB STORAGE 12x600GB 36x2TB FAT node (2TB RAM) 1 GB Ethernet (SP/iLO/mgmt) 1 GB Ethernet (NFS) 40 GB Infiniband (LUSTRE/MPI) 10 GB Ethernet (iscsi) 1 GB (LAN) Calcolo Scientifico e Bioinformatica oggi 13 gennaio

9 HPC CRIBI masternode 32 blades (2x6 cores, 24,48,96GB RAM) GPU node GPU node STORAGE 12x600GB 36x2TB STORAGE 12x600GB 36x2TB FAT node (2TB RAM) 1 GB Ethernet (SP/iLO/mgmt) 1 GB Ethernet (NFS) 40 GB Infiniband (LUSTRE/MPI) 10 GB Ethernet (iscsi) 1 GB (LAN) Calcolo Scientifico e Bioinformatica oggi 13 gennaio

10 HPC CRIBI masternode 32 blades (2x6 cores, 24,48,96GB RAM) GPU node GPU node STORAGE 12x600GB 36x2TB STORAGE 12x600GB 36x2TB FAT node (2TB RAM) 1 GB Ethernet (SP/iLO/mgmt) 1 GB Ethernet (NFS) 40 GB Infiniband (LUSTRE/MPI) 10 GB Ethernet (iscsi) 1 GB (LAN) Calcolo Scientifico e Bioinformatica oggi 13 gennaio

11 challenges heterogeneous environment (GPU, HIMEM/FAT, BLADES w/ or w/o HT, BLADES w/ HIMEM, storage nodes) access policies and queues complex network configuration remote management parallel filesystem Calcolo Scientifico e Bioinformatica oggi 13 gennaio

12 software solution: tools EPICO - exact Procedure for Installation and COnfiguration - LINUX based distributed and unattended installation framework - fully integrated with standard tools and services for HPC torquemaui ganglia and various monitoring tools / utilities standard compilers and scientific libraries LUSTRE parallel filesystem hardware and software tuning needed to increase I/O performance Calcolo Scientifico e Bioinformatica oggi 13 gennaio

13 LUSTRE and storage solution Calcolo Scientifico e Bioinformatica oggi 13 gennaio

14 accessing LUSTRE filesystem Calcolo Scientifico e Bioinformatica oggi 13 gennaio

15 accessing LUSTRE filesystem Calcolo Scientifico e Bioinformatica oggi 13 gennaio

16 accessing LUSTRE filesystem Calcolo Scientifico e Bioinformatica oggi 13 gennaio

17 accessing LUSTRE filesystem Calcolo Scientifico e Bioinformatica oggi 13 gennaio

18 accessing LUSTRE filesystem Calcolo Scientifico e Bioinformatica oggi 13 gennaio

19 accessing LUSTRE filesystem RAID Calcolo Scientifico e Bioinformatica oggi 13 gennaio

20 accessing LUSTRE filesystem RAID Calcolo Scientifico e Bioinformatica oggi 13 gennaio

21 accessing LUSTRE filesystem RAID Calcolo Scientifico e Bioinformatica oggi 13 gennaio

22 accessing LUSTRE filesystem RAID Calcolo Scientifico e Bioinformatica oggi 13 gennaio

23 why parallel filesystem? Calcolo Scientifico e Bioinformatica oggi 13 gennaio

24 why parallel filesystem? Calcolo Scientifico e Bioinformatica oggi 13 gennaio

25 why parallel filesystem? Calcolo Scientifico e Bioinformatica oggi 13 gennaio

26 why parallel filesystem? Calcolo Scientifico e Bioinformatica oggi 13 gennaio

27 why parallel filesystem? Calcolo Scientifico e Bioinformatica oggi 13 gennaio

28 why parallel filesystem? RAID Calcolo Scientifico e Bioinformatica oggi 13 gennaio

29 why parallel filesystem? RAID RAID Calcolo Scientifico e Bioinformatica oggi 13 gennaio

30 why parallel filesystem? RAID RAID Calcolo Scientifico e Bioinformatica oggi 13 gennaio

31 why parallel filesystem? RAID RAID Calcolo Scientifico e Bioinformatica oggi 13 gennaio

32 why parallel filesystem? RAID RAID Calcolo Scientifico e Bioinformatica oggi 13 gennaio

33 overall LUSTRE performance Client #1 Client #5 Client #2 Client #6 Client #3 Client #7 Client #4 Client #8 sequential write/read by iozone 1 ~ 8, 1 ~ 4 proc/client 32 GB files writing 64 GB files reading IB QDR Network #1 #2 #3 #4 ~ 1.7 GB/sec writing 32, 32 GB files ~ 1.2 GB/sec reading 32, 64 GB files Calcolo Scientifico e Bioinformatica oggi 13 gennaio

34 LUSTRE can be disappointing too... writing 1 file with variable block size MB/s kb 2 kb 4 kb 8 kb 64 kb 512 kb 1 Mb 2 Mb 4 Mb # Gbytes Calcolo Scientifico e Bioinformatica oggi 13 gennaio

35 tips to use lustre at best identify bottlenecks s, RAID,, volumes and virtual disks, 10GE, iscsi, LUSTRE fs, local caching/buffers, InfiniBand or Gigabit network, LUSTRE client, access patterns when accessing files: read/write in large chunks, not line by line [e.g. mmap instead of readline] when running multiple I/O operations: optimize data distribution and disk access tune cache/buffer size, sync, and priorities Calcolo Scientifico e Bioinformatica oggi 13 gennaio

36 Thanks Calcolo Scientifico e Bioinformatica oggi 13 gennaio

Agenda. HPC Software Stack. HPC Post-Processing Visualization. Case Study National Scientific Center. European HPC Benchmark Center Montpellier PSSC

Agenda. HPC Software Stack. HPC Post-Processing Visualization. Case Study National Scientific Center. European HPC Benchmark Center Montpellier PSSC HPC Architecture End to End Alexandre Chauvin Agenda HPC Software Stack Visualization National Scientific Center 2 Agenda HPC Software Stack Alexandre Chauvin Typical HPC Software Stack Externes LAN Typical

More information

Lustre failover experience

Lustre failover experience Lustre failover experience Lustre Administrators and Developers Workshop Paris 1 September 25, 2012 TOC Who we are Our Lustre experience: the environment Deployment Benchmarks What's next 2 Who we are

More information

Appro Supercomputer Solutions Best Practices Appro 2012 Deployment Successes. Anthony Kenisky, VP of North America Sales

Appro Supercomputer Solutions Best Practices Appro 2012 Deployment Successes. Anthony Kenisky, VP of North America Sales Appro Supercomputer Solutions Best Practices Appro 2012 Deployment Successes Anthony Kenisky, VP of North America Sales About Appro Over 20 Years of Experience 1991 2000 OEM Server Manufacturer 2001-2007

More information

Hadoop on the Gordon Data Intensive Cluster

Hadoop on the Gordon Data Intensive Cluster Hadoop on the Gordon Data Intensive Cluster Amit Majumdar, Scientific Computing Applications Mahidhar Tatineni, HPC User Services San Diego Supercomputer Center University of California San Diego Dec 18,

More information

HPC Cluster Decisions and ANSYS Configuration Best Practices. Diana Collier Lead Systems Support Specialist Houston UGM May 2014

HPC Cluster Decisions and ANSYS Configuration Best Practices. Diana Collier Lead Systems Support Specialist Houston UGM May 2014 HPC Cluster Decisions and ANSYS Configuration Best Practices Diana Collier Lead Systems Support Specialist Houston UGM May 2014 1 Agenda Introduction Lead Systems Support Specialist Cluster Decisions Job

More information

Exadata HW Overview. Marek Mintal

Exadata HW Overview. Marek Mintal Exadata HW Overview Marek Mintal marek.mintal@phaetech.com Oracle Day 2011 20.10.2011 Exadata Hardware Architecture Scalable Grid of industry standard servers for Compute and Storage Eliminates long-standing

More information

U-LITE: a proposal for scientific computing at LNGS. S. Parlati, P. Spinnato, S. Stalio LNGS 13 Sep. 2011

U-LITE: a proposal for scientific computing at LNGS. S. Parlati, P. Spinnato, S. Stalio LNGS 13 Sep. 2011 U-LITE: a proposal for scientific computing at LNGS S. Parlati, P. Spinnato, S. Stalio LNGS 13 Sep. 2011 20 years of Scientific Computing at LNGS Early 90s: highly centralized structure based on VMS cluster

More information

CMS Tier-3 cluster at NISER. Dr. Tania Moulik

CMS Tier-3 cluster at NISER. Dr. Tania Moulik CMS Tier-3 cluster at NISER Dr. Tania Moulik What and why? Grid computing is a term referring to the combination of computer resources from multiple administrative domains to reach common goal. Grids tend

More information

Cluster Implementation and Management; Scheduling

Cluster Implementation and Management; Scheduling Cluster Implementation and Management; Scheduling CPS343 Parallel and High Performance Computing Spring 2013 CPS343 (Parallel and HPC) Cluster Implementation and Management; Scheduling Spring 2013 1 /

More information

Sun Constellation System: The Open Petascale Computing Architecture

Sun Constellation System: The Open Petascale Computing Architecture CAS2K7 13 September, 2007 Sun Constellation System: The Open Petascale Computing Architecture John Fragalla Senior HPC Technical Specialist Global Systems Practice Sun Microsystems, Inc. 25 Years of Technical

More information

Lustre & Cluster. - monitoring the whole thing Erich Focht

Lustre & Cluster. - monitoring the whole thing Erich Focht Lustre & Cluster - monitoring the whole thing Erich Focht NEC HPC Europe LAD 2014, Reims, September 22-23, 2014 1 Overview Introduction LXFS Lustre in a Data Center IBviz: Infiniband Fabric visualization

More information

Demonstrating the Improvement in the Performance of a Single Lustre * Client from Version 1.8 to Version 2.6

Demonstrating the Improvement in the Performance of a Single Lustre * Client from Version 1.8 to Version 2.6 Demonstrating the Improvement in the Performance of a Single Lustre * Client from Version 1.8 to Version 2.6 With a Stop Along the Way at Versions 2.1 and 2.4 Andrew Uselton, Gabriele Paciucci, Jinshan

More information

Investigation of storage options for scientific computing on Grid and Cloud facilities

Investigation of storage options for scientific computing on Grid and Cloud facilities Investigation of storage options for scientific computing on Grid and Cloud facilities Overview Hadoop Test Bed Hadoop Evaluation Standard benchmarks Application-based benchmark Blue Arc Evaluation Standard

More information

Online Remote Data Backup for iscsi-based Storage Systems

Online Remote Data Backup for iscsi-based Storage Systems Online Remote Data Backup for iscsi-based Storage Systems Dan Zhou, Li Ou, Xubin (Ben) He Department of Electrical and Computer Engineering Tennessee Technological University Cookeville, TN 38505, USA

More information

Technical Bulletin: High Performance Lustre Filesystems using Dell PowerVault MD Storage. Dell/Cambridge HPC Solution Centre University of Cambridge

Technical Bulletin: High Performance Lustre Filesystems using Dell PowerVault MD Storage. Dell/Cambridge HPC Solution Centre University of Cambridge Technical Bulletin: High Performance Lustre Filesystems using Dell PowerVault MD Storage Dell/Cambridge HPC Solution Centre University of Cambridge Wojciech Turek, Dr Paul Calleja Dell Introduction This

More information

for my computation? Stefano Cozzini Which infrastructure Which infrastructure Democrito and SISSA/eLAB - Trieste

for my computation? Stefano Cozzini Which infrastructure Which infrastructure Democrito and SISSA/eLAB - Trieste Which infrastructure Which infrastructure for my computation? Stefano Cozzini Democrito and SISSA/eLAB - Trieste Agenda Introduction:! E-infrastructure and computing infrastructures! What is available

More information

Lustre SMB Gateway. Integrating Lustre with Windows

Lustre SMB Gateway. Integrating Lustre with Windows Lustre SMB Gateway Integrating Lustre with Windows Hardware: Old vs New Compute 60 x Dell PowerEdge 1950-8 x 2.6Ghz cores, 16GB, 500GB Sata, 1GBe - Win7 x64 Storage 1 x Dell R510-12 x 2TB Sata, RAID5,

More information

Computational infrastructure for NGS data analysis. José Carbonell Caballero Pablo Escobar

Computational infrastructure for NGS data analysis. José Carbonell Caballero Pablo Escobar Computational infrastructure for NGS data analysis José Carbonell Caballero Pablo Escobar Computational infrastructure for NGS Cluster definition: A computer cluster is a group of linked computers, working

More information

IRODS use case : Ciment, the Univ. Grenoble-Alpes HPC center. B.Bzeznik / X.Briand Irods users group meeting 11/06/2015

IRODS use case : Ciment, the Univ. Grenoble-Alpes HPC center. B.Bzeznik / X.Briand Irods users group meeting 11/06/2015 IRODS use case : Ciment, the Univ. Grenoble-Alpes HPC center B.Bzeznik / X.Briand Irods users group meeting 11/06/2015 IRODS rocks! We like rocks here... Irods is used (famous) in the French Alps since

More information

Estonian Scientific Computing Infrastructure (ETAIS)

Estonian Scientific Computing Infrastructure (ETAIS) Estonian Scientific Computing Infrastructure (ETAIS) Week #7 Hardi Teder hardi@eenet.ee University of Tartu March 27th 2013 Overview Estonian Scientific Computing Infrastructure Estonian Research infrastructures

More information

Supercomputer Upgrade (HPC Phase 3) Friday 15 th March 2013

Supercomputer Upgrade (HPC Phase 3) Friday 15 th March 2013 Supercomputer Upgrade (HPC Phase 3) Friday 15 th March 2013 Agenda Item Presented By Time Agenda & Team Introduction Peter Fattoush 5mins Overview Neville Smith 5mins SDC Update Barry Nugent 5mins Oracle

More information

Scientific Computing Data Management Visions

Scientific Computing Data Management Visions Scientific Computing Data Management Visions ELI-Tango Workshop Szeged, 24-25 February 2015 Péter Szász Group Leader Scientific Computing Group ELI-ALPS Scientific Computing Group Responsibilities Data

More information

New Storage System Solutions

New Storage System Solutions New Storage System Solutions Craig Prescott Research Computing May 2, 2013 Outline } Existing storage systems } Requirements and Solutions } Lustre } /scratch/lfs } Questions? Existing Storage Systems

More information

IOmark- VDI. HP HP ConvergedSystem 242- HC StoreVirtual Test Report: VDI- HC- 150427- b Test Report Date: 27, April 2015. www.iomark.

IOmark- VDI. HP HP ConvergedSystem 242- HC StoreVirtual Test Report: VDI- HC- 150427- b Test Report Date: 27, April 2015. www.iomark. IOmark- VDI HP HP ConvergedSystem 242- HC StoreVirtual Test Report: VDI- HC- 150427- b Test Copyright 2010-2014 Evaluator Group, Inc. All rights reserved. IOmark- VDI, IOmark- VM, VDI- IOmark, and IOmark

More information

Advanced Techniques with Newton. Gerald Ragghianti Advanced Newton workshop Sept. 22, 2011

Advanced Techniques with Newton. Gerald Ragghianti Advanced Newton workshop Sept. 22, 2011 Advanced Techniques with Newton Gerald Ragghianti Advanced Newton workshop Sept. 22, 2011 Workshop Goals Gain independence Executing your work Finding Information Fixing Problems Optimizing Effectiveness

More information

IT of SPIM Data Storage and Compression. EMBO Course - August 27th! Jeff Oegema, Peter Steinbach, Oscar Gonzalez

IT of SPIM Data Storage and Compression. EMBO Course - August 27th! Jeff Oegema, Peter Steinbach, Oscar Gonzalez IT of SPIM Data Storage and Compression EMBO Course - August 27th Jeff Oegema, Peter Steinbach, Oscar Gonzalez 1 Talk Outline Introduction and the IT Team SPIM Data Flow Capture, Compression, and the Data

More information

An introduction to Fyrkat

An introduction to Fyrkat Cluster Computing May 25, 2011 How to get an account https://fyrkat.grid.aau.dk/useraccount How to get help https://fyrkat.grid.aau.dk/wiki What is a Cluster Anyway It is NOT something that does any of

More information

Performance, Reliability, and Operational Issues for High Performance NAS Storage on Cray Platforms. Cray User Group Meeting June 2007

Performance, Reliability, and Operational Issues for High Performance NAS Storage on Cray Platforms. Cray User Group Meeting June 2007 Performance, Reliability, and Operational Issues for High Performance NAS Storage on Cray Platforms Cray User Group Meeting June 2007 Cray s Storage Strategy Background Broad range of HPC requirements

More information

Building a Top500-class Supercomputing Cluster at LNS-BUAP

Building a Top500-class Supercomputing Cluster at LNS-BUAP Building a Top500-class Supercomputing Cluster at LNS-BUAP Dr. José Luis Ricardo Chávez Dr. Humberto Salazar Ibargüen Dr. Enrique Varela Carlos Laboratorio Nacional de Supercómputo Benemérita Universidad

More information

HA Certification Document Armari BrontaStor 822R 07/03/2013. Open-E High Availability Certification report for Armari BrontaStor 822R

HA Certification Document Armari BrontaStor 822R 07/03/2013. Open-E High Availability Certification report for Armari BrontaStor 822R Open-E High Availability Certification report for Armari BrontaStor 822R 1 Executive summary After successfully passing all the required tests, the Armari BrontaStor 822R is now officially declared as

More information

Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre

Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre University of Cambridge, UIS, HPC Service Authors: Wojciech Turek, Paul Calleja, John Taylor

More information

Best Practices for IBM Spectrum Scale: Announcing a new partnership

Best Practices for IBM Spectrum Scale: Announcing a new partnership Best Practices for IBM : Announcing a new partnership Bernard Shen, VP and Chief Engineer Re-Store, Infrastructure Engineering Tonya Witherspoon, Executive Director Ennovar, Institute of Emerging Technologies

More information

CNR-INFM DEMOCRITOS and SISSA elab Trieste

CNR-INFM DEMOCRITOS and SISSA elab Trieste elab and the FVG grid Stefano Cozzini CNR-INFM DEMOCRITOS and SISSA elab Trieste Agenda/Aims Present elab ant its computational infrastructure GRID-FVG structure basic requirements technical choices open

More information

Transforming the UL into a Big Data University. Current status and planned evolutions

Transforming the UL into a Big Data University. Current status and planned evolutions Transforming the UL into a Big Data University Current status and planned evolutions December 6th, 2013 UniGR Workshop - Big Data Sébastien Varrette, PhD Prof. Pascal Bouvry Prof. Volker Müller http://hpc.uni.lu

More information

Analyzing Large-scale Network Boot Workload for Windows Qiang Wang Microsoft Corporation

Analyzing Large-scale Network Boot Workload for Windows Qiang Wang Microsoft Corporation Analyzing Large-scale Network Boot Workload for Windows Qiang Wang Microsoft Corporation Agenda Network Boot Basics Windows iscsi Diskless Boot Challenges of Large-scale Network Boot Workload Profiling

More information

(Scale Out NAS System)

(Scale Out NAS System) For Unlimited Capacity & Performance Clustered NAS System (Scale Out NAS System) Copyright 2010 by Netclips, Ltd. All rights reserved -0- 1 2 3 4 5 NAS Storage Trend Scale-Out NAS Solution Scaleway Advantages

More information

Clusters: Mainstream Technology for CAE

Clusters: Mainstream Technology for CAE Clusters: Mainstream Technology for CAE Alanna Dwyer HPC Division, HP Linux and Clusters Sparked a Revolution in High Performance Computing! Supercomputing performance now affordable and accessible Linux

More information

EMC IsIlon scale-out storage for sas grid CoMputIng a shared file system solution for sas grid E M C P E r s P E C t i v E

EMC IsIlon scale-out storage for sas grid CoMputIng a shared file system solution for sas grid E M C P E r s P E C t i v E EMC Isilon Scale-out Storage for SAS Grid Computing A Shared File System Solution for SAS Grid EMC Perspective EMC Isilon Scale-out NAS storage platform delivers scale out performance for your SAS Grid

More information

ZABBIX FOR HPC MONITORING AND SUPPORT

ZABBIX FOR HPC MONITORING AND SUPPORT ZABBIX FOR HPC MONITORING AND SUPPORT Mikhail Serkov Delivery Manager/HPC Engineer 2016 AGENDA #1 High Performance Computing what is it about #2 Overview of the customer infrastructure and software stack

More information

Panasas at the RCF. Fall 2005 Robert Petkus RHIC/USATLAS Computing Facility Brookhaven National Laboratory. Robert Petkus Panasas at the RCF

Panasas at the RCF. Fall 2005 Robert Petkus RHIC/USATLAS Computing Facility Brookhaven National Laboratory. Robert Petkus Panasas at the RCF Panasas at the RCF HEPiX at SLAC Fall 2005 Robert Petkus RHIC/USATLAS Computing Facility Brookhaven National Laboratory Centralized File Service Single, facility-wide namespace for files. Uniform, facility-wide

More information

Best Practice Guide Anselm

Best Practice Guide Anselm Bull Extreme Computing at IT4Innovations / VSB Roman Sliva, IT4Innovations / VSB - Technical University of Ostrava Filip Stanek, IT4Innovations / VSB - Technical University of Ostrava May 2013 1 Table

More information

IMPLEMENTING GREEN IT

IMPLEMENTING GREEN IT Saint Petersburg State University of Information Technologies, Mechanics and Optics Department of Telecommunication Systems IMPLEMENTING GREEN IT APPROACH FOR TRANSFERRING BIG DATA OVER PARALLEL DATA LINK

More information

High Performance Computing Specialists. ZFS Storage as a Solution for Big Data and Flexibility

High Performance Computing Specialists. ZFS Storage as a Solution for Big Data and Flexibility High Performance Computing Specialists ZFS Storage as a Solution for Big Data and Flexibility Introducing VA Technologies UK Based System Integrator Specialising in High Performance ZFS Storage Partner

More information

Introduction. Need for ever-increasing storage scalability. Arista and Panasas provide a unique Cloud Storage solution

Introduction. Need for ever-increasing storage scalability. Arista and Panasas provide a unique Cloud Storage solution Arista 10 Gigabit Ethernet Switch Lab-Tested with Panasas ActiveStor Parallel Storage System Delivers Best Results for High-Performance and Low Latency for Scale-Out Cloud Storage Applications Introduction

More information

NetApp High-Performance Computing Solution for Lustre: Solution Guide

NetApp High-Performance Computing Solution for Lustre: Solution Guide Technical Report NetApp High-Performance Computing Solution for Lustre: Solution Guide Robert Lai, NetApp August 2012 TR-3997 TABLE OF CONTENTS 1 Introduction... 5 1.1 NetApp HPC Solution for Lustre Introduction...5

More information

HPC performance applications on Virtual Clusters

HPC performance applications on Virtual Clusters Panagiotis Kritikakos EPCC, School of Physics & Astronomy, University of Edinburgh, Scotland - UK pkritika@epcc.ed.ac.uk 4 th IC-SCCE, Athens 7 th July 2010 This work investigates the performance of (Java)

More information

M-grid: Linux in the First Production Grid Environment in Finland

M-grid: Linux in the First Production Grid Environment in Finland M-grid: Linux in the First Production Grid Environment in Finland Arto Teräs Linux & Open Source training Hotel Kämp, Helsinki, November 1, 2005 (English version of the slides presented

More information

Scalability of ANSYS Applications on Multi-core and Floating point Accelerator Processor Systems from Hewlett- Packard

Scalability of ANSYS Applications on Multi-core and Floating point Accelerator Processor Systems from Hewlett- Packard Scalability of ANSYS Applications on Multi-core and Floating point Accelerator Processor Systems from Hewlett- Packard Don Mize Technical Consultant donald.mize@hp.com Forewords With this presentation

More information

HPC and Big Data. EPCC The University of Edinburgh. Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk

HPC and Big Data. EPCC The University of Edinburgh. Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk HPC and Big Data EPCC The University of Edinburgh Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk EPCC Facilities Technology Transfer European Projects HPC Research Visitor Programmes Training

More information

The Panasas Parallel Storage Cluster. Acknowledgement: Some of the material presented is under copyright by Panasas Inc.

The Panasas Parallel Storage Cluster. Acknowledgement: Some of the material presented is under copyright by Panasas Inc. The Panasas Parallel Storage Cluster What Is It? What Is The Panasas ActiveScale Storage Cluster A complete hardware and software storage solution Implements An Asynchronous, Parallel, Object-based, POSIX

More information

Configuration Maximums VMware Infrastructure 3

Configuration Maximums VMware Infrastructure 3 Technical Note Configuration s VMware Infrastructure 3 When you are selecting and configuring your virtual and physical equipment, you must stay at or below the maximums supported by VMware Infrastructure

More information

CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT

CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT SS Data & Storage CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT HEPiX Fall 2012 Workshop October 15-19, 2012 Institute of High Energy Physics, Beijing, China SS Outline

More information

Flexible Scalable Hardware independent. Solutions for Long Term Archiving

Flexible Scalable Hardware independent. Solutions for Long Term Archiving Flexible Scalable Hardware independent Solutions for Long Term Archiving More than 20 years of experience in archival storage 2 OA HPA 2010 1992 2000 2004 2007 Mainframe Tape Libraries Open System Tape

More information

HPC Update: Engagement Model

HPC Update: Engagement Model HPC Update: Engagement Model MIKE VILDIBILL Director, Strategic Engagements Sun Microsystems mikev@sun.com Our Strategy Building a Comprehensive HPC Portfolio that Delivers Differentiated Customer Value

More information

A Holistic Model of the Energy-Efficiency of Hypervisors

A Holistic Model of the Energy-Efficiency of Hypervisors A Holistic Model of the -Efficiency of Hypervisors in an HPC Environment Mateusz Guzek,Sebastien Varrette, Valentin Plugaru, Johnatan E. Pecero and Pascal Bouvry SnT & CSC, University of Luxembourg, Luxembourg

More information

Building a Linux Cluster

Building a Linux Cluster Building a Linux Cluster CUG Conference May 21-25, 2001 by Cary Whitney Clwhitney@lbl.gov Outline What is PDSF and a little about its history. Growth problems and solutions. Storage Network Hardware Administration

More information

Hadoop: Embracing future hardware

Hadoop: Embracing future hardware Hadoop: Embracing future hardware Suresh Srinivas @suresh_m_s Page 1 About Me Architect & Founder at Hortonworks Long time Apache Hadoop committer and PMC member Designed and developed many key Hadoop

More information

INDIAN INSTITUTE OF TECHNOLOGY KANPUR Department of Mechanical Engineering

INDIAN INSTITUTE OF TECHNOLOGY KANPUR Department of Mechanical Engineering INDIAN INSTITUTE OF TECHNOLOGY KANPUR Department of Mechanical Engineering Enquiry No: Enq/IITK/ME/JB/02 Enquiry Date: 14/12/15 Last Date of Submission: 21/12/15 Formal quotations are invited for HPC cluster.

More information

Quantum StorNext. Product Brief: Distributed LAN Client

Quantum StorNext. Product Brief: Distributed LAN Client Quantum StorNext Product Brief: Distributed LAN Client NOTICE This product brief may contain proprietary information protected by copyright. Information in this product brief is subject to change without

More information

Overview of HPC systems and software available within

Overview of HPC systems and software available within Overview of HPC systems and software available within Overview Available HPC Systems Ba Cy-Tera Available Visualization Facilities Software Environments HPC System at Bibliotheca Alexandrina SUN cluster

More information

Sussex University Lustre Benchmarking Report

Sussex University Lustre Benchmarking Report Sussex University Lustre Benchmarking Report Alces Order number: DELL04 Table of Contents Sussex University...1 Lustre Benchmarking Report...1 IOZONE benchmark test configuration and method...2 IOZONE

More information

Lustre on Hyperion. Marc Stearman. April, 2009

Lustre on Hyperion. Marc Stearman. April, 2009 Lustre on Hyperion Marc tearman marc@llnl.gov April, 2009 This work performed under the auspices of the U.. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.

More information

Configuration Maximums VMware Infrastructure 3: Update 2 and later for ESX Server 3.5, ESX Server 3i version 3.5, VirtualCenter 2.

Configuration Maximums VMware Infrastructure 3: Update 2 and later for ESX Server 3.5, ESX Server 3i version 3.5, VirtualCenter 2. Topic Configuration s VMware Infrastructure 3: Update 2 and later for ESX Server 3.5, ESX Server 3i version 3.5, VirtualCenter 2.5 When you are selecting and configuring your virtual and physical equipment,

More information

Cloud Computing through Virtualization and HPC technologies

Cloud Computing through Virtualization and HPC technologies Cloud Computing through Virtualization and HPC technologies William Lu, Ph.D. 1 Agenda Cloud Computing & HPC A Case of HPC Implementation Application Performance in VM Summary 2 Cloud Computing & HPC HPC

More information

NFS SERVER WITH 10 GIGABIT ETHERNET NETWORKS

NFS SERVER WITH 10 GIGABIT ETHERNET NETWORKS NFS SERVER WITH 1 GIGABIT ETHERNET NETWORKS A Dell Technical White Paper DEPLOYING 1GigE NETWORK FOR HIGH PERFORMANCE CLUSTERS By Li Ou Massive Scale-Out Systems delltechcenter.com Deploying NFS Server

More information

Low-cost BYO Mass Storage Project. James Cizek Unix Systems Manager Academic Computing and Networking Services

Low-cost BYO Mass Storage Project. James Cizek Unix Systems Manager Academic Computing and Networking Services Low-cost BYO Mass Storage Project James Cizek Unix Systems Manager Academic Computing and Networking Services The Problem Reduced Budget Storage needs growing Storage needs changing (Tiered Storage) I

More information

Cornell University Center for Advanced Computing

Cornell University Center for Advanced Computing Cornell University Center for Advanced Computing David A. Lifka - lifka@cac.cornell.edu Director - Cornell University Center for Advanced Computing (CAC) Director Research Computing - Weill Cornell Medical

More information

Stovepipes to Clouds. Rick Reid Principal Engineer SGI Federal. 2013 by SGI Federal. Published by The Aerospace Corporation with permission.

Stovepipes to Clouds. Rick Reid Principal Engineer SGI Federal. 2013 by SGI Federal. Published by The Aerospace Corporation with permission. Stovepipes to Clouds Rick Reid Principal Engineer SGI Federal 2013 by SGI Federal. Published by The Aerospace Corporation with permission. Agenda Stovepipe Characteristics Why we Built Stovepipes Cluster

More information

SMB Direct for SQL Server and Private Cloud

SMB Direct for SQL Server and Private Cloud SMB Direct for SQL Server and Private Cloud Increased Performance, Higher Scalability and Extreme Resiliency June, 2014 Mellanox Overview Ticker: MLNX Leading provider of high-throughput, low-latency server

More information

HPC Cloud. Focus on your research. Floris Sluiter Project leader SARA

HPC Cloud. Focus on your research. Floris Sluiter Project leader SARA HPC Cloud Focus on your research Floris Sluiter Project leader SARA Why an HPC Cloud? Christophe Blanchet, IDB - Infrastructure Distributing Biology: Big task to port them all to your favorite architecture

More information

Introduction to High-Speed InfiniBand Interconnect

Introduction to High-Speed InfiniBand Interconnect Introduction to High-Speed InfiniBand Interconnect What is InfiniBand? Industry standard defined by the InfiniBand Trade Association Originated in 1999 InfiniBand specification defines an input/output

More information

Demartek September Intel 10GbE Adapter Performance Evaluation for FCoE and iscsi. Introduction. Evaluation Environment. Evaluation Summary

Demartek September Intel 10GbE Adapter Performance Evaluation for FCoE and iscsi. Introduction. Evaluation Environment. Evaluation Summary Intel 10GbE Adapter Performance Evaluation for FCoE and iscsi Evaluation report prepared under contract with Intel Corporation Introduction As the interest in converged networks grows, and as the vendors

More information

A National Computing Grid: FGI

A National Computing Grid: FGI A National Computing Grid: FGI Vera Hansper, Ulf Tigerstedt, Kimmo Mattila, Luis Alves 3/10/2012 FGI Grids in Finland : a short history 3/10/2012 FGI In the beginning, we had M-Grid Interest in Grid technology

More information

Scala Storage Scale-Out Clustered Storage White Paper

Scala Storage Scale-Out Clustered Storage White Paper White Paper Scala Storage Scale-Out Clustered Storage White Paper Chapter 1 Introduction... 3 Capacity - Explosive Growth of Unstructured Data... 3 Performance - Cluster Computing... 3 Chapter 2 Current

More information

RFP-MM-1213-11067 Enterprise Storage Addendum 1

RFP-MM-1213-11067 Enterprise Storage Addendum 1 Purchasing Department August 16, 2012 RFP-MM-1213-11067 Enterprise Storage Addendum 1 A. SPECIFICATION CLARIFICATIONS / REVISIONS NONE B. REQUESTS FOR INFORMATION Oracle: 1) What version of Oracle is in

More information

Configuration Maximums

Configuration Maximums Topic Configuration s VMware vsphere 5.1 When you select and configure your virtual and physical equipment, you must stay at or below the maximums supported by vsphere 5.1. The limits presented in the

More information

Dell Storage for HPC with Intel Enterprise Edition 2.3 for Lustre

Dell Storage for HPC with Intel Enterprise Edition 2.3 for Lustre Dell Storage for HPC with Intel Enterprise Edition 2.3 for Lustre A Dell Technical White Paper Quy Ta Dell HPC Engineering October 2015 Version 1.0 THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY,

More information

Openstack and SCITAS

Openstack and SCITAS Openstack and HPC @ SCITAS ricardo.silva@epfl.ch EPFL - SCITAS June 29, 2015 ricardo.silva@epfl.ch (EPFL - SCITAS) Openstack and HPC @ SCITAS June 29, 2015 1 / 11 Why virtualization? Use Cases Most of

More information

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com Parallels Cloud Storage White Paper Performance Benchmark Results www.parallels.com Table of Contents Executive Summary... 3 Architecture Overview... 3 Key Features... 4 No Special Hardware Requirements...

More information

Performance in a Gluster System. Versions 3.1.x

Performance in a Gluster System. Versions 3.1.x Performance in a Gluster System Versions 3.1.x TABLE OF CONTENTS Table of Contents... 2 List of Figures... 3 1.0 Introduction to Gluster... 4 2.0 Gluster view of Performance... 5 2.1 Good performance across

More information

I/O Performance of Virtualized Cloud Environments. Devarshi Ghoshal, Richard S. Canon, Lavanya Ramakrishnan

I/O Performance of Virtualized Cloud Environments. Devarshi Ghoshal, Richard S. Canon, Lavanya Ramakrishnan I/O Performance of Virtualized Cloud Environments Devarshi Ghoshal, Richard S. Canon, Lavanya Ramakrishnan Data Volumes are Increasing Data is a critical component of scientific process Joint Genome Institute

More information

The PHI solution. Fujitsu Industry Ready Intel XEON-PHI based solution. SC2013 - Denver

The PHI solution. Fujitsu Industry Ready Intel XEON-PHI based solution. SC2013 - Denver 1 The PHI solution Fujitsu Industry Ready Intel XEON-PHI based solution SC2013 - Denver Industrial Application Challenges Most of existing scientific and technical applications Are written for legacy execution

More information

Cluster Scalability of ANSYS FLUENT 12 for a Large Aerodynamics Case on the Darwin Supercomputer

Cluster Scalability of ANSYS FLUENT 12 for a Large Aerodynamics Case on the Darwin Supercomputer Cluster Scalability of ANSYS FLUENT 12 for a Large Aerodynamics Case on the Darwin Supercomputer Stan Posey, MSc and Bill Loewe, PhD Panasas Inc., Fremont, CA, USA Paul Calleja, PhD University of Cambridge,

More information

Kerrighed / XtreemOS cluster flavour

Kerrighed / XtreemOS cluster flavour Kerrighed / XtreemOS cluster flavour Jean Parpaillon Reisensburg Castle Günzburg, Germany July 5-9, 2010 July 6th, 2010 Kerrighed - XtreemOS cluster flavour 1 Summary Kerlabs Context Kerrighed Project

More information

Desktop Strategy. 28 th CompRep Gianluca Capitani. Head of Servers and Desktops Section. ECMWF June 7, 2016

Desktop Strategy. 28 th CompRep Gianluca Capitani. Head of Servers and Desktops Section. ECMWF June 7, 2016 Desktop Strategy 28 th CompRep 2016 Gianluca Capitani Head of Servers and Desktops Section gianluca.capitani@ecmwf.int ECMWF June 7, 2016 Once upon a time but not long ago 2 What is currently happening

More information

HP + HPC Storage. Gavin Brebner USD Engineering

HP + HPC Storage. Gavin Brebner USD Engineering HP + HPC Storage Gavin Brebner USD Engineering 2009 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Where storage is used in HPC. Scratch

More information

Investigation of storage options for scientific computing on Grid and Cloud facilities

Investigation of storage options for scientific computing on Grid and Cloud facilities Investigation of storage options for scientific computing on Grid and Cloud facilities Overview Context Test Bed Lustre Evaluation Standard benchmarks Application-based benchmark HEPiX Storage Group report

More information

ECDF Infrastructure Refresh - Requirements Consultation Document

ECDF Infrastructure Refresh - Requirements Consultation Document Edinburgh Compute & Data Facility - December 2014 ECDF Infrastructure Refresh - Requirements Consultation Document Introduction In order to sustain the University s central research data and computing

More information

An Introduction to the Lustre Parallel File System Tom Edwards

An Introduction to the Lustre Parallel File System Tom Edwards An Introduction to the Lustre Parallel File System Tom Edwards tedwards@cray.com Agenda Introduction to storage hardware RAID Parallel Filesystems Lustre Mapping Common IO Strategies to Lustre Spokesperson

More information

HP Proliant BL460c G7

HP Proliant BL460c G7 HP Proliant BL460c G7 The HP Proliant BL460c G7, is a high performance, fully fault tolerant, nonstop server. It s well suited for all mid-level operations, including environments with local storage, SAN

More information

3 Red Hat Enterprise Linux 6 Consolidation

3 Red Hat Enterprise Linux 6 Consolidation Whitepaper Consolidation EXECUTIVE SUMMARY At this time of massive and disruptive technological changes where applications must be nimbly deployed on physical, virtual, and cloud infrastructure, Red Hat

More information

David Rioja Redondo Telecommunication Engineer Englobe Technologies and Systems

David Rioja Redondo Telecommunication Engineer Englobe Technologies and Systems David Rioja Redondo Telecommunication Engineer Englobe Technologies and Systems About me David Rioja Redondo Telecommunication Engineer - Universidad de Alcalá >2 years building and managing clusters UPM

More information

LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance

LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance 11 th International LS-DYNA Users Conference Session # LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance Gilad Shainer 1, Tong Liu 2, Jeff Layton 3, Onur Celebioglu

More information

CHESS DAQ* Introduction

CHESS DAQ* Introduction CHESS DAQ* Introduction Werner Sun (for the CLASSE IT group), Cornell University * DAQ = data acquisition https://en.wikipedia.org/wiki/data_acquisition Big Data @ CHESS Historically, low data volumes:

More information

Experiences & Performance of SFS/Lustre Cluster File System in Production

Experiences & Performance of SFS/Lustre Cluster File System in Production Experiences & Performance of SFS/Lustre Cluster File System in Production Computing Centre (SSCK) University of Karlsruhe Laifer@rz.uni-karlsruhe.de page 1 Outline» Underlying hardware at SSCK» Performance

More information

Michael Kagan. michael@mellanox.com

Michael Kagan. michael@mellanox.com Virtualization in Data Center The Network Perspective Michael Kagan CTO, Mellanox Technologies michael@mellanox.com Outline Data Center Transition Servers S as a Service Network as a Service IO as a Service

More information

Preview of a Novel Architecture for Large Scale Storage

Preview of a Novel Architecture for Large Scale Storage Preview of a Novel Architecture for Large Scale Storage Andreas Petzold, Christoph-Erdmann Pfeiler, Jos van Wezel Steinbuch Centre for Computing STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the

More information

Installation of LLNL s Sequoia File System

Installation of LLNL s Sequoia File System Installation of LLNL s Sequoia File System Marc Stearman Parallel File Systems Operations Lead marc@llnl.gov April 2012 This work performed under the auspices of the U.S. Department of Energy by Lawrence

More information

Work Environment. David Tur HPC Expert. HPC Users Training September, 18th 2015

Work Environment. David Tur HPC Expert. HPC Users Training September, 18th 2015 Work Environment David Tur HPC Expert HPC Users Training September, 18th 2015 1. Atlas Cluster: Accessing and using resources 2. Software Overview 3. Job Scheduler 1. Accessing Resources DIPC technicians

More information

How E/EF-Series can help solve HPC Challenges MEW25 3 rd December 2014

How E/EF-Series can help solve HPC Challenges MEW25 3 rd December 2014 How E/EF-Series can help solve HPC Challenges MEW25 3 rd December 2014 Mohinder Toor Business Development Executive NEMEA E-Series Sales and Product Specialist Emerging Products Group 1 2014 NetApp, Inc.

More information