Infiniband Monitoring of HPC Clusters using LDMS (Lightweight Distributed Monitoring System)
|
|
|
- Erick Simmons
- 8 years ago
- Views:
Transcription
1 OpenFabrics Software User Group Workshop Infiniband Monitoring of HPC Clusters using LDMS (Lightweight Distributed Monitoring System) Christopher Beggio Sandia National Laboratories Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy s National Nuclear Security Administration under contract DE-AC04-94AL SAND NO XXXXP
2 Goals Understand how user/applications are utilizing network resources Intra-job communications File system communications PFS NFS Understand network congestion Intensity, longevity, extent Drive job scheduling and resource allocation based on: Dynamic state information Knowledge of user/application historic utilization File system locations
3 Current Impediments Understanding of user/application network needs is occluded by possible congestion both internal and external to application Opacity of core switches only lends itself to vague notion of congestion characteristics No snapshot ability because counter readings aren t synchronized Rapid sampling of core fabric adversely impacts application traffic
4 Deployment Configuration: Data Collection, Transport, and Storage RDMA Socket ldmsd
5 Chama: 1232 node TLCC2 cluster (SNL) Socket over Private Internal Network Socket over Private Internal Network RDMA over IB RDMA over IB
6 LDMS Architecture Memory Metric Set Metric Set Metric Set Sampler Plug-in Interface Memory Sampler LDMS API (libldms) HSN Sampler Transport Driver Interface LDMS Daemon Storage Plug-in Interface CSV Store LDMS API (libldms) Other Store Storage CSV MySQL Metric Set RDMA Transport Socket Transport Flat File 6
7 Metric Set Format 7
8 IB Host Based Metric Set chama466/sysclassib: consistent, last update: Tue Jan 20 20:36: [6129us] U64 0 ib.symbol_error#qib0.1 U64 0 ib.link_error_recovery#qib0.1 U64 0 ib.link_downed#qib0.1 U64 0 ib.port_rcv_errors#qib0.1 U64 0 ib.port_rcv_remote_physical_errors#qib0.1 U64 0 ib.port_rcv_switch_relay_errors#qib0.1 U64 0 ib.port_xmit_discards#qib0.1 U64 0 ib.port_xmit_constraint_errors#qib0.1 U64 0 ib.port_rcv_constraint_errors#qib0.1 U64 0 ib.local_link_integrity_errors#qib0.1 U64 0 ib.excessive_buffer_overrun_errors#qib0.1 U64 0 ib.vl15_dropped#qib0.1 U ib.port_xmit_data#qib0.1 U ib.port_rcv_data#qib0.1 U ib.port_xmit_packets#qib0.1 U ib.port_rcv_packets#qib0.1 U64 0 ib.port_xmit_wait#qib0.1 U ib.port_unicast_xmit_packets#qib0.1 U ib.port_unicast_rcv_packets#qib0.1 U ib.port_multicast_xmit_packets#qib0.1 U ib.port_multicast_rcv_packets#qib0.1
9 Host Side Collection (would also like from every core link) Errors: symbol_error link_error_recovery port_rcv_errors port_rcv_remote_physical_errors port_rcv_switch_relay_errors port_xmit_constraint_errors port_rcv_constraint_errors local_link_integrity_errors excessive_buffer_overrun_errors Data Info: port_xmit_data port_rcv_data port_xmit_packets port_rcv_packets Misc. Info: port_xmit_wait port_xmit_discards VL15_dropped link_downed
10 Other LDMS Metric Set Examples shuttle-cray.ran.sandia.gov_1/meminfo U MemFree U Buffers U Cached U SwapCached U Active shuttle-cray.ran.sandia.gov_1/procstatutil U cpu0_user_raw U cpu0_sys_raw U cpu0_idle_raw U cpu0_iowait_raw shuttle-cray.ran.sandia.gov_1/vmstat U nr_free_pages U nr_interactive_anon U nr_active_anon U nr_inactive_file U nr_active_file Metric sets: (datatype, value, metricname) tuples optional per metric user metadata e.g., component id API: ldms_get_set ldms_get_metric ldms_get_u64 Same API for on-node and offnode transport
11 Would Like Synchronized counter collection across all switches Configurable at boot Fixed set On-the-fly collection frequency change Offload IB counter collection from computes Currently working on reference implementation using LDMS protocol
12 IB Traffic 12/9/2014
13 Lustre Traffic 12/9/2014
14 IB Traffic 1/22/2014
15 Lustre Traffic 1/22/2015
16
17 IB Traffic Information = No information = Have information
18 Summary Goals Impediments LDMS Architecture IB Host Metrics IB Traffic Lustre Traffic Path forward
19 Questions?
20 Thank You OpenFabrics Software User Group Workshop #OFSUserGroup
GSAND2015-1899C. Demonstrating Improved Application Performance Using Dynamic Monitoring and Task Mapping
Exceptional service in the national interest GSAND2015-1899C Demonstrating Improved Application Performance Using Dynamic Monitoring and Task Mapping J. Brandt, K. Devine, A. Gentile, K. Pedretti Sandia,
The Lightweight Distributed Metric Service: A Scalable Infrastructure for Continuous Monitoring of Large Scale Computing Systems and Applications
The Lightweight Distributed Metric Service: A Scalable Infrastructure for Continuous Monitoring of Large Scale Computing Systems and Applications Anthony Agelastos, Benjamin Allan, Jim Brandt, Paul Cassella,
Leveraging In-Memory Key/Value Stores in HPC Kelpie Craig Ulmer
Photos placed in horizontal position with even amount of white space between photos and header Leveraging In-Memory Key/Value Stores in HPC Kelpie Craig Ulmer Sandia National Laboratories is a multi-program
How To Monitor Infiniband Network Data From A Network On A Leaf Switch (Wired) On A Microsoft Powerbook (Wired Or Microsoft) On An Ipa (Wired/Wired) Or Ipa V2 (Wired V2)
INFINIBAND NETWORK ANALYSIS AND MONITORING USING OPENSM N. Dandapanthula 1, H. Subramoni 1, J. Vienne 1, K. Kandalla 1, S. Sur 1, D. K. Panda 1, and R. Brightwell 2 Presented By Xavier Besseron 1 Date:
Lustre Networking BY PETER J. BRAAM
Lustre Networking BY PETER J. BRAAM A WHITE PAPER FROM CLUSTER FILE SYSTEMS, INC. APRIL 2007 Audience Architects of HPC clusters Abstract This paper provides architects of HPC clusters with information
Cloud Development Strategies
Photos placed in horizontal position with even amount of white space between photos and header Cloud Development Strategies A 2015 NLIT Fueling the Fire in IT presentation John G Miner, Mgr of Enterprise
Introduction to Infiniband. Hussein N. Harake, Performance U! Winter School
Introduction to Infiniband Hussein N. Harake, Performance U! Winter School Agenda Definition of Infiniband Features Hardware Facts Layers OFED Stack OpenSM Tools and Utilities Topologies Infiniband Roadmap
InfiniBand Software and Protocols Enable Seamless Off-the-shelf Applications Deployment
December 2007 InfiniBand Software and Protocols Enable Seamless Off-the-shelf Deployment 1.0 Introduction InfiniBand architecture defines a high-bandwidth, low-latency clustering interconnect that is used
Can High-Performance Interconnects Benefit Memcached and Hadoop?
Can High-Performance Interconnects Benefit Memcached and Hadoop? D. K. Panda and Sayantan Sur Network-Based Computing Laboratory Department of Computer Science and Engineering The Ohio State University,
Application Performance and Security Testing
Photos placed in horizontal position with even amount of white space between photos and header Application Performance and Security Testing NLIT May 1 4, 2016 Christi Forsythe and Roy Life, Sandia National
OS/Run'me and Execu'on Time Produc'vity
OS/Run'me and Execu'on Time Produc'vity Ron Brightwell, Technical Manager Scalable System SoAware Department Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation,
So#ware Tools and Techniques for HPC, Clouds, and Server- Class SoCs Ron Brightwell
So#ware Tools and Techniques for HPC, Clouds, and Server- Class SoCs Ron Brightwell R&D Manager, Scalable System So#ware Department Sandia National Laboratories is a multi-program laboratory managed and
Web Application Platform for Sandia
Web Application Platform for Sandia Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for
A Micro-benchmark Suite for Evaluating Hadoop RPC on High-Performance Networks
A Micro-benchmark Suite for Evaluating Hadoop RPC on High-Performance Networks Xiaoyi Lu, Md. Wasi- ur- Rahman, Nusrat Islam, and Dhabaleswar K. (DK) Panda Network- Based Compu2ng Laboratory Department
Building a Scalable Storage with InfiniBand
WHITE PAPER Building a Scalable Storage with InfiniBand The Problem...1 Traditional Solutions and their Inherent Problems...2 InfiniBand as a Key Advantage...3 VSA Enables Solutions from a Core Technology...5
Achieving Mainframe-Class Performance on Intel Servers Using InfiniBand Building Blocks. An Oracle White Paper April 2003
Achieving Mainframe-Class Performance on Intel Servers Using InfiniBand Building Blocks An Oracle White Paper April 2003 Achieving Mainframe-Class Performance on Intel Servers Using InfiniBand Building
Integration of Supercomputers and Data-Warehouse Appliances
August, 2013 Integration of Supercomputers and Data-Warehouse Appliances Approved for Public Release: SAND 2013-6833 C Ron A. Oldfield, George Davidson, Craig Ulmer, Andy Wilson Sandia National Laboratories
A Tour of the Linux OpenFabrics Stack
A Tour of the OpenFabrics Stack Johann George, QLogic June 2006 1 Overview Beyond Sockets Provides a common interface that allows applications to take advantage of the RDMA (Remote Direct Memory Access),
Storage at a Distance; Using RoCE as a WAN Transport
Storage at a Distance; Using RoCE as a WAN Transport Paul Grun Chief Scientist, System Fabric Works, Inc. (503) 620-8757 pgrun@systemfabricworks.com Why Storage at a Distance the Storage Cloud Following
Network Contention and Congestion Control: Lustre FineGrained Routing
Network Contention and Congestion Control: Lustre FineGrained Routing Matt Ezell HPC Systems Administrator Oak Ridge Leadership Computing Facility Oak Ridge National Laboratory International Workshop on
The Importability of Performance Tools: The Good, the Bad, and the Ugly of PMUs
The Importability of Performance Tools: The Good, the Bad, and the Ugly of PMUs Jeanine Cook Scalable Architectures SNL ABQ Sandia National Laboratories is a multi-program laboratory managed and operated
Who is Watching You? Video Conferencing Security
Who is Watching You? Video Conferencing Security Navid Jam Member of Technical Staff March 1, 2007 SAND# 2007-1115C Computer and Network Security Security Systems and Technology Video Conference and Collaborative
Cloud Defense. Kevin Hall Cyber Security Technology Department 4/17/2012. Sandia National Laboratories SAND 2012-2581C
4/17/2012 Cloud Defense Kevin Hall Cyber Security Technology Department Sandia National Laboratories SAND 2012-2581C Sandia National Laboratories is a multi program laboratory managed and operated by Sandia
Chemical Safety and Security Officer Training for Laboratory Settings
Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy s National
New Storage System Solutions
New Storage System Solutions Craig Prescott Research Computing May 2, 2013 Outline } Existing storage systems } Requirements and Solutions } Lustre } /scratch/lfs } Questions? Existing Storage Systems
HPC Software Requirements to Support an HPC Cluster Supercomputer
HPC Software Requirements to Support an HPC Cluster Supercomputer Susan Kraus, Cray Cluster Solutions Software Product Manager Maria McLaughlin, Cray Cluster Solutions Product Marketing Cray Inc. WP-CCS-Software01-0417
Building Enterprise-Class Storage Using 40GbE
Building Enterprise-Class Storage Using 40GbE Unified Storage Hardware Solution using T5 Executive Summary This white paper focuses on providing benchmarking results that highlight the Chelsio T5 performance
Cluster Implementation and Management; Scheduling
Cluster Implementation and Management; Scheduling CPS343 Parallel and High Performance Computing Spring 2013 CPS343 (Parallel and HPC) Cluster Implementation and Management; Scheduling Spring 2013 1 /
Energy Storage Safety Plan Implementation Kickoff
Energy Storage Safety Plan Implementation Kickoff Web Meeting July 8, 2015 Hosted by DOE-OE Energy Storage Program Sandia National Laboratories Pacific Northwest National Laboratory SNL Document Number:
Exadata Database Machine Administration Workshop NEW
Exadata Database Machine Administration Workshop NEW Duration: 4 Days What you will learn This course introduces students to Oracle Exadata Database Machine. Students learn about the various Exadata Database
Mellanox Academy Online Training (E-learning)
Mellanox Academy Online Training (E-learning) 2013-2014 30 P age Mellanox offers a variety of training methods and learning solutions for instructor-led training classes and remote online learning (e-learning),
High Availability with Windows Server 2012 Release Candidate
High Availability with Windows Server 2012 Release Candidate Windows Server 2012 Release Candidate (RC) delivers innovative new capabilities that enable you to build dynamic storage and availability solutions
Running Native Lustre* Client inside Intel Xeon Phi coprocessor
Running Native Lustre* Client inside Intel Xeon Phi coprocessor Dmitry Eremin, Zhiqi Tao and Gabriele Paciucci 08 April 2014 * Some names and brands may be claimed as the property of others. What is the
The Retail Payment System - Disruption of RPS
2012_6458P by Teresa Chambers FUNDED BY THE DEPARTMENT OF HOMELAND SECURITY, OFFICE OF SCIENCE AND TECHNOLOGY, AWARD #2009-ST-062-000031 Creation of a Community College-University Collaborative to Promote
SMB Advanced Networking for Fault Tolerance and Performance. Jose Barreto Principal Program Managers Microsoft Corporation
SMB Advanced Networking for Fault Tolerance and Performance Jose Barreto Principal Program Managers Microsoft Corporation Agenda SMB Remote File Storage for Server Apps SMB Direct (SMB over RDMA) SMB Multichannel
Globus and the Centralized Research Data Infrastructure at CU Boulder
Globus and the Centralized Research Data Infrastructure at CU Boulder Daniel Milroy, daniel.milroy@colorado.edu Conan Moore, conan.moore@colorado.edu Thomas Hauser, thomas.hauser@colorado.edu Peter Ruprecht,
Intel Cluster Ready Appro Xtreme-X Computers with Mellanox QDR Infiniband
Intel Cluster Ready Appro Xtreme-X Computers with Mellanox QDR Infiniband A P P R O I N T E R N A T I O N A L I N C Steve Lyness Vice President, HPC Solutions Engineering slyness@appro.com Company Overview
Achieving Real-Time Business Solutions Using Graph Database Technology and High Performance Networks
WHITE PAPER July 2014 Achieving Real-Time Business Solutions Using Graph Database Technology and High Performance Networks Contents Executive Summary...2 Background...3 InfiniteGraph...3 High Performance
High Speed I/O Server Computing with InfiniBand
High Speed I/O Server Computing with InfiniBand José Luís Gonçalves Dep. Informática, Universidade do Minho 4710-057 Braga, Portugal zeluis@ipb.pt Abstract: High-speed server computing heavily relies on
Appro Supercomputer Solutions Best Practices Appro 2012 Deployment Successes. Anthony Kenisky, VP of North America Sales
Appro Supercomputer Solutions Best Practices Appro 2012 Deployment Successes Anthony Kenisky, VP of North America Sales About Appro Over 20 Years of Experience 1991 2000 OEM Server Manufacturer 2001-2007
An Oracle White Paper October 2012. InfiniBand Network Troubleshooting Guidelines and Methodologies
An Oracle White Paper October 2012 InfiniBand Network Troubleshooting Guidelines and Methodologies EXECUTIVE OVERVIEW... 3 NETWORK BUILDING BLOCKS... 4 OPEN SYSTEMS INTERCONNECT (OSI)... 4 NETWORK BUILDING
IEEE Congestion Management Presentation for IEEE Congestion Management Study Group
IEEE Congestion Management Presentation for IEEE Congestion Management Study Group Contributors Jeff Lynch IBM Gopal Hegde -- Intel 2 Outline Problem Statement Types of Traffic & Typical Usage Models Traffic
Know your Cluster Bottlenecks and Maximize Performance
Know your Cluster Bottlenecks and Maximize Performance Hands-on training March 2013 Agenda Overview Performance Factors General System Configuration - PCI Express (PCIe) Capabilities - Memory Configuration
Implementing Network Attached Storage. Ken Fallon Bill Bullers Impactdata
Implementing Network Attached Storage Ken Fallon Bill Bullers Impactdata Abstract The Network Peripheral Adapter (NPA) is an intelligent controller and optimized file server that enables network-attached
TCP Servers: Offloading TCP Processing in Internet Servers. Design, Implementation, and Performance
TCP Servers: Offloading TCP Processing in Internet Servers. Design, Implementation, and Performance M. Rangarajan, A. Bohra, K. Banerjee, E.V. Carrera, R. Bianchini, L. Iftode, W. Zwaenepoel. Presented
Why Compromise? A discussion on RDMA versus Send/Receive and the difference between interconnect and application semantics
Why Compromise? A discussion on RDMA versus Send/Receive and the difference between interconnect and application semantics Mellanox Technologies Inc. 2900 Stender Way, Santa Clara, CA 95054 Tel: 408-970-3400
High Performance Data-Transfers in Grid Environment using GridFTP over InfiniBand
High Performance Data-Transfers in Grid Environment using GridFTP over InfiniBand Hari Subramoni *, Ping Lai *, Raj Kettimuthu **, Dhabaleswar. K. (DK) Panda * * Computer Science and Engineering Department
Lustre * Filesystem for Cloud and Hadoop *
OpenFabrics Software User Group Workshop Lustre * Filesystem for Cloud and Hadoop * Robert Read, Intel Lustre * for Cloud and Hadoop * Brief Lustre History and Overview Using Lustre with Hadoop Intel Cloud
High Performance Computing OpenStack Options. September 22, 2015
High Performance Computing OpenStack PRESENTATION TITLE GOES HERE Options September 22, 2015 Today s Presenters Glyn Bowden, SNIA Cloud Storage Initiative Board HP Helion Professional Services Alex McDonald,
LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance
11 th International LS-DYNA Users Conference Session # LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance Gilad Shainer 1, Tong Liu 2, Jeff Layton 3, Onur Celebioglu
Investigations on InfiniBand: Efficient Network Buffer Utilization at Scale
Investigations on InfiniBand: Efficient Network Buffer Utilization at Scale Galen M. Shipman 1, Ron Brightwell 2, Brian Barrett 1, Jeffrey M. Squyres 3, and Gil Bloch 4 1 Los Alamos National Laboratory,
An Alternative Storage Solution for MapReduce. Eric Lomascolo Director, Solutions Marketing
An Alternative Storage Solution for MapReduce Eric Lomascolo Director, Solutions Marketing MapReduce Breaks the Problem Down Data Analysis Distributes processing work (Map) across compute nodes and accumulates
BRIDGING EMC ISILON NAS ON IP TO INFINIBAND NETWORKS WITH MELLANOX SWITCHX
White Paper BRIDGING EMC ISILON NAS ON IP TO INFINIBAND NETWORKS WITH Abstract This white paper explains how to configure a Mellanox SwitchX Series switch to bridge the external network of an EMC Isilon
Mellanox Technical Training and Certification Program
Mellanox Technical Partner Certification Program is a world-class certification program, designed for Mellanox partners' technical professionals to ensure and validate the technical and sales required
3G Converged-NICs A Platform for Server I/O to Converged Networks
White Paper 3G Converged-NICs A Platform for Server I/O to Converged Networks This document helps those responsible for connecting servers to networks achieve network convergence by providing an overview
Globus Striped GridFTP Framework and Server. Raj Kettimuthu, ANL and U. Chicago
Globus Striped GridFTP Framework and Server Raj Kettimuthu, ANL and U. Chicago Outline Introduction Features Motivation Architecture Globus XIO Experimental Results 3 August 2005 The Ohio State University
Headline in Arial Bold 30pt. The Need For Speed. Rick Reid Principal Engineer SGI
Headline in Arial Bold 30pt The Need For Speed Rick Reid Principal Engineer SGI Commodity Systems Linux Red Hat SUSE SE-Linux X86-64 Intel Xeon AMD Scalable Programming Model MPI Global Data Access NFS
Lessons learned from parallel file system operation
Lessons learned from parallel file system operation Roland Laifer STEINBUCH CENTRE FOR COMPUTING - SCC KIT University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association
Accelerating From Cluster to Cloud: Overview of RDMA on Windows HPC. Wenhao Wu Program Manager Windows HPC team
Accelerating From Cluster to Cloud: Overview of RDMA on Windows HPC Wenhao Wu Program Manager Windows HPC team Agenda Microsoft s Commitments to HPC RDMA for HPC Server RDMA for Storage in Windows 8 Microsoft
Intel s Vision for Cloud Computing
Intel s Vision for Cloud Computing Author: Jake Smith, Advanced Server Technologies Date: April 5 th, 2011 Location: Monterrey, California, USA www.openfabrics.org 1 By 2015 More Users More Devices More
Performance Evaluation of the RDMA over Ethernet (RoCE) Standard in Enterprise Data Centers Infrastructure. Abstract:
Performance Evaluation of the RDMA over Ethernet (RoCE) Standard in Enterprise Data Centers Infrastructure Motti Beck Director, Marketing motti@mellanox.com Michael Kagan Chief Technology Officer michaelk@mellanox.com
ovirt and Gluster hyper-converged! HA solution for maximum resource utilization
ovirt and Gluster hyper-converged! HA solution for maximum resource utilization 31 st of Jan 2016 Martin Sivák Senior Software Engineer Red Hat Czech FOSDEM, Jan 2016 1 Agenda (Storage) architecture of
Netvisor Software Defined Fabric Architecture
Netvisor Software Defined Fabric Architecture Netvisor Overview The Pluribus Networks network operating system, Netvisor, is designed to power a variety of network devices. The devices Netvisor powers
Pedraforca: ARM + GPU prototype
www.bsc.es Pedraforca: ARM + GPU prototype Filippo Mantovani Workshop on exascale and PRACE prototypes Barcelona, 20 May 2014 Overview Goals: Test the performance, scalability, and energy efficiency of
ovirt and Gluster hyper-converged! HA solution for maximum resource utilization
ovirt and Gluster hyper-converged! HA solution for maximum resource utilization 21 st of Aug 2015 Martin Sivák Senior Software Engineer Red Hat Czech KVM Forum Seattle, Aug 2015 1 Agenda (Storage) architecture
Network Infrastructure Services CS848 Project
Quality of Service Guarantees for Cloud Services CS848 Project presentation by Alexey Karyakin David R. Cheriton School of Computer Science University of Waterloo March 2010 Outline 1. Performance of cloud
Hyper-V over SMB Remote File Storage support in Windows Server 8 Hyper-V. Jose Barreto Principal Program Manager Microsoft Corporation
Hyper-V over SMB Remote File Storage support in Windows Server 8 Hyper-V Jose Barreto Principal Program Manager Microsoft Corporation Agenda Hyper-V over SMB - Overview How to set it up Configuration Options
SMB Direct for SQL Server and Private Cloud
SMB Direct for SQL Server and Private Cloud Increased Performance, Higher Scalability and Extreme Resiliency June, 2014 Mellanox Overview Ticker: MLNX Leading provider of high-throughput, low-latency server
INAM - A Scalable InfiniBand Network Analysis and Monitoring Tool
INAM - A Scalable InfiniBand Network Analysis and Monitoring Tool N. Dandapanthula 1, H. Subramoni 1, J. Vienne 1, K. Kandalla 1, S. Sur 1, D. K. Panda 1, and R. Brightwell 2 1 Department of Computer Science
Implementation of Hadoop Distributed File System Protocol on OneFS Tanuj Khurana EMC Isilon Storage Division
Implementation of Hadoop Distributed File System Protocol on OneFS Tanuj Khurana EMC Isilon Storage Division Outline HDFS Overview OneFS Overview HDFS protocol on OneFS HDFS protocol server implementation
A parallel file system made in Germany Tiered Storage and HSM
A parallel file system made in Germany Tiered Storage and HSM Mai 7th 2012 HLRS Stuttgart Franz-Josef Pfreundt Competence Center for HPC Sven Breuner Fraunhofer Institut for Industrial Mathematics Mathematical
White Paper Solarflare High-Performance Computing (HPC) Applications
Solarflare High-Performance Computing (HPC) Applications 10G Ethernet: Now Ready for Low-Latency HPC Applications Solarflare extends the benefits of its low-latency, high-bandwidth 10GbE server adapters
InfiniBand Clustering
White Paper InfiniBand Clustering Delivering Better Price/Performance than Ethernet 1.0 Introduction High performance computing clusters typically utilize Clos networks, more commonly known as Fat Tree
OFA Training Program. Writing Application Programs for RDMA using OFA Software. Author: Rupert Dance Date: 11/15/2011. www.openfabrics.
OFA Training Program Writing Application Programs for RDMA using OFA Software Author: Rupert Dance Date: 11/15/2011 www.openfabrics.org 1 Agenda OFA Training Program Program Goals Instructors Programming
Architecting a High Performance Storage System
WHITE PAPER Intel Enterprise Edition for Lustre* Software High Performance Data Division Architecting a High Performance Storage System January 2014 Contents Introduction... 1 A Systematic Approach to
Agile VPN for Carrier/SP Network. ONOS- based SDN Controller for China Unicom MPLS L3VPN Service
Agile VPN for Carrier/SP Network ONOS- based SDN Controller for China Unicom MPLS L3VPN Service Introduction In order to meet strong service demand on their network and respond to the new competition (who
Measuring MPI Send and Receive Overhead and Application Availability in High Performance Network Interfaces
Measuring MPI Send and Receive Overhead and Application Availability in High Performance Network Interfaces Douglas Doerfler and Ron Brightwell Center for Computation, Computers, Information and Math Sandia
Zadara Storage Cloud A whitepaper. @ZadaraStorage
Zadara Storage Cloud A whitepaper @ZadaraStorage Zadara delivers two solutions to its customers: On- premises storage arrays Storage as a service from 31 locations globally (and counting) Some Zadara customers
Scaling 10Gb/s Clustering at Wire-Speed
Scaling 10Gb/s Clustering at Wire-Speed InfiniBand offers cost-effective wire-speed scaling with deterministic performance Mellanox Technologies Inc. 2900 Stender Way, Santa Clara, CA 95054 Tel: 408-970-3400
White Paper ClearSCADA Architecture
White Paper ClearSCADA Architecture ClearSCADA has 3 major components or software applications; the ClearSCADA server, a windows client known as, and a web client known as Webx. The software is designed
ALPS Supercomputing System A Scalable Supercomputer with Flexible Services
ALPS Supercomputing System A Scalable Supercomputer with Flexible Services 1 Abstract Supercomputing is moving from the realm of abstract to mainstream with more and more applications and research being
Accelerating Spark with RDMA for Big Data Processing: Early Experiences
Accelerating Spark with RDMA for Big Data Processing: Early Experiences Xiaoyi Lu, Md. Wasi- ur- Rahman, Nusrat Islam, Dip7 Shankar, and Dhabaleswar K. (DK) Panda Network- Based Compu2ng Laboratory Department
Improvement Options for LHC Mass Storage and Data Management
Improvement Options for LHC Mass Storage and Data Management Dirk Düllmann HEPIX spring meeting @ CERN, 7 May 2008 Outline DM architecture discussions in IT Data Management group Medium to long term data
Enhanced Transmission Selection
Enhanced Transmission Selection Background Priorities/classes are being used to separate traffic with different QOS characteristics It is desirable to enable sharing network bandwidth between classes For
Low Latency 10 GbE Switching for Data Center, Cluster and Storage Interconnect
White PAPER Low Latency 10 GbE Switching for Data Center, Cluster and Storage Interconnect Introduction: High Performance Data Centers As the data center continues to evolve to meet rapidly escalating
Current Status of FEFS for the K computer
Current Status of FEFS for the K computer Shinji Sumimoto Fujitsu Limited Apr.24 2012 LUG2012@Austin Outline RIKEN and Fujitsu are jointly developing the K computer * Development continues with system
From Ethernet Ubiquity to Ethernet Convergence: The Emergence of the Converged Network Interface Controller
White Paper From Ethernet Ubiquity to Ethernet Convergence: The Emergence of the Converged Network Interface Controller The focus of this paper is on the emergence of the converged network interface controller
Configuring LNet Routers for File Systems based on Intel Enterprise Edition for Lustre* Software
Configuring LNet Routers for File Systems based on Intel Enterprise Edition for Lustre* Software Partner Guide High Performance Data Division Software Version: 2.4.0.0 World Wide Web: http://www.intel.com
"Charting the Course...... to Your Success!" MOC 50290 A Understanding and Administering Windows HPC Server 2008. Course Summary
Description Course Summary This course provides students with the knowledge and skills to manage and deploy Microsoft HPC Server 2008 clusters. Objectives At the end of this course, students will be Plan
Mellanox Academy Course Catalog. Empower your organization with a new world of educational possibilities 2014-2015
Mellanox Academy Course Catalog Empower your organization with a new world of educational possibilities 2014-2015 Mellanox offers a variety of training methods and learning solutions for instructor-led
Leveraging the SharePoint Investment (Thinking beyond file storage)
Photos placed in horizontal position with even amount of white space between photos and header To replace these boxes with images open the slide master Leveraging the SharePoint Investment (Thinking beyond
Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers over InfiniBand
Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers over InfiniBand P. Balaji, K. Vaidyanathan, S. Narravula, K. Savitha, H. W. Jin D. K. Panda Network Based
Deploying 10/40G InfiniBand Applications over the WAN
Deploying 10/40G InfiniBand Applications over the WAN Eric Dube (eric@baymicrosystems.com) Senior Product Manager of Systems November 2011 Overview About Bay Founded in 2000 to provide high performance
A Software and Hardware Architecture for a Modular, Portable, Extensible Reliability. Availability and Serviceability System
1 A Software and Hardware Architecture for a Modular, Portable, Extensible Reliability Availability and Serviceability System James H. Laros III, Sandia National Laboratories (USA) [1] Abstract This paper
SDN v praxi overlay sítí pro OpenStack. 5.10.2015 Daniel Prchal daniel.prchal@hpe.com
SDN v praxi overlay sítí pro OpenStack 5.10.2015 Daniel Prchal daniel.prchal@hpe.com Agenda OpenStack OpenStack Architecture SDN Software Defined Networking OpenStack Networking HP Helion OpenStack HP
Bright Cluster Manager
Bright Cluster Manager For HPC, Hadoop and OpenStack Craig Hunneyman Director of Business Development Bright Computing Craig.Hunneyman@BrightComputing.com Agenda Who is Bright Computing? What is Bright
Storage Networking Foundations Certification Workshop
Storage Networking Foundations Certification Workshop Duration: 2 Days Type: Lecture Course Description / Overview / Expected Outcome A group of students was asked recently to define a "SAN." Some replies
Cray DVS: Data Virtualization Service
Cray : Data Virtualization Service Stephen Sugiyama and David Wallace, Cray Inc. ABSTRACT: Cray, the Cray Data Virtualization Service, is a new capability being added to the XT software environment with
Sockets vs. RDMA Interface over 10-Gigabit Networks: An In-depth Analysis of the Memory Traffic Bottleneck
Sockets vs. RDMA Interface over 1-Gigabit Networks: An In-depth Analysis of the Memory Traffic Bottleneck Pavan Balaji Hemal V. Shah D. K. Panda Network Based Computing Lab Computer Science and Engineering
High Performance OpenStack Cloud. Eli Karpilovski Cloud Advisory Council Chairman
High Performance OpenStack Cloud Eli Karpilovski Cloud Advisory Council Chairman Cloud Advisory Council Our Mission Development of next generation cloud architecture Providing open specification for cloud