I/O Scheduling Service for Multi-Application Clusters
|
|
- Shanna Morton
- 7 years ago
- Views:
Transcription
1 I/O Scheduling Service for Multi-Application Clusters, Guillaume Huard, Yves Denneulin Laboratoire ID-IMAG (UMR 5132), Grenoble, France. BULL - HPC, Échirolles, France. Przemyslaw Sowa sowa@icis.pcz.pl Institute of Computer and Information Sciences Czestochowa University of Technology, Poland.
2 Plan Part 1 - Parallel Input/Output and Clusters Part 2 - Controlling and Scheduling Multi-application I/O Part 3 - aioli, an Input/Output Scheduler for HPC Part 4 - Conclusion 2/23
3 Plan Part 1 - Parallel Input/Output and Clusters 1 Introduction Context Parallel I/O 2 Parallel I/O and Clusters Available Solutions 3 Objectives Part 2 - Controlling and Scheduling Multi-application I/O Part 3 - aioli, an Input/Output Scheduler for HPC Part 4 - Conclusion 3/23
4 Context Environment Clusters of SMPs Linux High Performance Computing Intensive I/O applications CPU bounded application I/O bounded application Remote hard drive I/O Parallel I/O Handling concurrent accesses to a same resource (a file) Accesses: different in size, in offset Example: a parallel matrix product 4/23
5 Parallel I/O - Example Parallel matrix product Specific parts to fetch according to the data distribution (columns/rows, BLOCK/BLOCK...) Matrix A Matrix B Matrix C a b c 1 2 3??? d e f 4 5 6??? g h i 7 8 9??? Matrices are stored "row by row" in files Matrix A a b c d e f g h i Matrix B /23
6 Parallel I/O - Example Parallel matrix product Specific parts to fetch according to the data distribution (columns/rows, BLOCK/BLOCK...) Matrix A Matrix B Matrix C a b c 1 2 3??? d e f 4 5 6??? g h i 7 8 9??? P0 1 read (n) n read (1) Matrices are stored "row by row" in files Matrix A a b c d e f g h i Matrix B /23
7 Parallel I/O - Example Parallel matrix product Specific parts to fetch according to the data distribution (columns/rows, BLOCK/BLOCK...) Matrix A Matrix B Matrix C a b c 1 2 3??? d e f 4 5 6??? g h i 7 8 9??? P0 1 read (n) n read (1) 1 read (n) n read (1) Matrices are stored "row by row" in files Matrix A a b c d e f g h i Matrix B /23
8 Parallel I/O - Example Parallel matrix product Specific parts to fetch according to the data distribution (columns/rows, BLOCK/BLOCK...) File decomposition: Lot of disjoint/contiguous requests at the same time lethal behaviour for I/O subsystem Matrix A Matrix B Matrix C a b c 1 2 3??? d e f 4 5 6??? g h i 7 8 9??? P0 Pi 1 read (n) n read (1) 1 read (n) n read (1) 1 read (n) n read (1) Matrix A Matrices are stored "row by row" in files a b c d e f g h i Pp 1 read (n) n read (1) Matrix B No defined order between requests many disk head movements n2 * read (n) n2 * n read (1) 5/23
9 Parallel I/O and Clusters Parallel I/O bottlenecks SMP 1 SMP 2 Compute nodes SMP 3 Network interconnection I/O server (NFS) SMP 4 SMP n Application 1 6/23
10 Parallel I/O and Clusters Parallel I/O bottlenecks SMP 1 SMP 2 Compute nodes SMP 3 Network interconnection I/O server (NFS) SMP 4 SMP n Application 1 Hypothesis: network has fewer impact than the I/O subsystems. 6/23
11 Parallel I/O Solutions (1/2) Performance constraints Reduce the number of requests: decrease overhead implied by the different syscalls Requests Scheduling: avoid expensive seeks and maximize large accessses Exploit cache mechanisms (read-ahead strategies,...) Available solutions Parallel File Systems : balance requests between several data servers Designed for Parallel I/O (PIOUS, VESTA,...): gave up in the time More generic (PVFS, GPFS, Lustre, Parallel NFS) +/- complete / +/- efficient Libraries (MPI I/O): Focus on performance and portability aspects Sophisticated API Development overhead / Language bindings No global coordination 7/23
12 Parallel I/O Solutions (2/2) Libraries in multi-application environment Application 1 Compute nodes SMP 1 SMP 2 SMP 3 ROMIO ROMIO Network Interconnection I/O server 1 I/O server 2 Storage nodes SMP 4 I/O server p SMP n Application 2 8/23
13 Parallel I/O Solutions (2/2) Libraries in multi-application environment I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)? Appli 1 Appli 2 No informations about applications, only about files File 1 File 2 SEEK? 8/23
14 Parallel I/O Solutions (2/2) Libraries in multi-application environment I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)? Appli 1 Appli 2 No informations about applications, only about files File 1 File 2 8/23
15 Parallel I/O Solutions (2/2) Libraries in multi-application environment I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)?! Appli 1 Appli 2 No informations about applications, only about files File 1 File 2 SEEK 8/23
16 Parallel I/O Solutions (2/2) Libraries in multi-application environment I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)?! Appli 1 Appli 2 No informations about applications, only about files File 1 File 2 8/23
17 Parallel I/O Solutions (2/2) Libraries in multi-application environment I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)?!! Appli 1 Appli 2 No informations about applications, only about files File 1 File 2 SEEK 8/23
18 Parallel I/O Solutions (2/2) Libraries in multi-application environment I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)?!! Appli 1 Appli 2 No informations about applications, only about files File 1 File 2 8/23
19 Parallel I/O Solutions (2/2) Libraries in multi-application environment I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)?!!! Appli 1 Appli 2 No informations about applications, only about files File 1 File 2 SEEK 8/23
20 Parallel I/O Solutions (2/2) Libraries in multi-application environment I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)?!!!! Appli 1 Appli 2 To switch from one file to the other implies a disk head movement File 1 File 2 Libraries are not suited global synchronization is required 8/23
21 A New Approach Objectives Supply Parallel I/O algorithms scheduling / aggregating / overlapping access mono-application efficiency Only through the use of the ubiquitous POSIX calls: open/read/write/lseek/close portability / simplicity Address requests in a global manner multi-application efficiency Naive approach: Processing all the requests from one application before serving another one Not suited for a cluster Tradeoff between fairness and performance 9/23
22 Plan Part 1 - Parallel Input/Output and Clusters Part 2 - Controlling and Scheduling Multi-application I/O 4 Scheduling in Multi-application Environment General Algorithm 5 Synchronous Behaviour Part 3 - aioli, an Input/Output Scheduler for HPC Part 4 - Conclusion 10/23
23 Multi-application Scheduling Appli 1 Appli2 SMP 1 Appli 3 SMP 2 I/O server 1 Compute nodes Appli 5 SMP 3 Appli 4 Network interconnection I/O server 2 Storage nodes SMP 4 Appli 7 I/O server p Appli n SMP n 11/23
24 Multi-application Scheduling Appli 1 Appli2 SMP 1 Appli 3 SMP 2 I/O server 1 Compute nodes Appli 5 SMP 3 Appli 4 Network interconnection I/O server 2 Storage nodes SMP 4 Appli 7 I/O server p Appli n SMP n 11/23
25 Multi-application Scheduling Appli 1 Appli2 SMP 1 Appli 3 SMP 2 Compute nodes Appli 5 SMP 3 Appli 4 Network interconnection I/O server (NFS) SMP 4 Appli 7 Appli n SMP n 11/23
26 Multi-application Scheduling Online problem: several requests from distinct applications are delivered to the file systems. Wished criteria : Efficiency with fairness constraints Maximize the minimum of instantaneous throughput for each application Algorithm : Multi-Level Feedback variant (quantum approach) Step 1 Processing Waiting A1 A2 A1 A2 A1 A2 A1 A3 Pre processing (offset dependance) A1 A1 A1 A1 A2 A2 q = 10, T = 20 Scheduling A2 A3 q = 10, T = 15 q = 10, T = 5 A3 A1 A1 A1 A1 A2 A2 A2 1 element requires 5 time units to be processed 11/23
27 Multi-application Scheduling Online problem: several requests from distinct applications are delivered to the file systems. Wished criteria : Efficiency with fairness constraints Maximize the minimum of instantaneous throughput for each application Algorithm : Multi-Level Feedback variant (quantum approach) Step 2 Processing Waiting A3 A1 A1 A1 A1 A2 A2 A2 A2 A2 A3 Pre processing (offset dependance) A1 A1 A1 A1 A2 A2 A2 A2 A2 A3 q = 20, T = 20 q = 20, T = 25 q = 10, T = 5 Scheduling A1 A1 A1 A1 A2 A2 A2 A2 A2 A3 1 element requires 5 time units to be processed 11/23
28 Multi-application Scheduling Online problem: several requests from distinct applications are delivered to the file systems. Wished criteria : Efficiency with fairness constraints Maximize the minimum of instantaneous throughput for each application Algorithm : Multi-Level Feedback variant (quantum approach) Step 3 Processing Waiting A1 A1 A1 A1 A2 A2 A2 A2 A2 A3 A3 A2 A3 A3 Pre processing (offset dependance) A2 A2 A2 A2 A2 A2 A3 A3 A3 A3 q = 40, T = 30 q = 20, T = 20 Scheduling A2 A2 A2 A2 A2 A2 A3 A3 A3 A3 1 element requires 5 time units to be processed 11/23
29 Multi-application Scheduling Online problem: several requests from distinct applications are delivered to the file systems. Wished criteria : Efficiency with fairness constraints Maximize the minimum of instantaneous throughput for each application Algorithm : Multi-Level Feedback variant (quantum approach) The grow of a quantum could be set for each application (QoS) Step 3 Processing Waiting A1 A1 A1 A1 A2 A2 A2 A2 A2 A3 A3 A2 A3 A3 Pre processing (offset dependance) A2 A2 A2 A2 A2 A2 A3 A3 A3 A3 q = 40, T = 30 q = 20, T = 20 Scheduling A2 A2 A2 A2 A2 A2 A3 A3 A3 A3 1 element requires 5 time units to be processed 11/23
30 Manage Synchronous Behaviour in an Efficient Way I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)?!!!! Appli 1 Appli 2 To switch from one file to the other implies a disk head movement Serialize and define dedicated windows File 1 File 2 12/23
31 Manage Synchronous Behaviour in an Efficient Way I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)? Coordination server Appli 1 Appli 2 Serialize and define dedicated windows File 1 File 2 SEEK? 12/23
32 Manage Synchronous Behaviour in an Efficient Way I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)? Coordination server Appli 1 Appli 2 Serialize and define dedicated windows File 1 File 2 12/23
33 Manage Synchronous Behaviour in an Efficient Way I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)? Coordination server Appli 1 Appli 2 Serialize and define dedicated windows File 1 File 2 12/23
34 Manage Synchronous Behaviour in an Efficient Way I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)? Coordination server Appli 1 Appli 2 Serialize and define dedicated windows File 1 File 2 12/23
35 Manage Synchronous Behaviour in an Efficient Way I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)?! Coordination server Appli 1 Appli 2 Serialize and define dedicated windows File 1 File 2 SEEK 12/23
36 Manage Synchronous Behaviour in an Efficient Way I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)?! Coordination server Appli 1 Appli 2 Serialize and define dedicated windows File 1 File 2 12/23
37 Manage Synchronous Behaviour in an Efficient Way I/O server Synchronous behaviour (2 concurrent applications execute a cat like operation)?! Coordination server Appli 1 Appli 2 Serialize and define dedicated windows File 1 File 2 Decrease the number of seeks and exploit read-ahead mechanism 12/23
38 Plan Part 1 - Parallel Input/Output and Clusters Part 2 - Controlling and Scheduling Multi-application I/O Part 3 - aioli, an Input/Output Scheduler for HPC 6 aioli - Generic Framework 7 aioli - Evaluations Evaluations - Multi-nodes Eval - 2 applications Evaluations - 10 applications Part 4 - Conclusion 13/23
39 aioli - Generic Framework SMP 1 SMP 2 I/O server 1 Compute Network Storage nodes nodes interconnection SMP 3 I/O server 2 SMP 4 I/O server p Objectives SMP n A central point is required to apply our global strategy : model Client/Server like PANDA (explicit centralization) is not scalable Exploit available central point Major change : I/O systems become clients of our framework! 14/23
40 aioli - Framework Implementation Virtual File System (server side) syscalls layer userspace kernelspace NFS server XDR RPC Network layer aioli VFS Client Virtual File System ExtX ReiserFS aioli Optimize the VFS on server side 15/23
41 aioli - Framework Implementation Virtual File System (server side) NFS server (Version 3) NFS server syscalls layer aioli NFS client aioli userspace kernelspace XDR RPC Network layer Virtual File System ExtX ReiserFS Optimize the NFS server 15/23
42 aioli - Framework Implementation Virtual File System (server side) NFS server (Version 3) Technical aspects Linux kernel module - 3 functions to plug an I/O system Schedulers Pool Sched. Instance Sched. Instance Sched. Instance Request Queue Management Statistics Control I/O Controller aioli System I/O Controller I/O Controller aioli Client 1 (Virtual File System)... I/O Systems aioli Client n (Network File System) Block Device Block Device Network 15/23
43 aioli - Evaluations Platform Grid5000 : Sophia-Antipolis cluster (AMD 64, IDE HDD, Giga Ethernet) 1 to 96 nodes 1 dedicated NFS server IOR benchmark (LLNL) Experiments 1 parallel application : Valid parallel I/O detection and transparent optimisations 2 applications : Analyse mutual impact and interest of a global strategy 10 applications : Evaluate a real case 16/23
44 Evaluations : One Parallel Application 4 GB File decomposition including 32 MPI instances kernel , sophia cluster, NFS version 3, mpich 1.2.5, ROMIO POSIX 1 POSIX 32 Observations Time! Completion time (sec) File access granularity (KBytes) 32 processes deployed on 32 nodes Mono-application OK! next step: global coordination 17/23
45 Evaluations : One Parallel Application 4 GB File decomposition including 32 MPI instances kernel , sophia cluster, NFS version 3, mpich 1.2.5, ROMIO POSIX 1 POSIX 32 MPI IO 32 Observations Time! Completion time (sec) File access granularity (KBytes) 32 processes deployed on 32 nodes Mono-application OK! next step: global coordination 17/23
46 Evaluations : One Parallel Application 4 GB File decomposition including 32 MPI instances kernel , sophia cluster, NFS version 3, mpich 1.2.5, ROMIO Completion time (sec) POSIX 1 POSIX 32 MPI IO 32 POSIX + aioli 32 Observations Time! aioli provides significant improvements 11 < T P osix T aioli < < T MP IIO T aioli < File access granularity (KBytes) 32 processes deployed on 32 nodes Mono-application OK! next step: global coordination 17/23
47 Evaluations : One Parallel Application 4 GB File decomposition including 32 MPI instances kernel , sophia cluster, NFS version 3, mpich 1.2.5, ROMIO POSIX + aioli 32 POSIX 1 POSIX 32 MPI IO 32 Observations Time! Bandwidth (MB/s) aioli provides significant improvements 11 < T P osix T aioli < < T MP IIO T aioli < File access granularity (KBytes) 32 processes deployed on 32 nodes Mono-application OK! next step: global coordination 17/23
48 Bandwidth (MB/s) Evaluations : One Parallel Application 4 GB File decomposition including 32 MPI instances kernel , sophia cluster, NFS version 3, mpich 1.2.5, ROMIO POSIX 1 POSIX + aioli File access granularity (KBytes) 1 process ( prefetch ) Mono-application OK! next step: global coordination Observations Time! aioli provides significant improvements 11 < T P osix T aioli < 50 T MP IIO 3.5 < < 6.5 T aioli Synchronous behaviours benefit from aioli 17/23
49 Evaluations : One Parallel Application 4 GB File decomposition including 32 MPI instances kernel , sophia cluster, NFS version 3, mpich 1.2.5, ROMIO Bandwidth (MB/s) POSIX + aioli 1 POSIX + aioli 32 POSIX 1 POSIX 32 MPI IO File access granularity (KBytes) Observations Time! aioli provides significant improvements 11 < T P osix T aioli < 50 T MP IIO 3.5 < < 6.5 T aioli Synchronous behaviours benefit from aioli Mono-application OK! next step: global coordination 17/23
50 Evaluations : Multi-application Mode - Case 1 Impact of a 4 GB decomposition (32 processes - 32 nodes) over a cat of 16 MB kernel , sophia cluster, NFS version 3, mpich 1.2.5, ROMIO IOR NFS cat NFS IOR NFS cat NFS Completion time (sec) IOR file access granularity (KBytes) cat granularity 32KBytes IOR file access granularity (KBytes) "cat" granularity 32KBytes read - POSIX read - MPI I/O Observations Impact : 16MB in 100 sec (Y axis : log scale), The use of MPI I/O reduces the load on the server /23
51 Evaluations : Multi-application Mode - Case 1 Impact of a 4 GB decomposition (32 processes - 32 nodes) over a cat of 16 MB kernel , sophia cluster, NFS version 3, mpich 1.2.5, ROMIO IOR NFS cat NFS IOR NFS cat NFS Completion time (sec) IOR aioli cat aioli IOR file access granularity (KBytes) cat granularity 32KBytes IOR file access granularity (KBytes) "cat" granularity 32KBytes read - POSIX read - MPI I/O Observations Impact : 16MB in 100 sec (Y axis : log scale), The use of MPI I/O reduces the load on the server aioli improves the performances for both applications IOR aioli cat aioli /23
52 Evaluations : Multi-application Mode - Case 2 10 concurrent applications, 96 nodes, 6 GB kernel , sophia cluster, NFS version 3, mpich 1.2.5, ROMIO Completion time Application details POSIX MPI IO 4 decompos. 6 sequential. 6 GB Time are given in seconds. 19/23
53 Evaluations : Multi-application Mode - Case 2 10 concurrent applications, 96 nodes, 6 GB kernel , sophia cluster, NFS version 3, mpich 1.2.5, ROMIO Application details Completion time NFS POSIX MPI IO read decompos. - 2GB (32 nodes, granularity=128kb) write decompos. - 2GB (32 nodes, granularity=128kb) read decompos MB (16 nodes, granularity=8kb) write decompos MB (8 nodes, granularity=64kb) read sequential. - 1GB (1 node, granularity=2mb) write sequential MB (1 node, granularity=2mb) read sequential. - 32MB (1 node, granularity=4kb) write sequential. - 32MB (1 node, granularity=4kb) read sequential. - 4MB (1 node, granularity=32kb) write sequential. - 4MB (1 node, granularity=32kb) 39 2 Time are given in seconds. 19/23
54 Evaluations : Multi-application Mode - Case 2 10 concurrent applications, 96 nodes, 6 GB kernel , sophia cluster, NFS version 3, mpich 1.2.5, ROMIO Application details Completion time NFS POSIX MPI IO read decompos. - 2GB (32 nodes, granularity=128kb) write decompos. - 2GB (32 nodes, granularity=128kb) read decompos MB (16 nodes, granularity=8kb) write decompos MB (8 nodes, granularity=64kb) read sequential. - 1GB (1 node, granularity=2mb) write sequential MB (1 node, granularity=2mb) read sequential. - 32MB (1 node, granularity=4kb) write sequential. - 32MB (1 node, granularity=4kb) read sequential. - 4MB (1 node, granularity=32kb) write sequential. - 4MB (1 node, granularity=32kb) 39 2 Time are given in seconds. 19/23
55 Evaluations : Multi-application Mode - Case 2 10 concurrent applications, 96 nodes, 6 GB kernel , sophia cluster, NFS version 3, mpich 1.2.5, ROMIO Completion time Application details NFS NFS+aIOLi POSIX MPI IO POSIX MPI IO 4 decompos. 6 sequential. 6 GB Time are given in seconds. 19/23
56 Evaluations : Multi-application Mode - Case 2 10 concurrent applications, 96 nodes, 6 GB kernel , sophia cluster, NFS version 3, mpich 1.2.5, ROMIO Completion time Applications NFS NFS+aIOLi POSIX MPI IO POSIX MPI IO read decompos. - 2GB (32 nodes, granularity=128kb) write decompos. - 2GB (32 nodes, granularity=128kb) read decompos MB (16 nodes, granularity=8kb) write decompos MB (8 nodes, granularity=64kb) read sequential - 1GB (1 node, granularity=2mb) write sequential MB (1 node, granularity=2mb) read sequential. - 32MB (1 node, granularity=4kb) write sequential. - 32MB (1 node, granularity=4kb) read sequential. - 4MB (1 node, granularity=32kb) write sequential. - 4MB (1 node, granularity=32kb) Time are given in seconds. 19/23
57 Evaluations : Multi-application Mode - Case 2 10 concurrent applications, 96 nodes, 6 GB kernel , sophia cluster, NFS version 3, mpich 1.2.5, ROMIO Completion time Applications NFS NFS+aIOLi POSIX MPI IO POSIX MPI IO read decompos. - 2GB (32 nodes, granularity=128kb) write decompos. - 2GB (32 nodes, granularity=128kb) read decompos MB (16 nodes, granularity=8kb) write decompos MB (8 nodes, granularity=64kb) read sequential - 1GB (1 node, granularity=2mb) write sequential MB (1 node, granularity=2mb) read sequential. - 32MB (1 node, granularity=4kb) write sequential. - 32MB (1 node, granularity=4kb) read sequential. - 4MB (1 node, granularity=32kb) write sequential. - 4MB (1 node, granularity=32kb) Time are given in seconds. 19/23
58 Plan Part 1 - Parallel Input/Output and Clusters Part 2 - Controlling and Scheduling Multi-application I/O Part 3 - aioli, an Input/Output Scheduler for HPC Part 4 - Conclusion 8 Conclusion 9 Current and Future Works 20/23
59 Conclusion Performances and I/O in multi-application environment Control and schedule I/O requests in a global way : multi-criteria problem : efficiency and fairness MLF variant proposal aioli, an I/O scheduler for HPC Generic framework to evaluate new strategies for I/O scheduling Implementation in kernel space : intrusive from system point of view but efficient Code available under GPL Joint project since the end of 2005 with ICIS University (Poland) 21/23
60 Current and Future Works aioli, works in progress Future Take into account data stripping (on RAID devices and Parallel File Systems) Collaboration around a parallel version of NFS to evaluate the interest of an higher I/ scheduler (Brasil/France/Poland) Interconnexion with Lustre File system Control I/O requests at different points : multi-level scheduler on client side (compute node) / on server and hard drive side cascade scheduling Exploiting the meta-node concept used by modern FS to provide consistency as a central point to plug aioli 22/23
61 Questions? LIPS Project BULL - INRIA - ID-IMAG Laboratory - ICIS Institute Thanks 23/23
Overlapping Data Transfer With Application Execution on Clusters
Overlapping Data Transfer With Application Execution on Clusters Karen L. Reid and Michael Stumm reid@cs.toronto.edu stumm@eecg.toronto.edu Department of Computer Science Department of Electrical and Computer
More informationOperating System for the K computer
Operating System for the K computer Jun Moroo Masahiko Yamada Takeharu Kato For the K computer to achieve the world s highest performance, Fujitsu has worked on the following three performance improvements
More informationAn Alternative Storage Solution for MapReduce. Eric Lomascolo Director, Solutions Marketing
An Alternative Storage Solution for MapReduce Eric Lomascolo Director, Solutions Marketing MapReduce Breaks the Problem Down Data Analysis Distributes processing work (Map) across compute nodes and accumulates
More informationBlobSeer: Towards efficient data storage management on large-scale, distributed systems
: Towards efficient data storage management on large-scale, distributed systems Bogdan Nicolae University of Rennes 1, France KerData Team, INRIA Rennes Bretagne-Atlantique PhD Advisors: Gabriel Antoniu
More informationSun Storage Perspective & Lustre Architecture. Dr. Peter Braam VP Sun Microsystems
Sun Storage Perspective & Lustre Architecture Dr. Peter Braam VP Sun Microsystems Agenda Future of Storage Sun s vision Lustre - vendor neutral architecture roadmap Sun s view on storage introduction The
More informationWrite a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical
Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or
More informationMOSIX: High performance Linux farm
MOSIX: High performance Linux farm Paolo Mastroserio [mastroserio@na.infn.it] Francesco Maria Taurino [taurino@na.infn.it] Gennaro Tortone [tortone@na.infn.it] Napoli Index overview on Linux farm farm
More informationPetascale Software Challenges. Piyush Chaudhary piyushc@us.ibm.com High Performance Computing
Petascale Software Challenges Piyush Chaudhary piyushc@us.ibm.com High Performance Computing Fundamental Observations Applications are struggling to realize growth in sustained performance at scale Reasons
More informationTHE EXPAND PARALLEL FILE SYSTEM A FILE SYSTEM FOR CLUSTER AND GRID COMPUTING. José Daniel García Sánchez ARCOS Group University Carlos III of Madrid
THE EXPAND PARALLEL FILE SYSTEM A FILE SYSTEM FOR CLUSTER AND GRID COMPUTING José Daniel García Sánchez ARCOS Group University Carlos III of Madrid Contents 2 The ARCOS Group. Expand motivation. Expand
More informationUsing Linux Clusters as VoD Servers
HAC LUCE Using Linux Clusters as VoD Servers Víctor M. Guĺıas Fernández gulias@lfcia.org Computer Science Department University of A Corunha funded by: Outline Background: The Borg Cluster Video on Demand.
More informationGlobus Striped GridFTP Framework and Server. Raj Kettimuthu, ANL and U. Chicago
Globus Striped GridFTP Framework and Server Raj Kettimuthu, ANL and U. Chicago Outline Introduction Features Motivation Architecture Globus XIO Experimental Results 3 August 2005 The Ohio State University
More information- An Essential Building Block for Stable and Reliable Compute Clusters
Ferdinand Geier ParTec Cluster Competence Center GmbH, V. 1.4, March 2005 Cluster Middleware - An Essential Building Block for Stable and Reliable Compute Clusters Contents: Compute Clusters a Real Alternative
More informationDSS. Diskpool and cloud storage benchmarks used in IT-DSS. Data & Storage Services. Geoffray ADDE
DSS Data & Diskpool and cloud storage benchmarks used in IT-DSS CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/it Geoffray ADDE DSS Outline I- A rational approach to storage systems evaluation
More informationDistributed RAID Architectures for Cluster I/O Computing. Kai Hwang
Distributed RAID Architectures for Cluster I/O Computing Kai Hwang Internet and Cluster Computing Lab. University of Southern California 1 Presentation Outline : Scalable Cluster I/O The RAID-x Architecture
More informationCOSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters
COSC 6374 Parallel I/O (I) I/O basics Fall 2012 Concept of a clusters Processor 1 local disks Compute node message passing network administrative network Memory Processor 2 Network card 1 Network card
More informationCray DVS: Data Virtualization Service
Cray : Data Virtualization Service Stephen Sugiyama and David Wallace, Cray Inc. ABSTRACT: Cray, the Cray Data Virtualization Service, is a new capability being added to the XT software environment with
More informationCOSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters
COSC 6374 Parallel Computation Parallel I/O (I) I/O basics Spring 2008 Concept of a clusters Processor 1 local disks Compute node message passing network administrative network Memory Processor 2 Network
More informationCSAR: Cluster Storage with Adaptive Redundancy
CSAR: Cluster Storage with Adaptive Redundancy Manoj Pillai, Mario Lauria Department of Computer and Information Science The Ohio State University Columbus, OH, 4321 Email: pillai,lauria@cis.ohio-state.edu
More informationPOWER ALL GLOBAL FILE SYSTEM (PGFS)
POWER ALL GLOBAL FILE SYSTEM (PGFS) Defining next generation of global storage grid Power All Networks Ltd. Technical Whitepaper April 2008, version 1.01 Table of Content 1. Introduction.. 3 2. Paradigm
More informationRunning a Workflow on a PowerCenter Grid
Running a Workflow on a PowerCenter Grid 2010-2014 Informatica Corporation. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise)
More informationNew Storage System Solutions
New Storage System Solutions Craig Prescott Research Computing May 2, 2013 Outline } Existing storage systems } Requirements and Solutions } Lustre } /scratch/lfs } Questions? Existing Storage Systems
More informationCan High-Performance Interconnects Benefit Memcached and Hadoop?
Can High-Performance Interconnects Benefit Memcached and Hadoop? D. K. Panda and Sayantan Sur Network-Based Computing Laboratory Department of Computer Science and Engineering The Ohio State University,
More informationBottleneck Detection in Parallel File Systems with Trace-Based Performance Monitoring
Julian M. Kunkel - Euro-Par 2008 1/33 Bottleneck Detection in Parallel File Systems with Trace-Based Performance Monitoring Julian M. Kunkel Thomas Ludwig Institute for Computer Science Parallel and Distributed
More informationCERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT
SS Data & Storage CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT HEPiX Fall 2012 Workshop October 15-19, 2012 Institute of High Energy Physics, Beijing, China SS Outline
More informationWhy Compromise? A discussion on RDMA versus Send/Receive and the difference between interconnect and application semantics
Why Compromise? A discussion on RDMA versus Send/Receive and the difference between interconnect and application semantics Mellanox Technologies Inc. 2900 Stender Way, Santa Clara, CA 95054 Tel: 408-970-3400
More informationOracle Database Scalability in VMware ESX VMware ESX 3.5
Performance Study Oracle Database Scalability in VMware ESX VMware ESX 3.5 Database applications running on individual physical servers represent a large consolidation opportunity. However enterprises
More informationPerformance Monitoring of Parallel Scientific Applications
Performance Monitoring of Parallel Scientific Applications Abstract. David Skinner National Energy Research Scientific Computing Center Lawrence Berkeley National Laboratory This paper introduces an infrastructure
More informationViolin: A Framework for Extensible Block-level Storage
Violin: A Framework for Extensible Block-level Storage Michail Flouris Dept. of Computer Science, University of Toronto, Canada flouris@cs.toronto.edu Angelos Bilas ICS-FORTH & University of Crete, Greece
More informationAdvanced Techniques with Newton. Gerald Ragghianti Advanced Newton workshop Sept. 22, 2011
Advanced Techniques with Newton Gerald Ragghianti Advanced Newton workshop Sept. 22, 2011 Workshop Goals Gain independence Executing your work Finding Information Fixing Problems Optimizing Effectiveness
More informationE) Modeling Insights: Patterns and Anti-patterns
Murray Woodside, July 2002 Techniques for Deriving Performance Models from Software Designs Murray Woodside Second Part Outline ) Conceptual framework and scenarios ) Layered systems and models C) uilding
More informationAvid ISIS 7000. www.avid.com
Avid ISIS 7000 www.avid.com Table of Contents Overview... 3 Avid ISIS Technology Overview... 6 ISIS Storage Blade... 6 ISIS Switch Blade... 7 ISIS System Director... 7 ISIS Client Software... 8 ISIS Redundant
More informationCloud Computing through Virtualization and HPC technologies
Cloud Computing through Virtualization and HPC technologies William Lu, Ph.D. 1 Agenda Cloud Computing & HPC A Case of HPC Implementation Application Performance in VM Summary 2 Cloud Computing & HPC HPC
More informationDistributed File Systems
Distributed File Systems Paul Krzyzanowski Rutgers University October 28, 2012 1 Introduction The classic network file systems we examined, NFS, CIFS, AFS, Coda, were designed as client-server applications.
More informationParallel Programming Survey
Christian Terboven 02.09.2014 / Aachen, Germany Stand: 26.08.2014 Version 2.3 IT Center der RWTH Aachen University Agenda Overview: Processor Microarchitecture Shared-Memory
More informationDavid Rioja Redondo Telecommunication Engineer Englobe Technologies and Systems
David Rioja Redondo Telecommunication Engineer Englobe Technologies and Systems About me David Rioja Redondo Telecommunication Engineer - Universidad de Alcalá >2 years building and managing clusters UPM
More informationSystem Software for High Performance Computing. Joe Izraelevitz
System Software for High Performance Computing Joe Izraelevitz Agenda Overview of Supercomputers Blue Gene/Q System LoadLeveler Job Scheduler General Parallel File System HPC at UR What is a Supercomputer?
More informationVers des mécanismes génériques de communication et une meilleure maîtrise des affinités dans les grappes de calculateurs hiérarchiques.
Vers des mécanismes génériques de communication et une meilleure maîtrise des affinités dans les grappes de calculateurs hiérarchiques Brice Goglin 15 avril 2014 Towards generic Communication Mechanisms
More informationAgenda. HPC Software Stack. HPC Post-Processing Visualization. Case Study National Scientific Center. European HPC Benchmark Center Montpellier PSSC
HPC Architecture End to End Alexandre Chauvin Agenda HPC Software Stack Visualization National Scientific Center 2 Agenda HPC Software Stack Alexandre Chauvin Typical HPC Software Stack Externes LAN Typical
More informationThe High Performance Internet of Things: using GVirtuS for gluing cloud computing and ubiquitous connected devices
WS on Models, Algorithms and Methodologies for Hierarchical Parallelism in new HPC Systems The High Performance Internet of Things: using GVirtuS for gluing cloud computing and ubiquitous connected devices
More informationCommoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre
Commoditisation of the High-End Research Storage Market with the Dell MD3460 & Intel Enterprise Edition Lustre University of Cambridge, UIS, HPC Service Authors: Wojciech Turek, Paul Calleja, John Taylor
More informationUsing Linux Clusters as VoD Servers
HAC LUCE Using Linux Clusters as VoD Servers Víctor M. Guĺıas Fernández gulias@lfcia.org Computer Science Department University of A Corunha funded by: Outline Background: The Borg Cluster Video on Demand.
More informationBeyond Embarrassingly Parallel Big Data. William Gropp www.cs.illinois.edu/~wgropp
Beyond Embarrassingly Parallel Big Data William Gropp www.cs.illinois.edu/~wgropp Messages Big is big Data driven is an important area, but not all data driven problems are big data (despite current hype).
More informationA Comparison on Current Distributed File Systems for Beowulf Clusters
A Comparison on Current Distributed File Systems for Beowulf Clusters Rafael Bohrer Ávila 1 Philippe Olivier Alexandre Navaux 2 Yves Denneulin 3 Abstract This paper presents a comparison on current file
More informationHow To Install Linux Titan
Linux Titan Distribution Presented By: Adham Helal Amgad Madkour Ayman El Sayed Emad Zakaria What Is a Linux Distribution? What is a Linux Distribution? The distribution contains groups of packages and
More informationTechnological Overview of High-Performance Computing. Gerolf Ziegenhain - TU Kaiserslautern, Germany
Technological Overview of High-Performance Computing Gerolf Ziegenhain - TU Kaiserslautern, Germany Outline of This Talk Give a glance at the important technologies The most important stuff is mentioned
More informationNetwork Attached Storage. Jinfeng Yang Oct/19/2015
Network Attached Storage Jinfeng Yang Oct/19/2015 Outline Part A 1. What is the Network Attached Storage (NAS)? 2. What are the applications of NAS? 3. The benefits of NAS. 4. NAS s performance (Reliability
More informationLustre * Filesystem for Cloud and Hadoop *
OpenFabrics Software User Group Workshop Lustre * Filesystem for Cloud and Hadoop * Robert Read, Intel Lustre * for Cloud and Hadoop * Brief Lustre History and Overview Using Lustre with Hadoop Intel Cloud
More informationPortable, Scalable, and High-Performance I/O Forwarding on Massively Parallel Systems. Jason Cope copej@mcs.anl.gov
Portable, Scalable, and High-Performance I/O Forwarding on Massively Parallel Systems Jason Cope copej@mcs.anl.gov Computation and I/O Performance Imbalance Leadership class computa:onal scale: >100,000
More informationHow To Monitor Infiniband Network Data From A Network On A Leaf Switch (Wired) On A Microsoft Powerbook (Wired Or Microsoft) On An Ipa (Wired/Wired) Or Ipa V2 (Wired V2)
INFINIBAND NETWORK ANALYSIS AND MONITORING USING OPENSM N. Dandapanthula 1, H. Subramoni 1, J. Vienne 1, K. Kandalla 1, S. Sur 1, D. K. Panda 1, and R. Brightwell 2 Presented By Xavier Besseron 1 Date:
More informationEnabling High performance Big Data platform with RDMA
Enabling High performance Big Data platform with RDMA Tong Liu HPC Advisory Council Oct 7 th, 2014 Shortcomings of Hadoop Administration tooling Performance Reliability SQL support Backup and recovery
More informationLS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance
11 th International LS-DYNA Users Conference Session # LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance Gilad Shainer 1, Tong Liu 2, Jeff Layton 3, Onur Celebioglu
More informationCluster Implementation and Management; Scheduling
Cluster Implementation and Management; Scheduling CPS343 Parallel and High Performance Computing Spring 2013 CPS343 (Parallel and HPC) Cluster Implementation and Management; Scheduling Spring 2013 1 /
More informationImplementing MPI-IO Shared File Pointers without File System Support
Implementing MPI-IO Shared File Pointers without File System Support Robert Latham, Robert Ross, Rajeev Thakur, Brian Toonen Mathematics and Computer Science Division Argonne National Laboratory Argonne,
More informationHow To Visualize Performance Data In A Computer Program
Performance Visualization Tools 1 Performance Visualization Tools Lecture Outline : Following Topics will be discussed Characteristics of Performance Visualization technique Commercial and Public Domain
More informationRodrigo Fernandes de Mello, Evgueni Dodonov, José Augusto Andrade Filho
Middleware for High Performance Computing Rodrigo Fernandes de Mello, Evgueni Dodonov, José Augusto Andrade Filho University of São Paulo São Carlos, Brazil {mello, eugeni, augustoa}@icmc.usp.br Outline
More informationRedpaper. Performance Metrics in TotalStorage Productivity Center Performance Reports. Introduction. Mary Lovelace
Redpaper Mary Lovelace Performance Metrics in TotalStorage Productivity Center Performance Reports Introduction This Redpaper contains the TotalStorage Productivity Center performance metrics that are
More informationIntel DPDK Boosts Server Appliance Performance White Paper
Intel DPDK Boosts Server Appliance Performance Intel DPDK Boosts Server Appliance Performance Introduction As network speeds increase to 40G and above, both in the enterprise and data center, the bottlenecks
More informationChronicle: Capture and Analysis of NFS Workloads at Line Rate
Chronicle: Capture and Analysis of NFS Workloads at Line Rate Ardalan Kangarlou, Sandip Shete, and John Strunk Advanced Technology Group 1 Motivation Goal: To gather insights from customer workloads via
More informationSockets vs. RDMA Interface over 10-Gigabit Networks: An In-depth Analysis of the Memory Traffic Bottleneck
Sockets vs. RDMA Interface over 1-Gigabit Networks: An In-depth Analysis of the Memory Traffic Bottleneck Pavan Balaji Hemal V. Shah D. K. Panda Network Based Computing Lab Computer Science and Engineering
More informationUse of Hadoop File System for Nuclear Physics Analyses in STAR
1 Use of Hadoop File System for Nuclear Physics Analyses in STAR EVAN SANGALINE UC DAVIS Motivations 2 Data storage a key component of analysis requirements Transmission and storage across diverse resources
More informationGPFS Storage Server. Concepts and Setup in Lemanicus BG/Q system" Christian Clémençon (EPFL-DIT)" " 4 April 2013"
GPFS Storage Server Concepts and Setup in Lemanicus BG/Q system" Christian Clémençon (EPFL-DIT)" " Agenda" GPFS Overview" Classical versus GSS I/O Solution" GPFS Storage Server (GSS)" GPFS Native RAID
More informationGraySort on Apache Spark by Databricks
GraySort on Apache Spark by Databricks Reynold Xin, Parviz Deyhim, Ali Ghodsi, Xiangrui Meng, Matei Zaharia Databricks Inc. Apache Spark Sorting in Spark Overview Sorting Within a Partition Range Partitioner
More informationCeph. A file system a little bit different. Udo Seidel
Ceph A file system a little bit different Udo Seidel Ceph what? So-called parallel distributed cluster file system Started as part of PhD studies at UCSC Public announcement in 2006 at 7 th OSDI File system
More informationInfrastructure for Load Balancing on Mosix Cluster
Infrastructure for Load Balancing on Mosix Cluster MadhuSudhan Reddy Tera and Sadanand Kota Computing and Information Science, Kansas State University Under the Guidance of Dr. Daniel Andresen. Abstract
More informationDistributed Data Management
Introduction Distributed Data Management Involves the distribution of data and work among more than one machine in the network. Distributed computing is more broad than canonical client/server, in that
More informationLast Class: OS and Computer Architecture. Last Class: OS and Computer Architecture
Last Class: OS and Computer Architecture System bus Network card CPU, memory, I/O devices, network card, system bus Lecture 3, page 1 Last Class: OS and Computer Architecture OS Service Protection Interrupts
More informationConfiguration Maximums VMware Infrastructure 3
Technical Note Configuration s VMware Infrastructure 3 When you are selecting and configuring your virtual and physical equipment, you must stay at or below the maximums supported by VMware Infrastructure
More informationPCI Express High Speed Networks. Complete Solution for High Speed Networking
PCI Express High Speed Networks Complete Solution for High Speed Networking Ultra Low Latency Ultra High Throughput Maximizing application performance is a combination of processing, communication, and
More informationUnstructured Data Accelerator (UDA) Author: Motti Beck, Mellanox Technologies Date: March 27, 2012
Unstructured Data Accelerator (UDA) Author: Motti Beck, Mellanox Technologies Date: March 27, 2012 1 Market Trends Big Data Growing technology deployments are creating an exponential increase in the volume
More informationHigh Performance Computing. Course Notes 2007-2008. HPC Fundamentals
High Performance Computing Course Notes 2007-2008 2008 HPC Fundamentals Introduction What is High Performance Computing (HPC)? Difficult to define - it s a moving target. Later 1980s, a supercomputer performs
More informationSystem Models for Distributed and Cloud Computing
System Models for Distributed and Cloud Computing Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF Classification of Distributed Computing Systems
More informationLustre Networking BY PETER J. BRAAM
Lustre Networking BY PETER J. BRAAM A WHITE PAPER FROM CLUSTER FILE SYSTEMS, INC. APRIL 2007 Audience Architects of HPC clusters Abstract This paper provides architects of HPC clusters with information
More informationNetwork Infrastructure Services CS848 Project
Quality of Service Guarantees for Cloud Services CS848 Project presentation by Alexey Karyakin David R. Cheriton School of Computer Science University of Waterloo March 2010 Outline 1. Performance of cloud
More informationQuantum StorNext. Product Brief: Distributed LAN Client
Quantum StorNext Product Brief: Distributed LAN Client NOTICE This product brief may contain proprietary information protected by copyright. Information in this product brief is subject to change without
More informationThe Assessment of Benchmarks Executed on Bare-Metal and Using Para-Virtualisation
The Assessment of Benchmarks Executed on Bare-Metal and Using Para-Virtualisation Mark Baker, Garry Smith and Ahmad Hasaan SSE, University of Reading Paravirtualization A full assessment of paravirtualization
More informationMaking Multicore Work and Measuring its Benefits. Markus Levy, president EEMBC and Multicore Association
Making Multicore Work and Measuring its Benefits Markus Levy, president EEMBC and Multicore Association Agenda Why Multicore? Standards and issues in the multicore community What is Multicore Association?
More informationInfoScale Storage & Media Server Workloads
InfoScale Storage & Media Server Workloads Maximise Performance when Storing and Retrieving Large Amounts of Unstructured Data Carlos Carrero Colin Eldridge Shrinivas Chandukar 1 Table of Contents 01 Introduction
More informationCHAPTER 2 MODELLING FOR DISTRIBUTED NETWORK SYSTEMS: THE CLIENT- SERVER MODEL
CHAPTER 2 MODELLING FOR DISTRIBUTED NETWORK SYSTEMS: THE CLIENT- SERVER MODEL This chapter is to introduce the client-server model and its role in the development of distributed network systems. The chapter
More informationWindows 8 SMB 2.2 File Sharing Performance
Windows 8 SMB 2.2 File Sharing Performance Abstract This paper provides a preliminary analysis of the performance capabilities of the Server Message Block (SMB) 2.2 file sharing protocol with 10 gigabit
More informationDepartment of Computer Sciences University of Salzburg. HPC In The Cloud? Seminar aus Informatik SS 2011/2012. July 16, 2012
Department of Computer Sciences University of Salzburg HPC In The Cloud? Seminar aus Informatik SS 2011/2012 July 16, 2012 Michael Kleber, mkleber@cosy.sbg.ac.at Contents 1 Introduction...................................
More informationEOFS Workshop Paris Sept, 2011. Lustre at exascale. Eric Barton. CTO Whamcloud, Inc. eeb@whamcloud.com. 2011 Whamcloud, Inc.
EOFS Workshop Paris Sept, 2011 Lustre at exascale Eric Barton CTO Whamcloud, Inc. eeb@whamcloud.com Agenda Forces at work in exascale I/O Technology drivers I/O requirements Software engineering issues
More informationSRNWP Workshop. HP Solutions and Activities in Climate & Weather Research. Michael Riedmann European Performance Center
SRNWP Workshop HP Solutions and Activities in Climate & Weather Research Michael Riedmann European Performance Center Agenda A bit of marketing: HP Solutions for HPC A few words about recent Met deals
More informationFile System & Device Drive. Overview of Mass Storage Structure. Moving head Disk Mechanism. HDD Pictures 11/13/2014. CS341: Operating System
CS341: Operating System Lect 36: 1 st Nov 2014 Dr. A. Sahu Dept of Comp. Sc. & Engg. Indian Institute of Technology Guwahati File System & Device Drive Mass Storage Disk Structure Disk Arm Scheduling RAID
More informationA REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM
A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM Sneha D.Borkar 1, Prof.Chaitali S.Surtakar 2 Student of B.E., Information Technology, J.D.I.E.T, sborkar95@gmail.com Assistant Professor, Information
More informationMixing Hadoop and HPC Workloads on Parallel Filesystems
Mixing Hadoop and HPC Workloads on Parallel Filesystems Esteban Molina-Estolano *, Maya Gokhale, Carlos Maltzahn *, John May, John Bent, Scott Brandt * * UC Santa Cruz, ISSDM, PDSI Lawrence Livermore National
More informationPerformance Analysis of RAIDs in Storage Area Network
Performance Analysis of RAIDs in Storage Area Network Sneha M. Assistant Professor, Department of Computer Science and Engineering, R V College of Engineering Bengaluru-560059 ABSTRACT Direct Attached
More informationAIX NFS Client Performance Improvements for Databases on NAS
AIX NFS Client Performance Improvements for Databases on NAS October 20, 2005 Sanjay Gulabani Sr. Performance Engineer Network Appliance, Inc. gulabani@netapp.com Diane Flemming Advisory Software Engineer
More informationMulticore Parallel Computing with OpenMP
Multicore Parallel Computing with OpenMP Tan Chee Chiang (SVU/Academic Computing, Computer Centre) 1. OpenMP Programming The death of OpenMP was anticipated when cluster systems rapidly replaced large
More informationInternational Journal of Computer & Organization Trends Volume20 Number1 May 2015
Performance Analysis of Various Guest Operating Systems on Ubuntu 14.04 Prof. (Dr.) Viabhakar Pathak 1, Pramod Kumar Ram 2 1 Computer Science and Engineering, Arya College of Engineering, Jaipur, India.
More informationPARALLEL & CLUSTER COMPUTING CS 6260 PROFESSOR: ELISE DE DONCKER BY: LINA HUSSEIN
1 PARALLEL & CLUSTER COMPUTING CS 6260 PROFESSOR: ELISE DE DONCKER BY: LINA HUSSEIN Introduction What is cluster computing? Classification of Cluster Computing Technologies: Beowulf cluster Construction
More informationMaximizing Hadoop Performance and Storage Capacity with AltraHD TM
Maximizing Hadoop Performance and Storage Capacity with AltraHD TM Executive Summary The explosion of internet data, driven in large part by the growth of more and more powerful mobile devices, has created
More informationConfiguration Maximums VMware vsphere 4.0
Topic Configuration s VMware vsphere 4.0 When you select and configure your virtual and physical equipment, you must stay at or below the maximums supported by vsphere 4.0. The limits presented in the
More informationAgenda. Enterprise Application Performance Factors. Current form of Enterprise Applications. Factors to Application Performance.
Agenda Enterprise Performance Factors Overall Enterprise Performance Factors Best Practice for generic Enterprise Best Practice for 3-tiers Enterprise Hardware Load Balancer Basic Unix Tuning Performance
More informationSymmetric Multiprocessing
Multicore Computing A multi-core processor is a processing system composed of two or more independent cores. One can describe it as an integrated circuit to which two or more individual processors (called
More informationThe Hadoop Distributed Filesystem: Balancing Portability and Performance
The Hadoop Distributed Filesystem: Balancing Portability and Performance Jeffrey Shafer, Scott Rixner, and Alan L. Cox Rice University Houston, TX Email: {shafer, rixner, alc}@rice.edu Abstract Hadoop
More informationOptimizing Shared Resource Contention in HPC Clusters
Optimizing Shared Resource Contention in HPC Clusters Sergey Blagodurov Simon Fraser University Alexandra Fedorova Simon Fraser University Abstract Contention for shared resources in HPC clusters occurs
More informationIT of SPIM Data Storage and Compression. EMBO Course - August 27th! Jeff Oegema, Peter Steinbach, Oscar Gonzalez
IT of SPIM Data Storage and Compression EMBO Course - August 27th Jeff Oegema, Peter Steinbach, Oscar Gonzalez 1 Talk Outline Introduction and the IT Team SPIM Data Flow Capture, Compression, and the Data
More informationCluster Grid Interconects. Tony Kay Chief Architect Enterprise Grid and Networking
Cluster Grid Interconects Tony Kay Chief Architect Enterprise Grid and Networking Agenda Cluster Grid Interconnects The Upstart - Infiniband The Empire Strikes Back - Myricom Return of the King 10G Gigabit
More informationHPC and Big Data. EPCC The University of Edinburgh. Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk
HPC and Big Data EPCC The University of Edinburgh Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk EPCC Facilities Technology Transfer European Projects HPC Research Visitor Programmes Training
More informationDesign Issues in a Bare PC Web Server
Design Issues in a Bare PC Web Server Long He, Ramesh K. Karne, Alexander L. Wijesinha, Sandeep Girumala, and Gholam H. Khaksari Department of Computer & Information Sciences, Towson University, 78 York
More information