Chao Chen 1 Michael Lang 2 Yong Chen 1. IEEE BigData, Department of Computer Science Texas Tech University

Size: px

Start display at page:

Download "Chao Chen 1 Michael Lang 2 Yong Chen 1. IEEE BigData, 2013. Department of Computer Science Texas Tech University"

Herbert Hunt
8 years ago
Views:

1 Chao Chen 1 Michael Lang Data-Intensive Scalable Laboratory Department of Computer Science Texas Tech University 2 Los Alamos National Laboratory IEEE BigData, 2013

2 Outline

3 Outline

4 Data Explosion Scientific applications still rely on High (HPC) facilities. These applications are becoming more more data-intensive, generating hundreds of TBs data. Sloan Digital Sky Survey (SDSS) collected 100TB data Arctic Systems Reanalysis (ASR) produced 23.14TB for 20-year simulation Storage capacity requirement of Exascale system is estimated more than 960PB. Poor I/O performance makes it difficult to analyze such large scale of data efficiently.

Sloan Digital Sky Survey (SDSS) collected 100TB data Arctic Systems Reanalysis (ASR) produced 23.

5 Expensive Data Movement I/O will further lag the system performance in exascale time performance (FLOPs) improves by 1024x I/O bwidth improves only by 34x Reducing the data movement is the key to improve I/O system performance.

improves by 1024x I/O bwidth improves only by 34x

6 explores the rest computing resources of storage nodes for analysis codes, reduce the data movements.

7 Limitations of Current Systems Limited computing resources available at storage nodes in practice systems for analysis. CPU usage of each Lustre server node at Hrothagar cluster, Texas Tech University, is as high as 86% percentage. Limited analysis operations (simple, little computing) can be offloaded Complex analysis codes will cause overload of storage nodes, degrade the system performance.

CPU usage of each Lustre server node at Hrothagar cluster, Texas Tech University, is as high as

8 Outline

9 of MAS Two-level active storage services Storage level: where the data located, simple operations, very large data volume Dedicated data nodes: complex operations, large data volume Flexible user interface Support executing user-implemented operations at data nodes level.

data nodes: complex operations, large data volume Flexible user

10 Architecture of MAS

11 Software Stack of MAS MAS API hosts at each compute node providing application interface MAS runtime deployed at data nodes Storage-level leverages existing design.

12 MAS API ed as an extension to stard MPI-IO operation indicates the analysis operation name. op type indicates analysis level: 0 for storage level, 1 for data nodes level Table: API MPI File read ex(mpi File fh, void *buf, int count, MPI datatype type, char * operation, bool op type, MPI Status * status); MPI File write ex(mpi File fh, void *buf, int count, MPI datatype, char * operation, bool op type, MPI Status * status);

ex(mpi File fh, void *buf, int count, MPI datatype type, char * operation, bool op type, MPI Status *

13 MAS Runtime Provides execution environment for analysis codes. Analysis codes implemented separately from application codes. mpicc script is enhanced to compile analysis codes to dynamic library separately Parameters of analysis functions are abstracted by enhanced mpicc script used by MAS runtime for data exchange. PART I: /* Function definition */ void func(mpi File fh, int var1, char *var2, int var3, int var4,..., int *outbuf); PART II: /* Generated configuration file */ func function name MPI File file hle 4 number of arguments int the type of first argument char * the type of second argument int the type of third argument int the type of fourth argument... int * output buf for results returned to compute processes PART III: /* the framework of generated code */ #include <stdlib.h>... int main(argc, argv) {... MPI File fh data; memcopy = (fh data, argv[1],...); int var1 = atoi(argv[2]); char *var2; memcopy(var2, argv[3],...)... int *buf = argv[n]; } func(fh data, var2, var3, var3, var4,..., buf); return 0;

PART I: /* Function definition */ void func(mpi File fh, int var1, char *var2, int var3, int var4,.

14 Outline

15 -I Platform DISCFarm cluster at Texas Tech University. It contains 16 nodes with total of 32 processors 128 cores. Default configuration 2 storage nodes deployed with PVFS2 4 data nodes 8 compute nodes

16 -II Evaluated Operations Summation Lookup Ensemble Kalman Filter (EnKF) Evaluated Schemes (AS) Traditional Storage (TS)

17 Limitation of Current Evaluate the performance of Active Storage with increasing CPU usage of each storage nodes. Figure: Execution time of Summation Observed performance degradation when CPU usage reached 86%, a common case in practical systems.

Figure: Execution time of Summation Observed performance

18 Overall of MAS Observed CPU usage 1.3%. EnKF operation default configuration 30% 40% improvement compared to TS 10% 25% improvement compared to AS.

19 Overall of MAS Figure: comparison of MAS AS against TS Varying CPU usage. AS could perform even worse than TS EnKF operation MAS still achieved 25% improvement default configuration

20 Scalability of MAS Figure: Execution time of EnKF under MAS scheme with fixed data volume While system scaled 2x, the performance increased 1.8x

21 Outline

22 is a promising solution for reducing data movement but can not sufficiently afford complex analyses. is proposed Storage level for simple data-intensive operations Data nodes level for complex data-intensive operations A flexible interface is designed for supporting user-implemented analysis codes. Initial evaluation results are promising Continue investigating active storage, programming model, runtime solutions, to better support data-intensive HPC

23 Thank You For more information, please visit: This research is sponsored in part by the National Science Foundation under the grant CNS

Data Deduplication in a Hybrid Architecture for Improving Write Performance

Data Deduplication in a Hybrid Architecture for Improving Write Performance Data-intensive Salable Computing Laboratory Department of Computer Science Texas Tech University Lubbock, Texas June 10th, 2013