Interfaculty efforts to approach High Performance Computing (HPC) needs. A decade history at Uniandes

Transcription

1 Interfaculty efforts to approach High Performance Computing (HPC) needs. A decade history at Uniandes Alejandro Reyes Ph.D. Assistant Professor Department of Biological Sciences Universidad de los Andes

2 Interfaculty efforts to approach High Performance Computing (HPC) needs. A decade history at Uniandes Harold Castro, Mario Villamizar, Andrés Holguin, Michael Perez, Diego M Riaño, Silvia Restrepo, Alejandro Reyes

3

4 Biology is Computer Sciences next Physics Physics invented internet Tim Berners-Lee, a British scientist at CERN, invented the World Wide Web (WWW) in 1989.

5 Physics was the birth of HPC at Uniandes Before 2005 two small computer clusters, one in Computer Engineering and one in Physics and university wide resources from IT department DSIT School of Sciences School of Engeneering

6 Growing need in physics for more computational power How to better use the computational power within the university for scientist inside and outside? Solution? GRID Cloud computing Cluster computing Dedicated infrastructure usually requires large financial investments

7 EELA E-Infraestructure shared between Europe and Latin America (2006)

8 EELA E-Infraestructure shared between Europe and Latin America (2006) Usuarios G R I D Usuarios Usuarios Uniandes VO EDTEAM VO DTEAM VO EELA VO UNIANDES INTERNET Otras VOs Usuarios VO OPT VO CMS Usuarios

9 EELA E-Infraestructure shared between Europe and Latin America (2006) Eventually leads to project grid-colombia to interconect different universities and regions (2009) Usuarios Uniandes Site Física Site Biología DTI Infraestructura de administración Site Ingeniería Sistemas MOX Site DTI VO UNIANDES

10 HPC and Biological Sciences? We all know Moore s law

11 HPC and Biological Sciences? 10000$ 1000$ 100$ Hiseq' 2000/2500' Hiseq'X' Hiseq2500'RR' NextSeq'500' 10$ Proton' 1$ SOLiD' MiSeq' GS'FLX' ABI$3730xl$ Roche/454$GS$ 0.1$ 0.01$ GA'II' PGM' GS'Junior' PacBio'RS' Illumina$GA$ Series4$ SOLiD$ Illumina$MiSeq$ Ion$Torrent$PGM$ 0.001$ PacBio$RS$ 454$GS$Junior$ $ Ion$Proton$ Illumina$Hiseq$2500$RR$ Sanger ' Illumina$Hiseq$X$ Lex$Nederbragt$(2014)$h3 p://dx.doi.org/ /m9.figshare $ Illumina$NextSeq$500$ $ 10$ 100$ 1000$ 10000$

12 HPC and Biological Sciences? Biology s equivalent to Moore s law: Just Better!

13 New generation of computer scientist now focused on solving biological problems PROYECTOS Aplicaciones Descripción Usuario BLAST HMMER Identifica una cadena de ADN o de proteínas con un grupo de bases de datos (ADN y proteínas) conocidas. Suite de programas para la creación y análisis de modelos estadísticos de un alineamiento múltiple Biología InterproScan Flujo de trabajo (pipeline) para la caracterización funcional de secuencias biológicas. CMS SW Software del proyecto CMS. Física PovRay Crea imágenes, animaciones mediante código. MPICH Implementación de MPI para paralelizar procesos Phedex Administración de información distribuida. Física Aplicaciones Administrativas Descripción Usuario Ganglia Provee monitoreo del estado de los servidores. DTI Genius Interfaz de usuario gráfica para usar el Grid. Todos

14 Usuarios Uniandes What was Biology s HPC capability (2010) Webserver Entrance Workstation Site Física Site Biología MOX DTI Infraestructura de administración Site Ingeniería Sistemas Site DTI Mobyle Galaxy VO UNIANDES NAS: Storage Biosge Computing nodes 3x 24 cores, 32Gb RAM 1x 24 cores, 128GB RAM NFS 2x0,5TB

15 Application: Metagenomic analysis of extreme environments (GEBIX) METAGENOMES WHAT IS COMMON? WHAT IS DIFFERENT? Corporación Corpogen; Universidad Nacional; Universidad del Cauca; Universidad del Valle; Universidad Javeriana; Universidad de Caldas Uniandes

16 gebix 2 Bioinformatics for GEBIX gebix 3 gebix 8 o Internet Almacenamient o gebix 7 gebix 4 gebix 5 gebix 6 biow n PU J bioinfmac bioin f UNIANDES

17 But most Bioinformatics users don t like command line software! Loni Pipeline Cortesía Javier Tabima

18 Building pipelines and running them from a web server allows different infrastructures to be used The computing cluster was heavily used while other computational resources such as teaching clasrooms and laboratories remained idle for most of the time

19 Solution: Bio-UnaGrid

20 Solution: Bio-UnaGrid A virtual machine is executed on each computer of a lab and it works as a slave on a cluster. There is the need for a dedicated node as cluster master

21 Solution: Bio-UnaGrid Virtual cluster can be defined by research groups, custom application environments, etc. A grid solution (several virtual clusters) can be deployed to fulfill different needs. Allow heterogeneous resources, not a good idea in cluster computing.

22 Solution: Bio-UnaGrid Aplicating LONI Pipeline on Grid infraestructure.

23 Solution: Bio-UnaGrid Aplicating LONI Pipeline on Grid infraestructure.

24 What if we want to select what type of machines to use? We want to deploy on-demand Computing Services. CLOUD COMPUTING And if we still want to use idle computing machines at the university? UnaCLOUD

25 The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UnaCloud THE PROBLEM More than 2000 CPU cores

26 The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UnaCloud THE DESIRED SOLUTION Debian with PBS

27 UnaCloud UnaCloud validates the convergence of cloud computing and virtual clusters offering promising opportunities to meet customized computational requirements through the use of an open source, low cost, extensible, interoperable, efficient, scalable, secure and opportunistic IaaS model. UnaCloud provides a multipurpose cloud computing experimental platform to deploy Customizable Virtual Clusters that support new specific computational requirements of academic and research projects. UnaCloud represents an economically attractive solution for building and deploying large scale computing infrastructures. UnaCloud cloud computing features are promising to reduce the development cycle and the generation of results depending on the agile and flexible provisioning and sharing of low cost computing resources Mario Villamizar, Harold Castro

28 Current status of HPC in Uniandes Joint effort Sciences, DSIT, Engineering Current infrastructure 10 servers 4 cores y 8 GB RAM 2 servers 8 cores y 16 GB RAM 6 servers 16 cores y 32 GB RAM Total Cores: servers 24 cores y 32 GB RAM 1 servers 24 cores y 128 GB RAM Total Cores: 96 6 servers 64 cores y 128 GB RAM Total Cores: 384 DSIT - Biology Storage: 15.5 TB /Users /Applications /Scratch Total current users : 30 Engineering (MOX) Storage: 10 TB /Users /Applications /Scratch Total current users: 68 Installed infrastructure 17 servers 24 cores y 192 GB RAM 2 servers 24 cores y 512 GB RAM Total Cores: 456 Storage: 90 TB Total Cores: servers 64 cores y 128 GB RAM Total Cores: 384 Centralized administration

29 Study of viruses (phages) in the human gut

30 Why study phages? Phages are the most abundant biological group on the planet. Phages are more diverse than their bacterial prey, by an estimated ratio of 10 phages per microbe. They play important roles in marine microbial communities. Important drivers of energy balance and nutrient recycling. Shape microbial communities and generate diversity at strain level. Rodriguez-Brito B, et al. (2010) Viral and microbial community dynamics in four aquatic environments. ISME J 4(6):

31 Why study human gut phages? Ecological importance: community dynamics lytic cycles. Evolutionary importance: Predator prey dynamics role in shaping community adaptations/diversity. Lysogenic cycle Horizontal gene transfer, response to environmental signals. Virome diversity fingerprint that can provide more resolution than bacterial species health versus disease.

32 Phage lifecycle Reyes, A.,et al (2012). Going viral: next-generation sequencing applied to phage populations in the human gut. Nature Reviews Microbiology. 10(9)

33 Initial characterization of human virome Initial MOAFTs samples: 4 families MZ twin pairs + Mother 3 Time points (0, 2, 12 months) Virus purification VLPs from frozen fecal samples 454 Sequencing (MDA amplified VLP DNA) Comparison against reference DB. NR_Viral_DB (tblastx) Sample Nomenclature: F: Family T1, T2: Twins M: Mother (R): Technical Replicate F1T1.1 F1T1.3 F1T2.1 F1T2.1(R) F1T2.2 F1T2.3 F1M.1 F1M.2 F2T1.1 F2T1.1(R) F2T1.2 F2T1.3 F2T2.1 F2T2.1(R) F2T2.2 F2M.1 F2M.1(R) F2M.2 F2M.3 F3T1.1 F3T1.2 F3T1.3 F3T2.1 F3T2.2 F3T2.3 F3M.1 F3M.2 F5T2.1 F5T2.1(R) F4T1.1 F4T1.2 F4T1.3 F4T2.1 F4T2.3 F4M.1 F4M.2 F4M.3 F4M.3(R) Percent assignable reads Average unknown 81±6% Reyes, A., et al (2010). Viruses in the faecal microbiota of monozygotic twins and their mothers. Nature, 466(7304),

34 The Malawi viromes and malnutrition Marasmus Kwashiorkor F93 F229 F112 F194 F284 F23 F56 F268 F196 F57 F138 F26 F10 Dz Mz Dz Mz Dz = Dizygotic Mz = Monozygotic Sibling Mother RUTF Kwashiorkor Marasmus Moderate Malnutrition Total: 231 samples Average 56, reads/sample Healthy F95 F37 F301 F47 F121 F209 F259 Dz Mz Age (Months) Reyes, A., et al (2015). Manuscript in preparation

35 New assembly strategies Linear contigs Circular contigs Number of samples Contig Length (bp) % 70% 90% 100% Percent of data used Median coverage of contig Average 90% of data assembled. Contigs 500nt - 200Kb in length. Large number of circular contigs (potential full genomes). Reyes, A., et al (2015). Manuscript in preparation

36 Contigs specific for twin pairs Healthy Kwashiorkor Marasmus Contigs present at high abundance in twin pair over time. Higher number of discriminatory contigs in twinpairs that developed malnutrition. VLP-derived contigs Log (RPMM) F301 F10 F37 F47 F121 F209 F259 F95 F56 F196 F268 F26 F57 F138 F93 F112 F194 F229 F284 F23 Mother Siblings Reyes, A., et al (2015). Manuscript in preparation

37 High diversity of new Eukaryotic viruses Reyes, A., et al (2015). Manuscript in preparation

38 Different lifestyles provides different advantages in the human gut. Reyes A, Semenkovich NP, Whiteson K, Rohwer F, & Gordon JI (2012) Going viral: next-generation sequencing applied to phage populations in the human gut. Nat Rev Microbiol:1-11.

39 Evaluate in a controlled environment viral:bacterial interactions First, introduce 15 prominent, sequenced members of the human gut microbiota into groups of 5 adult germ-free C57Bl/6 mice Then, after model microbiota assembles in mice, add a pool of previously characterized purified VLPs from 5 healthy adults. Microbes + VLPs Microbes + Heat killed VLPs No microbes + VLPs

40 Community changes observed during assembly Add live VLPs Add heat killed VLPs

41 Temporal changes on microbial community -> staged VLP attack 0.15 Bacteroides caccae Time of addition of Live VLPs B. caccae Relative Abundance B. caccae Live VLP Time (d)

42 Temporal changes on microbial community -> staged VLP attack 0.15 Change not seen with heat killed VLPs B. caccae Relative Abundance B. caccae Live VLP B. caccae Heat Killed VLP Time (d)

43 ϕhsc01 Absolute abundance Square root of viral genome equivalents per mg fecal pellet Temporal changes on microbial community -> staged VLP attack B. caccae Relative Abundance ϕhsc01 Live VLP B. caccae Live VLP B. caccae Heat Killed VLP Time (d)

44 ϕhsc01 Absolute abundance Square root of viral genome equivalents per mg fecal pellet Temporal changes on microbial community -> staged VLP attack ϕhsc01 37,323 bp B. caccae Relative Abundance Assembled 37kb circular Phage, contains: - Phage genes - Terminase - Helicase - DNA polymerase - Bacteroides-associated carbohydrate binding protein Anaerobic bacterial stress response transcription factor ϕhsc01 Live VLP B. caccae Live VLP B. caccae Heat Killed VLP Time (d)

45 Other 4 novel viral genomes identified Model Community + Live VLP Model Community + Heat-Killed VLP No obvious associations with a particular bacterial host. No evidence of integration into bacterial genomes. 4,800 4,200 5,400 3,600 6,000 0 ϕhsc02 6,209 bp 3, ,400 1,200 1,800 Viral abundance (square root of genome equivalents per mg fecal pellet weight) 3000 ϕhsc03 153,451 bp ϕhsc ,253 bp ϕhsc05 95,864 bp Time (d) 0 10, ,000 20, ,000 30, ,000 40, ,000 50, ,000 60,000 90,000 80,000 70, , ,000 90,000 20,000 80,000 30,000 70,000 40,000 60,000 50, ,000 9,000 81,000 18,000 72,000 27,000 63,000 36,000 54,000 45,000

46 New Sequencing/Computing technologies has helped us see the clear picture Early Sanger sequencing

47 New Sequencing/Computing technologies has helped us see the clear picture Current Metagenomics

48 New Sequencing/Computing technologies has helped us see the clear picture Hopefully in the close future However, this doesn t allow to learn enough about the biology!

49 Harold Castro Mario Villamizar Andrés Holguin Michael Perez Diego M Riaño Silvia Restrepo Thank You! bcem.uniandes.edu.co

50