Deployment of BioXSDenabled services on a Cloud. christophe.blanchet@ibcp.fr

Transcription

1 Deployment of BioXSDenabled services on a Cloud

2 Outline IBCP, provider of BioXSD-enabled services Cloud Computing RENABI GRISBI, French infrastructure

3 Bioinformatics Integrated s gbio-pbil.ibcp.fr/ws

4 GBIO WS resources Biologist 's Machine HTTP SOAP Jobs Web Services HOST Virtual Machines Computing Nodes Virtualization VMware ESX/Xen Physical Machines NFS / S3 Central Storage NFS / S3

5 Web Services clients

6

7 EMBRACE Usecase «Usage of Generic WS» Proteins Master User's sequences splitting 10s to 1,000s Results gathering Protein Similarity s - BLAST, FastA, SSearch Read only Shared Storage Knowledge Bases - MB to 100s GB - Swiss-Prot, TrEMBL Protein sequence analysis at large scale with generic services, User: get the most information possible regarding their protein(s) Bioinformatics tools use case with common software BLAST, ClustalW, GOR4, CATH-Gene3D HMMscan,... Biological data Analyzing large sets of proteins obtained for example from Next Generation Sequencing Using international databases, i.e. UniProt

8 BioXSD Services

9 Benefits of Controlled Vocabulary, Ontology and Schema Providers verify the quality of the input/ ouput data expected data type : integer, string,... expected value: size, allowed range,... identify abnormal data verify the format of the input data return detailled error message: what s wrong in the format convert the data in a good format evaluate complexity/cost for the data input data and its transfer/ computation cost (time and ) ouput data and its storage/transfer cost (time and ) workflow group different process according to the data Users make the Web services «user-friendly» Web services selection BioCatalogue, Seekda Web services composition Taverna, Triana,... switch to other provider in case of failure Web services customization to let users map/connect to their own ontology rich plugins to input data specific forms, with data converters, an adequat help/doc about the data to input, on different hardware/support data conversion in workflow rich plugins to display data

10 Outline IBCP, provider of BioXSD-enabled services Cloud Computing RENABI GRISBI, French infrastructure

11 Cloud?

12 Cloud: principles Cloud Computing Virtual Infrastructure Biologist 's Machine Web Services HTTP WebServices Web CLOUD Portal and Services Cloud Node Cloud Node Cloud Node Cloud Node Physical Infrastructure CLOUD Site

13

14 Cloud: middlewares Open source hipernet (HIPCAL project), eucalyptus (Amazon EC2/S3 like), open nebula (OGF, StratusLab EU FP7) others: nimbus, cloudstack,... Commercial Amazon EC2, Google Apps, IBM, Microsoft, Yahoo,...

15 Cloud Comparison

16 Cloud Console

17 Deploying on Cloud Master User's sequences splitting s - BLAST, FastA, SSearch 10s to 1,000s Read only Shared Storage s Results gathering Knowledge Bases - MB to 100s GB - Swiss-Prot, TrEMBL Protein Similarity Virtual Infrastructure IBCP s cloud 5 servers 40 cores, 160GB Biologist 's Machine SSH WS... SSH WS... SSH WS...

18 Bioinformatics Services on a CLOUD Cloud Storage (S3) Biologist 's Machine 2. Register Biological Data 1. & 5. HTTP SOAP Bioinformatics Virtual Apps Cloud Console 3. Jobs Virtualization VMware ESX/Xen Virtual Machines Mgmt VMs Computing Nodes Connect to Cloud Choose VMs Deploy VMs Submit SOAP Process SOAP Compute job Yet done by IBCP 4. Get Results Physical Machines NFS / S3 Central Storage NFS / S3 Results

19 Cloud Storage (S3) Biologist 's Machine 1. Register Biological Data 3. Get Results 2. HTTP SOAP Cloud Console (EC2, ONE, hipcal) 0. HTTP/S SSH Bioinformatics Virtual Apps Perspectives Cloud Console Jobs Virtualization VMware ESX/Xen Virtual Machines Mgmt VMs Computing Nodes Connect to Cloud Choose VMs Deploy VMs Submit SOAP Process SOAP Compute job Physical Machines NFS / S3 Central Storage Virtual Appliances Deployment NFS / S3 Results

20 What next? Platform-as-a-Service Provide scientists with pre-defined VM to deploy on the Research Infrastructure, e.g. RENABI GRISBI on their own computer/cloud Bioinformatics centers switch to virtual appliances provider Infrastructure-as-a-Service User can connect to the community cloud to deploy required VM Deploy the required infrastructure according to the workflow: VXDL/VPXI developed by HIPCAL project Haizea with the OpenNebula.org

21 Outline IBCP, provider of BioXSD-enabled services Cloud Computing RENABI GRISBI, French infrastructure

22 GRISBI - Grid Support to Bioinformatics - Make possible challenging bioinformatics applications dealing with large scale biological systems GenOuest CBiB GR SB CIB Migale i i 6 centers 1000 cores - RAM 2TB Storage 150 TB PRABI BIPS RENABI GRISBI National Research Infrastructure RENABI, IBISA , Institut des Grilles centers from RENABI PRABI, MIGALE, GenOuest, CBIB Bordeaux, BIPS, CIB 8 sites, with 7 CNRS institutes IBCP Lyon, SBR Roscoff, CBiB Bordeaux, CIB Lille, IRISA Rennes, LBBE Lyon, MIGALE Jouy-en-Josas, BIPS Strasbourg 40 participants Computig resources 1200 cores, 220 TB storage GRISBI - Grid, Support to Bioinformatics,

23 GRISBI site SE CE UI Virtual Machines Virtualization VMware ESX GRID Computing Nodes Physical Machines GRISBI - Grid, Support to Bioinformatics,

24 GRISBI Infrastructure CIB 128 c 1.5 TB 06/ c 3.2 GB 216 c 62 TB 120 c 2 TB 06/ c,1TB 32 c 5 TB GenOuest CBiB GR SB Migale i i 376 c 50 TB PRABI 08/2010 BIPS 412 c 75 TB 48 c 30 TB 150 c 10 TB 09/ c 5 TB + Core services (IDG) WMS (GRIF), LFC&VOMS (CC) 1212 c 224 TB 426 c 17 TB GRISBI - Grid, Support to Bioinformatics,

25 GRISBI usecase Will be used tomorrow as hands-on material by LABRI people (RENABI GRISBI partner) GRISBI - Grid, Support to Bioinformatics,

26 Conclusions Standard Integration of bioinformatics tools Freely available to community Annotated tools with BioXSD/EDAM Cloud perspective Use standard infrastructures Managed infrastructure with Web interface Platform- and Infrastructure-as-a-Service Perspectives Research Infrastructures based on public/private cloud Provide the community with bioinformatics Virtual Appliances

27 Acknowledgment CNRS - Centre National de la Recherche Scientifique University of Lyon 1 The ANR throught the HIPCAL project GIS IBISA through the project GRISBI PF 2008 The European Commission through the EU FP7 EGEE III project, contract number INFSO-RI CNRS IBCP: E. Bettler, C. Combet, G. Deléage, C. Eloto, C. Gauthey, A. Joseph, A. Michon, J Pessey, F. Penin. HIPCAL: Pascale Vicat-Blanc and partners EMBRACE partners GRISBI partners CNRS IBCP Institute of Biology and Chemistry of Proteins 7 passage du Vercors, LYON, FRANCE

28 Thanks Questions?