CLOUD GRID Integration S MADHUSUDHANA RAO CENTRE FOR DEVELOPMENT OF ADVANCED COMPUTING, CHENNAI

Transcription

1 CLOUD GRID Integration S MADHUSUDHANA RAO CENTRE FOR DEVELOPMENT OF ADVANCED COMPUTING, CHENNAI

2 AGENDA Introduction CDAC Cloud CDAC GRID-GARUDA Integration of CLOUD & GRID

3 C-DAC an overview C-DAC : seeded at PuneCentre in 1987 High Performance Computing (HPC) the focal area then Later started Multilingual Computing (1988) Advanced Computing Training School (ACTS) (1993) Subsequently added activities Healthcare e-governance Multimedia Geo-Matics Merger of 3 premier societies of DIT with C-DAC in December, 2002 ( ERDCI , NCST-1985, CEDTI-1990 ) ( Electronics, Software, Training & skills development)

4 C-DAC an overview 10 Locations 10 Labs 2500 members

5 Thematic focus areas of C-DAC High Performance Computing & Grid Computing Hardware, Software, Systems, Applications, Research, Technology, Infrastructure Multilingual Computing and Heritage Computing Tools, Fonts, Products, Solutions, Research, Technology Development Health Informatics Hospital Information System, Telemedicine, Decision Support System, Tools, Traditional Knowledge-base and DSS for Medicine Software Technologies, including FOSS OSS, Multimedia, ICT for masses, E-Governance, Geo-Matics, ICT4D

6 Thematic focus areas of C-DAC Professional Electronics including VLSI & Embedded Systems Digital Broadband and Wireless Systems, Network Technologies, Power Electronics, Real-Time Systems, Control Electronics, Embedded Systems, VLSI/ASIC Design, AgriElectronics, Strategic Electronics Cyber Security & Cyber Forensics Cyber Security tools, technologies & solution development, Research & Training

7 Kaleidoscope of C-DAC products

8 International Presence RUSSIA TAJIKISTA N MAURITIUS GHANA UZBEKISTAN MYANMAR SEYCHELLES TANZANIA

9 GRID - GARUDA

10 GARUDA India Map

11 GARUDA India s National Grid Computing Initiative Distributed across 45 institutions in 17 locations. Total:1560 CPUs and 15.2 TF Network : National Knowledge Network (linking all research & educational institutes on high speed backbone ) Certificate : IGCA (Indian Grid Certificate Authority) Applications include -Disaster Management; Computer Aided Engineering; Climate Modeling; Medical and Health Care; Collaborative Class Room

12 Garuda Some Applications Development and parallelization of 2D model based Seismic Waveform inversion algorithm and software for estimating the elastic properties of the earth. Portal which provides a web-interface for Bioinformatics applications in Sequence Analysis and Molecular Modeling running on a High Performance Computer. imoldock Molecular Modeling docking application, which helps the user to find out appropriate ligandsfor particular proteins. EQ-Check To tackle problems related to checking and design of earthquake resistant structures, potentially saving money, structures and lives. SD2000 SD2000 is a supercomputing based Decision Support System using Mathematical, Statistical and Artificial Intelligence more..

13 CLOUD INFRASTRUCTURE

14 Private Cloud Infrastructure Computational three locations Hyderabad, Bengaluru, Chennai Make Model Processor RAM HDD Quantity HP HP HP DL 380 G5 DL 380G5 DL 160 G5 Dual Socket Quad Core Intel Xeon 3.16GHz 32 GB 4 x 146GB 3 Dual Socket Quad Core Intel Xeon 3.16GHz 16 GB 2 x 250GB 1 Dual Socket Quad Core Intel Xeon 3.16GHz 16 GB 2 x 250GB 40 HP HP DL 160G5 DL 360G4 Quad Core Intel Xeon 2.00GHz 2 GB 2 x 250GB 1 Dual Socket Dual Core Intel 3.60GHz 3 GB 160GB 8 HCL Global infinity Dual Core Intel 3.20GHz 2 GB 2 x 146GB 4 Computational resources TOTAL 57 Make Description Capacity 20 x 300 GB 15K RPM SAS disks and 27 x 750 GB SATA disks TB EMC2 2 TB TOTAL 28 TB Network : NATIONAL KNOWLEDGE NETWORK

15 Underlying Technical components CDAC Chennai Cloud powered by BOSS Advanced Server 1.0 Xen Hypervisor 3.2 Eucalyptus 2.0 Hadoop Appscale Hyperic HQ CDAC Scripts for Metering & Billing integrated with Eucalyptus CDAC Scripts for Elasticity integrated with Eucalyptus Garuda Grid powered by Globus Toolkit 4.* Torque Gridway

16 ON-GOING PROJECTS In Cloud Open Source Cloud Middleware development End to End Security for Cloud Integration of Garuda(India s National Grid Computing Initiative) with Cloud Chennai. Cloud stack for e-governance projects (State Data centre pilot@kerala)

17 Integration Requirements The cloud resources hosting virtualized grid environment throughthe images bundled with the grid middleware should be customized for execution of high performance, parallel processing distributed grid applications without any issues or hindrances. The grid environment should be capable of hosting cloud instances and applications. The virtual grid hosted over cloud should be incorporated into the virtual organization formation across cloud and grid or within the cloud.

18 Integration - Requirements Resource management across networking barriers Execution management Monitoring and control of jobs & instances Backup and disaster recovery, live migration Enhanced security Common user interface

19 SCOPE OF THE INTEGRATION Private cloud lacks with resources resulting in Cloud burst Integrates cloud middleware with grid middleware along with its schedulers Optimal utilization of the grid resources by deploying cloud appliances and applications into the grid Return on investment to the grid service providers High performance grid jobs are executed over virtual environmenton cloud instance Dedicated resource for grid not needed.

20 Integrator with Gridway Interfacing Gridway with Integrator. If the job state is pending gridway must invoke integrator Additional no of nodes to be created in cloud resources Image & Security Eucalyptus VM Creation Network Management between grid and cloud Execution Management Data Management Gridway has DRMAA (Distributed Resource Management Application API) for user interface

21 Integrator Submits Job If Job state is pending Request to create VM Meta Scheduler (Gridway) Selects Resource & Submit Eucalyptus Cloud Creating VM in Cloud Computing Nodes Cloud Resource

22 Integrator with Eucalyptus Interfacing Eucalyptus with Integrator If Eucalyptus cloud can't create VM, it must invoke integrator Additional number of nodes to be created in grid resource Image & Network Management Security Management between grid and cloud VM Creation in grid resources

23 Integrator Request Request to Create VM Meta Scheduler (Gridway) Creating VM in Grid Nodes Eucalyptus Cloud Creating VM in Cloud Nodes Computing Nodes Cloud Resource

24 CLOUD GRID INTEGRATION ARCHITECTURE

25 INTEGRATION MODULE Hierarchy of both grid and cloud is almost the same Components of Integrator Execution Manager Resource Manager Network Manager Imaging component Security component Portal

26 GRID JOB EXECUTION IN CLOUD Grid jobs under execution when need additional resources from cloud, request is sent by meta scheduler to integrator Resource management component in integrator approaches cloud controller Bundled grid image from imaging component deployed into cloud with concurrence from the cloud controller Cloud controller through cluster and node controller deploys theimage on n number of virtual instances Local scheduler runs on newly formed grid cluster over cloud resources and is directly controlled by the meta scheduler at the grid site. Virtual organization can be formed across the new grid cluster in cloud and clusters functioning at the grid site

27 CLOUD APPLICATION IN GRID Cloud image bundled and residing in the cloud controller fetchedby integrator imaging component Deployed into grid through meta scheduler after concurrence frommeta scheduler Cloud controller along with integrator resource manager raises a request Meta scheduler deploys image bundle and application is hosted onthe instances Grid instances contains node controller and all nodes controllers operating in grid site is controlled by a Cluster controller deployed internally Cluster controller directly controlled by Cloud controller operating in the cloud site. Integrator is responsible for Metering of Cloud instances, Elasticity, Backup and disaster recovery, Elastic Ipaddressing of instances deployed onto Grid.

28 Image Management Image Management is mainly used to manage the virtual images. Image Management module maintains the metadata as an xml DB which contains all the image related information. Virtual Image is the image which has created with operating system and applications.

29 Security Management It is mainly used to authenticate and authorize the grid and cloud users. The security management should have the intelligent for mapping both the grid and cloud users to access the resource.

30 Network Management It is responsible for assigning the IP address, host name and managing the network translation between grid and cloud resources

31 REFERENCES Application Level Interoperability between Clouds and Grids, Andre Merzky, Katerina Stamou, Shantenu Jha, uting Conference /09 $ IEEE Using Clouds to address Grid Limitations, GiacomoV. Mc EvoyBruno Schulze, Laboratório Nacional de Computação Científica Av. Getúlio Vargas, 333 Quitandinha Petrópolis, RJ, Brazil MGC 08, December 1-5, 2008 Leuven, Belgium. Copyright 2008 ACM /08/12... Integration of Cloud and Grid Middleware at DGRZR, Stefan Freitag, Robotics Research Institute Dortmund University of Technology International Symposium on Grid Computing 2010.

32 THANK YOU Please visit us Contact: