Clusterix Dynamic Clusters Administration Tutorial

Transcription

1 Clusterix Dynamic Clusters Administration Tutorial Marcin Pawlik Jan Kwiatkowski IIS, WIZ, PWr

2 Tutorial outline Clusterix and dynamic clusters introduction Clusterix security infrastructure Clusterix project internals Dynamic cluster attachment procedure Cumulus dynamic cluster demonstration

3 Presentation outline Cumulus metacluster description Dynamic clusters attachment procedure Clusterix utilization

4 Work motivation Creation of a parallel processing environment Joining the Clusterix Access to a nation-wide Grid environment Higher computational power Higher availability

5 Cumulus computational environment Grid environment with full access to the underlying hardware and software infrastructure Requirements: Cost effective utilizes existing hardware infrastructure Not invasive no large modifications to the existing infrastructure needed Cohabitative no degeneration of the existing functionality Useful meets our scientific and educational requirements

6 Cumulus metacluster implementation Assumptions network boot remotely mounted file systems optional local swap and scratch space Features easy to control and modify higher server load

7 Cumulus environment characteristics Fully controllable the nodes are fully dedicated to the cluster Modifications only in the cluster space Operates when the machines are not utilized Not fully available works only part-time

8 Model extensions Addition of dedicated nodes - full availability for testing purposes Dual mode of operations - as a local computing environment and as a part of a computational Grid 100Mbps

9 Clusterix structure

10 Dynamic Computing Resources Architecture Dynamic cluster attachment Requirements needs to be checked against new clusters Installed software SSL certificates Communication through router/firewall Network Monitoring System will automatically discover new resources Regular Cluster Local Switch PIONIER Backbone Switch Dynamic Resources Internet New cluster can serve computing power like the regular ones Router Firewall

11 Connection of a Dynamic Cluster Nodes Access node Firewall Nodes Local switch Internet DYNAMIC CLUSTER Firewall/ router Switch Not concerned about the internal structure of dynamic cluster Anyway it needs to have a public firewall Dynamic cluster must build a connection through the untrusted Internet Backbone network Connection via the local cluster firewall to the core

12 Dynamic cluster connection necessary steps Initial procedure Initial contact Dynamic cluster software requirements Security certificates Clusterix firewall selection Necessary Clusterix software installation Attachment/detachment

13 Globus Toolkit Security Infrastructure + Resource Management Information Services Data Management

14 Virtual User Account System john js jsmith jsmith Virtual User smith System Account foo acc01 Wrocław, June 2006

15 Grid Resource Management System In Clusterix, we base on GRMS developed in Gridlab project The main functionality of GRMS include: ability to choose the best resource for the task execution, according to the job description and chosen mapping algorithm ability to submit the GRMS task according to the job description ability to migrate GRMS tasks to better resources, according to the provided job description ability to cancel the task provides information about the task status, and other information about tasks, e.g., name of host where the task is/was running ability to transfer input and output files

16 Attachment procedure LCF ssh connection DCF /24 Internet Step 1: DCF connects to LCF through SSH and by logging to the special account invokes dclctl script Step 2: The script on LCF determines DCF IP and finds appropriate configuration settings and send the address range to DCF Step 3: LCF updates LCF firewall and routing configuration, creates the tunnel from the local cluster side Step 4: DCF receives its address and prepares its end of the tunnel Step 5: Using ICF protocol and X.509 certificates DCF and LCF create IPSec tunnel

17 Clusterix utilization Portal interface GRMS client Globus job submision system Direct login (individually negotiated)

18 Thank you for your attention (for now)