How To Build Westgrid

Transcription

1 WESTGRID ORIENTATION SESSION October 28, 2003 Simon Fraser University

2 What is WestGrid? PROJECT SUMMARY WestGrid is a $48 million project to acquire and install grid-enabled computational, data storage and collaboration facilities at 8 institutions, including: New MIC Simon Fraser University The Banff Centre TRIUMF University of Alberta University of British Columbia University of Calgary University of Lethbridge Five co-principal investigators lead the project (Jonathan Borwein, Grenfell Patey, Jonathan Schaeffer, Brian Unger, Michel Vetterli) with the involvement of over 250 researchers and users, a chief technology officer, distributed systems architect, technical support staff at partner institutions, Netera Alliance, BCNET and CANARIE.

3 Components 1. UofA: Large shared memory computer: SGI Origin 256 processor system for shared-memory parallel computing, plus a 5 Terabyte disk storage system and 10 Terabytes of tape storage. 2. UofC: Cluster of Multi-Processors (CluMP): HP SC processors for message passing parallel computing, plus a 5 Terabyte disk storage system. Genematcher2 genome sequence analyzer. 3. UBC/TRIUMF: Large commodity Linux farm: 1008 processor IBM blade cluster for naturally parallel computing jobs, plus a 10 Terabyte disk and 70 Terabyte tape storage facility. 4. SFU: Network storage: A scalable network storage facility consisting initially of 24 Terabytes of disk and 135 Terabytes of tape silo capacity. 5. Collaboration and Visualization facilities: Video conferencing and document sharing capabilities, built on the Access Grid technology, enhanced with visualization, virtual reality and other enhanced collaborative facilities. 6. Grid services: Grid computing tools will form an integral part of WestGrid.

4 Funding Funding and other support has been provided by: Alberta Innovation and Science BC Knowledge Development Fund Canada Foundation for Innovation Hewlett Packard IBM SGI BCNET CANARIE Netera Alliance NewMIC Simon Fraser University The Banff Centre TRIUMF University of Alberta University of British Columbia University of Calgary University of Lethbridge

5 WestGrid Governance and Management

6 WestGrid Capital Budget: Revenues CFI 11,990,839 ASRIP 5,795,420 BCKDF 5,711,591 HP 7,167,218 IBM 5,524,174 SGI 7,224,457 Other private sector 1,030,517 Institutions 384,453 TOTAL: 44,828,669

7 WestGrid Capital Budget: Expenditures HPC 25,605,844 Collaborative Visualization 1,661,597 Storage 9,366,616 Network Interconnect 980,781 Personnel 1,337,505 Software 1,005,941 Warranties 3,664,453 Renovations 1,120,966 Training 85,000 TOTAL 44,828,669

8 Access Grid Collaboration 180 AG nodes worldwide State-or-the-art SGI visualization server (SFU) WestGrid will add scientific visualization and virtual reality SFU GridRoom in Collaboratory 2nd Gridroom coming Presenter camera Ambient mic (tabletop) Presenter mic Audience camera

9 Networking WestGrid HPC sites connected through a GigE core network BCNET NeteraNet CA*net 4

10 Annual User Requirements Annually - users are expected to complete a CFI Impact Report via an on-line web form. Ongoing - users are expected to acknowledge WestGrid in publications facilitated by use of WestGrid resources. Publications - when updating project descriptions, users are requested to provide references for the above publications that acknowledge the support of WestGrid resources.

11 Technical Overview - CTO Computing/Storage resources Network infrastructure Grid/Middleware components Software availability Getting an account

12 Initial HPC Resources 1008 processor IBM (Xeon) blade cluster 256 processor SGI Origin processor HP AlphaServer-SC processor Parasol Genematcher-II Also access to: 160 processor HP Alpha Cluster 192 processor AMD Athlon Cluster 236 processors in SGI Origin servers

13 Initial storage resources Central storage site (IBM) 24TB of disk,135tb of tape Will be expanded as needed Local storage Total of over 30TB disk, 30TB tape distributed between 4 campuses

14 Initial Network HPC/storage sites connected by layer-2, gigabit-per-second network Network appears as a local subnet at SFU, UBC, UofC and UofA Uses components provided by BCNet, CANARIE, Netera and the local sites Lethbridge and Banff connected via NeteraNet

15 WestGrid-Core network

16 Network performance UofC to SFU

17 Grid Computing Grid services provide interoperability between resources in different management domains Global user namespace using Certificate Authority (CA) model each CA is unique (signing key is unique) each CA issues unique DNs to users Grid user is mapped to local user on Grid enabled resource Grid use is all about trust - and developing technologies to ensure trusting environments Many Grid tools are still rudimentary. Much more research still remains to be done.

18 Grid Enabled Resources WestGrid resources WestGrid-Big NUMA WestGrid-DataStore WestGrid-XeonCluster WestGrid-AthlonCluster WestGrid-MPP

19 Timmy s World Timmy s own machines Timmy-Workstation Timmy-DeptCluster Other (non-westgrid) resources that Timmy can access Bobs-PIII-cluster HPCVL-UMA SharcNet-MPP1 SharcNet-ItaniumCluster

20 Timmy s Grid Timmy-Workstation Timmy-DeptCluster Bobs-PIII-cluster WestGrid-AthlonCluster WestGrid-Big NUMA SharcNet-ItaniumCluster WestGrid-DataStore WestGrid-XeonCluster SharcNet-MPP1 HPCVL-UMA WestGrid-MPP

21 WestGrid Grid philosophy Promote use of Grid technologies to users/projects that can benefit from them Encourage use of robust Grid tools in place of traditional alternatives Don t impose the use of Grid technologies on users that don t need them, or that already have good solutions that are not Grid enabled WestGrid needs world class scientific discovery; we don t want to get in the way of this

22 Grid Components Basic services: Security/authentication service Remote job starting service Information discovery service Data movement service High level systems/services: Meta-scheduling Repository management tools

23 Single-signon Using Globus toolkit (2.4.X) for basic grid services GSI-enabled SSH provides single-signon across management domains

24 Job starting on WestGrid systems Have access to multiple job starting mechanisms Use what is easiest for you PBS batch scheduler used on HPC sites Note that some options available from MACI scheduler have changed!! Now using MAUI scheduling engine Globus job starters can be used to start jobs on remote resources globus-job-run / globus-job-submit

25 Meta-scheduling A meta-scheduler submits jobs to collections of batch schedulers e.g., to ones on WestGrid HPC systems Condor-G meta-scheduler will be run on a number of WestGrid systems Condor-G submits jobs to any system running a Globus gatekeeper

26 Distributed Data Management Initially, only simple distributed data management tools will be supported Scp, rsync and globus-url-copy (GridFTP) Will help projects with large data needs to adopt data-replica-management systems Form collections, locate instances of collections and replicate collections Investigating high performance distributed file-system solutions

27 Grid Portals Portals provide a web interface to Grid computing environment WestGrid will provide Information gathering portals Generic job starting portals We will help user groups create application specific job starting portals if required

28 WestGrid portal

29

30

31

32

33

34 Software Availability Vendor proprietary development tools available on each system Will provide core set of open source tools at each site Commercial software subject to vendor s licensing rules

35 Getting an account All users will need to apply for a WestGrid account Web-based account request system Every account must be associated with a faculty member/senior researcher Request that these people register so that their grad students / research assistants can apply for their accounts Link to registration pages available from WestGrid web site

36

37