Grid Sun Carlo Nardone. Technical Systems Ambassador GSO Client Solutions

Transcription

1 Grid Sun Carlo Nardone Technical Systems Ambassador GSO Client Solutions

2 Phases of Grid Computing Cluster Grids Single user community Single organization Campus Grids Multiple user communities Single organization Global Grids Multiple user communities Multiple organizations 2

3 Convergence of Grid and Web Services World Sun is actively participating Platinum Sponsor of Global Grid Forum from its start Enterprise Grid Alliance OASIS Other standardization efforts OGSA/OGSI WS RF (Web Services Resource Framework) WS GAF (Web Services Grid Application Framework) OMII (Open Middleware Infrastructure, UK) NMI (NSF Middleware Initiative) 3

4 Sun Grid Services Environment Small to Large Grid Computing Solutions Global Grid Infrastructure Enterprise Grid Infrastructure Cluster Grid Infrastructure Service Discovery Authentication/ Authorization Data Management Policy Management Resource Management System Management Data Access Industry Standards and partner technologies Java Enterprise System, J2EE OGSA, WS-RF... Globus Toolkit Avaki, SRB... Sun N1 Provisioning System Sun N1 Grid Engine 6 Solaris Containers Solaris Resource Manager Sun Management Center Sun Control Station Sun QFS/SAM-FS Solaris ZFS, CacheFS 4

5 Java Enterprise System for EDU 5

6 JES Components 6

7 N1 Grid Engine Distributed Resource Management in Cluster & Enterprise Grids Multi platform, open source +10,000 downloads since 2000 Queueing, prioritizing, scheduling Policy based sharing btw groups & departments Community project at gridengine.sunsource.net New features in N1GE 6 Improved throughput and scalability New scheduler: resource reservation with backfilling Standard DRMAA compliant More supported platforms, Mac OS/X included Optional BerkeleyDB based logs ARCo: Accounting & Reporting Console (Sun product only) Future Windows exec client (Sun product only) 7

8 Grid Engine Portal An Open Source project based on Java Portal Server 8

9 Campus Grid Computing 9

10 A Successful Example White Rose University Consortium Collaboration btw Univ. of York, Sheffield and Leeds Critical mass for R&D amongst University departments White Rose Grid In production since 2002 Globus Toolkit 2.0 middleware GridEngine used for fair sharing of resources btw sites 10

11 Grid Projects

12 Some Interesting Grid Project Using Sun Technologies JGrid JxGrid GrIDE Many others: EPCC TOG (Transfer-queue Over Globus), see gridengine.sunsource.net/project/gridengine/tog.html EPCC JOSH (JOb Scheduling Hierarchically), see and gridengine.sunsource.net/project/gridengine/josh.html JiniGrid, see 12

13 JGrid A Java Interface to Grid Engine Provides a useful remote API for submitting objects to the grid Lets Grid Engine do the scheduling and accounting Allows remote class loading Minimizes overhead, enables scalability Alpha available at gridengine/howto/jgrid/jgrid.html 13

14 JGrid Architecture Master Exec Host RMI interface Compute Proxy Result Channel GE Master Result Channel GE exec daemon Native Peer Compute Agent Gridlet Native Peer Gridlet Shared Storage Serialized Gridlet 14

15 J2EE JGrid Environment App Server Servlet Container Servlet EJB Container EJB EJB EJB N1GE Master Host JGrid Server qmaster Execution Host execd JGrid Agent Execution Host execd JGrid Agent Execution Host execd JGrid Agent 15

16 Project JxTA Peer to peer Networking via XML Protocols 16

17 JxTA Peer Groups 17

18 JxGrid A JXTA Approach to Resource Discovery for Grid Engine 18

19 GrIDE Developed at APSTC Asia-Pacific Science & Technology Centers, a network of R&D Centers in partnership btw Sun Microsystems EDU LOB and Academia GrIDE is an integrated development environment for Grids Workflow Editor Crosscompiler Grid Debugger Performance Tuning Data Grid Access Project Collaboration 19

20 GriDE Architecture NetBeans Tools Platform Flow Editor Apps Templates Cross Compiler Apps API Grid debugger Performance Turning Data Grid Access Project Collaboration GriDE Layer Java C/C++ Fortran CoGKit MPI-G2 Grid Simulation Programming languages/ Libraries Layer Globus SGE NimRod Middleware Layer Computing Resources 20

21 21

22 GrIDE Workflow Editor 22

23 Data Grids

24 Data Grids Federated, transparent services, independent of Location Name Distribution Replication Ownership Heterogeneity Change of schema Support for domain specific and general data access layer Replication management and consistency Consistency services appropriate to nature of the Grid Globus GridFTP is not enough! 24

25 Data Grids: EPCC OGSA DAI Main Grid effort at EPCC Open Grid Services Architecture Data Access and Integration Main tenet: while compute on demand is important, it is a niche market Distributed data on demand is much more interesting prospect ELDAS J2EE version of OGSA DAI Check out 25

26 Data Grids: Avaki Commercial result of Legion research project Enterprise Information Integration Unified data catalogue Single data service layer Access via Web services, ODBC, JDBC, file I/O Caching for performance Data integration accessible as a web service regardless of location Available via Sun Client Solutions 26

27 Data Grids: SRB Storage Resource Broker Research project from SDSC, San Diego, see Client server middleware Uniform interface for connecting to heterogeneous data over networks to enable data grids, digital libraries, etc. Works with MCAT (Metadata Catalog) to: Access data sets based on attributes and logical names Various interfaces: inq, MySRB, Jargon, Matrix Commercial version from Nirvana Storage 27

28 Thanks! cell: The Network is the Computer