Grid Computing @ Sun Carlo Nardone Technical Systems Ambassador GSO Client Solutions
Phases of Grid Computing Cluster Grids Single user community Single organization Campus Grids Multiple user communities Single organization Global Grids Multiple user communities Multiple organizations 2
Convergence of Grid and Web Services World Sun is actively participating Platinum Sponsor of Global Grid Forum from its start Enterprise Grid Alliance OASIS Other standardization efforts OGSA/OGSI WS RF (Web Services Resource Framework) WS GAF (Web Services Grid Application Framework) OMII (Open Middleware Infrastructure, UK) NMI (NSF Middleware Initiative) 3
Sun Grid Services Environment Small to Large Grid Computing Solutions Global Grid Infrastructure Enterprise Grid Infrastructure Cluster Grid Infrastructure Service Discovery Authentication/ Authorization Data Management Policy Management Resource Management System Management Data Access Industry Standards and partner technologies Java Enterprise System, J2EE OGSA, WS-RF... Globus Toolkit Avaki, SRB... Sun N1 Provisioning System Sun N1 Grid Engine 6 Solaris Containers Solaris Resource Manager Sun Management Center Sun Control Station Sun QFS/SAM-FS Solaris ZFS, CacheFS 4
Java Enterprise System for EDU 5
JES Components 6
N1 Grid Engine Distributed Resource Management in Cluster & Enterprise Grids Multi platform, open source +10,000 downloads since 2000 Queueing, prioritizing, scheduling Policy based sharing btw groups & departments Community project at gridengine.sunsource.net New features in N1GE 6 Improved throughput and scalability New scheduler: resource reservation with backfilling Standard DRMAA compliant More supported platforms, Mac OS/X included Optional BerkeleyDB based logs ARCo: Accounting & Reporting Console (Sun product only) Future Windows exec client (Sun product only) 7
Grid Engine Portal An Open Source project based on Java Portal Server 8
Campus Grid Computing 9
A Successful Example White Rose University Consortium Collaboration btw Univ. of York, Sheffield and Leeds Critical mass for R&D amongst University departments White Rose Grid In production since 2002 Globus Toolkit 2.0 middleware GridEngine used for fair sharing of resources btw sites 10
Grid Projects
Some Interesting Grid Project Using Sun Technologies JGrid JxGrid GrIDE Many others: EPCC TOG (Transfer-queue Over Globus), see gridengine.sunsource.net/project/gridengine/tog.html EPCC JOSH (JOb Scheduling Hierarchically), see www.epcc.ed.ac.uk/sungrid/wp5/ and gridengine.sunsource.net/project/gridengine/josh.html JiniGrid, see http://pds.irt.vein.hu/jgrid/jgrid.html 12
JGrid A Java Interface to Grid Engine Provides a useful remote API for submitting objects to the grid Lets Grid Engine do the scheduling and accounting Allows remote class loading Minimizes overhead, enables scalability Alpha available at http://gridengine.sunsource.net/project/ gridengine/howto/jgrid/jgrid.html 13
JGrid Architecture Master Exec Host RMI interface Compute Proxy Result Channel GE Master Result Channel GE exec daemon Native Peer Compute Agent Gridlet Native Peer Gridlet Shared Storage Serialized Gridlet 14
J2EE JGrid Environment App Server Servlet Container Servlet EJB Container EJB EJB EJB N1GE Master Host JGrid Server qmaster Execution Host execd JGrid Agent Execution Host execd JGrid Agent Execution Host execd JGrid Agent 15
Project JxTA Peer to peer Networking via XML Protocols 16
JxTA Peer Groups 17
JxGrid A JXTA Approach to Resource Discovery for Grid Engine 18
GrIDE Developed at APSTC Asia-Pacific Science & Technology Centers, a network of R&D Centers in partnership btw Sun Microsystems EDU LOB and Academia GrIDE is an integrated development environment for Grids Workflow Editor Crosscompiler Grid Debugger Performance Tuning Data Grid Access Project Collaboration 19
GriDE Architecture NetBeans Tools Platform Flow Editor Apps Templates Cross Compiler Apps API Grid debugger Performance Turning Data Grid Access Project Collaboration GriDE Layer Java C/C++ Fortran CoGKit MPI-G2 Grid Simulation Programming languages/ Libraries Layer Globus SGE NimRod Middleware Layer Computing Resources 20
21
GrIDE Workflow Editor 22
Data Grids
Data Grids Federated, transparent services, independent of Location Name Distribution Replication Ownership Heterogeneity Change of schema Support for domain specific and general data access layer Replication management and consistency Consistency services appropriate to nature of the Grid Globus GridFTP is not enough! 24
Data Grids: EPCC OGSA DAI Main Grid effort at EPCC Open Grid Services Architecture Data Access and Integration Main tenet: while compute on demand is important, it is a niche market Distributed data on demand is much more interesting prospect ELDAS J2EE version of OGSA DAI Check out www.ogsadai.org.uk 25
Data Grids: Avaki Commercial result of Legion research project Enterprise Information Integration Unified data catalogue Single data service layer Access via Web services, ODBC, JDBC, file I/O Caching for performance Data integration accessible as a web service regardless of location Available via Sun Client Solutions 26
Data Grids: SRB Storage Resource Broker Research project from SDSC, San Diego, see www.npaci.edu/dice/srb Client server middleware Uniform interface for connecting to heterogeneous data over networks to enable data grids, digital libraries, etc. Works with MCAT (Metadata Catalog) to: Access data sets based on attributes and logical names Various interfaces: inq, MySRB, Jargon, Matrix Commercial version from Nirvana Storage 27
Thanks! carlo.nardone@sun.com cell: 335 5828197 The Network is the Computer