Using Resource Virtualization Techniques to Grid-enable Coupled Coastal Ocean Models

Transcription

1 Using Resource Virtualization Techniques to Grid-enable Coupled Coastal Ocean Models Renato Figueiredo Arijit Ganguly Advanced Computing and Information Systems Lab Peter Sheng, Justin Davis, Vladimir Paramygin Civil and Coastal Engineering University of Florida

2 Outlook Focus: Grid application in the context of SURA s SCOOP project Collaborative work: coastal ocean and Grid middleware research groups

3 The ACIS Laboratory Fundamental and applied research on systems that integrate computing and information processing 5 faculty: architecture, distributed systems, Web services, peer-to-peer 2 post-doctorate associates, 25 students Key mission: Internet-scale systems research Grids Purdue s PUNCH (v1-v5, ) In-VIGO (v1, 2001-)

4 Projects NSF Middleware Initiative (NMI) R&D Research (Northwestern), 03-06, $750K Deployment (Purdue, Wisconsin), 04-07, $450K/3M Applications: Nanoelectronics centers: NSF ncn: 7 institutions, , $750K/$11M NASA INAC: 7 institutions, , $1.1M/$15M ARO DURINT: 10 institutions, , $300K/$6.5M SURA/SCOOP coastal ocean model Grid Other funding, $2M+ Infrastructure NSF CISE-RR, IBM SUR, corporate donations, $6M processors, 10TB+ storage

5 Outline Resource virtualization in Grid computing Application case study Dynamic coupling of coastal ocean models: CH3D+SWAN+ADCIRC Summary

6 The problem: Grid computing The grid: Flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resources 1 1 The Anatomy of the Grid: Enabling Scalable Virtual Organizations, I. Foster, C. Kesselman, S. Tuecke. International J. Supercomputer Applications, 15(3), 2001

7 Approach Leverage indirection layers virtualization Decouple user-perceived resources from physical infrastructure Stronger isolation and security Enable on-demand instantiation Facilitates sharing Present software environments where existing, unmodified applications can execute Flexibility

8 In-VIGO virtualization middleware UIML Virtual interfaces XUL Nanohub XML OGSA HTTP Scoop Netcare UDDI Virtual information grids Other portal WSDL SOAP Services Services Services Services Virtual computing grids Globus//Condor.net JINI VMware IBMz800 Virtual machines Java C# Virtual applications NFS Virtual data Virtual networks Machines Applications Data Networks SQL NFS TCP/IP UDP

9 Virtualization techniques Virtual machines (VMs) Single machine, multiple isolated kernels Support unmodified application binaries Execution environment encapsulated with application Use VM monitors (VMware, User-Mode Linux, Xen) Logical user accounts Middleware maps users to pool of accounts On-demand; accounts are not bound to any user Dynamic role-based access control Do not require VM monitors

10 Virtualization techniques (cont.) Virtual file systems (VFS) On-demand distributed file systems Grid-enabled applications access file-based data in the same manner they would in a local environment Use and extend Network File Systems (NFS) Cross-domain authentication, encryption, performance Virtual applications (VAP) User-perceived interface to the real application Possible to extend, compose, customize Accessed via portal

11 Virtualization techniques (cont.) Virtual networks Ongoing research On-demand setup of virtual networks Distributed applications communicate as if in LAN Powerful complement to virtual machines Routing through firewalls, NATs Ethernet or IP-level tunneling E.g.: VNET (Northwestern Univ.)

12 Application portal

13 Application case study Application: modeling inundation of coastal areas during a hurricane Coupling of basin-scale hydrodynamics with regional-scale hydrodynamics and waves important for accuracy Existing implementations of individual models: Basin-scale hydrodynamics (ADCIRC) Regional-scale hydrodynamics (CH3D) Regional-scale wave model (SWAN)

14 Model coupling Different possible approaches: Develop new coupled model from scratch: Substantial development effort Existing models have been evolving for several years (ADCIRC, 1992; CH3D, 1987; SWAN, 2000) New application; combine source code: Expertise and intellectual property are distributed Copyrights, up-to-date versioning, languages, Coupling via shared file system: Leverage virtualization middleware for communication/synchronization Coupling can be achieved without source code modifications

15 Coupled model experiment Western North Atlantic Ocean, Gulf of Mexico and Caribbean Sea Tampa Bay, FL ADCIRC (Windows XP) CH3D (Debian Linux) SWAN (Debian Linux)

16 Virtualization and Grid-enabling 1) Resource manager: Create VMs, allocate logical accounts ADCIRC CH3D SWAN University of Florida (UFL) Virginia Institute of Marine Sciences (VIMS) Louisiana State (LSU)

17 Virtualization and Grid-enabling 2) Data manager: Create distributed virtual file system session Shared file system ADCIRC Linux VM Samba/NFS proxy CH3D SWAN UFL VIMS LSU

18 Virtualization and Grid-enabling Every timestep Shared file system ADCIRC CH3D Samba/NFS proxy SWAN UFL VIMS LSU

19 Virtualization and Grid-enabling Shared file system Every 30 Timesteps ADCIRC CH3D Samba/NFS proxy SWAN UFL VIMS LSU

20 User interface: Virtual application

21 Results Case study: Hurricane Frances, Tampa Bay: Local-area (86min) vs. wide-area (97min): 13% overhead Two VMs, two physical machines with logical accounts Marked improvement in accuracy of simulated water level of coupled model with respect to CH3D alone Simulated Measured CH3D CH3D+SWAN+ADCIRC

22 User interface: File manager

23 Summary Virtualization facilitates Grid enabling: Physical resources Provide core virtualization support Middleware: Provide resource, data, and user management; allocate accounts, VMs, VFSs, enforce permissions dynamically Creation of virtual environments as a service Application installers: Design virtual applications; build upon middleware to create application-tailored environments on demand Users: Authenticate to middleware, not end-resources Configure, compose and interact with virtual applications

24 Acknowledgments Sponsors: National Science Foundation SURA/SCOOP DURINT program, ARO IBM VMware In-VIGO development team

25 For more information Publications Logical accounts, single sign-on [HCW 01, IPDPS 04] Virtual file system [HPDC 01, HPDC 04, Cluster Computing 04] Virtual machines, sandboxes [IPDPS 02, ICDCS 03, JPDC 03, SC 04] In-VIGO architecture [FGCS 05]

26 Questions

27 User interface: Virtual workspace

28 In-VIGO: Web portal and Middleware software simulation services educational resources software applications 1) In-VIGO grid computing middleware 2) computing resources 3) results Linux cluster

29