Grid Technology in Civil Engineering

Transcription

1 University of Ljubljana Faculty of Civil and Geodetic Engineering, Institute of Civil Enegineering, Earthquake Engineering and Construction IT, Chair of Construction Informatics Grid Technology in Civil Engineering M. Dolenc, V. Stankovski and Ž. Turk

2 University of Ljubljana Faculty of Civil and Geodetic Engineering, Institute of Civil Enegineering, Earthquake Engineering and Construction IT, Chair of Construction Informatics AEC sector Grid Technology in Civil Engineering M. Dolenc, V. Stankovski and Ž. Turk

3 1 page presentation Grid dream... is to allow users to tap into resources off the Internet as easily as electrical power can be drawn from a wall socket. Grid is like a distributed computer system except it has better security,... it is build on open standards, and... it usually spans across different domains. Grid service is like a Web Service except supports grid security infrastructure, and... it is usually stateful. No single Grid but many different types.... but as nothing is as simple as that there are another 50 slides about this

4 Grid hype Hyped technologies for 2005: Gartner rates the buzz-words,

5 Content Introduction to grids definition virtual organization types of grids grid hype generally lots of it but not in this presentation ;-) System architectures distributed objects architecture (COM/DCOM, CORBA,...) Service Oriented Architecture (Web Services) Grid standards Open Grid System Architecture (OGSA) WS-Resource Framework (WSRF)

6 Content Grid middleware / toolkits Globus Toolkit, Condor,... Grid projects AEC related: NEESGrid, G-Civil, InteliGrid EU FP6 projects Community grids: SETI@Home, Distributed.net,... Conclusions / comments

7 What is a grid Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations I. Foster & C. Kesselman, ``Concepts and Architecture'', The Grid 2: Blueprint for a New Computing Infrastructure, Enables integration of distributed resources 2. Using general-purpose protocols & infrastructure 3. To achieve better-than-best-effort service

8 Scenario (1/3) A Research Laboratory A

9 Scenario (2/3) A group of experts B B

10 Scenario (3/3) C Computing and data resources C

11 Virtual Organization (VO) A Administrator VO C B The demo is from the GridLab project.

12 Grid security Terminology Authentication (identity) and Authorization (rights) Message protection (integrity and confidentiality) Digital signature Accounting Certificate Authority (CA) Why is grid security hard? Because being used may be valuable & the problems being solved sensitive... are often located in distinct administrative domains... set of resources used by a single computation may be large, dynamic, and unpredictable... it must be broadly available & applicable (standard, well-understood, well-tested, integrated with a variety of tools)

13 Create Processes at A and B that Communicate & Access Files at C GSI in action Single sign-on via grid-id & generation of proxy cred. user Site A (Kerberos) User Proxy Proxy credential Or: retrieval of proxy cred. from on-line repository Remote process creation requests* GSI-enabled Authorize Map to local id GRAM server Create process Generate cred. Authorize Map to local id Create process Generate cred. Kerberos ticket Site B (Unix) Computer Computer Process GSI-enabled GRAM server Restricted proxy Process Communication* Local id Local id Remote file access request* * With mutual authentication Site C (Kerberos) Storage system Restricted proxy GSI-enabled FTP server Authorize Map to local id Access file

14 A three point checklist I. Foster, ``What is the Grid? A Three Point Checklist., Grid Today, July 20, Coordinates resources that are not subject to central control integrates and coordinates resources within different control domains addresses the issues of: security, policy, payment, membership,... Uses standard, open, general-purpose protocols and interfaces built on top of multi-purpose protocols and interfaces that address issues like authentication, authorization, resource discovery, and resource access. open and standard protocols and interfaces Delivers non-trivial quality of service deliver various qualities of service (response time, throughput, availability, security,...)

15 Grid types Computation intensive Interactive simulation (climate modeling) Very large-scale simulation and analysis (galaxy formation, gravity waves, battlefield simulation) Engineering (parameter studies, linked component models) Data intensive Experimental data analysis (high-energy physics) Image and sensor analysis (astronomy, climate study, ecology) Distributed collaboration On-line instrumentation (microscopes, x-ray devices, sensors, etc.) Remote visualization (climate studies, biology) Engineering (large-scale structural testing, chemical engineering) In all cases, the problems were big enough that they required people in several organizations to collaborate and share computing resources, data, instruments.

16 Grid types (AEC view) Computation intensive Interactive simulation (climate modeling) Very large-scale simulation and analysis (galaxy formation, gravity waves, battlefield simulation) Engineering (parameter studies, linked component models) Data intensive Experimental data analysis (high-energy physics) Image and sensor analysis (astronomy, climate study, ecology) Distributed collaboration On-line instrumentation (microscopes, x-ray devices, sensors, etc.) Remote visualization (climate studies, biology) Engineering (large-scale structural testing, chemical engineering) In all cases, the problems were big enough that they required people in several organizations to collaborate and share computing resources, data, instruments (this is true for all AEC projects, even the smallest one).

17 Grid as distributed operating system A move away from traditional mainframe systems... computing, storage, data processing... to geographically distributed, loosely coupled fabric of computing systems Why? To take advantage of commodity systems But we now need to replicate functions of mainframe systems resource management security monitoring accounting If successful start to commoditize the OS

18 Distributed object systems Semantics of object initialization and method invocation are exposed to remote systems Proprietary or standardized mechanisms, broker requests across system boundaries, marshall and unmarshall method argument data, etc. Distributed objects systems typically characterized by: objects maintaining internal state required to support their methods fine grained interaction between an object and a program using it shared implementation type systems and interface hierarchy between object and program using it Implementations: RPC, Java RMI, CORBA, COM/DCOM,...

19 Impl. of distributed objects systems Java RMI RPC Java RMI interface RMI interface implementation class Client RPC Compiler RPC IDL interface Object Server Client RPC Compiler Server JRMP Client stub Server skeleton Client stub Java RMI Nameserver RPC runtime TCP/IP TCP/IP COM/DCOM CORBA MIDL interface Client Server skeleton MIDL Compiler CORBA IDL interface COM Server Client ORPC DCOM proxy DCOM skeleton Server IIOP Client stub Server skeleton RPC ORB TCP/IP TCP/IP Object IDL Compiler

20 Service Oriented Architecture (SOA) SOA is characterized by the following properties: Logical view: service is an abstracted; programs, databases, etc. defined in terms of what it does. Message orientation: service formally defined in terms of the messages exchanged between provider/requester agents, not the properties of the agents Description orientation: machine-processable metadata, only public information is described Granularity: small number of operations / large and complex messages. Network orientation: Services used over a network, though not required Platform neutral: Messages are sent in a platform-neutral, standardized format delivered through the interfaces.

21 Web Services (WS) WSDL interface Client WSDL Compiler Service Server SOAP Client stub Server skeleton SOAP engine HTTP

22 OGSA key requirements Interoperability and support for dynamic and heterogeneous environments Resource sharing across organizations Job execution Data services Security Optimization Quality of service (QoS) assurance Administrative cost reduction Scalability Availability Ease of use and extensibility

23 OGSA defines basic capabilities Infrastructure services Security services Execution management services Self-management services Data services Information services Resource management services Security considerations Standardization Domain-Specific Services Program Execution Data Services Core Services WS-Resource Framework Web Services Messaging, Security,...

24 Stateless Web Service

25 Stateful Web Service

26 Grid and Web Services (first try) OGSI Grid Started far apart in apps & tech Web GT1 GT 2 OGS I Have been converging HTTP, WSDL WS-*? 2, WSDL WSDM Several (some partial) implementations Issues: technical, political & commercial Successes: a number of operational grids

27 Grid and Web Services (second try) WSRF Grid Started far apart in apps & tech Web GT1 GT 2 OGS I Have been converging HTTP, WSDL WS-* WSRF 2, WSDL WSDM Support from major WS vendors CA, HP, IBM, Fujitsu, BEA, SAP,...

28 WS-Resource: stateful resources WS-Resource = Web Service + WSRF A stateful resource is something that exists even when you're not interacting with it. e.g. database back-end service Stateful resources have properties that define state these properties are how you interact with them properties have values add/remove/change properties and values dynamically WSRF Specification: a WS-Resource is the combination of a Web service and a stateful resource on which it acts.

29 WS-Resource approach to state Typical approach: Put the state in the Web service (thus making it stateful, which is generally regarded as a bad thing) WSRF approach: Store state in a separate entity called a resource Each resource has a unique key, A Web service can have multiple resources To connect to service: URI + WS-Addressing

30 WSRF service invocation - stateful

31 WSRF specifications Core: WS-Resource Framework (WSRF) WS-ResourceProperties (WSRF-RP) WS-ResourceLifetime (WSRF-RL) WS-ServiceGroup (WSRF-SG) WS-Base Faults(WSRF-BF) Related: WS-Notifications WS-Addressing Different WSRF implementations Globus Toolkit (Java, Python, C) WSRF:Lite (Perl), WSRF.Net (.Net),... Interoperability between implementations:?

32 OGSA / WSRF / Implementation relationship Other software package (WSRF.NET, ) Globus Toolkit 4 implements high-level services for grid applications meet requirements of implements implements implemented on top of OGSA WSRF requires specifies Stateful Web service extends Web service

33 Grid middleware / toolkits General grid middleware Globus Toolkit de-facto standard OMII e-science UK ( glite EGEE project ( like Linux distributions :-( Cluster management / Workload management Condor Sun Grid Engine (

34 Globus Toolkit A software toolkit addressing key technical problems in the development of grid enabled tools, services, and applications Offer a modular bag of technologies Enable incremental development of grid-enabled tools and applications Implement standard grid protocols and APIs Make available under liberal open source license Globus Consortium: HP, Intel, IBM, SUN,... to advance commercial use of Globus software (GT) Available at www-unix.globus.org/toolkit

35 Globus Toolkit 4 components components

36 Globus Toolkit 4 components components runs only on Unix!!

37 Condor system Developed since 1986 The project goal: develop, implement, deploy, and evaluate mechanisms and policies that support HTC cycle steeling software dynamic environment (ex. student classrooms or office machines) Runs on (Linux, Solaris,...)/Windows/MacOSX Integration with GT4 (Condor-G) Available at

38 AEC related grid projects / development On-line collaboration NEESGrid (National Earthquake Engineering System) AccessGrid Sensor monitoring G-Civil Project management InteliGrid (Interoperability of Virtual Organisations on Complex Semantic Grid) Test drive a grid go to GENIUS testbed (

39 AEC related grid projects / development On-line collaboration NEESGrid (National Earthquake Engineering System) AccessGrid Sensor monitoring G-Civil Project management InteliGrid (Interoperability of Virtual Organisations on Complex Semantic Grid) Test drive a grid testbed go to GENIUS testbed (

40 NEESGrid Encourage collaboration among earthquake engineering researchers and practitioners. Provide remote access to large-scale earthquake engineering facilities. Provide distributed collaboration tools. Provide easy-to-use simulation capabilities. Allow integration of physical and simulation capabilities. Provide a community data repository for sharing data generated by use of the system. Create a cyberinfrastructure for earthquake engineering. Define and implement grid based integration points for system components. C. Kesselman, Introduction to Grids, Middleware and applications,

41 NEESGrid

42 G-Civil Mission:... to build a prototype system responsible for the collection, distribution and visualization of data collected from civil engineering sites or from infrastructure monitoring schemes Work on: integration of real-time data sharing site monitoring data visualization industrial uses More at

43 InteliGrid The goal:... to provide industries with challenging integration and interoperability needs a flexible, secure, robust, ambient accessible, interoperable, pay-perdemand access to information, communication and processing infrastructure. Sept 2004 Feb 2007 (30 months) Effort 360 person months More at

44 AccessGrid Enables group-group communication audio / video sharing of applications VNC open source Runs on Windows, Linux it works if you have multicast connection (unicast / multicast bridge) More at

45 AccessGrid Enables group-group communication PSNC, Poznan, Poland audio / video sharing of applications VNC open source Runs on OPB, Munchen, Germany Windows, Linux it works if you have multicast connection (unicast / multicast bridge) More at Picture taken at one of InteliGrid project on-line meetings. Ljubljana, University of Ljubljana, Slovenia

46 EU FP6 grid projects More info at

47 Community grids / distributed computing SETI@Home, search for extraterrestrial intelligence started in May 17, 1999 World Community Grid, to create largest public computing grid benefiting humanity Human Proteome Folding Project Distributed.net, world-wide distributed computing effort that is attempting to solve large scale problems

48 Conclusions / comments Is grid technology relevant for AEC sector? YES.... ready for research institutions / universities? Yes. high performance computing (HPC) high throughput computing (HTC) sensor monitoring on-line collaboration... ready for AEC industry deployment / uptake? Not yet, but we are nearly there. project management building life cycle management data management