Bartering-based Resource Negotiation in P2P Grids

Transcription

1 BEgrid Seminar, 27 th October 2006 Bartering-based Resource Negotiation in P2P Grids Cyril Briquet Department of EE & CS University of Liège, Belgium

2 Overview Convergence of Grid and P2P Bartering in P2P Grids Lightweight Bartering Grid Architecture Summary 2

3 What is the Grid? Ian Foster's 3-point checklist «A Grid is a system that coordinates resources that are not subject to centralized control, using standard, open, general-purpose protocols and interfaces, to deliver nontrivial qualities of service.» 3

4 Current state of Grid computing Most popular Grid efforts relate to interoperability, standards Most current production grids are «top-down» 4

5 Convergence of Grid and P2P Convergence between Grid computing and Peer-to-Peer technologies (in Foster & Iamnitchi, «On Death, Taxes, and the Convergence of Peer-to-Peer and Grid Computing», 2003) One possible goal of P2P Grid computing: enabling «bottom-up» Grids that emerge from the sharing of Resources by independent Peers 5

6 Negotiation enables resource sharing Strong requirement to «negotiate resource-sharing arrangements among a set of participating parties» (in Foster & al., «Anatomy of the Grid», 2001) 6

7 What is negotiation? Negotiation = to have influence over / be able to convince a trading partner to act in a particular way by making proposals, trading options or offering concessions to come to a mutually acceptable agreement. (in Jennings & al., «Automated Negotiation: Prospects, Methods and Challenges», 2001) 7

8 Market-based resource exchange Observation 1: Grid participants have to be motivated to supply their resources Observation 2: each Peer makes its decisions independently Idea: computational economy where access to resources is supplied in exchange of reciprocal access Grid real 8

9 Examples of market-based methods Auctions Commodity markets Bartering Note: monetary methods require a central bank 9

10 Overview Convergence of Grid and P2P Bartering in P2P Grids Lightweight Bartering Grid Architecture Summary 10

11 What is bartering? Bartering = decentralized, non-monetary, market-based resource exchange method Each Peer accounts its own Resource consumption and supplying e.g.: OurGrid and its Network of Favors 11

12 Network of Favors model OurGrid is the «free-to-join P2P grid» developed in Brazil it's based on the Network of Favors model a favor = utilization of a resource resource suppliers give favors to consumers each Peer counts the favors exchanged with every other Peer the model is totally decentralized 12

13 Network of Favors model (accounting) a supplied favor is removed from favors count, a consumed favor is added favors_count(peer_i) >= 0 amount of debt towards Peer_i Reputation of Peer_i correctly accounting resource usage, or computing value of a favor, is mission-critical (see Robson & al., «Accurate Autonomous Accounting of Resource Usage in Peer-to-Peer Grids», 2005) the model is totally decentralized 13

14 Network of Favors model (ranking) Peers always use their Resources to compute their own Tasks first Peers prioritize the supplying of their Resources to other Peers according to their ranking (generous Peers, with a higher favors count, are served first) 14

15 Network of Favors model (soundness) What about freeriding? local Tasks have priority Peers with a good ranking have priority freerider may gain access to unused Resources Does it work? several papers from the OurGrid team demonstrate the robustness of NoF OurGrid has been deployed in production since december 2004 (see 15

16 Negotiation model in bartering Resource exchange between 2 Peers = a Resource of Peer 1 is supplied to run a Task of Peer 2, with expected reciprocity simple, direct negotiation protocol (e.g. I want to consume X resources, you are granted Y resources ) 16

17 Overview Convergence of Grid and P2P Bartering in P2P Grids Lightweight Bartering Grid Architecture Summary 17

18 The art of negotiation Are inflexible negotiators successful? Learning from past interactions would be useful 18

19 Convergence of Grids and Agents Widening the perspective on Grid computing: independence of decision making... (RMS = Agent) => (Grid = MAS) (in Foster & al., «Brain meets Brawn», 2004) 19

20 Motivation Study bartering in P2P Grids (i.e. scheduling, negotiation done by Peers) where Peers model their environment through interactions with other Peers in order to optimize future interactions (i.e. consume reliable resources, when needed) 20

21 Modelling the behaviour of Peers Independent agents are opaque, by design Unknown Peers cannot be trusted PI-Resources => Behaviour of Peers must be modelled from externally observable patterns of data (negotiation, consumption, supplying) 21

22 Lightweight Bartering Grid Architecture Resources, Peers, Users 22

23 Architecture details Each User acts as a client to a given Peer and submits Requests (Bag-of-Tasks) Each Peer owns heterogeneous Resources Goal of each Peer is to compute Requests Free computational power of Peer may be used to augment ranking with other Peers Consumer and Supplier models: Network of Favors, augmented with reliability modelling 23

24 LBG: 1 architecture, 2 deployments Lightweight Bartering Grid (LBG) Architecture enables to build P2P Grids where Peers model their environment can be deployed as a Grid middleware and as a Grid simulator - from the same Peers code - with minimum code reimplementation of Users and Resources 24

25 Overview Convergence of Grid and P2P Bartering in P2P Grids Lightweight Bartering Grid Architecture Peer managers Simulator Middleware Summary 25

26 Grid Register 3 databases of management data model the environment through interactions: Grid Negotiation Profile Grid Bartering Profile Peer Profiles Reminder: collected data is externally observable Little trust => Limited access to other Peers' data 26

27 Modelling the behaviour of other Peers Machine Learning is aimed at identifying patterns in data Machine Learning algorithms operate on a set of examples (e.g. management data items) each composed of a set of attributes and output relationships between or about these examples 27

28 Machine Learning tools classification: put data items into some predefined classes e.g. QoS, qualitative reliability measure regression: predict a numeric quantity instead of a class e.g. runtime, time before preemption time series: predict the next element in a serie of past elems e.g. resource availability clustering: group elements that belong together 28

29 Relationship between Negotiation and Learning Some authors argue that: «Negotiation can be viewed as a distributed search through a space of potential agreements» «Machine Learning is searching through a model space» Negotiation = a form of distributed learning? 29

30 Other Peer managers Request Manager Resource Manager Task Manager Scheduler: local, nonpreemptive, preemptive algorithms Negotiator: random, NoF, NoF+reliability algorithms 30

31 Overview Convergence of Grid and P2P Bartering in P2P Grids Lightweight Bartering Grid Architecture Peer managers Simulator Middleware Summary 31

32 Simulator discrete-event system simulator Job submission events Task completion/cancellation/failure events scenario-based negotiation, scheduling after each time step with at least 1 event (= new submitted Tasks or free Resources) 1 Thread only good for memory usage trivially // activities may be //-ized in a few Threads 32

33 Related simulators GridSim SimGrid OurGrid Sim LBG Sim Java C Java Java any Grid any Grid P2P Grid P2P Grid threads threads 1 thread 1 thread - code sharing - code sharing 33

34 Overview Convergence of Grid and P2P Bartering in P2P Grids Lightweight Bartering Grid Architecture Peer managers Simulator Middleware Summary 34

35 Middleware communications: serialized Java objects over TCP sockets Grid Task: 1 Jar file (several classes, 1 implements an interface) dynamic code uploading (to owned Resources, other Peers) automatic data transfer Peer discovery via a basic, centralized directory 35

36 Lightweight Bartering Grid Architecture Resources controlled by Peer, Scheduler in Peer 36

37 OurGrid Architecture Resources controlled by User, Scheduler in User 37

38 Related middleware: OurGrid OurGrid advanced User «simple» Peer Network of Favors no bartering opt. used in production LBG simple User advanced Peer Network of Favors reliability bartering used in testbed 38

39 Overview Convergence of Grid and P2P Bartering in P2P Grids Lightweight Bartering Grid Architecture Peer managers Simulator Middleware Summary 39

40 Summary The Lightweight Bartering Grid architecture enables to build P2P Grids Goal = study scheduling, negotiation algorithms in P2P Grids where Peers model their environment Can be deployed as a simulator, but also as a middleware, with the same Peer managers Java-based, supports Java Grid applications 40

41 BEgrid Seminar, 27 th October 2006 Thank You! Cyril Briquet Department of EE & CS University of Liège, Belgium

42 Abstract Automated resource exchange and negotiation between participants of a Virtual Organization, or Peers of a Peer-to-Peer Grid, is an important feature of Grid computing because it enables scalable cooperation between entities under separate administrative control. This presentation will report on our work on resource Bartering in Peer-to-Peer Grids, including a simulator of resource exchange algorithms. 42