Trading Grid Services A Multi-attribute Combinatorial Approach

Transcription

1 Trading Grid Services A Multi-attribute Combinatorial Approach Björn Schnizler 1, Dirk Neumann 1, Daniel Veit 2, Christof Weinhardt 1 1 Institute of Information Systems and Management, University of Karlsruhe, Germany 2 Chair of Business Administration and Information Systems, University of Mannheim, Germany Abstract The Grid is a promising technology for providing access to distributed high-end computational capabilities. Thus, computational tasks can be performed spontaneously by other resources in the Grid that are not under the user s control. However, one of the key problems in the Grid is deciding which jobs are to be allocated to which resources at what time. In this context, the use of market mechanisms for scheduling and allocating Grid resources is a promising approach toward solving these problems. This paper proposes an auction mechanism for allocating and scheduling computer resources such as processors or storage space which have multiple quality attributes. The mechanism is evaluated according to its economic and computational performance as well as its practical applicability by means of a simulation. 1 Introduction The increasing interconnection between computers has created a vision for Grids. Within these Grids, computing resources such as processing power, storage space or applications are accessible to any participant. This accessibility has major ramifications for organizations since they can reduce costs by outsourcing nonessential elements of their IT infrastructure to various forms of service providers. Such emerging e-utilities providers offering on-demand access to computing resources enable organizations to perform computational jobs spontaneously through other resources in the Grid that are not under the control of the (temporary) user (Foster et al., 2002). Most Grid research has been devoted to the development of hard and software infrastructures so that access to resources is dependable, consistent, pervasive and inexpensive (Foster and Kesselman, 2004). The specification of open standards that defines the interactions between different computing resources across organizational entities hallmarks a significant milestone in this development. With the Open Grid Service Architecture (OGSA), which specifies fundamental middleware components for the Grid, the Grid Community has laid the foundation for future developments. OGSA defines computer and storage resources as well as networks, programs, databases and the like as services, i.e. Corresponding author, schnizler@iism.uni-karlsruhe.de 1

2 network-enabled entities that provide some capability. Using resources as service paves the road for interoperability among heterogeneous computing and application environments (Foster et al., 2002). OGSA and one of its reference implementations, Globus Toolkit 1, provide the technical infrastructure for accessing computational services over the Grid. Computers equipped with a Globus installation can easily access computational resources over the Grid, where access for the user is transparent. Resource owners can become providers by offering their services using standardized interfaces in a global directory. The global directory allocates the services to appropriate service consumers. With the implementation of Grid middleware, institutional arrangements are becoming increasingly important. In essence, current Grid middleware provides insufficient incentive to participate in the Grid. This lack of incentive stems from the fact that most Grid middleware service directories employ batch algorithms which determine the allocation based on either naive heuristics such as firstcome first-serve or on idiosyncratic cost functions (Buyya et al., 2002). From an economic point of view, however, these algorithms are harmful, as they strive to maximize a system-wide performance objective such as throughput or mean response time rather than maximize aggregate value. These batch algorithms can not guarantee that the buyers who value some services will really receive them. Market-based approaches are considered to work well in Grid settings. By assigning a value (also called utility) to their service requests, users can reveal their relative urgency or costs to the service, which is subject to service usage constraints. If the market mechanism is properly defined, users may be provided incentives to express their true values for service requests and offers. This in turn marks the prerequisite for attaining an efficient allocation of services, which maximizes the sum of aggregate valuations (Schnizler et al., 2005). In recent times, the idea of incorporating market mechanisms into Grid technology has increasingly gained attention. Among others, the proposals include the establishment of Open Grid Markets, where either idle or all available resources can be traded (Shneidman et al., 2005; Lai, 2005). Despite this recent interest in market-based approaches, research regarding market mechanisms for Grid resources remains in its infancy. The canon of available market mechanisms only insufficiently copes with the requirements imposed by the Grid. This paper seeks to supply the need for a market mechanism that is Grid-compatible and can thus be used for resource management in federated environments such as an Open Grid Market. The remainder of this paper is structured as follows: the economic properties which the proposed market mechanism should satisfy are presented in section 2. In section 3, comparable efforts in designing mechanisms for Grids and related domains are discussed. Since none of these mechanisms is directly applicable, section 4 tailors a mechanism for the Grid. In section 5, the proposed mechanism is evaluated according to the properties laid out in section 2. Section 6 concludes with a brief summary and an outlook for future research. 1 See for details. 2

3 2 Desirable Properties and Requirements The objective of an adequate market mechanism for the Grid is the efficient and reliable provision of resources to satisfy demand. Economic approaches explicitly assume that users act strategically in a way that does not reveal their true valuations for resources aggregated in services. Incentive compatible mechanisms are accordingly advantageous, as they encourage users to report their true valuations. This allows the mechanism to maximize allocative efficiency (i.e. the sum of the individuals utilities). A critical step in designing markets is to understand the nature of the trading object. This paper considers services which respect resource functionalities (e.g. storage) and quality characteristics (e.g. size), dependencies and time attributes. Relying on services instead of computational resources removes many technical problems. For instance, the resource CPU may technically not be offered without an appropriate amount of hard disk space on the same computer, while a computation service offering CPU cycles already includes the complementary resources. In the remainder of the paper, it is abstracted from those technical details by treating resources and services as synonyms. The following presents the design objectives and the Grid specific requirements for the market mechanism. 2.1 Design Objectives The theoretical basis for designing mechanisms has emerged from a branch of game theory called mechanism design (Milgrom, 2004). Within the scope of practical mechanism design, the primary design objective is to investigate a mechanism that has desirable properties. The following comprises common economic properties of a mechanism s outcome: Allocative efficiency: An allocation is efficient if the sum of individual utilities is maximized. It is assumed that utility is transferable among all participants. A mechanism can only attain allocative efficiency if the market participants report their valuation truthfully. This requires incentive compatibility in equilibrium. Incentive compatibility: A mechanism is incentive compatible, if every participant s expected utility maximizing strategy in equilibrium with every other participant is to report its true preferences (Parkes et al., 2001). Individual rationality: The constraint of individual rationality requires that the utility following participation in the mechanism must be greater or equal to the previous utility. Budget balance: A mechanism is said to be budget balanced if the prices add up to zero for all participants (Jackson, 2002). In case the mechanism runs a deficit, it must be subsidized by an outside source and is therefore not feasible per-se. 3

4 Computational tractability: Computational tractability considers the complexity of computing a mechanism s outcome. With an increasing number of participants, the allocation problem can become very demanding and may delimit the design of choice and transfer rules. It is clear that the objective allocative efficiency meets the general design goal that the mechanism designer wants to achieve, whereas the remaining categories are constraints upon the objective (Neumann, 2004). 2.2 Domain-Specific Requirements In addition to those mechanism properties pertaining to the outcome, the mechanism must also account for the underlying environment. The constraints of the market participants impose very rigid requirements upon the design: Double-sided mechanism: A Grid middleware usually provides a global directory enabling multiple service owners to publish their services and multiple service requesters to discover them. Since a market mechanism replaces these directories, it has to allow many resource owners (henceforth sellers) and resource consumers (buyers) to trade simultaneously. Language includes bids on attributes: Participants in the Grid usually have different requirements for the quality characteristics of Grid services and require these in different time spans. For example, a rendering job could require a storage service with at least 250 GB of free space for four hours in any slot between 9 a.m. and 4 p.m. The Grid community takes these requirements into account by defining service level agreement protocols, e.g. WS-Agreement (Ludwig et al., 2004). To facilitate the adherence of these agreement protocols, a market mechanism fitting the Grid is required to support bids on multiple quality attributes of services as well as their time objectives. Language includes combinatorial bids: Buyers usually demand a combination of different Grid services as a bundle in order to perform a task (Subramoniam et al., 2002). As such, Grid services are complementarities, meaning that participants have super-additive valuations for the services, since the sum of valuations for single services is less than the valuation for the whole bundle (v(a)+v(b) v(ab)). Suppose a buyer requires services for storage, computation and rendering. If any service, e.g. the storage service, is not allocated to him, the remaining bundle has no value for him. In order to avoid this exposure risk (i.e. receiving only a subset of the bundle), the mechanism must allow for bids on bundles. The buyer may want to submit more than one bid on a bundle as well as many that exclude each other. In this case, the resources for the bundles are substitutes, i.e. participants have sub-additive valuations for the services (v(a) + v(b) v(ab)). For instance, a buyer is willing 4

5 to pay a high price for a service during the day and a low price if the service is executed at night. However, this service may only be computed once. To express this, the mechanism must support XOR 2 bids to express substitutes. For the sake of simplicity, a seller s bid is restricted to a set of OR 3 bids. This simplification can be justified by the fact that Grid services are non-storable commodities, e.g. a computation service currently available cannot be stored for a later time. Language includes co-allocation constraints: Capacity-demanding Grid applications usually require the simultaneous allocation of several homogenous service instances from different providers. For example, a large-scale simulation may require several computation services to be completed at one time. Research literature often refers to the simultaneous allocation of multiple homogenous services as co-allocation. A mechanism for the Grid has to enable co-allocations and provide functionality to control it. In this context, two cases must be considered: Firstly, it is desirable to limit the maximum number of service co-allocations, i.e. the maximum number of service divisions. Secondly, it may be logical to couple multiple services of a bundle in order to guarantee that these resources are allocated from the same seller and more importantly will be executed on the same machine. An adequate market mechanism for the Grid must satisfy these requirements stemming from the economic environment and ideally meet the design objectives. 3 Related Work For instance, Waldspurger et al. (1992) and Regev and Nisan (2000) propose the application of Vickrey auctions for allocating homogenous computational resources in distributed systems. Vickrey auctions achieve truthful bidding as a dominant strategy and hence result in efficient allocations. Buyya et al. (2002) were among the first researchers to motivate the transfer of market-based systems from distributed systems to Grids. Nonetheless, they propose classical one-sided auction types which cannot account for combinatorial bids. Wolski et al. (2001) compare classical auctions with a bargaining market, coming to the conclusion that the bargaining market is superior to an auction-based market. This result is less surprising since the authors only consider classical auction formats where buyers cannot express bids on bundles. Eymann et al. (2003) introduce a decentralized bargaining system for resource allocation in Grids. In their simulation the bargaining systems work fairly well; however, bids on bundles are largely ignored. Subramoniam et al. (2002) account for combinatorial bids by providing a tâtonnement process for allocating and pricing Grid resources. Furthermore, Ng et al. (2005) propose repeated combinatorial auctions as a microeconomic resource allocator in distributed systems. Nonetheless, the resources are still considered to be standardized commodities. Standardization of the resources would either imply that the number of resources are limited compared to the number of all possible resources or that 2 A XOR B (A B) means either, A, or B, but not AB 3 A OR B (A B) means, A, B, or AB 5

6 there are many mechanisms which are likely to suffer due to meager participation. Both implications result in rather inefficient allocations. Additionally, state-of-the-art mechanisms widely neglect time attributes for bundles and quality constraints for single resources. Hence, the use of these mechanisms in the Grid environment is considerably diminished. The introduction of time attributes redefines the Grid allocation problem as a scheduling problem. To account for time attributes, Wellman et al. (2001) model single-sided auction protocols for allocating and scheduling resources under different time constraint considerations. However, the proposed approach is single-sided and favors monopolistic sellers or monopsonistic buyers in a way that allocates greater portions of the surplus. Installing competition on both sides is deemed superior, since no particular market side is systematically given an advantage. Demanding competition on both sides suggests the development of a combinatorial exchange. In research literature, Parkes et al. (2001) introduce the first combinatorial exchange as a single-shot sealed bid auction. As payment scheme, Vickrey discounts are approximated. The approach results in approximately efficient outcomes; however, it neither accounts for time nor for quality constraints and is thus not directly applicable to the Grid allocation problem. Counteractively, Bapna et al. (2005) propose a family of combinatorial auctions for allocating Grid services. Although the mechanism accounts for quality and time attributes and enables the simultaneous trading of multiple buyers and sellers, there is no competition on the sellers side as all orders are aggregated to one virtual order. Moreover, the mechanism does not take co-allocation constraints into account. In reviewing the related mechanisms according to the requirements presented in section 2, it is revealed that no market mechanism installs competition on both sides, includes combinatorial bids, allows for time constraints, manages quality constraints, or considers co-allocation restrictions. This paper intends to address these deficiencies by outlining the design of a multi-attribute combinatorial exchange for allocating and scheduling Grid services. 4 MACE: A Multi-attribute Combinatorial Exchange The design of a market mechanism mainly affects two components: (i) the communication language which defines how bids can be formalized and (ii) the outcome determination by means of winner determination and pricing rules. The following auction format follows common assumptions of mechanism design: participants are assumed to be risk neutral, have linear utility functions, as well as independent private valuations and reservation prices. Hence, the sellers reservation prices can be linearly transformed to any partial execution of any bundle. Firstly, a bidding language will be introduced which fits the requirements specified in section 2. Furthermore, a winner determination model (allocation rule) and a family of pricing schemes will be proposed. 6

7 4.1 Bidding Language Meeting the requirements specified in section 2 requires a formal bidding language that enables buyers and sellers to submit combinatorial bids, including multiple attributes and co-allocation constraints: Let N be a set of N buyers and M be a set of M sellers, where n N defines an arbitrary buyer and m M an arbitrary seller. There are G discrete resources G = {g 1,..., g G } with g k G and a set of S bundles S = {S 1,..., S S } with S i S and S i G as a subset of resources. For instance, S i = {g k, g j } denotes that the bundle S i consists of two resources g k and g j, where g k could be a computation service and g j a storage service. A resource g k has a set of A k cardinal quality attributes A gk = {a gk,1,..., a gk,a k } where a gk,j A gk represents the j.th attribute of the resource g k. For instance, in the context of a Grid service, a quality attribute can be the size of a storage service. A buyer n can specify the minimal required quality characteristics for a bundle S i S with q n (S i, g k, a gk,j) 0, where g k S i is a resource of the bundle S i and a gk,j A gk is a corresponding attribute of the resource g k. As such, the minimal required size of a storage service can be denoted by q n (S i, g k, a gk,j), e.g. 200GB. Accordingly, a seller m can specify the maximum offered quality characteristics with q m (S i, g k, a gk,j) 0. For each resource g k S i, a buyer n can specify the maximum number of co-allocations in each time slot with γ n (S i, g k ) 0. This means that a buyer n can limit the number of partial executions for each resource g k. Let γ n (S i, g k ) = D, if the resource g k has no divisibility restrictions, where D is a large enough constant 4. The coupling of two resources in a bundle is represented by the binary variable ϕ n (S i, g k, g j ) where ϕ n (S i, g k, g j ) = 1 if resources g k and g j have to be allocated from the same bundle bid of a seller and ϕ n (S i, g k, g j ) = 0 otherwise. It is assumed that all services offered in a bundle are located on the same machine. Resources in the form of bundles S i can be assigned to a set of maximal T discrete time slots T = {0,..., T 1}, where t T specifies one single time slot. A buyer n can specify the minimum required number of time slots s n (S i ) 0 for a bundle S i. The earliest time slot for any allocatable bundle S i can be specified by e n (S i ) 0 for a buyer n and e m (S i ) 0 for a seller m; the latest possible allocatable time slot by l n (S i ) 0 for a buyer n and by l m (S i ) 0 for a seller m. A buyer n can express the valuation for a single slot for a bundle S i by v n (S i ) 0, i.e. the maximum price for which the buyer n is willing to trade. The reservation price for allocating a single slot of a bundle S i is denoted by r m (S i ) 0, which represents the minimum price for which the seller m is willing to trade. Subsequently, a buyer n can submit a set of XOR bundle bids as an order B n = {B n,1 (S 1 )... B n,u (S j )}, 4 The constant D has to be greater than the total number of seller bids. 7

8 where u is the number of bundle bids in the order and B n,f (S i ) is defined as the tuple B n,f (S i ) =(v n (S i ), s n (S i ), e n (S i ), l n (S i ), q n (S i, g 1, a g1,1),..., q n (S i, g G, a gg,a j ), γ n (S i, g 1 ),..., γ n (S i, g G ), ϕ n (S i, g 1, g 2 ), ϕ n (S i, g 1, g 3 ),..., ϕ n (S i, g 1, g G ),..., ϕ n (S i, g G, g G 1 )). For example, the bid B n,1 (S 1 ) = {1, 4, 2, 10, (3000, 30), (2, 4), 0} with S 1 = {g 1, g 2 }, g 1 = {Computation} with one attribute A g1 = {Speed}, and g 2 = {Storage} with one attribute A g2 = {Space} expresses that a buyer n wants to buy the bundle S 1 and has a valuation of v n (S 1 ) = 1 per slot for it. The requested computation service g 1 should at least be capable of providing 3000 MIPS 5, and the storage service g 2 should have at least 30 GB of available space. The computation service g 1 can split in two parts at the most, while storage service g 2 can run on four different machines simultaneously. Furthermore, the two services have no coupling requirements. The buyer requires four slots of this bundle which must be fulfilled within a time range of slots two and ten. The sellers orders are formalized in a way similar to the buyers orders. However, they do not include maximum divisibility and coupling properties and assume that the number of time slots is equal to the given time range. An order B m is defined as a concatenated set of OR bundle bids B m = {B m,1 (S 1 )... B m,u (S j )}, where u is the number of bundle bids. A single bid B m,f (S i ) for a bundle S i is defined as the tuple B m,f (S i ) = (r m (S i ), e m (S i ), l m (S i ), q m (S i, g 1, a g1,1),..., q m (S i, g j, a gj,a j )). In the following it is assumed that the bid elicitation has already taken place. 4.2 Winner Determination Based upon this notation, the multi-attribute combinatorial winner determination problem with the objective of achieving allocative efficiency can be represented as a linear mixed integer program. For formalizing the model, the decision variables x n (S i ), z n,t (S i ), y m,n,t (S i ), and d m,n,t (S i ) have to be introduced. The binary variable x n (S i ) denotes whether bundle S i is allocated to buyer n (x n (S i ) = 1) or not (x n (S i ) = 0). Furthermore, the binary variable z n,t (S i ) is assigned to a buyer n and is associated in the same way as x n (S i ) with the allocation of S i in time slot t. For a seller m, the real-valued variable y m,n,t (S i ) with 0 y m,n,t (S i ) 1 indicates the percentage contingent of bundle S i allocated to the buyer n in time slot t. For example, y m,n,t (S i ) = 0.5 denotes that 50 percent of the quality characteristics of bundle S i are allocated from seller m to buyer n in time slot t. In the previous example including the storage service with 30GB of free space, a partial allocation of 15 GB from seller m to buyer n in time slot t would lead to y m,n,t (Storage) = 0.5. The binary variable d m,n,t (S i ) is linked with y m,n,t (S i ) and denotes whether the seller m allocates bundle S i to buyer n in time slot t (d m,n,t (S i ) = 1) or not (d m,n,t (S i ) = 0). 5 Million instructions per second (MIPS) is a measure for a computer s processor speed. 8

9 By means of these variables, the multi-attribute winner determination model can be formulated as follows: max v n (S i )z n,t (S i ) r m (S i )y m,n,t (S i ) (1) n N S i S t T m M n N S i S t T s.t. S i S x n (S i ) 1, n N (2) z n,t (S i ) x n (S i )s n (S i ) = 0, n N, S i S (3) t T y m,n,t (S i ) 1, m M, S i S, t T (4) n N The objective function (1) maximizes the surplus V which is defined as the difference between the sum of the buyer s valuations v n (S i ) and the sum of the sellers reservation prices r m (S i ). Assuming bidders are truthful, the objective function reflects the goal of maximizing economic social welfare. The first constraint (2) guarantees that each buyer n can be allocated only to one bundle S i. This constraint is necessary to fulfill the XOR constraint of a buyer order. Constraint (3) ensures that for any allocated bundle S i, a buyer n receives exactly the required slots within the time set T. The constraint has thus to formulated as an equality in to order to be binding. For each time slot t, constraint (4) ensures that each seller cannot allocate more than the seller possesses. The constraints (2-4) consider the basic allocation functionality of the exchange. In designing an adequate mechanism for the Grid, quality characteristics and dependencies between resources must also to be addressed: z n,t (S i )q n (S i, g k, a gk,j) y m,n,t (S i )q m (S i, g k, a gk,j) 0, S i g k d m,n,t (S i ) S i g k m M S i g k m M n N, g k G, a gk,j A gk, t T (5) γ n (S i, g k )z n,t (S i ) 0, S i g k n N, g k G, t T (6) ϕ n (S j, g k, g l )( d m,n,t (S i ) d m,n,t (S i )) = 0, S j g k,g l S i g k S i g l n N, m M, g k, g l G, t T (7) ϕ n (S j, g k, g l )( S j g k,g l d m,n,t (S i )+ d m,n,t (S i ) 2z n,t (S j )) 0, S i g k m M S i g l m M n N, g k, g l G, t T (8) Constraint (5) guarantees that for any allocated bundle in an arbitrary time slot t, all required resources have to be fulfilled in the same slot in at least the demanded qualities. Constraint (6) 9

10 ensures that a resource will be provided by at most γ n (S i, g k ) different suppliers. For simplicity, it is assumed that a good g k with restricted co-allocations is not part of further XOR concatenated bids of the buyer n. Constraints (7) and (8) account for the coupling of two resources. Constraint (7) ensures that two resources must be provided by the same seller, in case they should be coupled. This constraint alone does not suffice for coupling resources requirements, since it would be possible for two sellers to co-allocate a coupled computation service with 3000 MIPS and a storage service with 30 GB in different quality characteristics, e.g. to allocate a computation service with 2998 MIPS and a storage service with 1 GB from one seller, and a computation service with 2 MIPS and a storage service with 29 GB from another. To exclude these undesirable allocations, constraint (8) imposes the restriction that coupled resources cannot be co-allocated. Simplifying the model, this also includes free-disposal resources. For instance, if the computation service with 3000 MIPS and the storage service with 30 GB are allocated from one particular seller as a bundle, another seller cannot allocate a bundle containing a rendering service and another storage service to the same buyer. However, the seller may allocate any bundle without a storage and computation service to the buyer, e.g. the rendering service alone. Furthermore, it is assumed that coupled resources are only part of one particular bundle bid B n,f (S i ) in case a buyer submits two XOR concatenated bids containing coupled resources. The time restrictions of the bids are given by: (e n (S i ) t)z n,t (S i ) 0, n N, S i S, t T (9) (t l n (S i ))z n,t (S i ) 0, n N, S i S, t T (10) (e m (S i ) t) n N y m,n,t (S i ) 0, m M, S i S, t T (11) (t l m (S i )) n N y m,n,t (S i ) 0, m M, S i S, t T (12) Essentially, constraints (9)-(12) indicate that slots cannot be allocated before the earliest and after the latest time slot of either buyer (constraint (9), (10)) or seller (constraint (11), (12)). Finally, the establishment of the relationship between the real valued decision variable y m,n,t (S i ) and the binary variable d m,n,t (S i ) needs to be addressed and the decision variables of the optimization problem have to be defined: y m,n,t (S i ) d m,n,t (S i ) 0, n N, m M, S i S, t T (13) d m,n,t (S i ) y m,n,t (S i ) < 1, n N, m M, S i S, t T (14) x n (S i ) {0, 1}, n N, S i S (15) z n,t (S i ) {0, 1}, n N, S i S, t T (16) 10

11 y m,n,t (S i ) 0, n N, m M, S i S, t T (17) d m,n,t (S i ) {0, 1}, n N, m M, S i S, t T (18) Constraints (13) and (14) incorporate an if-then-else constraint, i.e. if a seller m partially allocates a bundle S i to a single buyer n (y m,n,t (S i ) > 0), the binary variable d m,n,t (S i ) has to be d m,n,t (S i ) = 1 (constraint (13)); otherwise, it has to be d m,n,t (S i ) = 0 (constraint (14)). Finally, the constraints (15) - (18) specify the decision variables of the optimization problem. As multiple optimal solutions may exist, ties are broken in favour of maximizing the number of traded bundles and then at random. Example (Winner determination): Suppose there are two buyers n 1, n 2, two sellers m 1, m 2 and two services g 1 = {Computation} and g 2 = {Storage} with G = {g 1, g 2 } each with single attributes, namely a g1,1 = {Speed} and a g2,1 = {Size}. The buyers and sellers can submit bids on the bundles S 1 = {g 1 }, S 2 = {g 2 }, and S 3 = {g 1, g 2 }. The bundles can be allocated within a time range T = {t 0,..., t 4 } of T = 5 slots. Each buyer submits a set of XOR bids (shown in table 1) and each seller a set of OR bids (see table 2). N S i v n (S i ) q n (S i, g k, a gk,j) e n (S i ) l n (S i ) s n (S i ) γ n (S i, g k ) ϕ n (S i, g i, g j ) n 1 S 3 2 g 1 500, g g 1, g 2 1 n 2 S 1 3 g S 3 2 g 1 300, g g 2 1 Table 1: XOR orders of the buyers. M S i r m (S i ) q m (S i, g k, a gk,j) e m (S i ) l m (S i ) m 1 S 3 1 g 1 500; g m 2 S 3 2 g 1 700; g S 1 2 g Table 2: OR orders of the sellers. For instance, buyer n 2 submits two XOR concatenated bids to bundles S 1 and S 3. The buyer has a valuation of v n2 (S 1 ) = 3 for the bundle S 1 which consists of the computation service g 1. The service must have at least 400 GB of free space and can be allocated between the slots e n2 (S 1 ) = 1 and l n2 (S 1 ) = 4. The buyer requires s n2 (S 1 ) = 3 slots of the service and has no co-allocation restrictions. An optimal solution for the winner determination problem is an allocation of the bundles S 3 and S 1 to the buyers n 1 and n 2 with x n1 (S 3 ) = 1 and x n2 (S 1 ) = 1. Seller m 1 is part of the allocation with bundle S 3 and seller m 2 with bundle S 1. The maximized value V of the winner determination problem is V = 8.6. The corresponding schedule for this allocation is given in table 3. 11

12 M S t 0 t 1 t 2 t 3 m 1 S 3 n 1 : g 1 500, g 2 40 n 1 : g 1 500, g 2 40 m 2 S 1 n 2 : g n 2 : g n 2 : g Table 3: Allocation schema. Buyer n 1 receives bundle S 3 = {g 1, g 2 } from seller m 1 in time slots 0 and 1. The bundle is allocated from one seller and as such the buyer s requested coupling property is satisfied. Buyer n 2 receives requested computation service S 1 = {g 1 } in time slots 1, 2, and 3. Although n 1 does not require the entire allocated space of the storage service g 2, a partial execution of the bundle is not possible due to the computation service g 1 requirements. Bundles can only partially executed as a whole. If an isolated partial execution of single resources in a bundle would be possible, these single resources would also have to be valued. However, as resources may be complementarities or substitutes, valuation and reservation prices for a single resource of a bundle do not exist (Milgrom, 2004). As such, resources of a bundle cannot be partially executed. The outcome of the winner determination problem is allocative efficient as long as buyers and sellers reveal their valuations truthfully. The incentive to set bids according to the valuation is induced by an adequate pricing mechanism introduced in the following subsection. 4.3 Pricing The question how to determine payments made by participants to the exchange and vice versa after the mechanism has determined the winners is referred to as pricing problem. With respect to the objective of achieving an efficient allocation, a pricing scheme based on the well-known Vickrey-Clarke-Groves (VCG) mechanism would attain this objective (Vickrey, 1961; Clarke, 1971; Groves, 1973). Moreover, VCG mechanisms are the only allocative-efficient and incentive compatible mechanisms (Green and Laffont, 1977). Hence, it is logical to turn to the VCG mechanism. The basic idea behind a VCG mechanism is to grant any participant a discount (called Vickrey discount) on that bid. The discount reflects the impact of that bid on the social welfare. A VCG mechanism is efficient, incentive-compatible, and individually rational for participants with quasi linear utility functions. However, Myerson and Satterthwaite (1983) proved that it is impossible to design an exchange which is incentive compatible, (interim) individually rational, and budget balanced that achieves efficiency in equilibrium. The theorem is comprehensive and also applies to the presented mechanism. Obviously, the VCG pricing schema cannot be applied since it runs a permanent deficit requiring outside subsidization. The VCG mechanism can, however, serve as a benchmark VCG Pricing Let N be the set of buyers and M be the set of sellers who are part of the allocation (hence, x n (S i ) = 1 with n N and y m,n,t (S i ) > 0 with m M). The union of both sets is defined as W = N M, where 12

13 N (V n ) v n (S i ) V ICK,n p V ICK,n (S i ) M (V m ) r m (S i ) V ICK,m p V ICK,m (S i ) n m n m Table 4: Vickrey discounts and prices. w W is a participant that is part of the allocation. Let V be the maximized value of the winner determination problem and (V w ) be the maximized value without the participant w. Thus, the Vickrey discount for a participant w can be calculated by V ICK,w = V (V w ). With regards to the Vickrey discount V ICK,w, the price p V ICK,n (S i ) for a bundle S i and a buyer n can be calculated by p V ICK,n (S i ) = v n (S i )s n (S i ) V ICK,n, (19) and the price p V ICK,m (S i ) for a bundle S i and a seller m by p V ICK,m (S i ) = r m (S i ) n N t T y m,n,t (S i ) + V ICK,m. (20) α As sellers submit OR concatenated bids, they can participate in the allocation with multiple bundles. Because the Vickrey discount refers to the seller s overall impact, the discount has to be portioned among all of the seller s successful bids. Thus the discount is divided by α, where α is the number of bundles with which a seller m is participating in the allocation. Applying the VCG pricing scheme to the example presented above (V = 8.6) results in the prices p V ICK,w (S i ) and the discounts V ICK,w shown in table 4 with v n (S i ) = v n (S i )s n (S i ) and r m (S i ) = t T,n N y m,n,t(s i )r m (S i ). As stated by the Myerson-Satterthwaite impossibility theorem, a VCG mechanism is efficient and individually rational but non budget balanced in an exchange. Aggregating the net payments of the example leads to a negative value with ( ) = In this case, the auctioneer has to subsidize the exchange. Naturally, such a situation cannot be sustained for a long period of time, making the VCG mechanism unfeasible. Relaxing the requirement of having an efficient allocation opens up the possibility for a second-best mechanism that is budget balanced. These ideas gave rise to the development of a k-pricing scheme which is presented in the next section K-Pricing The underlying idea of the k-pricing scheme is to determine prices for a buyer and a seller on the basis of the difference between their bids (Sattherthwaite and Williams, 1993). For instance, suppose that a buyer wants to purchase a computation service for $5 and a seller wants to sell a computation service for at least $4. The difference between these bids is π = $1, where π is the surplus of this transaction that can be distributed among the participants. For a single commodity exchange, the k-pricing scheme can be formalized as follows: let v n (S i ) = a be the valuation of a buyer n and r m (S i ) = b be the reservation price of the buyer s counterpart m. It 13

14 is assumed that a b, i.e. the buyer has a valuation for the commodity which is at least as high as the seller s reservation price. Otherwise no trade would occur. The price for a buyer n and a seller m can be calculated by p(s i ) = ka + (1 k)b with 0 k 1. In regard to the above-mentioned bids for the computation service g 1, the resulting price for both participants is p(g 1 ) = (1 0.5) 2.4 = 5.7 for k = 0.5. In the following, it is assumed that the number of buyers equals the number of sellers. Therefore, k is set to k = 0.5 since it favors neither the buyers side nor the sellers side. The k-pricing schema can also be applied to a multi-attribute combinatorial exchange: in each time slot t in which a bundle S i is allocated from one or more sellers, the surplus generated by this allocation is distributed among a buyer and the sellers. Suppose a buyer n receives a computation service S 1 = {g 1 } with 1000 MIPS in time slot 4 and values this slot with v n (S 1 ) = 5. The buyer obtains the computation service S 1 = {g 1 } by a co-allocation from seller m 1 (400 MIPS) with a reservation price of r m1 (S 1 ) = 1 and from seller m 2 (600 MIPS) with r m2 (S 1 ) = 2. The distributable surplus of this allocation is π n,4 (S 1 ) = 5 (1 + 2) = 2. Buyer n gets k π n,4 (S 1 ) of this surplus, i.e. the price buyer n has to pay for this slot is p k,n,4 (S i ) = v(s 1 ) kπ n,4 (S 1 ). Furthermore, the sellers have to divide the other part of this surplus, i.e. (1 k)π n,4 (S 1 ). This will be done by considering each proportion a seller s bid has on the surplus. In the example, this proportion o m,n,t (S i ) for seller m 1 is o m1,n,4(s 1 ) = 1 3 and for seller m 2 is o m2,n,4(s 1 ) = 2 3. The price a seller m receives for a single slot 4 is consequently calculated as p k,n,4 (S i ) = r m (S 1 ) + (1 k)π n,4 (S 1 )o m,n,4 (S 1 ). Expanding this scheme to a set of time slots results in the following formalization: let π n,t (S i ) be the surplus for a bundle S i of a buyer n with all corresponding sellers for a time slot t: π n,t (S i ) = z n,t (S i )v n (S i ) m M S l S y m,n,t (S l )r m (S l ) (21) For the entire job (i.e. all time slots), the price for a buyer n is calculated as p k,n (S i ) = x n (S i )v n (S i )s n (S i ) k t T π n,t (S i ). (22) This means that the difference between the valuation for all slots (v n (S i )s n (S i )) of the bundle S i and the k-th proportion of the sum over all time slots of the corresponding surpluses is determined. The price of a seller m is calculated in a similar way: First of all, the proportion o m,n,t (S i ) of a seller m allocating a bundle S i to the buyer n in time slot t is given by o m,n,t (S i ) = m M y m,n,t (S i )r m (S i ) S l S y m,n,t(s l )r m (S l ). (23) Having computed π n,t (S i ) and o m,n,t (S i ), the price a seller receives for a bundle S i is calculated as: p k,m (S i ) = y m,n,t (S i )r m (S i ) + (1 k) o m,n,t (S l )π n,t (S l ). (24) n N t T n N S l S t T Applying this pricing scheme to the example presented above results in the prices given table 5 with v n (S i ) = v n (S i )s n (S i ) and r m (S i ) = t T,n N y m,n,t (S i )r m (S i ). 14

15 In this case, the exchange does not have to subsidize the participants since it fulfills the budget balance property in a way that no payments towards the mechanism are necessary. Hence, the k-pricing schema qualifies as a candidate pricing schema for the Grid. Since the Myerson-Satterthwaite impossibility theorem is strict, the implications of the pricing schema on the allocation need to be investigated. N v n (S i ) p k,n (S i ) M r m (S i ) p k,n (S i ) n m n m Table 5: Prices using k-pricing with k = Evaluation The presented auction formats are evaluated by means of a stochastic simulation. The winner determination problem is solved using the optimization engine CPLEX The evaluation assesses the impact of the auction schemas on the outcome requirements defined in section Data Basis As a data basis, a random bid stream including Decay distributed bundles is generated. The decay function has been recommended by Sandholm (2002) because it creates hard instances of the allocation problem. At the beginning, a bundle consists of one random resource. Afterwards, new resources are added randomly with a probability of α = This procedure is iterated until resources are no longer added or the bundle already includes the same resource. The effects that can be obtained by the Decay distribution will be amplified. Hence, the Decay function is used to create a benchmark for upper bounds of the effects. As an order, buyers and sellers submit a set of bids on 10 different bundles, where a bundle is Decay-distributed from 5 possible resources. Each resource has two different attributes drawn from a uniform distribution in line with most of the quality characteristics of computer resources described by Kee et al. (2004). The time attributes are each uniformly distributed where the earliest time slot has a range of [1..3], the latest time slot is in [4..6], and the number of slots lies in [1..3]. Forty percent of the buyers bids have co-allocation restrictions. Sixty percent of the resources contained in these bundles have uniform distributed restrictions on the maximum number of co-allocations. The number of resources which have to be allocated from the same machine (coupling) is also drawn from an uniform distribution. The corresponding valuations and reservation prices for a bundle are drawn from the same normal distribution, whereas the number of resources in a bundle and the quality characteristics affect the mean and variance of the distribution. This means the probability is greater that a buyer has a higher valuation for a storage service with 200 GB than for a storage service with 10 GB of space. In any 6 CPLEX is commercial product and is currently the state of the art optimization engine. 15

16 problem instance, new orders for buyers and sellers are randomly generated. Subsequently, demand and supply are matched against each other, determining the winning allocation and corresponding prices. In total, 120 different problem instances are generated, where a problem consists of 10 or 20 participants. The proportion of buyers and sellers is equally selected out of {10/10, 8/12, 12/8, 5/5, 1/4, 4/1}. In each instance, the buyers and sellers each submit one single bid. 5.2 Truthful Bidders In the first treatment, it is assumed that bidders are truthful. The average results of the treatment regarding the budget balance and participant utilities are shown in table 6. Price Mechanism Budget Utility Buyers Utility Sellers Utility VCG k-price Table 6: Average Utility and Budget using Decay distributed bids. As the Myerson-Satterthwaite theorem suggests, the budget using a VCG auction is always negative. The mechanism requires subsidization from outside the system. Note that these subsidies amount to half the total utility. Obviously, incentive compatibility is extremely expensive, as these subsidizations are required to motivate the buyers and sellers to truthfully report their valuations and reservation prices. By applying a k-price auction, the budget deficit can be converted into a mechanism, which is independent from outside subsidies. Regarding its formal construction, the k-price auction is budget balanced; however, the average total utility shrinks by the amount of the budget deficit (including a small approximation increment). The share between buyers and sellers surplus remains unchanged. In the k-pricing schema, however, the utilities of the buyers and sellers are significantly smaller than in the VCG auction and denote a measure for the incentive compatibility loss. As table 6 shows, the sellers utilities in a VCG auction are higher than those of the buyers. Regarding the data basis, sellers expect to offer more available time slots than buyers request. Buyers submit bids on a number of time slots s n (S i ) between a time range e n (S i ) and l n (S i ). The number of requested slots is less than the number of allocatable slots, i.e. s n (S i ) l n (S i ) e n (S i ). A seller, however, offers a service for the entire time range. As the time ranges of the buyers and sellers are drawn from the same distribution, the number of tradeable slots a seller offers is greater than the buyer s requested slots. The greater the number of tradeable slots, the higher the impact of the bid in the VCG mechanism. As such, the sellers average discounts are higher than those of the buyers; therefore the sellers have a higher average utility than the buyers. Because the simulation assumes truthful bidding, this rather naive analysis is only accurate for the VCG mechanism. If the pricing rule is changed for example from the VCG mechanism to the k-price auctions strategic bidding behavior may change since the k-price auction is not incentive 16

17 compatible. 5.3 Manipulating Bidders In a second treatment, it is assumed that buyers and sellers using the k-pricing schema misrepresent their true valuation or reservation price respectively. Only simple misrepresentations by β% are considered, where λ% of the buyers and sellers reduce or increase their reported values by β%. Instead of observing only symmetric Nash-equilibria as in the analysis by Parkes et al. (2001), where buyers and sellers either misrepresent their valuation or reservation price by 0 or by β%, the ratio of misrepresenting buyers and sellers to the total number of participants is also varied. A ratio of β = 50%, for instance, denotes that 50% of the buyers and sellers misrepresent their valuations and reservation prices by λ%, while the other 50% report truthfully. By exploring the joint strategy space (i.e. varying the share of misrepresentative and truthful participants as well as the percentage of misrepresentation) the average efficiency loss for the k-pricing schema can be analyzed: Let n N be the buyers and m M be the sellers who are part of the allocation in a truthful bidding scenario, i.e. λ = β = 0. Furthermore, let ˆn ˆN be the buyers and ˆm ˆM be the sellers of the allocation having manipulating bidders (λ 0, β 0). Subsequently, the efficiency loss EL can be calculated as the difference between the true preferences of the participants who are part of the allocation in both the truthful and manipulating scenarios: EL = n N S i S ( ˆn ˆN s n (S i )v n (S i ) + S i S sˆn (S i )vˆn (S i ) + m M n N S i S t T ˆm ˆM ˆn ˆN S i S t T y m, n,t (S i )r m (S i ) y ˆm,ˆn,t (S i )r ˆm (S i )). Figure 1 shows the percentage efficiency loss and the percentage decrease of the number of successful bidders with a varying number of manipulating participants as a function of the manipulation factor. For instance, if β = 40% of the participants manipulate their valuations and reservation prices by λ = 6%, the resulting efficiency loss is 15%. It is apparent that the efficiency loss is higher when more participants use manipulation and thereby increase the manipulation factor. With a manipulation factor greater than λ = 25%, the efficiency curves stagnate. This stagnation results from the fact that either non-manipulating bidders are the only participants of the allocation (in case of β < 100%) or no one is allocated further (in case of β = 100%). The valuation and reservation prices with a manipulation factor λ > 25% are so low respectively so high that no bid from the manipulating participants is allocated. This means that no manipulated offer for a bundle can be matched with any manipulated request. This effect is supported by the percentage decrease of the number of successful bidders by manipulating participants. For instance, if β = 20% of the participants manipulate their bids by λ = 21%, the number of allocated bids is 17% smaller than in the non-manipulating scenario. Furthermore, if all participants are manipulating by λ = 30%, the size decreases by 100%, i.e. the set of allocated participants is empty. The curves recurrently stagnate with a manipulation factor greater than λ = 25%. In another analysis, the aggregated average utility gain of manipulating participants is measured. 17

18 Effiency loss ,1 0,2 0,4 0,6 0,7 0, ,03 0,06 0,09 0,12 0,15 0,18 0,21 0,24 0,27 0,3 Decrease of successful participants ,1 0,2 0,4 0,6 0,7 0, ,03 0,06 0,09 0,12 0,15 0,18 0,21 0,24 0,27 0,3 Manipulation factor Manipulation factor Figure 1: Efficiency loss (left) and decrease of the number of successful bidders (right) by manipulating participants. This gives information on whether or not the total utility of participants can be improved through manipulation. The utility which participants can gain by manipulation only depends on the prices they have to pay and is thus calculated as UG = ˆn ˆN S i S ˆpˆn (S i ) + ˆm ˆM S i S ˆp ˆm (S i ) ( n N p n (S i ) + S i S m M S i S p m (S i )), where ˆN and ˆM are the sets of buyers and sellers who are part of the allocation with manipulating participants and ˆpˆn (S i ) and ˆp ˆm (S i ) are the corresponding prices. Sets N and M denote the participants of the allocation in the non-manipulated scenario with p n (S i ) and p m (S i ) as their prices. The percentage utility gain of manipulating participants in the k-price auction is shown in figure 2. The total utility of the participants can be increased by manipulation; at the highest point where all participants manipulate by λ = 6%, the utility gain is more than 26% compared to the non-manipulating scenario. Moreover, small manipulation factors between λ = 1% and λ = 9% always result in a positive utility gain. However, participants have a negative utility gain when the manipulation factor is greater than λ = 13%. In the worst scenario, the utility loss is 100% if all participants manipulate by λ = 30%. In summary, the simulation has shown it reasonable to believe that participants will not strongly deviate from their truth valuations and reservation prices. Although participants average utility gain can be improved through manipulation, the participants increasingly risk not being executed in the auction (cf. figure 1). This risk actually increases the more participants use manipulation. The simulation result suggests that the k-pricing schema has accurate incentive properties resulting in fairly mild allocative efficiency losses. As such, the pricing schema is a practical alternative to the VCG mechanism and is highly relevant for an application in the Grid. 18

19 Utility gain ,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 Participants manipulating 0,3 0,21 0,12 0, Manipulation factor Figure 2: Utility gain using k-pricing. 5.4 Computational Tractability As denoted in section 2, a computational tractable outcome determination is required. However, the multi-attribute combinatorial winner determination problem (MWDP) is N P-complete: consider the special case of the MWDP with multiple buyers, one seller, no quality attributes, one time slot, and no coupling and maximal divisibility restrictions. In this case, the problem is equivalent to the combinatorial allocation problem (CAP) formulated by Rothkopf et al. (1998). CAP is equivalent to the set-packing problem (SPP) on hypergraphs (Rothkopf et al., 1998) which is known to be N P- complete (Karp, 1972). Since CAP can be reduced to the MWDP, the MWDP is also N P-complete. Due to the N P-completeness of the winner determination problem, the auction schema is computationally intractable in large-scaled scenarios. For settings with a limited number of participants, however, the auction schema seems practicable. Ideally, an upper boundary of participants can be defined for which the problem is still computationally tractable within a meaningful timeframe. In the context of trading services, a meaningful timeframe is consistent with the maximum time limit that an allocation process may last. Experiences suggest that an allocation process that is shorter than 4 minutes is adequate. Preliminary studies with the auction schema confirmed that solving the winner determination problem can become computationally intractable, even in settings with less than 100 participants. This implies that exact solutions of the winner determination problem may require too much computation time and cannot be directly applied for the Grid market. To remedy this obstacle, we have to rely on approximations. A common way is to employ algorithms that compute solutions of the winner determination problem within a threshold r away from optimality, say 2%. Such deviations from optimal solutions are often conceived to be tolerable. However, in some cases even finding solutions within the specified threshold of 2% becomes impossible within a meaningful timeframe. In those cases the approximations need to have a termination condition that forces the algorithm to return the best solution found within a predefined time range. Combinatorial auction experiments have shown that these so-called anytime algorithms perform fairly well. For instance, Andersson et al. (2000) and 19

20 Participants Time Limit 2min Time Limit 4min No Time Limit µ 2 σ 2 CV 2 µ 4 σ 4 CV 4 µ σ CV s 25.34s s s s s s 50.09s s s s 598.4s s 42.74s s s s s s 24.06s s s s s s 13.73s s s s s s 0s s s s 0.8 Table 7: CPU time required to solve the problem. Sandholm et al. (2005) report that feasible and approximate solutions can be determined in a fraction of time that would be required to find and prove an optimal solution. The applied standard solver CPLEX provides these functionalities. For our computational tractability analysis, we perform a series of simulations with anytime approximations for computing the solutions. The 4 minutes introduced above will serve as maximum timeframe. To compare the results, we also employ a simulation setting with a more optimistic limit of 2 minutes. As the anytime algorithms diverges from the theoretical optimum, we need a measure that captures the welfare loss imputed to the approximation. Nevertheless, computing the theoretical optimum may take very long, so that we use benchmark solution that lies within a 2% gap from the optimum. If CPLEX cannot find any feasible solution within the specified timeframe, the problem instance is rated as intractable. For instance, if no feasible solution is found within 2 minutes, the problem is measured as an empty allocation with welfare of 0 and runtime of 2 minutes. This way, we include those hard instances in our analysis. In order to perform the simulation, nearly the same random order flow is used as described above. However, the time attributes are widened in order to generate harder and more realistic settings. The earliest time slots are within a range of [1..5], the latest time slots within [6..10], and the number of slots within a range of [1..5]. In the simulation, the proportions of buyers and sellers are equal and the number of participants is varied. In total, 20 treatments are computed and the results are averaged. Table 7 summarizes the runtime results of the simulations with 2 minutes time limit, 4 minutes time limit, and benchmark solution with no time limit. 7 The table shows the mean µ, the standard deviation σ, and the coefficient of variation CV = σ/µ of the CPU times required to solve the winner determination problems. For example, with 200 participants the average runtime with 2 minutes time limit is µ 2 = 67.6 seconds, with 4 minutes limit µ 4 = seconds, and without a time limit µ = seconds. Note that the differences in the runtime of settings with equal number of participants are caused by the underlying time limits. For 200 participants, several instances require more than 2 minutes, some even more than 4 minutes to find a solution within the specified gap. As CPLEX terminates after the predefined timeframe, the average runtime in the 2 minutes setting is less than 7 The analysis was performed on a Pentium XEON with 3.2GHz and 2GB ram using CPLEX 9.1 with a MIP gap of