Automated Haggling: Building Artificial Negotiators Nick Jennings Intelligence, Agents, Multimedia (IAM) Group, Dept. of Electronics and Computer Science, University of Southampton, UK. nrj@ecs.soton.ac.uk http://www.ecs.soton.ac.uk/~nrj 1
2 Why do we need software that can negotiate?
3 Cheapest Phone Call Call to New York
4 Cheapest Phone Call Phone company 1 Call to New York Phone company 2 Phone company 3
5 Cheapest Phone Call Phone company 1 What price? Call to New York What price? Phone company 2 What price? Phone company 3
6 Cheapest Phone Call 1.9 Phone company 1 What price? Call to New York What price? 2.6 Phone company 2 1.4 What price? Phone company 3
7 Cheapest Phone Call 1.9 Phone company 1 What price? Call to New York What price? 2.6 Phone company 2 1.4 What price? Phone company 3
8 Virtual Organisations Local tourist attractions Travel Companies Hotels
9 Virtual Organisations
10 Virtual Organisations
What is the software model? Software acts on owner s behalf: capable of autonomous action in pursuit of its objectives make commitments on behalf of owner Software interacts with other entities to help achieve its objectives negotiating to make agreements 11
12
13 I. Negotiation Issues Talk Outline II. III. IV. One-to-Many Negotiation One-to-One Negotiation Argumentation & Persuasion V. Conclusions and Future Work
14 I. Negotiation Issues Talk Outline II. III. IV. One-to-Many Negotiation One-to-One Negotiation Argumentation & Persuasion V. Conclusions and Future Work
15 What is Negotiation? Process by which group of agents communicate with one another to try and come to a mutually acceptable agreement on some matter Oxford English Dictionary
Important because agents are autonomous: to influence an acquaintance, needs to be convinced it should act in a particular way since it represents a different stakeholder achieved by: making proposals trading options offering concessions and (hopefully) coming to an agreement 16
17 Negotiation Components: Protocols Specification of legal actions rules of the game permissible types of participants e.g. negotiators & third parties negotiation states e.g. accepting bids, negotiation closed events that cause state transitions e.g. no more bidders, bid accepted valid actions of participants take it or leave it (accept/reject) suggest changes (counter-propose)
Negotiation Components: Agreement Object Issues over which agreement is required: price quality volume delivery date.. 18
19 Negotiation Components: Agent s Strategy Decision making apparatus agent employs to achieve its negotiation objectives typically aims to maximise its benefit Shaped by: negotiation protocol structure of agreement object varies from: simply bidding true valuation complex speculation and argumentation
Maximising Benefit Preferences for world states captured by utility function Toss a fair coin: receive 20 if heads, lose 10 if tails U(heads) = 20 U(tails) = -10 Expected utility of action combine utility with outcome probabilities EU(take-bet) = (0.5 x 20) + (0.5 x -10) = 5 EU(not-take-bet) = 0 20
Maximising Benefit (cont.) Dealing with multiple attributes e.g. deciding which car to buy assuming mutual preference independence between attributes: U(make, colour, year) = U(make) + U(colour) + U(year) 21
22 BT s Provide Customer Quote Process (Jennings et al.) Provide Quote Customer Service Agents Service producer ServiceName Service consumer
BT s Provide Customer Quote Process (Jennings et al.) Provide Quote Customer Service Agents VetCustomer Service producer ServiceName Service consumer A B C D... n Customer Vetting Agents 23
BT s Provide Customer Quote Process (Jennings et al.) Cost&DesignNetwork Design Agents Provide Quote Customer Service Agents ProvideLegalAdvice Legal Agents VetCustomer Service producer ServiceName Service consumer A B C D... n Customer Vetting Agents 24
BT s Provide Customer Quote Process (Jennings et al.) Survey Agents SurveyPremises Cost&DesignNetwork Design Agents Provide Quote Customer Service Agents ProvideLegalAdvice Legal Agents VetCustomer Service producer ServiceName Service consumer A B C D... n Customer Vetting Agents 25
In ADEPT, Interaction = Negotiation Two agents must agree about conditions under which service will be executed price quality start and end times 26
27 I. Negotiation Issues Talk Outline II. III. IV. One-to-Many Negotiation (auctions) One-to-One Negotiation Argumentation & Persuasion V. Conclusions and Future Work
One-to to-many Negotiation (Vulkan and Jennings) Reverse auction: One buyer, many sellers Customer Service Agents VetCustomer A B C D... n Customer Vetting Agents 28
Auction-Based Models Used throughout history In Babylon in about 500BC, annual auctions of women for marriage! Efficient for 1:many negotiations quickly find partner with highest valuation 29
Auction-Based Models But which protocol? English: auctioneer raises price until only one taker remains Dutch: auctioneer continuously lowers price until bidder accepts price First-Price Sealed Bid: bids made simultaneously & highest bid wins auction and that bid is price paid Vickrey: same as first-price sealed bid, except winner pays second highest price 30
31 Underlying Assumptions Agents are utility maximisers Private vs. Common value goods private: value depends only on agent s preferences (e.g. cake to eat by self) common: value determined entirely by others values (e.g. bank notes) VetCustomer service is private value (information cannot be resold) Opponents valuations are not known
Coping with Uncertainty about Opponents Valuations Model (likely) beliefs of opponent Predict behaviour accordingly Difficult because may end up with infinite regress Construct belief-independent mechanism Agent s strategy is independent of its beliefs about opponents 32
Dominant Strategy Yields expected payoff higher than others whatever opponent s behaviour B1 Player B B2 Player A A1 A2 6 4 7 2 3 3 5 0 Dominant strategy for player A No dominant strategy for player B want to design protocol based upon agents selecting dominant strategies (if possible) 33
Which Protocol? English Dutch First-Price Sealed Bid Vickrey All mechanisms are revenue equivalent (for buyer) for private value objects English & Vickrey rely on agents playing dominant strategy English since works better in multi-dimensional contexts 34
35 Multi-Dimensional English Auction Initiation Buyer announces declared utility function (U) maximum price will pay (P) minimum acceptable price time will wait between offers (T) minimum percentage increase that next offer must exceed (X) can lie about U and P Auction Sellers submit offers (on all dimensions) can lie and speculate Offer accepted if: protocol has not terminated offer exceeds last accepted one by X% Buyer makes acceptable bid public Terminates T seconds after last acceptable offer
Analysis of the Protocol Can prove that: no point in buyer lying about U and P truth telling is dominant strategy no point in seller speculating about competitors bid X% more than current price, up to reservation level is the dominant strategy in buyer s best interest to use suggested protocol no other protocol (either auction or direct negotiation) can yield a better result for it 36
37 I. Negotiation Issues Talk Outline II. III. IV. One-to-Many Negotiation One-to-One Negotiation (heuristic model) Argumentation & Persuasion V. Conclusions and Future Work
Shortcomings of Game Theoretic Approaches Based on assumption of perfect rationality unbounded computational power complete knowledge Want to have flexibility in negotiation protocol cannot impose strategy in advance since it may depend on dynamic context 38
39 BT s Provide Customer Quote Process One buyer, one seller Customer Service Agents Cost&DesignNetwork Survey Agents Design Agents SurveyPremises ProvideLegalAdvice Legal Agents
Agent s Negotiation Architecture Evaluation decision (Faratin, Sierra and Jennings) Negotiation mode Trade-Off Issue Manipulation Concede 40
41 Evaluation Reasoner Rank incoming (counter) proposal (P) V(P) = ( i P) Σ w i f i (mutual preference independence) Compute offer would have returned (R) Decision: REJECT if: negotiation constraints violated ACCEPT if: V(P) V(R) [subject to specified constraints being satisfied] COUNTER: otherwise
Agent s Negotiation Architecture Evaluation decision Negotiation mode Defined by set of heuristics Trade-Off Issue Manipulation Concede 42
Agent s Negotiation Architecture Evaluation decision Negotiation mode Don t want to give ground Trade-Off Issue Manipulation Concede Select proposal most similar to opponent s last offer uses fuzzy similarity (Faratin et al, 2000) 43
Agent s Negotiation Architecture Evaluation decision Negotiation mode Want to break a deadlock Trade-Off Issue Manipulation Concede Add issue with greatest value to agent Remove issue to make proposal most similar to opponent s last offer (Faratin et al, 1999) 44
45 Agent s Negotiation Architecture Evaluation decision Negotiation mode Willing to give ground Trade-Off Issue Manipulation Concede
On Conceding Utility Decay functions Reservation Value Time Deadline 46
Enacting Concessions High Desirability Time until needed Num. suppliers Resource consumed Negotiation Drivers Low Negotiation Reasoner Strategic Choice: - balance drivers - determine key issues - monitor progress - modify stance Service Name Cost Start Time End Time Quality of Service Penalty for Violation Tactical Enactment - instantiate SLA values 47
48 Strategic Reasoning Issues Cost Start Time End Time Quality of Service Penalty for Violation w1 w2 w3 Drivers Time until deadline Number providers Behaviour of opponent Resources consumed Varied according to ongoing negotiation history
Tactical Reasoning wrt deadline wrt number providers wrt opponent s behaviour Price Price Price Time Num. providers Concession rate PRICE =f( w1 * v1, w2 * v2, w3 * v3) from strategic reasoner 49
50 Evaluation Negotiation tournaments in which agents with different styles interact Boulware: stay firm Conceder: give ground quickly Linear: concede at constant rate Tit-for-Tat: mirror opponent s behaviour Outcome characteristics depend on environment and agent population Agents with flexible strategies more successful More deals Better deals
51 I. Negotiation Issues Talk Outline II. III. IV. One-to-Many Negotiation One-to-One Negotiation Argumentation & Persuasion V. Conclusions and Future Work
52 Object-Level Negotiation Feedback through modification of negotiation object: Counter-Proposal: (more favourable) alternative generated in response to a proposal Recipient infers constraints and objectives from way original proposal is re-constituted Agents cannot: justify their negotiation stance may be compelling reason for behaviour e.g. requesting illegal or impossible act persuade opponent of their stance e.g. seller promotes importance of car safety, buyer threatens to go elsewhere
Exploiting Meta-Information Add arguments to support stance many argument types: threats, reward, appeals, etc. role of argument: modify recipient s region of acceptability or its acceptance threshold benefits: increase likelihood of agreement persuading opponent to accept deal would not previously have countenanced increase speed of agreement accept position on given issue and cease negotiating over it Argumentation-based negotiation 53
Argumentation Basics General schema: δ - (α, G) δ: set of formulae for building arguments -: suitable consequence relation α: proposition for which argument is made G: set of formulae used to infer α Argument (α 1, G 1 ) rebuts (α 2, G 2 ) if α 1 attacks α 2 (goto-beach, nice-day) (~goto-beach, work-to-do) Argument (α 1, G 1 ) undercuts (α 2, G 2 ) if exists s G 2 such that α 1 attacks s (~nice-day, raining-outside) (goto-beach, nice-day) 54
55 Implemented using Multi-Context Systems (Sabater, Parsons, Sierra and Jennings) Planner Goal Manager Social Manager Resource Manager Rebutting Module Undercutting Module
Implementing a Module (the Goal Manager) Goal manager G DONE R Resource list MONITOR ASK P (active) Plan list PLAN Communication unit CU RESOURCE 56
57 Sample Bridge Rules ASK: G:goal(X),G:not(done(X)),R:not(X), P:not(plan(X,Z), G:not(done(ask(X))) CU:ask(G,All,goal(X),[]), G:done(ask(X)) MONITOR: G:goal(X),R:not(X),P:plan(X,P) G:monitor(X,P)
58 S + S + > Hang Mirror + + > Hang Mirror + + > Hang Picture Hang Picture Hang Mirror Agents are trusting and cooperative
59 S + S + > Hang Mirror + + > Hang Mirror + + > Hang Picture I know agent B has a nail
60 S + S + > Hang Mirror + + > Hang Mirror + + > Hang Picture
61 S + S + > Hang Mirror + + > Hang Picture + + > Hang Mirror REBUT + + > Hang Mirror
UNDERCUT 62 S + S + > Hang Mirror + + > Hang Mirror + + > Hang Picture S + + > Hang Mirror + S + > Hang Mirror
63 S + S + > Hang Mirror + + > Hang Picture + + > Hang Mirror + S + > Hang Mirror
64 SS + S + > Hang Mirror + + > Hang Picture + + > Hang Mirror + S + > Hang Mirror
65 SS + S + > Hang Mirror + + > Hang Picture + + > Hang Mirror + S + > Hang Mirror
66 SS + S + > Hang Mirror + + > Hang Picture + + > Hang Mirror + S + > Hang Mirror
67 SS + S + > Hang Mirror + + > Hang Picture + + > Hang Mirror + S + > Hang Mirror S
68 S S + S + > Hang Mirror + + > Hang Picture + + > Hang Mirror + S + > Hang Mirror
69 S S + S + > Hang Mirror + + > Hang Picture + + > Hang Mirror + S + > Hang Mirror
70 I. Negotiation Issues Talk Outline II. III. IV. One-to-Many Negotiation One-to-One Negotiation Argumentation & Persuasion V. Conclusions and Future Work
Conclusions Automated negotiation is key enabler for wide range of applications enables software to respond to prevailing situation in an uncertain and dynamic world Diverse methods of approach eclectic stance required for realising full potential 71
Major Challenges Obtaining user s trust Elicitation of negotiation knowledge from users Producing predictable negotiation behaviour in realistic domains Knowing which techniques to use in which circumstances negotiation cook book 72
73 Collaborators
References P. Faratin, C. Sierra and N. R. Jennings (2000) Using similarity criteria to make negotiation trade-offs Proc. 4th Int. Conf on Multi-Agent Systems, Boston, 119-126. P. Faratin, C. Sierra, and N. R. Jennings (1998) Negotiation decision functions for autonomous agents Int. J. of Robotics and Autonomous Systems 24 (3-4) 159-182. N. R. Jennings, P. Faratin, A. R. Lomuscio, S. Parsons, C. Sierra and M. Wooldridge (2001) Automated negotiation: prospects, methods and challenges Int. J. of Group Decision and Negotiation 10 (2) 195-215. N. R. Jennings, P. Faratin, T. J. Norman, P. O Brien and B. Odgers (2000) Autonomous agents for business process management Int. J. of Applied Artificial Intelligence 14 (2) 145-190. S. Parsons, C. Sierra and N. R. Jennings (1998) Agents that reason and negotiate by arguing J. of Logic and Computation 8 (3) 261-292. J. Sabater, C. Sierra, S. Parsons and N. R. Jennings (2001) Engineering executable agents using multi-context systems J. of Logic and Computation. (to appear) C. Sierra, P. Faratin and N. R. Jennings (1997) A service-oriented negotiation model between autonomous agents Proc. European Wshop on Modelling Autonomous Agents in a Multi-Agent World, Ronneby, Sweden, 17-35. C. Sierra, N. R. Jennings, P. Noriega, and S. Parsons (1997) A framework for argumentation-based negotiation Proc. 4th Int. Workshop on Agent Theories, Architectures and Languages, Rode Island, USA, 177-192. N. Vulkan and N. R. Jennings (2000) Efficient mechanisms for the supply of services in multi-agent environments Int J. of Decision Support Systems 28 (1-2) 5-19. 74