Endowing a virtual assistant with intelligence: a multi-paradigm approach

Transcription

1 Endowing a virtual assistant with intelligence: a multi-paradigm approach Josefa Z. Hernández, Ana García Serrano Department of Artificial Intelligence Technical University of Madrid (UPM), Spain {phernan,agarcia}@dia.fi.upm.es AMEC SIG meeting, 4th February 2003, Barcelona

2 A virtual assistant on risk management: The VIP-Advisor project Target virtual assistant Target virtual assistant: Listens, understands and responds in the context of a conversation maintained via speech and/or with mouse clicks. Requires a multi-paradigm approach combining speech recognition and synthesis, on-line translation and natural language processing with a 3D avatar. Competent advice through the combination of a knowledge-based assistant on risk management with additional case-based reasoning and information retrieval support. Assistant intelligence supported on conversation retrieval: current and past user-system dialogues are the basis to expand a user query and retrieve the dialogue contribution.

3 A virtual assistant on risk management: The VIP-Advisor project Visualisation Visualisation Online Translation Risk Management Assistance Speech Synthesis Speech Recognition Natural Language Analyser Dialog-based Interaction Risk Management Assistance Intelligence Language Processing Natural Language Generator Case-based Reasoning The present: Winterthur s online assistant Intelligence The VIP-Advisor vision

4 VIP-Advisor cognitive architecture The agent-based view offers a potential support for distributed heterogeneous software using agent as an abstraction concept key The VIP-Advisor agent-based architecture reflects the conceptual and functional distribution of the decision support INTERFACE AGENT INTERACTION AGENT INTELLIGENT AGENT Graphical Interface Updated Dialogue Memory Risk manager Internet NL processor Speech processor Session model Discourse maker INTELLIGENT AGENT 3D Avatar Dialogue manager Case-based reasoner

5 VIP-Advisor: UPM role communicative acts (mouse clicks) text voice Speech recognition text Natural language interpreter Communicative acts Interaction Manager Risk Manager & CBR text voice Speech synthesis text Natural language generator Communicative acts communicative acts (gestures indications)

6 Communicative acts: representing intentions and meanings Communicative act: Atomic piece to represent semantic and pragmatic of a communicative expression (Searle s speech acts) Classes of communicative acts (for the moment): directive acts (request, command), representative acts (informative, confirmative), authoritative acts, courtesy acts, null Request Type (kind of request) data explanation choice Matter (what is requested) feature nextaction process permission qualify process definition nextaction type format Subject (about what) system user task print work task business activity system report Content general null business sector end repeat save null privacy omega consulting repercussion/policies repercussion/global advice/phase4 results/reports Management summary/risk management/catalogue of risks and actions printed/displayed I need to know the business sector your company is working in request(data, feature, user, business sector) What is meant by consulting? request(explanation, definition, business activity, consulting)

7

8 NL processing: analysis and generation Task: to retrieve not only the meaning, but also the intention of the speaker (and even the emotional state). Codify (meaning + intention) using extended semantic structures (communicative acts). Sentence templates adaptable to the user specificities (expertise, language formality, etc...) are used for generation text Natural Language Analyser intention meaning emotion Comm. acts Interaction Manager Comm. acts Natural Language Generator template template template text

9 NL processing: analysis and generation Text extraction Preprocessing Interpretation Delivery Xml2Text Text2Terms Sentences Contractions Pattern Matching Analysis method #1 Text2Xml Grammar Lexicon Natural Language Interpreter Robust processing Analysis method #2 Lexicon Tagger Parser Natural Language Generator Reception Generation Delivery Xml2Terms Generator Text2Xml Templates

10 Comunicative Acts INTERACTION AGENT Static information Dialogue steps Dynamic information Session Model Dialogue States Dialogue Manager Thread Joint User Model Main tasks: Manage the evolution of the conversation in a coherent way. keep track of the on-going dialogue. decide how to proceed next. Ask the Intelligent Agents for information when the answer to the user requires it. Deliver the answer together with indications on how to provide it. Comunicative Acts Dialogue Maker Discourse Generator Task Model Risk Manager & CBR

11 Interaction management: an example Interventions Threads Static Information S1 U1 S2 U2 S3 U3 I will first ask you some questions to get to know your company as well as possible, is it OK?. Are you ready? Fine. But before starting, how does the application work? I will show you the Quick Tour, there you can see how everything takes place, okay? Ok. Let s go [Quick Tour] Do you have any questions? Oh! no, all is alright. The Quick Tour is very clear and concise. Ø command authorise commit request authorise commit solve authorise solve solve S4 U4 OK. First of all, What is the name of your company? It is called "Starsailor car trade". request solve COMPANY NAME = Starsailor car trade

12 System abilities: the task model Discourse Generator Component to the dialogue manager Thread state context a) tasks b) changes dialogue c) discourse parsing Task Model request Communicative Acts (Pattern Based) Discourse Maker Pattern Selection & Fulfillment

13 Requesting information from a external source: the CBR When information is required to provide an answer, the Task Model will first ask the Risk Manager and next the CBR for it. If the information is provided by the CBR, the response could be represented as a dialogue script Deviations between the real dialogue and the script provided will end up in a new call Dialogue Scripts: Tree knowledge structures containing series of actions to solve a task. These actions could be represented by means of communicative acts. Every branch ends in a leaf node labelled END IST

14 Requesting information from a external source: the CBR Example: dialogue previous action script branch But before starting, how does the application work? request(data,feature, system,general) I will show you the Quick Tour, there you can see how everything takes place, okay? inform(explanation,nextaction,system, Quicktour) authorise(continue,open,generic) Yes. confirmation(approve,offer,omega) [Quick Tour] Do you have any questions? inform(explanation,video,system, Quicktour) authorise(refine,open,ask) Oh! no, all is alright. The Quick Tour is very clear and concise. confirmation(deny,offer,omega) inform(data,qualify,quicktour,clear) inform(data,qualify,quicktour,concise)

15 Requesting information from a external source: the CBR Example: dialogue script inform(explanation,nextaction,system,quicktour) authorise(continue,open,generic) confirmation(approve,offer,omega) confirmation(deny,offer,omega) inform(explanation,video,system,quicktour) authorise(refine,open,task) END confirmation(deny,offer,omega) END confirmation(approve,offer,omega) Deviation between dialogue and script: The act does not appear in the script, so the CBR would be called again since the task cannot be ended.

16 A view on the architecture linguatec Speech recognition voice text/xml GUI text/xml html voice html AME 3D Advisor gesture/ xml voice ppch html page_id Communicative acts/xml Communicative acts/xml Presentation Manager text/xml text/xml linguatec Online translation text/xml Communicative acts/xml UPM Natural language interpreter Communicative acts/xml text/xml UPM Natural language generator text/ xml voice linguatec Speech synthesis UPM Interaction manager ppch text/xml text/xml Risk Manager Communicative acts/xml FhG Case-based reasoner

17 <?xml version="1.0" encoding="utf-8"?> <!DOCTYPE problem SYSTEM "im2cbr.dtd"> <!-- example1: concept definition example--> <problem> <problemtype> concept definition </problemtype> <message> <sessionid> </sessionid> <utterance> <ComActList> <request> <type>explanation</type> <matter>definition</matter> <subject>businessactivity</subject> <content>consulting</content> </request> <text> What is meant by consulting? </text> </ComActList> </utterance> </message> </problem> Asking information to the CBR Getting the answer <?xml version="1.0" encoding="utf-8"?> <!DOCTYPE node SYSTEM "CBR2im.dtd"> <node> <nodecontent> <message> <sessionid> </sessionid> <utterance> <ComActList> <inform> <type>explanation</type> <matter>definition</matter> <subject>businessactivity</subject> <content>consulting</content> </inform> <text> Consulting is the process of providing professional or technical expert knowledge or guidance to a business. A fee is usually charged by the consultant for the service provided. </text> </ComActList> </utterance>... <nodecontent>

18 Communicating Presentation Manager, Interaction Manager and Risk Manager

19

20 Challenges Improve robustness and generality of the components developed in the ADVICE project (XML, Java, Ciao Prolog) Connect speech processing and interaction management via text/communicative acts Manage unexpected events in a dialogue with CBR techniques Influence gestures of a 3D avatar