Multimodality: The Next Wave of Mobile Interaction White Paper Multimodality is exciting new technology that promises to dramatically enhance the mobile user experience by enabling network operators to combine speech, touch and onscreen displays for intuitive and powerful mobile applications. What is Multimodality? Multimodality combines visuals, voice and touch to produce rich interactive experiences for mobile users Multimodality technology combines voice and touch (via a keypad or stylus) with relevant onscreen displays to enhance the mobile user experience and expand network operator service offerings. Blending multiple access channels provides new avenues of interaction to users. For example, a mobile user could call a friend by saying Call Rick to his or her mobile phone. The multimodal-enabled device could speak Select which Rick you want to call using your keypad. In turn, the phone could display everyone named Rick in the user s directory and place the call after the user selects the appropriate person from the mobile device s display. This simple yet powerful interchange between voice and visuals opens many opportunities for mobile network operators. Although the preceding example is a basic demonstration of multimodality, it has significant implications. For users, multimodality represents a very fast and efficient way to interact with a mobile device. There is no need to listen to or scroll through a long list of names to make a selection; instead, users simply and quickly make visual choices that were verbally produced. For network operators, the combination of audible and visual functions represents the future of mobile communications. Soon, powerful applications such as mobile commerce will take advantage of multiple simultaneous channels of communication, providing operators with a new wave of service offerings.
Multimodality will be introduced in stages. When performing functions, multimodal devices will initially move between modes of interaction such as visual, voice and touch. This sequential process will restrict mobile devices to a single mode of input at any single time. As next-generation networks and devices are adopted, multimodal devices will allow simultaneous modes of communication. Known as boundless multimodality, this technology will enable more advanced application development. Why Should Network Operators Consider Multimodality? Multimodality will significantly enhance the user experience, leading to a number of benefits including easy-touse, value-added services and reduced churn When communicating through an object such as a mobile phone, it has been found that a poor user experience deters consumers from using many revenuegenerating services offered by carriers (Datamonitor, 3/2002). The combination of visual and audible communication cues delivered by multimodality will enhance the user experience and enable the development of highly efficient and intuitive multimedia communication tools that enhance human interactions and bring unique value-added service revenue to network operators. Market Drivers and Benefits There are a number of market forces driving the need for multimodal-enabled services. Consider the growth of the mobile workplace. Mobile professionals in the U.S. users who spend more than 20% of their time on the road will grow from 13.1 million in 2001 to 19.2 million by 2006 (Datamonitor, 3/2002). Because keeping in touch and up to date are essential for their survival, mobile workers are ideal users of multimodal devices. Carriers need to increase the average revenue per user (ARPU) by offering value-added services that are both easy to use and widely accepted by consumers. According to a study by the Yankee Group (2002), users are more likely to use multimodal-enhanced services, resulting in increased call volumes and greater ARPU for carriers. The strongest value proposition for multimodality adoption is the ease of use it brings to mobile applications. Today s most advanced mobile phone applications are still difficult to use because they lack the ability to combine voice and visual communication. For example, mobile applications are mostly unusable when consumers are driving or walking while using their phones. Easy-to-use applications are the key to mass adoption of mobile services. Consider that today s mobile applications such as mobile e- mail and mobile text-based portals are largely unused. The primary barrier to wide-scale adoption of these applications is their complexity. When consumers do not use value-added features and services, network operator profitability is Comverse White Paper: Multimodality: The Next Wave of Mobile Interaction Page 2
dramatically affected. Multimodal applications will significantly increase the adoption rate of a range of mobile applications. Multimodality will differentiate carriers so they can compete based on services and not simply price points Reducing churn and differentiating carriers, Multimodality will also increase the popularity and use of value-added mobile applications (Datamonitor, 3/2002). The biggest benefit of multimodal for operators lies in retaining customers and keeping them happy, (Datamonitor, 3/2002). The compelling services enabled by multimodality will help reduce the likelihood that users will switch carriers. In addition, multimodal services will help differentiate carriers and make it easier for them to compete based on services rather than simply price. Another benefit of multimodality includes providing services that were impractical or not possible prior to multimodality. With multimodality, not all users need to have devices with the same capabilities. For example, a device with just voice capabilities could still receive a text message using text-to-speech services. Multimodal applications will: Combine visuals, voice and touch for powerful mobile applications Provide users the freedom to choose from multiple modes of interaction Break down barriers to the mass adoption of value-added services Enable spontaneous and intuitive communication Create new and richer services for mobile users Attract new customers and deepen loyalty of existing users Generate increased ARPU via value-added services Offer richer experiences to mobile users Simplify the use of value-added services Comverse White Paper: Multimodality: The Next Wave of Mobile Interaction Page 3
The Evolution of Mobile Applications By examining some historical trends in the telecommunications marketplace, it is clear that multimodality is a natural next step for mobile network operators. From Touch to Speech Recognition The first known attempt to standardize the transmission of nonvoice signals over voice channels was developed in 1960 by AT&T s Bell Labs. Known as Dual Tone Multi-Frequency (DTMF) and later referred to as touchtone, DTMF was designed to send sounds distinct from a human s voice over long distances to central offices. The sounds were produced when a person pressed keys on his or her telephone. DTMF sped the calling process and eventually formed the foundation for automated call handling. Today, DTMF tones are used for a variety of purposes including placing calls and navigating touchtone-based automated systems. Interactive voice response (IVR), an extension of DTMF, is the use of computer technology to provide prerecorded or computer-generated audio responses to customer touchtone input. The IVR industry began in the mid 1980s and found success in virtually all marketplaces where telephone-based customer service was important. An IVR system often works in conjunction with live agents to identify the nature of a call and possibly a caller s account-related information. IVR systems are also able to process more complex inquiries, such as the balance of a checking account or the status of a pending credit card transaction. However, touchtone IVR systems are limited to a small list of options for any given location within their tree structure. In addition, they can be very confusing to callers who may get lost within the system or not understand how to back up or start over again. The lack of verbal communication and visual feedback makes IVR systems difficult for many users to navigate. The limitations of IVR systems have led to call centers enhancing their customer support solutions with speech recognition-based systems. Significant Comverse White Paper: Multimodality: The Next Wave of Mobile Interaction Page 4
advancements have been made in speech recognition, enabling callers to speak commands using natural language. This has opened the door for the development of speech-driven applications that are virtually impossible to design with touchtone-based IVRs. For example, now airline reservations can be automated, whereas in the past it would have been very difficult to design a system that listed all departure and arrival cities using a touchtone IVR. The next step is to combine the power of speech recognition with onscreen display information. From Short Messaging to Wireless Applications SMS began in Europe and became mainstream as younger mobile users used the technology to communicate with friends Short message service (SMS) is a globally adopted mobile service that enables the transmission of alphanumeric messages between wireless devices. First introduced in 1991 in Europe, SMS initially gained popularity among younger mobile users and is expected to reach 500 million users worldwide by 2003 (Webopedia.com). SMS allows users to enter messages up to 160 alphanumeric characters in length. However, typing messages is a timeconsuming process, as users are limited to the standard numeric keypad or T9 predictive text input (a process that anticipates popular words based on numeric values). Despite the difficulty of using SMS, a new language has emerged using abbreviations such as C U L8er for see you later. Another limiting factor of SMS messages is that they cannot contain images or graphics (unlike computerbased instant messaging applications). SMS proved that mobile users were interested in alternative methods of communication via their mobile devices. The wireless application protocol (WAP), introduced in the late 1990s, is an open, global specification that empowers users to access information via mobile devices and display content and simple graphics on the device s display. WAP bridges the gap between the mobile world and the Internet by delivering an unlimited range of mobile value-added services to subscribers, independent of their network or mobile device. WAP-enabled mobile users benefit from secure access to relevant Internet information and services such as unified messaging, banking and entertainment. A mobile WAP-enabled device uses a microbrowser to display information appropriate for its small display, low memory constraints and the low bandwidth restrictions of its wireless network. Although WAP supports HTML and XML, the WAP s display language is specially designed for small screens and one-hand navigation without a keypad. WAP Comverse White Paper: Multimodality: The Next Wave of Mobile Interaction Page 5
lacks the ability to interact with voice and its input is limited to the 12 keys on most mobile phones. Convergence and Multimodality The convergence of mobile phones and computer technology is helping to validate the need for multimodalenabled devices Today, the trend toward converging many different technologies has produced powerful new mobile devices. Many of today s mobile personal digital assistants (PDAs) provide the benefits of mobile phones and computers such as larger, color displays. These devices are pointing toward a future where mobile devices will provide powerful network services. This will be further enhanced by new, soon-to-be-released, Class A mobile devices that enable simultaneous voice and data communications channels, helping to bring multimodality into the mainstream. A royalty-free, platform-independent standard is being developed that will allow multimodal telephony-enabled access to information, applications and web services from mobile devices and wireless PDAs. The Speech Application Language Tags (SALT) Forum, founded by Comverse, Microsoft, Intel, Philips, Cisco and SpeechWorks, is developing the SALT standard, a protocol that extends existing mark-up languages such as HTML, XHTML and XML. SALT will let users interact with an application using speech, a keypad, a mouse or a stylus and it will produce data as synthesized speech, audio, plain text, motion video or graphics. Each of these modes will be able to be used independently or concurrently. Multimodality in Action Multimodal applications will combine powerful forms of user input and data output to help network operators deliver new services. The key options are: Speech Input: Using speech recognition technology, users will be able to search for information (such as the name of an airport), navigate within and between applications, fill in data fields and perform other hands-free functions. Keypad Input: Using the navigation capabilities of their mobile devices (arrows, joysticks, stylus, keypad, touch-screen, etc.) users will be able to make selections, enter numbers such as a password or PIN, and perform a wide range of other functions. Comverse White Paper: Multimodality: The Next Wave of Mobile Interaction Page 6
Spoken Output: By simply listening, users will be able to hear both synthesized, prerecorded, streaming or live instructions, feedback, requests, sounds and music. Visual Output: Using the mobile device s display, lists of options, pictures, maps, graphics and endless other possibilities can be presented to users. Multimedia Message Example Multimedia messaging is expected to be a popular multimodal application. With a multimodalenabled mobile device, users will be able to combine verbal commands and onscreen visuals to send and receive multimedia messages. For example, a user will be able to ask her phone to display new mail by simply saying open mail. The user specifies a message to open by saying something such as open Judy s mail. If the multimedia message has a picture attached, the user simply says, view picture. The entire process will be done hands-free, combining voice recognition and onscreen graphics and text displays. Multimedia Conferencing Example Another example of a potential multimodal service is multimedia conferencing. Consider a user who wants to conference a group of her friends for a chat. She checks to see who is available and says conference Mary, Kristin and Elissa. While on the conference call, she posts a recent picture she took with her friend Tom for the whole group to see. This is a powerful example of combining Comverse White Paper: Multimodality: The Next Wave of Mobile Interaction Page 7
verbal discussions and commands with simultaneous onscreen visuals. Interactive Directions Example Another multimodal service that is expected to be popular with travelers is interactive directions. In this example, a user can verbally search for a repair shop, make an onscreen selection and interact with a map that helps guide him or her to the destination. Combining verbal search, onscreen displays and audio directions along with an onscreen map, this application is expected to be very popular with mobile professionals. Steps Necessary for Multimodal Implementations Comverse has conducted extensive research and development with the goal of creating powerful multimodal applications for mobile service providers. There are a number of fundamental requirements that must be in place prior to the introduction and adoption of multimodal services. First, mobile devices should be capable of processing multiple channels of communication and these types of devices are expected with the introduction of Class A handsets. Second, existing applications should be enhanced to take advantage of multimodality. Those applications include voicemail systems and voice-activated dialing. Finally, a robust and secure platform is necessary to control all aspects of multimodality. Comverse InSight platform has been developed to accomplish just this. Able to integrate session control and synchronization, speech control (including TTS, ASR and VoiceXML) and data, Comverse InSight is the ideal multimodal platform. Comverse expects wide-scale implementation and user adoption of multimodal services as early as 2004. Comverse White Paper: Multimodality: The Next Wave of Mobile Interaction Page 8
About Comverse Comverse Technology, Inc. (NASDAQ: CMVT), is the world's leading provider of software and systems-enabling, network-based, multimediaenhanced communications services. More than 390 wireless and wireline telecommunications network operators in more than 100 countries have selected Comverse's enhanced services systems and software. Comverse solutions enable the provision of revenue-generating value-added services including call answering with one-touch call return, short messaging services, IP-based unified messaging (voice, fax, and e-mail in a single mailbox), 2.5G/3G multimedia messaging (MMS), wireless instant messaging, wireless data and Internet-based services, voice-controlled dialing, messaging and browsing, prepaid wireless services and additional personal communication services. Other Comverse Technology business units include Verint Systems Inc., a leading provider of analytic solutions for communications interception, digital video security and surveillance, and enterprise business intelligence; and Ulticom, a leading provider of service-enabling network software for wireless, wireline and Internet communications. Comverse Technology is an S&P 500 and NASDAQ-100 Index company. For additional information, visit the Comverse web site at www.comverse.com. Comverse White Paper: Multimodality: The Next Wave of Mobile Interaction Page 9