An open standard for instant messaging: extensible Messaging and Presence Protocol (XMPP) Pin Nie Helsinki University of Technology Pin.Nie@hut.fi Abstract This paper discusses the extensible Messaging and Presence Protocol (XMPP), which is the first effort in creating an open standard for instant messaging and presence information. XMPP was introduced by the Jabber Software Foundation (JSF) and formalized in the IETF. Numerous extensions called Jabber Enhancement Proposals (JEP) have evolved through subsequent work. This paper explores the XMPP specifications and technology behind XMPP, accompanied with examples of the application program. Both advantages and disadvantages are analyzed. KEYWORDS: XMPP, Instant Messaging, presence, XML streaming 1 Introduction Instant Messaging (IM) is a service, which enables people to communicate with each other in realtime. It supports different communicating styles, such as unicast, multicast and group talk fashion. Quite a few protocols and frameworks that support the IM service have been created already. For example, AIM, MSN, Yahoo Messenger and ICQ are examples of existing IM systems. In addition, Session Initiation Protocol (SIP) [10] based design, SIMPLE (SIP for Instant Messaging and Presence Leveraging Extensions) [14] and IM extension [9] also exploit the same functionality. However, each of them has serious disadvantages. Commercial products are based on proprietary protocols, which block interoperability between different implementations and limit further development. SIMPLE [14] [4] is quite complex, because it is built on the top of the SIP architecture. In order to overcome these problems and facilitate the use of IM service, another new solution emerges. The extensible Messaging and Presence Protocol (XMPP), is a standard specified by the IETF for carrying IM service. It is an open XML protocol for near-real-time messaging, presence, and request/response services. XMPP was first proposed by Jabber open-source community and is still under the development. Later, it was formally approved and archived by the IETF in four Internet specifications [5] [6] [7] [8]. Due to its inherent good features, the problems aforementioned can be solved. Furthermore, Jabber Software Foundation keeps improving this framework and producing enhancement proposals, known as JEP(s) to carry out new requirements. The aim of this paper is to present the basic principle of XMPP and discuss its features. A few typical extensions of XMPP are covered. The paper is organized as follows. Section 2 describes the essential knowledge of XMPP and addresses fundamental actions and key elements in XMPP working process. Section 3 introduces some extensions of XMPP, which enhance the power of XMPP. Section 4 gives a short review of previous work with an overall evaluation of XMPP and summarizes the paper. 2 Basic Framework of XMPP This section explores the essential principles in XMPP. Technologies are introduced with evaluation. Core functions, security issues and basic procedures are covered. Examples of the XMPP-based software are presented to illustrate typical features. 2.1 XMPP Core The XMPP architecture is composed of three elements, XMPP client, XMPP server and gateways to foreign networks. Fig. 1 illustrates this architecture. The XMPP server is responsible for connection management and message routing. The gateway serves a bridge between different networks and has to manipulate at least two different protocols. The underlying connection between any two XMPP elements is TCP. Before the content session, a TCP connection for XML streaming must be established first. A single TCP connection can carry multiple sessions, which are identified by their unique addresses. Similar to the addressing scheme in SMTP, the identifier for XMPP entity, named JID resembles the pattern of an email address followed by an URL: user@domain/resource, for example, niepin@tml.hut.fi/laptop. Application level routing policy is defined on the server side, but the specification [5] leaves the process details to the implementation. In XMPP, there is no routing discovery capability, which is specified in the SIP framework. In most cases, a direct server-to-server stream between two domains is negotiated at the initiation phase. Three essential primitives called XML stanzas are defined in XMPP. XML stanza is the actual payload message in XML format that can be exchanged over the XML stream [1]. In addition, there are five common attributes (to, from, id, type, xml:lang) for these stanza. Three key stanzas are described below:
Figure 1: The XMPP Architecture [3] <message/>: it is carried out in a "store and push" mechanism through which one entity pushes information to another, for example, transferring messages between two endpoints. <message from niepin@tml.hut.fi/laptop to= peter@hut.fi /> <body>hello</body> </message> <presence/>: it is a broadcast through which multiple entities receive information about a given entity to which they ve subscribed, for example, entity s availability. <presence from= niepin@tml.hut.fi/laptop to= peter@hut.fi /> <iq/>: it is a request-response mechanism, similar to HTTP, that lets entities make requests and receive responses from each other, for example: file transfer, roster retrieve. <iq type= get from= niepin@tml.hut.fi/laptop > <query xmlns= jabber:roster /> </iq> All primitive XML stanzas must reside in the <stream/> block, which stands for a XML stream. It means these XML stanzas should be treated as the content of a XML stream. All stream-level errors are considered unrecoverable and a <error/> stanza with description inside is sent by the detector. The list of error conditions is defined in the specification [5]. As a result of these characteristics, XMPP has five positive features as follows. 1. Message box: since messages are stored before being forwarded. XMPP implementations can keep messages for later delivery if the recipient is offline. 2. Service discovery: with <iq/> stanza for query, service discovery is possible. Further, DNS SRV is supported to locate network services. A detailed extension, JEP- 0030 has been proposed [13]. 3. Resource binding: resource location is attached in the address to distinguish different resources of a single user. There is no need for an additional protocol to handle this issue. 4. Internationalization: <xml:lang> common attribute enables internationlization. Figure 2: Security layers for XMPP 5. Reliable communication: a TCP connection to carry out sessions offers a guarantee of the delivery, which can not be accomplished with UDP transmission. However, due to the human-readable data format, the XMPP message size is bigger than those who use binary format and leads to a heavier traffic load. Further, for some cases, TCP connection is too heavy and UDP connection produces less traffic and is more efficient, for example, casual communication between two nodes. 2.2 Security Overview Security for the communication is also specified in the core of XMPP [5]. Two mechanisms are employed to achieve confidentiality, data integrity and mutual authentication. The first protocol is the Transport Layer Security (TLS) [11]. It encrypts XML streams to ensure the confidentiality and data integrity between two entities. The second is the Simple Authentication and Security Layer protocol (SASL) [12], which can provide a reliable mechanism for validating the identity of an entity. Both of them are required to be supported for client-to-server and server-to-server communication. The order of layers for the protocols stack is showed in Fig 2. TLS is used by XMPP to protect a channel from tampering and eavesdropping. Clients should use TLS to secure the streams prior to attempting the completion of SASL negotiation. It is triggered by the "STARTTLS" extension and has its own namespace "urn:ietf:params:xml:ns:xmpp-tls". SASL defines a generalized method for adding authentication support to connection-based protocols, and XMPP uses a generic XML namespace ("urn:ietf:params:xml:ns:xmppsasl") profile for SASL that conforms to the profiling requirements. Supported mechanisms are announced within a <mechanisms/> element. The general procedure of a successful TLS+SASL negotiation is as follows. Fig. 3 illustrates the security setup process between the client and the server. 1. The client establishes the TCP connection to the server and initiates the XML streams. 2. The server sends a STARTTLS extension to the client, including the supported authentication mechanisms. 3. The client responds to the STARTTLS command. 4. The server informs the client that it is ok to proceed. 5. The client and the server complete the TLS setup of the TCP connection.
Figure 3: A successful security setup with TLS and SASL 6. Given that the success of the previous step, the client initiates a new stream to the server. 7. The server responds by sending a stream header to client along with any available stream features (SASL authentication). 8. The client picks up an appropriate authentication mechanism. 9. The server sends a Base64-encoded challenge to the client. 10. The client responds to the challenge with the credential. 11. The server sends another challenge to the client, as the session token. 12. The client again responds to the challenge. 13. The server informs the client of successful authentication. 14. The client initiates a new stream to the server for the application-purpose communication. In addition to these solutions applied in the core specification of XMPP [5], another specification [8] defines methods of end-to-end signing and object encryption for the XMPP. The idea is based on the XMPP-to-Common Presence and Instant Messaging (CPIM) [7] mapping, and must follow the procedure below: 1. Generate a "Message/CPIM" object. 2. Sign and/or encrypt both the headers and content of the "Message/CPIM" object. Figure 4: A ping-pong interaction between XMPP client and server [3] 3. Put the resulting signed and/or encrypted object into an XML CDATA section contained in an <e2e/> child of a <message/> stanza. 2.3 XMPP for IM and Presence As the name of XMPP implies, one primary purpose of the XMPP/Jabber technologies is the instant messaging and presence service. To describe extensions and the applications of the XMPP core features to this aim, another specification [6] was developed as a complement for the core document. Fig. 4 illustrates a simple communication process at one side between the client "A" and the server. It is almost the same situation at the other side. The first two messages are to initiate a XML stream. At this step, some key parameters, such as the stream tag "id" and security mechanisms (e.g. TLS and SASL) should be negotiated between two entities. Then, the content transfer can be processed. The third message is sent by the client "A" to the remote node "B" who is in the "server.com" domain. The language is set to "english". The fourth message is the response from the callee party. After the simple conversation, client "A" tears down the connection by sending a </stream:stream> to the server. The server confirms the action by replying the same XML stanza to indicate the end of the XML stream. The TCP connection is closed at last. The IM session above is very simple. In reality, the scenario could be much more complicated. There are five types of instant messages in XMPP, chat (unicast), groupchat (multicast), headline (broadcast), normal and error. Every
kind is designed to fit a specific environment. Besides, the XMPP IM specifies a set of child elements (subject, body and thread) for instant messages. Concerning presence, seven types of the state are defined for presence information. They are unavailable, subscribe, subscribed, unsubscribe, unsubscribed, probe and error. Presence stanza has three child elements: show, status and priority. The "show" depicts four statuses (away, chat, Do Not Disturb (dnd) and extended Away (xa)) that are reflected on the IM client program. The "status" could contain any words that we set about our states, and it is showed to our friends on the contact list. Note that presence is a service that has privacy issue. It needs the entity to decide whether to subscribe it and how to apply the subscription. Some presence configuration should be moved to the client side to let user do it. The roster, meaning the contact list, can be stored at either client s side or server s side. Although it reduces the workload on the server if the roster is locally managed, it is recommended to store at the server side, because the roster may be lost in case the local file is destroyed or removed, for example, reinstall operating system, disk failure or accident deletion. Moreover, in the XMPP, the IQ stanza is effectively used in the roster management. 2.4 Software According to the XMPP architecture in Fig. 1, there should be at least three kinds of software: client, server and gateway. The first two form the base of the XMPP network. In this section, an example for each element is introduced. The criterion to choose them is based on their functionality and an "open and free" condition. Both of them provide different version for different popular operating systems (Windows, Linux and Macintosh). The client example is the Jabber Client [15], which was developed by Jabber Community. It can enable the user talk to his friends who are using different IM systems, for example, "ICQ link", "AIM link", "Yahoo! Transport" and "MSN Transport", at the same time on a single program, depending on the capability of the connected server. Jabber Client has another browser-based version, called web messenger, which takes advantage of the HTTP protocol. One surpassing feature of the browser-based web messenger is that it can still function, when normal IM clients are blocked. This situation is quite normal in many companies for security reason. Only the necessary ports are opened, while other ports are closed and no traffic is allowed. The port "80" for HTTP is usually permitted, so web messenger can work by encapsulating its traffic in HTTP packets. The server example is the wildfire [16], which was developed by JiveSoftware organization. It was called Jive Messenger at first. The Wildfire server was developed with Java and it applies web-based administration console. The server has cross-platform capability and can be administrated remotely. The basic installation supports the core specification [5]. Other specifications and some extensions are sustained by several plugins of the server. 3 Extensions The XMPP extensions are composed of two major branches. The first part is JEPs, which are produced and managed by Jabber Software Foundation (JSF). It is the biggest source of XMPP extensions and provides substantial support to the functional improvement and generic standards. The second source derives from various research work and implementations, most of which focuses on the interoperability of XMPP-based applications, one key purpose of the XMPP. One example is given here. It is a XMPP-based design of IM service extended from Short Message Service (SMS). 3.1 JEPs The JSF develops extensions to XMPP through a standard process centered on JEPs. The process is managed by the JEP editor and involves intensive discussion on the Standard- JIG mailing list, formal review and voting by the Jabber Council, and eventual inclusion in the set of approved protocols. A list of all JEPs and details of each proposal is available on the Jabber website (www.jabber.org). So far, 175 JEPs have been published. 106 pieces of them have been approved and the rest are either rejected or obsolete. Apart from rejected, deprecated and obsolete JEPs, there are five kinds of status for every approved enhancement to the Jabber/XMPP protocols as well as proposals that are under active consideration. They are listed below. 1. Final The protocol defined may be considered as a stable technology for implementation and deployment. 2. Draft The protocol defined is encouraged and is appropriate for deployment in production system, but some changes are possible before it becomes a Final Standard. 3. Experimental No approval guarantee for this JEP. It is only encouraged in exploratory implementations. Production deployment should not be done until it advances to a status of Draft. 4. Proposed This JEP is currently within Last Call or under consideration by the Jabber Council for advancement to the next stage in the JSF standards process. 5. Active The process or activity defined has been approved. It is currently working and will do so until this JEP is deprecated or obsolete. In addition, there are five types of JEP(s), i.e. Historical, Humorous, Informational, Procedural and Standards Track. Among them, Informational and Standards Track are important for XMPP application developers. Historical and Procedural are related with JSF community, other than XMPP extension protocols. Five important JEPs have been finalized. They are Data Forms (JEP-0004), Jabber-RPC (JEP-0009), In-Band Registration (JEP-0077), Non-SASL Authentication (JEP-0078)
and Service Discovery (JEP-0030). Besides, there are 31 draft JEPs, 31 experimental JEPs, 36 active JEPs and 3 proposed JEPs. Thanks to these JEPs, many new functions and improvements have been added in some XMPP-based programs and software, for example, personalized avatar, file transfer and gateway interaction. Consequently, XMPP applications are becoming more powerful and popular. Note that standard error responding procedure was also specified in the core and extension protocols. It can make sure the clients that does not support some extension would not behave incorrectly, when communicating with those extension-enabled clients. 3.2 Extension on Interoperability In order to enable the XMPP-based applications to work with other protocols or networks, extensions must be provided to coordinate different protocols and connect them seamlessly. Effort has been made on this area. The XMPP gateway to commercial protocols, such as MSN, Yahoo Messenger and AIM, are available. Here a new design to combine XMPPbased IM with Short Message Service (SMS) is offered for analysis. Similar to the IM service, SMS is another popular instant text communication. But unlike IM working in IP-based networks, SMS serves in mobile networks, e.g. GSM. To bridge the two is becoming a new promising niche because people could then enjoy the instant message service anywhere at any time. Now there is a common, unified and open standard available [2]. The infrastructure, based on XMPP is able to interconnect IM systems and SMS systems. From the architecture showed in the Fig. 1, XMPP client can talk with an external node through the XMPP server that connects to the foreign network by the help of the gateway. In this design, the gateway acts as a translator and bridge. The underlying protocol in SMS system is Short Message Peer-to-Peer (SMPP). It is in binary format, different from the human readable text in XMPP. The gateway must be capable to perform two basic missions. One is the conversion between XML packets and SMPP Protocol Data Units (PDU), and the other is the session management. The first requirement includes both XML packet parsing and SMPP PDU wrapping. The second one is for keeping track of short messages within an IM session, because the SMPP does not provide session management. In addition, there should be some way to ensure in-order delivery. Based on the analysis above, a gateway design between IM systems and SMS systems was proposed [2]. Fig. 5 shows the logic structure of the XMPP server and Fig. 6 depicts the XMPP gateway. In order to satisfy the second requirement, that is, session management, the session module of the XMPP gateway creates a pseudo XMPP client. It acts for the corresponding SMS user to execute functions, such as login, data transmission and so on. The queue mechanism and store-and-forward mode enable orderly delivery of messages. In reality, unexpected failures, e.g. unreachable or power off, may occur on a mobile handset from time to time. Retrial and delivery need a confirmation from the recipient as the guarantee. To solve this problem, a feature of SMPP, delivery receipt, is utilized to indicate the final de- Figure 5: Basic functional modules of the XMPP server [2] Figure 6: Basic functional modules of the XMPP gateway [2] livery status for each message. The philosophy of this design can be extended to bridge XMPP system and other foreign networks, for example, proprietary protocols. The details depend on the attributes of the non-xmpp protocols and the interface provided by the protocol owner. 4 Conclusion This paper presented the XMPP framework and discussed both the specifications in the IETF and extensions in the JSF community and wild world. Advantages of the XMPP protocols were mentioned, along with the corresponding features. They explain why the XMPP is designed in this way and how it fits the requirements of the users. Examples of application programs were also given to gain a direct view of the XMPP-based software and its current development. The XMPP/Jabber technology has achieved big success, especially since the IETF approval of the core protocols. It led to significant implementations, major deployments and renewed activity by open-source projects and commercial software developers. New features and capabilities are being built into the framework. Nevertheless, the XMPP/Jabber still has to face challenge from competing technologies, such as SIMPLE stated above. In addition, to interoperate with other IM systems requires more effort on the gateway development. Compared to commercial products and enterprise applications, several topics, for example, quality of service, value-added services and multimedia support demands further work. Thanks to the extensibility of XML, the above
issues can be considered and solved by adding new elements to realize desired rules in the future, with backward compatibility. References [1] Peter Saint-Andre. Streaming XML with Jabber/XMPP. In Proc. 2005 IEEE Internet Computing, Sep 2005, IEEE Computer Society Press. [14] Robert Sparks and Hisham Khartabil. SIP for Instant Messaging and Presence Leveraging Extensions (simple). At http://www.ietf.org/html.charters/simple-chart [15] Jabber Community. Jabber Client At http://jajc.ksn.ru/download.php?id=4. [16] JiveSoftware Organization. XMPP server wildfire. At http://www.jivesoftware.org/wildfire/. [2] Heng-Te Chu, Wen-Shiung Chen, Yi-Hung Huang and Jeng-Yueng Chen. A novel design of instant messaging service extended from short message service with XMPP. Fifth IEE International Conference on 3G Mobile Communication Technologies, 2004 Page(s):504-508, IEEE Computer Society Press. [3] Mikko Laukkanen. Extensible Messaging and Presence Protocol (XMPP). At https://www.cs.helsinki.fi/u/kraatika/courses/ipsem04s/slides/xmpp.pdf University of Helsinki, Department of Computer Science, 2004. [4] Simone Leggio. SIP for Instant Messaging and Presence Leveraging Extensions. At https://www.cs.helsinki.fi/u/kraatika/courses/ipsem04s/simple_leggio.pdf University of Helsinki, Department of Computer Science, 2004. [5] Peter Saint-Andre. Extensible Messaging and Presence Protocol (XMPP): Core. IETF RFC 3920, Oct 2004. [6] Peter Saint-Andre. Extensible Messaging and Presence Protocol (XMPP): Instant Messaging and Presence. IETF RFC 3921, Oct 2004. [7] Peter Saint-Andre. Mapping the Extensible Messaging and Presence Protocol (XMPP) to Common Presence and Instant Messaging (CPIM). IETF RFC 3922, Oct 2004. [8] Peter Saint-Andre. End-to-End Signing and Object Encryption for the Extensible Messaging and Presence Protocol (XMPP). IETF RFC 3923, Oct 2004. [9] B. Campbell, J. Rosenberg, H. Schulzrinne, C. Huitema and D. Gurle. Session Initiation Protocol (SIP) Extension for Instant Messaging. IETF RFC 3428, Dec 2002. [10] J. Rosenberg, G. Camarillo, A. Johnston, H. Schulzrinne, J. Peterson, R. Sparks, M. Handley and E. Schooler. Session Initiation Protocol (SIP). IETF RFC 3261, June 2002. [11] T. Dierks and C. Allen. The TLS Protocol Version 1.0. IETF RFC 2246, January 1999. [12] J. Myers. Simple Authentication and Security Layer (SASL). IETF RFC 2222, October 1997. [13] Joe Hildebrand, Peter Millard, Ryan Eatmon and Peter Saint-Andre. JEP-0030: Service Discovery. At http://www.jabber.org/jeps/jep-0030.html.