Design, implementation and analysis of a large scale audio conferencing system using SIP

Transcription

1 Design, implementation and analysis of a large scale audio conferencing system using SIP A n t ó n i o V a r a n d a & T i a g o L o u r e i r o Master of Science Thesis Stockholm, Sweden 2006 ECS/ICT

2 Contents Introduction 3 Overview 3 Description of the problem 3 Motivation 4 Thesis goals 4 Background 5 Overview 5 Conferencing models 6 Centralized model 6 One endpoint mixing 7 Full Mesh 8 IP Multicast 9 Components and Design of a Conferencing Server 10 Signalling Layer 10 SIP 10 Control Layer 15 Conference Engine 18 Study of different available conferencing solutions 21 Scalable architecture design 24 Centralized controlling and management 26 Conference scheduling and listing 27 Call establishment and termination 29 Conference management 33 Bridging between the different conferencing servers 36 Scalability, availability and maintainability 39 Testing and analysing the implementation 46 Result analysis 58 Conclusion and future work 59 2

3 Introduction Overview In this thesis it is proposed to build a highly scalable system for large scale audio conferencing. It is also aimed to evaluate several open source solutions with the basic functionality of conferencing servers (mixing and encoding/decoding functions) and build an architecture on top of it to allow further scaling and introduce monitoring, moderation and booking tools to turn it into a fullyfunctional conferencing server. Description of the problem With the growing popularity of VoIP and Peer-to-Peer systems, there has been appearing a need for conferencing solutions. However, due to the high computational complexity which is conferencing (mainly due to intensive CPU usage for decoding packets, mixing them and re-encoding), it is extremely hard to develop a solution on the client side. Therefore, serverside conferencing is the natural way to go when it comes to scale conferences up to several participants. Even with the state-of-the-art servers that are available nowadays, a server cannot be scaled further than 100~150 participants using a commonly used speech codec in VoIP such as ilbc[4,5]: this is the problem to be solved and to do so it is necessary to design a truly scalable system which will not depend on the capacity of a single server but on a cluster of servers connected and communicating with each other allowing a single conference to be spawned across multiple servers. In this way, conferences will not be confined to a single CPU being the burden of doing the decoding, mixing and encoding shared by the several CPU's available. The final goal is to develop some centralized monitoring, moderation and booking tools in order to allow this architecture to act as a single conferencing server to the outter world. 3

4 Motivation The need for audio conferencing is growing every day. Companies from all around the world often need to provide some sort of contact among their employees (this is especially important in companies with branches in different cities or countries) and need to use some sort of conferencing tool which combines good quality with simplicity and low costs. There are already quite some open source solutions for this but, unfortunately, they can only serve small companies since the amount of simultaneous participants that they can handle is limited to the processing power of the computer on which the conferencing application is running: it is with this goal in mind that the architecture will be designed, being desirable that the only issue in the architecture be the amount of computers that a company owns: if a company wishes to hold more participants all it needs to do is buy a new machine, put it up and running and the system will then hold more participants. Thesis goals This thesis is divided in three main goals: First, a study of different available solutions for conferencing will be carried out in order to assess what is already out there in terms of conferencing solutions. Second, a design of an architectural solution for conferencing will be made which will provide the desired scalability which is proposed in this thesis. Finally, tests will take place to make sure that what was studied in the first part and, then theoretically proposed in the second, may in fact be implemented. 4

5 Background Overview The main goal of this Master Thesis is to design and implement a highly scalable VOIP conferencing solution. A VOIP conferencing server is an application that allows users to talk with each other by calling the same destination address (the identifier of the conference) using a VOIP signalling protocol. A conferencing server provides functionalities for creation, call acceptance, management and moderation of conferences. Even though a conferencing server can handle many conferences and participants, the main goal of this thesis is to build any architecture that would scale to thousands of conferences and participants talking at the same time, and this is not possible to achieve with only one conferencing server. To achieve the main goal of this thesis, several VOIP conferencing servers were tested and compared in terms of performance with different codecs, maximum number of participants, maximum number of conferences, management and moderation functionality and scalability options. In order to give a better understanding to the reader about which components are necessary to make a VOIP conference call, a brief overview of the protocols and applications needed will be given. To make any call over the internet, whether it is and endpoint to endpoint call, or a conference call, a signalling protocol is needed in order to establish, modify and terminate the call. There are several internet telephony protocols, but the IETF's SIP [7] protocol is, so far, the most used and known protocol available. SIP stands for Session Initiation Protocol, and it is an application layer signalling protocol. SIP is used to create, modify and terminate sessions between users. Sessions can either be internet telephone calls, conference calls, video calls, etc. 5

6 But SIP is not the only protocol needed to make an internet phone call, SIP is only used to make the communication possible, the real media (i.e. the voice stream) will be sent using another protocol, usually RTP [8]. RTP protocol is used to transport real-time multimedia data (audio, video, etc) over the internet, the protocol splits the multimedia data into packets and takes care of its transportation from endpoint to endpoint. Another protocol is also needed to specify the capabilities and negotiate the characteristics of the SIP session between the participants. The most common protocol to do this is SDP [9]. SDP stands for Session Description Protocol and based on the capabilities of the client for example, what codecs the client supports; SDP negotiates what codec will be used to encode the media stream during the session and also what protocol will be used to transport the media stream, the most common one is RTP. All the solutions that were tested and used, are based on SIP, using RTP for transporting the media stream and SDP for negotiation of the media capabilities. Conferencing models There are several conferencing models that can be defined based on the topology of the signalling process and media paths. Only the most used models will be presented. Centralized model The most common and easiest to manage is the centralized model, where all the clients connect to a central server, sending their media stream to the server. The conference server is responsible for decoding, mixing the media streams together, encoding it and sending back the appropriate media stream to each client. Since each participant is not supposed to hear itself, the server has to encode a different media stream to each one of the participants that is talking in the conference. One must bear in mind that it is not possible to mix encoded voice streams; they need to be described in a linear way (usually known as raw audio data). 6

7 A A B+C+D+E+F+G+H H G B Conferenc e Server F C D E Illustration 1: Centralized model This way, for a conference with N participants, there will be N decoding and N encoding processes, since it is necessary to subtract the contribution of the participant where each media stream is being sent which severely limits the number of participants that a server can hold. Naturally, some optimisations can be done, such as reusing the same media stream to everyone who is a listener in the same conference. For instance, for a conference with N listeners and M speakers, only N decodings and N+1 encodings will be necessary, as the same encoded media stream will be sent to the M speakers of this conference. One endpoint mixing Another way of creating a conference is to have one of the participants in the conference to do the mixing and the others just receive the mixed stream from him. For example, participant A is talking with participant B, and then participant C calls A, and A instead of dropping the call that it he is having with B, just joins C to the ongoing call with B. When this happens, A is the host of the conference call, and is responsible for mixing the media streams, and to send the appropriate media stream to each of the participants in the call. So A receives media streams from B and C, and sends back to B the sum of the media of A and C, and sends 7

8 the sum of A and B to C. This model is only suitable for small conference calls, because it depends on the capabilities (processing power and bandwidth) of the computer of the participant who is hosting the conference. It also introduces some problems like, if the participant that is mixing the conference leaves, the conference is dissolved. A A B+C+D B B A+C+D D C C B+A+D Full Mesh Illustration 2: One endpoint mixing In the Full Mesh model each participant sends its media stream to every other participant in conference, and each participant mixes all the media streams it receives from every other participant. This model works quite good when there is silence suppression in the clients, since usually only one participant is speaking at a time, so the decoding/mixing/encoding overhead in each client is not that high. The biggest problem with this model is that usually participants don't have enough bandwidth to send the media stream to every other participant in the conference. 8

9 A A D D A D B A C B C B B C C IP Multicast Illustration 3: Full mesh conference model Using IP multicast addresses also enables the possibility of creating scalable conferences. The idea is to associate an IP multicast address with a conference, and every participant sends its media stream to that IP address. All participants are also part of the multicast group that is listening that multicast IP address. The advantage of this model compared to the full mesh one, is that each participant only needs to send his media stream once, instead of sending it to every single participant, so uplink bandwidth is spared. One problem is that the multicast is not a technology widely supported outside universities and test networks. Apart from the centralized conferencing model, all of these models increase the complexity of booking, managing and controlling conferences in a centralized way. Since this was also one main requirement for this thesis, it was decided to focus the work more in the centralized model. 9

10 Components and Design of a Conferencing Server A conferencing server consists of 3 main modules: A signalling layer, a management and control layer and a conference engine. A signalling stack is needed to establish, modify, and terminate conference sessions. A management and control layer is used to book/create/delete conferences, to kick/mute/unmute participants, to setup conference details, like passwords, maximum number of users, moderator, etc. The conference engine is the module responsible for decoding, mixing and encoding the media streams. Signalling Layer A signalling layer is the part of the conferencing server that enables the users to interact with the conferencing server in order to establish/terminate a call and to negotiate the details of the media session. This layer is in charge of accepting new participants into the conferencing server, provide them an address and a port to send/receive the media streams and to terminate the calls releasing all the used resources. As said before, there are several signalling protocols for establishment of multimedia sessions (audio, video, whiteboard, etc). The most common ones are H.323 [10], that is a bit outdated and not so used any more, and SIP [7], the Session Initiation Protocol, defined by the IETF, wish is an open standard and is the most widely used telephony signalling protocol these days. SIP was the chosen protocol for the signalling layer of conferencing server. SIP SIP [7] stands for Session Initiation Protocol and it is an application layer signaling protocol. The design and development of this protocol was done by the IEFT (Internet Engineering Task Force) [11] and had scalability and flexibility in mind. 10

11 The SIP protocol is specified in many different RFCs, the most important one is the RFC 3261, where the core of the SIP protocol is specified. The aim of SIP is to establish, modifying and terminating sessions between two or more participants. A session describes the communication process between several participants, and the state of each participant during that communication process. Some examples of sessions that can be created using SIP are: internet telephone calls, distribution of media over the internet (video, sound, sideshows) and multimedia conferencing. Despite that SIP is used to create the sessions between the participants, SIP is not the only protocol needed for them to communicate with each other. SIP is only responsible for making this communication possible, the real media is sent over other protocol, usually RTP, Real-time Transport protocol. Another protocol that is also usually used together with SIP is SDP, SDP stands for Session Description protocol that is used to negotiate the characteristics of the media session between the end users, for example to decide what codec should be used to encode the media in a internet phone call. SIP design is based on the Internet model, which means that almost all the logic is in the endpoints of the communication. Endpoints are also the only components that store the state of the connection: this type of design is very good for systems that need to scale well. The only disadvantage of such a design is the fact that introduces message overhead, since all the messages that are needed to setup a connection are needed to be sent from end-to-end. This model is completely different from the model of the normal PSTN (Public Switched Telephone Network) where usually all the logic is kept in the components of the network and the endpoints (normal telephones) are very basic and primitive devices. With this new end-to-end design, SIP is able to provide the same functionality as the old PSTNs, but also is much easier to extended it for supporting new services that might appear in the future. 11

12 SIP messages headers inherited their format from RFC 822, the same way as the protocol HTTP. Some things in the SIP protocol are based in the HTTP protocol. How does SIP work SIP URI All SIP entities are identified by a SIP URI (Uniform Resource Identifier). SIP URIs are similar to addresses, their format is the following: sip:username@domain.com, they are composed by a username and by a domain delimited by an "@". The URIs identify the SIP entities when they want to establish a session between each other. SIP Network Elements The SIP protocol define several SIP elements, they can be user agents clients and servers, stateful and stateless proxies, registrars and redirect servers. In this thesis only the ones that are relevant to achieve the goals of this study will be referred. A brief description of the elements that matter to this thesis will be done in this section. User Agents SIP endpoints are called SIP user agents, these are the SIP elements that establish sessions between each other. A SIP user agent is usually an application controlled by the end user and that enables the user to make internet phone calls, but a user agent can also be a PSTN gateway, a PBX, a phone and so on. 12

13 There are two different logical entities in the User Agent element, the User Agent Client (UAC) and User Agent Server (UAS). Every User Agent has both logical entities the UAC and the UAS. The UAC is responsible for sending requests and receiving responses, while the UAS is listening for requests and sending responses to that request. Whenever a user agent receives a request, for example an INVITE message, it is said that it is behaving as a UAS, or when it is sending a request, to start a telephone call for example, it is said that it is behaving as a UAC. Proxy Servers Another important entities in a SIP infrastructure are the SIP Proxies. A SIP proxy can perform routing of SIP messages received from other SIP entities such as User Agents or other SIP proxies, based on the SIP URI of the message. They can be either stateful or stateless, and they can also perform authentication, accounting and so on. One of the most important functions of a SIP Proxy is to route session invitation messages closer to the destination User Agent. Many times the User Agent who wants to start a session does not know the exact location of the other User Agent that he or she wants to communicate with, so he sends the invitation to the proxy and just expects that the proxy knows where to forward his message. Sometimes the message can traverse several SIP proxies until it reaches one proxy that actually knows the exact location of destination. That proxy will forward the message directly to the final User Agent that will accept or reject the session invitation. There are basically two types of SIP Proxies, the stateless and the stateful ones. Stateless Proxies 13

14 A stateless proxy only forwards the SIP messages without keeping any state. Stateless Proxies are much faster and simpler than stateful proxies, since they do not keep any track of the ongoing SIP transactions. They are usually used to perform load balancing or message translation. One of the problems of Stateless Proxies is that they can not absorb message retransmissions due to timeout in the caller User Agent. Stateful Proxies A Stateful Proxy keeps a state whenever it receives a new SIP request, and this state is kept until the SIP transaction is finished. Some of these transactions can last for quite amount of time, for example, the transaction initiated by an INVITE message lasts until the destination User Agent accepts or declines the call. The performance of a Stateful Proxy is more limited than the perfomance of a Stateless Proxy, because it has to maintain state of all transactions until they finish. Since Stateful Proxies maintain the state of the transactions they can absorb message retransmissions since they know in what state exactly is each transaction, so they can check if they already received the same SIP message, Stateless Proxies can not perform this. They can also execute more complicated tasks like waiting for a user to answer his office phone, and in case the session is not established with the office phone, the call can be redirected to the user mobile phone. On the other hand, a stateless proxy could not do this as it does not possess a mechanism which allows it to find out whether a transaction is finished or not. Another interesting feature of Stateful Proxies is that they can perform forking, that means that when a message is received several messages can be sent out to several destinations. 14

15 A Stateful Proxy can perform accounting, NAT traversal and a lot of other features, that is why today most of the proxies are stateful. SIP Messages All SIP communication is based in exchanging SIP messages, these messages follow a specific format. Usually they are transported using UDP, but the network protocol can change. A SIP message consists always of a "First Line", message header, an body. This first line identifies what is the type of the message. There are two types of SIP Messages, requests and responses. Requests are used to initiate some kind of action, or to request something, what is being requested goes on the first line of the message. Responses are messages that are used to answer the request messages, and they contain always a status code that indicates the success of the request. The following image shows an example of a SIP Message: Illustration 4: SIP request message 15

16 Control Layer When there is a conversation with more than three participants, it starts to get difficult to understand everybody if there is not any kind of moderation that controls who's the next person that will talk. When there are people in a class room, or in a conference, there is always someone controlling who can talk at any given time, there are also ways to ask the permission to talk, like raising the hand, or turning on the microphone, etc. The same type of control is also needed in a conference call, if not, it would be impossible to understand who is talking, or who wants to talk. This is the reason why a control layer is needed. The control layer, will receive requests from the participants and the moderator, and it will make the correct actions. The next use case shows an example of what type of control request are sent, and what type of actions are done: 1. A conference call starts where Bob is giving a lecture. 2. Bob decides to mute everyone besides him, in order to explain how the conference call will work, without being interrupted. 3. Bob clicks in the button mute all, and everyone besides him gets muted. 4. Alice is one of the listeners in the lecture. 5. Alice misunderstood some part of Bob's speech because she was listenning to some music in the mean while. 6. Alice clicks on the button request microphone, to ask Bob if he can repeat what he was saying before. 7. Bob sees Alice's request in his controlling window. 8. Bob clicks on the button allow to talk in order to let Alice speak. 9. Alice sees in her controlling window that she is now unmuted. 10. Alice makes her question to Bob. 11. Bob answers back ands asks if she has anything else to say. 12. She answers negatively. 16

17 13. Bob clicks on the button mute, in order to mute Alice again and continue his lecture. 14. Alice's controlling windows gets updated, showing that she is now muted. The control layer is also responsible for notifying all the participants from the events that happen during the conference call, so for example, whenever a participant joins or leaves the conference, everyone else is notified. To accomplish this, there has to be a way for a conference server to receive requests from the participants and to send back events to all the participants in that conference. Without a control layer, it is impossible to manage a conference with many participants. 17

18 Conference Engine The conference engine is the most important component in a conference server, it is responsible to decode the voice streams from the participants, to mix them all together, and to send back an encoded mixed stream to each participant in the conference. The voice streams are encoded with a audio codec (i.e. an abbreviation for coder-decoder, a software algorithm that transforms a analog audio signal into a digital format that can be used to send it over the internet). There are several codecs available to compress audio, some have higher complexity but better quality(for example, Speex [11] and AMR-WB), others have worse quality but their encode/decode algorithm is less complex (for example, g729 and ilbc). A conference engine can decode/encode multiple codecs, it can also have a jitter buffer, a jitter buffer is something that eliminates jitter (i.e. The variation of the arrival time of the packets, due to different routes, network congestion, etc ). A jitter buffer, keeps the packets that are arriving in a buffer, it orders the packets in the exact playback order, and adds a bit of delay between the arrival time and the processing time, so that the conference engine processes the packets as if they would arrive all in the right order, and at a constant rate. The next picture shows how a conference engine works; the voice streams arrive from several participants, encoded with multiple codecs. The engine uses the specific decoder for each of the incoming streams, the raw audio data goes then to the Playout Delay buffer (also known as, Jitter Buffer). Every fixed time, a timer is triggered and an audio linear frame from each participant is taken out from the Jitter Buffer, and all the frames are then mixed together. After the mix process, each frame passes a subtraction process, where the audio produced by the participant who will receive the audio packet is subtracted from the mixed frame, this is done to avoid participants of hearing themselves in a conference call. Afterwords, a frame for each participant is encoded using the right codec that the client negotiated in the beginning, when the call was established. When the 18

19 frames are encoded, are then sent to the participants. The software clients receive the respective frame and decode it to play it to the participants. Illustration 5: Conference engine This component of a conference server is the one that consumes more resources of the machine, most of cpu utilization in a conference server, is used to decode and encode voice packets. So any efforts to optimize a conference server, should always start by optimizing the engine. There are several optimizations that can duplicate the performance of the server, like mixing only the N loudest participants, reusing the encoded bitstreams among the participants who are using the same codec and are only listening, etc. 19

20 Study of different available conferencing solutions One of the main purposes of this thesis was to study, compare and test some of the already available solutions for Voice over Ip Conferencing. The study was focused on all the open source projects that have conferencing capabilities and that seemed to be still actively developed by the community. Asterisk Asterisk [13] is a open source PBX, created by Mark Spencer the founder of Digium Inc.. Asterisk provides a huge set of features, Voic , Interactive Voice Response, Conferencing calls, Call queueing, etc. It also supports several signalling protocols, SIP, H.323, IAX, and it can also interoperate with most of the standards of traditional telephony, using inexpensive hardware adapters. Asterisk can run on several operating systems, Linux, BSD, Windows and Mac OS X. All the codecs that are supported in Asterisk can be used by different softphone clients, supporting different codecs, to connect to the conference calls. Even if some clients do not have compatible codecs between each other, they can still communicate if Asterisk supports the two used codecs of the softphones. Asterisk supports the following codecs: g711, g729, SPEEX, ilbc, GSM, g723.1 and g726. In this study, only ilbc and g729 were used, since they are both low bitrate codecs. A low bitrate codec was needed, in order to allow users with low bandwidth to join a conference. Since Asterisk was the most known and documented application, it was pretty easy to setup it up and try the conferencing capabilities. There are two Asterisk modules that can be used for conferencing. The first one, and the most widely used is called MeetMe [14]. The other one is called App_conference[15], and it is 20

21 not so known, but it can have much better performance in some specific types of conferences. MeetMe MeetMe [14] application allows users to join a conference call, dynamic created, or predefined in the dial-plan configuration file. It provides moderation capabilities to the moderator of the conference using DTMF commands (i.e. By pushing specific numbers in the dial pad). It has also other features available to all the participants: increase/decrease their listening and speaking volume, play entry and leaving sounds, etc. The conferencing MeetMe module for Asterisk is so far the one with more features and also the most stable one. App_conference App_conference [15] is the second conference application for Asterisk. It was designed to be more efficient when the softphones are all using the same codec. It minimizes decode and encode processes and avoids transcoding when it is not needed. Some usual conference types are done in a much more efficient way than MeetMe. For example, when a conference has only one speaker and all the participants are using the same codec, his frame is sent to all the listeners without even being decoded or encoded on the server. Another good optimization happens for conferences with two participants using the same codec, once again, no decode, mix or decode is needed, and the only thing that happens in the conference server is the process of sending one frame of one user to the other user, and vice-versa. App_conference has also Voice Activity Detection (i.e. It detects when a user is silent or talking ), this feature can be used to know who is speaking at a certain point in a conference. 21

22 Even though App_conference was designed to work better than MeetMe, it is much less stable and widely used, so there are less developers involved in maintaining the application, which makes it unsuitable for a production environment. Tests with different codecs and with different types of conferences where made, in order to compare both conferencing modules of Asterisk. The tests and results will be presented later in this document. SER and SEMS SER stands for SIP Express Router [reference], and it is a open source SIP router that can act as a proxy, registrar server or redirect server. It was designed to be a high-performance SIP Router that can run in Linux or Solaris. It has also a big set of extra features, like presence support, SMS gateway, Simple2Jabber gateway, etc. SEMS stands for SIP Express Media Server [reference] and it is a highperformance SIP media server that has conferencing, voic and IVR capabilities. SEMS is also open source, and it uses SER as it signalling layer. So all the signalling communication is done between SER and the SIP phone, and only the media, is sent to SEMS over RTP. SER and SEMS communicate with each other using a unix socket. SEMS acts as a conferencing engine, decoding the audio channels of the participants, mixing and encoding again the mixed frames. 22

23 Illustration 6: SER and SEMS communication process After setting up the environment, performance tests were done in order to compare it with both of the Asterisk solutions. SER and SEMS could handle much less participants than Asterisk using the same codec, for this reason this solution was dropped since it did not perform very well. Chosen Conferencing solution After analysing the pros and cons of all the available solutions, the choice was easy. Asterisk with the Meetme module for enabling conferencing, was chosen due to stability, performance and support by the development community. Scalable architecture design Having studied all the solutions and adopted asterisk as the most suitable, the big question remains. How to build a scalable architecture on top of it so that it holds up thousands of participants? This is no trivial question and indeed the big challenge in this thesis. It seems quite straightforward though, that the servers must somehow communicate among themselves (creating a bridge between two or more servers) in order for a single conference to be able to take place in several servers simultaneously. However, they must share the same management and control interface, acting as a single server. 23

24 Besides all these requirements, service availability and maintainability must also be of great concern. How to replace a server, how to add a new server, what to do when a server crashes are all questions that are answered throughout this thesis. Illustration 7: General architecture of the conferencing server system The above picture describes the general architecture of the solution proposed. To give a short description, all the asterisk servers must be registered in a database which is used by all the entities involved in the solution. Each asterisk server has a designated number (specified when they are added on to the database) with which they will be uniquely identified afterwards. Besides the asterisks and the central database, there shall be a SIP proxy load-balancer (a very simple, stateless proxy, used to control call establishment and forward INVITE messages), a webinterface (for conference booking and controlling purposes) as well as an astman proxy manager (whose purpose is to provide to the web-interface means to book/cancel/modify conferences and to multiplex requests from the web server to asterisk servers and vice-versa). Detailed functionality of these components will be understood in subsequent chapters. The process of booking takes place in the web-interface and works as follows: when a conference is booked (the person who books the conference is known as the host), a certain SIP URI is assigned to it. In the SIP URI, which has the form 24

25 of the username will uniquely identify the conference number whereas the domain will be the same for all conferences and must always be the address of the SIP proxy load balancer to which all calls must be directed to. The username consists of a server number (of known length, decided upon cluster startup depending on the need), which is randomly assigned and concatenated with a unique conference number (there can never be two simultaneous conferences with same conference number). After the conference is booked, users can now join that conference. Once a call is made from a SIP client, it arrives at the SIP proxy load-balancer which will then forward the call to the proper asterisk server where the call is finally established (note that a call establishment might generate the creating of a bridge between asterisks. How this is done will be explained shortly). When the call is established, information about the participant is written to the database which will be used afterwards for controlling the conference (muting, unmuting and kicking participants). Once the call is established, the host of the conference will have the possibility to control his/her conference through a web-interface in which the list of participants should be presented alongside with their status. This web-interface connects only to the asterisk manager proxy which makes all the communication between the users and the actual asterisk servers, once the call is established. Centralized controlling and management There are three main parts that must be taken into account in this topic: conference scheduling and listing, call establishment and termination and, finally, conference management 25

26 Conference scheduling and listing This feature is a "must-have" in any modern conferencing solution. Just as people book conference rooms and agree upon starting time for common face-to-face meetings, conference calls must follow a similar procedure. It is desirable to provide a very simple interface to the person booking the conference (also known as the conference host or moderator) and a web-interface seemed to us as the obvious way to go. This web-interface provides the conferencing server with booking and listing features. Within the booking interface, the host is able to decide the starting time of the conference, its duration and maximum number of users allowed in the conference call. The usefulness of these elements is rather straightforward and is used in call establishment and termination logic. Users shall not be able to join conferences that have not started nor already passed its end time and should as well be rejected if they decide to try and join a conference which is already full. Illustration 8: Simple conferencing booking interface The web-server on which the booking and listing pages (we will refer to it as directory server from now on) are hosted has a connection to a centralized server which holds connections to all asterisks. This server (previously refered to as astman proxy manager) is able to receive conferencing booking requests and 26

27 confirms the booking by writing it on the conferences database and sending an "OK" reply back together with the conference's SIP URI. Whenever there is a request for conference booking, the astman proxy manager will make sure all the data is correct and generate a SIP URI for it. As it was earlier mentioned, the SIP URI is in the form of "sip:username@domain" and the detailed the process of the URL creation shall now be described. In the implementation, the SIP URI looks as: "sip:server_number conference_number@address_of_sip_proxy_loader". After all data received from the directory server is checked for consistency, the astman randomly chooses an active asterisk server listed in the database and assign its number to the URI. This random choice will allow us to make an early load-balancing decision (as example of load-balancing server have a look at [6] in the implementation done during this thesis, the logic behind it is slightly different as the load-balancer gets the destination server from a database function call), trying to book conferences to different servers as evenly as possible. Step two of the process is to generate a conference number, which the astman must make sure that is unique for the time frame in which the conference is taking place. These two numbers are concatenated and then the domain is added to the URL: for example, let's say SIP load-balancer running at IP , chooses server '1' and conference number '23456' then the URI created would have been: "sip:123456@ ". This must be the "to field" used on the SIP request from the client once it is trying to connect to the conference server. Regarding the conference listing, all the people interested in joining conferences should be able to see, at least, the currently ongoing conferences as well as future conferences. This list should be composed of the conference subject or topic, current number of users and the conference link (the SIP URI). How this link is to be interpreted depends on client implementation and is out of the scope of this project. 27

28 Illustration 9: Listing of the currently available conferences Call establishment and termination The establishment of a call consists of all the events that take place from the moment a user contacts the other end-system until the call is terminated (whether that is done by caller or callee). Asterisk already provides such mechanism and uses SIP protocol for creating, modifying, and terminating sessions with one or more participants. The purpose of SIP protcocol is simply to make the communication possible as communication itself must be achieved by other means. Two protocols that are often used along with SIP are SDP and RTP which are exactly the ones asterisk uses to fully establish a call. The first one, SDP, is used to describe and encode capabilities of session participants. Such description is then used to negotiate the characteristics of the session so that all the devices can participate (that includes, for example, negotiation of codecs used to encode media so all the participants will be able to decode it, negotiation of transport protocol used and so on) [2]. As for RTP, it is used to carry the real-time multimedia data: the protocol makes it possible to encode and split the data into packets and transport such packets over the Internet. As the delivery of the actual data for audio is typically delay sensitive, the lighter weight UDP protocol is used as the Layer 4 delivery 28

29 mechanism, although TCP might also be used in environments that suffer higher packet loss. In asterisk's case, UDP was chosen. Having discussed the technology behind the call establishment, it will now be described the process to make this work in a centralized way. All calls are directed to the SIP proxy load-balancer which analyses the SIP request and forwards it to the proper server. It does so by looking at the SIP request URI and determining the server prefix of it. The size of the server prefix must be known by the proxy so that, when looking at a SIP URI, it knows to which server this request is supposed to be sent to. For instance, if a SIP URI is such as "sip:123456@ " and is defined that server prefix size is '1', then the proxy will know that this request must be forwarded to the server number '1'. As described earlier, all servers must be mapped in database so the SIP proxy loadbalancer will look up for server number 1 on the table which holds the corresponding IP address of the server. Illustration 10: Messages exchanged in the establishment of a call However, as it was said before, the server might be down, not active or simply full. Therefore, there might occur the case in which the request will not be forwarded to server number '1'. The forwarding decision is taken based on some rules and for that some small explanation of database is necessary. 29

30 Two tables take part in this decision, the servers and the bridges tables. In the servers table we have the following information per server: server number, ip address, current number of users, maximum number of users, cpu load and last update (each asterisk server updates database every 5 seconds so the SIP proxy balancer knows that this asterisk server has not crashed). The bridges table has information regarding bridges: for each conference that is bridged, an entry is added with origin and destination server (one conference may have more than one bridges). By definition, accepting another user in a given server must fulfill the following requirements: server is not full, has not passed the CPU threshold, is active and running at the moment. The rules to forward a given request are as follows: 1. Check if the original server to which conference was booked can accept another user - if so, this is the chosen server 2. If first rule does not apply, check for bridges for this conference. If they exist, and any of the bridged servers can accept another user, choose this server 3. If second rule does not apply, check for the least loaded server and choose that one. 4. If none of the above rules apply, then it means that all servers are full and no more users can be accepted. When this happens, user gets the call rejected. This is a sign that the service is reaching its limit and new servers should be added to the cluster. Having figured out where to send the packet to, the SIP proxy simply forwards it to the proper IP address. 30

31 Illustration 11: General interaction diagram of the call establishment process Once the call arrives in asterisk, typical SIP call establishment takes place. Asterisk checks the SIP request (especifically, the SDP contents making sure that codecs supported are compatible, and needs to know where to send the RTP traffic to) and does some extra validation (regarding conferecing logic - for example, we do not want the same SIP user to be able to connect to the same conference from two different places) to make sure client is allowed to join the conference. If everything is comformant, asterisk sends an "OK" message back to the client (some important information that goes in the message consist of the list of available codecs for the call and where to send RTP traffic to). Finally, if everything is correctly agreed upon, client sends back an "ACK" to asterisk and RTP media stream is established according to the values previously agreed upon. Besides the call establishment itself, other information from client must be stored. Whenever a client connects to a conference, that information is written to database. Information regarding user's name, status (whether the user is muted and unmuted), its location (which server is connected to) and conference that the user is currently in is stored in database. Usefulness of this information is explained in the "conference management" section. 31

32 The call can be terminated by both parties and this process is activated by sending a SIP "BYE" message. Either one of the parties can trigger this event, and its counterpart must reply with an "OK" message to end the session. Once the "OK" is receiving, the session is torn down and regarding information deleted from database. Conference management Managing a conference means, basically, the ability to control who is allowed to speak and who is not, as well as who is allowed or not to take part in the conference. For every conference there must be a person in charge of it, the host, who is, typically, the person who books the conference call. Both the host and simple participants will have access to a web-interface which will display all the connected participants, their names (obtained through SIP identity), state (muted or unmuted), total number of participants and the time the conference has been going on for. Main difference between the host and participants is that the host will also have access to conference management controls that will allow the host to mute, unmute or kick a participant from the conference that he is hosting. 32

33 Illustration 12: Web-interface for the host monitoring his conference Asterisk already possesses a mechanism allowing to mute, unmute and kick participants from a conference call. However, it is necessary to further enhance this functionality to allow it to be accessed through the web-interface and to be centralized/controlled in only one server, due to the possibility that a conference may be spawned across many asterisk servers. This is achieved through the astman proxy, which is able to accept requests from a given host of a conference, process them and send them to the proper server(s). When the host wants to take some action, an event is sent to the astman proxy which is permanently listening to SOAP requests from the web-interface. The proxy will then see what kind of request that is and forward it to the right asterisk server(s). The events (mute, unmute or kick) will be dealt differently depending on whether they are meant to be executed on a single user or on the whole conference. If they are directed to a single user, then astman looks up in database for that specific conference and user and sends that request to the proper asterisk server. If the request is to take a specific action on the whole conference (muting, unmuting or kicking all the participants) then astman must look up in the database for information regarding which server(s) are currently holding the given talk. Having obtained this information, it then sends the request to all the asterisk server(s). 33

34 Illustration 13: Interaction diagram of the message exchanged by the several components when the host mutes all the users Once asterisk receives a request from astman, it goes through a series of validation. Checks whether that conference exists, verifies that the given user is on that conference and makes sure that the desired action is a valid one (for instance, an already muted user cannot be muted). Also to be noted that the commands that affect the whole conference do not affect the host of the conference: if the host wants to mute all the participants, then everyone is muted with the exception of the host himself. Communication between the asterisk server(s) and astman is quite simple and uses the well known asterisk's manager API [3] for issuing the commands. Explaining briefly, this manager API allows programs to remotely connect to asterisk, issuing commands or events over a TCP stream. The astman proxy is, periodically, doing a database lookup, checking for new servers added, and logging into them using the "login" action. Only after login is done conferences will be booked onto that server. After astman has logged in, actions can be done 34

35 on the conferences there hosted. Possible actions (coming from web-interface) besides the 'login action' are: mute, unmute and kick a single participant or mute, unmute and kick all the participants. These are the possible commands that astman will issue to asterisk and astman does not wait for any confirmation from asterisk since the messages are delivered over TCP and so it can be sure that they will arrive there. As for the opposite communication way, asterisk will inform astman about the arrival and departure of clients sending a "leave" or "enter" event with the user information. Bridging between the different conferencing servers It was one of the primary goals of this thesis to make a scaling solution in the most transparent way possible. The bridges between conferences are no more than a participant in two conferences, the original one, and the bridge in the other asterisk server. This participant is a speaker in both conferences so that everyone connected to any of these conferences listens to everything that goes on in both sides and acting seeminglessly as a single conference. Of course this requires the several asterisk servers to be physically near each other, with a high bandwidth connection. For the connections between asterisks, it is necessary to use a high bitrate codec such as G.711 alaw. This is a requirement for two reasons: first, using a high bitrate codec will hardly increase the CPU load of the asterisk servers, since the encode/decode process for such codec is a very lightweight one. The second reason is that these connections will not introduce any loss in speech quality that is being mixed in other servers (which happens with codecs such as ilbc). 35

36 Illustration 14: Several users connected to different servers but talking in the same conference The decision of sending a certain user to an asterisk server is based mainly on CPU usage and number of users connected to a server. Whenever a server is added to the database, the maximum number of users which it will hold as well as the maximum CPU load (among other fields) must be specified. This is strongly dependant on the machine's processing power, some machines can hold up to 100 users while others cannot go over 50. Asterisk servers are, periodically, updating the database with their CPU load values and current number of users. This way, whenever a call reaches the SIP proxy-load balancer, it will check the database for current number of users and CPU load of the destination asterisk server (it knows which server to forward the call to by analysing the "username" on the SIP URL). If all these values are lower than the threshold defined for the given server, the normal flow will take place and the call is simply forwarded to the expected server and added to the conference (this is the scenario described in illustration 15). If any of these values exceeds the threshold, then the call is forwarded to another asterisk server based on two rules: first, if there is any other asterisk server with a bridge to this conference (this information exists in database) that fulfills the requirements previously described, then the call is forwarded to that server and the user simply joins the conference with the same number (which is 36

37 already bridged to the original one and, thus, the user simply joins this conference without any extra actions being taken). If the previous scenario does not apply, then the call is forwarded to the least loaded server (named Y from this point on): there, the call is established and a bridge is created to the conference in the original server (named X from this point on), adding the new user (which is actually a connection from another asterisk) to the existing conference. At the moment the bridge is created, that information is stored in the database so that, the next time the SIP proxy load-balancer receives a call, it knows there is a bridge connecting to this specific conference from server Y to server X. Servers must, somehow, differentiate normal SIP user calls from the ones made from the other servers. When generating a bridge call, asterisk will, in this implementation, use a different protocol named IAX (Inter-Asterisk exchange protocol) which was especifically designed to enable VoIP connections between asterisk servers (can also be used between servers and clients that use the IAX protocol). This way, when a call arrives at an asterisk server the first task to do is to check the type of channel request: if it is a SIP, then a series of validations follow. In case of a IAX call, these validations are to be skipped and the channel must simply be dumped onto the conference enabling the bridging between the two servers. 37

38 Illustration 15: Call establishment forcing the creation of a bridge. User goes to server Y but, since there is a bridge to server X, the users connected to conference 1 in asterisk X can also talk with users in conference 1 in asterisk Y and vice-versa This bridge (and corresponding conference) will cease to exist when (and only when) all the users connected to the conference in server Y have terminated their calls. In case all the users connected to server X terminate their calls, all bridges must be kept. They may not be destroyed for the following reasons: 1. There may be more than one bridge connected to server X and, if the conference is terminated or bridges are destroyed, the users connected to the other bridged servers (Y and, possibly W and Z) will no longer be able to talk with each other since there is no longer a central point where all the conferences in the bridged servers connect to. 2. There is always the possibility that the CPU usage on the original server lowers and, in that case, new users will be again directed to the server originally booked for that conference: had this bridge been destroyed, the new and old users, supposedly in same conference, would not be able to hear each other since the arrival of a client to an original server would not, logically, force the creation of a bridge. 38

39 Scalability, availability and maintainability In such a huge architecture, service availability and ease of maintainability is essential. With so many components, any updates might cause downtime to a certain extent and that was one of the major concerns during the development of a solution for this thesis. Regarding scalability and starting with the easiest task, adding a new asterisk server should be a simple task. The solution found to solve this issue is indeed rather simple and is no more than adding the server information to the database and to put the server up and running. Adding a server to database is achieved through a very simple web-interface in which the admin must enter: hostname (name of the machine on which asterisk is running), the server's IP address, the server number (recall that each server must have a unique number to be mapped in the SIP URI) and the maximum number of users that the machine holds (this value depends on hardware used). Illustration 16: Administrator interface: should display the status of current online asterisk servers and allow adding/removing servers After validation is done, information is stored in the database. Having done this, it is now only a matter of starting the (asterisk) server itself. From this point on, asterisk server is completely ready to host conferences, astman will then start 39

40 using this new server to book conferences, generating SIP URI's with this new server number. Providing full service availability is a bit more tricky but very important and, therefore, big effort must be made in order to achieve this. There are several ways of detecting that an asterisk server is down and the system will react in a way that will make the service unavailable, at worst, for a couple of seconds. The following scenarios may occur: Consider that an asterisk server (named X) which was not holding any conferences suddenly crashes or becomes unavailable. It will obvisouly stop updating the database since the application is no longer running (all the asterisk servers periodically every 5 seconds - update the database with the current timestamp). When a new call arrives to the SIP proxy load-balancer, it will check in the database (among other things), for the last timestamp for the server which the user is trying to connect. If it was longer ago then a certain threshold (e.g., 15 seconds is currently being used) then it assumes that it has crashed and forwards the call to another asterisk server (let us call it Y). The Y asterisk server will detect that this call is meant to asterisk server X and tries to bridge to it. Since this operation will fail, it decides that the server X is down and "replaces" it. This replacement means that a new entry will be added in the database stating that server with the IP of server X is down and that calls directed to it are now to be sent to server Y (this is called the substitution process). After this is done, call gets established and the client will not even know that there is a certain server down. Any new calls which have the prefix of the server X will now be automatically directed to server Y, without the client having any idea that is actually not connecting to the server that he was meant to. 40

41 Illustration 17: Flow of events when a user joins a conference which causes the replacement of a server which has just crashed A slightly different scenario happens when a server which is holding some conferences crashes. Naturally, all the calls will be dropped since RTP traffic will no longer flow between client and server. From this point on, the flow of events that will occur will be the same as the one explained earlier: a user will call in to the crashed server, will be redirected to a different which will become its substitute server, and call is established in the substitute server. Finally, the most complex situation takes place when a server X which has bridges to other conferencing servers crashes. In this situation, again, all the clients connected to the server will be dropped as well as the IAX connections between the servers: however, the bridged servers (for instance, Y and Z) will retry to establish connection: when this fails (asterisks should retry since the connection may be down for a number of reasons), then the bridged servers will try to replace the crashed server (if there is more than one server connected, only one will suceed since the replacement process is, in practice, a database transaction which will only executed by one server). 41

42 Illustration 18: First picture describes the servers configuration when a conference is being bridged to 2 servers: after the crash of asterisk-x, userb gets disconnected, asterisk-y becomes the substitute server and creates a bridge to asterisk-z. UserB dials in again and gets connected directly to asterisk-y However, imagine the following scenario: 1. Conference is booked to server X 2. At a certain point, server X is too loaded and a new call gets established in server Y which bridges the conference to server X 3. After some time, server Y gets too loaded and a new call gets established in server Z which bridges the conference to server X 4. Server X crashes. As it was described previously, either server Y or Z will replace server X. Still, there are some users connected to the same conference in servers Y and Z which are not able to talk to each other since they were connected to each other through 42

43 server X. To solve this issue, the following rule must be added to the substitution process: Whenever a server is substituting a crashed server, it must check for any existing bridges before the crash and try to recreate those bridges Since information regarding bridges is kept in the database, then the replacement server simply needs to look up in the database for the bridges which were originated from server X. Having obtained this information, it is then only a matter of making new IAX connections to the conferences and servers specified in the database and update that same information. Illustration 19: Flow of messages exchanged when an asterisk server holding bridges goes down. Asterisk-X crashes causing userb to get disconnected and destroying the bridges from asterisk-x to Y and Z. Asterisk-Y will manage to become the substitute server, creating a bridge to asterisk-z and keeping the conference running. When userb dials in again, he will join server asterisk-y There is still a question to be answered which is how to put a server back up. One cannot simply put a server back up and say that it is now ready to accept calls 43