L3DGEWorld Asterisk Management System (LAMS)

Transcription

1 L3DGEWorld Asterisk Management System (LAMS) HET556 Design & Development 2 by Alana Huebner Carl Javier under the guidance of Professor Grenville Armitage Faculty of Information Communication Technology Swinburne University of Technology Melbourne, Australia November 2008

2 Abstract High performance computing is needed for evaluating computationally "bulky" problems. For years, computers have progressed in architecture and hardware aspects to exploit parallelism, but the algorithmic or software aspects are yet to be discovered fully in many fields. There are many aspects related to high performance computing, which we need to know before developing algorithms. Linear algebra lies at the heart of most calculations in scientific computing. Thus there is a need for developing computationally "rich" algorithms for linear algebra. In this report we present some ways to exploit parallelism for achieving high performance for linear algorithms, along with a summary of parallel processing and architecture overview with performance evaluation analysis.

3 Contents 1 Introduction L3DGEWorld Asterisk Management System Background Growth of Industry VoIP Telephony D verses 3D Monitoring Object and Scope Aim Our Approach Elements we do not address Overview of LAMS Visual Monitoring Collaborative Monitoring Variation of LAMS setups System Requirement Implementation of LAMS prototype Overview of the LAMS prototype The Scenario The LAMS Virtual World User Experience Simulation of a working VSP L3DGEWorld Map & Model Construction Asterisk Manager Interface Interaction Asterisk Manager Interface Connecting to AMI AMI Scripting and Programming Asterisk Call Records Call Records i

4 5.4.2 Statistics extracted for LAMS Ganglia Cluster Monitoring Utlisation Ganglia Overview Ganglia Installation Ganglia Configuration Custom Metric Insertion FreeBSD Jails Scenario of testing Hardware Specification FreeBSD Installation & Jail Building Creation of first jail Host network settings Jail Management Building Asterisk Jail Cloning Asterisk Jail Extraction of Telephony Statistics Overview of Software Modules AMI-scripts CDR-scripts Gmetric-grazer Scripts Grazer L3DGEWorld input daemon SIP call generation Testing Procedures & Results Impact of system on network Traffic Analaysis Polling Intervals Visual Interpretation Discussion Limitations of system Geographic & Latency issues Development Problems Encountered Timing Issues D verses 3D visual monitoring analysis Future Work More Management & Monitoring Features L3DGEWorld Development VoIP Protocol integration Incorporate Billing Statistics ii

5 8.5 Security Development & Implementation Conclusion 23 A Acronyms & Abbreviations 26 B Pythonising a C library using SWIG 27 B.1 SWIG interface file B.2 Compile C file B.3 Create and link new python module B.4 Testing new l3dgecomms python module iii

6 Chapter 1 Introduction 1.1 L3DGEWorld Asterisk Management System 1

7 Chapter 2 Background 2.1 Growth of Industry 2.2 VoIP Telephony 2.3 2D verses 3D Monitoring 2

8 Chapter 3 Object and Scope 3.1 Aim 3.2 Our Approach 3.3 Elements we do not address 3

9 Chapter 4 Overview of LAMS 4.1 Visual Monitoring 4.2 Collaborative Monitoring 4.3 Variation of LAMS setups 4.4 System Requirement 4

10 Chapter 5 Implementation of LAMS prototype 5.1 Overview of the LAMS prototype The Scenario Banana Beans is an espresso machine rental company that lends machines to Melbourne venues and businesses. They provide a help line for their customers, which is implemented through an IP PBX on the company s private network. Over the years the success of the business has warrented three separate VoIP servers to handle calls coming from differnt regions of Melbourne. Due to a growing number of customers in Geelong, Banana Beans plans to setup a VoIP server to dedicated to handling calls from Geelong customers. Due to the personnel costs of the help line, the company would like to monitor the help line closely. The want to know how often calls occur, who is calling the help line and the average call duration. Currently each server provides 9 channels to the help-line for customers in it s service area The LAMS Virtual World The Banana Beans scenario has been transfered to LAMS three-tiered heirachy. The top of the heirachy represents a map of Victoria. Two entities are placed on this map representing summaries of the VoIP services provided in Geelong and Melbourne. The middle tier of the heirachy represents the VoIP servers within a city. The middle tier representing Melbourne has three pairs of entites. Each pair represents a single server: the VoIP server (Asterisk) is represented by one of the entities and the underlying hardware and operating system represented by the other. The bottom tier of the heirachy represents the 9 help-line channels that a single VoIP server provides. What does it look like, what VR entities represent real world objects. Discuss mappings of metrics to visuals. Why were these choices made? 5

11 5.1.3 User Experience What the user can and cant do. Describe some scenarios and how the world would appear if, for example, a VoIP server went down, was very busy, was not busy at all Simulation of a working VSP Discuss architecture of the prototype network and introduce the methods used to produce traffic. Discussed in detail later. 5.2 L3DGEWorld Map & Model Construction 5.3 Asterisk Manager Interface Interaction L3DGEWorld Asterisk Management System (LAMS) provides network administrators, VoIP and Internet service providers to monitoring a cluster of VoIP servers. Future telecommunicaion companies may roll out VoIP and managing many servers remotely is definitely an advantage. Asterisk is VoIP PBX system. Asterisk can be remotely managed and monitored through ÒAsterisk Management InterfaceÓ (AMI) Asterisk Manager Interface Asterisk has an interface that allows external programs to interface with it for management and monitoring. All Asterisk CLI commands can be executed. AMI listens on a TCP port for a connection and once the connection is established, commands are send through this connection. Asterisk will respond to these commands appropriately. To enable AMI, managers.conf needs to be configured where general and security settings are set. [general] displaysystemname = yes enabled = yes #webenabled = yes port = 5038 [lams] secret=lams18 permit= / ; Authorization for various classes read = system,call,log,verbose,command,agent,user,config write = system,call,log,verbose,command,agent,user,config 6

12 *NB AMI should be avoided on enabling it on a public IP address Connecting to AMI Any TCP application written can connect to AMI. AMI expects key sequences written in clear text. A simple TCP application that can be used directly with AMI is telnet. Our use of telnet quickly proves you can manage and monitor Asterisk remotely. Though there are some security issues with telnet, they are of no concern in the first prototype and we expect AMI interface to be only enabled on a private and secured network. The sequence of actions AMI expects are the following: Action: <action type> Key 1: Value 1 Key 2: Value 2 etc... Variable: Value Variable: Value etc... An example using telnet to connect to AMI: # telnet localhost 5038 Trying Connected to localhost. Escape character is ^]. Asterisk Call Manager/1.0 Action: Login Username: lams Secret: lams18 Response: Success Message: Authentication accepted Action: Logoff Response: Goodbye Message: Thanks for all the fish. Connection closed by foreign host. AMI tends to show status and event messages so it is also ideal at login to AMI to turn this off with the following sequence: 7

13 Events: off A list of AMI commands can be viewed here though nearly every Asterisk CLI command is executable by the ÒAction: CommandÓ sequence AMI Scripting and Programming AMI is designed to be used by external applications and not by humans. Thus the sequence of commands are in a certain fashion. To script the connection we used a combination of shell, python and system calls to connect. An example script can be showed below and how it is executed. #!/bin/sh # HOST="pbx.javier.home" PORT="5038" echo "open $HOST $PORT" sleep 1 echo "Action: Login" echo "Username: lams" echo "Secret: lams18" echo ÒEvents: offó echo echo "Action: Command" echo "Command: Show Channels" echo /bin/sleep 1 echo echo "Action: Logoff" echo "END" which is then used in the following manner: #./asterisk_showchannels.sh telnet The output of AMI can be used to input data into the LAMS server to show the statistics and status of Asterisk remotely. The following diagram describes the logical and actually physical set up of our LAMS system. 8

14 System Metrics CPU Utlil Memory Disk Space Network Statistics VoIP Server 1 gmond UDP unicast gmetad VoIP Metrics Current Calls Channels in use Queues Application gmetric 5.4 Asterisk Call Records L3DGEWorld Asterisk Management System (LAMS) provides network administrators, VoIP and Internet service providers a way of monitoring a cluster of VoIP servers. An important task for providers is to keep records and charge their users. Asterisk uses a few ways to save all data about what users are using the system for. One of the simple ways Asterisk does this is through ÒCall RecordsÓ Call Records Asterisk by default will save all call statistics in call records. specified in asterisk.conf Logs and statistics are astlogdir => /var/log/asterisk Call records are found in /var/log/asterisk/cdr-csv/ in a file called ÒMaster.csvÓ. Master.csv is a simple file with the following fields. accountcode: What account number to use: Asterisk billing account, (string, 20 characters) src: Caller*ID number (string, 80 characters) dst: Destination extension (string, 80 characters) 9

15 dcontext: Destination context (string, 80 characters) clid: Caller*ID with text (80 characters) channel: Channel used (80 characters) dstchannel: Destination channel if appropriate (80 characters) lastapp: Last application if appropriate (80 characters) lastdata: Last application data (arguments) (80 characters) start: Start of call (date/time) answer: Anwer of call (date/time) end: End of call (date/time) duration: Total time in system, in seconds (integer), from dial to hangup billsec: Total time call is up, in seconds (integer), from answer to hangup disposition: What happened to the call: ANSWERED, NO ANSWER, BUSY, FAILED (on some CDR backends, e.g. ODBC, these may be integers; note that more detailed info can be found in the dialplan variable $HANGUPCAUSE) amaflags: What flags to use: see amaflags: DOCUMENTATION, BILLING, IG- NORE etc, specified on a per channel basis like accountcode. user field: A user-defined field, maximum 255 characters Master.csv will be written to once a call has ended. So Call Records cannot be used to view current statistics. AMI can be used for current calls Statistics extracted for LAMS LAMS will use the call records to provide a summary of how much users of a PBX have been using. Usually a telecommunications provider would use this information to bill their users. As LAMS is only a proof of concept, we will only summarise a few statistics such as Average call duration over all users Number of internal calls Number of local calls Number of interstate calls Number of international calls 10

16 5.5 Ganglia Cluster Monitoring Utlisation L3DGEWorld Asterisk Management System (LAMS) provides network administrators, VoIP and Internet service providers to monitoring a cluster of VoIP servers. As LAMS is a prototype, a tool known as ÔGanglia Monitor CoreÕ will be used to assist in the monitoring of clusters of VoIP servers. Ganglia monitoring core is designed for super computing. Most often researchers use ganglia to manage and monitor a super computer, which comprises of multiple cluster nodes. Each nodeõs metrics such as CPU utilisation, memory allocation and disk space is centralised to one main server. The end product is a tabulated web page that displays the entire super computerõs status at a glance. Our decision to use ganglia is to save time on development and also use gangliaõs method of using multicast to pass statistics to one node Ganglia Overview Ganglia monitor is implemented on every VoIP server. Ganglia comprises of daemons that run on the host system. The following section explains the daemons used by the LAMS system. gmond Daemon that collects information about the host system (eg CPU utilisation, memory usage and disk space). It will tabulate the information gathered and send it to the ganglia core server (gmetad). Also offers statistics on request using simple TCP packets. gmetad Daemon that resides on a central machine that collects all the statistics from other host machines running gmond. All the information collected is stored in a database. The database is then used to create a tabulated webpage of all the cluster node statistics. gmetric Tool to inject and monitor custom metrics of the host system. gmetric communicates with the host gmond daemon which in turn informs the central gmetad daemon running on the LAMS server. gmetric is one of the reasons we can adapt ganglia to a cluster of VoIP servers. All daemonsõ on each component of the system talk to each using multicast and pass around small XML files that include system statistics. Having this system already in built has saved us time at developing a way to transfer data from each VoIP server to the LAMS server. The following is diagram on how ganglia monitor core works. The following diagram describes the logical and actually physical set up of our LAMS system. 11

17 System Metrics CPU Utlil Memory Disk Space Network Statistics VoIP Server 1 gmond UDP unicast gmetad VoIP Metrics Current Calls Channels in use Queues Application gmetric Ganglia Installation Ganglia monitoring core and web interface under FreeBSD 7.0 can be built from ports: #cd /usr/ports/sysutils/ganglia-monitor-core #make install #cd /usr/ports/sysutils/ganglia-webfrontend #make install *NB. In order to use the ganglia webfront, a webserver such as Apache that supports PHP must be installed and configured. To test gmond and invoke an XML response: #gmond #telnet localhost Ganglia Configuration gmond.conf is used to set the configuration for the gmond daemon. By default the gmond config file is located in /usr/local/etc/gmond.conf. Each VoIP server will have the same gmond configuration. We leave the default system metrics. The following are the lines needed for LAMS setup: 12

18 cluster { name = "LAMS 1.0 Server" owner = "HET556" latlong = "unspecified" url = " } host { location = "Core Network" } gmetad.conf sets how many clusters gmetad will display. The lines edited in this file is the following : data_source "LAMS 1.0" localhost gridname ÒLAMS 1.0Ó For security reasons, trusted hosts can be set by the following: trusted_hosts cjavier.caia.siwn.edu.au Custom Metric Insertion Custom metrics can be inserted into the ganglia system. gmetric is the tool that is used to achieve this. To insert a custom metric use the following metric: #gmetric -name MetricName -value MetricValue -unit MetricUnit Ðtype MetricType *NB. MetricValue can be a script that writes to stdout. For example a script or tool called ÔnocurrentcallsÕ returns the value of how many calls are made on the VoIP server. The following will inject the custom metric value into the ganglia system. #gmetric -name PBX1-currentcalls -value nocurrentcalls --unit Calls -type int8 Once geometric is executed, the database and any other gmetad daemons listening will receive the update. In order to renew the value of the custom metric, the command needs to be run again. A cron job can be used to continually update the custom metric. 5.6 FreeBSD Jails L3DGEWorld Asterisk Management System (LAMS) provides network administrators, VoIP and Internet service providers to monitoring a cluster of VoIP servers. As LAMS 13

19 is a prototype and due to hardware availability and time restraints, VoIP clusters can be created by using FreeBSD jails. FreeBSD jails are sub systems within a host FreeBSD system that act as real systems. Jails are used to imprison services such as web and name servers. If the server is attacked or corrupted, the whole system is not compromised, only the jail. Jails are also useful for small scale system testing. This report outlines the steps taken to implement more than one identical FreeBSD jails for networking and testing purposes. Each jail is configured to run Asterisk 1.4 and act as an individual IP PBX Scenario of testing For testing/prototyping purposes we want to simulate a scaled down LAMS system. These consist of three VoIP servers, at least four to ten VoIP clients for each VoIP server and one LAMS monitoring server. Due to time and hardware availability, we are forced to simulate a working scaled down version of LAMS. The following diagram describes the logical and actually physical set up of our LAMS system Hardware Specification According to Asterisk PBX documentation a 3 GHz system with 1 GB of RAM can support up to 25 channels [insert reference]. Have three Asterisk jail systems with 3 5 phones requiring a channel each leaves us with left over CPU power for others applications we will be running. Our host jail system has the following system specifications: CPU: Intel Celeron 2.8GHz RAM: 1024 MB Hard Drive: 60 GB FreeBSD Installation & Jail Building A standard FreeBSD 7.0 installation with the source files and ports tree was made on our host jail system Creation of first jail Majority of the steps followed correspond to the FreeBSD handbook documentation [insert reference], but with added steps here and there. #mkdir - p /usr/home/jail/asterisk1 #cd /usr/src #make world DESTDIR=/usr/home/jail/asterisk1 #cd etc/ 14

20 #make distribution DESTDIR=/usr/home/jail/askterisk1 #mount - t devfs devfs /usr/home/jail/asterisk1/dev Host network settings Each jail can be assigned another address. The host system needs to alias the address to the network card. #ifconfig fxp netmask alias To start jail #jail /usr/home/jail/asterisk1 pbx1.lams.home /bin/tcsh A shell prompt and you should now be inside your jail pbx1# Create an empty /etc/fstab file so at startup there is no complaining. Edit /etc/rc.conf with the following IP address Hostname rpcbind_enable= NO default route sshd_enable= YES Edit /etc/resolv.conf for DNS resolving Add a primary user Create a root password Once created, run the following to load /etc/rc.conf pbx1# /bin/sh /etc/rc sshd and other services you have enabled in /etc/rc.conf should be running Jail Management From the host system, jails can be managed with the following: // can be used to start a jail #jail [path to jail] [hostname] [ip address] [shell to start] #jls // lists all the jails running on the host system jexec // can be used to enter an existing jail from the host system. 15

21 5.6.7 Building Asterisk Jail 1 From the host system, the jail should have a basic FreeBSD file system. To be able to build asterisk, the src and port tree should be added. #mkdir -p /usr/home/jail/asterisk1/usr/ports #mount_nullfs /usr/ports /usr/home/jail/asterisk1/usr/ports #mount_nullfs /usr/src /usr/home/jail/asterisk1/usr/src Once inside the jail, the ports tree is read only, thus the WRKDIRPREFIX and DISTDIR need to be specified as an environmental variable. pbx1#mkdir /usr/portswork pbx1#set env WRKDIRPREFIX=/usr/portswork pbx1#set env DISTDIR=/usr/portsworks/usr/ports/distfiles Now from within the jail, ports building will work Cloning Asterisk Jail 1 Our first jail is up and running with all the modifications we want. We want another two identical. To clone the first jail all that is required is to copy the jail root directory to another location making sure to preserve. cp - pr asterisk1 asterisk2 cp - pr asterisk1 asterisk3 Each jail needs the following tasks edit the [asterisk]/etc/rc.conf - for the hostname and ip address touch the etc/fstab any sripts or daemons specific to hosts. assign the IP address alias on the host system. 5.7 Extraction of Telephony Statistics L3DGEWorld Asterisk Management System (LAMS) provides network administrators, VoIP and Internet service providers a way of monitoring a cluster of VoIP servers. A standard Asterisk server has been set up with simulated calls coming in and out to and from any kinds of numbers. One task that LAMS needs to achieve is to extract the telephony and server statistics from Asterisk. 16

22 5.7.1 Overview of Software Modules To ease the asterisk information extraction development process, scripts were divided into three distinct purposes. AMI-scripts, CDR-scripts and Gmetric scripts AMI-scripts Scripts that interact with the Asterisk Manager Interface CDR-scripts Scripts that use the Call Records to extract information Gmetric-grazer Scripts Scripts that call ami and cdr scripts then inject that information into the ganglia system. A diagram of the scripts the information flow is shown below. The functionality of these scripts can be used to create one daemon that sits on the PBX server. 5.8 Grazer L3DGEWorld input daemon 5.9 SIP call generation 17

23 Logical Setup fxp0 LAMS fxp fxp0 Physical Setup fxp0 jail1 fxp0 alias1 jail2 fxp0 alias2 fxp0 alia fxp0 alias2 fxp0 alias3 LAMS jail3 fxp0 alias3 18

24 gmetric-grazer.py Asterisk Manager Interface Scripts Call Record Scripts active_calls.py active_channels.py asterisk-clients.py asterisk-uptime.sh asterisk_hangchannel.sh asterisk_reload.sh asterisk_showchannel.sh asterisk_showpeers.sh no_internal_calls.py no_local_calls.py no_insterstate_calls.py no_internal_calls.py call_duration.py 19

25 Chapter 6 Testing Procedures & Results 6.1 Impact of system on network Traffic Analaysis 6.2 Polling Intervals 6.3 Visual Interpretation 20

26 Chapter 7 Discussion 7.1 Limitations of system Geographic & Latency issues 7.2 Development Problems Encountered Timing Issues 7.3 2D verses 3D visual monitoring analysis 21

27 Chapter 8 Future Work 8.1 More Management & Monitoring Features 8.2 L3DGEWorld Development 8.3 VoIP Protocol integration 8.4 Incorporate Billing Statistics 8.5 Security Development & Implementation 22

28 Chapter 9 Conclusion 23

29 Acknowledgment We would like to express my deep sense of gratitude to Professor Grenville Armitage, for hist invaluable help and guidance during the course of project. We are highly indebted to them for constantly encouraging me by giving their critics on my work. We are grateful to them for having given me the support and confidence. Gaurav Trivedi May 2000 Indian Institute of Technology, Bombay 24

30 References 25

31 Appendix A Acronyms & Abbreviations 26

32 Appendix B Pythonising a C library using SWIG L3DGEWorld Asterisk Management System (LAMS) provides network administrators, VoIP and Internet service providers to monitoring a cluster of VoIP servers. Primarily its purpose is for intrusion detection systems for networks. L3DGEWorld has been used for many other monitoring systems in the past [INSERT REFERENCE]. As L3DGEWorld is a diverse concept, various input daemons have been written to interact with the Quake III engine that L3DGEWorld sits on. The various daemon inputs all implement a variation on a C library called Ôl3dgecommsÕ. As it is written in C, developers are sometimes force to write the entire daemon in C. SWIG is a solution to porting a C library to higher level languages such as Perl, Java and Python. This report outlines how Ôl3dgecommsÕ library was ported as a Python module. B.1 SWIG interface file SWIG needs an interface file into the C library. The interface file allows SWIG to create a wrapper for the C. The following Ôl3gecomms.iÕ interface file was created. \%module l3dgecomms \%{ /* Place include tags in this section */ #include "l3dgecomms.h" \%} \%include "l3dgecomms.h" /* Define all extern functions in C file */ Then swig is used to create a wrapper c file. 27

33 #swig -python l3dgecomms.i B.2 Compile C file Compile the c file and wrapper c file. ÐI is the location of the c header files on your system. It will vary from system to system. #gcc -c l3dgecomms.c l3dgecomms_wrap.c -I /usr/local/include/python2.5 B.3 Create and link new python module #ld -shared l3dgecomms.o l3dgecomms_wrap.o Ðo _l3dgecomms.so B.4 Testing new l3dgecomms python module Python can now use the l3dgecomms module and use all functions that are in the c library. To test the library, we use the three functions needed to update the LAMS server. import os, sys, string import l3dgecomms l3dgecomms.setupcomms(none,0," ",27960) l3dgecomms.sendmessage(1,1,"r","6") l3dgecomms.closecomms() 28