How to Push CDR Files from Asterisk to SDReporter. September 27, 2013

Transcription

1 How to Push CDR Files from Asterisk to SDReporter September 27, 2013

2 Table of Contents Revision History Introduction Build Asterisk Configure Asterisk Load CDR Modules Configure CDR Support Verify CDR Support Status Utility Scripts Rotate Asterisk CDR File Export Asterisk CDR Files... 9 Asterisk is a trademark of Digium, Inc. TransNexus, OSP Secured and SDReporter are trademarks of TransNexus, Inc. 2

3 Revision History Revision Date of Issue Sep 2012 Draft Description Sep 2013 Correct minor typographic errors. Add comments to the cdrpush.sh script in section

4 1 Introduction This documentation explains how to build and configure Asterisk to generate CDR file in CSV format, rotate the CDR file, and export them to SDReporter. This documentation is for the Asterisk users who want to use SDReporter for CDR collection. This documentation is based on Asterisk version Build Asterisk To generate CDR in CSV format, Asterisk must be built with CDR CSV support. cdr_csv module must be built to provide CSV format support for CDR. func_cdr module may also be built to provide CDR dialplan function support. Both modules are built by default for Asterisk. They can be selected in menuselect application GUI (Call Destail Recoring->extended->cdr_csv, and Dialplan Functions->core- >func_cdr). They also can be explicitly selected by the following command lines. menuselect/menuselect --enable cdr_csv menuselect.makeopts menuselect/menuselect --enable func_cdr menuselect.makeopts 3 Configure Asterisk Assume Asterisk has been installed in $Asterisk_HOME folder. To generate the CDR file in CSV format for SDReporter, Asterisk must be configured to load CDR modules in order to generate CDRs in the correct format and with correct fields. 3.1 Load CDR Modules cdr_csv module must be loaded when Asterisk runs. func_cdr module may be loaded too. These two modules can be loaded by $Asterisk_HOME/etc/asterisk/modules.conf. load => func_cdr.so load => cdr_csv.so 3.2 Configure CDR Support There are several parameters in $Asterisk_HOME/etc/asterisk/cdr.conf that should be configured for Asterisk to generate CDRs in CSV format and with correct fields. 4

5 Asterisk Call Detail Record engine configuration A CDR is Call Detail Record, which provides logging services via a variety of pluggable backend modules. Detailed call information can be recorded to databases, files, etc. Useful for billing, fraud prevention, compliance with Sarbanes-Oxley aka The Enron Act, QOS evaluations, and more. [general] Define whether or not to use CDR logging. Setting this to "no" will override any loading of backend CDR modules. Default is "yes". enable=yes Define whether or not to log unanswered calls. Setting this to "yes" will report every attempt to ring a phone in dialing attempts, when it was not answered. For example, if you try to dial 3 extensions, and this option is "yes", you will get 3 CDR's, one for each phone that was rung. Default is "no". Some find this information horribly useless. Others find it very valuable. Note, in "yes" mode, you will see one CDR, with one of the call targets on one side, and the originating channel on the other, and then one CDR for each channel attempted. This may seem redundant, but cannot be helped. In brief, this option controls the reporting of unanswered calls which only have an A party. Calls which get offered to an outgoing line, but are unanswered, are still logged, and that is the intended behaviour. (It also results in some B side CDRs being output, as they have the B side channel as their source channel, and no destination channel.) unanswered = yes Normally, CDR's are not closed out until after all extensions are finished executing. By enabling this option, the CDR will be ended before executing the "h" extension so that CDR values such as "end" and "billsec" may be retrieved inside of of this extension. The default value is "no". endbeforehexten=no Normally, the 'billsec' field logged to the backends (text files or databases) is simply the end time (hangup time) minus the answer time in seconds. Internally, asterisk stores the time in terms of microseconds and seconds. By setting initiatedseconds to 'yes', you can force asterisk to report any seconds that were initiated (a sort of round up method). Technically, this is when the microsecond part of the end time is greater than the microsecond part of the answer time, then the billsec time is incremented one second. The default value is "no". initiatedseconds=no Define the CDR batch mode, where instead of posting the CDR at the end of every call, the data will be stored in a buffer to help alleviate load on the asterisk server. Default is "no". WARNING WARNING WARNING Use of batch mode may result in data loss after unsafe asterisk termination ie. software crash, power failure, kill -9, etc. WARNING WARNING WARNING 5

6 batch=no Define the maximum number of CDRs to accumulate in the buffer before posting them to the backend engines. 'batch' must be set to 'yes'. Default is 100. size=100 Define the maximum time to accumulate CDRs in the buffer before posting them to the backend engines. If this time limit is reached, then it will post the records, regardless of the value defined for 'size'. 'batch' must be set to 'yes'. Note that time is in seconds. Default is 300 (5 minutes). time=300 The CDR engine uses the internal asterisk scheduler to determine when to post records. Posting can either occur inside the scheduler thread, or a new thread can be spawned for the submission of every batch. For small batches, it might be acceptable to just use the scheduler thread, so set this to "yes". For large batches, say anything over size=10, a new thread is recommended, so set this to "no". Default is "no". scheduleronly=no When shutting down asterisk, you can block until the CDRs are submitted. If you don't, then data will likely be lost. You can always check the size of the CDR batch buffer with the CLI "cdr status" command. To enable blocking on submission of CDR data during asterisk shutdown, set this to "yes". Default is "yes". safeshutdown=yes CHOOSING A CDR "BACKEND" (what kind of output to generate) To choose a backend, you have to make sure either the right category is defined in this file, or that the appropriate config file exists, and has the proper definitions in it. If there are any problems, usually, the entry will silently ignored, and you get no output. Also, please note that you can generate CDR records in as many formats as you wish. If you configure 5 different CDR formats, then each event will be logged in 5 different places! In the example config files, all formats are commented out except for the cdr-csv format. Here are all the possible back ends: csv, custom, manager, odbc, pgsql, radius, sqlite, tds (also, mysql is available via the asterisk-addons, due to licensing requirements) (please note, also, that other backends can be created, by creating a new backend module in the source cdr/ directory!) Some of the modules required to provide these backends will not build or install unless some dependency requirements are met. Examples of this are pgsql, odbc, etc. If you are not getting output as you would expect, the first thing to do is to run the command "make menuselect", and check what modules are available, by looking in the "2. Call Detail Recording" option in the main menu. If your backend is marked with XXX, you know that the "configure" command could not find 6

7 the required libraries for that option. To get CDRs to be logged to the plain-jane /var/log/asterisk/cdr-csv/master.csv file, define the [csv] category in this file. No database necessary. The example config files are set up to provide this kind of output by default. To get custom csv CDR records, make sure the cdr_custom.conf file is present, and contains the proper [mappings] section. The advantage to using this backend, is that you can define which fields to output, and in what order. By default, the example configs are set up to mimic the cdr-csv output. If you don't make any changes to the mappings, you are basically generating the same thing as cdr-csv, but expending more CPU cycles to do so! To get manager events generated, make sure the cdr_manager.conf file exists, and the [general] section is defined, with the single variable 'enabled = yes'. For odbc, make sure all the proper libs are installed, that "make menuselect" shows that the modules are available, and the cdr_odbc.conf file exists, and has a [global] section with the proper variables defined. For pgsql, make sure all the proper libs are installed, that "make menuselect" shows that the modules are available, and the cdr_pgsql.conf file exists, and has a [global] section with the proper variables defined. For logging to radius databases, make sure all the proper libs are installed, that "make menuselect" shows that the modules are available, and the [radius] category is defined in this file, and in that section, make sure the 'radiuscfg' variable is properly pointing to an existing radiusclient.conf file. For logging to sqlite databases, make sure the 'cdr.db' file exists in the log directory, which is usually /var/log/asterisk. Of course, the proper libraries should be available during the 'configure' operation. For tds logging, make sure the proper libraries are available during the 'configure' phase, and that cdr_tds.conf exists and is properly set up with a [global] category. Also, remember, that if you wish to log CDR info to a database, you will have to define a specific table in that databse to make things work! See the doc directory for more details on how to create this table in each database. [csv] usegmtime=yes log date/time in GMT. Default is "no" loguniqueid=yes log uniqueid. Default is "no" loguserfield=yes log user field. Default is "no" accountlogs=no create separate log file for each account code. Default is "yes" [radius] usegmtime=yes log date/time in GMT loguniqueid=yes log uniqueid loguserfield=yes log user field Set this to the location of the radiusclient-ng configuration file 7

8 The default is /etc/radiusclient-ng/radiusclient.conf radiuscfg => /usr/local/etc/radiusclient-ng/radiusclient.conf 3.3 Verify CDR Support Status $ $Asterisk_HOME/sbin/asterisk -Wcdgvvvvv Asterisk , Copyright (C) Digium, Inc. and others. Created by Mark Spencer <markster@digium.com> Asterisk comes with ABSOLUTELY NO WARRANTY type 'core show warranty' for details. This is free software, with components licensed under the GNU General Public License version 2 and other licenses you are welcome to redistribute it under certain conditions. Type 'core show license' for details. ========================================================================= Asterisk Ready. == Parsing '/etc/asterisk/cli.conf': == Found *CLI> cdr show status Call Detail Record (CDR) settings Logging: Enabled Mode: Simple Log unanswered calls: Yes * Registered Backends csv 4 Utility Scripts To collect Asterisk CDRs at SDReporter, the Asterisk CDR file must be rotated and pushed to the correct folders on SDReporter. This section provides two utility scripts for this purpose. 4.1 Rotate Asterisk CDR File According to Asterisk documentation, the Asterisk application does not hold the CDR file (keep it open). The CDR file rotation script does the following things, 1. Generate the new CDR file name. The file name format is defined by SDReporter. 2. Rename the Asterisk CDR file to the new name 3. Generate a new empty Asterisk CDR file.!/bin/sh cdrrotate.sh This cdrrotate script is used to rotate Asterisk Master.csv file. It should be triggered by cron of the user runs Asterisk. When this file is run, the current Master.csv file is renamed. The new file name is Master-YYYYMMDDHHMMSS.csv. 8

9 Where YYYYMMDDHHMMSS stands for the year, month, day, hour, minute and second when the CDR file was renamed. When the Master.csv file is renamed, a new empty Master.csv file is automatically generated. This script should be run by user runs Asterisk. NEW_CDR_FILE_NAME=Master-`date +%Y%m%d%H%M%S`.csv if [! "$Asterisk_HOME" ] then echo "The Asterisk_HOME variable is not set." exit 1 fi cd $Asterisk_HOME/var/log/asterisk/cdr-csv mv Master.csv $NEW_CDR_FILE_NAME touch Master.csv 4.2 Export Asterisk CDR Files The rotated Asterisk CDR files should be exported to SDReporter. SDReproter may be installed on the same host or on a remote host. It should do the following things, 1. Define SCP parameters 2. Every rotated Asterisk CDR file should be copied to the SDReporter folder using scp. 3. If Asterisk CDR file is copied successfully to SDReporter, then remove the copied Asterisk CDR file from Asterisk directory.!/bin/sh cdrpush.sh This script automates the process of pushing rotated Asterisk CDR files to SDReporter. The remote SDReporter must support public key authentication. To enable public key authentication add the following lines to /etc/ssh/sshd_config on the remote SDReporter. RSAAuthentication=yes PubkeyAuthentication=yes Next, restart the ssh daemon (sshd) on the remote SDReporter by executing the command: /etc/init.d/sshd restart Then generate a public RSA key on the Asterisk and copy it to the list of authorized keys on the SDReporter. On the Asterisk machine execute (leave passphrase empty): ssh-keygen -t rsa 9

10 This will create public key at /home/<user>/.ssh/id_rsa.pub. Next, copy the public generated key to the SDReporter (for example by 'scp' command): scp /home/<user>/.ssh/id_rsa.pub After the Asterisk public key is copied to /tmp/id_rsa.pub of the SDReporter, login to the SDReporter and perform the following operation to add the Asterisk public key to the list of authorized keys by executing the command: mkdir /home/ossadmin/.ssh (only if it's needed) cat /tmp/id_rsa.pub >> /home/ossadmin/.ssh/authorized_keys When done, try to login to SDReporter from Asterisk: ssh If you can successfully ssh into the SDReporter from the Asterisk, then Asterisk is ready for scp without a password from this script. This script should be run by user runs Asterisk. Edit the variables shown below to match your systems. REMOTE_USER should be set to the SDReporter user name. The default user name for SDReporter is ossadmin. REMOTE_HOST is the IP address of the server hosting SDReporter. REMOTE_PATH is the SDReporter directory for Asterisk CDR files. If CDR files are created by only one Asterisk then this variable should be set as follows: REMOTE_PATH=/home/ossadmin/OSS/SDReporter/cdrconverter/data/INPUT/Asterisk/1 If there are multiple Asterisk servers, then each Asterisk should push CDR file to a different directory on SDReporter. For example, if there are two Asterisk servers, the second server should push CDR files to INPUT/Asterisk/2 REMOTE_USER=ossadmin REMOTE_HOST= REMOTE_PATH=/home/ossadmin/OSS/SDReporter/cdrconverter/data/INPUT/Asterisk/1 if [! "$Asterisk_HOME" ] then echo "The Asterisk_HOME variable is not set." exit 1 fi FILE_MASK=$Asterisk_HOME/var/log/asterisk/cdr-csv/Master-*.csv for file in $FILE_MASK do scp $file $REMOTE_USER@$REMOTE_HOST:$REMOTE_PATH > /dev/null 2>&1 if [ $? -eq 0 ] then 10

11 done fi rm $file 11