CDR-Pusher Installation¶
CDR-Pusher is a Go Application that will push your CDRs (Call Detail Record) from your Telco Switch (Asterisk, FreeSWITCH or other supported switch http://www.cdr-stats.org/pricing/switch-connectors/) to the centralized PostgreSQL Database CDR-Pusher on the CDR-Stats server.rebo
3.1 Install / Run¶
Install Golang dependencies (Debian/Ubuntu):
$ apt-get -y install mercurial git bzr bison
$ apt-get -y install bison
Install GVM to select which version of Golang you want to install:
$ bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)
$ source /root/.gvm/scripts/gvm
$ gvm install go1.4.2 --binary
$ gvm use go1.4.2 --default
Make sure you are running by default Go version >= 1.4.2, check by typing the following:
$ go version
To install and run the cdr-pusher application, follow these steps:
$ mkdir /opt/app
$ cd /opt/app
$ git clone https://github.com/cdr-stats/cdr-pusher.git
$ cd cdr-pusher
$ export GOPATH=`pwd`
$ make build
$ ./bin/cdr-pusher
The config file cdr-pusher.yaml is installed at the following location: /etc/cdr-pusher.yaml
3.2 Configuration file¶
Config file /etc/cdr-pusher.yaml:
# CDR FETCHING - SOURCE
# ---------------------
# storage_source_type: DB backend type where CDRs are stored
# (accepted values: "sqlite3" and "mysql")
storage_source: "sqlite3"
# db_file: specify the database path and name
db_file: "/usr/local/freeswitch/cdr.db"
# Database DNS
# Use this with Mysql
db_dns: ""
# db_table: the DB table name
db_table: "cdr"
# db_flag_field defines the table field that will be added/used to track the import
db_flag_field: "flag_imported"
# max_fetch_batch: Max number of CDR to push in batch (value: 1-1000)
max_fetch_batch: 100
# heartbeat: Frequency of check for new CDRs in seconds
heartbeat: 1
# cdr_fields is list of fields that will be fetched (from SQLite3) and pushed (to PostgreSQL)
# - if dest_field is callid, it will be used in riak as key to insert
cdr_fields:
- orig_field: uuid
dest_field: callid
type_field: string
- orig_field: caller_id_name
dest_field: caller_id_name
type_field: string
- orig_field: caller_id_number
dest_field: caller_id_number
type_field: string
- orig_field: destination_number
dest_field: destination_number
type_field: string
- orig_field: hangup_cause_q850
dest_field: hangup_cause_id
type_field: int
- orig_field: duration
dest_field: duration
type_field: int
- orig_field: billsec
dest_field: billsec
type_field: int
# - orig_field: account_code
# dest_field: accountcode
# type_field: string
- orig_field: "datetime(start_stamp)"
dest_field: starting_date
type_field: date
# - orig_field: "strftime('%s', answer_stamp)" # convert to epoch
- orig_field: "datetime(answer_stamp)"
dest_field: extradata
type_field: jsonb
- orig_field: "datetime(end_stamp)"
dest_field: extradata
type_field: jsonb
# CDR PUSHING - DESTINATION
# -------------------------
# storage_dest_type defines where push the CDRs (accepted values: "postgres" or "riak")
storage_destination: "postgres"
# Used when storage_dest_type = postgres
# datasourcename: connect string to connect to PostgreSQL used by sql.Open
pg_datasourcename: "user=postgres password=password host=localhost port=5432 dbname=cdr-pusher sslmode=disable"
# Used when storage_dest_type = postgres
# pg_store_table: the DB table name to store CDRs in Postgres
table_destination: "cdr_import"
# Used when storage_dest_type = riak
# riak_connect: connect string to connect to Riak used by riak.ConnectClient
riak_connect: "127.0.0.1:8087"
# Used when storage_dest_type = postgres
# riak_bucket: the bucket name to store CDRs in Riak
riak_bucket: "cdr_import"
# switch_ip: leave this empty to default to your external IP (accepted value: ""|"your IP")
switch_ip: ""
# cdr_source_type: write the id of the cdr sources type
# (accepted value: unknown: 0, csv: 1, api: 2, freeswitch: 3, asterisk: 4, yate: 5, kamailio: 6, opensips: 7, sipwise: 8, veraz: 9)
cdr_source_type: 0
# SETTINGS FOR FAKE GENERATOR
# ---------------------------
# fake_cdr will populate the SQLite database with fake CDRs for testing purposes (accepted value: "yes|no")
fake_cdr: "no"
# fake_amount_cdr is the number of CDRs to generate into the SQLite database for testing (value: 1-1000)
# this amount of CDRs will be created every second
fake_amount_cdr: 1000
3.3 Deployment¶
CDR-Pusher application aims to be run as Service, it can easily be run by Supervisord.
3.3.1 Install Supervisord¶
Some Linux distributions offer a version of Supervisor that is installable through the system package manager. These packages may include distribution-specific changes to Supervisor:
$ apt-get install supervisor
3.3.2 Configure CDR-Pusher with Supervisord¶
Create an Supervisor conf file for cdr-pusher:
$ vim /etc/supervisor/conf.d/cdr-pusher-prog.conf
A supervisor configuration could look as follow:
[program:cdr-pusher]
autostart=true
autorestart=true
startretries=10
startsecs = 5
directory = /opt/app/cdr-pusher/bin
command = /opt/app/cdr-pusher/bin/cdr-pusher
user = root
redirect_stderr = true
stdout_logfile = /var/log/cdr-pusher/cdr-pusher.log
stdout_logfile_maxbytes=50MB
stdout_logfile_backups=10
Make sure the director to store the logs is created, in this case you should create ‘/var/log/cdr-pusher’:
$ mkdir /var/log/cdr-pusher
3.3.4 Supervisord Manage¶
Supervisord provides 2 commands, supervisord and supervisorctl:
supervisord: Initialize Supervisord, run configed processes
supervisorctl stop programX: Stop process programX. programX is config name in [program:mypkg].
supervisorctl start programX: Run the process.
supervisorctl restart programX: Restart the process.
supervisorctl stop groupworker: Restart all processes in group groupworker
supervisorctl stop all: Stop all processes. Notes: start, restart and stop won't reload the latest configs.
supervisorctl reload: Reload the latest configs.
supervisorctl update: Reload all the processes where the config has changed.
3.3.5 Supervisord Service¶
You can also use supervisor using the supervisor service:
$ /etc/init.d/supervisor start
3.4 Configure CDR-Pusher¶
Edit /etc/cdr-pusher.yaml
Get started by configuring the CDR source, this is your original CDR backend, for instance on Asterisk this can be MySQL, SQlite or Postgresql.
For Mysql & PostgreSQL you will need to configure the DNS too: https://github.com/go-sql-driver/mysql
Some of the settings to configure:
# storage_source_type: DB backend type where CDRs are stored
# (accepted values: "sqlite3" and "mysql")
storage_source: "mysql"
# Database DNS
db_dns: "root:password@/accounting"
Then configure the ‘CDR Pushing’ section, here you will need to define where the CDRs will go, this will ‘almost’ always be the ‘cdr-pusher’ database living on your CDR-Stats server.
Check your CDR-Stats installation, you should find the Database settings for cdr-pusher database in settings_local.py
Some of the settings to configure:
# storage_dest_type defines where push the CDRs (accepted values: "postgres", "riak" or "both")
storage_destination: "postgres"
# Used when storage_dest_type = postgres
pg_datasourcename: "user=postgres password=password host=localhost port=5432 dbname=cdr-pusher sslmode=disable"
3.5 Configure your Switch CDR with CDR-Pusher¶
You will need to configure CDR-Pusher and you Telco Switch to work together, for this we put some individual instructions for :
> Configure FreeSWITCH with CDR-Stats and CDR-Pusher - Configure FreeSWITCH with CDR-Stats and CDR-Pusher
> Configure Asterisk with CDR-Stats and CDR-Pusher - Configure Asterisk with CDR-Stats and CDR-Pusher
> Configure Kamailio with CDR-Stats and CDR-Pusher - Configure Kamailio with CDR-Stats and CDR-Pusher
3.6 Restart Supervisord¶
After changes in CDR-Pusher configuration you will need to restart supervisord, you can do so with gently with:
/etc/init.d/supervisor stop
/etc/init.d/supervisor start
3.7 Troubleshooting¶
An easy way to verify that CDR-Stats is running smoothly is to look at the logs.
Find the import log activity on CDR-Stats at:
tail -f /var/log/cdr-stats/djcelery_error.log
Find the import log activity on CDR-Pusher at:
tail -f /var/log/cdr-pusher/cdr-pusher.log
Check out the CDR-Stats Database ‘import_cdr’ to see realtime import:
python manage.py dbshell --database=import_cdr