Guide: Homer & CaptAgent for Kazoo Installation

[Anthony Manzella]

Hi everyone,

I've recently installed and configured a Homer Server and CaptAgent for my Kazoo Cluster. Since we have a nice new forum I wanted to create a quick and easy guide on what I've done to get homer working with Kazoo.

FIRST, CREDIT WHERE IT'S DUE:

The following sources were used.

https://github.com/sipcapture/homer-installer

https://www.powerpbx.org/content/homer-voip-monitoring-install-guide-v1

https://github.com/sipcapture/homer/wiki/Example%3A-GeoIP

I also have to thank Lorenzo Mangani over at Sipcatpure/QXIP as he assisted via email in getting the maps/GeoIP working correctly.

Also working on getting the "Dangerous Demo" dashboards and etc working. Will post back here on GeoIP and Dangerous Demo once I have them 100% completed. 

 

Now, on to install!

 

I used Debian for my Homer Server and the very easy to use "Install Script"

Homer Server:

Run the following script on your homer server

bash <( curl -s https://cdn.rawgit.com/sipcapture/homer-installer/master/homer_installer.sh )

Follow the prompts and at the end, you will have a working Homer Server.

 

Now onto Kazoo and CaptAgent

CaptAgent v6 Install

You will do this on every FreeSwitch and Kamailio Server in your cluster.

• Prerequisites
  • Debian 8 (jessie)

  • apt-get install build-essential git-core libexpat-dev libpcap-dev libjson0-dev libtool automake flex bison libuv-dev

    CentOS v6 & v7 

  • yum -y install epel-release
    yum -y install git gcc json-c-devel expat-devel libpcap-devel flex-devel automake libtool bison flex libuv-devel

     

 

• Install

cd /usr/src
git clone https://github.com/sipcapture/captagent.git
cd captagent
./build.sh
./configure
make && make install

 

• Configure (You will edit 3 files:)
  • socket_pcap.xml - Change RTCP Socket On by enable=true

  • nano /usr/local/captagent/etc/captagent/socket_pcap.xml

    <?xml version="1.0"?>
    <document type="captagent_module/xml">
        <module name="socket_pcap" description="HEP Socket" serial="2014010402">
        <profile name="socketspcap_sip" description="HEP Socket" enable="true" serial="2014010402">
            <settings>
            <param name="dev" value="any"/>
            <param name="promisc" value="true"/>
            <param name="reasm" value="false"/>
            <param name="tcpdefrag" value="false"/>
            <param name="capture-plan" value="sip_capture_plan.cfg"/>
            <param name="filter">
                <value>portrange 5060-5091</value>
            </param>
            </settings>
        </profile>
        <profile name="socketspcap_rtcp" description="RTCP Socket" enable="true" serial="2014010402">
            <settings>
            <param name="dev" value="any"/>
            <param name="promisc" value="true"/>
            <param name="reasm" value="false"/>
            <!-- size in MB -->
            <param name="ring-buffer" value="20"/>
            <!-- for rtp && rtcp < 250 -->
            <param name="snap-len" value="256"/>
            <param name="capture-filter" value="rtcp"/>
            <param name="capture-plan" value="rtcp_capture_plan.cfg"/>
            <param name="filter">
                <value>portrange 5060-50000 and len >=50 </value>
            </param>
            </settings>
        </profile>
        </module>
    </document>

    transport_hep.xml - You will make 3 changes here. Change capture-host value to the IP address of your Homer server, change capture-port to 9060 and change your capture-id to a unique number to identify this CaptAgent.

  • nano /usr/local/captagent/etc/captagent/transport_hep.xml

    <?xml version="1.0"?>
    <document type="captagent_module/xml">
        <module name="transport_hep" description="HEP Protocol" serial="2014010402">
        <profile name="hepsocket" description="Transport HEP" enable="true" serial="2014010402">
            <settings>
            <param name="version" value="3"/>
            <param name="capture-host" value="x.x.x.x"/>
            <param name="capture-port" value="9060"/>
            <param name="capture-proto" value="udp"/>
            <param name="capture-id" value="100"/>
            <param name="capture-password" value="myhep"/>
            <param name="payload-compression" value="false"/>
            </settings>
        </profile>
        </module>
    </document>

    sip_capture_plan.cfg - Uncomment the if(sip_has_sdp() ) section

  • nano /usr/local/captagent/etc/captagent/captureplans/sip_capture_plan.cfg

    capture[pcap] {
    
        # here we can check source/destination IP/port, message size
        if(msg_check("size", "100")) {
    
            #Do parsing
            if(parse_sip()) {
            #Can be defined many profiles in transport_hep.xml  
            
            if(!send_hep("hepsocket")) {
                clog("ERROR", "Error sending HEP!!!!");
            }
            
            if(sip_has_sdp())
            {
              #Activate it for RTCP checks
              if(!check_rtcp_ipport())
                {
                  clog("ERROR", "ALREADY EXIST");
                }
            }
    
            #Duplicate all INVITEs to JSON transport
            # if(sip_is_method() && sip_check("method","INVITE")) {
            #    #Can be defined many profiles in transport_json.xml
            #    if(!send_json("jsonsocket")) {
            #   clog("ERROR", "Error sending JSON!!!");
            #    }
            # }
            }
        }
        drop;
    }

     

• Configure init or systemd and default
  • Debian8

  • cp /usr/src/captagent/init/deb/jessie/captagent.service /etc/systemd/system/
    cp /usr/src/captagent/init/deb/debian/captagent.default /etc/default/captagent
    systemctl daemon-reload
    systemctl enable captagent
    systemctl start captagent

    CentOS 7

  • cp /usr/src/captagent/init/el/7/captagent.service /etc/systemd/system/
    cp /usr/src/captagent/init/el/captagent.sysconfig /etc/sysconfig/captagent
    systemctl daemon-reload
    systemctl enable captagent
    systemctl start captagent

    CentOS 6

  • cp /usr/src/captagent/init/el/6/captagent.init /etc/init.d/captagent 
    cp /usr/src/captagent/init/el/captagent.sysconfig /etc/sysconfig/captagent
    
    # Modify init.d change directory name from "bin" to "sbin"
    nano /etc/init.d/captagent
    APP_FILE=/usr/local/captagent/sbin/$prog
    
    chmod 755 /etc/init.d/captagent
    chkconfig --add captagent
    chkconfig captagent on
    service start captagent

     

This should give you a working install of Homer that captures SIP, RTCP and produce QOS reports. Below are screenshots from my install. If you have any issues/questions, post them and I'll do my best to answer.

 

[mc_]

If you have the time and inclination, create this as a doc on the Sysadmin repo. This will add it to the Sysadmin docs site and probably be more viewed.

This is great, btw!

[Darren Schreiber]

This is awesome. We are developing a sysadmin section - can we copy this to their? It will operate like a Wiki so people can edit it.

 

[Anthony Manzella]

Thanks guys. I wanted to give back to the community

@mc_ Yes I will post it there soon

@Darren Schreiber Feel free to copy

[amn]

On 7/13/2017 at 10:44 AM, Anthony Manzella said:

Thanks guys. I wanted to give back to the community

@mc_ Yes I will post it there soon

@Darren Schreiber Feel free to copy

If you want to "give back to the community" perhaps you should not be copying a lot of it from another website and taking credit without giving any to the work done by the website.  In particular the captagent part where the similarities are beyond coincidental

https://www.powerpbx.org/content/homer-voip-monitoring-install-guide-v1

Edited July 25, 2017 by amn (see edit history)

[Anthony Manzella]

1 hour ago, amn said:

If you want to "give back to the community" perhaps you should not be copying a lot of it from another website and taking credit without giving any to the work done by the website.  In particular the captagent part where the similarities are beyond coincidental

https://www.powerpbx.org/content/homer-voip-monitoring-install-guide-v1

You have to do a little better to find all of the sources I used to compile this  

It was 3 sites and my notepad in windows. 

The homer server install was more than that.

[amn]

I think it is obvious you copy/pasted key parts verbatim and are claiming it as your own work. 

Edited October 23, 2017 by amn (see edit history)

[Anthony Manzella]

I think you misunderstood my post. I stated I used a few sources out there. You identified one. 

It's funny to me how you are attempting to make it look like I tried to "claim as my own work"

I was able to achieve a working homer with Kazoo. I wanted to give something to the 2600hz forums since searching homer didn't really do me any good here. 

I posted everything that I did to get mine working here in this thread. This now makes 1 thread that users can view, follow and have a working homer with Kazoo. 

If this bothers you that much, I suggest you don't view the thread that I'll be doing for a working Graylog server with Kazoo for logging. It too will contain other sources and whatever else I combine to make a single post from start to finish on a working Graylog. I expect to have it posted this week. 

[amn]

90% of that is from one website.  You took out some comments and put in a link to the installer script.  10 minutes of copying/pasting and trying to take credit for it.  

Edited July 25, 2017 by amn (see edit history)

[Darren Schreiber]

Hi guys,

     Let's not turn this into a flamewar. There are plenty of older open source products out there whose forums are flame wars, and it impacts the product negatively. Let's keep it positive.

      Anthony - can you edit your original post please and simply cite the sources you used? Something like "I've compiled a list of instructions primarily from XXX but also from YYY, so credit to those guys. I am reposting the instructions here solely for convenience".

      I think that's pretty reasonable, the instructions are not some sort of secret but someone did spend time crafting them so they should get the credit for sure.

[Anthony Manzella]

1 minute ago, Darren Schreiber said:

Hi guys,

     Let's not turn this into a flamewar. There are plenty of older open source products out there whose forums are flame wars, and it impacts the product negatively. Let's keep it positive.

      Anthony - can you edit your original post please and simply cite the sources you used? Something like "I've compiled a list of instructions primarily from XXX but also from YYY, so credit to those guys. I am reposting the instructions here solely for convenience".

      I think that's pretty reasonable, the instructions are not some sort of secret but someone did spend time crafting them so they should get the credit for sure.

Yes, I have no problem with that. I'll post shortly when I'm back on my pc. 

[Anthony Manzella]

Darren,

Doesn't look like I'm able to edit the post, but the following sources were used.

https://github.com/sipcapture/homer-installer

https://www.powerpbx.org/content/homer-voip-monitoring-install-guide-v1

https://github.com/sipcapture/homer/wiki/Example%3A-GeoIP

I also have to thank Lorenzo Mangani over at Sipcatpure/QXIP as he assisted via email in getting the maps/GeoIP working correctly.

Also working on getting the "Dangerous Demo" dashboards and etc working. Will post back here on GeoIP and Dangerous Demo once I have them 100% completed. 

[Darren Schreiber]

I've edited the post on your behalf.

[Te Matau]

@Anthony Manzella Thanks for this. Helped me!

[Uzair Mahmud]

@Anthony Manzella  Thankyou for the updated guide in one place.

I am implementing homer on my servers. This guide helped a lot. I had a point of confusion. i was hoping you could clear it for me.  I found that there are two ways to capture packets to send to homer server. One is to use captagent. Other is to use inbuilt capabilities of Kamailio and Freeswitch to send packets to the Homer Server. Can you elaborate on the differences between the two and which way would you recommend?

Also have you ever implemented a strategy of multiple homer servers for redundancy. is it even possible? We have a client that wants everything to be in pairs of two or more.

[amn]

I think you are better off using CaptAgent.  I don't think the Freeswitch native agent does anything with RTP so it's a more basic implementation.  Using CaptAgent also simplifies things instead of different configs for Freeswitch and Kamailio.

[martin]

as in:

https://2600hz.atlassian.net/wiki/spaces/docs/pages/74743810/Homer+and+Kazoo

 

from 2015

[Anthony Manzella]

On 12/13/2017 at 3:57 AM, Uzair Mahmud said:

@Anthony Manzella  Thankyou for the updated guide in one place.

I am implementing homer on my servers. This guide helped a lot. I had a point of confusion. i was hoping you could clear it for me.  I found that there are two ways to capture packets to send to homer server. One is to use captagent. Other is to use inbuilt capabilities of Kamailio and Freeswitch to send packets to the Homer Server. Can you elaborate on the differences between the two and which way would you recommend?

Also have you ever implemented a strategy of multiple homer servers for redundancy. is it even possible? We have a client that wants everything to be in pairs of two or more.

Use captagent. As for multiple homer servers, I have not set this up but it shouldn’t be hard. 

[Uzair Mahmud]

@Anthony Manzella I see. Ok installing it right now.

i had another question. Do you use a seperate ethernet interface for captagent to send packets to homer. I am guessing if  i use the same interface its going to decrease the number of calls throgh my voip ethernet interface by a little bit more than half unless i do some sort of compression on the data being sent.

 

I could not install homer using your centos instructions.

The ones that worked for me are these: 

yum install redhat-lsb-core wget git 
cd /usr/src
wget https://cdn.rawgit.com/sipcapture/homer-installer/master/homer_installer.sh
chmod +x homer_installer.sh
./homer_installer.sh

Taken from Homer Git Repo

 

Edit: 

I also had to add the following two lines in sipinterface_1.xml to get the rtcp information working

<param name="rtcp-audio-interval-msec" value="5000"/>
<param name="rtcp-video-interval-msec" value="5000"/>

 

Now freeswitch and captagent is sending rtcp information. It is ggetting saved in the databse with the correct corellation id.  I am stuck at not being able to view it with Homer gui.  

[amn]

12 hours ago, Uzair Mahmud said:

@Anthony Manzella I see. Ok installing it right now.

i had another question. Do you use a seperate ethernet interface for captagent to send packets to homer. I am guessing if  i use the same interface its going to decrease the number of calls throgh my voip ethernet interface by a little bit more than half unless i do some sort of compression on the data being sent.

 

I could not install homer using your centos instructions.

The ones that worked for me are these: 

yum install redhat-lsb-core wget git 
cd /usr/src
wget https://cdn.rawgit.com/sipcapture/homer-installer/master/homer_installer.sh
chmod +x homer_installer.sh
./homer_installer.sh

Taken from Homer Git Repo

 

Edit: 

I also had to add the following two lines in sipinterface_1.xml to get the rtcp information working

<param name="rtcp-audio-interval-msec" value="5000"/>
<param name="rtcp-video-interval-msec" value="5000"/>

 

Now freeswitch and captagent is sending rtcp information. It is ggetting saved in the databse with the correct corellation id.  I am stuck at not being able to view it with Homer gui.  

I think one of the goals of HEP is to keep bandwidth low, so you probably don't have to worry about it adding too much extra traffic.  Another interface makes stuff more complicated so I would try avoid that if I were you.

Edited December 18, 2017 by amn (see edit history)

[Uzair Mahmud]

@amn After using homer i just realized only sip signalling packets are being sent and some rtcp packets after the interval i specify in freeswitch xml. Since there are also no RTP packets being captured,  the bandwidth used should not be a lot. there is also a payload compression parameter in /usr/local/captagent/etc/captagent/transport_hep.xml which should further reduce the payload size of monitoring data. 

[Uzair Mahmud]

Alright ended up fixing the RTP  QOS reporting. Apparently for centos 7 the homer script ends up installing kamailio 4.4 instead of 5 . in kam 4.4 the default setting is to store RTCP data in one table called rtcp_capture. Kamailio 5.5 partitions the data by date and stores it in rtcp_capture_date1 ,  rtcp_capture_date2 etc.

I had to set define('RTCP_TABLE_PARTITION', 1); to define('RTCP_TABLE_PARTITION', 0); on second last line of /var/www/html/homer/api/preferences.php

Now onto manually upgradation of kamailio 5 and enable partitoning of rtcp data. 

[Baze]

Thanks for getting this all in one place!  We were able to get working other than one piece here.  Has anyone been able to get the correlation between a leg and b leg working?  From your initial screenshots @Anthony Manzella it looks like your call ID has the b2b tag there, but from what we are looking at we don't have any sort of x-cid or other call leg id we can use to correlate the two legs.  @safarov had a bit of a patch/hack but as we'd love to avoid needing to reload this every time: https://github.com/sergey-safarov/kazoo/commit/2478d500e7f108fcba598bc7ca4001285a70dceb.  

In any case, just looking to see if anyone has that working.  Thanks!

[Vic]

Hello Baze,

Could you please elaborate with some degree of details about the X-CID (for general learning purpose for me)? I seen this https://github.com/sipcapture/homer/issues/601 and apply Sergey Safarov's  X-CID on Kazoo 4.3.

Should I add this correlation script "somewhere"?

Thanks,

Victor