Thursday, May 6, 2010

ipt_netflow-1.6 - Netflow Based Monitoring Solution for Linux and iptables

After several days of investigation for the right solution for the network monitoring needs of my small company, I found a very powerful tool that helped me solve the problem and today I want to share it with you. Its name is ipt_netflow and it is an iptables module that can be used to understand what's going on in your network.

The company was needing "Eyes" on the network infrastructure, mainly on its firewalls to see what was going trough them and to understand the flow of the information inside of the corporate structure. Because it is a small company , we are using open source software everywhere including iptables for the firewalls and it made sense, then, to use open source software for the monitoring solution as well.

I have created an installation guide for ipt_netflow-1.6 under CentOS 5.4 that I would like to share with those interested on this subject and here it goes...Next post will be covering the same installation under Ubuntu 9.10.

First we will need to make sure our system complies with some basic packages needed to compile the ipt_netflow module.

Under CentOS 5.4 , and as root, install:

yum install iptables-devel.i386
yum install gcc.i386
yum install kernel-devel.i686
yum install kernel.i686
yum install gcc gcc-c++ kernel-devel

Then download ipt_netflow from:

http://sourceforge.net/projects/ipt-netflow/files/ipt-netflow/1.6/ipt_netflow-1.6.tgz/download

or execute the following command on your Linux box:

wget http://downloads.sourceforge.net/project/ipt-netflow/ipt-netflow/1.6/ipt_netflow-1.6.tgz

Unpack ipt_netflow-1.6.tgz:

tar -xzvf ipt_netflow-1.6.tgz

Move into the ipt_netflow folder :

cd ipt_netflow-1.6.tgz

Because CentOS normally goes a little behind with its package selection we need to edit one of the files provided within the ipt_netflow-1.6.tgz for this to work under CentOS 5.4. So let's edit the libipt_NETFLOW.c file:

vi libipt_NETFLOW.c

Search for "void _init(void)" and Replace the line:

void _init(void)

For the following line:

void __attribute((constructor)) my_init(void)

After this, save changes and exit.

Now we are ready to compile the module so execute:

./configure

and if it returns no errors then execute:

make all install; depmod

If the compilation returns no errors then the installation is done and now you need to load the module and setup the corresponding iptables rules.

First copy the ipt_NETFLOW.ko (wherever it was placed on your system) into the kernel modules folder (if the right kernel was installed it should be there already and this step is not needed):

cp /lib/modules/2.6.18-164.11.1.el5.centos.plus/extra/ipt_NETFLOW.ko /lib/modules/2.6.18-164.11.1.el5/extra/

Second load the module using modprobe:

modprobe ipt_NETFLOW destination=netflowserverip:port

Third add rules to the iptables at the top of the list:

iptables -A INPUT -j NETFLOW
iptables -A OUTPUT -j NETFLOW
iptables -A FORWARD -j NETFLOW



Now you only need to restart iptables for the flows to start being exported to your netflow server. In my case I'm using ManageEngine Netflow Analizer Free Edition which allows me to monitor a maximum of to interfaces for free. I strongly recommend you use the same application for this purpose since it is unbelievably good.

Before finishing the setup we need to tune up a little bit the ipt_netflow module. For this I like to set up the active_timeout to 10 because it is 1800 by default and this cause a delay in the flow reporting. To do this execute:

sysctl -w net.netflow.active_timeout=10

or if you want to make this permanent you would like to edit the /etc/sysctl.conf file and add this line to it:

net.netflow.active_timeout=10

It is also a good idea to load the ipt_netflow module automatically every time you restart your Linux box, for this you need to add the following line to the /etc/rc.local file:

modprobe ipt_NETFLOW destination=netflowserverip:port

and, of course, change the "netflowserverip:port" for the corresponding Server IP Address and Port.

You can take a look at some of the stats by issuing the following command in your Linux box:

cat /proc/net/stat/ipt_netflow

and you can also use tcpdump to check if the flows are being exported properly.

You will also need to add an extra rule to you iptables rule set, that allows the flows to get out of the Linux box and go to the Netflow Server. Something like the following line, but of course, changing the Interface, IP and Port shown, for the ones corresponding with your infrastructure:

iptables -A OUTPUT -o eth0 -p udp -d 192.168.0.70 --dport 9996 -j ACCEPT

Please let me know if you run into some problem while following this guide. I will be more than pleased to help any of you out there.

Enjoy.

P.D. - There is also an RPM for this but I couldn't make it work properly (http://centos.alt.ru/?p=306). Please let me know if anyone have made this work.

Here you are a couple of interesting links regarding ipt_netflow-1.6 module:

http://centos.alt.ru/?s=ipt_netflow

6 comments:

  1. Dude, super awesome post. I may be showing my iptables ignorance here, but do you think it would be possible to use ipt-netflow as a pure netflow probe? That is have one or two promiscuous interfaces (preferably without IP addresses) connected to a switch's mirror/SPAN port and forwarding that data as netflow to a collector.

    Can ipt even listen to an unnumbered promiscuous interface? I spose I could number the interfaces and forward all traffic to /dev/null or something. Any thoughts?

    ReplyDelete
  2. Hi James, I have tried that setup as well but for that scenario I have used a different tool named fprobe instead of ipt-netflow. fprobe will allow you to listen to the traffic coming from a mirror port on a switch, convert that traffic to flows and export those flows to the netflow server specified on the fprobe config file.

    fprobe is installed as a service on Linux and you can use the command line instead of a config file to pass options to the probe, using a command like the following:

    fprobe OPTIONS="-i eth0 -f "vlan&&ip" -p -B 4096 -r2 -q 10000 -t 10000:10000000 127.0.0.1:9996"

    Of course, adjust the command to suit your scenario. I hope this explanation was helpful.

    ReplyDelete
  3. Your post sure helped. We got fprobe working in the lab easily. Unfortunately now that we've deployed a live fprobe server it's only telling us about flows in one direction only. That or it's transposing all headers so that the local IPs are always the destination. I have confirmed through tcpdump that packets in both directions are reaching the unumbred promiscious sniffer interface. However the the flow records it produces always have the local IP as the destination. We haven't determined if fprobe is mangling the source/dest or if it is simply not reporting on outbound traffic. Do you have any places we can go for some good fprobe tips, docs or friendlies?

    ReplyDelete
  4. Old post, but on the top of g*gle output. So.. ipt-netflow is still developed, and you can try to participate in testing. Latest features is: IPv6 support, NetFlow v9/IPFIX protocols (which provide extended info about flows), NEL (NAT events logging), compatibility with latest linux kernels, many minor changes (like support of nexthop field). Of course many stability improvements since 1.6 mentioned in above atricle. git clone https://github.com/aabc/ipt-netflow and try.

    ReplyDelete
  5. This comment has been removed by the author.

    ReplyDelete