Friday, February 1, 2008

How to userspace l7 filter on Ubuntu

L7-filter is a classifier for Linux's Netfilter that identifies packets based on application layer data. It can classify packets as Kazaa, HTTP, Jabber, Citrix, Bittorrent, FTP, Gnucleus, eDonkey2000, etc., regardless of port. It complements existing classifiers that match on IP address, port numbers and so on. L7 filter is usefull if you want to limit or monitor different network protocols in your network. This article assumes that you have the basic knowledge how to use the terminal and set up a linux router.

This how to describes the step by step installation of Userspace L7 filter on Ubuntu Linux.

Installation

1. Method:

Add my PPA to you repository

deb http://ppa.launchpad.net/kuscsik/ubuntu hardy main
deb-src http://ppa.launchpad.net/kuscsik/ubuntu hardy main
and install l7-filter-userspace l7-protocols

2. Method:
  • Add the universe repositories to your sources list.
  • Install some necessary packages

    sudo apt-get install g++ build-essential
    sudo apt-get install libnetfilter-conntrack-dev libnetfilter-conntrack1 libnetfilter-queue-dev libnetfilter-queue1


  • Download the source package of userspace l7 filter and the archive of the protocol definitions.
  • Unpack the protocol definitions and install it by

    sudo make install

  • Unpack the userspace l7 filter, compile and install it

    # tar -xzf l7-filter-userspace-v0.10.tar.gz
    # cd l7-filter-userspace-v0.10/

  • Download this patch

    # wget http://kuscsik.com/download/patch_l7_Zoltan_Kuscsik.patch

    # cd
    l7-filter-userspace-v0.10
    # patch -p1 < ../patch_l7_Zoltan_Kuscsik.patch
    # /configure
    # make


  • Load a necessary kernel module and add to the automatically loaded modules

    # sudo modprobe ip_conntrack_netlink
    # sudo bash -c "echo ip_conntrack_netlink >> /etc/modules"

Testing

Let assume that you have a Ubuntu server with two network devices with WAN device eth0 and LAN device eth1. With iptables we redirect forwarded packages to the userspace

# sudo iptables -A FORWARD -j NFQUEUE --queue-num 0

Create a l7 config file in your favorite editor

sudo vi /etc/l7_filter.conf

and add some protocols

ssh 5
bittorrent 6


The list of all supported protocols can be found here. From the config we can read that l7 filter will assign the mark 5 to the packets of ssh protocol and it will sign with mark 6 the bittorrent packets.

Start l7 filter in debuging mode

l7-filter -vv -f /etc/l7_filter.conf

On your LAN client run a bittorrent session or make an ssh connection to an external server. If everything goes fine l7-filter will recognize your traffic and inform you trough the standard output.

Usage

Now, we are able to mark the connections. The next step is to set up the traffic control. For this you can use the HTB packet sheduller.
[to be improved...]

You can set up the start of l7-filter automatically by adding a line to /etc/rc.local

nohup /usr/local/bin/l7-filter -f /etc/l7-filter.conf &

Warning! This is a dangerous solution on a production server. If the program terminates unexpectedly your users will be cut off from the internet. You should set up some kind of monitoring for the l7-filter process.
You will notice, that the filter uses significant amount of system resources. You can save some computation time and memory by improving the forwarding rule.
For example, if you plan to shape the outgoing peer-to-peer connections it is a good idea to add some port specific selection to the userspace redirection rule

sudo iptables -A FORWARD -i eth0 -p tcp --dport 1024: -j NFQUEUE --queue-num 0

which will pass to the filter only the packets targeting higher ports than 1024.

I will improve this article, so please subscribe to the feed. If
you have any question or comments don't hesitate to drop me a message.

Please read the home page of Layer 7 Userspace

http://l7-filter.sourceforge.net/HOWTO-userspace

Especially the section why it is a bad idea to use the packet filtering for service blocking.

10 comments:

dixon said...

Hi,
when I modprobe the ip_conntrack_netlink module the nf_conntrack_netlink module is loaded instead of ip_con... Then when I try to run l7-filter I get warning "The ip_conntrack_netlink module does not appear to be loaded...."

Anonymous said...

Dixon > I just found solution that works. Open file using text editor 7-filter-userspace-v*.*/l7-filter.cpp

and change
"if(!check_for_module("ip_conntrack_netlink"))" to "if(!check_for_module("nf_conntrack_netlink"))"
and compile it.

Diablo said...

Hi ! First of all, nice post here :-)

Second, I have an issue, after I edit l7_filter.conf and put bittorent with mark 6, l7-filter marks all the packets with 6, so I could just simple use htb-tools to make a class with mark 6, and the speed limit I want ? I'm a liitle confused about this :)
Thanks a lot !

Anonymous said...

l7-filter-userspace seems to fail on compile with ubuntu hardy :(

BH said...

To compile on ubuntu intreprid x64:

change l7-filter.cpp and l7-conntrack.cpp

#include netinet/in.h must be in fist position

extern "C" {
#include netinet/in.h
#include linux/netfilter.h

Anonymous said...

Cheers BH for the compile tip!

(and of course zoltan for the wee howto)

On an non production server running music and torrent stuff, not routing etc. is there any reason not to drop packets?

Anonymous said...

Cheers BH for the compile tip!

(and of course zoltan for the wee howto)

On an non production server running music and torrent stuff, not routing etc. is there any reason not to drop packets?

Andrew said...

Old post, I know, but do you have any plans to through together an L7 package for Intrepid? So far I've been failing at compiling either into the kernel (tried once) or for userspace (tried a large number of times... hours fixing things, etc, etc). I'm no dope when it comes to compiling things, but the make error output was not useful.

Zoltan Kuscsik said...

Andrew, I created a small patch that will fix the issue with intrepid ibex. Please read the updated post.

Anonymous said...

How can i drop the packet i want to drop with layer7 in ubuntu (p2p, bittorrents, ...) ? I've tried with iptables but i don't know how....

Thanks