unix: traffic shaping with linux

Examples

This is by no means comprehensive. I may add to this when I get more of a chance. There are Wonder Shaper or the ADSL Bandwidth Management HOWTO. (though I feel that they are inadequate or employ the wrong strategies).

tc, qdiscs, classes, filters, oh my!

tc, the traffic control tool, is used to configure the Linux kernel to accomplish the shaping, scheduling, policing, and dropping of packets.
Each interface by default has a root qdisc. By default, it uses pfifo_fast algorhythm (in our case, it will be configured to use HTB). Think of the root qdisc as the main container that everything resides. Inside the root qdisc, we can classify various types of traffic into classes and attach them to the root handle. After the classes have been defined, filters are used to match and redirect the packets into the right classes.

Using tc

Some example tc commands to create a root handle and attach some classes to it:

tc qdisc add dev eth0 root handle 1: htb default 60 - creates a root handle attached to eth0 using the HTB qdisc and classify packets as classid 60 by default
tc class add dev eth0 parent 1: classid 1:1 htb rate 116kbit - create a parent class using HTB with a maximum (capped) rate of 116 kilobits/s (about 90% of 128 kilobits/s)
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 25kbit ceil 116kbit prio 0 - create a leafnode class using HTB with a guaranteed minimum of 25 kilobits/s and a maximum ceiling of 116 kilobits/s and a priority of zero (highest - note that available bandwidth is offered in order of priority)
tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10 - attach another qdisc handle to class 1:10 using SFQ - this implements the SFQ round robin qdisc inside the HTB that kicks in only when the queue is saturated to ensure that no one "conversation" hogs the pipe.

Note: You always have to have a root handle, and that classes can only be attached to handles. The reason why dev (eth0) is specified for each command is that you can attach handles and classes with the same names to each of network interface (if you have multiple interfaces). Also, all of the commands below depend on these classes created by the commands above.
Some example tc commands to filter packets to the desired classes:

Create a filter that classifies acknowledgement packets (small ACK packets):tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32\
match ip protocol 6 0xff \
match u8 0x05 0x0f at 0 \
match u16 0x0000 0xffc0 at 2 \
match u8 0x10 0xff at 33 \
flowid 1:10
Creates a filter that classifies SSH packets:tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32\
match ip dport 22 0xfffe flowid 1:10

Note: You don't actually need to use any other tool other than tc to do traffic shaping. However, since the syntax is arcane and not generally considered human readable, I recommend using iptables (with classify if available) even though this adds overhead. Please don't ask me any questions on writing other tc filters to classify packets (I copied and pasted the above rules from other docs).

Using iptables (marking) with tc

Some example iptables commands to do similar as above:

Mark acknowledge packets of an established session between 40 and 100 bytes:
iptables -t mangle -A PREROUTING -p tcp --tcp-flags ALL ACK -m state --state ESTABLISHED -m length --length 40:100 -j MARK --set-mark 20
Mark SSH packets that starts new sessions with a packet length between 40 and 68 bytes:
iptables -t mangle -A PREROUTING -p tcp --dport 22 --syn -m state --state NEW -m length --length 40:68 -j MARK --set-mark 22

Now use the fw filter to classify the marked packets:

Create a filter that classifies packets based on the fwmark (20) on the packet as belonging to classid 1:10:
tc filter add dev eth0 parent 1:0 prio 0 protocol ip handle 20 fw flowid 1:10
Creates a filter that classifies packets based on the fwmark (22) on the packet as belonging to classid 1:10:
tc filter add dev eth0 parent 1:0 prio 0 protocol ip handle 22 fw flowid 1:10

Note: The advantage of marking packets with iptables is that the marking facilities are generally supported by stock kernels. If you have a kernel that supports iptables classify, use that instead (see below).

Using iptables (classify)

Mmm, classify:

Classify packets of an established session between 40 and 100 bytes:
iptables -t mangle -A POSTROUTING -p tcp --tcp-flags ALL ACK -m state --state ESTABLISHED -m length --length 40:100 -J CLASSIFY --set-class 1:10
Classify SSH packets that starts new sessions with a packet length between 40 and 68 bytes:
iptables -t mangle -A POSTROUTING -p tcp --dport 22 --syn -m state --state NEW -m length --length 40:68 -j CLASSIFY --set-class 1:10

Note:

Verify that /lib/modules/<kernel version>/kernel/net/ipv4/netfilter/ipt_CLASSIFY.o and /usr/lib/iptables/libipt_CLASSIFY.so exist to see if you have the required kernel support (note that the path may vary depending on your distribution).
If you do not have the kernel support and are planning to compile your own kernel, download the source for iptables and the latest patch-o-matic, apply the CLASSIFY patch, enable it in the .config, and also remember to recompile iptables.
The CLASSIFY target only work in the mangle table and the POSTROUTING chain.
It's possible to use both the iptables classifier and the tc filters together. Any traffic that does not get classified by iptables will be go through the tc filters.

Some sample scripts

/etc/init.d/shaper:

#!/bin/sh
# init script written by shane at knowplace dot org
# this script only creates the qdiscs and classes required for shaping, it
# does NOT create the necessary filters

INTERFACE='eth0'
rc_done="  done"
rc_failed="  failed"

return=$rc_done

TC='/sbin/tc'

tc_reset ()
{
	# Reset everything to a known state (cleared)
	$TC qdisc del dev $INTERFACE root 2> /dev/null > /dev/null
}

tc_status ()
{
    echo "[qdisc - $INTERFACE]"
    $TC -s qdisc show dev $INTERFACE
    echo "------------------------"
    echo
    echo "[class - $INTERFACE]"
    $TC -s class show dev $INTERFACE
}

tc_showfilter ()
{
    echo "[filter - $INTERFACE]"
    $TC -s filter show dev $INTERFACE
}

case "$1" in

    start)
	echo -n "Starting traffic shaping"
	tc_reset
	U320="$TC filter add dev $INTERFACE protocol ip parent 1:0 prio 0 u32"
	#
	# dev eth0 - creating qdiscs & classes
	#
	$TC qdisc add dev $INTERFACE root handle 1: htb default 60
	$TC class add dev $INTERFACE parent 1: classid 1:1 htb rate 116kbit
	$TC class add dev $INTERFACE parent 1:1 classid 1:10 htb rate 32kbit ceil 116kbit prio 0
	$TC class add dev $INTERFACE parent 1:1 classid 1:20 htb rate 22kbit ceil 116kbit prio 1
	$TC class add dev $INTERFACE parent 1:1 classid 1:30 htb rate 22kbit ceil 116kbit prio 2
	$TC class add dev $INTERFACE parent 1:1 classid 1:40 htb rate 20kbit ceil 116kbit prio 3
	$TC class add dev $INTERFACE parent 1:1 classid 1:50 htb rate 18kbit ceil 116kbit prio 4
	$TC class add dev $INTERFACE parent 1:1 classid 1:60 htb rate 2kbit ceil 116kbit prio 5
	$TC qdisc add dev $INTERFACE parent 1:10 handle 10: sfq perturb 10
	$TC qdisc add dev $INTERFACE parent 1:20 handle 20: sfq perturb 10
	$TC qdisc add dev $INTERFACE parent 1:30 handle 30: sfq perturb 10
	$TC qdisc add dev $INTERFACE parent 1:40 handle 40: sfq perturb 10
	$TC qdisc add dev $INTERFACE parent 1:50 handle 50: sfq perturb 10
	$TC qdisc add dev $INTERFACE parent 1:60 handle 60: sfq perturb 10
	tc_status
	;;

     stop)
	echo -n "Stopping traffic shaper"
	tc_reset || return=$rc_failed
	echo -e "$return"
	;;
	
    restart|reload)
	$0 stop && $0 start || return=$rc_failed
	;;
				
    stats|status)
	tc_status
	;;

    filter)
	tc_showfilter
	;;

    *)
	echo "Usage: $0 {start|stop|restart|stats|filter}"
	exit 1

esac
test "$return" = "$rc_done" || exit 1

Script placeholder:

I'm too lazy to write a custom script for you. =) I use the narc-custom.conf file to insert custom commands via narc to classify traffic. Below are some example commands. They are provided here only to show you what could be done. Please feel free to adapt them to your own needs.

# give "overhead" packets highest priority
iptables -t mangle -A POSTROUTING -o eth0 -p tcp --syn -m length --length 40:68 -j CLASSIFY \
  --set-class 1:10
iptables -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags ALL SYN,ACK -m length --length 40:68 \
  -j CLASSIFY --set-class 1:10
iptables -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags ALL ACK -m length --length 40:100 \
  -j CLASSIFY --set-class 1:10
iptables -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags ALL RST -j CLASSIFY --set-class 1:10
iptables -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags ALL ACK,RST -j CLASSIFY \
  --set-class 1:10
iptables -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags ALL ACK,FIN -j CLASSIFY \
  --set-class 1:10
# interactive SSH traffic
iptables -t mangle -A POSTROUTING -o eth0 -p tcp --sport ssh -m length --length 40:100 \
  -j CLASSIFY --set-class 1:20
iptables -t mangle -A POSTROUTING -o eth0 -p tcp --dport ssh -m length --length 40:100 \
  -j CLASSIFY --set-class 1:20
# interactive mail or web traffic
iptables -t mangle -A POSTROUTING -o eth0 -p tcp -m multiport --sport http,pop,imap,https,imaps \
  -j CLASSIFY --set-class 1:30
# dns lookups
iptables -t mangle -A POSTROUTING -o eth0 -p tcp --dport domain -j CLASSIFY --set-class 1:30
# ICMP, UDP
iptables -t mangle -A POSTROUTING -o eth0 -p udp -j CLASSIFY --set-class 1:40
iptables -t mangle -A POSTROUTING -o eth0 -p icmp -m length --length 28:1500 -m limit \
  --limit 2/s --limit-burst 5 -j CLASSIFY --set-class 1:40
# bulk traffic
iptables -t mangle -A POSTROUTING -o eth0 -p tcp --sport ssh -m length --length 101: \
  -j CLASSIFY --set-class 1:50
iptables -t mangle -A POSTROUTING -o eth0 -p tcp --dport ssh -m length --length 101: \
  -j CLASSIFY --set-class 1:50
iptables -t mangle -A POSTROUTING -o eth0 -p tcp --sport 25 -j CLASSIFY --set-class 1:50
iptables -t mangle -A POSTROUTING -o eth0 -p tcp --dport 6667 -j CLASSIFY --set-class 1:50

http://www.knowplace.org/pages/howtos/traffic_shaping_with_linux/examples.php

unix

Thursday, October 6, 2011

traffic shaping with linux