[Gllug] iptables with 1000s of IP addresses

[C. Cooke]

On Sun, Dec 28, 2008 at 05:51:25PM +0000, Richard Jones wrote:
> I've been slowly adding the IP addresses of people who (try to) add
> comment spam to my sites to a big IP drop list.  Currently each IP in
> the list is just added to a DROP rule in the INPUT table.
> 
> The list hit the 1000 mark recently (in fact, 1221 addresses right
> now) and is growing at ~ 50 new addresses / day.
> 
> At the moment, iptables seems to be handling all of this OK, but ...
> 
> Can I measure the overhead?
> 
> Are there more efficient solutions?  I've heard about nfqueue, but has
> anyone used it?  It seems like it would be quite inefficient because
> it involves a transition to userspace and back to the kernel for each
> incoming packet.

I used to work for a company that had a firewall to remap two distinct
networks that happened to use overlapping address space - those two
networks being "Internal company traffic" and "BT's ISDN network". My
employer at the time was one of the largest single ISDN and ADSL
customer of BT, having a line installed to tens of thousands of venues.
The ISDN users were presented as a VPN onto the BT network. Their
firewall allowed us to talk to any of the ISDN sites we were authorised
to.

Because the networks overlapped, we needed to do bidirectional NAT at
the point the VPN terminated. This meant a Linux box handling
redirections for tens of thousands of NAT sites. The iptables config -
which was around 30 lines of setup stuff and then screeds of NAT rules -
ran to over five megabytes by the time I left. We could have shortened
that by around 60-70% by using the NETMAP module - but that would need a
kernel recompile, which would have meant downtime...

Things I learned dealing with that system:

1) Iptables - even if it takes around five minutes to read in its config
- is atomic. Thank God. 

2) Iptables can handle an absolutely staggering number of firewall rules
without any effect on routing speed. Those boxes may have only been
ISDN, but put enough of them together and that's a hell of a lot of
data. 

3) At least at that time, Linux was more capable than Cisco or any of
the other big network vendors when it comes to massive rulesets - before
I joined, the company had tried to do the job on some decent firewall
kit and found serious performance problems. 

-- 
d=(1 0 6 0 1 0 5 5 41 5 3 12 4 5 15 1 4 -2 5 5 0 5 4 24 3 5 27 1 3 -2 1 3 6)
a=0;while :;do ((v=(c=a)+3));((x=d[d[a]]-d[d[a+1]]));d[d[a]]=$x;((a=d[d[a]]\
<0?${d[a+2]}:v));case $a in -1)read d[d[c]];a=$v;;-2)echo ${d[d[c+1]]};a=$v\
;;0)exit;;esac;done 2>&- # Charles Cooke, Sysadmin.  
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug