BSD firewall / Router Performance

Goals of this testing was to understand the following :

1) the new em driver to make sure it works well for me and give it a good shake out under load for RELENG_6

2) understand the implications of various configs on forwarding performance of SMP vs UP vs Polling vs Fast Interrupts and to avoid livelock when there is a lot of pps.

 

For more discussion, see http://lists.freebsd.org/pipermail/freebsd-performance/2006-November/002320.html

dmesg from box acting as router (r2)
r2 kernel config for freebsd
dmesg from 192.168.44.1
dmesg from 192.168.88.176
dmesg from 192.168.44.244
dmesg from 192.168.88.218
 

switch is Netgear GSM7212 using port based vlans.  All tests are done with a dual port Intel PCIe PT1000 NIC unless otherwise stated. There are a couple of tests that use a pair of Broadcom PCIe single port NICs (see dmesg for more detail)

Added Nov 24--

Test on HEAD from Nov 23rd with PMC in the kernel. UP results at  FastForward Off   FastForward On  and with ipfw in the kernel
and SMP results with ipfw in the kernel  and one firewall rule:  fastforward on, fastforward off (aka just HEAD) and net.isr.direct=1

Interface is always EM (Intel dualport Pro 1000PT PCIe 4x in 16x slot ) unless stated otherwise. All RELENG_4 tests are UP as SMP does not get detected on an AMD X2

HEAD is Always from Nov24th sources, unless otherwise noted.
firewall tests with NetBSD are done with ipfilter

*BGE Patch http://lists.freebsd.org/pipermail/freebsd-net/2006-November/012389.html
** ipfw might not be working properly on DragonFly Dev
*** Driver did wedge at one point in testing, but speed looks promising. See dmesg for detail