FreeBSD Device Polling Results for Gigabit Copper

In my post FreeBSD Device Polling I ran my tests over Gigabit fiber connections. I thought I would repeat the tests for Gigabit copper, connected by normal straight-through cables. (One benefit of Gigabit copper Ethernet NICs is there's no need for crossover cables.)

Although I booted my two test boxes, asa633 and poweredge, with kernels offering polling, neither interface had polling enabled by default. This is asa633's NIC:

em1: flags=8843 mtu 1500
        options=b
        inet6 fe80::20e:cff:feba:e726%em1 prefixlen 64 scopeid 0x4 
        inet 172.16.7.1 netmask 0xffffff00 broadcast 172.16.7.255
        ether 00:0e:0c:ba:e7:26
        media: Ethernet autoselect (1000baseTX )
        status: active

This is poweredge's NIC.

em1: flags=8843 mtu 1500
        options=b
        inet6 fe80::207:e9ff:fe11:a0a0%em1 prefixlen 64 scopeid 0x4 
        inet 172.16.7.2 netmask 0xffffff00 broadcast 172.16.7.255
        ether 00:07:e9:11:a0:a0
        media: Ethernet autoselect (1000baseTX )
        status: active

First I ran unidirectional TCP tests, from asa633 to poweredge, without polling.

asa633:/root# iperf -c 172.16.7.2 -t 60 -i 5
------------------------------------------------------------
Client connecting to 172.16.7.2, TCP port 5001
TCP window size: 32.5 KByte (default)
------------------------------------------------------------
[  3] local 172.16.7.1 port 58672 connected with 172.16.7.2 port 5001
[  3]  0.0- 5.0 sec  90.2 MBytes    151 Mbits/sec
[  3]  5.0-10.0 sec  91.1 MBytes    153 Mbits/sec
[  3] 10.0-15.0 sec  90.0 MBytes    151 Mbits/sec
[  3] 15.0-20.0 sec  91.2 MBytes    153 Mbits/sec
[  3] 20.0-25.0 sec  89.8 MBytes    151 Mbits/sec
[  3] 25.0-30.0 sec  90.9 MBytes    153 Mbits/sec
[  3] 30.0-35.0 sec  91.7 MBytes    154 Mbits/sec
[  3] 35.0-40.0 sec  92.0 MBytes    154 Mbits/sec
[  3] 40.0-45.0 sec  89.9 MBytes    151 Mbits/sec
[  3] 45.0-50.0 sec  90.1 MBytes    151 Mbits/sec
[  3] 50.0-55.0 sec  90.4 MBytes    152 Mbits/sec
[  3] 55.0-60.0 sec  91.0 MBytes    153 Mbits/sec
[  3]  0.0-60.0 sec  1.06 GBytes    152 Mbits/sec

Here is what the server saw.

poweredge:/root# iperf -s -B 172.16.7.2
------------------------------------------------------------
Server listening on TCP port 5001
Binding to local address 172.16.7.2
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[  4] local 172.16.7.2 port 5001 connected with 172.16.7.1 port 58672
[  4]  0.0-60.0 sec  1.06 GBytes    152 Mbits/sec

Interrupt levels for both systems was similar to the Gigabit copper results.

Here is the change with polling enabled via 'ifconfig em1 polling'. First, the client.

asa633:/root# iperf -c 172.16.7.2 -t 60 -i 5
------------------------------------------------------------
Client connecting to 172.16.7.2, TCP port 5001
TCP window size: 32.5 KByte (default)
------------------------------------------------------------
[  3] local 172.16.7.1 port 52789 connected with 172.16.7.2 port 5001
[  3]  0.0- 5.0 sec  79.2 MBytes    133 Mbits/sec
[  3]  5.0-10.0 sec  76.3 MBytes    128 Mbits/sec
[  3] 10.0-15.0 sec  80.4 MBytes    135 Mbits/sec
[  3] 15.0-20.0 sec    123 MBytes    207 Mbits/sec
[  3] 20.0-25.0 sec    126 MBytes    212 Mbits/sec
[  3] 25.0-30.0 sec    110 MBytes    185 Mbits/sec
[  3] 30.0-35.0 sec  89.1 MBytes    149 Mbits/sec
[  3] 35.0-40.0 sec  77.0 MBytes    129 Mbits/sec
[  3] 40.0-45.0 sec  76.8 MBytes    129 Mbits/sec
[  3] 45.0-50.0 sec    103 MBytes    172 Mbits/sec
[  3] 50.0-55.0 sec    128 MBytes    215 Mbits/sec
[  3] 55.0-60.0 sec    120 MBytes    201 Mbits/sec
[  3]  0.0-60.0 sec  1.16 GBytes    166 Mbits/sec

Now the server.

poweredge:/root# iperf -s -B 172.16.7.2
------------------------------------------------------------
Server listening on TCP port 5001
Binding to local address 172.16.7.2
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[  4] local 172.16.7.2 port 5001 connected with 172.16.7.1 port 52789
[  4]  0.0-60.0 sec  1.16 GBytes    166 Mbits/sec

Polling didn't improve the situation much for TCP.

Here are the results for unidirectional UDP tests, without polling.

This is the client.

asa633:/root# iperf -c 172.16.7.2 -u -t 60 -i 5 -b 500M
------------------------------------------------------------
Client connecting to 172.16.7.2, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size: 9.00 KByte (default)
------------------------------------------------------------
[  3] local 172.16.7.1 port 60193 connected with 172.16.7.2 port 5001
[  3]  0.0- 5.0 sec    129 MBytes    217 Mbits/sec
[  3]  5.0-10.0 sec    129 MBytes    217 Mbits/sec
[  3] 10.0-15.0 sec    129 MBytes    217 Mbits/sec
[  3] 15.0-20.0 sec    127 MBytes    212 Mbits/sec
[  3] 20.0-25.0 sec    129 MBytes    217 Mbits/sec
[  3] 25.0-30.0 sec    129 MBytes    217 Mbits/sec
[  3] 30.0-35.0 sec    129 MBytes    217 Mbits/sec
[  3] 35.0-40.0 sec    129 MBytes    217 Mbits/sec
[  3] 40.0-45.0 sec    129 MBytes    217 Mbits/sec
[  3] 45.0-50.0 sec    129 MBytes    216 Mbits/sec
[  3] 50.0-55.0 sec    129 MBytes    216 Mbits/sec
[  3]  0.0-60.0 sec  1.51 GBytes    216 Mbits/sec
[  3] Sent 1102470 datagrams
[  3] Server Report:
[  3]  0.0-60.0 sec    787 MBytes    110 Mbits/sec  0.042 ms 541153/1102469 (49%)
[  3]  0.0-60.0 sec  1 datagrams received out-of-order

Notice the huge drops. Here is the server.

poweredge:/root# iperf -s -u -B 172.16.7.2
------------------------------------------------------------
Server listening on UDP port 5001
Binding to local address 172.16.7.2
Receiving 1470 byte datagrams
UDP buffer size: 41.1 KByte (default)
------------------------------------------------------------
[  3] local 172.16.7.2 port 5001 connected with 172.16.7.1 port 60193
[  3]  0.0-60.0 sec    787 MBytes    110 Mbits/sec  0.042 ms 541153/1102469 (49%)
[  3]  0.0-60.0 sec  1 datagrams received out-of-order

Here are results with polling enabled.

The client:

asa633:/root# iperf -c 172.16.7.2 -u -t 60 -i 5 -b 500M
------------------------------------------------------------
Client connecting to 172.16.7.2, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size: 9.00 KByte (default)
------------------------------------------------------------
[  3] local 172.16.7.1 port 53387 connected with 172.16.7.2 port 5001
[  3]  0.0- 5.0 sec    163 MBytes    274 Mbits/sec
[  3]  5.0-10.0 sec    164 MBytes    275 Mbits/sec
[  3] 10.0-15.0 sec    164 MBytes    275 Mbits/sec
[  3] 15.0-20.0 sec    164 MBytes    275 Mbits/sec
[  3] 20.0-25.0 sec    164 MBytes    275 Mbits/sec
[  3] 25.0-30.0 sec    164 MBytes    275 Mbits/sec
[  3] 30.0-35.0 sec    163 MBytes    274 Mbits/sec
[  3] 35.0-40.0 sec    164 MBytes    275 Mbits/sec
[  3] 40.0-45.0 sec    164 MBytes    275 Mbits/sec
[  3] 45.0-50.0 sec    164 MBytes    275 Mbits/sec
[  3] 50.0-55.0 sec    164 MBytes    275 Mbits/sec
[  3]  0.0-60.0 sec  1.92 GBytes    275 Mbits/sec
[  3] Sent 1401731 datagrams
[  3] Server Report:
[  3]  0.0-60.0 sec  1.86 GBytes    267 Mbits/sec  0.023 ms 40962/1401730 (2.9%)
[  3]  0.0-60.0 sec  1 datagrams received out-of-order

Now the server.

poweredge:/root# iperf -s -u -B 172.16.7.2
------------------------------------------------------------
Server listening on UDP port 5001
Binding to local address 172.16.7.2
Receiving 1470 byte datagrams
UDP buffer size: 41.1 KByte (default)
------------------------------------------------------------
[  3] local 172.16.7.2 port 5001 connected with 172.16.7.1 port 53387
[  3]  0.0-60.0 sec  1.86 GBytes    267 Mbits/sec  0.024 ms 40962/1401730 (2.9%)
[  3]  0.0-60.0 sec  1 datagrams received out-of-order

Like the copper tests, polling really helps with UDP performance.

Given that polling can be enabled and disabled at will via ifconfig, I would like to see 'options DEVICE_POLLING' added to the GENERIC FreeBSD kernel.

Comments

Anonymous said…
Polling really helps with datagram performance in general, not just UDP. It's a huge performance increase for straight IP routing, which includes routing (and filtering) TCP packets. It's probably a huge performance benefit for Snort-style IPS as well, even when Snort is looking inside TCP.
6:27 PM
Andrei said…
But aren't those rates a little too low for gigabit cards ?
4:01 AM
Post a Comment

Popular posts from this blog

Zeek in Action Videos

Image
This is a quick note to point blog readers to my Zeek in Action YouTube video series for the Zeek network security monitoring project .  Each video addresses a topic that I think might be of interest to people trying to understand their network using Zeek and adjacent tools and approaches, like Suricata, Wireshark, and so on.  I am especially pleased with Video 6 on monitoring wireless networks . It took me several weeks to research material for this video. I had to buy new hardware and experiment with a Linux distro that I had not used before -- Parrot .  Please like and subscribe, and let me know if there is a topic you think might make a good video.
Read more

MITRE ATT&CK Tactics Are Not Tactics

Image
Just what are "tactics"? Introduction MITRE ATT&CK  is a great resource, but something about it has bothered me since I first heard about it several years ago. It's a minor point, but I wanted to document it in case it confuses anyone else. The MITRE ATT&CK Design and Philosophy document from March 2020 says the following: At a high-level, ATT&CK is a behavioral model that consists of the following core components: • Tactics, denoting short-term, tactical adversary goals during an attack; • Techniques, describing the means by which adversaries achieve tactical goals; • Sub-techniques, describing more specific means by which adversaries achieve tactical goals at a lower level than techniques; and • Documented adversary usage of techniques, their procedures, and other metadata. My concern is with MITRE's definition of "tactics" as "short-term, tactical adversary goals during an attack," which is oddly recursive. The key word in the tacti
Read more

New Book! The Best of TaoSecurity Blog, Volume 4

Image
  I've completed the TaoSecurity Blog book series . The new book is  The Best of TaoSecurity Blog, Volume 4: Beyond the Blog with Articles, Testimony, and Scholarship .  It's available now for Kindle , and I'm working on the print edition.  I'm running a 50% off promo on Volumes 1-3 on Kindle through midnight 20 April. Take advantage before the prices go back up. I described the new title thus: Go beyond TaoSecurity Blog with this new volume from author Richard Bejtlich. In the first three volumes of the series, Mr. Bejtlich selected and republished the very best entries from 18 years of writing and over 18 million blog views, along with commentaries and additional material.  In this title, Mr. Bejtlich collects material that has not been published elsewhere, including articles that are no longer available or are stored in assorted digital or physical archives. Volume 4 offers early white papers that Mr. Bejtlich wrote as a network defender, either for technical or pol
Read more