Hello,
We have recently seen a performance regression when using gopacket's pcap capabilities. We noticed the regression when we saw the number of packets being processed dropped significantly. When we investigated further we identified that the pcap handle Stats() was reporting Dropped to be around 98% of the total packets processed.
When we started to investigate the issue we rolled back the commits on gopacket and found that the regression was introduced in commit 7d9ba85c47bb41298fed10f5769f793741de187f
Here is the statistics output.
Aug 28 22:18:06 time="2017-08-28T22:18:06Z" level=info msg="Statistics received: 2480538, dropped: 0, interface dropped 0"
Aug 28 22:18:21 time="2017-08-28T22:18:21Z" level=info msg="Statistics received: 2480630, dropped: 0, interface dropped 0"
Aug 28 22:18:36 time="2017-08-28T22:18:36Z" level=info msg="Statistics received: 2480726, dropped: 0, interface dropped 0"
Aug 28 22:18:51 time="2017-08-28T22:18:51Z" level=info msg="Statistics received: 2480812, dropped: 0, interface dropped 0"
Aug 28 22:19:06 time="2017-08-28T22:19:06Z" level=info msg="Statistics received: 2480884, dropped: 0, interface dropped 0"
Aug 28 22:19:21 time="2017-08-28T22:19:21Z" level=info msg="Statistics received: 2480972, dropped: 0, interface dropped 0"
<<< REDEPLOY WITH NEW VERSION OF gopacket >>>
Aug 28 22:20:03 time="2017-08-28T22:20:03Z" level=info msg="Statistics received: 64, dropped: 36, interface dropped 0"
Aug 28 22:20:18 time="2017-08-28T22:20:18Z" level=info msg="Statistics received: 150, dropped: 122, interface dropped 0"
Aug 28 22:20:33 time="2017-08-28T22:20:33Z" level=info msg="Statistics received: 230, dropped: 202, interface dropped 0"
Aug 28 22:20:48 time="2017-08-28T22:20:48Z" level=info msg="Statistics received: 326, dropped: 298, interface dropped 0"
Aug 28 22:21:03 time="2017-08-28T22:21:03Z" level=info msg="Statistics received: 402, dropped: 374, interface dropped 0"
Aug 28 22:21:18 time="2017-08-28T22:21:18Z" level=info msg="Statistics received: 488, dropped: 460, interface dropped 0"
Aug 28 22:21:33 time="2017-08-28T22:21:33Z" level=info msg="Statistics received: 566, dropped: 538, interface dropped 0"
Aug 28 22:21:48 time="2017-08-28T22:21:48Z" level=info msg="Statistics received: 654, dropped: 626, interface dropped 0"
Aug 28 22:22:03 time="2017-08-28T22:22:03Z" level=info msg="Statistics received: 732, dropped: 704, interface dropped 0"
Aug 28 22:22:18 time="2017-08-28T22:22:18Z" level=info msg="Statistics received: 796, dropped: 768, interface dropped 0"
Aug 28 22:22:33 time="2017-08-28T22:22:33Z" level=info msg="Statistics received: 876, dropped: 848, interface dropped 0"
Aug 28 22:22:48 time="2017-08-28T22:22:48Z" level=info msg="Statistics received: 966, dropped: 938, interface dropped 0"
Aug 28 22:23:03 time="2017-08-28T22:23:03Z" level=info msg="Statistics received: 1046, dropped: 1018, interface dropped 0"
Aug 28 22:23:18 time="2017-08-28T22:23:18Z" level=info msg="Statistics received: 1102, dropped: 1074, interface dropped 0"
From the logs we can see that when we redeployed with the latest commits the number of drops started to increase. When we then reverted back to the previous binary the drops returned to zero. We then started to investigate and found that the problem was only introduced in the commit listed above.
System Info
hostname:~# go version
go version go1.8.3 linux/amd64
hostname:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
hostname:~# dpkg --list | grep pcap
ii libpcap-dev 1.7.4-2 all development library for libpcap (transitional package)
ii libpcap0.8:amd64 1.7.4-2 amd64 system interface for user-level packet capture
ii libpcap0.8-dev 1.7.4-2 amd64 development library and header files for libpcap0.8