🚀 Listed in the CNCF Landscape and eBPF Foundation Emerging Project. If you like this project, give it a star on GitHub! ❤️

Blog

Why We Revived go-pcap for AI Infrastructure

tcpdump and libpcap already handle traditional packet capture well. However, using libpcap in a Go application requires CGO and system libraries. These dependencies also complicate static builds and cross-compilation.

In AI infrastructure, packets do not come only from network interfaces. Kernel packet drop tracing operates on skb; DPDK applications operate on rte_mbuf; and RDMA/RoCE traffic may traverse userspace data paths. These environments need a filter compiler that can be embedded into different observability pipelines.

We therefore resumed the development of go-pcap. It compiles tcpdump-style expressions into cBPF in pure Go, without libpcap or CGO. The resulting filters can be used for live capture, kernel observability, and userspace packet filtering.

What Problem Does It Solve?

go-pcap has two independent components:

  • The root pcap package opens devices, manages capture handles, attaches filters, and reads packets.
  • The filter package parses expressions and generates cBPF instructions for a specified link type and compilation target.

The main execution paths are:

tcpdump expression
Parser → AST → LinkType/Target → cBPF
                                  ├─ capture socket
                                  ├─ skb/eBPF
                                  └─ mbuf/eBPF (planned extension)

Separating filter compilation from packet capture allows the same expressions to be used in different observability scenarios. Each backend only needs to locate the packet bytes, translate the instructions, and inject the target program.

go-pcap is not intended to replace tcpdump or libpcap completely. Its current goal is to provide commonly used, verifiable, and embeddable filtering capabilities.

Live Capture: Compile and Attach Filters in Pure Go

For live capture, use OpenLive to open a device and SetBPFFilter to configure a filter:

handle, err := pcap.OpenLive(
	context.Background(),
	"eth0",
	1600,
	true,
	time.Second,
	pcap.DefaultSyscalls,
)
if err != nil {
	return err
}
defer handle.Close()

if err := handle.SetBPFFilter("tcp and port 443"); err != nil {
	return err
}

The expression is compiled into cBPF in the Go process and attached to the capture socket. The kernel discards non-matching packets, and the application reads only matching packets.

The repository also provides a pcap command-line tool. It supports common options including -i, -c, -n/-nn, -q, -v, -e, -X, -A, -s, and -p. It does not currently support reading or writing pcap files and is not intended to be a complete replacement for tcpdump.

skb Packet Drop Filtering

HUATUO dropwatch monitors tracepoint/skb/kfree_skb. The following command observes TCP packet drops on eth0 where the destination port is 443:

sudo dropwatch \
  --bpf-path bpf/dropwatch.o \
  --device eth0 \
  --filter "tcp dst port 443" \
  --duration 60 \
  --output json

When the eBPF program is loaded, HUATUO uses go-pcap to generate cBPF for link-layer and network-layer inputs. HUATUO then translates the cBPF into eBPF, adapts the program for the verifier, and injects it into the target program.

At runtime, dropwatch checks skb->mac_len to determine whether filtering should start at the MAC header or the network-layer header. Non-matching events return immediately in the kernel and do not enter the rate-limiting or userspace output paths.

The responsibilities are separated as follows:

  • go-pcap parses expressions and generates cBPF for a specified packet layout.
  • HUATUO locates the skb data, translates and injects the eBPF program, and reports the drop reason, device, protocol, and call stack.

This is the first non-traditional packet capture scenario in which go-pcap is used in production.

Extensions for mbuf and RDMA/RoCE

In DPDK, packets are stored in rte_mbuf. An observability tool can obtain an mbuf through a uprobe or USDT probe, locate the packet bytes, and apply the filter before submitting the event to perf/ringbuf.

ByteDance netcap already supports this observability model. It currently uses gopacket/pcap and libpcap to compile expressions, which adds deployment dependencies. go-pcap can replace libpcap for this compilation step.

Scenario Status Role of go-pcap
Live capture Built in Open handles, compile and attach filters, and read packets
HUATUO dropwatch Integrated Compile cBPF for L2/L3 skb input
DPDK mbuf / RDMA Planned extension Provide filter compilation; the mbuf backend is out of scope

Key Constraint

cBPF reads fields at byte offsets. The compiler must know whether byte 0 is an Ethernet header or an IP header. Otherwise, it generates incorrect offsets.

The packet layout for tcp and port 443 differs between the two input types:

Ethernet: Ethernet header | IP header | TCP header | Data
RAW:                      IP header | TCP header | Data

go-pcap requires the caller to specify the link type explicitly:

ethernet, err := filter.Compile(
	"tcp and port 443",
	filter.LinkTypeEthernet,
)

raw, err := filter.Compile(
	"tcp and port 443",
	filter.LinkTypeRaw,
)

LinkTypeEthernet corresponds to DLT_EN10MB and applies to input that contains an Ethernet header. LinkTypeRaw corresponds to DLT_RAW and applies to input that starts with an IPv4 or IPv6 header.

This distinction is particularly important for skb and mbuf. If rte_mbuf.data points to an Ethernet header, compile the filter for Ethernet. If the observation point provides an IP packet with the link-layer header removed, compile it for RAW input.

Link-layer rules cannot be used with RAW input. The compiler returns an explicit error instead of generating a program with undefined results:

_, err := filter.Compile("arp", filter.LinkTypeRaw)
if errors.Is(err, filter.ErrL2OnlyLinkType) {
	// Reject the rule or switch to the Ethernet layout.
}

Implementation Details

go-pcap currently supports IPv4/IPv6, ARP/RARP, TCP, UDP, SCTP, ICMP/ICMP6, VLAN/QinQ, MPLS. It also supports common conditions such as host, net, port, portrange, packet byte access, arithmetic comparisons, and len.

The main implementation challenges involve expression semantics and memory safety.

  • Control flow: and, or, and not must preserve operator precedence and short-circuit semantics. go-pcap first generates control flow with symbolic labels and then resolves the labels into relative cBPF jumps. This avoids manually maintaining jump offsets during recursive code generation.

  • Variable protocol offsets: VLAN and MPLS change the location of subsequent protocol headers. The compiler uses immutable cursors for expression branches so that an offset change in one branch does not affect another. Linux also requires support for VLAN offload. A VLAN tag may remain in the packet or may be stripped by hardware and stored in metadata. The socket target supports both paths.

  • Bounds checking: Packet byte access generates length checks. A truncated packet does not match the filter. An out-of-bounds read is never treated as zero and used in a subsequent calculation.

Validation

Different compilers do not need to generate identical cBPF instructions. They must produce the same accept or reject result for the same packet.

The current test suite includes:

  • Unit tests for parsing, validation, and instruction generation.
  • cBPF VM execution tests.
  • Tests for IPv4/IPv6, L2/L3/L4, VLAN/MPLS, and compound expressions.
  • Golden fixture comparisons against tcpdump 4.99.0 and libpcap 1.10.0.
  • Live capture integration tests on the loopback interface.

The project also provides benchmarks for parsing, compilation, and matching or rejecting packets in the VM. These benchmarks support repeatable measurements but do not claim performance results independent of the runtime environment.

Current Limitations

The following features are not yet implemented:

  • Numeric protocol identifiers and protochain.
  • IPv6 extension header traversal.
  • broadcast, which depends on the interface netmask.

Recognized but unsupported features return ErrUnsupportedFeature. Future development will prioritize link types backed by concrete requirements and packet samples.

Conclusion

AI infrastructure has extended network observability to skb, mbuf, and RDMA/RoCE data paths. go-pcap provides packet filter compilation that is implemented in pure Go, testable, embeddable, and programmable.

Project repository: github.com/huatuo-ai/go-pcap