A rising RX dropped or TX dropped counter does not automatically mean that a cable, switch, or upstream carrier is failing. Packets can be discarded at several points: the network adapter, its driver, an RX ring, the kernel networking backlog, a traffic-control queue, a virtual interface, or the cloud host underneath a virtual machine. The reliable approach is to identify where the counter is increasing, then correlate it with application symptoms, driver statistics, CPU load, and softirq activity.
1. Confirm Whether Applications Are Actually Losing Traffic
An interface counter can increase without causing a visible outage. Start by recording the time window, interface, destination, protocol, and traffic direction. At the same time, check latency, retransmissions, timeouts, and application errors. The following commands provide a useful baseline:
date ip -s link ss -s nstat -az sar -n DEV 1 ping -c 20 target.example.com mtr -rwzc 50 target.example.com
ping and mtr are supporting evidence, not a complete diagnosis. Intermediate routers may rate-limit ICMP. If one hop reports loss while later hops respond normally, that alone does not prove that production traffic is being dropped at that hop. For TCP workloads, also examine retransmissions, request timeouts, and logs on both ends.
2. Read the Interface Counters with ip
ip -s link ip -s -s link show dev eth0
Watch the RX and TX values for errors, dropped, and overrun. Capture two or more samples 30 to 60 seconds apart. A large lifetime counter is less useful than the rate at which it is currently increasing.
RX errors: frames that encountered an error while being received; the exact meaning depends on the device and driver.
RX dropped: packets discarded somewhere on the receive path, which may involve buffers, backlog processing, protocol handling, or a virtual interface.
TX errors: errors encountered during transmission.
TX dropped: packets discarded before transmission by a queue, traffic-control rule, driver, or device.
Drivers do not map every statistic in exactly the same way. Treat the generic interface counters as a starting point, not as proof of a specific root cause.
3. Use ethtool to Inspect the Adapter and Driver
ethtool eth0 ethtool -S eth0 ethtool -g eth0 ethtool -k eth0
ethtool eth0 reports properties such as link speed, duplex mode, and link detection. ethtool -S exposes extended statistics supplied by the driver or hardware. These may include CRC, frame, missed, no-buffer, timeout, reset, and per-queue drop counters. Names vary by driver, so interpret them with the adapter model and driver documentation in mind.
If CRC or frame-related counters continue to rise, investigate the physical link, transceiver, cable, switch port, and speed or duplex negotiation. If missed, no-buffer, or one specific RX queue grows instead, focus on receive queues, CPU processing capacity, and interrupt distribution.
4. Check CPU, Softirq, and IRQ Distribution
A server can drop packets before its total bandwidth is saturated. Small packets raise packets per second, and a single CPU, interrupt queue, or softirq path may become the bottleneck first.
mpstat -P ALL 1 cat /proc/interrupts cat /proc/net/softnet_stat sar -n DEV 1
Look for NIC interrupts concentrated on a small number of CPUs and for unusually high softirq utilization on those CPUs. /proc/net/softnet_stat can reveal drops or processing-budget pressure in the kernel networking path. Its field layout may change between kernel versions, so use the documentation or source that matches the running kernel instead of relying on an old one-line parser.
On a multiqueue adapter, also review RSS, RPS, RFS, XPS, irqbalance, and explicit CPU affinity. Record per-queue and per-CPU baselines before changing them. A poorly chosen affinity mask can move all packet processing onto an even smaller set of cores.
5. Examine the RX Ring and Kernel Backlog
When packets arrive faster than the kernel can process them, an RX ring or the kernel backlog may overflow. First inspect the supported and active settings:
ethtool -g eth0 sysctl net.core.netdev_max_backlog sysctl net.core.netdev_budget sysctl net.core.netdev_budget_usecs
Do not immediately set every buffer to its maximum value. Larger queues may absorb short bursts, but they can also increase memory use and queueing latency while hiding a CPU, driver, or application bottleneck. Change one variable at a time during a controlled window, monitor packet loss and latency, and keep the original value for rollback.
6. Troubleshoot TX Queues and Traffic Control
When the increase is mainly on the TX side, inspect the qdisc, transmit queue, and any egress shaping policy:
tc -s qdisc show dev eth0 ip -s link show dev eth0 ip link show dev eth0
The dropped, overlimits, and backlog fields in tc -s qdisc help show whether packets are queueing or being discarded by traffic control. Rate limits, burst size, queue length, and congestion on the downstream link can all raise TX drop counters. Export the current qdisc configuration and prepare a rollback command before making production changes.
7. Verify MTU and Offload Settings
ip link show dev eth0 ethtool -k eth0 ping -M do -s 1472 target.example.com
An MTU mismatch between a host, virtual interface, tunnel, load balancer, and upstream network can cause fragmentation, silent drops, or the familiar symptom where small requests work but larger transfers fail. VXLAN, GRE, VPN, and other encapsulation add headers, so the underlying network must leave room for that overhead.
GRO, GSO, TSO, LRO, and checksum offload also change what packets look like in a local capture. A checksum that appears incorrect in tcpdump may be completed by hardware after the capture point. Do not disable every offload feature at once. Test one setting temporarily and measure CPU utilization, throughput, retransmissions, and latency.
8. Trace Every Layer in VMs, Containers, and Cloud Servers
In a virtualized environment, a packet may be dropped at the guest interface, veth pair, bridge, tap device, host adapter, virtual switch, or cloud network layer. For containers, inspect both the container namespace and the host:
ip -s link ip -d link tc -s qdisc show nsenter -t <PID> -n ip -s link
If counters inside the VM look normal but the application still sees retransmissions or timeouts, correlate the incident with host and cloud-platform metrics. A typical cloud VM does not expose every physical NIC statistic. Preserve timestamps, instance identifiers, availability zone, connection tuples, and packet captures when escalating the issue to the provider.
9. A Practical Troubleshooting Order
Define the incident window, affected interface, traffic direction, and application symptoms.
Sample
ip -s linkrepeatedly to identify which counters are actively increasing.Use
ethtool -Sto separate physical-link, driver, and queue clues.Check CPU utilization, softirq load, IRQ distribution, and packets per second.
Inspect the RX ring, softnet backlog, and TX qdisc independently.
Verify MTU, encapsulation overhead, and offload behavior.
In virtualized systems, trace the guest, veth, bridge, host, and cloud layers.
Apply one reversible change and repeat the same measurements under comparable load.
10. Validate the Fix
A cumulative drop counter does not return to zero after a fix. Compare the rate of change before and after the modification:
ip -s -s link show dev eth0 ethtool -S eth0 sar -n DEV 1 ss -s nstat -az
Also review request success rate, P95 and P99 latency, TCP retransmissions, CPU softirq usage, queue backlog, and throughput. If drops fall but latency rises sharply, the change may only have converted packet loss into longer queueing. That is not a complete resolution.
Frequently Asked Questions
Does a rising RX dropped counter mean the NIC is failing?
No. Drops can also occur in driver queues, the kernel backlog, a virtual interface, or protocol processing. Correlate the generic counter with ethtool -S, softnet data, CPU usage, and physical error statistics.
Why are packets dropped when bandwidth is not fully used?
The limiting factor may be packets per second rather than bits per second. A small-packet workload, a hot receive queue, one overloaded softirq CPU, or a short traffic burst can cause drops well below the nominal link rate.
Should I increase netdev_max_backlog immediately?
Only after evidence shows that the softnet backlog is overflowing. Treat it as a measured tuning option, not a default fix. Monitor memory, CPU, and latency after the change and retain the previous value for rollback.
Why does tcpdump show an incorrect checksum?
Transmit checksum offload may calculate the final checksum after the packet passes the local capture point. Confirm the issue with a capture on the receiving side, driver counters, and the active offload settings.
Conclusion
The most important step in Linux packet-drop troubleshooting is to locate the layer that is discarding packets before changing kernel parameters. Start with ip, narrow the scope with driver and per-queue statistics, then examine CPU/IRQ pressure, softnet_stat, qdisc behavior, MTU, and virtual networking. Small, reversible changes backed by before-and-after measurements are safer and more informative than applying a collection of generic tuning values.
References
Linux man-pages: ip-link(8), checked September 7, 2026
Linux man-pages: ethtool(8), checked September 7, 2026
Linux Kernel Documentation: Interface Statistics, checked September 7, 2026
Linux Kernel Documentation: Scaling in the Linux Networking Stack, checked September 7, 2026
