系统与基础设施blog.cloudflare.com编辑精选
如何每秒接收一百万个数据包
How to receive a million packets per second
2015 — 2020，它一次次回到首页
黑点是上首页（≥ 50 分）的那次投稿，空心点是没冲上去的投稿，点越大分数越高。
导读
Cloudflare 工程师撰写的经典技术文章，讲解如何在 Linux 上不借助内核旁路等特殊手段，仅通过调优内核网络栈（如多队列网卡、RSS、recvmmsg 批量收包等）让单机每秒接收一百万个 UDP 数据包。文章回应了“Linux 网络栈每核最多只能收 5 万包”的说法，证明通过合理配置完全可以突破这一瓶颈。
为什么经久不衰：它把 Linux 内核网络收包路径的关键调优技巧讲得清晰透彻，是理解高性能网络编程绕不开的入门经典，多年后仍是相关领域的必读参考。
HN 评论精选
recvmmsg 的超时实现有缺陷，且内核态 IO 占用大量 CPU，实际应用空间有限
不错，但 recvmmsg() 是有缺陷的。man 手册里写明：timeout 参数不能按预期工作——超时只在收到每个数据报后才检查，所以如果收到 vlen-1 个数据报后超时还没到、之后再也没有数据报到来，调用会永远阻塞。这对任何需要短时限内处理数据的应用来说都没法用，唯一的绕过办法是“自时钟”方式：想要至少每 10ms 收到包，就设 10ms 超时……然后确保自己每 10ms 给自己发一个包。我用 UDP 做过类似测试，一个不太复杂、不用太多花招的测试程序在多核系统上也能达到 50 万 pps。问题是系统 80%~90% 的时间都花在内核做 IO 上，留给应用的时间所剩无几。另一个选择是 pcap 和 PF_RING，参见 robdns 项目。
adekok查看英文原文
Nice, except recvmmsg() is broken. http://man7.org/linux/man-pages/man2/recvmmsg.2.html The timeout argument does not work as intended. The timeout is checked only after the receipt of each datagram, so that if up to vlen-1 datagrams are received before the timeout expires, but then no further datagrams are received, the call will block forever. Which makes it useless for any application that wants to service data in a short time frame. The only way around it is to use a "self clocking" method. If you want to receive packets at least every 10ms, set a 10ms timeout... and then be sure to send yourself a packet every 10ms. I've done similar tests with UDP applications. It's possible to get 500K pps on a multi-core system with a test application that isn't too complex, or uses too many tricks. The problem is that the system spends 80% to 90% of its time in the kernel doing IO. So you have no time left to run your application. Another alternative is pcap and PF_RING, as seen here: https://github.com/robertdavidgraham/robdns That might be useful. Previous discussion on robdns: https://news.ycombinator.com/item?id=8802425
质疑为何不直接采用业界常见的内核旁路方案
嗯，我可能漏了什么，但大多数高性能网络应用不正是为了这个原因才绕过内核的吗？（比如 Snabb Switch 那篇文章，绕过操作系统拿到每秒 4000 万请求。）这让我好奇内核旁路在生产环境的网络应用中到底有多常见。
edude03查看英文原文
Hmm, I might be missing something here, but don't most high performance network applications skip the kernel for this exact reason? (IE http://highscalability.com/blog/2014/2/13/snabb-switch-skip-the-os-and-get-40-million-requests-per-sec.html) Makes me wonder how often bypassing the kernel is used in production networked applications.
指出按每核 5 万包的原始说法，24 核机器本就该能到 120 万 pps，标题并不算突破
文章开头引用同事的话：“Linux 网络栈很慢！每核别指望超过每秒 5 万个包！”而测试机是两颗六核 2GHz Xeon，开超线程后每台 24 个处理器。24 × 50,000 = 1,200,000。所以“每秒 100 万包”这个原始命题本来就是对的。
danpalmer查看英文原文
> Last week during a casual conversation I overheard a colleague saying: "The Linux network stack is slow! You can't expect it to do more than 50 thousand packets per second per core!" > They both have two six core 2GHz Xeon processors. With hyperthreading (HT) enabled that counts to 24 processors on each box. 24 * 50,000 = 1,200,000 > we had shown that it is technically possible to receive 1Mpps on a Linux machine So the original proposition was correct.
5 年后硬件和内核进步，普通配置即可轻松超过文中性能
“别指望实际应用不费大力气就能有这种性能。”——5 年后的今天，我的服务器不用折腾 NUMA 或网卡设置，多进程就能跑到每秒 200 万包。
ncmncm查看英文原文
A thousand nanoseconds per packet? Luxury! At 10Gbps, and hundred-odd byte packets as are common on finance multicast feeds, I get 73ns per packet. The amount that can be done in that time is limited. Typically, you just dump them in a big-ass ring buffer and let processes on a bunch of other cores pick out interesting ones to spend more time on. The way to get the packets is usually with some proprietary kernel-bypass library (some have success with DPDK), and a very carefully isolated core that the kernel is kept from interrupting, and just spins moving packets. NIC hardware-level ring buffers are usually pretty small, so pauses make for drops. Many NICs can be persuaded to filter packets out to multiple ring buffers, and then you can get multiple cores picking them off; that is necessary when you get to 40 and 100 Gbps. Multiple rings even with one core multiplies headroom, making occasional housekeeping without drops possible.
质疑既然不走内核旁路，为何选用以旁路见长的高端网卡
楼主用的是 Solarflare 网卡，那硬件上显然能超过 100 万 pps？但既然标题是“Linux 内核栈每秒 100 万包”，我理解内核旁路之类本来就不在讨论范围内，目标始终是走内核。可如果是这样，为什么用 Solarflare？为什么不在一块普通但不错的网卡上验证能否超过 100 万 pps？
7532yahoogmail查看英文原文
OP has Solarflare NICs. So surely > 1MM pps is possible on that HW? But as you opened with 1 million per second on the Linux stack, I gather that kernel by-pass and such were by definition out of scope. The goal here was to always pass through the kernel. But if so, why use Solarflare? Why not see if one can do + 1MM pps on a nice, even beefy but not Solarflare/Mellonx NICs whose motivation includes kernel by-pass.
历次讨论
同类还值得读
TTY 揭秘（2008）
The TTY demystified (2008)
经典文章深入剖析了 Unix/Linux 中 TTY 子系统的工作原理，包括线路规程、终端驱动、信号与控制字符等机制。
系统与基础设施上首页 11 次 · 2009–2025linusakesson.net编辑精选为什么 GNU grep 这么快
Why GNU grep is Fast
FreeBSD 邮件列表上一篇广为流传的帖子，解释 GNU grep 为什么快：核心在于 Boyer-Moore 字符串搜索算法配合精心优化的内部循环和 I/O，让程序在搜索时“…
系统与基础设施上首页 9 次 · 2010–2022lists.freebsd.orgXplain：为普通人讲解 X11
Xplain – Explaining X11 for the rest of us
《Xplain》是作者（HN 用户 Jasper_）写的一系列 X11 长文，用他在浏览器中重建的 X11 绘图模型做交互演示，讲解 X11 的绘图模型与窗口系统设计。
系统与基础设施上首页 6 次 · 2013–2020magcius.github.io编辑精选旋风教程：为 Linux 创建极小 ELF 可执行文件（1999）
A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux (1999)
breadbox 写于 1999 年的经典教程，主题是如何为 Linux 手工构造并裁剪出体积极小的 ELF 可执行文件。
系统与基础设施上首页 6 次 · 2007–2025muppetlabs.comPostgreSQL 内部原理
The internals of PostgreSQL
《The Internals of PostgreSQL》是一本免费在线书籍，系统讲解 PostgreSQL 的内部实现，包括查询处理、并发控制、WAL、缓冲区管理等核心机制。
系统与基础设施上首页 5 次 · 2017–2025interdb.jp编辑精选