系统与基础设施engineering.skroutz.gr
揭开 Linux 内核中一个24年前的 bug
Uncovering a 24-year-old bug in the Linux Kernel
2021 — 2022，它一次次回到首页
黑点是上首页（≥ 50 分）的那次投稿，空心点是没冲上去的投稿，点越大分数越高。
导读
Skroutz 工程团队排查 rsync 在内网传输异常缓慢的问题，最终在 Linux 内核 TCP 拥塞控制代码中发现一个潜伏 24 年的 bug（涉及序列号回绕比较的边界情况）。文章完整记录了从现象观察、抓包分析到定位内核源码的排查过程。
为什么经久不衰：它是一篇教科书级的深度技术排查记录，展示了如何用开源工具逐层下钻定位内核级 bug，多年后仍是学习网络调试和系统排障的经典范例。
HN 评论精选
这篇文章同时也是一次高明的低调招聘
除了是篇出色的技术文章，这还是一次非常低调而有效的公司招聘：展示了公司日常使用的工具和技术、解决问题的思路，证明团队里有优秀的工程师，并体现了乐于参与开源社区的文化。招聘信息并不咄咄逼人，只是页面上有个不显眼的'We're hiring'按钮，结尾邀请感兴趣的人联系，整体感觉做得非常好。
qchris查看英文原文
Besides being a great technical write-up, this does an absolutely fantastic job of doing low-key recruitment for Skroutz. It shows some of the main tools and technologies that the company uses on a day-to-day basis, provides a window into the way that they approach problems, makes a compelling case that you'd be working with some talented engineers, and showcases a culture willing to engage with the open source community. The hiring pitch isn't in your face, but there's a "We're hiring!" button in the banner, which fairly unobtrusively follows you down the page, and then ends with a "hey, if you're interested in working with us, reach out." Overall, it just feels really well done.
以土木工程视角类比：24 年的 bug 并不算老
写得很棒，我会让手下的技术员都读一遍。网络运维很难，网络编程更是真正的硬功夫。看到有人觉得 20 年的 bug 算老，我觉得挺有趣——土木工程比 IT 古老得多，我们仍在学习，24 年的 bug 不算什么。伦敦千禧桥就是忘了共振这类基础知识的经典例子。
gerdesj查看英文原文
Great write up. Think I'll get the kids, sorry technicians to walk through this. Actually, I think I'll learn just as much but I have to keep a little bit aloof as MD! Networks are tricky to run and networking is proper hard to do. TCP/UDP int al are pretty bloody good at shuffling data from A to B. I find it quite amusing when 20 years is considered old for a bug. The Millenium bridge in London is a classic example of forgetting the basics - in this case resonance and being too clever for your own good. It's a rather cool design for a bridge - a sort of a suspension bridge but flatter and some funky longitudinal stuff. I'm a Civ Eng grad. It looked too flat to me from day one. When people walk across a bridge and it starts to sway, they start to lock step and then resonance, where each step reinforces the last kicks in and more and more energy causes sway, shear and what have you forces. It gets worse and worse and then failure. Tacoma Narrows is another classic example of resonance but due to wind - that informed designs that don't fly! Civ Eng is way, way older than IT and we are still learning. 24 years is nothing for a bug. However, IT is capable of looking inward and monitoring itself (unit tests, ping etc) in a way that Civ Eng can't (OK we have strain gauges and a few other tools). The real difference between physical stuff and IT is that the Milli bridge rather obviously came close to failure visually and in a way that our other senses can perceive - it shook. The fix was to put hydraulic dampers along its length. In IT, we often try to fix things by using magic or papering over flaws with "just so" stories. Sometimes we get the tools out and do the job properly and these boys and girls did just that: the job properly.
这个 bug 可能正是 rsync '对网络不友好'传言的来源
我记得很久以前人们常说 rsync'对网络不友好'或'处理拥塞能力差'。我很有把握地猜，这些说法的根源就是这个 bug。
rancor查看英文原文
I have, admittedly old and very vague, memories of people talking about rsync being "hard on networks" or "dealing poorly with congestion." I'd put good odds that this bug is why those statements existed.
推测：正是网络性能太好、无丢包才让这个 bug 得以暴露
文章结尾问'为什么之前没人发现'，作者铺了逻辑但没推完，我来补上：他们最近升级了网络硬件后问题更频繁；丢包等网络问题会迫使协议栈走出快速路径、更新计数器从而避开 bug；走 ssh 也会避开 bug，只有内网才会不经 ssh 跑 rsync。所以这个 bug 只会在无丢包的高性能内网上跑 rsync 的人面前暴露——升级交换机可能恰好让网络好到足以触发它。
myself248查看英文原文
Okay so I got to the wrap-up at the end, about "why did nobody else find this", the author sets up some logical dominoes but doesn't knock them down. Allow me to try: Earlier in the article, the author mentions that they recently upgraded some network hardware, and the problem seemed to become more frequent after that. Packet loss or other network issues would force the stack to fall out of fast-path and update the counter, avoiding the bug. Running over ssh would avoid the bug. The only time you'd run rsync not over ssh would be within your own network. So it sounds like (this is my conjecture here) this would only appear to someone running rsync internally, over a high-performance network with no packet loss, and upgrading the switches might've finally gotten the network good enough to expose the bug?
历次讨论
同类还值得读
TTY 揭秘（2008）
The TTY demystified (2008)
经典文章深入剖析了 Unix/Linux 中 TTY 子系统的工作原理，包括线路规程、终端驱动、信号与控制字符等机制。
系统与基础设施上首页 11 次 · 2009–2025linusakesson.net编辑精选为什么 GNU grep 这么快
Why GNU grep is Fast
FreeBSD 邮件列表上一篇广为流传的帖子，解释 GNU grep 为什么快：核心在于 Boyer-Moore 字符串搜索算法配合精心优化的内部循环和 I/O，让程序在搜索时“…
系统与基础设施上首页 9 次 · 2010–2022lists.freebsd.orgXplain：为普通人讲解 X11
Xplain – Explaining X11 for the rest of us
《Xplain》是作者（HN 用户 Jasper_）写的一系列 X11 长文，用他在浏览器中重建的 X11 绘图模型做交互演示，讲解 X11 的绘图模型与窗口系统设计。
系统与基础设施上首页 6 次 · 2013–2020magcius.github.io编辑精选旋风教程：为 Linux 创建极小 ELF 可执行文件（1999）
A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux (1999)
breadbox 写于 1999 年的经典教程，主题是如何为 Linux 手工构造并裁剪出体积极小的 ELF 可执行文件。
系统与基础设施上首页 6 次 · 2007–2025muppetlabs.comPostgreSQL 内部原理
The internals of PostgreSQL
《The Internals of PostgreSQL》是一本免费在线书籍，系统讲解 PostgreSQL 的内部实现，包括查询处理、并发控制、WAL、缓冲区管理等核心机制。
系统与基础设施上首页 5 次 · 2017–2025interdb.jp编辑精选