系统与基础设施joyent.com编辑精选
TCP 疑难杂症
TCP Puzzlers
2016 — 2019,它一次次回到首页
黑点是上首页(≥ 50 分)的那次投稿,空心点是没冲上去的投稿,点越大分数越高。
导读
Joyent 的这篇博文通过几个 TCP 疑难问题(puzzlers)讲解 TCP 连接在半关闭、空闲超时、对端消失等场景下的反直觉行为。作者用实际案例说明这些边界情况在编写分布式系统代码时必须理解。
为什么经久不衰:TCP 的这些边界行为多年不变,而虚拟化和大规模集群让'罕见情况'频繁发生,任何写网络或分布式系统代码的人都绕不开这些坑。
HN 评论精选
分布式系统规模变大后,TCP 的罕见边界行为变得常见,必须完全理解
写分布式系统代码的人必须完全理解 TCP/IP 的这些行为。以前 100 个节点就算'大'网络时这类问题很罕见,如今虚拟化可以让 500 个节点跑在同一台主机上,大数定律说'东西多了,罕见的事就常发生'。趣闻:在 Blekko 时,爬虫会连续抓取 300 页搜索结果,我们用一台只完成大部分 TCP 握手就消失的机器对付它们,结果爬虫脚本永远挂起,有的甚至挂了几个月。
ChuckMcM查看英文原文
That is an awesome post. If you are thinking about writing distributed systems code you need to understand this behavior of TCP/IP completely. Back when a 100 nodes was a "big" network these sorts of problems were so rare that you could be forgiven for not taking them into account, but these days with virtualization you can have a 500 node network all on the same freakin' host! And the law of large numbers says "rare things happen often, when you have a lot of things." Fun anecdote, at Blekko we had people who tried to scrape the search engine by fetching all 300 pages of results. They would do that with some script or code and it would be clear they weren't human because they would ask for each page right after the other. We sent them to a process that Greg wrote on a machine that did most of the TCP handshake and then went away. As a result the scrapers script would hang forever. We saw output that suggested some of these things sat their for months waiting for results that would never come.
TCP 超时设计的历史演变:从'不该管'到如今为防 DoS 而缩短
空闲但半死的 TCP 连接最终会超时,但在某些系统上可能要几个小时。早期观点是 TCP 本身不该超时——Telnet 服务器可以有超时,但决定你午餐时间多长不是 TCP 的职责。如今大多数服务器超时更短,主要是防 DoS 攻击,而且往往是 HTTP 层而非 TCP 层先超时。
Animats查看英文原文
Idle but half-dead TCP connections will eventually time out. It can take hours, though, on some systems.[1] Early thinking was that if you have a Telnet session open, TCP itself should not time out. The Telnet server might have a timeout, but it wasn't TCP's job to decide how long you get for lunch. Today most servers have shorter timeouts, mostly as a defense against denial of service attacks. But it's often the HTTP server, not the TCP level, that times out first. [1] https://blogs.technet.microsoft.com/nettracer/2010/06/03/things-that-you-may-want-to-know-about-tcp-keepalives/
TCP 之下的网络层(ICMP)也会在握手和连接过程中报错,容易被忽视
还有一些有趣的边界情况:如果端口转发指向一台已关机的机器,握手 SYN 会收到 ICMP 网络不可达或主机不可达。这类 ICMP 消息(如管理禁止)也可能在连接的任何阶段出现。要记住 TCP 连接建立在底层网络栈之上,后者可以在 TCP 协议之外发出错误信号。
richm44查看英文原文
There are some other fun edge cases - for example: - If you have a port-forward to a machine that is switched off then you can get ICMP network unreachable or ICMP host unreachable as the response to the a SYN in the initial handshake. This can also happen at any point in the connection. Other ICMP messages can also occur like this (eg. admin prohibited). It's always worth remembering that the TCP connection is sitting on an underlying network stack that can also signal errors outside of the TCP protocol itself.
HTTP/2 的 PING 帧提供了应用层保活方案,可替代难以调优的 TCP keep-alive
HTTP/2 提供了一个很酷的东西:PING 帧,让客户端和服务器检查连接是否还活着。如果应用无法开启 TCP keep-alive,可以在应用层实现。这让双方都知道连接是否仍然存在,也避免 NAT 设备移除长连接。不用干等 5 分钟或调系统参数,你可以自己控制发现死连接的速度。
denormalfloat查看英文原文
A cool thing HTTP/2 provides is PING frames, which allow clients and servers to check if connections are still alive. In case your application can't turn on TCP keep-alives, you can enable it at the application layer. This allows both sides to know if the connection is still around and active, and avoids NAT boxes from removing long lived connections. Rather than wait around 5 minutes, or tune your system, you can control how quickly dead connections are discovered.
历次讨论
同类还值得读
TTY 揭秘(2008)
The TTY demystified (2008)
经典文章深入剖析了 Unix/Linux 中 TTY 子系统的工作原理,包括线路规程、终端驱动、信号与控制字符等机制。
系统与基础设施上首页 11 次 · 2009–2025linusakesson.net编辑精选为什么 GNU grep 这么快
Why GNU grep is Fast
FreeBSD 邮件列表上一篇广为流传的帖子,解释 GNU grep 为什么快:核心在于 Boyer-Moore 字符串搜索算法配合精心优化的内部循环和 I/O,让程序在搜索时“…
系统与基础设施上首页 9 次 · 2010–2022lists.freebsd.orgXplain:为普通人讲解 X11
Xplain – Explaining X11 for the rest of us
《Xplain》是作者(HN 用户 Jasper_)写的一系列 X11 长文,用他在浏览器中重建的 X11 绘图模型做交互演示,讲解 X11 的绘图模型与窗口系统设计。
系统与基础设施上首页 6 次 · 2013–2020magcius.github.io编辑精选旋风教程:为 Linux 创建极小 ELF 可执行文件(1999)
A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux (1999)
breadbox 写于 1999 年的经典教程,主题是如何为 Linux 手工构造并裁剪出体积极小的 ELF 可执行文件。
系统与基础设施上首页 6 次 · 2007–2025muppetlabs.comPostgreSQL 内部原理
The internals of PostgreSQL
《The Internals of PostgreSQL》是一本免费在线书籍,系统讲解 PostgreSQL 的内部实现,包括查询处理、并发控制、WAL、缓冲区管理等核心机制。
系统与基础设施上首页 5 次 · 2017–2025interdb.jp编辑精选