系统与基础设施danluu.com编辑精选
文件很难
Files Are Hard
2015 — 2019，它一次次回到首页
黑点是上首页（≥ 50 分）的那次投稿，空心点是没冲上去的投稿，点越大分数越高。
导读
Dan Luu 的经典文章，探讨为什么文件读写和持久化远比想象中困难：崩溃、断电等场景下，即使正确使用 fsync，本地文件系统仍可能返回损坏或丢失的数据。文章结合多篇学术研究说明文件系统一致性的种种坑，并解释为何桌面邮件客户端不如 Gmail 可靠。
为什么经久不衰：它把'写个文件而已'这一看似简单的操作背后的系统性风险讲透了，多年后仍是每个开发者处理数据持久化时的必读警示。
HN 评论精选
从业分布式文件系统开发者证实：本地文件系统历史包袱重，大家只能绕开它自己造轮子
文章基本说到点子上了。本地内核文件系统很难，部分源于其本质，部分源于历史——很多代码仍带着单核系统和早期 SATA 磁盘时代的烙印，开发测试方法也是同时代的。开发者总有时间优化古老微基准的数字，但新特性要么被推到 LVM 层（快照），要么长期搁置（联合挂载/overlay），要么干脆被无视（fsync 之外的任何替代方案）。我们做分布式文件系统的，只能把本地文件系统粗粒度地用于空间分配和缓存，其余尽量自己做。数据库（本地和分布式）处境也差不多。希望有一天这些分散的努力能合并成一个'把所有正确的事都做对'的公共库。
notacoward查看英文原文
Pretty much spot on. Local in-kernel file systems are hard, partly because of their essential nature and partly because of their history. A lot of the codebases involved still show their origins on single-core systems and pre-NCQ SATA disks, and the development/testing methods are from the same era. The developers always have time to improve the numbers on some ancient micro-benchmark, but new features often get pushed to the LVM layer (snapshots), languish for ages (unions/overlays), or are simply ignored (anything alternative to the fsync sledgehammer). The only way a distributed file system such as I work on can provide sane behavior and decent performance to our users is to use local file systems only for course-grain space allocation and caching. Sometimes those magic incantations from ten-year-old LKML posts don't really work, because they were never really tested for more than a couple of simple cases. Other times they have unexpected impacts on performance or space consumption. Usually it's easier and/or safer just to do as much as possible ourselves. Databases - both local and distributed - are in pretty much the same boat. Some day, I hope, all of this scattered and repeated effort will be combined into a common library that Does All The Right Things (which change over time) and adds features with a common API. It's not quite as good as if the stuff in the kernel had been done right, but I think it's the best we can hope for at this point.
ZFS 的设计目标正是保证读到的一定是完整写入的数据，但文章第一部分的问题它也解决不了
据我理解，ZFS 就是为应对这类数据损坏而设计的——一种设计描述是'read() 要么返回之前成功 write() 的内容，要么返回错误'。原则上这能防止文件出现'a boo'或'a far'这种内容。文章引用的作者之一写过一篇分析 ZFS 的论文（不过没测试崩溃时的行为）。这只涉及文章第二部分，第一部分的问题在 ZFS 上依然存在。
psi-squared查看英文原文
AIUI, ZFS was explicitly designed to deal with this sort of data corruption - one of the descriptions of the design I've heard is "read() will return either the contents of a previous successful write() or an error". That would (in principle) prevent the file containing "a boo" or "a far" at any point. It looks like one of the authors cited in this article has written a paper analysing ZFS - though they admittedly don't test its behaviour on crashes. Citation here, in PDF form: http://pages.cs.wisc.edu/~kadav/zfs/zfsrel.pdf (edited to add: This only deals with the second part of this article. The first part would still be important even on ZFS)
推荐'写临时文件 + 原子 rename'这一经典模式来避免文件损坏
有一个不损坏数据的简单写法：先把要改的文件复制成临时文件（或新建临时文件），在临时文件上修改，最后用 rename() 原子地替换旧文件。目录同理，但需要用链接实现真正的原子性。在特殊配置（如跨设备的 NFS）下可能不适用，但那种情况下可以通过捕获 rename() 的错误检测出来。不是银弹，但仍是好办法。
needs查看英文原文
There is an easy way to write data without corruption. First copy your file-to-be-changed as a temporary file or create a temporary file. Then modify the temporary file and write whatever you want in it. Finally, use rename() to atomically replace the old file by the temporary one. The same logic also apply to directories, although you will have to use links or symlinks to have something really atomic. It may not work on strangely configured systems, like if your files are spread over different devices over the network (or maybe with NFS). But in those cases you will be able to detect it if you catch errors of rename() and co (and you should catch them of course). So no silver bullet here, but still a good shot.
建议用久经考验的 SQLite 存文档，天然规避文件一致性问题
强烈推荐用 SQLite 作为自己的文档格式。SQLite 是地球上测试最充分的软件之一，支持 ACID。如果不需要极致性能，还可以配置得比默认更安全，尤其在开启完全同步、单线程使用时，非常抗崩溃和数据损坏。
JohnStrangeII查看英文原文
I strongly recommend using Sqlite for your own document format. Sqlite is one of the most well-tested pieces of software on earth and ACID-compliant. You can even make it safer than the default if you don't need maximum performance only need it for storing documents. It is very crash and corruption proof, especially with the full sync option and if you use it from one thread only.
历次讨论
同类还值得读
TTY 揭秘（2008）
The TTY demystified (2008)
经典文章深入剖析了 Unix/Linux 中 TTY 子系统的工作原理，包括线路规程、终端驱动、信号与控制字符等机制。
系统与基础设施上首页 11 次 · 2009–2025linusakesson.net编辑精选为什么 GNU grep 这么快
Why GNU grep is Fast
FreeBSD 邮件列表上一篇广为流传的帖子，解释 GNU grep 为什么快：核心在于 Boyer-Moore 字符串搜索算法配合精心优化的内部循环和 I/O，让程序在搜索时“…
系统与基础设施上首页 9 次 · 2010–2022lists.freebsd.orgXplain：为普通人讲解 X11
Xplain – Explaining X11 for the rest of us
《Xplain》是作者（HN 用户 Jasper_）写的一系列 X11 长文，用他在浏览器中重建的 X11 绘图模型做交互演示，讲解 X11 的绘图模型与窗口系统设计。
系统与基础设施上首页 6 次 · 2013–2020magcius.github.io编辑精选旋风教程：为 Linux 创建极小 ELF 可执行文件（1999）
A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux (1999)
breadbox 写于 1999 年的经典教程，主题是如何为 Linux 手工构造并裁剪出体积极小的 ELF 可执行文件。
系统与基础设施上首页 6 次 · 2007–2025muppetlabs.comPostgreSQL 内部原理
The internals of PostgreSQL
《The Internals of PostgreSQL》是一本免费在线书籍，系统讲解 PostgreSQL 的内部实现，包括查询处理、并发控制、WAL、缓冲区管理等核心机制。
系统与基础设施上首页 5 次 · 2017–2025interdb.jp编辑精选