系统与基础设施thume.ca
Telefork:把进程“传送”到另一台电脑
Teleforking a Process onto a Different Computer
2020 — 2022,它一次次回到首页
黑点是上首页(≥ 50 分)的那次投稿,空心点是没冲上去的投稿,点越大分数越高。
导读
作者实现了一个“telefork”工具:像 fork() 一样把进程复制到另一台计算机上继续运行,文章介绍了其实现原理和遇到的挑战。这一想法在分布式计算历史上被反复探索过,文章引发了大量相关经验的讨论。
为什么经久不衰:它把一个古老而迷人的分布式计算想法用现代工具重新实现,兼具趣味性和对进程迁移难点的深入剖析。
HN 评论精选
指出这是经典的应用检查点/进程迁移技术,并提到 CRIU 已内置于现代内核
很高兴看到人们重新发现老技术。这在集群计算中通常叫“应用检查点”,至今仍在许多系统中使用。要在应用中实现并行计算一般会用 PVM/MPI。SSI 集群曾试图让任意进程“telefork”到任意节点运行,但最持久、最困难的挑战是让共享内存和线程可靠工作。CRIU 支持从内核 3.11 起就已内置,在 Ubuntu 18.04 上可用,所以现在无需定制应用基本就能做到。
0xbadcafebee查看英文原文
It's nice to see people re-discover old school tech. In cluster computing this was generally called "application checkpointing"[1] and it's still in use in many different systems today. If you want to build this into your app for parallel computing you'd typically use PVM[2]/MPI[3]. SSI[4] clusters tried to simplify all this by making any process "telefork" and run on any node (based on a load balancing algorithm), but the most persistent and difficult challenge was getting shared memory and threading to work reliably. It looks like CRIU support is bundled in kernels since 3.11[5], and works for me in Ubuntu 18.04, so you can basically do this now without custom apps. [1] https://en.wikipedia.org/wiki/Application_checkpointing [2] https://en.wikipedia.org/wiki/Parallel_Virtual_Machine [3] https://en.wikipedia.org/wiki/Message_Passing_Interface [4] https://en.wikipedia.org/wiki/Single_system_image [5] https://en.wikipedia.org/wiki/CRIU#Use
指出这重新发明了 1979 年的 LOCUS 项目,并说明进程迁移难以普及的原因
恭喜,你刚刚重新发明了 UCLA 1979 年 LOCUS 分布式计算项目的核心思想。重新发明 LOCUS 也有很强的传承,比如贝尔实验室的 Plan 9 在 80 年代末部分做到了。虽然从未成为商业爆款,但 tele-fork 及其更高级的变体(机器间进程迁移、集群级进程池)吸引了分布式计算领域最聪明的人二十多年。可惜要实现好“很复杂”,尤其是当你要管理计算之外的资源(网络连接、文件、文件句柄等)时。
jonathaneunice查看英文原文
Congratulations! You have just reinvented the core idea of UCLA's LOCUS distributed computing project from 1979. https://en.wikipedia.org/wiki/LOCUS Reinventing LOCUS also has a strong heritage. Bell Lab's Plan 9, for example, did so in part in the late 1980s. While never a breakout commercial success, tele-forking and its slightly more advanced cousins machine-to-machine process migration and cluster-wide process pools intrigued some of the best minds in distributed computing for 20+ years. Unfortunately "it's complicated" to implement well, especially when you try to tele-spawn and manage resources beyond compute cycles (network connections, files, file handles, ...) that are important to scale up the idea.
列举 OpenMOSIX、MPI 和 Plan 9 等先例
这让我想起 OpenMOSIX,它以分布式方式实现了大部分 POSIX。MPI 也类似,但更侧重 IPC 机制。我一直很喜欢 Plan 9 的做法:每个 CPU 都是一个文件,通过向该文件写入来执行代码,即使它在远程文件系统上。
synack查看英文原文
This reminds me of OpenMOSIX, which implemented a good chunk of POSIX in a distributed fashion. MPI also comes to mind, but it's more focused on the IPC mechanisms. I always liked Plan 9's approach, where every CPU is just a file and you execute code by writing to that file, even if it's on a remote filesystem.
回忆 QNX 在 90 年代就有跨节点 spawn 进程的能力
旧瓶装新酒——我很确定 QNX 在 90 年代就能做到。QNX 有一套很酷的局域网进程间通信方式,用起来就像本地一样。不过后来发现 QNX4 不能跨局域网 fork,它有一个单独的 spawn() 调用可以跨节点操作。
ISL查看英文原文
What's old is new again -- I'm pretty sure QNX could do this in the 1990s. QNX had a really cool way of doing inter-process communication over the LAN that worked as if it were local. Used it in my first lab job in 2001. You might not find it on the web, though. The API references were all (thick!) dead trees. Edit: Looks like QNX4 couldn't fork over the LAN. It had a separate "spawn()" call that could operate across nodes. https://www.qnx.com/developers/docs/qnx_4.25_docs/qnx4/sysarch/proc.html
历次讨论
同类还值得读
TTY 揭秘(2008)
The TTY demystified (2008)
经典文章深入剖析了 Unix/Linux 中 TTY 子系统的工作原理,包括线路规程、终端驱动、信号与控制字符等机制。
系统与基础设施上首页 11 次 · 2009–2025linusakesson.net编辑精选为什么 GNU grep 这么快
Why GNU grep is Fast
FreeBSD 邮件列表上一篇广为流传的帖子,解释 GNU grep 为什么快:核心在于 Boyer-Moore 字符串搜索算法配合精心优化的内部循环和 I/O,让程序在搜索时“…
系统与基础设施上首页 9 次 · 2010–2022lists.freebsd.orgXplain:为普通人讲解 X11
Xplain – Explaining X11 for the rest of us
《Xplain》是作者(HN 用户 Jasper_)写的一系列 X11 长文,用他在浏览器中重建的 X11 绘图模型做交互演示,讲解 X11 的绘图模型与窗口系统设计。
系统与基础设施上首页 6 次 · 2013–2020magcius.github.io编辑精选旋风教程:为 Linux 创建极小 ELF 可执行文件(1999)
A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux (1999)
breadbox 写于 1999 年的经典教程,主题是如何为 Linux 手工构造并裁剪出体积极小的 ELF 可执行文件。
系统与基础设施上首页 6 次 · 2007–2025muppetlabs.comPostgreSQL 内部原理
The internals of PostgreSQL
《The Internals of PostgreSQL》是一本免费在线书籍,系统讲解 PostgreSQL 的内部实现,包括查询处理、并发控制、WAL、缓冲区管理等核心机制。
系统与基础设施上首页 5 次 · 2017–2025interdb.jp编辑精选