编程与软件工程thedailywtf.com编辑精选
Therac-25 事件
The Therac-25 Incident (2021)
2021 — 2025,它一次次回到首页
黑点是上首页(≥ 50 分)的那次投稿,空心点是没冲上去的投稿,点越大分数越高。
导读
这篇文章回顾了 1980 年代 Therac-25 放射治疗机的系列事故:由于软件缺陷(以及被移除的硬件联锁保护),多台设备向患者释放了致命的过量辐射。文章梳理了事故经过、调查结论以及从中得到的软件工程教训。
为什么经久不衰:它是软件工程史上最著名的安全关键系统事故之一,反复被提起,因为其中关于复用旧代码、过度信任软件、缺少硬件保护的教训至今仍未过时。
HN 评论精选
Therac-25 长期拖慢了医学信息化,且软件的离散型失效模式与医学监管假设的连续风险模型根本不同。
一位先读计算机学位、后读医学院的医生常把 Therac-25 当作“需要跨领域双料专家”的例子。他补充两点:其一,这起事故的影响远比人们记忆中的更久——他认为它让医学信息学倒退了 20 年,整个 80、90 年代医学界都弥漫着“计算机很危险”的感觉,直到 2000 年代中期美国联邦政府强制推行电子病历才真正改变。其二,医学界和监管机构习惯处理“连续型风险”——100mg 阿司匹林是 1 个单位风险,200mg 是 2,150mg 一定落在中间而不会是 100 万;而软件故障恰恰是这种不连续、可能突然放大百万倍的风险。
omginternets查看英文原文
The featured comment is great, for those who missed it: I am a physician who did a computer science degree before medical school. I frequently use the Therac-25 incident as an example of why we need dual experts who are trained in both fields. I must add two small points to this fantastic summary. 1. The shadow of the Therac-25 is much longer than those who remember it. In my opinion, this incident set medical informatics back 20 years. Throughout the 80s and 90s there was just a feeling in medicine that computers were dangerous, even if the individual physicians didn't know why. This is why, when I was a resident in 2002-2006 we still were writing all of our orders and notes on paper. It wasn't until the US federal government slammed down the hammer in the mid 2000's and said no payment unless you adopt electronic health records, that computers made real inroads into clinical medicine. 2. The medical profession, and the government agencies that regulate it, are accustomed to risk and have systems to manage it. The problem is that classical medicine is tuned to "continuous risks." If the Risk of 100 mg of aspirin is "1 risk unit" and the risk of 200 mg of aspirin is "2 risk units" then the risk of 150 mg of aspirin is strongly likely to be between 1 and 2, and it definitely won't be 1,000,000. The mechanisms we use to regulate medicine, with dosing trials, and pharmacokinetic studies, and so forth are based on this assumption that both benefit and harm are continuous functions of prescribed dose, and the physician's job is to find the sweet spot between them. When you let a computer handle a treatment you are exposed to a completely different kind of risk. Computers are inherently binary machines that we sometimes make simulate continuous functions. Because computers are binary, there is a potential for corner cases that expose erratic, and as this case shows, potentially fatal behavior. This is not new to computer science, but it is very foreign to medicine. Because of this, medicine has a built in blind spot in evaluating computer technology.
同类事故在本世纪仍不断发生,但现代案例很少被纳入程序员教育。
Therac-25 在 HN 上常被提及,但没人说这类问题其实从未真正消失——它只是最有名的一例。不必回到 1986 年,本世纪也有案例(如 2007 年佛罗里达的放疗过量事件、2001 年巴拿马放疗事故),根因有新有旧,但学校几乎不教这些更现代的案例,以至于不是每个程序员都听说过。
FiatLuxDave查看英文原文
I find it interesting how often the Therac 25 is mentioned on HN (thanks to Dan for the list), but nobody ever mentions that those kind of problems never entirely went away. Therac 25 is just the famous one. You don't have to go back to 1986, there are definitely examples from this century. The root causes are somewhat different, and somewhat the same. But no one seems to be teaching these more modern cases to aspiring programmers in school, at least not to the level where every programmer I know has heard of them. For example, the issue which caused this in 2007: https://www.heraldtribune.com/article/LK/20100124/News/605200351/SH Or the process issues which caused this in 2001: https://www.fda.gov/radiation-emitting-products/alerts-and-notices/fda-statement-radiation-overexposures-panama
面试安全关键医疗设备岗位时的真实心理挣扎:这类系统的道德重量不是人人都能承担。
多年前他面试过一套类似系统(比 Therac-25 新得多)。终面时一位高管问他,是否愿意参与开发可能给患者施加危险剂量辐射的设备;那一刻他脑海里闪过了这个故事。他自认是个谨慎的工程师,也相信现在有更多防护措施,但他意识到如果自己因此害了人,将无法原谅自己——于是他如实回答,对方道谢后双方就此结束。他强调这不是在评判做这类系统的人:社会显然需要优秀工程师去设计危险系统,只是他认清了自己没有那样的性格。
time0ut查看英文原文
Many years ago, I had an opportunity to work on a similar type of system (though more recent than this). In the final round of interviews, one of the executives asked if I would be comfortable working on a device that could deliver a potentially dangerous dose of radiation to a patient. In that moment, my mind flashed to this story. I try to be a careful engineer and I am sure there are many more safeguards in place now, but, in that moment, I realized I would not be able to live with myself if I harmed someone that way. I answered truthfully and he thanked me and we ended things there. I do not mean this as a judgement on those who do work on systems that can physically harm people. Obviously, we need good engineers to design potentially dangerous systems. It is just how I realized I really don't have the character to do it.
整理出该话题在 HN 上多年反复被讨论的历次链接。
(供好奇者)HN 上 Therac-25 的历次讨论串:2019、2018、2016、2015、2014、2010……还有别的吗?
dang查看英文原文
(For the curious) the Therac-25 stack on HN: 2019 https://news.ycombinator.com/item?id=21679287 2018 https://news.ycombinator.com/item?id=17740292 2016 https://news.ycombinator.com/item?id=12201147 2015 https://news.ycombinator.com/item?id=9643054 2014 https://news.ycombinator.com/item?id=7257005 2010 https://news.ycombinator.com/item?id=1143776 Others?
历次讨论
同类还值得读
开源应用架构
The Architecture of Open Source Applications
《开源软件架构》(AOSA)是一本在线免费书,邀请知名开源项目的作者亲自剖析其内部设计,涵盖 SQLAlchemy、LLVM、Audacity、Apache、Bash 等系统。
编程与软件工程上首页 9 次 · 2011–2024aosabook.org编辑精选编程语言简史:简短、不完整且多半错误
A Brief, Incomplete, and Mostly Wrong History of Programming Languages
一篇以戏谑、故意不准确的方式回顾编程语言史的博客文章,用夸张轶事和讽刺定义串起 Lisp、C、C++、Java、JavaScript、PHP 等语言。
编程与软件工程上首页 8 次 · 2009–2021james-iry.blogspot.comSQLite 是如何测试的
How SQLite is tested
SQLite 官方文档页,系统介绍 SQLite 的测试体系:多种测试方法(TCL 测试、模糊测试、回归测试、静态分析等)和海量测试用例,展示了这个小型开源项目如何靠极致测试获得…
编程与软件工程上首页 8 次 · 2009–2025sqlite.org编辑精选高级编程语言(2009)
Advanced programming languages (2009)
Matt Might 的经典文章,按用途推荐“高级编程语言”:如果追求就业就选主流语言,如果追求表达力、学术研究或创业,则推荐 Haskell、OCaml、Scala 等函数式语…
编程与软件工程上首页 8 次 · 2009–2022matt.might.net我的 Forth 与栈机器经历
Yosefk - My history with Forth & stack machines
作者以亲身经历回顾了使用 Forth 语言和栈式机器的历程:既赞叹其简洁优雅、代码量小、可自举的优点,也坦率指出栈操作在实际编程中的痛苦,以及 Forth 社区文化中的问题。
编程与软件工程上首页 8 次 · 2010–2024yosefk.com编辑精选