思考与写作blog.samaltman.com
H5N1
H5N1
2013 — 2020，它一次次回到首页
黑点是上首页（≥ 50 分）的那次投稿，空心点是没冲上去的投稿，点越大分数越高。
导读
Sam Altman 的这篇短文以 H5N1 禽流感为引子，讨论生物技术可能带来的极端风险：工程化病原体、实验室制造病毒，以及能改变人类动机的“快乐药”等情景。文章把这些低概率高后果的风险放进文明脆弱性与尾部风险的框架中，追问我们是否以及该如何提前防范。
为什么经久不衰：它把生物安全、尾部风险和文明韧性放在一起讨论，在新冠疫情之后读来尤其有现实感，也仍能提醒人们：最危险的不是最戏剧化的病毒，而是刚好逃过警报的那种。
HN 评论精选
造出“完美末日病毒”远没有想象中容易，生物复杂性和国际协作都是限制。
我们现在能在实验室造病毒。若有人造出极易传播、死亡率超过 50%、潜伏数周的病毒并被恶意释放，世界来不及准备，几个月内可能死掉一半以上人口。但 Sam 严重低估了这种病毒的技术可行性：搞计算机的人常高估生物学家对生物系统的掌握。我们对大多数生物学几乎一无所知，复杂动态系统和反馈回路不断击碎理解。我不敢说不可能，但我们甚至不知道高致死、长潜伏、易传播这三者能否同时存在。评论还犯了把超级能力给坏人、却假设好人坐视不管的谬误；H5N1 其实是国际协作的较好例子。
JunkDNA查看英文原文
>We now have the tools to create viruses in labs. What happens when someone creates a virus that spreads extremely easily, has greater than 50% mortality, and has an incubation period of several weeks? Something like this, released by a bad guy and without the world having time to prepare, could wipe out more than half the population in a matter of months. Misguided biotech could effectively end the world as we know it Sam is a smart guy, so I really don't want to come off as sounding like a jerk here, but this grossly underestimates the technical feasibility of creating such a virus. Computer folks routinely overestimate how much biologists actually know about the systems we study. We know jack about how the vast majority of biology works. We have the most fleeting glimpses of understanding that are regularly crushed by the complexity of dynamic systems with nested feedback loops and multiple semi-overlapping redundancies. I won't say it's impossible, but we don't even know enough to know whether the three things: high mortality, long incubation, and ease of transmission are even possible. While we can imagine it, there might be biological and epidemiological factors that prevent such a thing from existing. This also commits the logical fallacy of ascribing superpowers to the bad guys cooking up viruses while assuming the good guys are sitting on their duffs letting bad things happen. H5N1 was a pretty good example of international collaboration. There were academic competitors and industrial labs working around the clock collaboratively on it in the early days before much was known. Whole vaccine divisions at pharmas were all over it. If we're instead talking about a mythical time in the future when we do understand enough biology to engineer something like this, one would have to assume the good guys possess the knowledge to develop countermeasures. I'm not arguing that pandemics aren't something we should worry about. Europeans were almost wiped out by the plague and in modern times Africa has been decimated by HIV. These are real problems that the human race has faced and will likely face again, irrespective of lab-created stuff. Biotechnology is the primary mechanism by which we're going to be able to survive when the next one comes, wherever it comes from. EDIT: Fixed wrong word usage in 2nd sentence.
关键问题是文明本质上是脆弱还是稳健，这决定我们该为低概率高后果风险投入多少偏执。
尾部风险决策从来不容易，因为按定义我们缺乏足够数据。要不要重点防恐怖主义？如果 9/11 是最坏情况，那可能不值得；但如果一次恐袭能击垮整个国家，就值得偏执。假设恐怖分子污染食物和水导致全国性抢粮骚乱——文明离无政府状态只有九顿饭。核心问题是：我们的文明本质上是脆弱的还是稳健的？如果稳健，就专注增长，遇到挫折再处理，长期繁荣会上升。相反观点是文明脆弱：王国兴衰，愚蠢决策曾导致数百年倒退；近几百年的上升趋势不代表人类成熟。我们每发现一种新末日武器就拿来玩，和嗜血祖先没两样。
gizmo查看英文原文
Tail risk decisions are never easy. Because we lack sufficient data by definition. Should we focus on preventing terrorism? Well, if 9/11 was the worst case scenario then no. If on the other hand a terror attack could bring down the entire country it's certainly worth being paranoid about. Suppose terrorists poison our food and water supplies to the extent that we get country-wide food riots. A civilization is only 9 meals away from anarchy after all. So the essential question is this: - Is our civilization essentially fragile or fundamentally robust? If our civilization is fundamentally robust we can simply focus on growth and deal with setbacks (global warming, terrorism, imperialism, wars) as they come. In the long term prosperity will go up and up. Not always as fast as we'd like and not always in ways we deem fair but if we keep making progress we'll get there eventually. This is the whiggish view. The opposite view is that civilization is fragile. Kingdoms come and go and foolish decisions can and have lead to centuries of regression. The upward trend we've seen in the past couple of centuries does not mean our species has grown up in the slightest. Every new weapon of doom we discover we play with and we're no better than our imperialist and bloodthirsty forefathers. Our civilization is determined to self-destruct by either nuclear war, environmental disaster, political insanity or runaway capitalism. A civilization that is not capable of planning ahead will eventually walk like a lemming of a cliff. The best thing we can do is put tons of safeguards and regulations in place to improve our odds of surviving at all. Those are the two main views. And the kicker is we don't have enough data to know for certain which view is correct.
最危险的病毒不是最致命的，而是致死性与传播力平衡、能拖延警报的病毒；SARS-CoV-2 可能接近这种“甜点”。
病毒越致命，越会早早触发警报和恐慌；而过去几周我们都学到，疫情里天数很重要。完美的病毒是平衡的：致命得刚刚好。SARS-CoV-2 可能正好命中这个甜点。
_acco查看英文原文
The deadlier a virus is, the earlier it will trip alarm systems/panic. And as we've all learned in the past few weeks, in outbreaks days matter. The perfect virus is balanced: just deadly enough. It's possible SARS-CoV-2 hits the sweet spot.
高致死、高传染、长潜伏很难兼得，是“三选二”的取舍。
极致命、极传染、极长潜伏期的病毒相当不可能，甚至不可能。据我所知这是典型的“三选二”三角。
choeger查看英文原文
A virus that is extremely deadly, extremely contagious, and has an extremely long incubation period is quite unlikely, if not impossible. As far as I know this is a typical "pick two" triangle.
历次讨论
同类还值得读
保持你的身份认同足够小
Keep Your Identity Small (2009)
Paul Graham 2009 年的一篇短文：人一旦把某个话题当成自己身份的一部分（宗教、政治、编程语言皆然），讨论就会退化成站队和争吵，因为此时争论的不再是事实，而是"我是谁…
思考与写作上首页 5 次 · 2009–2021paulgraham.com编辑精选苏格拉底式教学法：通过提问而非讲授来教学
The Socratic Method: Teaching by Asking Instead of by Telling
记录了作者用纯提问的方式，引导一群小学生从零推导出二进制计数法的课堂实验，展示了苏格拉底式教学法的惊人效果。
思考与写作上首页 5 次 · 2011–2022garlikov.com你不能说的话
What You Can't Say (2004)
Paul Graham 2004 年的经典文章，探讨那些因道德时尚而不能公开说出的观点。
思考与写作上首页 5 次 · 2014–2022paulgraham.com编辑精选龙暴君的寓言（2005）
The Fable of the Dragon-Tyrant (2005)
哲学家 Nick Bostrom 于 2005 年写的一则寓言：一条每天吞噬数千人的巨龙（隐喻死亡与衰老）长期被人类当作不可抗拒的自然规律接受，直到有人提出并最终用导弹将其杀死，…
思考与写作上首页 5 次 · 2009–2021nickbostrom.com编辑精选现实有着惊人的细节量
Reality has a surprising amount of detail (2017)
2017 年的博文指出，现实远比我们以为的充满细节：我们习惯用简化的心智模型看世界，直到真正动手做事（比如作者举的造楼梯例子）才发现处处是未曾预料的细节。
思考与写作上首页 4 次 · 2018–2025johnsalvatier.org编辑精选