编程与软件工程chriskiehl.com
事件溯源很难
Event Sourcing is Hard
2019 — 2021,它一次次回到首页
黑点是上首页(≥ 50 分)的那次投稿,空心点是没冲上去的投稿,点越大分数越高。
导读
Chris Kiehl 结合自己的实践经验,总结出事件溯源(Event Sourcing)在实际落地时遇到的问题:事件日志膨胀、重放事件极慢、投影(projection)需要同步修改的节点分散、缺少成熟工具链等。结论是:事件溯源理论上很优雅,但对大多数业务系统而言,成本远大于收益。
为什么经久不衰:它是一篇来自实践经验的警示文章,并且常引发关于“事件溯源到底是什么、何时该用”的辩论,多年后仍被反复提起。
HN 评论精选
来自真实生产项目的惨痛经验:重放慢、存储浪费、数据分散,最后项目失败。
我完全同意作者。我在一个相当大的事件溯源系统上工作过,那是场没完没了的噩梦。事件基本就是数据库的提交日志,存着极其浪费空间,也没有想象中那么有用。每次改动事件处理逻辑都要重放,光在本地 CI 环境重放就要花好几天。去中心化数据存储是个坏主意:不仅有巨大的事件日志,每个服务还各有一份数据副本,改一个 User 这类公共对象要改 5-10 个项目的投影。实际上就是在“提交日志兼消息队列”上自己造一个糟糕的数据库,最后整个项目被自身的复杂度压垮了。
spricket查看英文原文
I agree with the author completely. I worked on a fairly large system using event sourcing, it was a never-ending nightmare. Maybe with better tooling someday it will be usable, but not now. Events are pretty much a database commit log. This is extremely space innefficient to keep around. And not nearly use useful as you might think. Re-runs need to happen pretty often as you change how events are handled. Even in our local CI environment, it eventually took DAYS to re-run the events. It was clear that the system would never survive production use for this reason alone. De-centralizing your data storage is a bad idea. We ended up not only with a stupidly huge event log, but multiple copies of data floating around at each service. Not fun to deal with changes to common objects like User. Sometimes you would have to update the "projection" in 5-10 different projects. In practice ES amounted to building our own (terrible) database on top of a commit log that's actually a message queue. Worst technology I've worked with in years, eventually the whole project collapsed under it's own weight. Some of these problems are fixable in theory. Perhaps a framework to manage projection updates, something to prune old events by taking snapshots, a DB migration style tool to "fixup" mistakes that are otherwise immortalized in the event stream. But right now, seriously stay away :)
先澄清事件溯源的定义和常见应用,同时认同它对多数场景并不合适。
事件溯源就是“发生的一切都是事件”,全部存下来,就能还原最新状态,也能看到任意历史时刻的样子。它在复杂工作流的企业应用(如支付、制造)中很常见,适合函数式风格,业务逻辑就是“对事件作出反应并产生新事件”。Redux 就是前端的事件溯源,数据库复制用 WAL 也是。但我也完全同意作者:它对大多数场景是错误方案,带来的问题多于解决。
manigandham查看英文原文
Event Sourcing = everything that happens is an event. Save all the events and you can always get to the latest state, as well as what things look like at any time in the past. What an "event" is depends on your business domain and the granularity of processing. It's very common in enterprise apps with complex workflows (like payment processing or manufacturing). Good fit for functional programming techniques and makes business logic easy to reason about since everything is in reaction to some event, and usually emits another event. With global ordering, you can have a single transaction ID that points to a snapshot of the entire system. The actual communication is usually handled by "service bus" style messaging like RabbitMQ that can support ordering, routing, acknowledgements, and retries. Kafka or a RDBMS can also be used but requires frameworks on top. This concept is used pretty much everywhere. Redux is event sourcing front-end state for React. Most database replication is event sourcing by using a write-ahead log as the stream of mutations to apply. All that being said, I completely with this article that it's the wrong solution for most cases and creates far more problems and limitations than it solves.
结论是“用 Postgres 的 WAL 就够了”,ES 只适合资源雄厚的基础设施级项目。
我做过几个事件溯源系统,还曾出于自负亲手实现过一个。每个 ES 项目都有文章说的那些问题。文章还没提到最糟的部分:吃了这么多苦之后,你发现整个系统只有公司里一个人用来生成一份无关紧要的报表,而这报表让实习生每季度花四小时手工做也能完成。99% 的情况下,当 ES 听起来是个好主意时,正确做法就是用 Postgres——用 wal2json 订阅 WAL 流,时间旅行和审计日志都便宜得多;要企业级就用 Debezium。ES 在基础设施类软件里效果很好(Postgres 本身就是),但没有几十上百人年资源的普通软件承受不了它的复杂度。
akiselev查看英文原文
I've worked with several event sourcing systems and was even seduced into implementing one out of sheer hubris once. These problems are ever present in every ES project I've had the misfortune of coming into contact with. It doesn't even mention the worst part that comes afterwards, when you realize after all of that pain that it is only used by a single person in the company to generate a noncritical report comparing a meaningless KPI that could have been manually done in four hours by a different intern every quarter. By the time the tooling is up and running enough to make a stable system, no one will trust it enough to use ES's landmark features except for a few developers still coming off the koolaid. 99% of the time when ES sounds like a good idea, the answer is to just use Postgres. Use wal2json and subscribe to the WAL stream - if you really need time travel or audit logs or whatever, they'll be much cheaper to implement using WAL. If you need something more enterprisey to sell to the VP-suite, use Debezium. Event sourcing sounds so awesome in theory and it is used to great effect in many demanding applications (like Postgres! WAL = event sourcing with fewer steps) but it's just too complex for non-infrastructure software without resources measured in the 10s or 100s of man years.
指出文章批评的其实是事件流(Kafka 式)而非真正的事件溯源,两者被混为一谈。
文章再次上榜,我想指出:它有些观点也许成立,但讲的不是事件溯源,而是事件流/事件驱动。在事件溯源里,事件就是状态:从流中取出某个实体的全部事件,按顺序逐个应用得到当前状态,运行完业务逻辑后追加新事件,不需要在别处缓存写模型状态。事件溯源有它自己的问题,但文中描述的多数是 Kafka 这类事件流工具的问题——那些工具负责搬运数据,不是持久化数据库。
oskar_dudycz查看英文原文
As this article pops out again, I'd like to the point that although it may have some valid points, those points are not about Event Sourcing. What's expressed in the article is the Event Streaming or Event-Driven approach. So when events are not the source of truth etc. All of the event stores that I know supports strong consistency on appends, optimistic concurrency. Many guarantee global ordering. Some help in idempotency. All of that helps to reduce those issues. In Event Sourcing, events are the state. So the flow looks like that: 1. You get the events from the stream (that represents all the facts that happened for the entity), 2. You apply it one by one in the order of appearance to build the current state, 3. You run the business logic and, as a result, create the event, 4. You append a new event. You don't need to cache the write model state anywhere, as the state is in events. Event Sourcing has its issues and troubles, but the stuff described in the article apply to tools like Kafka, etc. They're not tools for Event Sourcing but Event Streaming. They're designed to move things from one place to another, not to be used as durable databases. I've made a list of articles like that: https://github.com/oskardudycz/EventSourcing.NetCore#this-is-not-event-sourcing I'll repeat it again: this is not Event Sourcing.
历次讨论
同类还值得读
开源应用架构
The Architecture of Open Source Applications
《开源软件架构》(AOSA)是一本在线免费书,邀请知名开源项目的作者亲自剖析其内部设计,涵盖 SQLAlchemy、LLVM、Audacity、Apache、Bash 等系统。
编程与软件工程上首页 9 次 · 2011–2024aosabook.org编辑精选编程语言简史:简短、不完整且多半错误
A Brief, Incomplete, and Mostly Wrong History of Programming Languages
一篇以戏谑、故意不准确的方式回顾编程语言史的博客文章,用夸张轶事和讽刺定义串起 Lisp、C、C++、Java、JavaScript、PHP 等语言。
编程与软件工程上首页 8 次 · 2009–2021james-iry.blogspot.comSQLite 是如何测试的
How SQLite is tested
SQLite 官方文档页,系统介绍 SQLite 的测试体系:多种测试方法(TCL 测试、模糊测试、回归测试、静态分析等)和海量测试用例,展示了这个小型开源项目如何靠极致测试获得…
编程与软件工程上首页 8 次 · 2009–2025sqlite.org编辑精选高级编程语言(2009)
Advanced programming languages (2009)
Matt Might 的经典文章,按用途推荐“高级编程语言”:如果追求就业就选主流语言,如果追求表达力、学术研究或创业,则推荐 Haskell、OCaml、Scala 等函数式语…
编程与软件工程上首页 8 次 · 2009–2022matt.might.net我的 Forth 与栈机器经历
Yosefk - My history with Forth & stack machines
作者以亲身经历回顾了使用 Forth 语言和栈式机器的历程:既赞叹其简洁优雅、代码量小、可自举的优点,也坦率指出栈操作在实际编程中的痛苦,以及 Forth 社区文化中的问题。
编程与软件工程上首页 8 次 · 2010–2024yosefk.com编辑精选