系统与基础设施eng.uber.com编辑精选
Uber 工程团队为何从 Postgres 迁移到 MySQL
Why Uber Engineering Switched from Postgres to MySQL
2016 — 2018,它一次次回到首页
黑点是上首页(≥ 50 分)的那次投稿,空心点是没冲上去的投稿,点越大分数越高。
导读
Uber 工程团队在这篇文章中解释了他们为何从 PostgreSQL 迁移到 MySQL,重点涉及复制机制、表损坏与数据一致性、在线 schema 变更以及大规模分片等运维层面的考量。这是 Uber 在特定架构和规模下的经验,并非对两种数据库的普遍性判决。
为什么经久不衰:它是一篇经久不衰的数据库选型案例,评论区也持续揭示出复制、一致性与工程实践之间的真实权衡,今天读仍能帮助理解“按运维模型选数据库”而非只看基准测试。
HN 评论精选
Postgres 的多数“缺点”从整体看是优势,尤其 MVCC、索引和 WAL 复制;MySQL 复制隐患更多。
我认为,整体来看而不是只看 Uber 的具体用例,文中大多数 Postgres 的“缺陷”其实是相对 MySQL 的优势。Postgres 的 MVCC 更优(可回滚 DDL、在线加索引、长时间开启只读事务而不影响其他部分),索引类型也远不止 b-tree;它没有聚簇 b-tree 索引,这确实让 MySQL 在某些场景略好。Postgres 的 WAL 级复制能更好地保证数据被正确复制;我无数次不得不告诉老板“MySQL 从库可能和主库略有不同步”,因为各种复制问题,触发器/定时事件的处理很容易破坏复制或悄悄造成不一致。至于数据损坏,如果有 bug 导致损坏,那就是 bug,我不认为这是文章暗示的根本设计缺陷;你不该靠半吊子的复制设计来侥幸避免损坏 bug。MySQL 这种设计还有很多文章没列的缺点。
ledjon查看英文原文
I would argue that most of these Postgres "flaws" are actually advantages over MySQL when you look at them holistically rather than the very specific Uber use-case. Postgres's MVCC is superior (can rollback DDL, can add indexes online, can have open read transactions for a VERY long time without impacting other parts of the system) Postgres supports many types of indexes, not just b-tree. One thing it doesn't have is clustered b-tree indexes... which is really what MySQL does that makes it somewhat "better." I wonder how Uber adds an index to a table that already has 1B+ rows in it with mysql? Postgres have WAL level replication is a better guarantee of actually replicating the data correctly. I cannot tell you how many times I've had to tell my boss that the "mysql replicas might be slightly out of sync with the master" because of various replication issues. The way it handles triggers and scheduled events alone is garbage and can very easily break replication and/or silently cause inconsistency. As for data corruption, if there is a bug that causes corruption, then there is a bug. I don't think that is a fundamental design flaw as implied in this article. You shouldn't rely on 1/2 assed replication design to accidentally save you from the data corruption bug. There are many downsides to the design MySQL has that are simply not listed here. I have been both a professional MySQL administrator as well as Postgresql (as well as SQL Server and many NoSQL engines). Many of these Postgres issues are only issues at crazy huge scale, and I would say at that point you probably want to move away from relational anyway. MySQL has its own very large set of problems at scale as well. It sounds like Uber is using MySQL as just a data bucket with primary keys ("Schemaless") which is good -- because you can't alter tables to save your life with MySQL. At the end of the data each developer/business needs to use what works for them, but I would really shy away from pointing to this article as a linchpin in the "MySQL vs. Postgres" war (if there even is such a thing.)
批评 Uber 把工程实践问题归咎于 Postgres;语句复制并非 MySQL 独有优势,事务管理应靠抽象。
MySQL 支持多种复制模式:基于语句的复制会复制逻辑 SQL 语句……Postgres 也有这个能力(通过第三方工具,但它是官方支持的工具)。我们十年前在 reddit 就用过,它带来了很多问题。我不认为这是 MySQL 的优势。老实说,读下来总结像是:“我们没有遵循良好的工程实践,所以需要一个更宽容的数据库。”如果这就是你想经营业务的方式,那没问题,但这并不是 Postgres 的丧钟。举一个具体例子:……开发者应该理解数据库事务,但你应该通过抽象让他们不必理解……
jedberg查看英文原文
> MySQL supports multiple different replication modes: > Statement-based replication replicates logical SQL statements (e.g., it would literally replicate literal statements such as: UPDATE users SET birth_year=770 WHERE id = 4) Postgres has that too (using a 3rd party tool, but it's an officially supported tool). We were using it on reddit 10 years ago. It caused a lot of problems. I wouldn't call that an advantage for Mysql. Honestly, reading this it seems like the summary is: "We don't follow great engineering practices so we need a database more forgiving". Which is fine if that's how you want to run your business, but isn't really the death knell for Postgres. A specific example: > This problem might not be apparent to application developers writing code that obscures where transactions start and end. For instance, say a developer has some code that has to email a receipt to a user. Depending on how it’s written, the code may implicitly have a database transaction that’s held open until after the email finishes sending. While it’s always bad form to let your code hold open database transactions while performing unrelated blocking I/O, the reality is that most engineers are not database experts and may not always understand this problem, especially when using an ORM that obscures low-level details like open transactions. Your developer should understand database transactions. But you should make it easier for them by abstracting it so that they don't have to. And in this particular case, I'd say they shouldn't be using the database to do locking around sending a receipt. It should be put into a queue and that queue should be processed separately, which avoids the transaction problem altogether.
文章混淆分布式的物理限制与复制实现;MySQL 复制并不更健壮,Postgres 一致性更好。
这篇文章可以概括为“Postgres 不是分布式数据库”。MySQL 也不是,尽管它的复制技术确实更友好。更可能的情况是:他们设计的 “schemaless” schema 或其配套软件能够处理 MySQL 允许而 Postgres 不允许的那类软错误。我工作的地方有跨全国的 MySQL 复制,我绝不会称它健壮——它每 3-6 个月就出故障。MySQL 复制确实比 Postgres 更老、更易用,但 SQL 数据库本质上是 CP 系统。当文章说“复制从库经常落后主库几秒,因此容易写出导致事务被杀死的代码”时,听起来像是把物理问题归咎于复制实现方式。现实网络下,不可能设计出让两个高一致性数据库达到完美可用性的复制系统;更差的协议会加剧问题,但更好的协议也无法消除问题。我在 Postgres 上从没见过损坏(MySQL 则见过),只是没试过跨数据中心复制。除此之外,Postgres 在一致性上通常比 MySQL 好得多(MySQL 的 DDL 不是事务性的等)。
fusiongyro查看英文原文
The article could be summed up as "Postgres is not a distributed database." MySQL isn't either, although it certainly has more friendly replication technology. I think it's a lot more likely that what's really happening here is that they've designed their "schemaless" schema or its supporting software to handle the kind of soft errors that MySQL is permitting and Postgres was not. We have MySQL replication across the country where I work and I certainly wouldn't characterize it as robust; it fails every 3-6 months. MySQL replication is certainly a lot older and easier to use than Postgres's, but SQL databases are fundamentally CP systems. When you say "This design means that replicas can routinely lag seconds behind master, and therefore it is easy to write code that results in killed transactions" it sounds like you're blaming the way replication was implemented for a physical problem. There is no way to design a replication system such that two highly-consistent databases can achieve perfect availability in the face of real-world networks. A worse protocol can exacerbate the problem, but a better one can't make it go away. I have never seen corruption with Postgres (unlike MySQL), but I have never tried cross-datacenter replication with it. Apart from that, Postgres generally seems to do much better with consistency than MySQL does, where DDL statements are not transactional, etc. So I am not surprised to hear that their system trips harder on Postgres's more aggressive consistency. In short, I suspect a more robust solution to their problem is a NoSQL database. On the other hand, it sounds like they want a combination of availability and consistency that will be difficult to get off-the-shelf. I'm glad they found a way to make it work. I wouldn't generally choose Postgres for a scalable system with an aggressive availability constraint--but then again, I wouldn't choose MySQL either, and I generally avoid problems that demand highly scalable, highly available solutions.
讽刺文章把“数据可能缺失/无效但不停机”当成优点。
文章说:“MySQL 的复制架构意味着,如果 bug 确实导致表损坏,问题不太可能造成灾难性故障。复制发生在逻辑层,所以像 rebalancing B-tree 这样的操作永远不会导致索引损坏。典型的 MySQL 复制问题是语句被跳过(或较少见地被执行两次)。这可能导致数据缺失或无效,但不会导致数据库宕机。”所以,数据可能无效或缺失,但不会导致数据库宕机……
zaxomi查看英文原文
From the article: > MySQL’s replication architecture means that if bugs do cause table corruption, the problem is unlikely to cause a catastrophic failure. Replication happens at the logical layer, so an operation like rebalancing a B-tree can never cause an index to become corrupted. A typical MySQL replication issue is the case of a statement being skipped (or, less frequently, applied twice). This may cause data to be missing or invalid, but it won’t cause a database outage. So, data might be invalid or missing, but it will not cause a database outage...
历次讨论
同类还值得读
TTY 揭秘(2008)
The TTY demystified (2008)
经典文章深入剖析了 Unix/Linux 中 TTY 子系统的工作原理,包括线路规程、终端驱动、信号与控制字符等机制。
系统与基础设施上首页 11 次 · 2009–2025linusakesson.net编辑精选为什么 GNU grep 这么快
Why GNU grep is Fast
FreeBSD 邮件列表上一篇广为流传的帖子,解释 GNU grep 为什么快:核心在于 Boyer-Moore 字符串搜索算法配合精心优化的内部循环和 I/O,让程序在搜索时“…
系统与基础设施上首页 9 次 · 2010–2022lists.freebsd.orgXplain:为普通人讲解 X11
Xplain – Explaining X11 for the rest of us
《Xplain》是作者(HN 用户 Jasper_)写的一系列 X11 长文,用他在浏览器中重建的 X11 绘图模型做交互演示,讲解 X11 的绘图模型与窗口系统设计。
系统与基础设施上首页 6 次 · 2013–2020magcius.github.io编辑精选旋风教程:为 Linux 创建极小 ELF 可执行文件(1999)
A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux (1999)
breadbox 写于 1999 年的经典教程,主题是如何为 Linux 手工构造并裁剪出体积极小的 ELF 可执行文件。
系统与基础设施上首页 6 次 · 2007–2025muppetlabs.comPostgreSQL 内部原理
The internals of PostgreSQL
《The Internals of PostgreSQL》是一本免费在线书籍,系统讲解 PostgreSQL 的内部实现,包括查询处理、并发控制、WAL、缓冲区管理等核心机制。
系统与基础设施上首页 5 次 · 2017–2025interdb.jp编辑精选