工具与资源github.com
Dolt:可像 Git 一样分叉、克隆、分支和合并的 SQL 数据库
Dolt is Git for Data: a SQL database that you can fork, clone, branch, merge
2021 — 2022,它一次次回到首页
黑点是上首页(≥ 50 分)的那次投稿,空心点是没冲上去的投稿,点越大分数越高。
导读
Dolt 是一个把 Git 式版本控制做进 SQL 数据库本身的开源项目:可以像代码仓库一样 fork、clone、branch、merge、diff,也能回滚到任意历史提交或查询过去某一时刻的数据快照。它面向的是数据库长期缺乏可靠版本管理这一痛点——迁移脚本、备份恢复、多环境同步都靠外部工具拼凑。
为什么经久不衰:它把「数据库是不可变的、可分支可合并的版本历史」这一想法做成了能直接用的产品,思路至今仍被认为是数据库演进的方向,值得作为理解这一范式的入口。
HN 评论精选
数据库可变状态是迁移与回滚痛苦的根源,版本化数据库能像容器一样带来范式转变。
这是数据库的未来,但似乎还没人意识到。SQL 数据库最大的问题之一就是它是一大堆可变状态:迁移之所以存在,就是因为没法「直接」撤销任意改动、自动 diff、自动合并,只能靠一堆工具生成 DDL/DML 并按序执行、记录哪些已执行,而且改动一旦生效就无法回滚,除非再写更多逻辑。把数据库当作受版本控制的实体后,任何操作都可以是不可变的:随便改,不用担心冲突,随时退回上一个可用版本、撤销某个改动、把多个工作库的改动合并进来。生产库被人误删了主表?直接 revert 掉这次删除。它的影响会像容器普及一样具有颠覆性。
throwaway892238查看英文原文
This is the future of databases, but nobody seems to realize it yet. One of the biggest problems with databases (particularly SQL ones) is they're a giant pile of mutable state. The whole idea of "migrations" exists because it is impossible to "just" revert any arbitrary change to a database, diff changes automatically, merge changes automatically. You need some kind of intelligent tool or framework to generate DDL, DML, DCL, they have to be applied in turn, something has to check if they've already been applied, etc. And of course you can't roll back a change once it's been applied, unless you create even more program logic to figure out how to do that. It's all a big hack. By treating a database as version-controlled, you can treat any operation as immutable. Make any change you want and don't worry about conflicts. You can always just go back to the last working version, revert a specific change, merge in one or more changes from different working databases. Make a thousand changes a day, and when one breaks, revert it. No snapshotting and slowly restoring the whole database due to a non-reversible change. Somebody dropped the main table in prod? Just revert the drop. Need to make a change to the prod database but the staging database is different? Branch the prod database, make a change, test it, merge back into prod. The effect is going to be as radical as the popularization of containers. Whether you like them or not, they are revolutionizing an industry and are a productivity force multiplier.
时间旅行特性天然解决机器学习中训练数据的未来信息泄漏问题。
看起来很棒。列出的用例之一是「时间旅行」。我们希望在工作中使用它:我们在做时间序列预测,基础设施上有很多复杂度是为了保证用多年前的数据训练时,不会用到那个时间点之后才产生的数据(未来数据泄漏到过去)。按我的理解,用 Dolt 只要把数据库设到过去某个「未来数据尚不存在」的时间点就行,非常酷。
cosmic_quanta查看英文原文
That looks awesome. One of the listed use-cases is 'time-travel': https://dolthub.com/blog/2021-03-09-dolt-use-cases-in-the-wild/#other I wish we could use this at work. We're trying to predict time-series stuff. However, there's a lot of infrastructure complexity which is there to ensure that when we're training on data from years ago, that we're not using data that would be in the future from this point (future data leaking into the past). Using Dolt, as far as I understand it, we could simply set the DB to a point in the past where the 'future' data wasn't available. Very cool
可 fork 的数据库本可以让 CMS 的本地测试、合并与审批流程变得简单。
WordPress 本可以从中受益。很多站长想要的是:在本地测试站点,然后合并回去。很多人转向 Jekyll 或 Hugo,正是因为站点能签入 git,而 git 是可靠的;静态站点不会被黑,被 WordPress 安全事故坑过的人更愿意选静态站。更进一步:人们希望把新网站从设计师交给客户、再交给管理者——如果 WordPress 有一个可 fork 的数据库,也许根本不需要自己开发那套审批流程(权限、草稿/预览/发布)。
laurent92查看英文原文
Wordpress would have benefited from this. What a lot of webmasters want is, test the site locally, then merge it back. A lot of people turned to Jekyll or Hugo for the very reason that it can be checked into git, and git is reliable. A static website can’t get hacked, whereas anyone who has been burnt with Wordpress security fail knows they’d prefer a static site. And even more: People would like to pass the new website from the designer to the customer to managers — Wordpress might have not needed to develop their approval workflows (permission schemes, draft/preview/publish) if they had had a forkable database.
数据版本控制如何与多环境、匿名化和回填等实际流程结合仍不清楚。
我一直关注这个领域,但没能搞清楚这些数据版本控制方案在多环境场景下如何运作。有些方面更贴近时间旅行,比如与双时态数据库、XTDB 等数据建模方式对应;有些更贴近 git 式用例,比如模式演进、回填;还有些是组合。难点在于:数据上我看不出怎么把所有环境放在同一个地方/仓库里,而且还有方向性问题,比如从生产到非生产需要匿名化、从非生产到生产需要回填等。很想多读一些这个领域的实践经验,以及大家如何组合不同方案。
cgio查看英文原文
I have been interested in this space, but have failed to understand how these versioning solutions for data work in the context of environments. There are aspects of time travel that line up better e.g. with data modelling approaches (such as bitemporal DBs, xtdb etc.) others more with git-like use cases (e.g. schema evolution, backfilling) some combinations. The challenge is, with data I don’t see how you’d like to have all environments in same place/repo and there may be additional considerations coupled with directionality of moves, such as anonymisation for moving from prod to non-prod , back filling for moving from non-prod to prod etc. Keen to read more on other people experiences in this space and how they might be combining different solutions.
历次讨论
同类还值得读
直接在浏览器中启动 Linux 内核
Boot a linux kernel right inside your browser.
Fabrice Bellard 的 JSLinux 页面,用 JavaScript 在浏览器里启动一个 Linux 内核,并提供一个可交互的终端环境。
工具与资源上首页 5 次 · 2011–2025bellard.orgHacker News 未公开功能与行为清单(2018)
A List of Hacker News's Undocumented Features and Behaviors (2018)
数据科学家 Max Woolf 整理的 Hacker News 未公开功能与行为清单,涵盖投票机制、排名算法、账号惩罚等社区运作细节。
工具与资源上首页 5 次 · 2019–2025github.comMapSCII:在终端中用盲文和 ASCII 渲染地图
MapSCII – Braille and ASCII map renderer for the console
MapSCII 是一个在命令行终端里渲染地图的开源工具,用盲文字符和 ASCII 字符把地理数据画成可交互浏览的字符地图。
工具与资源上首页 5 次 · 2017–2025github.com顶尖大学的免费在线课程
Free Online Courses from Top Universities
Open Culture 整理的一份免费在线课程合集,汇集了全球顶尖大学(如 MIT、斯坦福、耶鲁等)向公众开放的网络课程资源,涵盖音频、视频等多种形式。
工具与资源上首页 5 次 · 2010–2022openculture.com淘气字符串:容易引发用户输入问题的字符串清单
Naughty Strings: A list of strings likely to cause issues as user-input data
一个 GitHub 仓库,收录了大量容易让程序出问题的用户输入字符串,如特殊字符、emoji、SQL 注入片段、系统命令等,供开发者用于测试输入处理逻辑。
工具与资源上首页 4 次 · 2015–2020github.com