编程与软件工程yuri.is
Julia 生态中的正确性与可组合性缺陷
Correctness and composability bugs in the Julia ecosystem
2022 — 2025,它一次次回到首页
黑点是上首页(≥ 50 分)的那次投稿,空心点是没冲上去的投稿,点越大分数越高。
导读
作者 Yuri 以资深 Julia 用户的角度,系统梳理了 Julia 生态中大量正确性与可组合性方面的 bug:由于缺乏接口约定和类型约束,互相独立的包在组合使用时容易因隐含假设而悄悄出错。文章引发了 Julia 核心开发者与社区成员的深入回应与讨论。
为什么经久不衰:它触及 Julia 类型系统与多重派发设计的根本权衡,连语言核心成员都认真回应,是理解该语言生态优劣的必读讨论。
HN 评论精选
Julia 核心开发者承认可组合性设计确实带来隐含假设问题,但对 bug 更多的说法持保留态度
这条评论对我来说很沉重,因为 Yuri 确实在 Julia 上花了大量时间,而且他是一位非常有能力的程序员,所以他的批评值得认真对待,听到他最终留下了负面印象我也很难过。文章涉及很多不同的问题,我觉得其中一类是“单纯的 bug”(比如提到的 HTTP、JSON 等问题),说这类问题在 Julia 中比其他系统更常见,我不知道该如何评判——不是说 Julia 生态没有 bug,只是就我的经验而言,无论用什么语言,每次尝试新软件时我基本都会遇到两三个严重问题。另一条主线是“很难知道什么应该能正常工作”,我认为这是合理的批评,也同意 Yuri 的看法:一些根本性的设计决策确实造成了这一点。Julia 非常努力地让可组合性生效,即使被组合的包的作者彼此一无所知。这是 Julia 强大能力的关键特性,但当然也容易导致某个包做出未文档化的隐含假设。
KenoFischer查看英文原文
So this one is a tough one for me, because Yuri has certainly spent significant time with Julia and I think he's a very competent programmer, so his criticism is certainly to be taken seriously and I'm sad to hear he ended up with a sour opinion. There's a lot of different issues mentioned in the post, so I'm not really sure what angle to best go at it from, but let me give it a shot anyway. I think there's a couple of different threads of complaints here. There's certainly one category of issues that are "just bugs" (I'm thinking of things like the HTTP, JSON, etc. issues mentioned). I guess the claim is that this happens more in Julia than in other systems. I don't really know how to judge this. Not that I think that the julia ecosystem has few bugs, just that in my experience, I basically see 2-3 critical issues whenever I try a new piece of software independent of what language it's written in. I think the other thread is "It's hard to know what's expected to work". I think that's a fair criticism and I agree with Yuri that there's some fundamental design decisions that are contributing here. Basically, Julia tries very hard to make composability work, even if the authors of the packages that you're composing don't know anything about each other. That's a critical feature that makes Julia as powerful as it is, but of course you can easily end up with situations where one or the other package is making implicit assumptions that are not documented (because the author didn't think the assumptions were important in the context of their own package) and you end up with correctness issues. This one is a bit of a tricky design problem. Certainly adding more language support for interfaces and verification thereof could be helpful, but not all implicit assumptions are easily capturable in interfaces. Perhaps there needs to be more explicit documentation around what combinations of packages are "supported". Usually the best way to tell right now is to see what downstream tests are done on CI and if there are any integration tests for the two packages. If there are, they're probably supposed to work together. To be honest, I'm a bit pained by the list of issues in the blog post. I think the bugs linked here will get fixed relatively quickly by the broader community (posts like this tend to have that effect), but as I said I do agree with Yuri that we should be thinking about some more fundamental improvements to the language to help out. Unfortunately, I can't really say that that is high priority at the moment. The way that most Julia development has worked for the two-ish years is that there are a number of "flagship" applications that are really pushing the boundary of what Julia can do, but at the same time also need a disproportionate amount of attention. I think it's overall a good development, because these applications are justifying many people's full time attention on improving Julia, but at the same time, the issues that these applications face (e.g. - "LLVM is too slow", better observability tooling, GC latency issues) are quite different from the issues that your average open source julia developer encounters. Pre 1.0 (i.e. in 2018) there was a good 1-2 year period where all we did was think through and overhaul the generic interfaces in the language. I think we could use another one of those efforts now, but at least that this precise moment, I don't think we have the bandwidth for it. Hopefully in the future, once things settle down a bit, we'll be able to do that, which would presumably be what becomes Julia 2.0. Lastly, some nitpicking on the HN editorialization of the title. Only of the issues linked (https://github.com/JuliaLang/julia/issues/41096) is actually a bug in the language - the rest are various ecosystem issues. Now, I don't want to disclaim responsibility there, because a lot of those packages are also co-maintained by core julia developers and we certainly feel responsibility to make those work wel…
正确性问题各语言都有,Julia 代码易读反而让 bug 更容易被发现
哪里都有正确性问题。Julia 发布了整个打补丁版本的 LLVM 来修复数值方法中的正确性 bug;它自己实现了软件层面的 FMA,因为 Windows 的 FMA 实现是错的;核心开发者因为要修的东西太多,成了 libuv 等项目的维护者。从这三点看,显然 Python、R 等的代码也有大量 Julia 所没有的错误。我觉得 Julia 有趣的地方在于代码都是 Julia 写的,很容易深入挖掘潜在 bug——标准库函数用 @edit sum(1:5) 就能打开源码随便改。代码越容易看,越容易发现问题,这也是 Julia 开发者与用户比例更高的原因。这有利有弊:它让开发过程民主化,但也意味着没有大量开发经验的人也能贡献。我认为长远看是好事,但会有波折。另外作者提到的 "for i in 1:length(A)",我同意,代码永远不该那样写,应该用 eachindex(A)。
ChrisRackauckas查看英文原文
Everything has correctness issues somewhere. Julia ships an entire patched version of LLVM to fix correctness bugs in numerical methods. It has its own implementations of things like software-side FMA because the FMA implementation of Windows is incorrect: https://github.com/JuliaLang/julia/pull/43530 . Core Julia devs are now the maintainers of things like libuv because of how much had to be fixed there. So from those three points, that clearly points out tons of cases where Python, R, etc. code is all incorrect where Julia isn't. I think what's interesting about Julia is that because the code is all Julia, it's really easy to dig in there and find potential bugs. The standard library functions can be accessed with @edit sum(1:5) and there you go, hack away. The easier it is to look at the code, the easier it is to find issues with it. This is why Julia has such a higher developer to user ratio. That has its pros and cons of course. It democratizes the development process, but it means that people who don't have a ton of development experience (plus Fortran or C knowledge) are not excluded from contributing. Is that good or bad? Personally I believe it's good in the long run, but can have its bumps. As an aside, the author highlights "for i in 1:length(A)". I agree, code should never do that. It should be `eachindex(A)`. In general things should use iterators which are designed for arbitrary indexing based on iterators. This is true in any language, though you'll always have some newcomers write code (and documentation) with this. Even experienced people who don't tend to use arrays beyond Array tend to do this. It's an interesting issue because coding style issues perpetuate themselves: explicitly using 1-base wasn't an issue before GPUs and OffsetArrays, but then loop code like that trains the next generation, and so more people use it. In the end the people who really know to handle these cases are the people who tend to use these cases, just like how people who write in styles that are ARM-safe tend to be people who use ARM. Someone should just run a bot that opens a PR for every occurrence of this (especially in Base), as that would then change the source that everyone learns from and completely flip the style.
社区老成员将问题根源归结为“类型无政府状态”:正确性责任归属不清
我是 Julia 社区的老成员,很久以前就和作者讨论过这些问题。首先说明 Yuri 是个很好的人,曾是社区中很有价值的成员,他推动了语言的边界,写过一些非常不错的包。他的担忧是真实的,应该在这个背景下被尊重和讨论。另外,我自己并没有遇到过这类 bug,但我对库的使用和花哨的组合一向非常保守。如果我有更多编程语言理论与实现的经验,也许能给文中描述的问题起个更好的名字。我的尝试是称之为“类型无政府状态”:在我看来,目前没有明确的方式来划分正确性的责任。以文中的数组为例,究竟是 sum 函数的实现者(没有应有的类型签名)的错,还是数据结构实现者的错?说实话我不确定。但既然 Julia 在类型系统和多重派发上开辟了新天地,这很可能仍是一个开放性问题。
ninjin查看英文原文
I am a long-time member of the Julia community and had a discussion with the author about these issues a long time ago – but did not give feedback on the post. Let me first state that Yuri is a great person and was a valuable member of the community. He pushed the boundaries of the language and produced some very nice packages in his time. His concerns are genuine and should be respected and discussed in that context. Also, let me say that encountering these kinds of bugs is not something I have had experience with. But, I tend to be very conservative with my usage of libraries and fancy composition. If I had more experience with programming language theory and implementation, perhaps I would have a better name to describe the source of the issues described. My attempt is to call it “type anarchy”. The way I see it, there is not a clear way to assign responsibility for correctness. In the case of the array used in the post, is it the fault of the implementer of the `sum` function (without a type signature, as it should be) or implementer of the data structure? I am honestly not sure. But as Julia breaks news ground with its type system and multiple dispatch, this could very much be an open question.
缺乏正式接口约束是最大痛点,linter 与 trait 系统可大幅缓解
我在 Julia 中开发过大型计算代码库,我认为这些问题中最隐蔽的一类源于没有正式的接口强制机制。用常见的包构建 trait 系统并保证某个 trait 所需的方法都已实现,能极大简化维护。这抓不住数学上的 bug(那些到处都会出现),但知道接口必须如何规定、从而可以信任自己的实现,并且知道实现何时失效,是无价的。我遇到过几个涉及大型项目的糟糕 bug,一个正式的接口/trait 系统会让事情成倍简化。有些编码风格需要改变,比如用 eachindex 而不是 1:length(A) 来迭代数组,但这些应该是一次性的教训,好的 linter 应该能抓住这类潜在错误。在好的 linter(或某种静态分析,我很看好 JET.jl)和正式接口规范之间,我真的认为 Julia 开发侧的大部分问题都能被平息。
QuackingTheQ查看英文原文
I've spent a lot of time developing large computational codebases in Julia, and I think the most insidious of these issues is a product of no formal way of enforcing interfaces. Using one of the common packages to build a trait system and add some sort of guarantee that all the right methods are implemented for a given trait simplifies maintenance dramatically. This doesn't catch mathematical bugs, but those crop up everywhere. Instead, knowing what the interfaces must be specified so you can trust your implementation is crucial, and being able to know when it is invalidated is invaluable. I've had a few awful bugs involving some of the larger projects in this language, but a proper interface/trait system would simplify things exponentially. There are some coding style things that need to be changed to address this, like using `eachindex` instead of `1:length(A)` for array iteration as the example in the article points out. However, these should be one-off lessons to learn, and a good code linter should be able to catch potential errors like this. Between a good code linter (or some static analysis, I'm pulling for JET.jl) and a formal interface spec, I really think most of Julia's development-side issues could be quelled.
历次讨论
同类还值得读
开源应用架构
The Architecture of Open Source Applications
《开源软件架构》(AOSA)是一本在线免费书,邀请知名开源项目的作者亲自剖析其内部设计,涵盖 SQLAlchemy、LLVM、Audacity、Apache、Bash 等系统。
编程与软件工程上首页 9 次 · 2011–2024aosabook.org编辑精选编程语言简史:简短、不完整且多半错误
A Brief, Incomplete, and Mostly Wrong History of Programming Languages
一篇以戏谑、故意不准确的方式回顾编程语言史的博客文章,用夸张轶事和讽刺定义串起 Lisp、C、C++、Java、JavaScript、PHP 等语言。
编程与软件工程上首页 8 次 · 2009–2021james-iry.blogspot.comSQLite 是如何测试的
How SQLite is tested
SQLite 官方文档页,系统介绍 SQLite 的测试体系:多种测试方法(TCL 测试、模糊测试、回归测试、静态分析等)和海量测试用例,展示了这个小型开源项目如何靠极致测试获得…
编程与软件工程上首页 8 次 · 2009–2025sqlite.org编辑精选高级编程语言(2009)
Advanced programming languages (2009)
Matt Might 的经典文章,按用途推荐“高级编程语言”:如果追求就业就选主流语言,如果追求表达力、学术研究或创业,则推荐 Haskell、OCaml、Scala 等函数式语…
编程与软件工程上首页 8 次 · 2009–2022matt.might.net我的 Forth 与栈机器经历
Yosefk - My history with Forth & stack machines
作者以亲身经历回顾了使用 Forth 语言和栈式机器的历程:既赞叹其简洁优雅、代码量小、可自举的优点,也坦率指出栈操作在实际编程中的痛苦,以及 Forth 社区文化中的问题。
编程与软件工程上首页 8 次 · 2010–2024yosefk.com编辑精选