编程与软件工程benhoyt.com
用 Python、Go、C++、C、Awk、Forth、Rust 统计词频的性能比较
Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
2021 — 2022，它一次次回到首页
黑点是上首页（≥ 50 分）的那次投稿，空心点是没冲上去的投稿，点越大分数越高。
导读
Ben Hoyt 比较了 Python、Go、C++、C、Awk、Forth、Rust 等语言实现单词计数程序的性能，每个语言都给出“惯用”和“优化”两个版本，并测量运行时间。文章还穿插了编程史上的经典轶事：1986 年 Knuth 用文学编程写了十页方案解决此问题，McIlroy 则用一行 Unix 管道回敬。
为什么经久不衰：它不只是一张语言性能表，更展示了“惯用代码”与“优化代码”之间的差距，以及 I/O 方式、哈希算法等细节如何显著左右基准结果，提醒读者对这类比较保持谨慎。
HN 评论精选
Knuth 的十页文学编程方案与 McIlroy 的一行 Unix 管道形成经典对比。
这对我来说绝对是文章中最有趣的部分：……1986 年，Jon Bentley 请 Donald Knuth 用“文学编程”解决这个问题，Knuth 写出了一篇精美的十页杰作。然后 Doug McIlroy（Unix 管道的发明者）用 tr、sort 和 uniq 回复了一行 Unix shell 版本。
Shorel查看英文原文
This is definitely the most interesting part of the article for me: Incidentally, this problem set the scene for a wizard duel between two computer scientists several decades ago. In 1986, Jon Bentley asked Donald Knuth to show off “literate programming” with a solution to this problem, and he came up with an exquisite, ten-page Knuthian masterpiece. Then Doug McIlroy (the inventor of Unix pipelines) replied with a one-liner Unix shell version using tr, sort, and uniq. http://www.leancrew.com/all-this/2011/12/more-shell-less-egg/
C 示例为单个整数做堆分配是浪费，改用指针存储可获得小幅性能提升。
这类文章（对我来说）是这里最好的内容之一。除了 C 我可能不算任何所用语言的专家……我震惊地发现示例代码为存储整数做了堆分配，每个“计数桶”都为单个整数分配一次。我会毫不犹豫地把计数存在数据指针里。我在本地改了之后，耗时从 0.11 降到 0.10。
unwind查看英文原文
Very interesting, these kinds of articles are (to me) some of the best content we get on here. I'm no expert in any of the languages used except maybe C, at least I'm quick to form opinions about C code. In C, I'm not so sure that there is a well-established "idiomatic" concept, since C is not so tightly bound to a community as more modern languages. At least that's my feeling. That said, I was shocked to see that the example code did heap-allocations to store integers. Single integers, one per allocation, per "counting bucket". I would store the count in the data pointer without a second thought, I just "can't" make 4/8-byte heap allocations and still sleep at night. Especially after all the concern about processing the input in large blocks for performance! Casting an integer to/from a void pointer should be pretty safe according to my gut feel. If that turns out to be false, I would dump the libc's hash API altogether, since it makes me do heap allocations to store single integers. I tried making the change locally, and the time taken (as measured with the OP's benchmarking program) dropped from 0.11 to 0.10 on my system (a Dell laptop running an Intel i5-7300). That's at least something.
Rust“优化”版的三个改动中，字节字符串其实改变了问题本身，另两个是合理的性能优化。
对于好奇 Rust 优化的人，“优化”版本相比“惯用”版本做了三个改变：使用字节字符串而不是 UTF-8 字符串，在我看来这不是优化，而是改变了问题；使用更快的哈希算法，Rust 默认使用 DOS 安全哈希，这在这类微基准测试中伤害了它；使用 get_mut+insert 而不是更方便的 HashMap::entry。
codeflo查看英文原文
For those curious about Rust optimization, the “optimized” version makes three changes compared to the “idiomatic” version: * It uses byte strings instead of UTF-8 strings. In my opinion, that’s not an optimization, that’s changing the problem. Depending on the question you’re asking, only one of the two can be correct. * It uses a faster hash algorithm. It’s not the first time this came up in a benchmark article. Rust’s decision to use a DOS-safe hash by default (and not provide a fast algorithm in the std, like other languages do) really seems to hurt it in that kind of microbenchmark. * It uses get_mut+insert instead of the more convenient HashMap::entry method, because the latter would require redundantly allocating the key even in the repeat case. I’ve hit this problem in the past as well. Maybe the upcoming HashMap::raw_entry_mut will make this kind of optimization cleaner.
基准测试环境差异可导致比文中多数语言差距更大的偏差，结论需谨慎。
Knuth-McIlroy 经常被提起……在这个例子里，我可以让一个非常类似的 Nim 程序跑得几乎和 wc -w 一样快，但优化过的 C 程序比 wc -w 快 1.2 倍而不是慢 1.15 倍——1.4 倍的差异，比表格中的许多比率都大。所以人们应该对从中得出的任何结论非常谨慎。
cb321查看英文原文
Knuth-McIlroy comes up a lot. Previous discussion [1]. For this example I can make a very similar Nim program [2] run almost exactly the same speed as `wc -w`, yet the optimized C program runs 1.2x faster not 1.15x slower than `wc -w` - a 1.4x discrepancy - bigger than many ratios in the table. So, people should be very cautious about conclusions from any of this. [1] https://news.ycombinator.com/item?id=24817594 [2] https://github.com/c-blake/adix/blob/master/tests/wf.nim
历次讨论
同类还值得读
开源应用架构
The Architecture of Open Source Applications
《开源软件架构》（AOSA）是一本在线免费书，邀请知名开源项目的作者亲自剖析其内部设计，涵盖 SQLAlchemy、LLVM、Audacity、Apache、Bash 等系统。
编程与软件工程上首页 9 次 · 2011–2024aosabook.org编辑精选编程语言简史：简短、不完整且多半错误
A Brief, Incomplete, and Mostly Wrong History of Programming Languages
一篇以戏谑、故意不准确的方式回顾编程语言史的博客文章，用夸张轶事和讽刺定义串起 Lisp、C、C++、Java、JavaScript、PHP 等语言。
编程与软件工程上首页 8 次 · 2009–2021james-iry.blogspot.comSQLite 是如何测试的
How SQLite is tested
SQLite 官方文档页，系统介绍 SQLite 的测试体系：多种测试方法（TCL 测试、模糊测试、回归测试、静态分析等）和海量测试用例，展示了这个小型开源项目如何靠极致测试获得…
编程与软件工程上首页 8 次 · 2009–2025sqlite.org编辑精选高级编程语言（2009）
Advanced programming languages (2009)
Matt Might 的经典文章，按用途推荐“高级编程语言”：如果追求就业就选主流语言，如果追求表达力、学术研究或创业，则推荐 Haskell、OCaml、Scala 等函数式语…
编程与软件工程上首页 8 次 · 2009–2022matt.might.net我的 Forth 与栈机器经历
Yosefk - My history with Forth & stack machines
作者以亲身经历回顾了使用 Forth 语言和栈式机器的历程：既赞叹其简洁优雅、代码量小、可自举的优点，也坦率指出栈操作在实际编程中的痛苦，以及 Forth 社区文化中的问题。
编程与软件工程上首页 8 次 · 2010–2024yosefk.com编辑精选