工具与资源github.com
pigz：多核机器上的并行 gzip 实现
pigz: A parallel implementation of gzip for multi-core machines
2016 — 2022，它一次次回到首页
黑点是上首页（≥ 50 分）的那次投稿，空心点是没冲上去的投稿，点越大分数越高。
导读
pigz 是 gzip 的多核并行实现，由 zlib 作者 Mark Adler 开发，可近乎无缝替代 gzip，大幅缩短大文件压缩时间。页面是其 GitHub 仓库，多次登上 HN 首页，常被用于数据库备份、生物信息学大文件处理等场景。
为什么经久不衰：压缩是几乎所有服务器运维都绕不开的瓶颈，pigz 简单、稳定、随处可得，即使今天有 zstd，它仍是经典工具，其实现思路也值得学习。
HN 评论精选
pigz 可与云存储拷贝管道并行叠加使用，但如今 zstd 往往是更优选择
最近做生物信息学的概念验证时正好用上。那个领域动辄几十 GB 的文本文件，压缩和拷贝都很痛苦。我发现 pigz 可以一边压缩一边并行上传云存储，比如 pigz input.fastq -c | azcopy ...，s3cmd 也有类似管道。不过如果工具支持 zstd，用 zstd -T0 或 pzstd 效率更高。另外希望这类工具能根据输出带宽自动调节压缩级别，以最大化总吞吐。
jiggawatts查看英文原文
Funny this comes up again so soon after I needed it! I recently did a proof-of-concept related to bioinformatics (gene assembly, etc...), and one quirk of that space is that they work with enormous text files. Think tens of gigabytes being a "normal" size. Just compressing and copying these around is a pain. One trick I discovered is that tools like pigz can be used to both accelerate the compression step and also copy to cloud storage in parallel! E.g.: pigz input.fastq -c | azcopy copy --from-to PipeBlob "https://myaccountname.blob.core.windows.net/inputs/input.fastq.gz?..." There is a similar pipeline available for s3cmd as well with the same benefit of overlapping the compression and the copy. However, if your tools support zstd, then it's more efficient to use that instead. Try the "zstd -T0" option or the "pzstd" tool for even higher throughputs but with same minor caveats. PS: In case anyone here is working on the above tools, I have a small request! What would be awesome is to automatically tune the compression ratio to match the available output bandwidth. With the '-c' output option, this is easy: just keep increasing the compression level by one notch whenever the output buffer is full, and reduce it by one level whenever the output buffer is empty. This will automatically tune the system to get the maximum total throughput given the available CPU performance and network bandwidth.
仓库中用纯 C99 实现 try/catch 的代码是个有趣的亮点
我觉得最有意思的其实是仓库里的 try.h 和 try.c，它们在 C99 里实现了 try/catch。
omoikane查看英文原文
The bit I found most interesting was actually: https://github.com/madler/pigz/blob/master/try.h https://github.com/madler/pigz/blob/master/try.c which implements try/catch for C99.
pbzip2 并行度更高，但 zstdmt 综合表现已全面胜出
如果真想让压缩和解压都用满所有核心，试试 pbzip2，pigz 的并行度不如它。（编辑：正如楼下所说，直接用 zstdmt 吧，实测比 pbzip2 快约 8 倍且压缩率更好，3.6GB 数据从 41 秒降到 3.5 秒，文件还更小。）
sitkack查看英文原文
If you really want to enable all cores for compression and decompression, give pbzip2 a try. pigz isn't as parallel as pbzip2 http://compression.ca/pbzip2/ *edit, as ac29 mentions below, just use zstdmt. In my quick testing it is approximately 8x faster than pbzip2 and gives better compression ratios. Wall clock time went from 41s to 3.5s for a 3.6GB tar of source, pdfs and images AND the resulting file was smaller. megs 3781 test.tar 3041 test.tar.zstd (default compression 3, 3.5s) 3170 test.tar.bz2 (default compression, 8 threads, 40s)
Facebook 曾用 pigz 加速 MySQL 备份，多年可靠运行
2010 年代初我们在 Facebook 用它做 MySQL 备份，效果很好。备份主机 CPU 大量闲置，相比 gzip 提速明显。后来我们换成了 zstd，但 pigz 从没让我们失望过。
ericbarrett查看英文原文
We used this to great effect at Facebook for MySQL backups in the early 2010s. The backup hosts had far more CPU than needed so it was a very nice speed-up over gzip. Eventually we switched to zstd, of course, but pigz never failed us.
历次讨论
同类还值得读
直接在浏览器中启动 Linux 内核
Boot a linux kernel right inside your browser.
Fabrice Bellard 的 JSLinux 页面，用 JavaScript 在浏览器里启动一个 Linux 内核，并提供一个可交互的终端环境。
工具与资源上首页 5 次 · 2011–2025bellard.orgHacker News 未公开功能与行为清单（2018）
A List of Hacker News's Undocumented Features and Behaviors (2018)
数据科学家 Max Woolf 整理的 Hacker News 未公开功能与行为清单，涵盖投票机制、排名算法、账号惩罚等社区运作细节。
工具与资源上首页 5 次 · 2019–2025github.comMapSCII：在终端中用盲文和 ASCII 渲染地图
MapSCII – Braille and ASCII map renderer for the console
MapSCII 是一个在命令行终端里渲染地图的开源工具，用盲文字符和 ASCII 字符把地理数据画成可交互浏览的字符地图。
工具与资源上首页 5 次 · 2017–2025github.com顶尖大学的免费在线课程
Free Online Courses from Top Universities
Open Culture 整理的一份免费在线课程合集，汇集了全球顶尖大学（如 MIT、斯坦福、耶鲁等）向公众开放的网络课程资源，涵盖音频、视频等多种形式。
工具与资源上首页 5 次 · 2010–2022openculture.com淘气字符串：容易引发用户输入问题的字符串清单
Naughty Strings: A list of strings likely to cause issues as user-input data
一个 GitHub 仓库，收录了大量容易让程序出问题的用户输入字符串，如特殊字符、emoji、SQL 注入片段、系统命令等，供开发者用于测试输入处理逻辑。
工具与资源上首页 4 次 · 2015–2020github.com