编程与软件工程gregable.com
学一点 Awk 就够了
Learn just a little Awk (2010)
2010 — 2019,它一次次回到首页
黑点是上首页(≥ 50 分)的那次投稿,空心点是没冲上去的投稿,点越大分数越高。
导读
这篇 2010 年的经典文章主张每个开发者都值得学一点 awk:它用简短的示例展示 awk 如何快速完成字段提取、过滤等日常文本处理任务,而无需写完整脚本。文章多年来多次登上 HN 首页,评论区也不断有人分享用两行 awk 替代臃肿工具或脚本的经历。
为什么经久不衰:awk 几乎预装在所有 *nix 系统上,处理日志和文本数据依然是最快的瑞士军刀,这篇短文是入门的最佳引子。
HN 评论精选
两行 awk 解决了臃肿工具崩溃后无法完成的实际数据分析任务
最近帮朋友分析近 1GB 的 RNA 样本数据,推荐的分析工具 mirexpress 崩溃了,其代码混乱到我花三四个小时也没法修。半夜醒来想起 awk,写了两行脚本就完美跑完所有数据,直接出了结果。
kumarharsh查看英文原文
Recently, I was helping out a friend in analyzing some RNA samples for her work. These samples are huge - like nearly a gigabyte of data. There was this tool which was recommended for the job - mirexpress. It was a small job, perhaps 10 minutes worth of effort. To make my work easier, I provisioned a beefy (and costly) machine on Azure to do the job, took a quick look at the clock (it said 11 PM), ran the tool, and relaxed. The tool crashed while reading the file. In an attempt to fix the bug, I opened mirexpress's code. And all my confidence in my programming ability vanished when I saw its innards. I understand that the code may have been written by scientists who had no experience in programming, but I have never been so utterly _disoriented_ by bad code. Anyways, after hacking away at the mess for about 3-4 hours, I realized that this was a fool's errand and thought I'll just phone it in the next day saying I couldn't do it. I went to sleep thinking that it was already late and I'd get late for work the next day. - 5 minutes later - I woke up with a start, recalling this nifty tool called awk. I had last used it maybe 3 years ago, and before that only in college. But I could see how awk could do some of the things which mirexpress was claiming to do. So I fire up my computer, write an awk script - 2 lines only! TWO FUCKING LINES! And it runs like a charm - eats away at megabytes of sample data and gives me results I can show. So then like any rational person, I spent the remaining hours re-discovering awk and forgot to sleep. Pissed away the whole next day (and some part of the day after that too!) :-D It's really fascinating that this nifty little tools invented DECADES ago are still going strong, and there's been no _evolutionary_ leap in areas where tools like awk/grep/sed excel at.
推荐权威教材《The AWK Programming Language》,awk 在贝尔实验室时代就已是数据处理利器
我 80 年代末在贝尔实验室第一次接触 awk,之前只用 Lisp 和 C,被它的简洁震撼。虽然当时 Perl 正流行,awk 仍是我多年的首选工具。想学 awk 强烈推荐 Aho、Kernighan 和 Weinberger 的《The AWK Programming Language》,写得和《C 程序设计语言》一样好。
MaDeuce查看英文原文
I was first exposed to Awk when I started work at Bell Labs in the late 80s. Until then, I'd been using either Lisp or C exclusively and was really blown away by how simple some things were in Awk. I used it with impunity to munge all sorts of data for input into fault prediction tools I was working on. Speed was never an issue for me, so I never explored the potential improvements offered by 'awkcc'. Although perl was becoming the new hotness at that time, Awk remained my goto tool for many years. If you are interested in learning Awk, I highly recommend "The AWK Programming Language" by Aho, Kernighan, and Weinberger. It's about the same size as the original "The C Programming Language" and is equally well-written. Previously on HN: https://news.ycombinator.com/item?id=13451454
awk 的价值在于简单且几乎所有系统原生自带
有人会问如今有 Python、Go 为什么还要学 awk,但很多任务(甚至大任务)用它更简单。而且大多数操作系统的基础系统都自带 POSIX 兼容版本,这对数据分析和系统管理/运维人员都非常有价值。
dbolgheroni查看英文原文
Some people might think "why learn awk today when there is Python, Go, etc". But a lot of tasks (even big ones) can be done easier with this tool. The fact that most OSs have a POSIX compliant version of it in the base system also make it very valuable, not just for the people crunching data but also to sysadmin/devops people.
给出 -F、正则匹配等实用 awk 单行命令示例
awk 是极好的瑞士军刀,每个 *nix 上都有。文章提到 FS 变量,但用 -F 更方便,比如快速处理 CSV:awk -F, '{print $2}'。内置正则匹配也超实用,比如 awk '/ERROR/{print $(NF-1)}',或从访问日志里挑出所有 503。
kylek查看英文原文
Awk is a fantastic "swiss army knife", and always included (in some form) on every *nix. The article mentions FS but this is easier to access with -F. e.g. for some quick and dirty CSV munging- awk -F, '{print $2}' Something else I find super useful is the built-in regex matching, e.g.- awk '/ERROR/{print $(NF-1)}' or maybe picking out all 503's from an web server access log- awk '/503/{print}' edit: formatting....wow hn is bad for this
历次讨论
同类还值得读
开源应用架构
The Architecture of Open Source Applications
《开源软件架构》(AOSA)是一本在线免费书,邀请知名开源项目的作者亲自剖析其内部设计,涵盖 SQLAlchemy、LLVM、Audacity、Apache、Bash 等系统。
编程与软件工程上首页 9 次 · 2011–2024aosabook.org编辑精选编程语言简史:简短、不完整且多半错误
A Brief, Incomplete, and Mostly Wrong History of Programming Languages
一篇以戏谑、故意不准确的方式回顾编程语言史的博客文章,用夸张轶事和讽刺定义串起 Lisp、C、C++、Java、JavaScript、PHP 等语言。
编程与软件工程上首页 8 次 · 2009–2021james-iry.blogspot.comSQLite 是如何测试的
How SQLite is tested
SQLite 官方文档页,系统介绍 SQLite 的测试体系:多种测试方法(TCL 测试、模糊测试、回归测试、静态分析等)和海量测试用例,展示了这个小型开源项目如何靠极致测试获得…
编程与软件工程上首页 8 次 · 2009–2025sqlite.org编辑精选高级编程语言(2009)
Advanced programming languages (2009)
Matt Might 的经典文章,按用途推荐“高级编程语言”:如果追求就业就选主流语言,如果追求表达力、学术研究或创业,则推荐 Haskell、OCaml、Scala 等函数式语…
编程与软件工程上首页 8 次 · 2009–2022matt.might.net我的 Forth 与栈机器经历
Yosefk - My history with Forth & stack machines
作者以亲身经历回顾了使用 Forth 语言和栈式机器的历程:既赞叹其简洁优雅、代码量小、可自举的优点,也坦率指出栈操作在实际编程中的痛苦,以及 Forth 社区文化中的问题。
编程与软件工程上首页 8 次 · 2010–2024yosefk.com编辑精选